Scribd
Upload
8 views

data science notes b

The document provides an overview of machine learning, defining it as a subset of AI that enables systems to learn from data. It covers key categories such as supervised, unsupervised, and reinforcement learning, along with their respective algorithms, evaluation metrics, and challenges. Additionally, it discusses future trends like AutoML and Explainable AI, emphasizing the importance of practical application for deeper understanding.

Uploaded by

fredrickbossy8
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
8 views

data science notes b

The document provides an overview of machine learning, defining it as a subset of AI that enables systems to learn from data. It covers key categories such as supervised, unsupervised, and reinforcement learning, along with their respective algorithms, evaluation metrics, and challenges. Additionally, it discusses future trends like AutoML and Explainable AI, emphasizing the importance of practical application for deeper understanding.

Uploaded by

fredrickbossy8
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as DOCX, PDF, TXT or read online on Scribd
You are on page 1/ 5

1.

Introduction to Machine Learning

 Definition: Machine Learning is a subset of Artificial Intelligence (AI) where systems learn
patterns and make decisions from data without explicit programming.
 Categories:
o Supervised Learning: Models are trained on labeled data (inputs and corresponding
outputs).
o Unsupervised Learning: Models work with unlabeled data to discover patterns and
structures.
o Reinforcement Learning: Models learn by interacting with an environment to maximize
rewards.

2. Supervised Learning

 Goal: Learn a mapping from inputs to outputs using labeled data.

Types of Supervised Learning:

1. Classification: Predicting a category or class label for a given input.


o Examples: Spam detection, Image recognition, Sentiment analysis.
o Algorithms:
 Logistic Regression: A simple algorithm for binary classification.
 K-Nearest Neighbors (KNN): Classifies based on the majority class among the
nearest neighbors.
 Support Vector Machines (SVM): Finds a hyperplane that best separates
classes.
 Decision Trees: Tree-like structures used for classification or regression.
 Random Forests: Ensemble method using multiple decision trees to improve
accuracy.
 Naive Bayes: Based on Bayes' theorem, commonly used for text classification.
 Neural Networks: Layers of interconnected nodes (neurons), particularly useful
for complex classification tasks.

2. Regression: Predicting a continuous value.


o Examples: Predicting house prices, Stock market prediction, Temperature forecasting.
o Algorithms:
 Linear Regression: Fits a linear relationship between input features and the
target.
 Polynomial Regression: Extends linear regression by adding polynomial terms.
 Ridge and Lasso Regression: Regularized versions of linear regression to prevent
overfitting.
 Support Vector Regression (SVR): Uses support vectors to predict continuous
values.
Evaluation Metrics for Supervised Learning:

 Classification:
o Accuracy: Proportion of correct predictions.
o Precision: Proportion of true positives among predicted positives.
o Recall (Sensitivity): Proportion of true positives among actual positives.
o F1-Score: Harmonic mean of precision and recall.
o ROC-AUC: Area under the receiver operating characteristic curve.
 Regression:
o Mean Absolute Error (MAE): Average of the absolute errors.
o Mean Squared Error (MSE): Average of the squared errors.
o Root Mean Squared Error (RMSE): Square root of MSE.
o R² (Coefficient of Determination): Proportion of variance explained by the model.

3. Unsupervised Learning

 Goal: Discover hidden patterns in data without labeled outputs.

Types of Unsupervised Learning:

1. Clustering: Grouping data points into clusters based on similarity.


o Algorithms:
 K-Means: Partitioning data into k clusters.
 Hierarchical Clustering: Builds a tree of clusters (dendrogram).
 DBSCAN: Density-based clustering algorithm.
 Agglomerative Clustering: Bottom-up approach to clustering.
2. Dimensionality Reduction: Reducing the number of features while preserving essential
information.
o Algorithms:
 Principal Component Analysis (PCA): Linearly transforms the features into a set
of orthogonal components that explain the most variance.
 t-Distributed Stochastic Neighbor Embedding (t-SNE): A non-linear technique
for dimensionality reduction.
 Linear Discriminant Analysis (LDA): A technique that finds a linear combination
of features that best separate multiple classes.

Evaluation Metrics for Unsupervised Learning:

 Clustering:
o Silhouette Score: Measures how similar a point is to its own cluster compared to other
clusters.
o Davies-Bouldin Index: Lower values indicate better clustering.
o Adjusted Rand Index (ARI): Measures the similarity between two clustering results.
4. Reinforcement Learning (RL)

 Goal: Learn an optimal action strategy by interacting with an environment to maximize


cumulative rewards over time.

Key Concepts:

 Agent: The learner or decision maker.


 Environment: The external system that the agent interacts with.
 Action: Choices made by the agent.
 State: The current condition of the environment.
 Reward: Feedback signal after each action, used to guide learning.
 Policy: The strategy that the agent follows to make decisions.
 Value Function: Estimate of the total accumulated reward from a given state.

Key Algorithms:

 Q-Learning: Off-policy algorithm where the agent learns the value of actions in states.
 Deep Q-Networks (DQN): Uses deep learning to approximate Q-values for large state spaces.
 Policy Gradient Methods: Directly learns the policy rather than the value function.
 Proximal Policy Optimization (PPO): A more stable reinforcement learning algorithm used in
complex environments.

5. Model Evaluation and Tuning

 Overfitting and Underfitting:


o Overfitting: Model performs well on training data but poorly on unseen data.
o Underfitting: Model fails to capture patterns in the data, leading to poor performance
on both training and testing data.

 Cross-Validation: Splitting the data into multiple subsets (folds) to evaluate the model
performance more reliably (e.g., K-fold cross-validation).
 Hyperparameter Tuning: Adjusting model parameters to optimize performance.
Techniques include:
o Grid Search: Exhaustively searches a hyperparameter space.
o Random Search: Randomly samples the hyperparameter space.
o Bayesian Optimization: Uses probabilistic models to guide the search for optimal
hyperparameters.

6. Ensemble Learning

 Goal: Combine multiple models to improve performance.


Popular Ensemble Techniques:

1. Bagging (Bootstrap Aggregating):


o Random Forest: Combines multiple decision trees by averaging their outputs to improve
accuracy and reduce overfitting.
2. Boosting:
o AdaBoost: Weighs classifiers to correct the mistakes of previous ones.
o Gradient Boosting: Iteratively improves the model by minimizing the residual error.
o XGBoost: Optimized gradient boosting method known for high performance and speed.
o LightGBM: A gradient boosting framework designed for speed and efficiency.
3. Stacking: Combines the predictions of several base models through a meta-model.

7. Neural Networks and Deep Learning

 Neural Networks: A network of interconnected nodes (neurons) inspired by the human brain,
useful for capturing complex patterns.

Types of Neural Networks:

1. Feedforward Neural Networks (FNN): The simplest type, where data flows in one direction from
input to output.
2. Convolutional Neural Networks (CNN): Primarily used for image processing and computer vision
tasks.
3. Recurrent Neural Networks (RNN): Best suited for sequential data such as time series or text.
o Long Short-Term Memory (LSTM): A type of RNN designed to handle long-range
dependencies.
4. Transformers: State-of-the-art architecture for NLP tasks (e.g., BERT, GPT).

Deep Learning Frameworks:

 TensorFlow, Keras, PyTorch, and MXNet are popular frameworks used for building neural
network models.

8. Common Challenges in Machine Learning

 Data Quality: Poor data quality, missing values, or noisy data can impact model performance.
 Scalability: Handling large datasets efficiently.
 Interpretability: Some models, especially deep learning models, can be difficult to interpret.
 Bias and Fairness: Ensuring models do not perpetuate bias or discriminatory outcomes.
9. Future Trends in Machine Learning

 AutoML: Automation of the end-to-end process of applying machine learning to real-world


problems.
 Explainable AI (XAI): Focus on making machine learning models more transparent and
interpretable.
 Federated Learning: A decentralized approach to training models using data from multiple
sources without sharing raw data.

These notes cover the essential concepts and algorithms in machine learning. Machine learning is
an expansive field, and a deeper understanding comes with practical application and
experimentation.

You might also like