Introduction to Machine Learning
Machine Learning is a subset of artificial intelligence that enables systems to learn and improve from experience without being explicitly programmed.
Types of Machine Learning
**Supervised Learning**: Learn from labeled data
**Unsupervised Learning**: Find patterns in unlabeled data
**Reinforcement Learning**: Learn through trial and error
Supervised Learning
Regression
Predicts continuous values.
**Linear Regression**: y = mx + c
Simple, interpretable
Assumes linear relationship
**Multiple Regression**: Multiple features
`y = β₀ + β₁x₁ + β₂x₂ + ... + βₙxₙ`
Classification
Predicts discrete categories.
**Logistic Regression**: Binary classification using sigmoid function
**Decision Trees**: Tree-based decision rules
**Random Forest**: Ensemble of decision trees
**SVM**: Finds optimal hyperplane for separation
**K-NN**: Classifies based on nearest neighbors
Evaluation Metrics
**Accuracy**: (TP + TN) / Total
**Precision**: TP / (TP + FP)
**Recall**: TP / (TP + FN)
**F1 Score**: 2 × (Precision × Recall) / (Precision + Recall)
**ROC-AUC**: Area under ROC curve
Unsupervised Learning
Clustering
**K-Means**: Partition into K clusters
**DBSCAN**: Density-based clustering
**Hierarchical Clustering**: Tree of clusters
Dimensionality Reduction
**PCA**: Principal Component Analysis
**t-SNE**: Visualization of high-dimensional data
**Autoencoders**: Neural network-based reduction
Neural Networks
Perceptron
The simplest neural network — a single neuron.
Multi-Layer Perceptron (MLP)
Input layer → Hidden layers → Output layer
Activation functions: ReLU, Sigmoid, Tanh
Backpropagation for training
Deep Learning
**CNNs**: For image data (convolutional layers)
**RNNs**: For sequential data (LSTM, GRU)
**Transformers**: Attention-based architecture (BERT, GPT)
Training Process
Forward pass: Compute predictions
Loss calculation: Compare predictions to targets
Backward pass: Compute gradients
Weight update: Gradient descent
Model Evaluation & Validation
Overfitting vs Underfitting
**Overfitting**: Model learns noise, poor generalization
**Underfitting**: Model too simple, poor on training data
Solutions
Cross-validation (k-fold)
Regularization (L1, L2)
Early stopping
Dropout (for neural networks)
More training data
Bias-Variance Tradeoff
High bias → Underfitting
High variance → Overfitting
Goal: Find optimal balance