Stochastic Gradient Descent (SGD)

Optimization algorithm that updates parameters using gradients from single training examples or small batches.