Scribd
Upload
49 views17 pages

Optimization Gradient Descent Method

The document discusses numerical optimization, emphasizing its applications in various fields such as machine learning, engineering, and economics. It details the gradient descent method, an iterative algorithm used to find the minimum or maximum of a function by moving against the gradient. Additionally, it covers one-dimensional and multi-dimensional gradient descent techniques and mentions program development in MATLAB for visualizing optimization results.

Uploaded by

mehdihasan261098
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
49 views17 pages

Optimization Gradient Descent Method

The document discusses numerical optimization, emphasizing its applications in various fields such as machine learning, engineering, and economics. It details the gradient descent method, an iterative algorithm used to find the minimum or maximum of a function by moving against the gradient. Additionally, it covers one-dimensional and multi-dimensional gradient descent techniques and mentions program development in MATLAB for visualizing optimization results.

Uploaded by

mehdihasan261098
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 17

Lab – 07

Optimization using Gradient Descent


Method
Numerical Analysis LAB
B.Sc. In ME
Section B
Numerical Optimization
Numerical optimization is a fundamental tool in mathematics, computer science, engineering, and
economics, designed to solve problems involving the minimization or maximization of a mathematical
function.

Numerical optimization is widely used across disciplines:

• Machine Learning: Training models by minimizing loss functions (e.g., stochastic gradient descent).

• Engineering Design: Optimizing structures, materials, or processes.

• Economics: Maximizing utility or profit under constraints.

• Robotics and Control: Path planning and motion optimization.

• Physics and Chemistry: Solving energy minimization problems.


Numerical Optimization Techniques
Gradient Descent Method
• Gradient descent is a first-order iterative optimization algorithm that seeks to find the minimum
(or maximum) of a function 𝑓(𝑥). It does this by moving step-by-step in the opposite direction of
the gradient of the function at the current point.

• The gradient is the vector of partial derivatives of the function, pointing in the direction of the
steepest ascent. By taking steps in the negative gradient direction, the algorithm ensures the
function value decreases at each step.
Concept
Concept
One Dimensional Gradient Descent
In one-dimensional gradient descent, the

variable 𝑓(𝑥). The algorithm seeks to minimize (or


optimization is performed on a function of a single

value of 𝑥.
maximize) the function by iteratively updating the
Multi Dimensional Gradient Descent

performed on a function of multiple variables 𝑓(𝑥), where


In multi-dimensional gradient descent, the optimization is

𝑥=[𝑥1,𝑥2,…,𝑥𝑛]. The goal is to minimize (or maximize) the


function by updating all variables simultaneously based on their
partial derivatives.
1-D vs Multi-D Gradient Descent
Algorithm of Gradient Descent Method
Algorithm of Gradient Descent Method
Program Development in MATLAB
Program Development in MATLAB

Visualization of the function in 3D Surface and Contour Plots


Visualization of the function in 3D Surface and Contour Plots
Program Development in MATLAB
Output Visualization
Thank You

You might also like