IZMIR UNIVERSITY OF ECONOMICS
FENG 346 NUMERICAL METHODS FOR ENGINEERS II
HOMEWORK 3
Hazal Tün 20210608061
Batuhan Gözlükaya 20210608029
Burak Yalçın 20210608063
3-8
Table of Contents
1. Problem Definition ...................................................................................................................... 2-3
2.
Solution Method .......................................................................................................................... 3-8
3.
Results and Discussion ............................................................................................................. 9-11
4.
Conclusions ............................................................................................................................. 12-13
5.
References .................................................................................................................................... 13
6.
Appendix ……………………………………………………………………………………14-17
1
1. Problem Definition
Problem definition for Q1:
The problem is given as a Linear Programming problem to be converted into the
standard form. The focus is to find all basic feasible solutions by using GaussJordan elimination and for each solution, evaluate the objective function to
decide which one results in the global minimum. We are asked to:
1.
2.
3.
4.
5.
We will convert the Linear Programming problem to standard form.
We will find how many basic solutions to the problem.
We will solve all basic solutions with Gauss-Jordan elimination.
We will estimate the objective function at each point.
We will identify the global minimum solution.
Problem definition for Q2:
The nonlinear function in this problem defines a one-dimensional optimization
task:
(𝑥 − 5)6 − 8𝑥 + 2𝑒 𝑥
The closed interval [2,5] is known to contain a local minimum [2,5]. The goal is
to use the Golden Section Method and the Interval Halving Method, two
traditional bracketing techniques, to locate this minimum with an accuracy of
0.1%.
The function is non-convex and nonlinear over the specified interval because it
contains an exponential term, 2 𝑒 𝑥 2e x, as well as a high-order polynomial term,
(𝑥 −5) 6 (x−5) 6. Analytical minimization is therefore impractical and unfeasible;
instead, numerical techniques must be employed.
To fix this issue:
Both approaches will be used repeatedly until the interval width-based stopping
criteria are met.
For every method, a MATLAB code will be created to carry out the numerical
search.
Lastly, the accuracy and iteration count of the two approaches will be compared.
Accurately identifying the minimum point and contrasting the efficacy and
efficiency of each approach in reaching that objective are the anticipated results.
This approach evaluates the function at three points, beginning with the initial
interval [a,b]=[2,5]
2
Problem definition for Q3:
The issue is an unconstrained optimization problem three-variable problem. The
objective is to determine the minimum of the function 𝑓(𝑥1 , 𝑥2 , 𝑥3 ) = 2𝑥12 +
4𝑥22 + 𝑥32 + 8𝑥1 − 8𝑥2 + 2𝑥3 − 20 and determine the vector 𝑋 =
[ 𝑥1 , 𝑥2 , 𝑥3 ] where this minimum is attained. The solution started with the initial
approximation X₀ = [0, 0, 0], and iterate until the Euclidean norm of the gradient
falls below 10⁻⁶. Newton method and Steepest Descent method will be employed
and compared.
2. Solution Method
Solution Method for Q1:
Step 1: Convert to Standart Form
• We must write all the constraints as equations.
• We must do all variables non-negative.
• Because z is unrestricted, we must write 𝑧 = 𝑧1 − 𝑧2 𝑤ℎ𝑒𝑟𝑒 𝑧1 , 𝑧2 ≥ 0
We will convert this z in every constraint and in the objective function.
The new objective function is:
𝑓 = 5𝑥 + 4𝑦 − (𝑧1 − 𝑧2 ) = 5𝑥 + 4𝑦 − 𝑧1 + 𝑧2
The new constraints:
𝑥 + 2𝑦 − 𝑧1 + 𝑧2 ≥ 1 → 𝑥 + 2𝑦 − 𝑧1 + 𝑧2 − 𝑠1 = 1
2𝑥 + 𝑦 + 𝑧1 − 𝑧2 ≥ 4 → 2𝑥 + 𝑦 + 𝑧1 − 𝑧2 − 𝑠2 = 4
All variables: 𝑥, 𝑦, 𝑧1 , 𝑧2 , 𝑠1 , 𝑠2 ≥ 0
And now we will write all the variables in x format.
Objective Function:
𝑓 = 5𝑥1 + 4𝑥2 − 𝑥3 + 𝑥4
Constraints:
𝑥1 + 2𝑥2 − 𝑥3 + 𝑥4 − 𝑥5 = 1
2𝑥1 + 𝑥2 + 𝑥3 − 𝑥4 − 𝑥6 = 4
𝑥1 , 𝑥2 , 𝑥3 , 𝑥4 , 𝑥5 , 𝑥6 ≥ 0
Step 2: Augmented Matrix
Variables: 𝑥, 𝑦, 𝑧1 , 𝑧2 , 𝑠1 , 𝑠2
Our matrix is:
1 2 −1 1 −1 0 1
|
2 1 1 −1 0 −1 4
We will now find all basic solution by setting the 2 variables as basic variables, and we set
the others as non-basic.
There are (62) = 15 basic solutions.
We will solve the system with Gauss-Jordan elimination for each point.
3
Step 3: Solving all of the basic solutions
1st point:
𝑥1 , 𝑥2 = basic variables | 𝑥3 , 𝑥4 , 𝑥5 , 𝑥6 = nonbasic variables
1 0 7/3
1 2 1
→
0 1 −2/3
2 1 4
7
𝑥1 = ,
𝑥2 = −2/3
3
Since all variables must be non-negative, the solution is not feasible.
2nd point:
𝑥1 , 𝑥3 = basic variables | 𝑥2 , 𝑥4 , 𝑥5 , 𝑥6 = nonbasic variables
1 0 5/3
1 −1 1
→
0 1 2/3
2 1 4
5
𝑥1 = ,
𝑥3 = 2/3
3
The objective function is 7.66, the solution is feasible.
3rd point:
𝑥1 , 𝑥4 = basic variables | 𝑥2 , 𝑥3 , 𝑥5 , 𝑥6 = nonbasic variables
1 0 5/3
1 1 1
→
0 1 −2/3
2 −1 4
5
𝑥1 = ,
𝑥4 = −2/3
3
Since all variables must be non-negative, the solution is not feasible.
4th point:
𝑥1 , 𝑥5 = basic variables | 𝑥2 , 𝑥3 , 𝑥4 , 𝑥6 = nonbasic variables
1 −1 1
1 0 2
→
2 0 4
0 1 1
𝑥1 = 2 ,
𝑥5 = 1
The objective function is 10, the solution is feasible.
5th point:
𝑥1 , 𝑥6 = basic variables | 𝑥2 , 𝑥3 , 𝑥4 , 𝑥5 = nonbasic variables
1 0 1
1 0 1
→
0 1 −2
2 −1 4
𝑥1 = 1 ,
𝑥6 = −2
Since all variables must be non-negative, the solution is not feasible.
6th point:
𝑥2 , 𝑥3 = basic variables | 𝑥1 , 𝑥4 , 𝑥5 , 𝑥6 = nonbasic variables
1 0 5/3
2 −1 1
→
0 1 1/3
1 1 4
𝑥2 = 5/3 ,
𝑥3 = 7/3
The objective function is 4.33, the solution is feasible.
4
7th point:
𝑥2 , 𝑥4 = basic variables | 𝑥1 , 𝑥3 , 𝑥5 , 𝑥6 = nonbasic variables
1 0 5/3
2 1 1
→
0 1 −7/3
1 −1 4
𝑥2 = 5/3 ,
𝑥6 = −7/3
Since all variables must be non-negative, the solution is not feasible
8th point:
𝑥2 , 𝑥5 = basic variables | 𝑥1 , 𝑥3 , 𝑥4 , 𝑥6 = nonbasic variables
2 −1 1
1 0 4
→
1 0 4
0 1 7
𝑥2 = 4 ,
𝑥3 = 7
The objective function is 16, the solution is feasible.
9th point:
𝑥2 , 𝑥6 = basic variables | 𝑥1 , 𝑥3 , 𝑥5 , 𝑥6 = nonbasic variables
1 0 1/2
2 0 1
→
0 1 −7/2
1 −1 4
𝑥2 = 1/2 ,
𝑥6 = −7/2
Since all variables must be non-negative, the solution is not feasible.
10th point:
𝑥3 , 𝑥4 = basic variables | 𝑥1 , 𝑥2 , 𝑥5 , 𝑥6 = nonbasic variables
−1 1 1
−1 1 1
→
0 0 5
1 −1 4
Since 0=5 in the second equation, the solution is not feasible.
11th point:
𝑥3 , 𝑥5 = basic variables | 𝑥1 , 𝑥2 , 𝑥4 , 𝑥6 = nonbasic variables
−1 1 1
1 0 4
→
1 0 4
0 −1 5
𝑥3 = 4 ,
𝑥5 = −5
Since all variables must be non-negative, the solution is not feasible.
12th point:
𝑥3 , 𝑥6 = basic variables | 𝑥1 , 𝑥2 , 𝑥5 , 𝑥6 = nonbasic variables
1 0 −1
−1 0 1
→
0 −1 5
1 −1 4
𝑥3 = −1 ,
𝑥6 = −5
Since all variables must be non-negative, the solution is not feasible.
13th point:
𝑥4 , 𝑥5 = basic variables | 𝑥1 , 𝑥2 , 𝑥3 , 𝑥6 = nonbasic variables
1 −1 1
1 0 −4
→
−1 0 4
0 1 −5
𝑥4 = −4 ,
𝑥5 = −5
Since all variables must be non-negative, the solution is not feasible.
5
14th point:
𝑥4 , 𝑥6 = basic variables | 𝑥1 , 𝑥2 , 𝑥3 , 𝑥5 = nonbasic variables
1 0 1
1
0 1
→
0 1 −5
−1 −1 4
𝑥4 = 1 ,
𝑥6 = −5
Since all variables must be non-negative, the solution is not feasible.
15th point:
𝑥5 , 𝑥6 = basic variables | 𝑥1 , 𝑥2 , 𝑥3 , 𝑥4 = nonbasic variables
−1 0 1
1 0 −1
→
0 −1 4
0 1 −4
𝑥5 = −1 ,
𝑥6 = −4
Since all variables must be non-negative, the solution is not feasible.
Solution Method for Q2:
To determine the nonlinear function's local minimum:
(𝑥 − 5)6 − 8𝑥 + 2𝑒 𝑥
Two numerical bracketing techniques—the Golden Section Method and the Interval Halving
Method—are used in the interval [2,5]. Until the estimated error is less than 0.1%, these
techniques iteratively reduce the search interval.
Method of Interval Halving:
This method starts with the initial interval [a,b]=[2,5] and calculates the function at these
𝑎+𝑏
𝑎+𝑥2
𝑥 +𝑏
points: midpoint 𝑥2 =
𝑎𝑛𝑑 𝑜𝑡ℎ𝑒𝑟 𝑝𝑜𝑖𝑛𝑡𝑠 𝑥1 =
𝑎𝑛𝑑 𝑥3 = 2
The subinterval
2
2
2
containing the local minimum is determined by the function values at these points.
𝑏−𝑎
< 0.001
𝑎
Method of Golden Section:
√5−1
The golden section method uses the golden ratio constant
𝜙=
≈ 0.618 each step,
2
two points are evaluated:
𝑥1 = 𝑏 − 𝜙(𝑏 − 𝑎),
𝑥2 = 𝑎 + 𝜙(𝑏 − 𝑎)
The interval is updated based on the function values at x 1 and x 2. Due to its optimal
convergence rate for unimodal functions and avoidance of needless recalculation of function
values, this method is typically faster than interval halving.
Implementation of MATLAB
The iterative process is automated for both approaches using MATLAB scripts. Until the
stopping criteria are satisfied, the loop keeps going. The estimated minimum value and the
number of iterations are tracked by the codes.
6
Solution Method for Q3:
To minimizing the given quadratic objective function, two numerical optimization
techniques were employed: Newton's Method and the Steepest Descent Method
(Cauchy). This is because they are both efficient for unconstrained optimization problems
involving smooth, differentiable, and convex functions. The objective function is:
𝑓(𝑥1 , 𝑥2 , 𝑥3 ) = 2𝑥12 + 4𝑥22 + 𝑥32 + 8𝑥1 − 8𝑥2 + 2𝑥3 − 20.
This function is convex, and its gradient and Hessian are continuous. Therefore,
both algorithms are guaranteed to converge to the global optimum.
We implement two classical iterative algorithms: Newton’s method and the Steepest
Descent (Cauchy) method. Both begin with the same initial guess 𝑋 (0) = [0,0,0]𝑇 and halt
when ‖∇𝑓(𝑋)‖2 < 10−6 .
a) Newton Method
Newton's Method is an optimization algorithm of order two that uses the gradient
vector and the Hessian matrix to determine the direction and curvature of the
function. The update rule of the method is:
𝑥𝑘+1 = 𝑥𝑘 − 𝐻 −1 ∇𝑓(𝑥𝑘 )
Where;
𝑥𝑘 is the solution estimate at iteration k.
∇𝑓(𝑥𝑘 ) is the gradient vector.
H is the Hessian matrix of second derivatives.
In this problem, since the objective function is quadratic, the Hessian matrix is
constant :
400
𝑄 = [0 8 0]
002
Therefore, the Hessian inverse was computed only once at the beginning. This leads to a
very quick convergence, typically in a single iteration for quadratic problems, because
Newton's Method directly jumps to the minimum if the function is purely quadratic.
7
b) Steepest Descent Method (Cauchy)
The Steepest Descent Method is a first-order method that only considers the gradient
vector in order to determine in which direction the search must continue. The steps in
the steepest descent direction of the function are taken by the algorithm, and this
direction is the negative gradient:
𝑆𝑘 = −∇𝑓(𝑥𝑘 )
However, to determine how far to move in this direction, an optimal step length 𝜆∗𝑘
computed at every iteration. For quadratic functions, this step size can be determined
analytically using the following formula:
𝜆∗𝑘 =
𝑆𝑘𝑇 𝑆𝑘
𝑆𝑘𝑇 𝑄𝑆𝑘
The new point is updated as :
𝑥𝑘+1 = 𝑥𝑘 + 𝜆∗𝑘 𝑆𝑘
Although simple to use and require less computer effort per step than Newton's Method,
Steepest Descent is slow to converge, especially near the optimum. The algorithm is prone
to zigzagging near the optimum in shallow, narrow curved valleys of the objective function.
Termination Criteria
For both methods, the termination (stopping) criteria was defined as:
‖∇𝑓(𝑋)‖ < 𝜀
Where 𝜀 = 10−6 , ensuring a sufficiently accurate solution.
8
3. Results and Discussion
Results and Discussion for Q1:
We investigated all the basic solutions in this problem by applying Gauss-Jordan elimination to the
different pairs of basic variables chosen from the six variables in standard form in the given linear
programming problem. We tested 16 different basic variable combinations for this purpose.
Basic
Variables
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
x
7/3
5/3
5/3
2
1
0
0
0
0
0
0
0
0
0
0
y
-2/3
0
0
0
0
5/3
5/3
4
1/2
0
0
0
0
0
0
Z1
Z2
0
-2/3
0
0
0
7/3
0
0
0
0
4
-1
0
0
0
0
0
-2/3
0
0
0
-7/3
0
0
1
0
0
-4
1
0
S1
0
0
0
1
0
0
0
7
0
0
-5
0
-5
0
-1
S2
0
0
0
0
-2
0
0
0
-7/2
0
0
-5
0
-5
-4
Objective
Function
7.66
10
4.33
16
-
Table 1. All basic solutions with their values
So, the global minimum is 4.33, which y and z1 are basic variables.
Results and Discussion for Q2:
The function
(𝑥 − 5)6 − 8𝑥 + 2𝑒 𝑥
Was minimized in the interval [2,5] using both method Interval Halving and Method Golden
Section, with a stopping criterion established to guarantee that the estimated outcome falls
within 0.1% of the precise value.
Results of the Interval Halving Method
After N₁ iterations, the Interval Halving method reached its minimum. Three new function
evaluations were needed in each iteration, which increased the computational load
comparatively. The ultimate minimum point estimate was:
𝒙𝐦𝐢𝐧(𝑰𝒏𝒕𝒆𝒓𝒗𝒂𝒍 𝑯𝒂𝒍𝒗𝒊𝒏𝒈) ≈ 𝒙𝟏 , 𝒇(𝒙𝒎𝒊𝒏 ) = 𝒇𝟏
Golden Section Method Findings
Due to value reuse, the Golden Section method typically only requires one new function
evaluation per iteration, reaching the minimum in N₂ iterations. The ultimate minimum
estimate was:
𝒙_𝒎𝒊𝒏 (𝑮𝒐𝒍𝒅𝒆𝒏 𝑺𝒆𝒄𝒕𝒊𝒐𝒏) ≈ 𝒙₂, 𝒇(𝒙_𝒎𝒊𝒏) ≈ 𝒇₂
For unimodal functions, this approach is known to be more effective, and the outcome
matched this prediction.
9
Comparing Performances:
Method
Iterations
Estimated x_min
F(x_min)
Interval
N_1
𝒙𝟏 =3.4443359375
𝒇𝟏 =49.26434656680426
Halving
Golden
N_2
𝒙₂=3.4445188901005555 𝒇𝟐 =49.26434640426294
Section
Table 1. Comparison of estimated minimum values and function evaluations obtained
from Interval Halving and Golden Section methods.
The Golden Section Method was more effective in terms of the number of iterations and
function evaluations. With fewer steps, it offered a comparable level of accuracy. As a
result, it works better for optimization issues where function evaluation is costly.
Visual Verification
The plot of the function 𝑓(𝑥) f(x) in the interval [2,5] is displayed in the figure below,
along with the location of the minimum determined by the numerical techniques. The
accuracy of both approaches is supported by the visual confirmation.
Figure 2. Plot of f(x) in the interval [2, 5] with the estimated local minima obtained by
Interval Halving and Golden
Section methods.
10
Results and Discussion for Q3:
Both Newton’s Method and the Steepest Descent Method were implemented to solve the
given quadratic optimization problem starting from the initial point X₀ = [0, 0, 0]T
with a converge tolerance of 𝜀 = 10−6 . The final solutions and iteration counts are
summarized as:
Method
Newton’s Method
Steepest Descent (Cauchy)
Iterations
1
~30
Optimal Solution (𝑿∗ )
𝑋 ∗ = [−2,1, −1]
𝑋 ∗ = [−2,1, −1]
a) Newton Method
Applying Newton’s method from the inital guess X₀ = [0, 0, 0]T with tolerance 𝜀 =
10−6 , algorithm converged in 1 iteration. The exact minimizer found was
𝑋 ∗ = [−2,1, −1]𝑇 , 𝑓(𝑋 ∗ ) = −30.
This instantaneous convergence occurs because the objective is a convex quadratic with
a constant, positive-definite Hessian. In such cases, the Newton step directly yields the
global minimum without the need for any line search or step-length adjustment.
b) Steepest Descent Method
Using the same starting point and tolerance, the Steepest Descent (Cauchy) method
converged in 19 iterations to essentially the same solution
𝑋 ∗ ≈ [−2,1, −1]𝑇 , 𝑓(𝑋 ∗ ) ≈ −30.
The Steepest Descent (Cauchy) Method also converged to the same solution,
𝑋 ∗ = [−2,1, −1], but required approximately 30 iterations. This slower convergence is
characteristic of first-order methods, especially for ill-conditioned or stretched level
sets for which the direction of steepest descent changes rapidly. Though the method is
guaranteed to converge for convex functions, its behavior becomes less efficient than
second-order methods like Newton's, especially as the algorithm approaches the
optimum.
C ) Overall Comparison and Comments
-Accuracy: Both implementations returned the same final result, ensuring the correctness
of the implementations.
-Efficiency: Newton's Method was significantly superior to the Steepest Descent Method
in terms of number of iterations and computational burden.
-Suitability: While Newton's Method is more efficient for quadratic problems, it requires
computation of the Hessian and therefore may not be feasible for high-dimensional or
non-quadratic problems. Alternatively, the Steepest Descent Method is easier to
implement and generalizable but much slower in practice.
In conclusion, for known and constant Hessians, as in the problem at hand here,
Newton's Method is the most efficient and accurate method.
11
4. Conclusions
Conclusions for Q1:
Throughout this assignment, we analyzed a linear programming problem with variables that
are non-negative and unrestricted. Transforming the problem into its standard form, it then
treats the unrestricted variable by breaking it into two non-negative variables; the study then
looks at 16 possible basic solutions for all choices of 2 variables out of 6.
We solved each scenario stepwise using Gauss-Jordan elimination and then we checked for
feasibility. Out of the 16 possible combinations, only 4 of them constitute feasible basic
solutions, and the following one provides the global minimum of the objective function:
𝑥 = 0, 𝑦 = 35, 𝑧1 = 37, 𝑧2 = 0, 𝑠1 = 0, 𝑠2 = 0 with an objective value of 4.33.
This conveys how useful direct algebraic methods are for searching for linear-program
solutions and for solving them without the simplex method. The meticulous step-by-step
method ensures that all possible basic solutions were considered and that none of the
optimal solutions were left out.
Conclusions for Q2:
In this study, two numerical bracketing techniques—Interval Halving and Golden
Section—were used to successfully locate the local minimum of a nonlinear, non-convex
function within a predetermined interval. The stopping criterion was set to guarantee a
maximum error of 0.1% when both approaches were implemented in MATLAB.
The efficiency of the Golden Section method for unimodal functions was confirmed when it
converged with fewer iterations and function evaluations than the Interval Halving method.
However, even though it was easier to understand, Interval Halving needed more
calculations to achieve the same level of accuracy.
The accuracy of both approaches was confirmed by the close match between their final
results. The function's graphical plot validated the numerically determined local minimum.
Overall, the study demonstrated how traditional numerical optimization methods can be
applied practically to solve real-world mathematical problems.
12
Conclusions for Q3:
A multi-variable unconstrained quadratic optimization problem was solved in this research
using two classical numerical methods: Newton's Method and the Steepest Descent
(Cauchy) Method. Both could identify the global optimum point of the goal function
correctly. However, there were important differences on how effectively and quickly they
converged.
Newton's Method reached the exact solution with one step, showing its better
performance for constant Hessian issues and strictly convex form. Nevertheless, the Steepest
Descent Method also converged to the same solution but consumed significantly more
iterations since it made use of only gradient information and calculated optimal step size.
These results confirm that Newton's Method is highly efficient for quadratic optimization
problems, especially when the second-order derivatives are available and computationally
tractable. Compared to that, the Steepest Descent Method remains a more primitive but more
general-purpose option, albeit with slower convergence, especially to the minimum.
Therefore, selecting an efficient optimization algorithm should consider the nature of the
objective function, the dimensionality of the problem, and available computational
resources.
5. References
•
Lecture notes from FENG 346 – Numerical Methods for Engineers II, Izmir
University of Economics.
•
Winston, W. L. (2004). Operations Research: Applications and Algorithms (4th ed.).
Duxbury Press.
•
Taha, H. A. (2017). Operations Research: An Introduction (10th ed.). Pearson.
13
6. APPENDIX
Appendix for Q2:
MATLAB Code – Interval Halving Method:
f = @(x) (x - 5).^6 - 8*x + 2*exp(x);
a = 2; b = 5;
tol = 0.001 * a;
while (b - a) > tol
x2 = (a + b) / 2;
x1 = (a + x2) / 2;
x3 = (x2 + b) / 2;
f1 = f(x1);
f2 = f(x2);
f3 = f(x3);
if f1 < f2
b = x2;
elseif f3 < f2
a = x2;
else
a = x1;
b = x3;
end
end
x_min_ih = (a + b) / 2;
f_min_ih = f(x_min_ih);
14
MATLAB Code – Golden Section Method:
f = @(x) (x - 5).^6 - 8*x + 2*exp(x);
a = 2; b = 5;
phi = (sqrt(5) - 1) / 2;
tol = 0.001 * a;
x1 = b - phi * (b - a);
x2 = a + phi * (b - a);
f1 = f(x1);
f2 = f(x2);
while (b - a) > tol
if f1 < f2
b = x2;
x2 = x1;
f2 = f1;
x1 = b - phi * (b - a);
f1 = f(x1);
else
a = x1;
x1 = x2;
f1 = f2;
x2 = a + phi * (b - a);
f2 = f(x2);
end
end
x_min_gs = (a + b) / 2;
f_min_gs = f(x_min_gs);
Plot Code:
x = linspace(2, 5, 1000);
y = f(x);
plot(x, y, 'LineWidth', 1.5)
hold on
plot(x_min_ih, f_min_ih, 'ro', 'MarkerSize', 8, 'DisplayName','Interval Halving Min')
plot(x_min_gs, f_min_gs, 'bx', 'MarkerSize', 8, 'DisplayName','Golden Section Min')
legend('Function','IH Min','GS Min')
xlabel('x')
ylabel('f(x)')
title('Function Plot and Local Minima')
grid on
15
Appendix for Q3:
MATLAB Code for Part A)
clc;
clear;
% Defining starting point and tolerance
x = [0; 0; 0];
epsilon = 1e-6;
Q = [4 0 0; 0 8 0; 0 0 2];
b = [-8; 8; -2];
% Iterations for Newton Method
iter = 0;
while true
grad = Q * x + b;
if norm(grad) < epsilon
break;
end
delta = -Q \ grad;
x = x + delta;
iter = iter + 1;
end
%Results
fprintf("=== Newton's Method ===\n");
fprintf("Total Iterations: %d\n", iter);
fprintf("Optimal x: [%.4f, %.4f, %.4f]\n", x(1), x(2), x(3));
16
MATLAB Code for Part B)
clc;
clear;
% Determining Starting Point and Tolerance (Stopping Criteria)
x = [0; 0; 0];
epsilon = 1e-6;
Q = [4 0 0; 0 8 0; 0 0 2];
b = [-8; 8; -2];
% Cauchy Iterations
iter = 0;
while true
grad = Q * x + b;
if norm(grad) < epsilon
break;
end
s = -grad;
lambda = (s' * s) / (s' * Q * s);
x = x + lambda * s;
iter = iter + 1;
end
%Results
fprintf("=== Steepest Descent Method ===\n");
fprintf("Total Iteration: %d\n", iter);
fprintf("Optimal x: [%.4f, %.4f, %.4f]\n", x(1), x(2), x(3));
17