ECE 6397, Fall, 2012
Selected Topic in Optimization
Zhu Han
Department of Electrical and Computer Engineering
Class 1
Aug. 27nd, 2012
Outline

Instructor information

Motivation to study optimization

Course descriptions and textbooks

What you will study from this course

Objectives

Coverage and schedule

Homework, projects, and exams

Other policies

Reasons to be my students

Background and Preview
Instructor Information

Office location: Engineering 2 W302

Office hours: M 10am-2pm or by appointment

Email: zhan2@mail.uh.edu or hanzhu22@gmail.com


Phone: 713-743-4437(o), 301-996-2011(c)
Course website:
http://www2.egr.uh.edu/~zhan2/ECE6397/

Research interests:
http://www2.egr.uh.edu/~zhan2
Wireless Networking, Signal Processing, and Security
http://wireless.egr.uh.edu/
Motivations

Optimization is the mathematical discipline which is
concerned with finding the maxima and minima of functions,
possibly subject to constraints.

Interdisciplinary
• Architecture
• Nutrition
• Electrical circuits
• Economics
• Transportation
•Examples:
• Determining which ingredients and in what quantities to add to a mixture
being made so that it will meet specifications on its composition
• Allocating available funds among various competing agencies
• Deciding which route to take to go to a new location in the city
Course Descriptions

What is the optimization framework?

What are the major types?

Convex vs. non-convex

Continuous vs. discrete

Centralized vs. distributed

Deterministic vs. stochatic

What are the theorems?

What are the applications?

What are the state-of-art research?

Can I find research topics?

How to conduct research and write technique paper
Textbook and Software


Require textbook:
1. Zhu Han, Dusit Niyato, Walid Saad, Tamer Basar, and Are
Hjorungnes,
Game
Theory
in
Wireless
and
Communication Networks: Theory, Models and
Applications, Cambridge University Press, UK, 2011.
2. Steven Boyd’s videos for convex optimization
3. Handout for parts of book, Zhu Han and K. J. Ray Liu,
Resource Allocation for Wireless Networks: Basics,
Techniques, and Applications, Cambridge University
Press, 2008.
4. Other handouts
Require Software: MATLAB
http://www.mathworks.com/ or type helpwin in Matlab environment
Schedule
• Introduction to optimization
• Convex optimization
• Steven Boyd’s class
http://www.stanford.edu/~boyd/cvxbook/
• 30% of the class
• Need to watch videos as homework (17 videos for 1 hour 15 min each)
• Watch the video before the class!!!
• Class is just review
• Integer/Combinatorial optimization
• Might based on Georgia tech class
• 15%
• Stochastic optimization
• Might based on UIUC class
• 15%
• Game Theory
• based on my book
• 40%
Homework, Project, and Exam


Homework

Watch videos for convex optimization

Some other homework
Projects: simple MATLAB programs



Based on the simulation at the end of each chapter
Exams

Two independent exams

Grading policy
Participations

Attendance and Feedback

Quiz if the attendance is low
Teaching Styles



Slides plus black board

Slides can convey more information in an organized way

Blackboard is better for equations and prevents you from
not coming.
Course Website

Print handouts with 3 slides per page before you come

Homework assignment and solutions

Project descriptions and preliminary codes
Feedback

Too fast, too slow

Presentation, Writing, English, …
Other Policies
Any violation of academic integrity will receive academic and
possibly disciplinary sanctions, including the possible awarding
of an XF grade which is recorded on the transcript and states that
failure of the course was due to an act of academic dishonesty.
All acts of academic dishonesty are recorded so repeat offenders
can be sanctioned accordingly.
• CHEATING
• COPYING ON A TEST
• PLAGIARISM
• ACTS OF AIDING OR ABETTING
• UNAUTHORIZED POSSESSION
• SUBMITTING PREVIOUS WORK
• TAMPERING WITH WORK
• GHOSTING or MISREPRESENTATION
• ALTERING EXAMS
• COMPUTER THEFT
Reasons to be my students

Wireless Communication and Networking have great market

Usually highly paid and have potential to retire overnight

Highly interdisciplinary

Do not need to find research topics which are the most difficult
part.

Research Assistant

Free trips to conferences in Alaska, Hawaii, Europe, Asia…

A kind of nice (at least looks like)

Work with hope and happiness

Graduate fast
Different Kinds of Optimization
Optimization Formulation and Analysis

We discuss how to formulate the problem as an optimization issue.

Specifically, we study what the objectives are, what the parameters are, what
the practical constraints are, and what the optimized performances across the
different layers are.

The tradeoffs between the different optimization goals and different users'
interests are also investigated.

The goal is to provide the students a new perspective from the optimization
point of view for variety of problems in engineering fields.
Mathematical Programming

If the optimization problem is to find the best objective function
within a constrained feasible region, such a formulation is
sometimes called a mathematical program.

Many real-world and theoretical problems can be modeled in
this general framework.

We discuss the four major subfields of the mathematical
programming:
– linear programming,
– convex programming,
http://www.stanford.edu/~boyd/cvxbook/
– nonlinear programming,
– dynamic programming.
What do we optimize?

A real function of n variables
f ( x1 , x2 ,, xn )

with or without constrains
– Without constraint
min f ( x, y)  x  2 y
2
– With constraint
2
min f ( x, y )  x 2  2 y 2
x0
or
min f ( x, y )  x 2  2 y 2
 2  x  5, y  1
or
min f ( x, y )  x 2  2 y 2
x y 2
Lets Optimize

Suppose we want to find the minimum of the function

What is special about a local max or a local min of a function f
(x)?
at local max or local min f’(x)=0
f”(x) > 0 if local min
f”(x) < 0 if local max
Review max-min for R3
Integer/Combinatorial Optimization

The discrete optimization is the problem in which the decision variables
assume discrete values from a specified set.

The combinatorial optimization problems, on the other hand, are problems of
choosing the best combination out of all possible combinations.

Most combinatorial problems can be formulated as integer programs.

Integer optimization is the process of finding one or more best (optimal)
solutions in a well defined discrete problem space.

The major difficulty with these problems is that we do not have any
optimality conditions to check if a given (feasible) solution is optimal or not.

We listed several possible solutions such as
– relaxation and decomposition,
– enumeration,
– knapsack problem
– cutting planes.
Example of Integer Program
(Production Planning-Furniture Manufacturer)

Technological data:
Production of 1 table requires 5 ft pine, 2 ft oak, 3 hrs labor
1 chair requires 1 ft pine, 3 ft oak, 2 hrs labor
1 desk requires 9 ft pine, 4 ft oak, 5 hrs labor


Capacities for 1 week: 1500 ft pine, 1000 ft oak,
Market data:
20 employees (each works 40 hrs).
profit demand
table
$12/unit
40
chair
$5/unit
130
desk

$15/unit
30
Goal: Find a production schedule for 1 week tomaximize the profit.
Production Planning-Furniture Manufacturer:
modeling the problem as integer program
The goal can be achieved
by making appropriate decisions.
First define decision variables:
Let xt be the number of tables to be produced;
xc be the number of chairs to be produced;
xd be the number of desks to be produced.
(Always define decision variables properly!)
Production Planning-Furniture Manufacturer:
modeling the problem as integer program

Objective is to maximize profit:
max 12xt + 5xc + 15xd

Functional Constraints
capacity constraints:
pine: 5xt + 1xc + 9xd  1500
oak: 2xt + 3xc + 4xd  1000
labor: 3xt + 2xc + 5xd  800
market demand constraints:
tables: xt ≥ 40

chairs:
xc ≥ 130
desks:
xd ≥ 30
Set Constraints
xt , xc , xd  Z+
Solutions to integer programs

A solution is an assignment of values to variables.

A feasible solution is an assignment of values to variables
such that all the constraints are satisfied.

The objective function value of a solution is obtained by
evaluating the objective function at the given point.

An optimal solution (assuming maximization) is one whose
objective function value is greater than or equal to that of all
other feasible solutions.

There are efficient algorithms for finding the optimal
solutions of an integer program.
Game Theory

Game theory is a branch of applied mathematics that uses models to study
interactions with formalized incentive structures (“games").

It studies the mathematical models of conflict and cooperation among
intelligent and rational decision makers.

Rational means that each individual's decision-making behavior is consistent
with the maximization of subjective expected utility.

Intelligent means that each individual understands everything about the
structure of the situation, including the fact that others are intelligent rational
decision makers.

We have discussed four different types of games, namely, the noncooperative game, repeated game, cooperative game, and auction theory.

Slides
http://wireless.egr.uh.edu/research.htm

The basic concepts are listed and simple examples are illustrated.
Game Theory Overview

What is game theory?
– The formal study of conflict or cooperation
– Modeling mutual interaction among rational decision makers
– Widely used in economics

Components of a “game”
–
–
–
–

Rational players with conflicting interests or mutual benefit
Strategies or actions
Utility as a payoff of player’s and other players’ actions
Outcome: Nash Equilibrium
Many types
–
–
–
–
–
Non-cooperative game theory
Cooperative game theory
Dynamic game theory
Stochastic game
Auction theory
Rich Game Theoretical Approaches
Non-cooperative static
game: play once

Prisoner Dilemma
Payoff: (user1, user2)
– Mandayam and Goodman (2001)
– Virginia tech

Repeated game: play multiple times
– Threat of punishment by repeated game. MAD: Nobel prize 2005.
– Tit-for-Tat (infocom 2003):

Dynamic game: (Basar’s book)
–
–
–
–

ODE for state
Optimization utility over time
HJB and dynamic programming
Evolutional game (Hossain and Dusit’s work)
Stochastic game (Altman’s work)
Auction Theory
Book of Myerson (Nobel Prize 2007), J. Huang, H. Zheng, X. Li
Term Project

Forming the group, 2-3 people per group
– Similar research background

Formulation of problems
– Is that a problem?
– What is the objective and constraints
– What is best optimization techniques

Simulation
– Matlab
– Victim algorithm

Analysis

Writing
– It will be a headache for everybody
Homework

Convex optimization I
– http://www.youtube.com/watch?v=McLq1hEq3UY
– Watch before Wed. class!!!

Form Term project group
– 2-3 people per group
– Let me know in the next class for grouping and basic interests