Angelia Nedich
angelia@uiuc.edu
University of Illinois at Urbana-Champaign
Department of Industrial and Enterprise Systems Engineering
Distributed Optimization (IE/GE 598 AN)
This course is an advanced research seminar for graduate students. The objective is to
study and learn the most recent advances in models and algorithmic developments for
analysis and optimization of large scale distributed systems. The course is mainly
motivated by the emergence of large scale networks, characterized by the lack of a
centralized coordinator or centralized access to information, and time-varying system
characteristics such as connectivity. The purpose of this course is to study such systems
by exploring the interplay of optimization theory, game theory, dynamical systems, and
to some extent graph theory. The topics to be covered include models and algorithms for
distributed rate allocation and congestion control over communication networks (such as
the internet), distributed coordination algorithms (such as consensus and gossip) over
networks with applications to multi-vehicle systems and sensor networks, coverage
problems, and distributed control, as well as quantization and synchronization
phenomena in engineered systems.
There is no text-book for this course. The list of selected papers for readings is posted on
the course web at: https://netfiles.uiuc.edu/angelia/www/nedichcourses.html
Credit: 4 hours
Offering Level: Graduate
Meeting Time: MW 3-4:40 MEB 256
Office Hours: M 2-3 TB 211
Prerequisites: Basic background in Multivariate Calculus, Linear Algebra, and
Optimization or instructor’s permission.
Requirements:
This course requires a lot of independent work. There are a few traditional lectures.
Students are expected to critically read and analyze research papers across various
disciplines.
Assignments: Everyone participating in the course shall read all the selected papers.
Each student will present several papers in a form of group lectures. Grades will be based
on the quality of the presentations and overall participation in the class discussions. Some
special projects may be assigned if necessary.
READING LIST
Decentralized “cooperative” agent systems
1. J. N. Tsitsiklis, Problems in Decentralized Decision Making and Computation,
Ph.D. Thesis, Department of EECS, MIT, November 1984; technical report
LIDS-TH-1424, Laboratory for Information and Decision Systems, MIT. (13 Mb)
(2 lectures)
2. J. N. Tsitsiklis, D. P. Bertsekas and M. Athans, Distributed Asynchronous
Deterministic and Stochastic Gradient Optimization Algorithms, IEEE
Transactions on Automatic Control, Vol. 31, No. 9, 1986, pp. 803-812. (1/2 lect.)
3. T. Vicsek, A. Czirok, E. Ben-Jacob, I. Cohen, and O. Schochet, “Novel type of
phase transitions in a system of self-driven particles,” Physical Review Letters,
Vol. 75, No. 6, 1995, pp. 1226-1229,
http://arxiv.org/pdf/cond-mat/0611743
(1/2 lecture)
4. A. Jadbabaie, J. Lin, and A. S. Morse,
Coordination of groups of mobile
autonomous agents using nearest neighbor rules,
IEEE Transactions on
Automatic Control, Vol. 48, No. 6, June 2003, pp. 988-1001.
(1/2 lecture)
5. V. D. Blondel, J. M. Hendrickx, A. Olshevsky, and J. N. Tsitsiklis, Convergence
in Multiagent Coordination, Consensus, and Flocking, in Proceedings of the Joint
44th IEEE Conference on Decision and Control and European Control
Conference (CDC-ECC'05), Seville, Spain, December 2005. (1/2 lecture)
6. A. Olshevsky and J. N. Tsitsiklis, Convergence Speed in Distributed Consensus
and Averaging, submitted, December 2006. (1/2 lecture)
7. R. Olfati-Saber and R. M. Murray Consensus Problems in Networks of Agents
with Switching Topology and Time-Delays IEEE Trans. on Automatic Control,
vol. 49(9), pp. 1520-1533, Sep., 2004. (1/2 lecture)
8. R. Olfati-Saber, Flocking for Multi-Agent Dynamic Systems: Algorithms and
Theory IEEE Trans. on Automatic Control, vol. 51(3), pp. 401-420, Mar. 2006.
(1/2 lecture)
9. L. Xiao and S. Boyd, Fast linear iterations for distributed averaging, Systems and
Control Letters, 53:65-78, 2004.
10. S. Boyd, A. Ghosh, B. Prabhakar, and D. Shah, Gossip algorithms: Design,
analysis, and applications, Proceedings IEEE Infocomm 2005, 3:1653-1664,
Miami, March 2005 (2/3 lecture).
11. S. Boyd, A. Ghosh, B. Prabhakar, and D. Shah, Randomized Gossip Algorithm,
IEEE Transactions on Information Theory, vol. 52, no. 6, June 2006 (1/3 lecture)
12. D. Kempe, A. Dobra, and J. Gehrke, Gossip-based computation of aggregate
information, ACM Symposium on Theory of Computing, 2004.
http://www.cs.cornell.edu/johannes/papers/2003/focs2003-gossip.pdf (1/4 lecture)
13. C.C. Moallemi and B. Van Roy, Consensus Propagation, IEEE Transactions on
Information Theory, vol. 52, no. 11, 2006, http://arxiv.org/pdf/cs/0603078v2 (3/4)
14. A.G. Dimakis, A.D. Sarwate, and M.J. Wainwright, Geographic Gossip: Efficient
Averaging for Sensor Networks, forthcoming in IEEE Transactions on Signal
Processing, http://arxiv.org/pdf/0709.3921v1 (2/3 lecture)
15. F. Benezit, A. G. Dimakis, P. Thiran and M. Vetterli, Gossip Along the Way:
Order-Optimal Consensus through Randomized Path Averaging, Proceedings of
Allerton Conference 2007.
http://www.eecs.berkeley.edu/~adim/Allerton_final_gossip.pdf (2/3 lecture)
16. A. Kashyap, T. Basar and R. Srikant, Quantized Consensus, Automatica, vol. 43,
no.7, p.1192-1203, July 2007.
http://www.ifp.uiuc.edu/~srikant/Papers/akshay_consensus.pdf
17. Ruggero Carli, Fabio Fagnani, Paolo Frasca, and Sandro Zampieri, Efficient
Quantized Techniques for Consensus Algorithms, NeCST workshop, Nancy,
France, 2007. http://calvino.polito.it/~frasca/due_metodi.pdf
18. Paolo Frasca, Ruggero Carli, Fabio Fagnani, and Sandro Zampieri, Average
Consensus on Networks with Quantized Communication, Preprint, 2008.
http://calvino.polito.it/~frasca/FCFZ_quantization.pdf
19. Q. Li and D. Rus, Global Clock Synchronization in Sensor Networks, IEEE
Transactions on Computers, vol. 55, no. 2, 2006. Download it through UIUC
library on line system.
20. R. Carli, A. Chiuso, L. Schenato, S. Zampieri, A PI Consensus Controller for
Networked Clocks Synchronization, submitted to IFAC World Congress, Seul’08.
HERE
21. A. Olshevsky and J. N. Tsitsiklis, "On the Nonexistence of Quadratic Lyapunov
Functions for Consensus Algorithms'', June 2007; IEEE Transactions on
Automatic Control, forthcoming (1/2 lecture)
22. R. Carli, A. Chiuso, L. Schenato, S. Zampieri, “A PI Consensus Controller for
Networked Clocks Synchronization,” Preprint 2007.
23. P.-A. Bliman and G. Ferrari-Trecate. Average consensus problems in networks of
agents with delayed.communications. Proc. 44th IEEE Conference on Decision
and Control and European Control Conference 2005, pages 7066--7071, 2005.
24. D. Angeli and P.-A. Bliman, “Stability of leaderless multi-agent systems.
Extension of a result by Moreau,” 2004.
oai:arXiv.org:math/0411338.
http://arxiv.org/pdf/math/0411338
25. S. Boyd, P. Diaconis, and L. Xiao, Fastest mixing Markov chain on a graph
(2003)
26. R. Raffard, C. Tomlin, and S. Boyd, Distributed optimization for cooperative
agents: Application to formation flight (2004)
27. S. Boyd, A. Ghosh, B. Prabhakar, and D. Shah, Mixing times for random walks
on geometric random graphs (2006)
28. S. Samar, S. Boyd, and D. Gorinevsky, Distributed estimation via dual
decomposition (2007)
29. L. Xiao, S. Boyd, and S.-J. Kim, Distributed average consensus with least-meansquare deviation (2007) 1 lecture
“Non-cooperative” agent systems
30. Basic intro on games, Equilibria, Stackelberg games, mechanism design (2-3
lectures)
31. Frank Kelly, Aman Maulloo and David Tan. Rate control in communication
networks: shadow prices, proportional fairness and stability, Journal of the
Operational Research Society 49 (1998) 237-252.
32. R.J. Gibbens and F.P. Kelly, Resource pricing and the evolution of congestion
control , Automatica 35 (1999) 1969-1985.
33. Steven H. Low and David E. Lapsley, "Optimization flow control, I: basic
algorithm and convergence," IEEE/ACM Transactions on Networking, December
1999. http://citeseer.ist.psu.edu/article/low99optimization.html More
34. T. Basar and R. Srikant. "A Stackelberg network game with a large number of
followers." Journal of Optimization Theory and Applications, 115(3):479-490,
December 2002.
35. R. Johari and D.K.H. Tan. End-to-end congestion control for the Internet: delays
and stability. IEEE/ACM Transactions on Networking, 9(6):818–832, 2001.
36. R. Johari and J.N. Tsitsiklis. Efficiency loss in a network resource allocation
game. Mathematics of Operations Research, 29(3):407–435, 2004. Awarded First
Place in the 2003 INFORMS George E. Nicholson Student Paper Competition.
37. R. Johari, S. Mannor, and J.N. Tsitsiklis. Efficiency loss in a network resource
allocation game: the case of elastic supply. IEEE Transactions on Automatic
Control, 50(11):1712–1724, 2005. A longer version containing all proofs is
available here.
38. R. Johari and J.N. Tsitsiklis. A scalable network resource allocation mechanism
with bounded efficiency loss. IEEE Journal on Selected Areas in
Communications, 24(5):992–999, 2006.
39. D. Acemoglu, R. Johari, and A. Ozdaglar. Partially optimal routing. IEEE Journal
on Selected Areas in Communications, 25(6):1148–1160, 2007.
40. S. Adlakha, R. Johari, and A. Goldsmith. Competition in wireless systems via
Bayesian interference games. 2007. Submitted.
41. R. Srikant, “On the positive recurrence of a Markov chain describing file arrivals
and departures in a congestion-controlled network,” Technical Report. Presented
at the IEEE Computer Communications Workshop, Oct. 2004.
http://www.ifp.uiuc.edu/~srikant/Papers/ccw04.pdf This is the later version of the
paper which will appear in the IEEE Transactions on Information Theory: X. Lin,
N.Shroff and R.Srikant On the Connection-Level Stability of CongestionControlled Communication Networks
http://www.ifp.uiuc.edu/~srikant/Papers/it_connection_level.pdf