ITEE Home » Engg7302 » Material: ENGG7302

Print version

The University of Queensland
School of Information Technology and Electrical Engineering
Semester 2, 2011

ENGG7302 - Advanced Computational Techniques in Engineering

Course Material

Lecture Notes

Numerical Linear Algebra

1. Matrix-Vector Multiplication.
2. Orthogonal Vectors and Matrices.
3. Norms.
4. The Singular Value Decomposition.
5. More on the SVD.
6. Projection Matrices.
7. QR Decomposition.
8. Householder Triangularisation.
9. Least Squares Problems.
   

Stochastic Processes

Listed below are a choice of references available for the lecture content.
Follow the reference you are comfortable with to study the content.

1. Probability.
   Kay, Chap:2-4
   OR Papoulis Chap 2-3
   STAT2202 notes UQ course notes on Probability & random variables.
2. Random Variables.
   Kay, Chap:5-6, 10-11
   OR Papoulis Chap 4-5
3. Multiple Random Variables (2011).
   Kay, Chap:7, 8 (upto 8.4), 9 , 12 (upto 12.8, 12.10), 13 (upto 13.5), 14, 15.4, 15.5
   OR Papoulis Chap 6-7
4. Stochastic Processes and PSD .
   Kay, Chap 16-17.4, Simon Haykin, Communication Systems, 4th edition pg. 31-41
   OR Papoulis pp. 373–393.
   Kay, 17.6 - 17.8, Simon Haykin Chap 1, pg: 41-54, also A. Oppenheim 2nd ed. (pg: 18,21)
   OR Papoulis pp. 393–420.
   Solution to a lecture Question (last slide) Solution
5. Discrete-time Stochastic Processes.
   Papoulis Pillai pp. 420–426, 506–509, (check solved examples in A. Oppenheim pg: 11-40)
6. Markov Chains.
   Grinstead Snell, Chap 11.
7. Markov chain Monte Carlo.
   Mackay pp. 357-369

Optimisation

1. Classical Mathematical and Numerical Optimization
• Additional slides on Nelder-Mead Simplex algorithm (we used slides 12-15 only). These slides are based on the book by Spall, listed in the Additional References.
2. Global Optimization and Metaheuristics
3. Evolutionary Computation
4. Swarm Intelligence
5. Simulated Annealing
6. Real World Applications of Optimisation

---

Primary Reference Texts

• Lloyd N. Trefethen & David Bau, III, Numerical Linear Algebra, SIAM, 1997.
• Steven Kay, Intuitive Probability and Random Processes using MATLAB, Springer, 2006.
  
• Athanasios Papoulis & S. Unnikirshna Pillai, Probability, Random Variables and Stochastic Processes, McGraw-Hill, 4th ed., 2002.
  
• C. M. Grinstead and J. L. Snell. Introduction to Probability (Ch. 11). Available Online
• M. T. Heath. Scientific Computing: An Introductory Survey (Ch. 6 available online from library). A local copy is also here.
• S. Luke. Essentials of Metaheuristics.
• J. Brownlee. Clever Algorithms.

Additional Reference Texts

• G. H. Golub and C. F. Van Loan. Matrix Computations. Johns Hopkins university press, 3rd edn. 2006.
• D. Mackay. Information Theory, Inference, and Learning Algorithms. Oxford, 2003 (see Chap.29 for MCMC).
• S. Boyd and L. Vandenberghe. Convex Optimization.
• J. C. Spall. Introduction to Stochastic Search and Optimization: Estimation, Simulation and Control. Wiley, 2003.
• A. Oppenheim, R. Schafer. Discrete time signal processing, Prentice Hall, 1999.

Other Reference Material

• Paper: C. Andrieu, N. De Freitas, A. Doucet and M. I. Jordan. An Introduction to MCMC for Machine Learning. Machine Learning, 50, 5-43, 2003.
• Laird Breyer's MCMC web pages (including demo applet).
• Paper: J. R. Shewchuk. An Introduction to the Conjugate Gradient Method Without the Agonizing Pain. Tech Rept. School of Computer Science, Carnegie-Mellon University, 1994.
• Paper: C. Blum and A. Roli. Metaheuristics in Combinatorial Optimization: Overview and Conceptual Comparison. ACM Computing Surveys, Vol. 35, No. 3, September 2003, pp. 268-308.
• Simulated Annealing TSP Applet.
• 1-D continuous problem simulated annealing applet.
• Ant colony optimization TSP applet.
• Particle Swarm Optimization applet.
• Another Particle Swarm Optimization applet.
• Optimisation Algorithm Toolkit (OAT)

Background Material

• Introduction to Probability Models, by Em Prof Tom Downs.
• Matlab primer (An Introduction to Matlab for Cognitive Programming) by Scott Bolland.

---

Assignments and Tutorials

Assignments

• DUE 5pm Friday, 9/9/11: Assignment 1
• DUE 5pm Monday, 10/10/11: Assignment 2
• DUE 5pm, Tuesday, 10/11/11: Assignment 3

Tutorials

• Weeks 2 - 3: Tutorial LA1
• Weeks 3 - 4: Tutorial LA2
• Weeks 4 - 5: Tutorial LA3
• Weeks 5 - 6: Tutorial OP1 (Genetic Algorithms)
• Weeks 7 - 8: Tutorial OP2 (Swarm Intelligence Algorithms)
• Weeks 8 - 9: Tutorial OP3 (Steepest Descent)
• Weeks 10: Tutorial SP1
• Weeks 11: Tutorial SP2
• Week 12: Tutorial SP3
• Week 13: Tutorial SP4

---

Exams

Past Exams

• Semester 1, 2007: Class Test 2 (Stochastic Processes)
• Semester 1, 2007: Class Test 1 (Optimization)
• Semester 1, 2007: Final Exam
• Semester 2, 2007: Class Test 1 (Linear Algebra)
• Semester 2, 2007: Class Test 2 (Stochastic Processes)
• Semester 2, 2007: Final Exam
• Semester 1, 2008: Class Test 1 (Linear Algebra)
  
• Semester 1, 2008: Class Test 2 (Stochastic Processes)
• Semester 1, 2008: Final Exam
• Semester 2, 2008: Class Test 1 (Linear Algebra)
• Semester 2, 2008: Class Test 2 (Stochastic Processes)
• Semester 2, 2008: Final Exam
• Semester 1, 2009: Class Test 1 (Linear Algebra)
• Semester 1, 2009: Class Test 2 (Stochastic Processes)
• Semester 1, 2009: Final Exam
  
• Semester 2, 2009: Class Test 1 (Linear Algebra)
• Semester 2, 2009: Class Test 2 (Stochastic Processes)
• Semester 2, 2009: Final Exam
• Semester 1, 2010: Class Test 1 (Linear Algebra)
  
• Semester 1, 2010: Class Test 2 (Stochastic Processes)
• Semester 1, 2010: Final Exam
• Semester 2, 2010: Class Test 1 (Linear Algebra)
• Semester 2, 2010: Class Test 2 (Optimization)
• Semester 2, 2010: Final Exam
• Semester 1, 2011: Class Test 1 (Linear Algebra)
• Semester 1, 2011: Class Test 2 (Stochastic Processes)
• Semester 1, 2011: Final Exam
• Semester 2, 2011: Class Test 1 (Linear Algebra)
• Semester 2, 2011: Class Test 2 (Optimisation)

---