Lecturer: Prof. Johannes Berg
Exercises: Prasanna Bhogale (email) and Chau Nguyen (email)
–Sheet 1: Review of random variables-
Due: Monday 15 October, 2012
Discussion: Wednesday 17 October, 2012
–Sheet 2: Entropy-
Due: Monday 29 October, 2012
Discussion: Wednesday 31 October, 2012
–Sheet 3: Source coding problem-
Due: Monday 12 November, 2012
Discussion: Wednesday 14 November, 2012
–Sheet 4: Communication over noisy channels-
Due: Monday 26 November, 2012
Discussion: Wednesday 28 November, 2012
–Sheet 5: Landauer’s principle-
Due: Monday 10 December, 2012
Discussion: Wednesday 12 December, 2012
–Sheet 6: Basic Bayesian inference-
Due: Monday 07 January, 2013
Discussion: Wednesday 09 January, 2013
–Sheet 7: Belief propagation on system of Ising spins-
Supplementary: sample source code. Please send us (Prasanna and Chau) your code and figures, or show your figures in the your solution for checking.
Due: Monday 21 January, 2013
Discussion: Wednesday 23 January, 2013
References
*D. J. C. MacKay, “Information Theory, Inference and Learing Algorithms” (Cambridge University Press, 2003).
*M. Opper and D. Saad, “Advanced Mean Field Methods”, (MIT 2001).
*W. J. Ewens and G. R. Gran, “Statistical Methods in Bioinformatics”, (Springer, 2001).
*T. M. Cover and J. A. Thomas, “Elements of Information Theory”, (John Wiley and Sons, 2006).
*A. I. Khinchin, “Mathematical Foundations of Information Theory”, translated by R. A. Silverman and M. D. Friedman (Dover, 1957).
*T. Sagawa, “Thermodynamics of Information Processing in Small Systems”, Progr. Theor. Exp. Phys. 127(1),1–56 (2012).
*F. Bavaud, “Information Theory, Relative Entropy and Statistics” in Formal Theory of Information, 54–78 (Springer, 2009).
*N. Merhav, “Physics at the Shannon Limits”, IEEE Transaction on Information Theory 56(9), 4274–4285 (2010).
*R. Landauer, “Irreversibility and Heat Generaton in the Computing Process”, IBM Journal of Reseacher and Development 5:3, 183 (1961).