An attempt to teach information theory through practice with problems grounded in neuroscience.
The repository contains 4 main pdf files that are regularly updated:
- problems.pdf: The problems to solve
- hints.pdf: Hints that can be used, if you have difficulties solving the problems alone.
- cheatsheet.pdf: A cheatsheet with all the necessary theoretical background to get started with problem solving. As more problems are solved, the insights from these problems will be added into this document.
- solutions.pdf: Solutions for past problems.
- Start trying to solve problems. If you have issues make sure first you understand the content in the cheatsheet, else you might be lacking some important background (either basics you need to catch up on or previous problems you need to revisit).
- If you cannot solve a problem, first try looking at the hints. If that does not help, ask a fellow problem solver for some insights.
- After solving a problem, write your answer on the whiteboard. Solutions will be posted online for everyone, when the problem has been solved.
If you have any questions, suggestions or spot any mistakes please write me an email.
Many online resources on information theory can be found online, but here is a shortlist of interesting resources:
- If you want a standard book that teaches information theory from the classical communication theory angle, I can recommend Elements of Information Theory by Thomas Cover and Joy Thomas (link).
- For a historical perspective, the seminal article A Mathematical Theory of Communication by Claude Shanon is a must read (link)