The Most Important (and Surprising) Result from Information Theory

The machine learning consultancy: truetheta.io/
Join my email list to get educational and useful articles (and nothing else!): mailchi.mp/truetheta/true-theta-email-list
Want to work together? See here: truetheta.io/about/#want-to-work-together

Information Theory contains one idea in particular that has had an incredibly impact on our society.

David MacKay's lecture:    • Information Theory, Pattern Recognition, a...…

SOCIAL MEDIA

LinkedIn : www.linkedin.com/in/dj-rich-90b91753/
Twitter : twitter.com/DuaneJRich
Github: github.com/Duane321

Enjoy learning this way? Want me to make more videos? Consider supporting me on Patreon: www.patreon.com/MutualInformation

SOURCES

Chapters 1, 9 and 10 from [1] were the primary source. Chapter 7 of [3] was a secondary source. [2] is inclusive of Shannon's 1948 original paper and provided useful historical context.

[1] D. MacKay. Information Theory, Inference and Learning Algorithms. Cambridge University Press. 2003.

[2] C. E. Shannon and W. Weaver. The Mathematical Theory of Communication. University of Illinois Press. 1964

[3] T. M. Cover and J. A. Thomas. The Elements of Information Theory, 2nd Edition. Wiley-Interscience. 2006.

TIMESTAMPS
0:00 Problem Statement and the R3 Coding Strategy
2:06 Bit Error Probability and Rate
3:30 The Trillion Dollar Question
4:12 Claude Shannon Proves Something Remarkable
6:11 Sidebar on other Educational Content
6:49 The Trick
7:38 Check out David Mackay's Textbook and Lectures, plus Thank You