Information theory

Topic intro

Information theory quantifies information, uncertainty, and the limits of communication and compression using ideas like entropy. It began in communications but connects naturally to thermodynamic entropy, machine learning, cryptography, and how biological systems encode and copy genetic instructions.

New here? Ignore the full sidebar for a minute. On a first pass, follow the core path in order. The sidebar is for jumping later, not for deciding where to begin.

First pass

Do these in order. Each step is short on purpose.

1. Start hereLearn · Overview
Get the big picture, the objectives, and the nearby connections before you start solving.
2. Work through the short lessonsPart 1 - Bits, binary questions, logarithm…
Use Next / Previous through the remaining 6 short lessons in order.
3. Check whether it sticksReview
Use this after the lessons to test recall without leaning on the exact sentence pattern from the page.
4. Try the labEntropy lab
Use the topic in a small challenge once the main ideas feel familiar.

Use after the first pass

These tools help with retrieval, expansion, and review once the main idea is already in your head.

SprintCompete under pressure
Use sprint after the first pass for faster, higher-yield mixed questions that stay anchored to the lesson ideas.
CardsRun short recall
Best after the first pass, when you want retrieval rather than first exposure.
QuizzesCheck under pressure
Use these when you want a sharper check, not when you are still figuring out the core idea.
MapSee the wider graph
Use the map when you want the dependency picture, not when you are deciding where to begin.