Measuring information. Entropy, mutual information and their meaning. Probabilistic source models and the source coding theorem. Data compression. Noisy channels and the channel coding theorem. Error correcting codes and their decoding. Applications to inference, learning, data structures and communication complexity.
NOTE: Students not enrolled in the Computer Science Major or Specialist program at A&S, UTM, or UTSC, or the Data Science Specialist at A&S, are limited to a maximum of 1.5 credits in 300-/400-level CSC/ECE courses.
