Entropy, Cross-Entropy & KL-Divergence

Here is a 10-minute video by Aurélien Géron explaining entropy, cross-entropy and KL-divergence using Information Theory.

He has some more interesting videos on his channel. Do check it out!


A new kind of interpretation


In all of computer programming, we have been solving problems by modeling the real world into code. Consider the reverse process, where we derive a relation between data structures and the real world. Can we model code in real world objects? Or in other words, can we represent information in real world entities? My effort in the following article will be to guide you through various scenarios which will (hopefully) change your perception towards our environment.


Continue reading