Entropy and Information

The term entropy is often avoided because it contains a certain complexity that cannot be argued away. But when we talk about information, we also have to talk about entropy. Because entropy is the measure of the amount of information. We cannot understand what information is without understanding what entropy is. 1948: Claude E. Shannon invents the expression "bit" and links the concepts of information and entropy. Information is always relative We believe that we can pack information, just as we store bits in a storage medium. Bits are then the information that is objectively available, like little beads in

By |2026-01-06T09:27:40+00:0011. September 2024|Categories: Information, Entropy|Tags: , , , , , |0 Comments

Information Reduction 5: The Classic Glass of Water

Information reduction in thermodynamics A very specific example of information reduction can be found in the field of thermodynamics. What makes this example so special is its simplicity. It clearly illustrates the basic structure of information reduction without the complexity found in other examples, such as those from biology. And it’s a subject many of us will already be familiar with from our physics lessons at school. What is temperature? A glass of water contains a huge amount of water molecules, all moving at different speeds and in different directions. These continuously collide with other water molecules, and their speed

Go to Top