I've heard a few times recently the phrase "Information only exists in a physical state". It come from the quantum computing world where they claim quantum changes the game when it comes to representing information.
As one who has spent his career studying theoretical information that has never and never will exist in a physical state, how can we reckon with such a statement? For starters let's consider the set of all primes--how does that infinite set exist in our finite world?
Information is physical but not directly, but rather as its description. We can discuss a computational process or more generally a mathematical model that captures the set of all primes and we can and have store that description physically.
Let's consider a single prime, the recently discovered Mersenne prime \(2^{136279841}-1\). Note how we must describe the number in a very compressed format, certainly not as a collection of \(2^{136279841}-1\) ping pong balls or even \(2^{136279841}-1\) atoms, far more than the roughly \(2^{365}\) atoms in the observable universe.
In a similar fashion, a large language model stores information through its weights--not a direct encoding of the sentences it can generate.
Now let's think of quantum computing. The quantum algorithm is always classically defined. All the information in quantum states has a classical description. An entangled quantum state may require an exponentially large explicit description, but the algorithm generating it provides a short classical physical description. So if we allow information to only physically represented by its description then it's hard to argue that quantum is somehow special. There are differences to how quantum works but when we try to simplify the message, it can confuse people into thinking quantum is more powerful than it really is.
Are you saying that information does *not* only exist in a physical state? Or are you saying that, if we think of information in terms of its smallest possible representation, then it does exist only in a physical state, but those finite physical states still allow us to reason about infinitely long strings which cannot be be explicitly represented (and in that case, the quantum world is really no different)?
ReplyDeleteThis is me trying to interpret that statement "All information must exist in a physical state", not try to argue whether or not it is true, a philosophical debate I'd rather not wade into.
DeleteIn college, an astrophysicist friend of mine computed the weight of an e-mail message. He started with (average) e-mail message length, converted to entropy, then to energy, then mass.
ReplyDeleteI'm not sure how serious he was being; but it is thought-provoking that the units (sort of) make sense.