Total Pixel Space

Last month the New York Times highlighted some AI generated short movies, including Total Pixel Space, by Jacob Adler that gets philosophical about information, à la infinite monkeys. It imagines the finite number of images and video that contain all human history, both real and fake, along with all possible creatures, and even the rise and fall of alien civilizations.

Let's talk about the math. According to the video there are roughly 7.8 x 10^7,575,667 possible 1024x1024 images, and 9.3 x 10^{1,309,075,411,322} possible two hour films, incomprehensibly large numbers. The narrator acknowledges that most of the images are random noise.

But within this ocean of pixel possibility, natural images are but a drop. Recognizable scenes, faces and objects are extremely rare islands in a vast sea of noise.

So how do we capture which images actually matter? Let's visit our friend Kolmogorov Complexity, which measures information by the amount we can compress. A JPEG image typically compresses (lossily) to about 200 KB, which reduces the number of images to 8.7 x 10^1,061,650. Still enormous.

All the images and videos were generated by short prompts. JPEG compression reduces raw pixel space by discarding details our eyes don’t notice. Prompts do something similar at a higher level: they compress ideas, not pixels.

Consider prompts of length 100 over a 10,000 word alphabet typically used in prompting and we're down to 10⁴⁰⁰ possibilities, a mere quintic of the number of atoms in the universe. And you could cut that down by considering grammatical structure.

Now from a prompt you won't get the same image each time, based on the randomness of machine learning. But that's the whole power of AI, the randomness just gives us examples of the structure embedded in the prompt. It's the structure that matters.

And perhaps those monkeys would be more efficient if they entered prompts into AI, instead of generating random text. What images they could produce!