Automatic Token Generation.
Inspired by the Surrealists’ practice of automatic writing, this experiment lets language unfold without a written prompt. A language model uses recent tokens to predict what could come next, then selects the next token by sampling from those probabilities. A 3D view tracks attention weights from a selected layer and maps tokens through their embeddings.
The model has no unconscious mind to uncover. This demo offers a look under the hood of how large language models generate text through learned patterns, attention, and probabilistic sampling within a limited context window.
Sign in to use your Hugging Face ZeroGPU quota. Sign-in opens the app in a new tab to avoid embedded-browser cookie issues. Generation uses short GPU sessions; Pause releases the GPU.
Small base models work best. Public safetensors models only; Preloaded models up to 8 GB; each visitor has a separate stream. Loading includes preparing a fixed 3D embedding projection.
GPT-2 XL (1.5B) is a larger base model for prose. Models are preloaded in 16-bit precision on ZeroGPU. Gemma uses BF16 to keep generation numerically stable.