Episode 27: Why AI answers appear word by word

1. 1 One piece, then the nextThe model builds an answer step by step. Each new piece is based on the text before it.Try: Watch a long answer. Does it grow from left to right?Simplified. Language models are trained to predict the next word from the text before it.
2. 2 Pieces, not always whole wordsThe small units are called tokens. A token can be a whole word, part of a word, or even a single letter.Try: A long or unusual word may show up in pieces.From an AI maker's glossary.
3. 3 Streaming shows it earlyStreaming means the app shows the start of the answer while the rest is still being written.Try: Text trickling in usually means the app is streaming.From the AI makers' developer docs.
4. 4 Some apps waitShowing words early is a choice the app makes. Some apps wait and show the full answer at once.Try: Notice which of your apps type it out and which show it all at once.From the AI makers' developer docs.
5. 5 Long answer, longer waitLong answers take time to write. Streaming lets you start reading while the rest is still coming.Try: Start reading the first lines instead of waiting.From the AI makers' developer docs.

Tap a step when you have done it. Your ticks stay on this device.