A million tokens stopped being a demo and became a working mode. The model holds a whole project in mind, not a single file.
About this eventThe workaround the industry lived on while context was short. Now a complement rather than a crutch.
RAG separates knowledge from model parameters: before answering, a system retrieves relevant documents and adds them to the input. This makes facts and citations easier to update, but quality depends on retrieval, document chunking and the model's ability to distinguish evidence from its own guess.
Source: arXiv · RAGAbout this eventA million tokens in production — a whole book or an hour of video fits inside.
About this eventCoding systems read the whole project.
About this eventOne-twentieth the compute per token, 9× prefill and 15× decode on million-token context.
About this eventLong context fits, but attention to its middle sags: the model recalls the start and the end better than what lies between.
About this eventContext resets with every new conversation. Memory that lives for years without growing forever is a separate unsolved problem.
Primary material behind this dossier: papers, lab publications and official reports.