leon.bottou.org

This short position paper argues that LLMs are best viewed as Fiction Machines, that machines able to write stories that might not be related to what is factual but are internally coherent. Considerable efforts are spent aligning these machines with our expectations, chasing hallucinations, and ignoring the fact that the ability to make up stories is key to intelligence.

• “Although the machine must know what makes sense in the context of the developing story, what is true in the world of the story need not be true in our world. As new words are printed on the tape, the story follows fresh twists and turns, borrowing facts from the training data and filling in the gaps with plausible confabulations.”

• “While we may know the facts of a historical battle, we can only truly understand them by imagining alternate timelines. What if the general had made a different choice? What if the weather had been more clement? We rarely draw on our own personal experiences to answer such questions; instead, we might consider what we have previously read about battles — including fictional ones. We understand the causal structure of true facts by conjuring counterfactual scenarios that we complete with more fiction. Indeed, we must accept that we ourselves are fiction machines.”

@article{siamnews-2025,
  title = {The Fiction Machine},
  author = {Bottou, L{\'e}on and Sch{\"o}lkopf, Bernhard},
  journal = {Siam News},
  volume = {58},
  number = {03},
  year = {2025},
  url = {http://leon.bottou.org/papers/siamnews-2025},
}

Related