Memorandum
- From
- Derek Weston via VentureBeat
- Date
- Filed
- News·4 min to read
- Re
Frontier AI models can recover up to 65% of facts they fail to recall by thinking longer
ReFrontier AI models can recover up to 65% of facts they fail to recall by thinking longer
A new study by Google Research and Technion shows that frontier models like GPT-5 and Gemini-3 encode 95-98% of tested facts, but fail to directly recall 26-34% of them. Inference-time thinking recovers 40-65% of those facts, suggesting recall, not knowledge, is the main bottleneck.
When large language models produce incorrect answers, developers usually assume the models lack the necessary information. The standard fix is to make the models bigger, feed them more data, or build complex retrieval systems. But a new study by researchers at Google Research and Technion suggests that in many cases the knowledge is not missing at all — the model simply fails to retrieve it during generation.
The study, which evaluated 13 large language models on more than 4 million responses, found that frontier models such as GPT-5 and Gemini-3 encode 95–98% of the facts tested. Yet without additional computational effort, they fail to directly recall 26–34% of those encoded facts. When given extra time to think — through inference-time computation like Chain-of-Thought — the models successfully recover 40–65% of the facts they initially could not recall.
The researchers propose a new framework called fact-level profiling, which shifts evaluation from simple question accuracy to a deeper analysis of whether a fact is stored in the model's parameters and whether it can be accessed under different conditions. They distinguish between a fact being "encoded" and being "known." A model encodes a fact if it can reproduce it when primed with its original training context. It knows a fact if it can reliably answer questions about it across varied phrasings and directions.
"Encoding and recall failures are indistinguishable under accuracy metrics, yet they imply different limitations and solutions," the researchers write. "Encoding failures call for pre-training interventions, such as scaling model size or data coverage. Recall failures suggest post-training interventions that often improve how models utilize what they already encode."
The paper categorizes knowledge into five profiles, using the example of Oasis's first gig at the Boardwalk club. Direct recall means the model can answer immediately. Encoding failure means the fact is absent entirely. Recall failure means the fact is stored but inaccessible. Recall with thinking means the model can retrieve the fact only through intermediate reasoning steps. Inference without encoding means the model deduces the answer by chaining together other facts it knows.
The researchers also found that scaling up model size does not automatically solve the recall problem. For example, scaling the Gemma3 model from 1 billion to 27 billion parameters reduced encoding failures from 85% to 23%, but the share of recall failures increased, peaking at 40% without thinking. This suggests that scaling mainly improves storage, not access, leaving a larger pool of knowledge trapped in an "encoded but inaccessible" state.
The findings have practical implications for engineering teams building AI applications. Instead of always reaching for larger models or external databases, developers may be able to improve reliability by designing prompts that encourage the model to reason through intermediate steps. The researchers compare the phenomenon to the human tip-of-the-tongue state, where deliberate effort helps retrieve a memory that is present but not immediately accessible.
"Our findings suggest that recall is tightly coupled to the conditions under which facts were learned, degrading when queries diverge from training-time patterns," the researchers write. The way a question is asked directly influences whether the model can unlock the stored answer.
