Large language models suffer from an architectural problem that the industry has not yet solved: after learning, knowledge in the model stagnates, and the attempt to update it causes what is called "catastrophic forgetting." A study in January 2026 found that about 15-23% of attention points in neural networks are severely disrupted, especially in the lower layers. And forgetting is more pronounced when new tasks are similar to those already studied. The bottom line is that the model operates on probabilities rather than facts, and without access to fresh data it begins to hallucinate.

The main working way around this problem today is generation with search (RAG): the model does not learn again, but simply takes up relevant information from external sources at the time of response. But companies that already have their own search infrastructure can implement this fully in the first place. For example, Microsoft integrated Bing with Copilot, Google linked its search to Gemini, and Yandex linked its search to YandexGPT. Companies without their own search are forced to either use someone else’s solutions or build infrastructure from scratch - which takes years of work.

Scientists are looking for ways to solve the problem at the model level: in 2026, strategies like O-LoRA, CLAIM, Nested Learning from Google Research appeared. However, the fundamental constraint has not yet been removed. Therefore, having your own search becomes a key competitive advantage for AI services: the better the search, the more accurate and reliable the answers from the network.