The Chinese company ByteDance, TikTok's parent company, has begun pre-training one of the world's largest artificial intelligence models. According to sources, the potential size of the new system can reach 10 trillion parameters, which will exceed the performance of the Chinese Kimi K3 model by about three times. The final volume will be determined at a later stage of development. The pre-training takes from three to six months, after which additional customization and product release are planned. This step is aimed at reducing the technological gap from the American developer Anthropic, whose most powerful models, according to industry experts, contain about 8 trillion parameters.

The number of parameters determines the ability of the model to process information, but does not guarantee automatic superiority in quality. The system's capabilities are also influenced by the amount of training data, architecture, and optimization methods. Despite the scale of the project, ByteDance adheres to a strategy of developing closed models, unlike a number of Chinese laboratories that publish their developments. At the same time, the company is already showing success: its Seedance video generation model is among the market leaders, and the Doubao consumer product has about 324 million monthly active users in China.

The Seed team, led by a former Google DeepMind researcher, includes about 2,000 specialists, is engaged in the development. The management of ByteDance relies on completely independent development, abandoning the distillation method, which involves training smaller models on the responses of large systems. The company's founder, Zhang Yimin, emphasized the need to focus on the global level of the model's capabilities in the long term. To achieve these goals, ByteDance is actively increasing investments in AI infrastructure, expanding its network of data centers, developing the Volcano Engine cloud division, and working on creating its own microprocessors. The company plans to invest up to 170 billion dollars in this area.