Andrew Ng, a pioneer in applying GPUs for deep learning, co-founder of Google Brain, and former Chief Scientist at Baidu, discussed the importance of the data-centric AI movement, which he considers crucial for developing effective AI systems. He pointed out that over the past decade, significant advancements in deep learning have been driven by the use of the most powerful models processing vast amounts of data. However, Ng expresses the opinion that this approach is not universal and will not work in industries where large datasets simply do not exist.
In particular, he noted: “In many industries where giant datasets simply do not exist, I believe the focus should shift from big data to quality data. Having 50 well-thought-out examples can be sufficient to teach a neural network what you want it to learn.” This statement underscores the importance of a quality approach to data rather than its quantity. Ng also mentions that collecting large volumes of data can be an expensive endeavor. “Collecting more data often helps, but if you try to gather more data for everything, it can turn into a very costly activity,” he added. In this context, he emphasizes that companies should focus on the quality selection of data rather than just increasing its volume.
He also pointed to the enormous potential offered by the data-centric AI movement. “I think it is quite likely that the biggest shift this decade will be related to data-centric AI,” Ng asserts. Given that current neural network architectures have significantly evolved, the movement's paradigm now requires a systematic approach to data engineering. In the context of the escalating issue of responsibility and bias in AI systems, Ng noted that the quality of data should be approached more responsibly. In this regard, he praised discussions at the recent NeurIPS workshop, where researchers exchanged ideas on how data-centric AI can be studied and applied to reduce bias. Looking to the future, Ng emphasized that only through a creative approach to data engineering can companies develop reliable and sustainable AI solutions that offer significant advantages in manufacturing and other sectors. He concluded that the data-centric AI movement poses a challenge: how to scale such solutions without hiring 10,000 ML specialists for 10,000 manufacturers.
Related articles
