Forbes December 20, 2024
Ilya Sutskever, a co-founder and former chief scientist of OpenAI claims, “Data is the fossil fuel of AI, and we used it all!“
Is this aphorism true?
This claim is made in the context of explaining that the limitation for AI (LLMs in particular) lies in the quality of data required to mimic intelligence—a limitation often referred to as the ‘entropy gap’.
Entropy, in information theory, is the measure of uncertainty or unpredictability in a probabilistic system. In the context of AI, entropy quantifies the variability and richness of information within a dataset, reflecting how evenly distributed or diverse the data points are across possible outcomes. This diversity contributes to the uncertainty by ensuring the dataset encompasses a broad range...
Today's Sponsors
Today's Sponsor