New! Sign up for our free email newsletter.
Science News
from research organizations

Charting new paths in AI learning

Date:
February 28, 2024
Source:
Ecole Polytechnique Fédérale de Lausanne
Summary:
Physicists explore different AI learning methods, which can lead to smarter and more efficient models.
Share:
FULL STORY

In an era where artificial intelligence (AI) is transforming industries from healthcare to finance, understanding how these digital brains learn is more crucial than ever. Now, two researchers from EPFL, Antonia Sclocchi and Matthieu Wyart, have shed light on this process, focusing on a popular method known as Stochastic Gradient Descent (SGD).

At the heart of an AI's learning process are algorithms: sets of rules that guide AIs to improve based on the data they're fed. SGD is one of these algorithms, like a guiding star that helps AIs navigate a complex landscape of information to find the best possible solutions a bit at a time.

However, not all learning paths are equal. The EPFL study reveals how different approaches to SGD can significantly affect the efficiency and quality of AI learning. Specifically, the researchers examined how changing two key variables can lead to vastly different learning outcomes.

The two variables were the size of the data samples the AI learns from at a single time (this is called the "batch size") and the magnitude of its learning steps (this is the "learning rate"). They identified three distinct scenarios ("regimes"), each with unique characteristics that affect the AI's learning process differently.

In the first scenario, like exploring a new city without a map, the AI takes small, random steps, using small batches and high learning rates, which allows it to stumble upon solutions it might not have found otherwise. This approach is beneficial for exploring a wide range of possibilities but can be chaotic and unpredictable.

The second scenario involves the AI taking a significant initial step based on its first impression, using larger batches and learning rates, followed by smaller, exploratory steps. This regime can speed up the learning process but risks missing out on better solutions that a more cautious approach might discover.

The third scenario is like using a detailed map to navigate directly to known destinations. Here, the AI uses large batches and smaller learning rates, making its learning process more predictable and less prone to random exploration. This approach is efficient but may not always lead to the most creative or optimal solutions.

The study offers a deeper understanding of the tradeoffs involved in training AI models, and highlights the importance of tailoring the learning process to the particular needs of each application. For example, medical diagnostics might benefit from a more exploratory approach where accuracy is paramount, while voice recognition might favor more direct learning paths for speed and efficiency.


Story Source:

Materials provided by Ecole Polytechnique Fédérale de Lausanne. Original written by Nik Papageorgiou. Note: Content may be edited for style and length.


Journal Reference:

  1. Antonio Sclocchi, Matthieu Wyart. On the different regimes of stochastic gradient descent. Proceedings of the National Academy of Sciences, 2024; 121 (9) DOI: 10.1073/pnas.2316301121

Cite This Page:

Ecole Polytechnique Fédérale de Lausanne. "Charting new paths in AI learning." ScienceDaily. ScienceDaily, 28 February 2024. <www.sciencedaily.com/releases/2024/02/240221160433.htm>.
Ecole Polytechnique Fédérale de Lausanne. (2024, February 28). Charting new paths in AI learning. ScienceDaily. Retrieved April 19, 2024 from www.sciencedaily.com/releases/2024/02/240221160433.htm
Ecole Polytechnique Fédérale de Lausanne. "Charting new paths in AI learning." ScienceDaily. www.sciencedaily.com/releases/2024/02/240221160433.htm (accessed April 19, 2024).

Explore More

from ScienceDaily

RELATED STORIES