In this episode of Gradient Dissent, Together AI co-founder and Stanford Associate Professor Percy Liang joins host, Lukas Biewald, to discuss advancements in AI benchmarking and the pivotal role that open-source plays in AI development.
🎙 *Listen on Apple Podcasts* : [ Ссылка ]
🎙 *Listen on Spotify* : [ Ссылка ]
He shares his development of HELM—a robust framework for evaluating language models. The discussion highlights how this framework improves transparency and effectiveness in AI benchmarks. Additionally, Percy shares insights on the pivotal role of open-source models in democratizing AI development and addresses the challenges of English language bias in global AI applications. This episode offers in-depth insights into how benchmarks are shaping the future of AI, highlighting both technological advancements and the push for more equitable and inclusive technologies.
✅ *Subscribe to Weights & Biases* → [ Ссылка ]
⏳Timestamps:
00:00 Introduction
09:16 Discussion on benchmarking incentives with references to Kaggle.
10:31 Nuanced challenges of language model overfitting.
10:42 How to effectively choose AI models using leaderboards.
14:00 Advice on selecting models for company use.
16:45 Importance of having a robust model evaluation framework.
18:38 How individuals can contribute to AI benchmarking.
21:13 Discussion on specialized vs. generalized model performance.
27:36 Insights into the complexities of benchmarking AI agents.
29:18 Real-world applications and limitations of AI agents.
36:18 Percy reflects on his TED talk regarding open vs. closed source models.
42:02 Introduction to TogetherAI and its mission for open AI development.
47:11 Combining AI with music creation.
🎙 Get our podcasts on these platforms:
Apple Podcasts: [ Ссылка ]
Spotify: [ Ссылка ]
Google: [ Ссылка ]
YouTube: [ Ссылка ]
Connect with Percy Liang:
[ Ссылка ]
[ Ссылка ]
Anticipatory Music Composer:
[ Ссылка ]
HELM Blog Post Referenced:
[ Ссылка ]
Follow Weights & Biases:
[ Ссылка ]
[ Ссылка ]
Join the Weights & Biases Discord Server:
[ Ссылка ]
