In this video we define the basics of quantization and look at how its benefits and how it affects large language models.
Links mentioned:
Perplexity vs model size [ Ссылка ]
Qualitative comparison [ Ссылка ]
#largelanguagemodels #quantization #artificialintelligence #chatgpt #llama
0:00 Intro
0:33 Basic concept
0:51 Benefits
1:30 Quantization 101
2:48 Impact on model size and perplexity
3:48 Impact on inference speed
4:25 Qualitative analysis
