Don't miss out! Join us at our next Flagship Conference: KubeCon + CloudNativeCon North America in Salt Lake City from November 12 - 15, 2024. Connect with our current graduated, incubating, and sandbox projects as the community gathers to further the education and advancement of cloud native computing. Learn more at [ Ссылка ]

Lightning Talk: Unleash the Power of Generative AI Using Only Open Source Technolgies - Christian Kadner, IBM

Generative AI powered by large language models has become the dominant topic in the world of machine learning. Integrating these large language models (LLMs) into applications is still a challenging task, requiring enormous resources and expertise in several fields computer science. In this talk we will present how to deploy LLMs on small Kubernetes clusters using cutting edge open source technologies like CaiKit, KServe and Text Generation Inference Server (TGIS). Caikit enables developers to consume AI models through APIs. It streamlines the management of AI models and lets developers focus on writing application code without the need for data science skills. KServe provides a scalable serving layer powered by Kubernetes providing performant, high abstraction interfaces for ML frameworks like Tensorflow, XGBoost, ScikitLearn, PyTorch, and ONNX. In combination with TGIS, KServe enables high-performance text generation for open-source LLMs.

свернуть