19-22 March
Learn More and Register to Attend

The Sched app allows you to build your schedule but is not a substitute for your event registration. You must be registered for KubeCon + CloudNativeCon Europe 2024 to participate in the sessions. If you have not registered but would like to join us, please go to the event registration page to purchase a registration.

Please note: This schedule is automatically displayed in Central European Standard Time (UTC +1). To see the schedule in your preferred timezone, please select from the drop-down menu to the right, above "Filter by Date." The schedule is subject to change and session seating is available on a first-come, first-served basis. 
Wednesday, March 20 • 14:30 - 15:05
Self-Hosted LLMs on Kubernetes: A Practical Guide - Hema Veeradhi & Aakanksha Duggal, Red Hat

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Feedback form is now closed.

Have you ever considered deploying your own large language model (LLM), but the seemingly complex process held you back from exploring this possibility? The complexities of deploying and managing LLMs in production environments often pose significant challenges. This talk will serve as a comprehensive introductory guide, empowering beginners to commence their LLM journey by effectively hosting their own models on Kubernetes. We will discuss the process of selecting appropriate open source LLM models, containerization of the models, and creating Kubernetes deployment manifests and resource provisioning to support the LLM's computational needs. Self-hosted LLMs offer enhanced data privacy, flexibility in model training, and reduced operational costs, making them an attractive option for organizations seeking greater control over their AI infrastructure. By the end of this talk, attendees will possess the necessary skills and knowledge to navigate the exciting path of self-hosting LLMs.

avatar for Hema Veeradhi

Hema Veeradhi

Senior Data Scientist, Red Hat
Hema Veeradhi is a Senior Data Scientist working in the Emerging Technologies team part of the office of the CTO at Red Hat. Her work primarily focuses on implementing innovative open AI and machine learning solutions to help solve business and engineering problems. Hema is a staunch... Read More →
avatar for Aakanksha Duggal

Aakanksha Duggal

Senior Data Scientist, Red Hat Inc
Aakanksha Duggal is a Senior Data Scientist in the Emerging Technologies Group at Red Hat. She is a part of the Data Science team and works on developing open source software that uses AI and machine learning applications to solve engineering problems.

Wednesday March 20, 2024 14:30 - 15:05 CET
Pavilion 7 | Level 7.1 | Room C
  ML/AI + Data Processing + Storage
  • Content Experience Level Beginner
  • Presentation Slides Attached Yes