Training and Deploying Large-Scale Models

Course objective

This course introduces the foundations and practices of training modern Large Language Models (LLMs) at scale. You will learn how deep learning models are trained across multiple GPUs, nodes, and clusters—and why distributed training is essential for today’s largest AI systems.

We will cover:

• Core techniques for distributed training
• Modern frameworks and scaling strategies
• Practical implementations with real-world toolchains
• Theoretical underpinnings of large-scale learning
• Inference and applications

As LLMs grow in complexity and impact, understanding how they are built and deployed has become essential for researchers and engineers. This series bridges engineering and theory.

Organization

• Enrollment limited to 60 students (no external auditors)
• 7 sessions: 2h lecture + 2h hands-on lab
• Final session: grading
• Projects and HWs are done in groups of 2

Bring your own laptop for the lab sessions, and make sure you can install the nightly version of PyTorch with GPU support before the first lab.