Schedule

Abstract
Large multimodal models (LMMs) are rapidly advancing the frontier of AI by enabling joint understanding and generation across text, images, video, and audio. Yet deploying these models efficiently in production remains a major challenge due to their heterogeneous architectures, multi-stage pipelines, and highly variable request patterns. In this guest lecture, I will first provide a brief overview of the systems challenges in multimodal inference and generation from both production traces analysis at Azure and open-source LMM characterization. Building on these insights, I will talk about ModServe, our modular serving system for efficient LMM inference. ModServe decouples model stages for independent optimization and adaptive scaling, and introduces modality-aware scheduling to meet tail latency SLOs under dynamic workloads. On production-scale traces, ModServe improves throughput by 3.3–5.5× and reduces costs by up to 41%, while maintaining latency guarantees. I will conclude by discussing broader implications of modular and adaptive design for efficient multimodal generation in datacenters.

Abstract
This lecture surveys how to turn body-worn cameras into end-to-end, outcome-driven products under real-world constraints. We first examine pragmatic strategies to overcome LLM context limits for long videos–favoring purpose-built representations over brute-force tokenization. We then map multi-persona product design (device user vs. platform operator) into an iterative, signal-driven development loop. Finally, we analyze edge–cloud trade-offs including the surprising shift toward audio-first analysis to maximize reliability, trust, and value via distributed AI processing.

Abstract
Scalable and efficient machine learning (ML) systems have been instrumental in fueling recent advancements in ML capabilities. However, further scaling these systems requires more than simply increasing the performance and quantity of accelerators such as GPUs. Modern ML deployments rely on complex pipelines composed of many diverse and interconnected systems beyond just accelerators. In this talk, I will emphasize the importance of building scalable systems across the entire ML pipeline. In particular, I will first explore how to build scalable data storage and ingestion systems to manage massive datasets for large-scale ML training pipelines, including those at Meta. To meet growing ML data demands, these data systems must be optimized for performance and efficiency. I will next illustrate how to leverage synergistic optimizations across the training data pipeline to unlock performance and efficiency gains beyond what isolated system optimizations can achieve. However, effectively deploying these optimizations requires navigating a complex system design space. To address this, I will finally introduce cedar, a framework that automates these optimizations and orchestrates ML data processing for diverse training workloads. Together, these systems enable scaling the entire ML pipeline, not just the GPUs.

Abstract
Agentic workflows commonly coordinate multiple models and tools with complex control logic. They are quickly becoming the dominant paradigm for AI applications. However, serving them remains inefficient with today’s frameworks. The key problem is that they expose workflows as opaque sequences of model and tool calls that tightly couple agent logic with model and hardware choices. Often, these workflow components are fragmented across different entities, preventing systems from reasoning about trade-offs across accuracy, latency, energy, and cost. This leads to resource waste and degraded service-level objectives (SLOs). We present Murakkab, a resource-efficient serving system for agentic workflows. Murakkab introduces a declarative abstraction that decouples workflow specification from execution configuration. A profile-guided optimizer and adaptive runtime jointly manage the full stack: orchestrating workflow components, mapping them to models and hardware, and dynamically reconfiguring execution to satisfy user-defined SLOs. By exposing the internal structure of agentic workflows, Murakkab enables cross-layer optimization that existing frameworks and cloud schedulers cannot achieve. Our evaluation on diverse workflows shows that Murakkab reduces GPU usage by up to 2.8×, energy consumption by 3.7×, and cost by 4.3× while maintaining SLOs.