Publications

Ekka: Automated Diagnosis of Silent Errors in LLM Inference

Yile Gu, Zhen Zhang, Shaowei Zhu, Xinwei Fu, Jun Wu, Yida Wang, Baris Kasikci — International Conference on Machine Learning (ICML) (2026)

TraceLab: Characterizing Coding Agent Workloads for LLM Serving

Inference & ServingAgents

Kan Zhu, Mathew Jacob, Chenxi Ma, Yi Pan, Stephanie Wang, Arvind Krishnamurthy, Baris Kasikci — (2026)

Piper: A Programmable Distributed Training System

TrainingProgrammable Systems

Megan Frisella, Shubham Tiwari, Andy Ruan, Yi Pan, Parker Gustafson, Mat Jacob, Gilbert Bernstein, Stephanie Wang — (2026)

M*: A Modular, Extensible, Serving System for Multimodal Models

Inference & ServingMultimodalProgrammable Systems

Atindra Jha, Naomi Sagan, Keisuke Kamahori, Irmak Sivgin, Rohan Sanda, Steven Gao, Mark Horowitz, Luke Zettlemoyer, Olivia Hsu, Jure Leskovec, Baris Kasikci, Stephanie Wang — (2026)

MURMUR: An Efficient Inference System for Long-Form ASR

Inference & ServingMultimodal

Wei-Tzu Lee, Keisuke Kamahori, Baris Kasikci — (2026)

VibeServe: Can AI Agents Build Bespoke LLM Serving Systems?

Inference & ServingAgents

Keisuke Kamahori, Shihang Li, Simon Peter, Baris Kasikci — (2026)

VoxServe: Streaming-Centric Serving System for Speech Language Models

Inference & ServingMultimodal

Keisuke Kamahori, Wei-Tzu Lee, Atindra Jha, Rohan Kadekodi, Stephanie Wang, Arvind Krishnamurthy, Baris Kasikci — (2026)

Reducing the GPU Memory Bottleneck with Lossless Compression for ML

Training

Aditya K Kamath, Arvind Krishnamurthy, Marco Canini, Simon Peter — European Conference on Computer Systems (EuroSys) (2026)

FlashInfer-Bench: Building the Virtuous Cycle for AI-driven LLM Systems

Inference & ServingAgents

Shanli Xing, Yiyan Zhai, Alexander Jiang, Yixin Dong, Yong Wu, Zihao Ye, Charlie Ruan, Yingyi Huang, Yineng Zhang, Liangsheng Yin, Aksara Bayyapu, Luis Ceze, Tianqi Chen — Annual Conference on Machine Learning and Systems (MLSys) (2026)

Accelerating Large-Scale Reasoning Model Inference with Sparse Self-Speculative Decoding

Inference & Serving

Yilong Zhao, Jiaming Tang, Kan Zhu, Zihao Ye, Chi-Chih Chang, Chaofan Lin, Jongseok Park, Guangxuan Xiao, Mohamed S. Abdelfattah, Mingyu Gao, Baris Kasikci, Song Han, Ion Stoica — Annual Conference on Machine Learning and Systems (MLSys) (2026)

DynaFlow: Transparent and Flexible Intra-Device Parallelism via Programmable Operator Scheduling

Inference & ServingProgrammable Systems

Yi Pan, Yile Gu, Jinbin Luo, Yibo Wu, Ziren Wang, Hongtao Zhang, Ziyi Xu, Shengkai Lin, Baris Kasikci, Stephanie Wang — Annual Conference on Machine Learning and Systems (MLSys) (2026)

Unleashing Scalable Context Parallelism for Foundation Models Pre-Training via FCP

Training

Yilong Zhao, Xiaonan Nie, Kan Zhu, Shuang Ma, Zhichao Lai, Hongxiang Hao, Yang Zhou, Baris Kasikci, Ion Stoica — Annual Conference on Machine Learning and Systems (MLSys) (2026)

TeleRAG: Efficient Retrieval-Augmented Generation Inference with Lookahead Retrieval

Inference & ServingML + Data

Chien-Yu Lin, Keisuke Kamahori, Yiyu Liu, Xiaoxiang Shi, Madhav Kashyap, Yile Gu, Rulin Shao, Zihao Ye, Kan Zhu, Stephanie Wang, Arvind Krishnamurthy, Rohan Kadekodi, Luis Ceze, Baris Kasikci — Annual Conference on Machine Learning and Systems (MLSys) (2026)

Tactic: Adaptive Sparse Attention with Clustering and Distribution Fitting for Long-Context LLMs

Inference & Serving

Kan Zhu, Tian Tang, Qinyu Xu, Yile Gu, Zhichen Zeng, Rohan Kadekodi, Liangyu Zhao, Ang Li, Arvind Krishnamurthy, Baris Kasikci — International Conference on Learning Representations (ICLR) (2026)

The Streaming Batch Model for Efficient and Fault-Tolerant Heterogeneous Execution

ML + DataTraining

Frank Sifei Luan, Ron Yifeng Wang, Yile Gu, Ziming Mao, Charlotte Lin, Amog Kamsetty, Hao Chen, Cheng Su, Balaji Veeramani, Scott Lee, SangBin Cho, Clark Zinzow, Eric Liang, Ion Stoica, Stephanie Wang — (2025)

Programmable and Adaptive Scheduling for Distributed Systems

Programmable Systems

Yuyao Wang, Xiangfeng Zhu, Ratul Mahajan, Stephanie Wang — Hot Topics in Networks (HotNets) (2025)

Piper: Towards Flexible Pipeline Parallelism for PyTorch

TrainingProgrammable Systems

Megan Frisella, Arvin Oentoro, Xiangyu Gao, Gilbert Bernstein, Stephanie Wang — Practical Adoption Challenges of ML for Systems (PACMI) (2025)

LiteASR: Efficient Automatic Speech Recognition with Low-Rank Approximation

Inference & ServingMultimodal

Keisuke Kamahori, Jungo Kasai, Noriyuki Kojima, Baris Kasikci — Conference on Empirical Methods in Natural Language Processing (EMNLP) (2025)

FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving

Inference & Serving

Zihao Ye, Lequn Chen, Ruihang Lai, Wuwei Lin, Yineng Zhang, Stephanie Wang, Tianqi Chen, Baris Kasikci, Vinod Grover, Arvind Krishnamurthy, Luis Ceze — Annual Conference on Machine Learning and Systems (MLSys) (2025)

Argos: Detecting Dynamic Anomalies in the Cloud with Rule Generation

AgentsReliability & Energy

Yile Gu, Hoang Doan Nguyen, Demirhan Celik, Sifat Hasan, Yifan Xiong, Jonathan Mace, Yuting Jiang, Yigong Hu, Baris Kasikci, Peng Cheng — arXiv preprint (2025) (2025)

NanoFlow: Towards Optimal Large Language Model Serving Throughput

Inference & Serving

Kan Zhu, Yufei Gao, Yilong Zhao, Liangyu Zhao, Gefei Zuo, Yile Gu, Dedong Xie, Tian Tang, Qinyu Xu, Zihao Ye, Keisuke Kamahori, Chien-Yu Lin, Ziren Wang, Stephanie Wang, Arvind Krishnamurthy, Baris Kasikci — Symposium on Operating Systems Design and Implementation (OSDI) (2025)

Magneton: Optimizing Energy Efficiency of ML Systems via Differential Energy Debugging

Reliability & Energy

Yi Pan, Wenbo Qian, Dedong Xie, Ruiyan Hu, Yigong Hu, Baris Kasikci — arXiv preprint (2025) (2025)

Towards ML System Extensibility

Programmable Systems

Weixin Deng, Andy Ruan, Megan Frisella, Kai-Hsun Chen, SangBin Cho, Jack Tigar Humphries, Rui Qiao, Stephanie Wang — Hot Topics in Operating Systems (HotOS) (2025)

Fiddler: CPU-GPU Orchestration for Fast Inference of Mixture-of-Experts Models

Inference & Serving

Keisuke Kamahori, Tian Tang, Yile Gu, Kan Zhu, Baris Kasikci — International Conference on Learning Representations (ICLR) (2025)

Datacomp-LM: In search of the next generation of training sets for language models

ML + Data

Jeffrey Li, Alex Fang, Georgios Smyrnis, Maor Ivgi, Matt Jordan, Samir Gadre, Hritik Bansal, Etash Guha, Sedrick Keh, Kushal Arora, Saurabh Garg, Rui Xin, Niklas Muennighoff, Reinhard Heckel, Jean Mercat, Mayee Chen, Suchin Gururangan, Mitchell Wortsman, Alon Albalak, Yonatan Bitton, Marianna Nezhurina, Amro Abbas, Cheng-Yu Hsieh, Dhruba Ghosh, Josh Gardner, Maciej Kilian, Hanlin Zhang, Rulin Shao, Sarah Pratt, Sunny Sanyal, Gabriel Ilharco, Giannis Daras, Kalyani Marathe, Aaron Gokaslan, Jieyu Zhang, Khyathi Chandu, Thao Nguyen, Igor Vasiljevic, Sham Kakade, Shuran Song, Sujay Sanghavi, Fartash Faghri, Sewoong Oh, Luke Zettlemoyer, Kyle Lo, Alaaeldin El-Nouby, Hadi Pouransari, Alexander Toshev, Stephanie Wang, Dirk Groeneveld, Luca Soldaini, Pang Wei Koh, Jenia Jitsev, Thomas Kollar, Alexandros G Dimakis, Yair Carmon, Achal Dave, Ludwig Schmidt, Vaishaal Shankar — Conference on Neural Information Processing Systems (NeurIPS) (2024)

Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference

Inference & Serving

Jiaming Tang, Yilong Zhao, Kan Zhu, Guangxuan Xiao, Baris Kasikci, Song Han — International Conference on Machine Learning (ICML) (2024)

Atom: Low-bit Quantization for Efficient and Accurate LLM Serving

Inference & Serving

Yilong Zhao, Chien-Yu Lin, Kan Zhu, Zihao Ye, Lequn Chen, Size Zheng, Luis Ceze, Arvind Krishnamurthy, Tianqi Chen, Baris Kasikci — Annual Conference on Machine Learning and Systems (MLSys) (2024)