Hello! My name is Saurabh Garg and I am a fifth year PhD student at Machine Learning Department at CMU where I am advised by Prof. Zachary Lipton and Prof. Siva Balakrishnan. I have also been lucky to collaborate with Prof. Zico Kolter and Prof. Aditi Raghunathan. I am broadly interested in building robust and deployable machine learning systems under distribution shift, now with a focus on large language models and multimodal models, e.g., CLIP. My PhD is supported by Bloomberg PhD Fellowship, JP Morgan AI PhD Fellowship and Amazon Graduate Fellowship.

I did my undergrad from IIT Bombay, India with major and honors in CS and minors in Applied Statistics in 2018. After that, I spent one amazing year at Samsung Headquaters, Korea. In the past, I have worked with Prof. Preethi Jyothi, Prof. Soumen Chakrabarti, Prof. Suyash Awate.

I am on job market now and I am actively seeking full-time positions!

Research/Mentorship Opportunities: I am always happy to discuss new research directions; If you’re a student at CMU interested in working on (fundamentals of) distribution shift related problems, in particular, domain adaptation or transfer learning, reach out to me at sgarg2@andrew.cmu.edu.


Oct 2023: My Apple internship work is out now: TiC-CLIP: Continual Training of CLIP Models. Quite excited about this work! Short version will appear as Oral at NeurIPS Dist Shift Workshop 2023.
Sept 2023: Our work on (i) Complementary Benefits of Contrastive Learning and Self-Training Under Distribution Shift (ii) Online Label Shift: Optimal Dynamic Regret meets Practical Algorithms (as Spotlight); and (iii) (Almost) Provable Error Bounds Under Distribution Shift via Disagreement Discrepancy got accepted at NeurIPS 2023. See you in New Orleans!
Sept 2023: We are organizing R0-FoMo: Robustness of Few-shot and Zero-shot Learning in Foundation Models (R0-FoMo) workshop at NeurIPS, 2023.
May 2023: We will be presenting our work on Downstream Datasets Make Surprisingly Good Pretraining Corpora at ACL 2023
May 2023: We will be presenting two papers at ICML 2023: (i) RLSbench; (ii) CHILS
Jan 2023: Our work on (i) Deconstructing Distributions; and (ii) Understanding SGD Noise got accepted at ICLR 2023
Sept 2022: Our work on (i) Domain adaptation under Open Set Label Shift, (ii) Unsupervised Learning under Latent Label Shift, and (iii) Characterizing Datapoints via Second-Split Forgetting got accepted at NeurIPS 2022.
July 2022: We are organizing Principles of Distribution Shift (PODS) workshop at ICML, 2022.
March 2022: Honored to receive the JP Morgan AI PhD Fellowship and Amazon Graduate Fellowship.
Feb 2022: Code for PU learning and RATT is out now.
Jan 2022: Work on investigate methods to predict target domain performance under distribution shift was accepted at ICLR 2022. [Arxiv link]
Sept 2021: Work on learning from positive and unlabeled data accepted at NeurIPS 2021 as a Spotlight!. [Arxiv link]
May 2021: Two papers at ICML: (i) Work on obtaining generalization bound with unlabeled data got accepted as Long talk at ICML 2021 [Paper]; (ii) Work on understanding heavy tails in PPO to appear as Short Talk at ICML 2021 [Paper].
April 2021: Our work on obtaining generalization gaurantees with unlabeled data will be presented at RobustML Workshop at ICLR 2021 [Paper] [Poster].
April 2021: Our work on understanding behaviour of gradients in PPO will be presented at SEDL Workshop at ICLR 2021. [Paper] [Talk] [Poster].
Feb 2021: Excited to be interning with Hanie Sedghi and Behnam Neyshabur at Google Brain during Summer 21.
Feb 2021: New work on understanding behaviour of gradients in PPO is out on arxiv.
Sept 2020: Our work on label shift got accepted at NeurIPs 2020 [Paper] [Poster].
July 2020: Our work on label shift estimation was accepted as Oral at ICML UDL 2020 [Talk] [Full Paper].
April 2020: Our work on Neural Architecture for Question Answering was an invited Oral at ECIR 2020 [Talk].
June 2019: I will be joining CMU ML Ph.D. in fall 2019.
April 2019: My B.Tech thesis titled "Estimating Uncertainty in MRF-based Image Segmentation: An Exact-MCMC Approach" got accepted at Medical Image Analysis 2019 journal
Dec. 2018: Received Excellence in Research Award from CSE dept, IIT Bombay
Nov. 2018: Presented my paper"Code-Switched Language models using Dual RNNs and Same-Source Pretraining" at EMNLP 2018, Brussels (poster)
Oct. 2018: Paper titled "Neural Architecture for Question Answering Using a Knowledge Graph and Web Corpus" got accepted at Information Retrieval Journal
Sept. 2018: Moved to Suwon, South Korea and joined Samsung Research Korea as Engineer
Sept. 2018: Presented my paper "Dual Language Models for Code Mixed Speech Recognition" at Interspeech 2018, Hyderabad (poster)
Aug. 2018: Graduated from IIT Bombay.
May 2018: Paper titled "Uncertainty Estimation in Segmentation with Perfect MCMC Sampling in Bayesian MRFs" got accepted at MICCAI, 2018 (poster)
Dec 2018: Invited to spend two weeks at Microsoft Research India to work on Indian language technologies with Prof. Preethi Jyothi
May 2017: Internship @ Samsung Research Korea
May 2016: Internship at Purdue Univeristy, US advised by Prof. Alex Pothen
July 2015: Changed branch from Electrical Engineering to Computer Science Engineering
July 2014: Joined IIT Bombay