Title: Order-Optimal Sample Complexity for Reinforcement Learning

Abstract: Deep Reinforcement Learning (DRL) has seen tremendous advancements, yet achieving optimal sample complexity remains a fundamental challenge. This talk presents recent progress in developing order-optimal sample complexity guarantees for RL algorithms with general parametrization. We first discuss the key difficulties in achieving optimal sample complexity and introduce an accelerated natural policy gradient (ANPG) approach that attains optimality in terms of order of sample complexity. We further explore improvements in actor-critic methods under Markovian sampling with neural network parameterization, reinforcement learning with human feedback for LLM alignment, and Quantum RL. Our results bridge crucial gaps in RL theory, offering practical implications for scalable and efficient decision-making systems.

Bio: Vaneet Aggarwal received the B.Tech. degree in 2005 from Indian Institute of Technology, Kanpur, India and the M.A. and Ph.D. degrees in 2007 and 2010, respectively from Princeton University, Princeton, NJ, USA, all in Electrical Engineering. He is currently a University Faculty Scholar and Professor at Purdue University, where he has been since Jan 2015. Prior to this, he worked as a researcher at AT&T Labs-Research, Florham Park, NJ for four and a half years. He was Adjunct Assistant Professor at Columbia University (EE, 2013-2014), VAJRA Adjunct Distinguished Professor at IISc Bangalore (ECE, 2018- 2019), Adjunct Faculty at IIIT Delhi (CS, 2022-2023), and Visiting Faculty at KAUST, Saudi Arabia (CS, 2022-2023). His research interests are in Reinforcement Learning, Generative AI, LLM Alignment, Quantum Machine Learning, and applications of ML. 

Dr. Aggarwal was the recipient of Princeton University's Porter Ogden Jacobus Honorific Fellowship in 2009. He received Vice President Award, Senior Vice President Award, and Key Contributor Award while at AT&T Labs – Research. He further received Purdue’s Most Impactful Faculty Innovator Award in 2020. He also received the 2017 Jack Neubauer Memorial Award recognizing the Best Systems Paper published in the IEEE Transactions on Vehicular Technology, the 2018 IEEE Infocom Workshop Best Paper Award, the 2021 NeurIPS Workshop Best Paper Award, and the 2024 IEEE William Bennett Prize Award recognizing the Best Paper published in IEEE/ACM Transactions on Networking. He has about 5 monographs/book chapters, 175 journal papers, 66 peer-reviewed conference papers in CS conferences ranked A/A* by CORE with at least 7 pages, and 130 other peer-reviewed conference papers in top-tier venues. He was on the Editorial Board of the IEEE Transactions on Green Communications and Networking, the IEEE Transactions on Communications, and the IEEE Transactions on Network Science and Engineering. He is currently serving on the Editorial Board of the IEEE/ACM Transactions on Networking and is founding co-Editor-in-Chief of the ACM Journal on Transportation Systems. He is also IEEE ComSoc Distinguished Lecturer (2024-2025).