Tutorial 2

Time and Venue

3 October 2025, 3:30 to 5 PM (ET)
Klaus 1116, Klaus Advanced Computing Building (KACB), Georgia Tech

Title and Abstract

Fine-Tuning (supervised and reinforcement learning) for Science

Fine-tuning large language models (LLMs) is key to adapting them for scientific problems, but choosing the right approach and preparing data can be challenging. We will motivate why fine-tuning is necessary, starting with the role of data and using examples from chemistry and other sciences to show how to identify, curate, and when necessary create datasets to address new research questions. This motivation will naturally lead to when different fine-tuning methods are appropriate.

We will then cover supervised fine-tuning with parameter-efficient methods such as Low-Rank Adaptation (LoRA) and other PEFT techniques, which make high-quality training feasible on modest GPU resources. Building on this foundation, we will introduce reinforcement learning approaches, including RL with human feedback (RLHF) and RL with symbolic feedback (RLSF) developed by our group, and touch on recent methods such as Direct Preference Optimization (DPO) and Grouped Reward Policy Optimization (GRPO).

By the end of this session, participants will understand how to move from a scientific problem to a fine-tuned model: deciding when to curate or create data, selecting appropriate fine-tuning methods, and recognizing when reinforcement learning is required.

Instructors

Piyush Jha

Ph.D. Student,
School of Computer Science,
Georgia Tech

Piyush Jha is a Ph.D. student in Computer Science at Georgia Tech advised by Prof. Vijay Ganesh. His research focuses on AI for Science and neurosymbolic techniques, developing computational methods that combine machine learning, symbolic reasoning, and reinforcement learning to discover new physics theories and design new materials with targeted properties. Piyush brings industry and research experience from Amazon Science, where he designed LLM-based autonomous model-building agents, and from Quadrical AI, a leading solar AI startup where he developed predictive analytics and digital-twin solutions for renewable energy systems. He has also contributed to multiple academic labs, building high-performance tools at the intersection of AI and scientific discovery. A two-time SMT-COMP winner, he has received the University of Waterloo Graduate Scholarship and the Pasupalak Scholarship in Robotics and AI, and his work has been published in top venues including ACM Transactions, IJCAI, AAAI, and ECAI.

Prithwish Jana

Ph.D. Student,
School of Computer Science,
Georgia Tech

Prithwish Jana is a Ph.D. student in Computer Science at Georgia Tech, advised by Prof. Vijay Ganesh and Prof. Sriram Vishwanath. He earned his M.Tech. (2022) from IIT Kharagpur and B.E. (2020) from Jadavpur University, graduating top of his class as the Institute Silver Medalist in his Master’s and the University Gold Medalist in his Bachelor’s. In addition to his academic research, he brings industry experience from Amazon Science (AWS, Seattle), where he worked as an Applied Scientist Intern with the Next Gen DevX (NGDE) team, designing LLM-based coding agents for Infrastructure-as-Code.
Prithwish’s research lies at the intersection of neuro-symbolic AI, formal methods, AI for code, and AI for mathematics. He develops neuro-symbolic techniques for fine-tuning large language models (LLMs), integrating symbolic reasoning tools and formal methods to enhance their reasoning capabilities in software engineering (e.g., code translation and code generation) and in mathematical reasoning (e.g., automated proof synthesis in Lean and proof auto-formalization).