Interplay Between Reinforcement Learning and Generative Artificial Intelligence (Fall 2025)

Overview

The project develops research reading, presentation, and writing skills through the study of a paper on the interplay between RL and GenAI. Students work in teams of two to present an assigned paper and prepare a joint report in LaTeX. The project is worth 50% of the course grade: 20% for the presentation and 30% for the report.

Fall 2025 deadlines: Presentation slides are due at the beginning of the assigned class. The final report is due December 2, 2025, at 11:59 pm. Late reports are not accepted. One team member submits the joint report through Canvas.

Project Guidelines (PDF)

Presentation Schedule

Date	Paper	Presenters
Nov 3	Decoding-Time Language Model Alignment with Multiple Objectives	Sanskar Amgain Tingwei Chen
Nov 5	Improving LLM General Preference Alignment via Optimistic Online Mirror Descent	Nolan Coffey Ethan Head
Nov 7	Accelerating RL for LLM Reasoning with Optimal Advantage Regression	Robert King Sheng Liu
Nov 10	DSPO: Direct Score Preference Optimization for Diffusion Model Alignment	Madhusree Mondal Kapildev Neupane
Nov 12	The Superposition of Diffusion Models Using the Itô Density Estimator	Faithful Mari Odoi Cameron Rader
Nov 14	Diffusion-based Reinforcement Learning via Q-weighted Variational Policy Optimization	Stephen Nathaniel Souther Utham Suresh
Nov 17	SafeDiffuser: Safe Planning with Diffusion Probabilistic Models	Qipan Xu Zecheng Zhan
Nov 19	Posterior Inference with Diffusion Models for High-dimensional Black-box Optimization	Bryce Bible

Reading Guideline

Connect the assigned paper to the course and discuss the following questions with your teammate:

What problem does the paper address, and how does it relate to the lectures?
What are its main ideas, methods, and contributions?
How do the results differ from related work?
What are the strengths and limitations?
What insights or different perspectives emerged from your discussion?

Presentation Guideline

Plan a 40-minute presentation, including transitions, followed by 10 minutes of questions and discussion. Each team member should present for approximately 20 minutes. A suggested structure is:

Introduction (4 minutes): Motivation, connection to the course, main contributions, and talk outline.
Problem setup (12 minutes): Formulation, notation, existing approaches, and key challenges.
Main results (20 minutes): New methods or theorems, the ideas behind them, and the experiments that support them.
Conclusion (4 minutes): Main takeaways, open questions, and connections to future research.

Aim for about 30 content slides. Use readable text, consistent notation, and clear illustrations. Focus on one or two main results instead of attempting to cover the entire paper. Prepare for questions throughout the talk and check the classroom projector before presenting. Audience questions count toward class participation.

For a one-person team, the presentation time is reduced by half.

Report Writing Guideline

Write a joint report in LaTeX using the NeurIPS 2025 format. The report must be at least 6 pages, excluding references and appendices, and explain the paper in your own words. Include:

Abstract: A one-paragraph summary.
Introduction: Motivation, course connections, challenges, and contributions.
Problem setup: The formulation and notation needed for the selected results.
Main results: The key formulations, algorithms, or theorems, with explanations of their meaning and implications.
Derivations and proofs: Selected important derivations or proofs, broken into clear steps.
Experiments: The experiments that best support the results, including the setting and data.
Conclusion: Main takeaways, followed by a separate contribution statement explaining each team member's role.
References and appendices: Citations and any supplementary material.

Maintain consistent notation, terminology, and formatting throughout the report. Students may arrange an appointment to discuss a draft. For a one-person team, the page requirement is reduced by half.

Optional Bonus

The project guidelines offer up to 10% of the course grade for optional work on the following topics:

Identify concrete open questions that are not discussed in the assigned paper, and explain possible approaches or why they are challenging.
Find and summarize additional papers in the same direction, explaining how they relate to the assigned paper.

Place bonus material after the conclusion and before the references. Identify the type of task and which team member wrote it; joint work is allowed.