Rui Liu

Ph.D. candidate in Computer Science, University of Maryland

I am a Ph.D. candidate in Computer Science at the University of Maryland, College Park, advised by Prof. Pratap Tokekar and Prof. Ming Lin. I have interned at Netflix Research, Tencent Hunyuan LLM Frontier & AI Lab, Apple, and Tencent Robotics X, and I received my bachelor’s degree from Shanghai Jiao Tong University.

My research focuses on post-training large language models: reinforcement learning, reward modeling, and agentic LLMs that can recursively improve themselves. My work aims to make RL for LLMs stable and sample-efficient, rewards harder to game, and reasoning robust to noisy or degraded inputs.

Research

RL for LLM post-training

Stable, efficient reinforcement learning with verifiable rewards for (multimodal) reasoning: group-free single-rollout optimization, uncertainty- and curiosity-driven exploration, parallel thinking, and robustness to degraded inputs.

MSSR
ROMA
DUPL
CDE
Parallel-R1

Reward modeling and verification

Generative reward models that plan and verify with self-generated rubrics, multimodal judges, process supervision that locates the first error, and self-rewarding vision-language models.

Agentic LLMs and recursive self-improvement

LLM agents that discover their own test-time scaling strategies and improve their exploration policies by replaying their own discovery history.

Earlier work: robust and multimodal learning for multi-agent systems and robotics, including CAML (NeurIPS 2025), MMCD (IROS 2025), IMRL (ICRA 2025), LAVA (IROS 2024, ICRA workshop best paper), and D³ROC (IROS 2023).

News

Sep 2026Two papers accepted to NeurIPS 2026: ROMA (RL for robust multimodal reasoning) and LLMs Improving LLMs (agentic discovery of test-time scaling strategies).
Sep 2026Three new papers: Dream-RSI on recursive self-improvement, CARE on certified VLA inference acceleration, and ATR, a multimodal lip-sync judge.
Aug 2026Two papers accepted to EMNLP 2026: RL-guided adaptive sampling for test-time scaling (Main) and DUPL (Findings).
Summer 2026Research internship at Netflix Research on LLM post-training, reasoning, and reward modeling.
May 2026Three new papers: DeltaRubric on multimodal reward modeling with self-generated rubrics, ROMA on robust multimodal reasoning, and AutoTTS on automated test-time scaling.
Apr 2026Parallel-Probe accepted to ICML 2026, and VPPO accepted to ACL 2026 Findings.

Show earlier newsHide earlier news

Feb 2026MSSR (Multimodal Stabilized Single-Rollout RL) accepted to CVPR 2026.
Feb 2026Three new papers: A2MAML for multi-agent multimodal learning, Parallel-Probe for efficient parallel thinking, and VPPO for process-supervised RL.
Jan 2026Four papers accepted to ICLR 2026: Parallel-R1, CDE, Vision-SR1, and AdaConG.
Jan 2026DUPL (Dual-Uncertainty Guided Policy Learning) for multimodal LLM reasoning is out.
Dec 2025MSSR (Multimodal Stabilized Single-Rollout RL) for multimodal LLM reasoning is out.
Sep 2025Three papers on (multimodal) LLM reasoning: Vision-SR1 for VLM reasoning, Parallel-R1 for parallel thinking with RL, and CDE (Curiosity-Driven Exploration) for LLM reasoning.
Sep 2025CAML (multimodal multi-agent learning) accepted to NeurIPS 2025. AdaConG on learning under uncertainty is out.
Aug 2025Paper on risk-sensitive policy gradient is out.
Jun 2025MMCD (multimodal collaborative decision-making) accepted to IROS 2025.
May 2025Started a research internship at Tencent (AI Lab and Hunyuan LLM Frontier), Bellevue, WA.
Jan 2025IMRL (robust imitation learning for robotic manipulation) accepted to ICRA 2025.
Jun 2024LAVA (visual action learning for robotic manipulation) accepted to IROS 2024.
Jun 2023D³ROC (distributionally robust optimal control) accepted to IROS 2023.
May 2023Started a research internship at Apple, Cupertino, CA.

Selected publications

All papers are in my CV and on Google Scholar.

Dream-RSI: Recursive Self-Improvement through Evolving Worlds

Tong Zheng, Xidong Wu, Zheng Zhang, Zhankui He, Chaoyi Zhang, Benjamin Coleman, Ruoqiao Wei, Di Bai, Haolin Liu, Rui Liu, Xue Wang, Yue Zhuan, Wang-Cheng Kang, Renkai Xiang, Heng Huang, Xinwu Cheng, Yunsong Guo

Tech Report 2026 Self-improvement

Dream-RSI turns an agent's completed discovery trees into a replay simulator, so new exploration policies can be evaluated and refined offline (by dreaming) before being redeployed online, closing a self-improvement loop at the exploration layer.

DeltaRubric: Generative Multimodal Reward Modeling via Joint Planning and Verification

Rui Liu, Dian Yu, Zhenwen Liang, Yucheng Shi, Tong Zheng, Runpeng Dai, Haitao Mi, Pratap Tokekar, Leoweiliang

Preprint 2026 Reward modeling

We propose DeltaRubric, a novel approach that reframes multimodal evaluation as an active, two-step visual investigation through a single shared MLLM, leveraging self-generated rubrics as guidance.

Align Then Reason: A Multimodal Lip-Sync Judge for Dubbing

Rui Liu, Bhavin Jawade, Haoqi Li, Shivam Mehta, Karan Saxena, Yinghong Lan, Cameron Wolfe

Preprint 2026 Reward modeling

We propose ATR, a temporally grounded multimodal multilingual lip-sync judge for dubbing.

Reinforcing Multimodal Reasoning Against Visual Degradation

Rui Liu, Dian Yu, Haolin Liu, Yucheng Shi, Tong Zheng, Runpeng Dai, Haitao Mi, Pratap Tokekar, Leoweiliang

NeurIPS 2026 RL post-training

We propose ROMA, an RL fine-tuning approach for MLLMs that enforces robustness to visual degradation.

Dual-Uncertainty Guided Policy Learning for Multimodal Reasoning

Rui Liu, Dian Yu, Tong Zheng, Runpeng Dai, Zongxia Li, Wenhao Yu, Zhenwen Liang, Linfeng Song, Haitao Mi, Pratap Tokekar, Dong Yu

EMNLP Findings 2026 RL post-training

A method that guides policy learning using measured dual-uncertainty (output and perceptual) as feedback signals to encourage exploration and enhance reasoning in multimodal LLMs.

Stable and Efficient Single-Rollout RL for Multimodal Reasoning

Rui Liu, Dian Yu, Lei Ke, Haolin Liu, Yujun Zhou, Zhenwen Liang, Haitao Mi, Pratap Tokekar, Dong Yu

CVPR 2026 RL post-training

A group-free RLVR approach that achieves both stable optimization and effective multimodal reasoning performance.

LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling

Tong Zheng, Haolin Liu, Chengsong Huang, Huiwen Bao, Sheng Zhang, Rui Liu, Runpeng Dai, Ruibo Chen, Chenxi Liu, Tianyi Xiong, Xidong Wu, Hongming Zhang, Heng Huang

NeurIPS 2026 Agentic LLMs

We propose an environment-driven framework, AutoTTS, that changes what researchers design: from individual TTS heuristics to environments where TTS strategies can be discovered automatically.

Parallel-R1: Towards Parallel Thinking via Reinforcement Learning

Tong Zheng, Hongming Zhang, Wenhao Yu, Xiaoyang Wang, Runpeng Dai, Rui Liu, Huiwen Bao, Chengsong Huang, Heng Huang, Dong Yu

ICLR 2026 RL post-training

An RL framework that enhances LLM reasoning capabilities by enabling parallel thinking through a progressive curriculum.

CDE: Curiosity-Driven Exploration for Efficient Reinforcement Learning in Large Language Models

Runpeng Dai, Linfeng Song, Haolin Liu, Zhenwen Liang, Dian Yu, Haitao Mi, Zhaopeng Tu, Rui Liu, Tong Zheng, Hongtu Zhu, Dong Yu

ICLR 2026 RL post-training

CDE enhances Reinforcement Learning with Verifiable Rewards (RLVR) by using intrinsic curiosity signals from the actor and critic to improve exploration and reduce premature convergence in LLMs.