Nicolas Baumann’s Post

What if LLMs could embrace a "learning by doing" approach, like robots do? Reinforcement Learning (RL) in robotics is inherently interactive, agents explore environments, fail, adapt, and overcome. In contrast, RL for LLMs feels static: think curated datasets with known correct answers i.e. RLVR (Reinforcement Learning with Verifiable Rewards). But shouldn’t LLMs be able to learn through embodied interaction too? In our #CoRL2025 paper, Liam Boyle Paviththiren Sivasothilingam Michele Magno Luca Benini and I bring closed-loop RL to LLMs in a robotic setting. We teach a 1.5B and 3B-parameter LLM to drive a car, not just from static demonstrations, but by interacting with its environment. 🔑 Key Highlights: • Combines Supervised Fine-Tuning (SFT) with closed-loop RL, mirroring how humans learn: theory first, then practice. • Our 3B Qwen2.5 model, trained on a single RTX 3090, outperforms the much larger GPT-4o in control adaptability (63.3% vs. 58.5%). • Deployed fully onboard on a Jetson Orin AGX—no cloud needed. This is the first demonstration of closed-loop RL for LLMs in a real-world robotic task. And it shows that smaller, grounded models can beat larger, cloud-bound ones, when they learn like humans do. 📄 Paper: https://lnkd.in/dxEYkxGZ 🎥 Demo video: https://lnkd.in/dxJUrDYf 💻 Code & models: https://lnkd.in/dt8xCTai 🗓️ Attending CoRL 2025 in Seoul next week? Come see our spotlight talk + poster on September 29th, in Spotlight Session 4 & Poster 2. 🧾 Paper ID: 325 🎯 Spotlight presentation order: #19 📍 Poster number: 40 Affiliations: Center for Project-Based Learning D-ITET Integrated Systems Laboratory, ETH Zurich ForzaETH by Autonomous Racing Zürich ETH Hangar