Publications

Guided Flow Policy: Learning from High-Value Actions in Offline Reinforcement Learning

Franki Nguimatsia Tiofack*, Théotime Le Hellard*, Fabian Schramm*, Nicolas Perrin-Gilbert, Justin Carpentier

Published in arXiv, 2025

We introduce Guided Flow Policy (GFP), which couples a multi-step flow-matching policy with a distilled one-step actor to focus on learning from high-value actions in offline reinforcement learning.

Recommended citation: Tiofack, F. N., Le Hellard, T., Schramm, F., Perrin-Gilbert, N., & Carpentier, J. (2025). "Guided Flow Policy: Learning from High-Value Actions in Offline Reinforcement Learning." arXiv preprint arXiv:2512.03973. https://arxiv.org/abs/2512.03973

First-order Sobolev Reinforcement Learning

Fabian Schramm, Nicolas Perrin-Gilbert, Justin Carpentier

Published in Workshop on Differentiable Systems and Scientific Machine Learning, EurIPS 2025, 2025

We propose a refinement of temporal-difference learning that enforces first-order Bellman consistency by training the value function to match both Bellman targets and their derivatives.

Recommended citation: Schramm, F., Perrin-Gilbert, N., & Carpentier, J. (2025). "First-order Sobolev Reinforcement Learning." Workshop on Differentiable Systems and Scientific Machine Learning, EurIPS 2025. https://arxiv.org/abs/2511.19165

Reference-Free Sampling-Based Model Predictive Control

Fabian Schramm, Pierre Fabre, Nicolas Perrin-Gilbert, Justin Carpentier

Published in arXiv, 2025

We present a sampling-based model predictive control framework that enables emergent locomotion without relying on handcrafted gait patterns or predefined contact sequences.

Recommended citation: Schramm, F., Fabre, P., Perrin-Gilbert, N., & Carpentier, J. (2025). "Reference-Free Sampling-Based Model Predictive Control." arXiv preprint arXiv:2511.19204. https://arxiv.org/abs/2511.19204

KernelSOS for Global Sampling-Based Optimal Control and Estimation via Semidefinite Programming

Antoine Groudiev, Fabian Schramm, Éloïse Berthier, Justin Carpentier, Frederike Dümbgen

Published in arXiv, 2025

This paper applies the Kernel Sum of Squares framework for global sampling-based optimal control and estimation via semidefinite programming.

Recommended citation: Groudiev et al. (2025). "KernelSOS for Global Sampling-Based Optimal Control and Estimation via Semidefinite Programming." arXiv preprint arXiv:2507.17572. https://arxiv.org/abs/2507.17572

PROXQP: an Efficient and Versatile Quadratic Programming Solver for Real-Time Robotics Applications and Beyond

Antoine Bambade, Fabian Schramm, Sarah El Kazdadi, Stéphane Caron, Adrien Taylor, Justin Carpentier

Published in IEEE Transactions on Robotics, 2025

This paper presents ProxQP, a new and efficient QP solver for robotics and beyond.

Recommended citation: Bambade et al. (2025). "PROXQP: an Efficient and Versatile Quadratic Programming Solver for Real-Time Robotics Applications and Beyond." IEEE Transactions on Robotics. https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=11027562

End-to-End and Highly-Efficient Differentiable Simulation for Robotics

Quentin Le Lidec, Louis Montaut, Yann de Mont-Marin, Fabian Schramm, Justin Carpentier

Published in arXiv, 2024

This paper introduces a unified and efficient method for computing analytical derivatives in robotics simulators.

Recommended citation: Le Lidec et al. (2024). "End-to-End and Highly-Efficient Differentiable Simulation for Robotics." arXiv preprint arXiv:2409.07107. https://arxiv.org/pdf/2409.07107

Leveraging Randomized Smoothing for Optimal Control of Nonsmooth Dynamical Systems

Quentin Le Lidec, Fabian Schramm, Louis Montaut, Cordelia Schmid, Ivan Laptev, Justin Carpentier

Published in Nonlinear Analysis: Hybrid Systems, International Federation of Automatic Control (IFAC) journal, 2024

This paper presents randomized smoothing to tackle non-smoothness issues commonly encountered in optimal control and provides key insights on the interplay between Reinforcement Learning and Optimal Control.

Recommended citation: Le Lidec et al. (2024). "Leveraging Randomized Smoothing for Optimal Control of Nonsmooth Dynamical Systems." NAHS24. https://arxiv.org/abs/2203.03986

Leveraging augmented-Lagrangian techniques for differentiating over infeasible quadratic programs in machine learning

Antoine Bambade, Fabian Schramm, Adrien Taylor, Justin Carpentier

Published in Twelfth International Conference on Learning Representations (ICLR), 2023

This paper presents primal-dual augmented Lagrangian techniques for computing derivatives of both feasible and infeasible QPs.

Recommended citation: Bambade et al. (2023). "Leveraging augmented-Lagrangian techniques for differentiating over infeasible quadratic programs in machine learning." ICLR24. https://hal.laas.fr/PRAIRIE-IA/hal-04133055v1

Reactive Stepping for Humanoid Robots using Reinforcement Learning: Application to Standing Push Recovery on the Exoskeleton Atalante

Alexis Duburcq, Fabian Schramm, Guilhem Boéris, Nicolas Bredeche, Yann Chevaleyre

Published in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022

Video

This paper presents a reinforcement learning framework capable of learning robust standing push recovery for bipedal robots with a smooth out-of-the-box transfer to reality, requiring only instantaneous proprioceptive observations.

Recommended citation: Duburcq et al. (2022). "Reactive Stepping for Humanoid Robots using Reinforcement Learning: Application to Standing Push Recovery on the Exoskeleton Atalante." IROS22. https://arxiv.org/abs/2203.01148