Domain invariant Q-learning for model-free robust continuous control under visual distractions - Institut Polytechnique de Paris
Conference Papers Year : 2022

Domain invariant Q-learning for model-free robust continuous control under visual distractions

Abstract

End-to-end reinforcement learning on images showed significant performance progress in the recent years, especially with regularization to value estimation brought by data augmentation (Yarats et al., 2020). At the same time, domain randomization and representation learning helped push the limits of these algorithms in visually diverse environments, full of distractors and spurious noise, making RL more robust to unrelated visual features. We present DIQL, a method that combines risk invariant regularization and domain randomization to reduce out-of-distribution (OOD) generalization gap for temporal-difference learning. In this work, we draw a link by framing domain randomization as a richer extension of data augmentation to RL and support its generalized use. Our model-free approach improve baselines performances without the need of additional representation learning objectives and with limited additional computational cost. We show that DIQL outperforms existing methods on complex visuo-motor control environment with high visual perturbation. In particular, our approach achieves state-of the-art performance on the Distracting Control Suite benchmark, where we evaluate the robustness to a number of visual perturbators, as well as OOD generalization and extrapolation capabilities.
Fichier principal
Vignette du fichier
132_domain_invariant_q_learning_fo.pdf (2.55 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-04135292 , version 1 (20-06-2023)

Identifiers

  • HAL Id : hal-04135292 , version 1

Cite

Tom Dupuis, Jaonary Rabarisoa, Quoc-Cuong Pham, David Filliat. Domain invariant Q-learning for model-free robust continuous control under visual distractions. NeurIPS 2022 - Deep Reinforcement Learning Workshop, Dec 2022, Virtual, United States. ⟨hal-04135292⟩
80 View
40 Download

Share

More