Reward Relabelling for combined Reinforcement and Imitation Learning on sparse-reward tasks - Mines Paris Accéder directement au contenu
Communication Dans Un Congrès Année : 2023

Reward Relabelling for combined Reinforcement and Imitation Learning on sparse-reward tasks

Fabien Moutarde

Résumé

In the search for more sample-efficient reinforcement-learning (RL) algorithms, a promising direction is to leverage as much external off-policy data as possible. For instance, expert demonstrations. In the past, multiple ideas have been proposed to make good use of the demonstrations added to the replay buffer, such as pre-training on demonstrations only or minimizing additional cost functions. We present a new method, able to leverage both demonstrations and episodes collected online in any sparse-reward environment with any off-policy algorithm. Our method is based on a reward bonus given to demonstrations and successful episodes (via relabeling), encouraging expert imitation and self-imitation. Our experiments focus on several robotic-manipulation tasks across two different simulation environments. We show that our method based on reward relabeling improves the performance of the base algorithm (SAC) on these tasks.
Fichier principal
Vignette du fichier
STIR2_AAMAS2023-final.pdf (1.04 Mo) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04375346 , version 1 (05-01-2024)

Identifiants

  • HAL Id : hal-04375346 , version 1

Citer

Jesus Bujalance, Fabien Moutarde. Reward Relabelling for combined Reinforcement and Imitation Learning on sparse-reward tasks. 22nd International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2023), May 2023, London, United Kingdom. ⟨hal-04375346⟩
20 Consultations
10 Téléchargements

Partager

Gmail Facebook X LinkedIn More