Reinforcement Learning & Foundation Models

Exploration under sparse rewards, intrinsic motivation and language-guided agents built on pretrained foundation models.

A robot agent finding its path to a goal flag through a grid of obstacles, with a rising reward curve

This line studies how reinforcement learning (RL) agents can learn efficiently when rewards are sparse, and how pretrained foundation models (vision-language models and LLMs) can guide them through intrinsic motivation, high-level language prompts and structured task decomposition.

It grew out of Alain Andrés Fernández’s doctoral thesis (2023), which studied intrinsic motivation mechanisms for sample-efficient deep RL in hard-exploration environments. That work covered self-imitation learning with prioritization and diversity, the use of offline data in procedurally generated environments, and collaborative training of heterogeneous agents. Unai Ruiz-Gonzalez’s ongoing thesis extends it to open-ended learning that uses the latent knowledge of foundation models. Other contributions in this line address multitask RL through evolutionary multifactorial optimization, curiosity-driven continual offline RL, and safe RL when agents meet out-of-distribution observations.

Main contributions

Publications in this area (11)

All my publications related to this research line, in reverse chronological order, including those mentioned above.

2025

  1. Neurocomputing
    Using Offline Data to Speed-up Reinforcement Learning in Procedurally Generated Environments
    Alain Andres, Lukas Schäfer, Stefano V. Albrecht, and Javier Del Ser
    Neurocomputing, 2025
  2. Machine Learning and Knowledge Extraction
    Large Language Models for Structured Task Decomposition in Reinforcement Learning Problems with Sparse Rewards
    Unai Ruiz-Gonzalez, Alain Andrés, and Javier Del Ser
    Machine Learning and Knowledge Extraction, 2025

2024

  1. Cognitive Computation
    Using Curiosity for an Even Representation of Tasks in Continual Offline Reinforcement Learning
    Pankayaraj Pathmanathan, Natalia Díaz-Rodríguez, and Javier Del Ser
    Cognitive Computation, 2024
  2. NeurIPS 2024 Workshop on Intrinsically Motivated Open-ended Learning (IMOL)
    Fostering Intrinsic Motivation in Reinforcement Learning with Pretrained Foundation Models
    A. Andres and Javier Del Ser
    In NeurIPS 2024 Workshop on Intrinsically Motivated Open-ended Learning (IMOL), 2024
  3. NeurIPS 2024 Workshop on Open-World Agents (OWA)
    Words as Beacons: Guiding RL Agents with High-Level Language Prompts
    U. Ruiz-Gonzalez, A. Andres, P. G. Bascoy, and Javier Del Ser
    In NeurIPS 2024 Workshop on Open-World Agents (OWA), 2024
  4. International Joint Conference on Neural Networks (IJCNN)
    Advancing Towards Safe Reinforcement Learning over Sparse Environments with Out-of-Distribution Observations: Detection and Adaptation Strategies
    Aitor Martínez-Seras, Alain Andrés, and Javier Del Ser
    In International Joint Conference on Neural Networks (IJCNN), 2024

2023

  1. IEEE Symposium Series on Computational Intelligence (SSCI)
    Enhanced Generalization Through Prioritization and Diversity in Self-Imitation Reinforcement Learning over Procedural Environments with Sparse Rewards
    Alain Andrés, Daochen Zha, and Javier Del Ser
    In IEEE Symposium Series on Computational Intelligence (SSCI), 2023
  2. Neural Computing and Applications
    Collaborative Training of Heterogeneous Reinforcement Learning Agents in Environments with Sparse Rewards: What and When to Share?
    Alain Andres, Esther Villar-Rodriguez, and Javier Del Ser
    Neural Computing and Applications, 2023

2022

  1. International Cross-Domain Conference, CD-MAKE 2022
    An Evaluation Study of Intrinsic Motivation Techniques Applied to Reinforcement Learning over Hard Exploration Environments
    Alain Andres, Esther Villar-Rodriguez, and Javier Del Ser
    In International Cross-Domain Conference, CD-MAKE 2022, 2022
  2. IEEE Symposium Series on Computational Intelligence (SSCI)
    Towards Improving Exploration in Self-Imitation Learning Using Intrinsic Motivation
    Alain Andres, Esther Villar-Rodriguez, and Javier Del Ser
    In IEEE Symposium Series on Computational Intelligence (SSCI), 2022
  3. IEEE Transactions on Evolutionary Computation
    Adaptive Multi-Factorial Evolutionary Optimization for Multi-Task Reinforcement Learning
    Aritz D. Martinez, Javier Del Ser, Eneko Osaba, and Francisco Herrera
    IEEE Transactions on Evolutionary Computation, 2022