Intern
Reinforcement Learning and Computational Decision-Making

Carlo D'Eramo

Prof. Dr. Carlo D'Eramo

Head
Reinforcement Learning and Computational Decision-Making
Julius-Maximilians-Universität
John-Skilton-Straße 8a
97074 Würzburg
Deutschland
Gebäude: JS8
Raum: 04.016
Telefon: +49 931 31-83457

 

 

About me

I am the head of the professorship for Reinforcement Learning and Computational Decision-Making at the Center for Artificial Intelligence and Data Science of Julius-Maximilians-Universität Würzburg. I am also independent group leader of hessian.AI.

The research of my LiteRL group revolves around the problem of how agents can efficiently acquire expert skills that account for the complexity of the real world. To answer this question, we are investigating lightweight methods to obtain adaptive autonomous agents, focusing on several RL topics including multi-task, curriculum, adversarial, options, and multi-agent RL.

Previously, I have studied Computer Engineering at Politecnico di Milano (Italy), where I obtained a B.Sc. degree in 2011 and an M.Sc. degree in 2015. I obtained a double degree in Computer Science at University of Illinois at Chicago (USA) in 2015. Afterwards, I conducted a Ph.D. in Information Technology at Politecnico di Milano (Italy), where I graduated in February 2019 with a thesis "On the exploitation of uncertainty to improve Bellman updates and exploration in Reinforcement Learning". Then, I have been a postdoctoral researcher at the Intelligent Autonomous Systems (IAS) group at TU Darmstadt from April 2019 to October 2022.

Follow me on X and on Google Scholar.

Check out my CV here.

Senior area chair: RLC.
Area chair: AAAI, ACML, AISTATS, NeurIPS, ICLR.
Reviewer for DFG and ERC proposals, and most of AI & ML conferences.

Full list here.

Journal articles

  • Tuan Dam, Carlo D'Eramo, Jan Peters, Joni Pajarinen. "A Unified Perspective on Value Backup and Exploration in Monte-Carlo Tree Search". Journal of Artificial Intelligence Research (JAIR). 2024.

  • Pascal Klink, Carlo D'Eramo, Jan Peters, Joni Pajarinen. "On the Benefit of Optimal Transport for Curriculum Reinforcement Learning". IEEE Transactions on Pattern Recognition and Machine Intelligence (PAMI). 2024.

  • Julen Urain, ..., Carlo D'Eramo, et al. "Composable Energy Policies for Reactive Motion Generation and Reinforcement Learning". International Journal of Robotics Research (IJRR). 2023.

  • Simone Parisi, ..., Carlo D'Eramo, et al. "Long-Term Visitation Value for Deep Exploration in Sparse-Reward Reinforcement Learning." Algorithms. 2022.

  • Pascal Klink, ..., Carlo D'Eramo, et al. "A Probabilistic Interpretation of Self-Paced Learning with Applications to Reinforcement Learning." Journal of Machine Learning Research (JMLR). 2021.

  • Carlo D'Eramo, et al. "Gaussian Approximation for Bias Reduction in Q-Learning." Journal of Machine Learning Research (JMLR). 2021.

  • Carlo D'Eramo, et al. "MushroomRL: Simplifying Reinforcement Learning Research." Journal of Machine Learning Research (JMLR). 2021.

  • Dorothea Koert, ..., Carlo D'Eramo, et al. "Multi-channel interactive reinforcement learning for sequential tasks." Frontiers in Robotics and AI. 2020.

Conference proceedings

  • Anna Riedmann, Julia Götz, Carlo D'Eramo, Birgit Lugrin. "Uli-RL: A Real-World Deep Reinforcement Learning Pedagogical Agent for Children". German Conference on Artificial Intelligence (KI). 2024.

  • Oliver Järnefelt, Mahdi Kallel, Carlo D'Eramo. "Cyclicity-Regularized Coordination Graphs". Reinforcement Learning Conference (RLC). 2024.

  • Erdi Sayar, Zhenshan Bing, Carlo D'Eramo, Ozgur S Oguz, Alois Knoll. "Contact Energy Based Hindsight Experience Prioritization". International Conference on Robotics and Automation (ICRA). 2024.

  • Aryaman Reddi, Maximilian Tölle, Jan Peters, Georgia Chalvatzaki, Carlo D'Eramo. "Robust Adversarial Reinforcement Learning via Bounded Rationality Curricula". International Conference on Learning Representations (ICLR) (Spotlight, 15% of the accepted papers). 2024.

  • Ahmed Hendawy, Jan Peters, Carlo D'Eramo. "Multi-Task Reinforcement Learning with Mixture of Orthogonal Experts". International Conference on Learning Representations (ICLR). 2024.

  • Mahdi Kallel, Debabrota Basu, Riad Akrour, Carlo D'Eramo. "Augmented Bayesian Policy Search". International Conference on Learning Representations (ICLR). 2024.

  • Gabriele Tiboni, Pascal Klink, Tatiana Tommasi, Jan Peters, Carlo D'Eramo, Georgia Chalvatzaki. "Domain Randomization via Entropy Maximization". International Conference on Learning Representations (ICLR). 2024.

  • Théo Vincent, Alberto Maria Metelli, Boris Belousov, Jan Peters, Marcello Restelli, Carlo D'Eramo. "Parameterized projected Bellman operator". Proceedings of the AAAI Conference on Artificial Intelligence (AAAI). 2024.

  • Carlo D'Eramo, and Georgia Chalvatzaki. "Prioritized Sampling with Intrinsic Motivation in Multi-Task Reinforcement Learning." International Joint Conference on Neural Networks (IJCNN). 2022.

  •  Pascal Klink, ..., Carlo D'Eramo et al. "Curriculum reinforcement learning via constrained optimal transport." International Conference on Machine Learning (ICML). 2022.

  •  Pascal Klink, Carlo D'Eramo, et al. "Boosted Curriculum Reinforcement Learning." International Conference on Learning Representations (ICLR). 2022.

  •  Tuan Dam, Carlo D'Eramo, et al. "Convex Regularization in Monte-Carlo Tree Search." International Conference on Machine Learning (ICML). 2021.

  •  Julen Urain, ..., Carlo D'Eramo, et al. "Composable energy policies for reactive motion generation and reinforcement learning." Robotics: Science and Systems (RSS), 2021.

  •  Andrew S. Morgan, ..., Carlo D'Eramo, et al. "Model predictive actor-critic: Accelerating robot skill acquisition with deep reinforcement learning." IEEE International Conference on Robotics and Automation (ICRA). 2021.

  •  Carlo D'Eramo, et al. "Sharing knowledge in multi-task deep reinforcement learning." International Conference on Learning Representations (ICLR), 2020.

  •  Pascal Klink, Carlo D'Eramo, et al. "Self-paced deep reinforcement learning." Advances in Neural Information Processing Systems (NeurIPS) (Oral, 2% of the accepted papers). (2020).

  •  Tuan Dam, Carlo D'Eramo, et al. "Generalized mean estimation in Monte-Carlo tree search." Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence. 2021.

  •  Samuele Tosatto, Carlo D'Eramo, et al. "Exploration driven by an optimistic bellman equation." International Joint Conference on Neural Networks (IJCNN). 2019.

  •  Carlo D'Eramo, Andrea Cini, and Marcello Restelli. "Exploiting action-value uncertainty to drive exploration in reinforcement learning." International Joint Conference on Neural Networks (IJCNN). 2019.

  •  Davide Tateo, Carlo D'Eramo, et al. "Exploiting structure and uncertainty of Bellman updates in Markov decision processes." IEEE Symposium Series on Computational Intelligence (SSCI). 2017.

  •  Samuele Tosatto, Carlo D'Eramo, et al. "Boosted fitted q-iteration." International Conference on Machine Learning (ICML). 2017.

  •  Carlo D'Eramo, et al. "Estimating the maximum expected value in continuous reinforcement learning problems." Proceedings of the AAAI Conference on Artificial Intelligence (AAAI). 2017.

  •  Carlo D'Eramo, Marcello Restelli, and Alessandro Nuara. "Estimating maximum expected value through gaussian approximation." International Conference on Machine Learning (ICML). 2016.

Workshop papers

  • Jose-Luis Holgado-Alvarez, Aryaman Reddi,  Carlo D'Eramo. "Dynamic Obstacle Avoidance with Bounded Rationality Adversarial Reinforcement Learning". LocoLearn: From Bioinspired Gait Generation to Active Perception. Conference on Robot Learning (CoRL). 2024.
  • Ahmed Hendawy, Jan Peters, Carlo D'Eramo. "Multi-Task Reinforcement Learning with Mixture of Orthogonal Experts". European Workshop on Reinforcement Learning (EWRL). 2024.
  • Mahdi Kallel, Debabrota Basu, Riad Akrour, Carlo D'Eramo. "Augmented Bayesian Policy Search". European Workshop on Reinforcement Learning (EWRL). 2024.
  • Oliver Järnefelt, Mahdi Kallel, Carlo D'Eramo. "Cyclicity-Regularized Coordination Graphs". European Workshop on Reinforcement Learning (EWRL). 2024.
  • Sebastian Griesbach, Carlo D'Eramo. "Deterministic Exploration via Stationary Bellman Error Maximization". European Workshop on Reinforcement Learning (EWRL). 2024.
  • Théo Vincent, Fabian Wahren, Jan Peters, Boris Belousov, Carlo D'Eramo. "Adaptive Q-Network: On-the-fly Target Selection for Deep Reinforcement Learning". European Workshop on Reinforcement Learning (EWRL). 2024.
  • Mahdi Kallel, Samuele Tosatto, Carlo D'Eramo. "Revisiting On-Policy Deep Reinforcement Learning". European Workshop on Reinforcement Learning (EWRL). 2024.
  • Théo Vincent, Fabian Wahren, Jan Peters, Boris Belousov, Carlo D'Eramo. Adaptive Q-Network: On-the-fly Target Selection for Deep Reinforcement Learning. Automated Reinforcement Learning: Exploring Meta-Learning, AutoML, and LLMs. International Conference on Machine Learning (ICML) 2024.
  • Henrik Metternich, Ahmed Hendawy, Pascal Klink, Carlo D'Eramo. Using Proto-Value Functions for Curriculum Generation in Goal-Conditioned RL. Goal-Conditioned Reinforcement Learning Workshop - Advances in Neural Information Processing Systems (NeurIPS). 2023.
  • Marcel Mittenbuehler, Ahmed Hendawy, Carlo D'Eramo, Georgia Chalvatzaki. Parameter-efficient Tuning of Pretrained Visual-Language Models in Multitask Robot Learning. Workshop on Learning Effective Abstractions for Planning (LEAP) - Conference On Robot Learning (CoRL). 2023.
  • Tuan Dam, Carlo D'Eramo, Joni Pajarinen, Jan Peters. A Unified Perspective on Value Backup and Exploration in Monte-Carlo Tree Search. European Workshop on Reinforcement Learning (EWRL). 2023.
  • Théo Vincent, Boris Belousov, Carlo D'Eramo, Jan Peters. Iterated Deep Q-Network: Efficient Learning of Bellman Iterations for Deep Reinforcement Learning. European Workshop on Reinforcement Learning (EWRL). 2023.
  • Théo Vincent, Alberto Maria Metelli, Jan Peters, Marcello Restelli, Carlo D'Eramo. "Parameterized projected Bellman operator". Frontiers4LCD - International Conference on Machine Learning (ICML). 2023.
  • Oliver Järnefelt, Carlo D'Eramo. "Modular Value Function Factorization in Multi-Agent Reinforcement Learning". Workshop on Decision-Making in Multi-Agent Systems - IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 2022.
  • Pascal Klink, Haoyi Yang, Carlo D'Eramo, Jan Peters, Joni Pajarinen. "Curriculum Reinforcement Learning via Constrained Optimal Transport". European Workshop on Reinforcement Learning (EWRL). 2022.

Theses

  • Carlo D'Eramo. "On the exploitation of uncertainty to improve Bellman updates and exploration in Reinforcement Learning." Ph.D. Thesis (2019).

  • Carlo D'Eramo. "On the use of deep Boltzmann machines for road signs classification." M.Sc. Thesis. Politecnico di Milano (2015).

Now
  • JMU - Introduction to Reinforcement Learning;
  • JMU - Introduction to Artificial Intelligence;
  • JMU - Seminar Introduction to Reinforcement Learning: from foundations to modern approaches.

Previously

  • TU Darmstadt - Introduction to Reinforcement Learning: from foundations to deep approaches.