Courses
Every Summer Semester we offer a course on Introduction to Reinforcement Learning. This course spans from basic notions of Reinforcement Learning (RL) to advanced topics involving planning and deep learning methodologies. Reinforcement Learning (RL) is a strongly interdisciplinary topic, that requires rigorous theoretical treatment and with numerous applications in real-world problems. Our course covers each topic from both a theoretical perspective, to properly gain the required knowledge of notions and algorithms, and from an empirical point of view, to understand the usefulness and impact of RL in real-world problems.
The course is organized in 12 lectures with the following topics:
Lecture 01: Learning agents;
Lecture 02: Markov Decision Processes (MDPs);
Lecture 03: Dynamic Programming;
Lecture 04: Multi-Armed Bandits and Exploration vs Exploitation;
Lecture 05: Model-Free Evaluation;
Lecture 06: Model-Free Control;
Lecture 07: Reinforcement Learning with value function approximation;
Lecture 08: Introduction to deep Reinforcement Learning;
Lecture 09: Policy Gradient;
Lecture 10: Actor-Critic approaches;
Lecture 11: Model-Based RL and Monte-Carlo Tree Search;
Lecture 12: Multi-agent Reinforcement Learning.
At the end of the course, the students will have developed the required theoretical and empirical knowledge about classic and modern reinforcement learning (RL) approaches. They will have a good knowledge of the core problems and goals of RL, its current state-of-the-art, and its future challenges. On a practical point of view, they will be able to design, implement, and carry out RL experiments using common tools, e.g., python libraries for RL and optimization. They will be able to conduct their M.Sc. theses on topics related to RL, and they will have good starting knowledge of RL for a future career in research or industry.
The evaluation consists of a written exam and 2 coding assignments.
The exam is in English and lasts 2 hours. There will be an exam at the end of the Summer Semester and one at the beginning of the Winter Semester. It will consist of open questions and/or quick application of algorithms by hand;
The coding assignments will be based on the technical skills developed in the exercise session, and will consist of autonomous implementation of algorithms, execution of experiments, and presentation of obtained results in a report of 2 pages. The two coding assignments will give, respectively, an additional bonus of 0.3 and 0.4 to the final grade of the written exam.
The course is organized into lectures and exercise sessions. The lectures will provide both theoretical and practical perspectives on the respective topics. The exercise session will consist of a quick refresh of the topic of the lecture and a guided coding session using the python programming language to implement and run reinforcement learning experiments.
The main reference book for the course is:
- Sutton, Richard S., and Andrew G. Barto. Reinforcement learning: An introduction. MIT press, 2018. (http://incompleteideas.net/book/the-book.html).
Additional references are:
- Szepesvári, Csaba. "Algorithms for reinforcement learning." Synthesis lectures on artificial intelligence and machine learning 4.1 (2010): 1-103. (https://sites.ualberta.ca/~szepesva/papers/RLAlgsInMDPs.pdf);
- Deisenroth, Marc Peter, A. Aldo Faisal, and Cheng Soon Ong. Mathematics for machine learning. Cambridge University Press, 2020.
Every Winter Semester we offer a course on Introduction to Artificial Intelligence (Intro2AI). The course aims to provide an introduction to AI by covering the main topics ranging from search and logic, to decision-making and learning. The course is highly recommended to whomever is interested in the exciting topic of AI for the current studies, and possibly for a future job in research or industry.
The course is organized in 15 lectures with the following topics:
Lecture 01: Introduction to Artificial Intelligence;
Lecture 02: Classical search;
Lecture 03: Advanced search;
Lecture 04: Propositional logic;
Lecture 05: First-order logic and inference;
Lecture 06: Planning;
Lecture 07: Knowledge representation;
Lecture 08: Uncertainty quantification;
Lecture 09: Probabilistic reasoning;
Lecture 10: Decision theory;
Lecture 11: Sequential decision theory;
Lecture 12: Multi-agent and game theory;
Lecture 13: Learning - Basics;
Lecture 14: Learning - Advanced;
Lecture 15: Reinforcement learning;
At the end of the course, the students will have acquired the required theoretical and empirical knowledge about classic and modern Artificial Intelligence (AI) approaches. They will have a good knowledge of the core problems and goals of AI, its current state-of-the-art, and its future challenges. They will be able to conduct their M.Sc. theses on topics related to AI, and they will have good starting knowledge of AI for a future career in research or industry.
The evaluation consists of a written exam.
The exam is in English and lasts 2 hours. There will be an exam at the end of the Winter Semester and one at the beginning of the Summer Semester. It will consist of open questions and/or quick application of algorithms by hand.
The course is organized into 15 lectures and 15 exercise sessions. The lectures will provide both theoretical and practical perspectives on the respective topics. The exercise sessions will aim at strengthening the knowledge of the topics presented in the lectures through a quick refresh and guided exercises. Some of the exercise sessions will be used to host invited lectures given by experts about cutting-edge topics of AI, e.g., Robotics and Virtual Reality, explaining them more in detail compared to the lecture. Through the expert knowledge of the invited speakers, the invited lectures aim to provide a rich knowledge of the topics that will shape the future of AI, with a particular focus on research aspects and their impact on society.
