## Summary

I have a awful memory, but its good enough most of the time, so I can remember where I left my coffee mug or when I am searching for it, where I have looked before. Imagine a person that has no recollection of what happened in their past. They might be running between room A and room B trying to find their coffee mug for ever, not realising they put it in the dishwasher.

What this person is lacking, is an episodic memory. A recollection of their, personal, previous experiences. Without them, they can only rely on what they observe and think about the world at the present moment.

Today on the Austrian Artificial Intelligence Podcast, I am talking to Fabian Paischer, PhD Student at the JKU in Linz and the ELISA PhD Program. Fabian is going to explain his research, developing an episodic memory system for reinforcement learning agents.

We will discuss his Semantic HELM paper in which they have been using pre-trained CLIP and LLM models to build an agents biography that serves the agent as an episodic memory.

How pre-trained foundation models help to build representations that generalize Reinforcement learning systems and help to understand and navigate in new environments.

This agent biography serves as a great help for the agent to solve specific memory related tasks, but in addition provides ways to interpret an agents behavior and thinking process.

I hope you enjoy this very interesting episode about current Reinforcement learning research.

## TOC

00:00:00 Beginning

00:02:08 Guest Introduction

00:07:15 Natural Language and Abstraction

00:10:37 European Ellis PhD Program

00:13:14 Episodic Memory in Reinforcement Learning

00:18:35 Symbolic State representation & Episodic Memory

00:27:04 Pre-trained Models for scene presentation

00:36:25 Semantic Helm Paper & Agent Interpretability

00:45:47 Improvements and Future research

## Sponsors

- Quantics: Supply Chain Planning for the new normal - the never normal - https://quantics.io/

- Belichberg GmbH: We do digital transformations as your innovation partner - https://belichberg.com/

## References

Fabian Paischer: https://www.jku.at/en/institute-for-machine-learning/about-us/team/fabian-paischer-msc/

Ellis PhD Program: https://ellis.eu/

SHELM Paper: https://arxiv.org/abs/2306.09312

HELM Paper: https://arxiv.org/abs/2205.12258

CLIP Explained: https://akgeni.medium.com/understanding-openai-clip-its-applications-452bd214e226