On 6 May, the DEEC TALK Policy Optimization in Reinforcement Learning took place, featuring Yuejie Chi, Charles C. and Dorothea S. Dilley Professor of Statistics and Data Science at Yale University, with a secondary appointment in Computer Science.

Yuejie Chi’s research aims to accelerate the learning process of algorithms and focuses on optimizing outcomes through reinforcement learning: the system progressively improves its decisions through interaction with the environment. These predictive mechanisms can be applied in various research and industrial areas, such as the design of chips for artificial intelligence and signal processing.

Starting with a reference to the film The Thinking Game, in which a researcher seeks to develop an artificial intelligence system capable of solving complex problems, Yuejie Chi explained that her team’s work focuses on optimizing the performance, efficiency, and viability of generative AI systems and large language models (LLMs), particularly in scenarios involving large datasets and limited resources. These tools can be used, for example, for information aggregation in multi-agent systems, where multiple agents rapidly adopt strategies. Each agent interacts with other agents and with its environment, taking into account the information it possesses.

Yuejie Chi also explained how each agent makes its decisions: the system receives rewards or penalties based on its actions (reward signal). As a result, instead of an isolated sequence of decisions, there is a continuous interaction with the environment. Because the environment is dynamic, the system is always based on probabilities related, for example, to the coordinates and positions of objects in space, depending on the current state.

On the other hand, each choice influences both future decisions and subsequent rewards. For this reason, she discussed the use of policy gradient methods: the system evaluates possible actions and adopts a policy that maximizes expected reward (soft policy). In this context, the Bellman operator is used—the algorithm iteratively updates value estimates until it converges to the optimal solution.

Finally, Yuejie Chi highlighted that the system can compute its strategy in two ways: one agent maximizes while another minimizes the obtained rewards. In this way, the system involves two agents in an adversarial setting, combining information from both until reaching an equilibrium through an entropy-based mechanism.

Throughout the session, the researcher demonstrated how these systems operate mathematically.

The DEEC TALK also included a Q&A session with questions from the audience.