Paper Title: "Understanding the Role of Population Experiences in Proximal Distilled Evolutionary Reinforcement Learning"
Student Author:
• Nguyễn Thái Huy – 20520547 – KHMT2020 – Main Author
Supervising Professor:
• Dr. Lương Ngọc Hoàng
Abstract of the Paper:
Reinforcement learning combined with evolutionary computation, known as Evolutionary Reinforcement Learning (ERL), harnesses the effectiveness of Reinforcement Learning (RL) and the ability to explore new experiences from population-based search in Evolutionary Computation. These methods have shown promising results in continuous control tasks but often suffer from instability. Some studies indicate that experiences obtained from individuals within the population can lead to a shift in the state distribution during weight updates for the RL agent. Recently, a method has been proposed to mitigate this issue by segregating experiences into two separate memories, RL memory and population memory. Subsequently, when updating the RL agent, samples from these memories are mixed according to a predetermined ratio. The effectiveness of this approach has been demonstrated through experiments on the ERL algorithm using Evolution Strategies to support an external RL agent. However, there is currently no clear research on the impact of this method on ERL based on Genetic Algorithms (GA). In this study, we analyze the impact of the data obtained from the GA population on the RL agent and the influence of the sample mixing method on an ERL method, specifically Proximal Distilled Evolutionary Reinforcement Learning (PDERL).
I would like to express my sincere gratitude to Dr. Lương Ngọc Hoàng for his dedicated support, guidance, and for pointing out limitations during the research to help improve the content and make it more comprehensive.
SOICT (Symposium on Information and Communication Technology) is an international specialized conference that covers important research areas, including AI and Big Data Foundations, Communication Networks and Security, Image Processing and Natural Language, Software Engineering and Digital Technology, Blockchain, and other research trends. The conference aims to provide an academic forum for researchers and students to share the latest research findings and identify the challenges in computer science for the future. Additionally, the conference will feature speeches and guidance from world-leading speakers. The SOICT 2023 proceedings will be published and archived in the ACM Digital Library as part of the International Conference Proceedings Series (ICPS). SOICT 2023 will be indexed by DBLP, Ei Compendex, Scopus, and Clarivate Analytics Web of Science (ISI Web of Science).
Further information: https://www.facebook.com/UIT.Fanpage/posts/pfbid0mMFZS9jzqWew53EtCcRZXrRfayoPxpeewRUUmmSoMKUicHLfCeiYSKeqAbJF2PJUl
Written by: Ha Bang
Translated by: Ngoc Diem