Aprendizagem por demonstração de planos coordenados em sistemas multiagentes

Abstract

One of the great challenges in Multiagent Systems (MAS) is the creation of cooperative plans to deal with the different scenarios that present themselves in a dynamic, real-time environment composed of teams of mobile robots. In this scenario, an agent of the MAS controls each robot, which needs to make complex decisions in a short time in a coordinated manner with the other robots on its team. Despite the many solutions developed based on multi-agent planning and reinforcement learning, a human observer usually sees opportunities for better cooperative plans in many scenarios where robots underperform. The research presented in this thesis consists of capturing the human spectator's knowledge to demonstrate how robot teams can better cooperate in solving the problem they must solve. The human watcher can indicate the situations in which a cooperative plan can better solve a given problem by watching the performance of a team of robots in action. Consequently, a dataset for training the agents that control the robots can gather the various human observations. For the development of this research, this work used the environment RoboCup 3D Soccer Simulation (3DSSIM) and the collection of human demonstrations was carried out through a set of tools developed from the adaptation of existing solutions in the RoboCup community using a strategy of crowdsourcing. In addition, fuzzy clustering was used to gather demonstrations with the same semantic meaning, even with small differences. With the data organized, this thesis used a reinforcement learning mechanism to learn a classification policy that allows agents to decide which group of plans is best suited to each situation that presents itself in the environment. The results show the ability of the robot team to evolve, from the learning of the suggested plays and its use in an appropriate way to the abilities of each robot.