Multi-Agent Reinforcement Learning
I’d like to learn more about the interplay between Reinforcement Learning and Multi-Agent Systems. Can you suggest some study resources such as books and scientific articles from where I can start learning?
Multi-agent reinforcement learning (MARL) is a hot topic. This is because in the future, multiple agents are more likely to be able to solve a problem faster and better than they could alone. But the problem is that it makes the problem highly non-stationary. I.e. one agent is trying to find an optimal policy whilst another is physically altering the environment.
It’s a fascinating subject and there are basically two angles you can approach it from. 1) Game theory or 2) RL. If you go down the game theory route, that is a lot more formal, has more research and might suit those with mathematical or computer science wizards. But RL is more practical, more focused towards practical industrial applications, but less formal, and suits people that prefer hacking.
So if you like the math, study the game theory. If you like the practical aspects, go down the RL route. Then move onto MARL.
Books: My book of course! 😊 Here are the links for further reading in my book, specifically related to MARL:
- Schwartz, H. M. 2014. Multi-Agent Machine Learning: A Reinforcement Approach. John Wiley & Sons.
- A great RL-focused recent review from Oroojlooy and Hajinezhad.^[OroojlooyJadid, Afshin, and Davood Hajinezhad. 2020. ‘A Review of Cooperative Multi-Agent Deep Reinforcement Learning’. ArXiv:1908.03963, June. http://arxiv.org/abs/1908.03963.]
- Another review that spans games and EFGs from Zhang et al..^[Zhang, Kaiqing, Zhuoran Yang, and Tamer Başar. 2019. ‘Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms’. ArXiv:1911.10635, November. http://arxiv.org/abs/1911.10635.]
- Decentralized MARL:
- A great introduction to decentralized MARL from Christopher Amato
- And accompanying book by Oliehoek and Amato.^[Oliehoek, Frans A., and Christopher Amato. 2016. A Concise Introduction to Decentralized POMDPs. 1st ed. Springer Publishing Company, Incorporated.]