Defining and Solving Multi-agent Environments

Description

Reinforcement learning has had success in solving challenging problems, ranging from games to robotics. However most of the success have been performed in single-agent environments, while real-world problems are mostly based on interactions between multiple agents. This challenge aims to explore approaches to defining and building environments that are based on multi-agent interactions. These are based on but not limited to cooperative communication, cooperative navigation, adversarial navigation and many others. Participants would also need to apply an algorithm to solve the defined environments.

Challenge Rules

• Participants may work individually or in teams (up to 3 people in a team)
• Participants must implement at least one environment with more than one agent
• Entries must represent the original work (any references have to be mentioned)
• At least one team member must present the work during the Challenge presentation

Guidelines

A strong entry will have one or more of these components:

• Employ multiple multi-agent environments with various scenarios and different complexity
• Include visualization or/and animation
• Applying different types of algorithms to solve the environments

Abstract

Abstract is max 2 pages includes: title, team members, preliminary results, visualizations if applicable, approach description.

Final Report

A 10-page paper describing the project, following NIPS template. This includes references as well. Final report should talk about two main parts:

• Environments
• Algorithms

Below are suggested but not limited questions to talk about in your submission:

Environment

• What kind of multiagent environments have been created?
• Talk about environment definitions, what strategies have been used to define the reward function?
• What approaches used to define the environments (include references if it is based on exciting environments)?
• What are the real-world applications of proposed environments?

Algorithms

• What types of algorithms have been used?
• Results explanation
• Is there any generic algorithm that can solve all problems?
• Talk about implementing the algorithms and hyper-parameter tuning.
• What is the novel contribution?

References

• NIPS Styles (docx, tex)
• Lowe, Ryan, Yi Wu, Aviv Tamar, Jean Harb, OpenAI Pieter Abbeel, and Igor Mordatch. "Multi-agent actor-critic for mixed cooperative-competitive environments." In Advances in Neural Information Processing Systems, pp. 6379-6390. 2017. (Read)
• GitHub Multi-Agent Particle Environment

Evaluation

Evaluation parameters includes:

• The range and complexity of multi-agent scenarios implemented
• The range of applied algorithms to solve proposed environments
• Clear explanation and visualization of results
• A novelty on creating multi-agent environments
• A novelty in algorithms

Updates

Congratulations to the teams that reached the Final Round of the challenge! The final presentation will be held on June 27 (Thursday) @3pm. How to prepare:

1. Preliminar report (max 10 pages). This report should contain all the details of your implementation and solution that you used to solve the environment, along with some background information. It should be done in NIPS format.
   Deadline: June 26, 11:59pm
2. Presentation slides should be sent to avereshc[at]buffalo.edu by 12:00pm July 27 .
3. Final write-up (max 10 pages). An improved version of your Preliminary report, with some sections added to it, if neccesary (e.g. Background, Methodology, etc). Selected write-ups will be published online on the challenge website Deadline: July 12

Presentation Details

• Presentation lengths: 15 mins
• UB branded ppt templates

Presentation Structure (recommended):

• Challenge Title / Team / Team's members / Date [1 slide]
• Background [max 2 slides]
• Environment Description (scenario, how many agents, their goal, possible states/actions, reward, etc) [max 3 slides]
• Environment Implementation (show your environment - ideally some images or snapshots) [max 2 slides]
• Solving the environment (what algorithm do you apply to solve, why do you choose it) [max 2 slides]
• Solving the environment Results (graphs, any visuals) [max 4 slides]
• Key Observations / Summary [1 slide]
• Thank you Page [1 slide]

Who is Eligible

UB students or alumnies from any department with an interest in reinforcement learning. Entries are accepted from single individuals or teams (up to 3 people).

Contact Details

Alina Vereshchaka - avereshc[at]buffalo.edu

Important Dates

• June 12 - Register your team and submit the abstract
• June 14 - Teams notifications
• June 26 - Submit final report
• June 27 - Challenge presentations & Results

Results:

This event was supported by the Department of Computer Science and Engenering and our judging commetee:

• Dr. Sargur N. Srihari
• Dr. Wen Dong
• Dr. Kenny Joseph

Congratulations to all the participants of the challenge on getting great results. Our winning teams:

1. First Place - Team of Nathan Margaglio
2. Second Place - Team of Anurag Saykar
3. Third Place - Team of Nitin Nataraj and Priyanka Pai

Selected works:

• Nazerke Sandibay. "Control of 2-degree of freedom robot using Advantage-Actor-Critic method", State University of New York at Buffalo, USA, 2019 (pdf)
• Nitin Nataraj, and Priyanka Pai. "Collaborative Drawing with Multi-Agent Reinforcement Learning", State University of New York at Buffalo, USA, 2019 (pdf)
• Zhi Wen Huang, and Weijin Zhu. "Tic-Tac-Toe with Deep Multi-Agent Reinforcement Learning", State University of New York at Buffalo, USA, 2019 (pdf)

Photos from the event

All images can be downloaded from Google Drive