UB Reinforcement Learning Challenge 2019

Defining and Solving Multi-agent Environments


Reinforcement learning has had success in solving challenging problems, ranging from games to robotics. However most of the success have been performed in single-agent environments, while real-world problems are mostly based on interactions between multiple agents. This challenge aims to explore approaches to defining and building environments that are based on multi-agent interactions. These are based on but not limited to cooperative communication, cooperative navigation, adversarial navigation and many others. Participants would also need to apply an algorithm to solve the defined environments.

Challenge Rules

  • Participants may work individually or in teams (up to 3 people in a team)
  • Participants must implement at least one environment with more than one agent
  • Entries must represent the original work (any references have to be mentioned)
  • At least one team member must present the work during the Challenge presentation


A strong entry will have one or more of these components:
  • Employ multiple multi-agent environments with various scenarios and different complexity
  • Include visualization or/and animation
  • Applying different types of algorithms to solve the environments


Abstract is max 2 pages includes: title, team members, preliminary results, visualizations if applicable, approach description.

Final Report

A 10-page paper describing the project, following NIPS template. This includes references as well. Final report should talk about two main parts:

  • Environments
  • Algorithms

Below are suggested but not limited questions to talk about in your submission:


  • What kind of multiagent environments have been created?
  • Talk about environment definitions, what strategies have been used to define the reward function?
  • What approaches used to define the environments (include references if it is based on exciting environments)?
  • What are the real-world applications of proposed environments?


  • What types of algorithms have been used?
  • Results explanation
  • Is there any generic algorithm that can solve all problems?
  • Talk about implementing the algorithms and hyper-parameter tuning.
  • What is the novel contribution?



Evaluation parameters includes:
  • The range and complexity of multi-agent scenarios implemented
  • The range of applied algorithms to solve proposed environments
  • Clear explanation and visualization of results
  • A novelty on creating multi-agent environments
  • A novelty in algorithms


Congratulations to the teams that reached the Final Round of the challenge! The final presentation will be held on June 27 (Thursday) @3pm. How to prepare:
  1. Preliminar report (max 10 pages). This report should contain all the details of your implementation and solution that you used to solve the environment, along with some background information. It should be done in NIPS format.
    Deadline: June 26, 11:59pm
  2. Presentation slides should be sent to avereshc[at]buffalo.edu by 12:00pm July 27 .
  3. Final write-up (max 10 pages). An improved version of your Preliminary report, with some sections added to it, if neccesary (e.g. Background, Methodology, etc). Selected write-ups will be published online on the challenge website Deadline: July 12

Presentation Details

Presentation Structure (recommended):
  • Challenge Title / Team / Team's members / Date [1 slide]
  • Background [max 2 slides]
  • Environment Description (scenario, how many agents, their goal, possible states/actions, reward, etc) [max 3 slides]
  • Environment Implementation (show your environment - ideally some images or snapshots) [max 2 slides]
  • Solving the environment (what algorithm do you apply to solve, why do you choose it) [max 2 slides]
  • Solving the environment Results (graphs, any visuals) [max 4 slides]
  • Key Observations / Summary [1 slide]
  • Thank you Page [1 slide]

Who is Eligible

UB students or alumnies from any department with an interest in reinforcement learning. Entries are accepted from single individuals or teams (up to 3 people).

Contact Details

Alina Vereshchaka - avereshc[at]buffalo.edu

Important Dates

  • June 12 - Register your team and submit the abstract
  • June 14 - Teams notifications
  • June 26 - Submit final report
  • June 27 - Challenge presentations & Results


This event was supported by the Department of Computer Science and Engenering and our judging commetee: Congratulations to all the participants of the challenge on getting great results. Our winning teams:
  1. First Place - Team of Nathan Margaglio
  2. Second Place - Team of Anurag Saykar
  3. Third Place - Team of Nitin Nataraj and Priyanka Pai

Selected works:

  • Nazerke Sandibay. "Control of 2-degree of freedom robot using Advantage-Actor-Critic method", State University of New York at Buffalo, USA, 2019 (pdf)
  • Nitin Nataraj, and Priyanka Pai. "Collaborative Drawing with Multi-Agent Reinforcement Learning", State University of New York at Buffalo, USA, 2019 (pdf)
  • Zhi Wen Huang, and Weijin Zhu. "Tic-Tac-Toe with Deep Multi-Agent Reinforcement Learning", State University of New York at Buffalo, USA, 2019 (pdf)

Photos from the event

UB Reinforcement Learning Challenge 2019
Judges, organizer and participants of the UB Reinforcement Learning Challenge 2019 and CSE4/510 Reinforcement Learning
Nazerke Sandibay
Nazerke Sandibay (finalist of the UB Reinforcement Learning Challenge), summer exchange student from the Nazarbayev University, Kazakhstan
Zhi Wen Huang
Zhi Wen Huang (finalist of the UB Reinforcement Learning Challenge)
Weijin Zhu
Weijin Zhu (finalist of the UB Reinforcement Learning Challenge)
 Nitin Nataraj and Priyanka Pai
Dr. Wen Dong, Nitin Nataraj (third place winner), Priyanka Pai (third place winner), Alina Vereshchaka (organizer), Dr. Kenny Joseph, Dr. Sargur Srihari
Anurag Saykar
Dr. Wen Dong, Anurag Saykar (second place winner), Alina Vereshchaka (organizer), Dr. Kenny Joseph, Dr. Sargur Srihari
Nathan Margaglio
Dr. Wen Dong, Nathan Margaglio (first place winner), Alina Vereshchaka (organizer), Dr. Kenny Joseph, Dr. Sargur Srihari
All images can be downloaded from Google Drive

We are grateful to the Computer Science and Engeneering Department for their support in organizing the event, our judges for their insights and all the participants for showing their impressive works. Looking forward to future events!