GCRL-Collection

This repo relates to the survey paper <Goal-Conditioned Reinforcement Learning: Problems and Solutions>. We collects widely used benchmark environments and conclude a series of research works for goal-conditioned reinforcement learning (GCRL).

Summary of collected algorithms

Approach	Resolved Goal Type	Sub-Goal	Relabel	Optimization
UVFA	Vector/Image			✅
HER	Vector/Image		✅
Laplacian Representation Learning	Vector/Image			✅
SR	Vector/Image			✅
DDL	Vector/Image	✅		✅
SoRB	Vector/Image	✅
Skew-Fit	Vector	✅
HER with Demonstrations	Vector			✅
GoalGAN	Vector	✅
Sub-goal Discovery	Vector	✅
HGG	Vector	✅
CHER	Vector		✅
G-HER	Vector		✅
Hindsight Planner	Vector	✅
GDP	Vector		✅
VDS	Vector	✅
MEGA	Vector	✅	✅
PlanGAN	Vector			✅
AIM	Vector			✅
I-HER	Vector			✅
GCSL	Vector			✅
MapGo	Vector		✅	✅
L3P	Vector			✅
RIG	Image	✅
DISCERN	Image	✅
HVF	Image	✅
AMIGo	Image	✅
GAP	Image			✅
LEXA	Image			✅

Widely used benchmark environments.

Update: Most of these environments has been integrated in Gymnasium-Robotics.

Note almost all environments support various kinds of goal structures, i.e., vector, image, language, etc.

Environment	Benchmark Type	Common Used Goal Structure	Description
FetchReach	Gym Robotics/Mujoco	Vector/Image	A robotic arm reaches a target position
FetchPush	Gym Robotics/Mujoco	Vector/Image	A robotic arm pushes a block to a target position
FetchPickAndPlace	Gym Robotics/Mujoco	Vector/Image	A robotic arm first picks up the block then places it to a target position
FetchSlide	Gym Robotics/Mujoco	Vector/Image	A robotic arm slides a buck to a goal position
HandManipulateBlock	Gym Robotics/Mujoco	Vector/Image	Orient a block using a robot hand
HandManipulateEgg	Gym Robotics/Mujoco	Vector/Image	Orient an egg using a robot hand
HandManipulatePen	Gym Robotics/Mujoco	Vector/Image	Orient a pen using a robot hand
HalfCheetah	Mujoco	Vector/Image	Make a 2D cheetah robot run and keep a specific speed
AntLocomotion/Ant Maze	Mujoco	Vector	Make a 3D four-legged robot walk to a target position
Sawyer	Mujoco	Vector	A sawyer robot reaches a target position
Reacher	Mujoco	Vector/Image	A reacher robot reaches a target position
N-D Mass	Mujoco	Vector	A ball reaches a target position
DoorOpening	Mujoco	Image	A sawyer robot opens a door by latching onto the handle
Dy-Reaching	Mujoco	Image	A robotic arm reaches a target position moving along a straight line
Dy-Circling	Mujoco	Image	A robotic arm reaches a target position moving along a circle
Dy-Pushing	Mujoco	Image	A robotic arm moves the box to a moving target position
Dy-Pouring	Mujoco	Image	A robotic arm grips the can and pours the water into a cup
WaterMaze	DeepMind Lab	Image	Get to a target position described by an image
Seaquest	Atari	Image	Get to a target position described by an image
Montezuma’s Revenge	Atari	Image	Get to a target position described by an image
Reacher	DeepMind Control Suit	Image	A two-link planar reacher poses an aimed shape
Manipulator	DeepMind Control Suit	Image	A 3-DoF finger robot rotates the body into a target orientation
Finger	DeepMind Control Suit	Image	A planar manipulator brings an object in a target location
Place Object in Tray	PyBullet	Image	A robotic arm puts toys into a tray
Opening Drawer	PyBullet	Image	A robotic arm opens the drawer
PickandPlace	PyBullet	Image	A robotic arm first picks up the block then places it to a target position
Five Object Manipulation	CLEVR-Robot/Mujoco	Image/Language	A point mass agent arranges, orders and sorts 5 fixed objects
Diverse Object Manipulation	CLEVR-Robot/Mujoco	Image/Language	A point mass agent orders objects in different shapes and colors
Playground	Playground	Image/Language	A 2D hand moves and grasps/releases different objects
VizDoom	ViZDoom	Image/Language	AI bots reach specific place in the first person shooter 3D game Doom