Baseline experimental results with an LLM agent and OpenAI Stable Baselines 3 RL algorithms on our Extended Gridworlds

By Roland Pihlakas

I have implemented an LLM agent that is able to navigate in our extended multi-objective multi-agent gridworlds environment. Also published the baseline experimental results of the LLM agent and OpenAI Stable Baselines 3 RL algorithms in an update to the working paper.

Summary: The LLM agent performed notably better than the RL algorithms on the resource sharing benchmark. Yet, all baseline algorithms, including the LLM agent, have difficulty in properly handling the multi-objective homeostasis and diminishing returns benchmarks.

Example image of the current system, where all features are turned on simultaneously:

Elements and metrics can be configured flexibly for each given benchmark. Examples of configuration options are: observation and state space of agents, scoring dimensions, adding NPC agents, object types and their dynamics.

Publication (arXiv)
Agent training and benchmarking repository
Extended gridworlds - environment building framework repository
Zoo to Gym multiagent adapter repository