About the subgoals of Alfworld dataset

[jnanliu]

Hi, thanks for the interesting work. I'm curious about the subgoals in the test set of alfworld, how to generate them?

[sunnynexus]

Hello, thanks for your interest!

For ALFWorld, we use the subgoal annotations provided by AgentBoard (Hugging Face dataset: https://huggingface.co/datasets/hkust-nlp/agentboard).

In AgentBoard, subgoals are generated by defining a set of necessary intermediate milestones and detecting their completion via pattern/regex matching on the environment’s observation text (with the final subgoal determined by the environment’s success flag).