An Analysis of Generative AI Agents in Automatic Task Completion in Interactive Web Environments

Abstract

Autonomous AI agents are software programs that can independently complete multi-step tasks given a prompt (e.g. “Tell me the full address of the nearest airport to George Mason University”). They are expected to complete mundane daily tasks that do not require human input in the near future. Autonomous agents have become more sophisticated in recent years owing to advancements in machine learning and natural language processing. However, these agents are still susceptible to mistakes in comprehension, reasoning, and action execution. WebArena is a standalone web environment that can host autonomous agents. In this project, we conducted an analysis of WebArena and its provided agent using the procedural guidelines and task prompts provided by the WebArena team. We ran WebArena using 100 unique prompts ranging from shopping and store management to mapping and navigation. We categorized errors and discovered which errors occurred most frequently. The autonomous agent successfully completed the task in 23% of runs. Many runs suffered from an incorrect, missing, or unnecessary action. Common reasons for errors included a lack of world knowledge or human-like visualization, ignoring part of a prompt, and hallucinations. In the future, training should ensure that autonomous agents have sufficient world knowledge and experience to navigate the web successfully. Developers can also add weights to prompts so that agents can focus more on relevant criteria and introduce memory so that agents can learn from past actions. Hopefully, with more time and research, autonomous agents will soon improve the lives of many.