Original Paper: https://jykoh.com/search-agents
By: Jing Yu Koh, Stephen McAleer, Daniel Fried, Ruslan Salakhutdinov
Abstract:
Autonomous agents powered by language models (LMs) have demonstrated promise in their ability to perform decision-making tasks such as web automation. However, a fundamental challenge remains: LMs, primarily optimized for natural language understanding and generation, struggle with multi-step reasoning, planning, and using environmental feedback when attempting to solve realistic computer tasks. Towards addressing this, we propose an inference-time search algorithm for LM agents to explicitly perform exploration and multi-step planning in interactive web environments. Our approach is a form of best-first tree search that operates within the actual environment space, and is complementary with most existing state-of-the-art agents. It is the first tree search algorithm for LM agents that shows effectiveness on realistic web tasks. On the challenging VisualWebArena benchmark, applying our search algorithm on top of a GPT-4o agent yields a 39.7% relative increase in success rate compared to the same baseline without search, setting a state-of-the-art success rate of 26.4%. On WebArena, search also yields a 28.0% relative improvement over a baseline agent, setting a competitive success rate of 19.2%. Our experiments highlight the effectiveness of search for web agents, and we demonstrate that performance scales with increased test-time compute. We conduct a thorough analysis of our results to highlight improvements from search, limitations, and promising directions for future work.
Enhancing Web Automation with Tree Search for Language Model Agents
Introduction
Imagine an autonomous agent that can navigate through complex web environments, making decisions and executing tasks just like a human. While this might sound futuristic, recent advancements in language models (LMs) have brought us closer to this reality. However, these agents often struggle with multi-step reasoning and planning, crucial for solving intricate web tasks. Enter tree search—a method to enhance LM agents by enabling them to explore, plan, and make informed decisions. This blog post delves into a groundbreaking research paper that introduces an inference-time tree search algorithm for LM agents, demonstrating significant improvements in web automation tasks.
Key Methodologies
The research addresses a critical limitation of current LM agents: their inability to efficiently explore and plan in dynamic web environments. The proposed solution is a best-first tree search algorithm that operates directly within the environment space. Here’s a breakdown of the methodologies used: