Building LLM-Based Agents: From Toolbox to Autonomous Architect

Valuable Content and Insights from Anthropic’s Guide on Agents

Recently, Anthropic published a blog post about how to efficiently build agent systems, sharing their experiences in the process of building LLM-based agents.

This article mainly presents the valuable content, vivid metaphors and insights I have gained from it.

Workflows and Agents

Workflows are systems that orchestrate LLMs and tools through predefined code paths.

Agents are systems where LLMs dynamically guide their own processes and tool usage, maintaining control over how they complete tasks.

Workflows provide predictability and consistency for well-defined tasks, while agents are better suited when tasks require flexibility and model-driven decision-making at scale.

From this we can see that workflows emphasize predefined paths, while agents emphasize autonomous control. This reminds me that many so-called agentic RAG designs are actually workflows, not agents.

Trade-offs

When building LLM applications, Anthropic recommends finding the simplest viable solution and adding complexity only when necessary. For many applications, optimizing a single LLM call with retrieval and in-context examples is sufficient.

Building more complex systems means we'll need to make a tradeoff - they may work better, but they'll also be slower and cost more to run. Before adding complexity, we should think carefully about whether these downsides are worth it.

Whether to Use Frameworks

Many frameworks, like LangGraph and others, abstract standard low-level tasks such as calling LLMs, defining and parsing tools, and chaining calls together.

These frameworks undoubtedly make it easier to get started. However, these extra layers of abstraction can make debugging difficult. They can also tempt developers to add complexity when a simpler setup would suffice.

Therefore, developers should begin with direct LLM API calls, as many patterns require only a few lines of code. If you do need to use a framework, make sure you understand the underlying code. Incorrect assumptions about the underlying implementation are a common source of customer errors.

Development

Anthropic outlined a development roadmap for building effective agents that moves from simple to complex. It starts with augmented LLMs as foundational building blocks, then gradually adds complexity as needed, progressing from basic workflows to autonomous agent systems:

1. Augmented LLMs (Simple Toolbox).

2. Workflows: Prompt Chaining (Puzzle Pieces), Routing (Navigator), Parallelization (Coordinated Dance), Orchestrator-workers (Orchestrator and Orchestra), Evaluator-optimizer (Refining Sculptor).

3. Autonomous Agents (Autonomous Explorer) capable of independently planning and executing complex tasks.

Simple Toolbox (The Augmented LLM)

Vivid metaphor: Imagine you’re putting together a versatile toolbox. At the heart of it is a strong, reliable tool (the LLM), but you’ve added extras like a search engine, memory, and external APIs. These additions make the toolbox more effective, but the key is in organizing them well so each piece works smoothly together.

Agentic systems are built on LLMs augmented with retrieval, tools, and memory capabilities. These models can independently generate searches, use tools, and manage information storage.