Three things every AI agent engineer should know: agent architecture, prompt engineering, and MCP

This guide covers three pieces of foundational knowledge for building reliable AI applications: agent architecture, prompt engineering, and the Model Context Protocol (MCP). None of it belongs to any one model vendor. It applies whether you use OpenAI, Anthropic or Gemini, and whether you build on LangChain, LlamaIndex or plain SDK calls. Learn the principles and you won’t be held hostage by a particular tool.

Part 1: Agent architecture

An agent, at its core, is a loop: let the LLM think, call a tool, look at the result, and think again, until the task is done. Understanding the structure of that loop is the first step to building anything reliable.

1.1 The core mechanism: the agentic loop

A traditional program calls an LLM once: send a request, get an answer, done. An agent is iterative. Each model response might be “I need to call a tool”; the program runs the tool, feeds the result back, and the next round begins.

  Your program                    LLM

  send messages ────────────────▶  thinking...
  (with tool definitions)          ↓
                                  stop_reason = "tool_use"
  ◀────── tool_call returned ────  "I need to call search_web"
  ↓
  execute the tool (real code runs here)
  ↓
  append result ────────────────▶  thinking...
                                  ↓
                                  stop_reason = "end_turn"
  ◀────── final answer ──────────  "Based on the search results..."

  The loop only stops when stop_reason == "end_turn"

The control flow, in pseudocode that fits any LLM SDK:

def run_agent(messages, tools):
    while True:
        response = llm.complete(messages=messages, tools=tools)

        # Task complete: exit the loop
        if response.stop_reason == "end_turn":
            return response.content

        # Tool call requested: keep looping
        if response.stop_reason == "tool_use":
            tool_results = []
            for tool_call in response.tool_calls:
                result = execute_tool(tool_call.name, tool_call.input)
                tool_results.append({
                    "tool_call_id": tool_call.id,
                    "content": result
                })

            # Append the tool results to the conversation history
            messages.append(response)        # assistant turn
            messages.append(tool_results)    # tool results
            # next iteration

Common trap: don’t decide whether the task is finished by parsing the text, for example by checking whether the reply contains the words “task complete”. Rely on stop_reason. LLM output is probabilistic, and text parsing fails at random.

1.2 Multi-agent orchestration: dividing the work

When a task gets too complex, one agent can’t handle it. Not because it isn’t “smart enough”, but because the context window is finite: doing too many things at once scatters attention and raises the error rate. The fix is to split the work across several agents.

The classic pattern is hub and spoke:

                    ┌──────────────┐
                    │  Coordinator │  ← the lead agent: decomposes, dispatches, aggregates
                    └──────┬───────┘
          ┌────────────────┼────────────────┐
          ▼                ▼                ▼
   ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
   │  SearchAgent │ │ AnalyzeAgent │ │  WriteAgent  │
   │  (web search)│ │  (documents) │ │  (the report)│
   └──────────────┘ └──────────────┘ └──────────────┘

  Sub-agents never talk to each other; everything routes through the Coordinator.
  Why: observability, uniform error handling, controlled information flow.

Key principle: sub-agents have no memory. Each sub-agent knows only what you pass it in this call. The parent agent’s conversation history is not inherited. You have to put the necessary context into the sub-agent’s prompt explicitly.

# Wrong: the sub-agent has no context at all
result = spawn_agent(
    agent="WriteAgent",
    prompt="Write a report based on the research findings"   # which findings?
)

# Right: pass every piece of context it needs
result = spawn_agent(
    agent="WriteAgent",
    prompt=f"""Write a report on "the impact of AI on the creative industries".

Research findings collected so far:
{search_results}     # output of the search agent

Summaries of the key sources:
{analysis_results}   # output of the analysis agent

Requirements: ..."""
)

Parallel or sequential depends on whether the tasks depend on each other:

Situation	Run	Why
Searching several unrelated topics	In parallel	Independent tasks; parallelism cuts total latency
Search, then analyse	Sequentially	Analysis depends on the search results
Checking many files	In parallel	Files are independent; processing them together is fastest
Authenticate, then act	Sequentially	Security-critical; order must be guaranteed

1.3 Passing context: how information flows

Context is the LLM’s working memory. Whatever it can’t see doesn’t exist. Managing context well is the central hard problem of agent engineering. Four common strategies:

Append-only history: every turn is appended. Simple, but tokens and cost keep growing.
Progressive compression: periodically summarise older history. Saves tokens, but numbers and dates get lost in summaries.
Key-fact extraction: pull important data (order IDs, amounts, timestamps) into a structured block that stays at the top of the context.
Tool-output trimming: an API returns 40 fields and 5 are useful. Left untrimmed, the rest wastes tokens and distracts the model.

There is also the “lost in the middle” problem. Research shows that LLMs attend most strongly to the start and end of the context, and the middle is easily ignored. Structure prompts accordingly:

system_prompt = """
[The most important instructions and rules]   ← first, so they are never forgotten

--- Background ---
{background_context}                          ← middle (relatively less important)

--- Key facts (always kept) ---
Customer ID: {customer_id}                    ← key data near the end
Order: {order_details}

--- Current task ---
{current_task}                                ← last, freshest in the model's attention
"""

1.4 Error propagation: what happens when things fail

Real tool calls fail: network timeouts, missing permissions, missing data. How you design error handling decides whether an agent system is actually reliable. Classify first:

Type	Example	Correct handling
Transient	Timeout, service temporarily unavailable	Retry locally with exponential backoff; escalate only if retries fail
Validation	Bad input format, missing required field	Don’t retry; report immediately with the reason
Business	Insufficient balance, out of scope	Don’t retry; return a user-friendly explanation
Permission	Unauthorised access	Don’t retry; may need escalation to a human

Tools should return structured errors, independent of any framework:

def search_customer(customer_id):
    try:
        result = db.query(customer_id)
        return { "success": True, "data": result }
    except TimeoutError:
        return {
            "success": False,
            "error_type": "transient",
            "is_retryable": True,
            "message": "Database connection timed out, please retry later"
        }
    except PermissionError:
        return {
            "success": False,
            "error_type": "permission",
            "is_retryable": False,
            "message": "No access to this customer record; needs manual review"
        }

Design principle: a sub-agent should first try to recover locally (for example, auto-retry transient errors) and only escalate to the Coordinator when it can’t. The escalation must include the failure type, what was already attempted, and any partial results. That is what lets the Coordinator make a sensible recovery decision.

Three anti-patterns that make systems impossible to debug:

Swallowing errors silently: the tool fails but returns an empty list as if it succeeded. Downstream agents assume there is no data and decide wrongly.
Aborting the whole flow on any error: one sub-agent’s search times out and the entire research task dies. Continue with partial results instead, and mark the gaps in the final output.
Vague error messages: "operation failed" tells the LLM nothing. It can’t tell whether to retry, change strategy, or escalate.

Part 2: Prompt engineering principles

Prompt engineering is not magic; it is engineering practice with rules. This part covers three core techniques: few-shot prompting so the model generalises from examples, structured output so programs can parse the result, and methods for cutting false positives.

2.1 Few-shot prompting: examples instead of instructions

When you describe a rule in words, the model can interpret it several ways. Give it two to four concrete input/output examples and it learns the pattern itself; ambiguity drops sharply.

Why it works: during pre-training an LLM has seen countless “example, pattern, application” structures, so it is naturally good at inducing a rule from examples and extending it to new cases. Concrete examples carry more information per token than abstract descriptions.

Instruction only (unstable output):

SYSTEM: Analyse the code review findings. Use a professional format, group by severity, and suggest fixes.

OUTPUT: This code has a few issues worth noting. First, the SQL query has a security risk… (sometimes a list, sometimes paragraphs, format varies)

Few-shot (stable format):

Analyse the code. Follow this output format exactly:

Example input: query = "SELECT * FROM users WHERE id = " + user_input
Example output:
🔴 [HIGH] SQL injection
Location: line 3
Problem: user input concatenated directly into the SQL statement
Fix: use a parameterised query, WHERE id = ?

---

Example input: def process(): pass  # TODO: implement
Example output:
🟡 [MEDIUM] Unfinished implementation
Location: line 1
Problem: empty function body with a TODO comment
Fix: implement the logic or remove the placeholder

Few-shot best practices:

Two to four examples. Fewer and the pattern is unstable; more and you burn tokens for diminishing returns.
Cover the edges. Show not just the normal case but also “this doesn’t need reporting” and “this is a false positive”.
Include negative examples. Tell the model which things look like problems but are normal. This is the most direct way to reduce false positives.
The format of the examples is the format constraint. Whatever format you show, the model imitates. No extra formatting instructions needed.

2.2 Structured output: making LLM output parseable

By default an LLM produces natural language. In an automated pipeline you need JSON or another structure your code can parse; a program cannot “understand” prose. Three methods, in increasing order of reliability.

Method A: prompt constraints. Simplest, not reliable enough.

# Ask the model to output only JSON, with no guarantee
prompt = """Output the analysis as JSON:
{"severity": "high/medium/low", "issue": "...", "fix": "..."}
Output only JSON, nothing else."""

# Problem: the model may wrap it in ```json ... ``` or add a sentence first.
# You need cleanup code, and there is still a small chance of malformed output.

Method B: JSON Schema constraints. More reliable.

# Most modern LLM APIs support a response_format parameter
response = llm.complete(
    prompt="Analyse this code for security issues",
    response_format={
        "type": "json_schema",
        "schema": {
            "type": "object",
            "properties": {
                "severity": {"type": "string", "enum": ["high", "medium", "low"]},
                "issues": {"type": "array", "items": {"type": "string"}},
                "explanation": {"type": "string"}
            },
            "required": ["severity", "issues"]
        }
    }
)
# The output is guaranteed to match the schema syntactically; semantics can still be wrong

Method C: forced structure via tool use. Most reliable. Disguise “extract structured data” as a tool call. The model then generates JSON that matches the tool’s parameter schema directly, rather than writing prose to be converted later.

# Define an "extraction tool" that exists only to force structured output
tools = [{
    "name": "extract_code_issues",
    "description": "Extract security and quality issues found in the code",
    "parameters": {
        "type": "object",
        "properties": {
            "issues": {
                "type": "array",
                "items": {
                    "type": "object",
                    "properties": {
                        "severity": {"enum": ["high", "medium", "low"]},
                        "line": {"type": "integer"},
                        "description": {"type": "string"},
                        "fix": {"type": "string"}
                    },
                    "required": ["severity", "description"]
                }
            }
        }
    }
}]

response = llm.complete(
    prompt="Analyse this code",
    tools=tools,
    tool_choice="any"  # force a tool call
)
# Read the data from the tool_call arguments; the format is always correct

Schema design tips:

Situation	Design	Why
The information may not exist in the document	Make the field nullable (optional)	Otherwise the model invents data to satisfy `required`
Fixed categories, but new ones may appear	Add “other” to the enum plus a description field	Stops the model forcing items into an ill-fitting known category
Numbers need consistency checks	Extract both a `calculated` and a `stated` field	Comparing them catches semantic errors, such as line items not summing to the total

2.3 Controlling false positives: raising precision

In code review, security scanning and content moderation, a false positive does more damage than a miss. Users lose trust in the system and end up ignoring every warning, including the ones that matter. Three root causes:

Vague criteria: instructions like “be conservative” or “only report high confidence” are too abstract to execute.
Too broad: “check for all potential issues” makes the model report every uncertain case.
No negative examples: with nothing showing what is acceptable, the model has no reference point.

The fix is concrete criteria instead of vague instructions.

Vague (high false-positive rate):

Check the code for problems. Be conservative, only report high-confidence findings, avoid false positives.

Concrete (low false-positive rate):

Report the following (explicitly in scope):
• Code behaviour that contradicts its comments
• Unhandled error paths that lose data
• SQL / command injection risks

Do not report the following (explicitly out of scope):
• Style preferences (indentation, naming)
• Patterns that are project conventions, even if unusual
• Redundant code that doesn't affect behaviour

Severity definitions:
HIGH: an exploitable security flaw, e.g. eval(user_input)
MEDIUM: an error path that corrupts data, e.g. a batch write without a transaction
LOW: an improvement that doesn't affect correctness

The validation-retry loop. For structured extraction, the model sometimes produces output with format problems. The right move is to feed the errors back and let it correct itself, rather than failing outright:

def extract_with_retry(document, max_retries=2):
    messages = [{"role": "user", "content": document}]

    for attempt in range(max_retries):
        result = llm.complete(messages, tools=[extract_tool])
        extracted = result.tool_calls[0].arguments

        errors = validate(extracted)
        if not errors:
            return extracted   # validation passed

        # Validation failed: feed the errors back to the model
        messages.append({"role": "assistant", "content": result})
        messages.append({
            "role": "user",
            "content": f"The extraction has these problems, please fix them:\n{errors}"
        })

    raise ExtractionError("Could not extract a valid result after retries")

When retrying doesn’t help: retries only fix format errors, where the model knows the information but structured it wrongly. If the information simply isn’t in the source document, no number of retries will produce it. Make the field nullable and let the model return null instead of inventing something.

Part 3: The Model Context Protocol

MCP (Model Context Protocol) is an open standard that lets AI models call external tools and data sources. Just as USB unified peripheral connectors, MCP tries to unify how AI models connect to the outside world: integrate once, and every model can use it.

3.1 What MCP is: the “USB port” for AI

Before MCP, every AI service called tools its own way, and tool developers had to adapt to each platform separately.

[Without MCP]

  OpenAI    ──custom interface──▶  your database tool (OpenAI version)
  Anthropic ──custom interface──▶  your database tool (Anthropic version)
  Gemini    ──custom interface──▶  your database tool (Gemini version)

  One tool, N copies to maintain

[With MCP]

  OpenAI    ─┐
  Anthropic ─┼──── MCP standard ────▶  your database tool (one copy)
  Gemini    ─┘

  Write once; every MCP-capable model can use it

MCP is an open protocol released by Anthropic in 2024 and is supported by many AI tools and platforms, including editors such as Cursor, Windsurf and Zed, and a range of AI assistants.

3.2 The three layers of MCP

  ┌──────────────────────┐
  │   MCP Host           │  ← the AI application itself (Claude Desktop, Cursor...)
  │   the model runs here│    initiates connections, routes requests
  └──────────┬───────────┘
             │ MCP protocol (JSON-RPC over stdio / HTTP)
  ┌──────────┴───────────┐
  │   MCP Client         │  ← built into the Host; manages the connection to a Server
  └──────────┬───────────┘
             │
     ┌───────┴─────────────────┐
     ▼                         ▼
  ┌──────────────┐      ┌──────────────┐
  │ MCP Server A │      │ MCP Server B │
  │ (filesystem) │      │ (database)   │
  │ • Tools      │      │ • Tools      │
  │ • Resources  │      │ • Resources  │
  │ • Prompts    │      │ • Prompts    │
  └──────────────┘      └──────────────┘

Each MCP server exposes three kinds of capability:

Capability	Role	Examples
Tools	Functions the model can call, with inputs, outputs and side effects	`create_file`, `send_email`, `run_sql`
Resources	Read-only content for the model to consult, no side effects	Document contents, a database schema, config files
Prompts	Predefined prompt templates, optionally parameterised	“code review template”, “weekly report template”

3.3 The wire protocol: MCP message format

MCP is built on JSON-RPC 2.0. Transport is either stdio (standard input/output, for local processes) or HTTP + SSE (for remote services). The handshake:

  Client                                     Server

  initialize ─────────────────────────────▶
  (declares the protocol version it supports)
                                           checks version compatibility
                         ◀─────────────── initialized
                                           (returns its capability list)
  notifications/initialized ──────────────▶
  (confirms the handshake)

  ═══════════════ normal traffic begins ════════════════

  tools/list ─────────────────────────────▶
                         ◀─────────────── [tool list + schemas]

  tools/call ─────────────────────────────▶
  {name: "search_db", arguments: {...}}
                         ◀─────────────── {content: [...], isError: false}

A tool call looks like this on the wire:

// Request: call a tool
{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "search_customer",
    "arguments": { "customer_id": "C-12345" }
  }
}

// Response: success
{
  "jsonrpc": "2.0",
  "id": 1,
  "result": {
    "content": [
      { "type": "text", "text": "{\"name\": \"Zhang San\", \"email\": \"zhang@example.com\"}" }
    ],
    "isError": false
  }
}

// Response: failure (note the isError flag)
{
  "result": {
    "content": [{ "type": "text", "text": "Customer not found" }],
    "isError": true
  }
}

Failure is expressed through the isError flag rather than an HTTP error, so the model can see that the call failed and decide what to do next.

3.4 Implementing an MCP server

The smallest useful MCP server, in Python, showing the core concepts:

# pip install mcp
from mcp.server import Server
from mcp.types import Tool, TextContent

app = Server("my-database-server")

# ① Declare the tool list (the model queries this on connect)
@app.list_tools()
async def list_tools():
    return [
        Tool(
            name="search_customer",
            # The description matters enormously: it is how the model decides when to call this tool
            description="""Search the customer database for a customer record.
Accepts a customer ID (e.g. C-12345) or an email address.
Returns: name, email, registration date, account status.
Note: this tool is read-only and never modifies data.""",
            inputSchema={
                "type": "object",
                "properties": {
                    "query": {"type": "string", "description": "Customer ID or email address"}
                },
                "required": ["query"]
            }
        )
    ]

# ② Implement the tool
@app.call_tool()
async def call_tool(name, arguments):
    if name == "search_customer":
        try:
            result = query_database(arguments["query"])
            return [TextContent(type="text", text=json.dumps(result))]
        except Exception as e:
            # Return the error with isError=True instead of raising
            return [TextContent(type="text", text=str(e))], True

# ③ Start the server (stdio mode, suited to local tools)
if __name__ == "__main__":
    import mcp.server.stdio
    mcp.server.stdio.run(app)

The most overlooked detail is the tool description. It is the model’s main basis for deciding whether to call a tool. The more detailed the description and the clearer its boundaries, the better the model chooses. A tool described as “search customers” and a tool whose description spells out input format, return fields, applicable situations and edge cases behave completely differently in practice. Check descriptions against this list:

What does the tool do? One sentence on its purpose.
When should it be used? How it differs from similar tools.
What is the input format? Give an example, such as “accepts IDs in the form C-12345”.
What does it return? List the key fields and say when they can be empty.
What are the side effects? Read-only, or does it modify data?

3.5 When to use MCP and when to call an API directly

Situation	Recommendation	Why
A tool shared by several AI applications	MCP server	Write once, use everywhere; standardisation eases maintenance
A team’s internal business workflow	MCP server	Shareable with everyone on the team
A simple one-off script	Direct API call	Setting up MCP costs more than it returns
The external system already has a community MCP server	Use the community one	GitHub, Jira, Slack and others already have solid implementations

Practical advice: check first whether a community MCP server already exists (github.com/modelcontextprotocol/servers). Standard tools such as databases, Git, email and calendars almost all have one. Write your own only for business-specific cases.

3.6 MCP in four points

Open standard: built on JSON-RPC, tied to no AI vendor; in principle any model can support it.
Three capabilities: Tools (act), Resources (read), Prompts (templates) cover most integration needs.
Description-driven: the quality of a tool’s description directly decides whether the model calls it correctly. This is the most easily neglected point.
The isError flag: errors come back in the isError field rather than as HTTP status codes, so the AI can perceive and handle failure.

About this guide

This piece focuses on general engineering principles. Every concept and code structure applies to the major LLM platforms (OpenAI, Anthropic, Google Gemini) and to open-source frameworks such as LangChain and LlamaIndex. It was originally a static documentation site and was reworked into this article in September 2026. 中文版.