Blog

Agentic workflows: how AI agents plan, act, and recover from failure

Agentic workflows let a large language model plan, call tools, and verify results in a loop; learn how they work, why they fail, and how to evaluate them.

On WebArena, a benchmark of 812 realistic web tasks, the best GPT-4-based agent finished 14.41% of tasks end to end, while human participants finished 78.24% (Zhou et al., 2024). Agentic workflows are artificial intelligence (AI) systems in which a large language model (LLM) decides its own sequence of steps, calls external tools, and checks results until it reaches a goal. Most of the gap comes from planning, tool use, and verification, not from language fluency. This article explains how agentic workflows operate, where they break in production, and how engineering teams measure and improve their reliability with human-labeled data.

What are agentic workflows?

Agentic workflows are multi-step processes in which an AI model receives a goal, breaks it into sub-tasks, calls external tools such as search, databases, or code interpreters, checks each result, and revises its plan until a defined success criterion is met. Unlike fixed automation scripts, the model decides the sequence of steps at run time.

They build on two reasoning techniques. Chain-of-thought prompting breaks problems into intermediate steps (Wei et al., 2022). ReAct interleaves reasoning traces with tool actions (Yao et al., 2023). Production designs usually keep a human in the loop for steps that are high-risk or can't be undone. Quality depends on agentic AI training data that shows the model what good multi-step behavior looks like.

How agentic workflow automation compares with other approaches

The table compares agentic workflow automation with robotic process automation (RPA) and a single-prompt LLM call.

Dimension Rule-based automation (RPA) Single-prompt LLM Agentic workflow automation
Control flow Fixed script written by a developer One input, one output Model plans and re-plans at run time
Exception handling Fails or escalates on unseen inputs None; no actions taken Retries, switches tools, or asks for help
Tool use Predefined UI or API actions None Selects and parameterizes tools per step
Reproducibility Deterministic Varies with sampling Varies across runs; must be measured
Auditability Script is the audit trail Prompt and output only Requires full trajectory logging
Evaluation unit Pass/fail per run Output quality Final state plus every intermediate step

For the model-level distinction, see agentic AI vs generative AI.

Why agentic AI workflows fail in production

Errors compound across steps. If each step independently succeeds 95% of the time, a 10-step task succeeds only about 60% of the time (0.95¹⁰ ≈ 0.60). On τ-bench, which tests agents in simulated customer-service domains, gpt-4o succeeded on fewer than 50% of tasks. In the retail domain, it succeeded on all eight repeated trials of the same task less than 25% of the time (Yao et al., 2024). An analysis of multi-agent frameworks identified 14 failure modes in three categories: system design issues, inter-agent misalignment, and task verification. Expert annotators agreed at a Cohen's kappa of 0.88 (Cemri et al., 2025).

  • Specification drift: The agent loses track of constraints, roles, or stop conditions over long sequences of steps.
  • Tool misuse: The agent picks the wrong tool or passes malformed arguments, and silent failures get read as success.
  • Weak verification: The agent declares the task complete without checking the final state against the goal.
  • Run-to-run inconsistency: The same task passes once and fails the next time, which hides risk if you only test single runs.

Teams that classify these errors systematically use a failure mode taxonomy to decide what to fix first.

How agentic workflows work: plan, act, observe, verify

Each cycle of an agentic workflow runs through four stages. The loop repeats until the success criteria pass or a step budget runs out.

Figure 1: How an agent breaks down goals, selects tools, and executes to deliver outcomes
  • Plan: The model breaks the goal into a task graph of sub-goals, dependencies, inputs, constraints, and stop conditions.
  • Act: The model maps each step to a tool in its registry and generates arguments that match the schema. Models can learn when and how to call tools from self-supervised examples (Schick et al., 2023).
  • Observe: The system parses the tool output, updates working memory, and flags errors or empty results.
  • Verify: The model checks progress against success criteria, then retries or revises the plan. Verbal self-reflection on failed attempts measurably improves later attempts (Shinn et al., 2023).

Single-agent vs multi-agent workflows

Multi-agent systems split work across specialized agents that talk to each other, a pattern implemented in frameworks such as AutoGen (Wu et al., 2023).

Dimension Single agent Multi-agent system
Specialization One model and prompt cover all sub-tasks Roles such as planner, coder, critic
Coordination overhead None Message passing, handoffs, shared state
Failure surface Planning and tool errors Adds inter-agent misalignment
Debuggability One trajectory to inspect Interleaved trajectories to reconstruct
Cost and latency Lower Higher per task
Best fit Bounded tasks with few tools Cross-domain tasks needing review steps

Both architectures depend on well-specified tasks and checks; see agentic rubrics and verifiers.

How to evaluate an agentic workflow before deployment

The numeric gates below are example starting points. Adjust them to your own risk tiers.

  1. Write the task contract: Define the goal, allowed tools, forbidden actions, stop condition, and a success check run by code where possible. Pass criterion: two reviewers working independently agree on whether a given final state counts as success.
  2. Log full trajectories: Record every reasoning step, tool call, argument, and observation. Pass criterion: you can replay 100% of production steps.
  3. Build a golden set: Start with at least 100 tasks covering common paths and known edge cases, each with an expert reference trajectory. Where more than one solution is valid, include each valid path.
  4. Measure consistency with pass^k: Run each task at least 8 times and report pass^1 alongside pass^k, the probability that all k runs succeed (Yao et al., 2024). Example gate: pass^8 ≥ 0.90 for read-only tasks.
  5. Label failures against a taxonomy: Have at least 3 annotators label each trajectory in a calibration batch. Start full labeling only when Cohen's or Fleiss' kappa reaches 0.61 ("substantial" agreement) (Landis & Koch, 1977), and aim for 0.80 or higher.
  6. Gate irreversible actions: Require human approval for payments, deletions, and external messages. This lines up with the human oversight duties in Article 14 of the EU AI Act for high-risk systems (European Parliament and Council, 2024) and the Manage function of the NIST AI Risk Management Framework (NIST, 2023).
  7. Turn failures into training signal: Feed corrected trajectories into supervised fine-tuning. Turn checks run by code into reward functions for reinforcement learning with verifiable rewards (RLVR).

Worked example: scoring a refund agent

A retail support agent receives this task: "Refund order #4471 if it arrived damaged within 30 days." The policy requires checking a damage photo before refunding. Three annotators label one trajectory:

Step Agent action Annotator A Annotator B Annotator C Consensus
1 lookup_order(4471) Correct Correct Correct Correct
2 check_policy(window=30) Correct Correct Correct Correct
3 issue_refund(amount=full) Tool-order error Tool-order error Correct Tool-order error

The final-state check passes because a refund was issued, yet the process broke policy. Across 10 runs, 7 pass the final-state check: pass^1 = 0.70, and pass^3 = C(7,3)/C(10,3) = 35/120 ≈ 0.29. Step-level review shows that 2 of those 7 skipped damage verification, so 5 of 10 runs comply with policy, which gives pass^3 = 10/120 ≈ 0.08. Checking only outcomes overstated reliability by more than 3x. Expert-built golden trajectory references catch exactly this gap.

How Appen supports agentic workflow evaluation

Research and engineering teams design the agent, choose its tools, and set its risk thresholds. Appen provides the human data and quality infrastructure that steps 3, 5, and 7 rely on, drawing on decades of running large human data programs.

  • Golden trajectories: Contributors matched to the task domain record step-by-step completions, explain the reasoning behind each action, and cover multiple valid paths so agents don't overfit to one route.
  • Failure labeling: Trajectories are classified against a taxonomy that separates planning, tool-call, and verification errors. Agreement is measured on calibration batches before full labeling starts.
  • Task and verifier design: Task environments, rubrics, and verifiers run by code are built to supply reward signals for RLVR training loops.
  • Annotator quality control: In the Appen Data Annotation Platform (ADAP), test questions with known answers gate entry to a task (quiz mode) and are embedded in live work (work mode). Contributors who fall below the project's accuracy threshold are flagged, and the job manager can remove them and their work from the job.
  • Targeted recruiting: Contributors are sourced by domain, region, and language across 500+ locales. That includes engineers for agentic software engineering (SWE) evaluation and reviewers for agentic RAG evaluation.
  • Off-the-shelf datasets: Licensed data from the Appen data catalog gives teams baseline training and test sets before custom collection begins.

To scope a golden set or failure-labeling pilot, talk to Appen's agentic AI data team.

Frequently asked questions

What does agentic mean in simple terms?

Agentic describes a system that works toward a goal on its own initiative instead of waiting for step-by-step instructions. In AI, the word signals three abilities: choosing the next action, using tools to carry it out, and judging whether the result moved it closer to the goal. A chatbot answers; an agentic system answers, acts, and checks. How much autonomy it gets varies, so deployments set explicit limits on what an agent may do unsupervised.

What is an example of an agentic workflow in software engineering?

A coding agent given a failing GitHub issue reads the repository, reproduces the bug, edits files, runs the test suite, and repeats until the tests pass. SWE-bench formalizes this with 2,294 real issues from 12 Python repositories, and scores a patch as correct only if it passes the project's own tests (Jimenez et al., 2024). A patch can still pass tests while adding unsafe or unmaintainable code, which is why expert human review stays in the loop.

Is agentic process automation replacing RPA?

Not wholesale. RPA stays cheaper and more predictable for stable, high-volume processes with structured inputs, such as pulling fields from a fixed invoice template. Agentic process automation earns its cost where inputs vary or exceptions are frequent. A common pattern is hybrid: the agent handles triage and exceptions, then passes structured instructions to existing RPA bots. That keeps the deterministic steps auditable and limits how many model decisions need evaluating.

Are agentic flows different from agentic workflows?

The terms are often used interchangeably. When engineers do draw a line, "agentic flow" usually means a predefined graph of steps where an LLM makes decisions at specific nodes, while "agentic workflow" means the model plans the sequence itself. The practical difference is how you evaluate them: fixed flows can be tested node by node, while open-ended workflows need full-trajectory review and repeated runs to measure consistency.

Which tools do agentic AI workflows connect to?

Typical tool registries include web search, retrieval over internal documents, SQL databases, code interpreters, browsers, and business application programming interfaces (APIs). Tools are exposed through function-calling schemas or open standards such as the Model Context Protocol (MCP), which defines how a model discovers and invokes external tools (Model Context Protocol, 2026). Every added tool widens the agent's options, so evaluation sets should include tasks where the right choice is to call no tool at all.

References

‍

    ‍