Skip to main content

Running coding agents in parallel with a task DAG

The DAG is the heart of how Kiwi turns one big task into safe, parallel work. This page covers what a DAG is, why Kiwi uses one, how it is created, how it is scheduled, and how you monitor it.

What is a DAG?​

A DAG — Directed Acyclic Graph — is a set of nodes connected by one-directional edges with no cycles. In Kiwi:

  • each node is a worker spec (one editing task with its own test command), and
  • each edge is a depends_on relationship: "this worker may only run after that worker has succeeded."

"Directed" means the dependency points one way. "Acyclic" means there are no loops, so there is always a valid order to run everything and the plan can never deadlock on itself.

In this example, A has no dependencies and runs first. Once A succeeds, B and C become eligible and run in parallel on different daemons. D waits until both B and C have succeeded.

Why a DAG, and not a linear script?​

A linear script forces everything to run one step at a time, in a fixed order, even when steps are independent. A DAG captures the actual structure of the work:

  • Parallelism where it is safe. Independent workers (no path between them) run at the same time across daemons, so a big task finishes faster.
  • Ordering where it is necessary. When one worker's change is a precondition for another (a shared type, a renamed function), the edge guarantees the dependency lands and is verified green before the dependent worker starts.
  • Correct partial failure. If a worker fails, only the parts of the graph that depend on it are affected — independent branches keep going.
  • A big task, safely split. The DAG is exactly what lets Kiwi decompose a task, parallelize the independent parts, and serialize the parts that must be ordered.

How the DAG is created​

The Planner produces the DAG. It asks a frontier model to decompose your task into a Plan{summary, workers[]}, where each PlannedWorker carries a depends_on list of the other worker ids it requires. The service then:

  1. hashes the plan into an immutable content-addressed manifest, and
  2. enqueues one QueuedTask per worker onto the lease queue, all in a single transaction.

The task id is deterministic — <job_id>-<worker_id> — so a worker's depends_on ids resolve directly to its sibling tasks' ids. No separate dependency table is needed; the edges travel inside each worker spec.

How the DAG is scheduled​

Scheduling happens in the lease queue (pkg/store/queue.go). Tasks are leased, not popped — see The Control Plane / Orchestrator for the full state machine. The DAG rule is enforced at lease time:

A task is leasable only when every task in its depends_on list has already SUCCEEDED.

When a daemon polls for work, LeaseNextTask scans the oldest queued tasks and picks the first one whose dependencies are all satisfied (firstLeasable). A task blocked on an unfinished dependency does not stall the queue — the scanner looks past it to find other leasable work. Because SUCCEEDED is a terminal state, a dependency that reads as satisfied stays satisfied.

Failure propagates the other way: if any dependency has FAILED, the dependent task can never satisfy its DAG, so it is failed fast rather than left queued forever. This cascades transitively down the graph — a task failed this way becomes a failed dependency for anything that depended on it.

Scheduling also respects org limits at lease time: a per-org concurrency cap, a per-job and per-month budget cap, and (on the Free tier) a monthly agent-minute ceiling. Fleet routing means a daemon only leases work for its own fleet plus unassigned (fleet_id = "") work.

How the DAG is monitored​

Every worker in the graph produces a job you can watch:

  • Jobs — one job per plan, with the summary and the per-worker status rolling up from the queued tasks.
  • Events — an append-only, per-job trace with a monotonic sequence and phase markers, so you can follow each worker as it plans, edits, tests, and opens its PR.
  • Live topology — the dashboard hangs each running job off the executor daemon that leased it, so you can see the whole swarm working the graph at once.

See Observability for the event trace, checkpoints, and topology view in detail.