Back to Blog
Aerial view of a multi-level highway interchange where converging lanes of traffic must be sequenced safely

AI Agent Concurrency Control: Prevent Lost Updates

11 min read

AI agent concurrency control becomes necessary as soon as two actors can change the same case. An agent reads version 12, a manager approves the case, and the agent later saves a proposal based on version 12. If that save replaces the current record, the approval disappears even though every individual operation returned success. Retries and idempotency keys do not solve this conflict because the two writes represent different intentions.

Prevent the overwrite at a versioned mutation boundary. Models can produce proposals outside the database transaction, but the application must commit each proposal through a short conditional write tied to the version the model saw. A failed condition is a conflict to classify, not a reason to repeat the write. This design applies to business records, checkpoints, approvals, and tool results. It also gives you a way to test each race on purpose.

Why successful requests still lose data

A lost update begins with two readers observing the same state. Each computes a valid next state. Writer A commits first. Writer B commits later using its stale copy and replaces some or all of A's change. Nothing has to crash, time out, or return an error.

Internal AI workflows are prone to lost updates because model inference is slow compared with a database write. While the model runs, a webhook can deliver new facts. An operator might correct a field, another branch might finish, or an approval might change the permitted action. The output can be reasonable for the snapshot the model saw and wrong for the record that now exists.

Parallel graph execution creates a related problem inside the orchestrator. LangGraph documents an INVALID_CONCURRENT_GRAPH_UPDATE error when concurrent nodes write one state key without a reducer that explains how to combine the values. A reducer works when both results are naturally additive. It is not a safe default for decisions such as approved, rejected, or assigned_owner, where order and authority matter.

Duplicate delivery is a separate problem. An idempotency key answers, "Have I already applied this logical command?" A version condition answers, "Has the state used to make this decision changed?" Most production workflows need both checks.

Define the mutation contract

Require four values on every state-changing operation:

  1. A stable command identifier for duplicate detection.
  2. The expected record or aggregate version.
  3. A narrow patch or domain command, not a complete stale object.
  4. The actor, reason, and evidence needed for an audit record.

The storage layer must apply the command only when its expected version equals the current version. AWS describes this mechanism as optimistic locking with a version number. A stale writer receives a condition failure rather than replacing a newer value. The same contract can be implemented with a relational update predicate, a compare and swap operation, or an API precondition.

For HTTP services, If-Match provides the standard conditional request. The client sends the entity tag it read. The server performs the mutation only if the current representation still matches. This moves conflict detection to the system that owns the record instead of asking the agent to compare a cached copy.

Keep patches narrow. A classifier that intends to set category should not send old values for approval_status, owner, and notes back to storage. Narrow commands leave fewer unrelated fields for a stale writer to erase, and the resulting conflicts are easier to classify.

Keep model inference outside the transaction

A database transaction should not stay open while a model runs. Long transactions retain resources and invite contention. They also cannot protect a later write to another service.

Use a read, propose, commit sequence:

  1. Read the minimum business facts plus the current version.
  2. Call the model and validate its proposed command.
  3. Recheck authorization and time-sensitive policy in application code.
  4. Attempt a conditional write using the version from step one.
  5. Record either the committed result or the conflict disposition.
class VersionConflict(Exception):
    pass


def classify_and_commit(case_id, command_id, model, store, policy):
    snapshot = store.read_case(case_id)

    proposal = model.classify(
        text=snapshot.text,
        allowed_categories=policy.allowed_categories(snapshot.tenant_id),
    )
    command = validate_proposal(proposal)

    policy.authorize(
        actor="classification-agent",
        action="set-category",
        current_case=store.read_case(case_id),
    )

    result = store.update_category_if_version(
        case_id=case_id,
        expected_version=snapshot.version,
        command_id=command_id,
        category=command.category,
        evidence_ref=command.evidence_ref,
    )

    if result.conflict:
        raise VersionConflict(result.current_version)

    return result

The example performs authorization against current state, but the conditional write remains the final guard. A change can occur after the policy read and before the commit. The storage operation must atomically check the version, detect a duplicate command, apply the change, increment the version, and append its event or audit record.

A separate "check version" query followed by an unconditional update still has a race. Another writer can commit between those statements. Put the comparison in the update predicate or use the database's conditional write primitive.

Choose a conflict policy by field and intent

A version conflict is a control signal, not an instruction to retry the same stale write. Load the current record and compare the fields relevant to the proposal. Then apply one of four policies.

Retry from current facts

Retry model inference when the changed field affects the decision. If a new customer message arrived while a support classifier was running, build a new prompt from the current conversation and produce a new command with a new expected version. Keep the same workflow identity, but use a new command identifier if the intended decision is genuinely new.

Limit attempts and elapsed time. A hot record can change faster than the agent completes. Once the budget is gone, pause the run or route the case to an operator instead of paying for more model calls that are likely to conflict again.

Merge independent changes

Merge only when the domain defines a deterministic rule. Appending a tagged observation to a set can be safe if each item has a unique identity and removal semantics are clear. Incrementing a monotonic counter can be safe through an atomic database operation. Updating category and adding an unrelated internal note may be mergeable when neither policy depends on the other.

A generic object merge is not a domain rule. Last writer wins is dangerous for approvals, revocations, assignments, financial states, and user corrections because it treats timing as authority.

Reject the stale proposal

Reject when the newer state makes the command invalid or unnecessary. If a manager closed the case while the agent prepared a reply, discard the proposed send action. Record that the proposal was evaluated and rejected against a newer version so operators can distinguish a controlled conflict from a missing action.

Escalate an ambiguous conflict

Escalate when both intentions are valid but cannot be combined automatically. An operator changing a refund amount while an agent prepares payment is not a mechanical retry. Present the old snapshot, current record, proposed command, changed fields, and policy reason. Do not ask the operator to reconstruct the race from raw logs.

Handle approvals and automation as competing actors

Human approval does not automatically outrank every later event. Define the state machine explicitly. An approval might authorize one immutable proposal, authorize any action below a limit for a short period, or remain valid until relevant facts change. Those are different contracts.

Bind approval to the proposal hash or record version it reviewed. If the agent changes the recipient, amount, target object, or action after approval, require a new approval. If an unrelated note changes, policy may allow the existing approval to stand. The conflict classifier should know which fields invalidate consent.

Temporal's message passing guidance documents that update and signal handlers can run concurrently and that workflows need explicit concurrency controls and handler completion behavior. Apply the same discipline even without Temporal: serialize transitions that share invariants, make message ordering visible, and wait for consequential handlers to finish before declaring the workflow complete.

Authority changes should fail closed. A revocation, denial, account suspension, or removed scope must prevent a stale approved action from committing. When authorization state has its own lifecycle, the conditional write should check both the expected business version and the policy version.

Protect checkpoints and parallel branches

Workflow checkpoints are shared state too. A checkpoint key based only on the workflow identifier can let two workers save incompatible snapshots. Include a monotonically increasing checkpoint version or lease generation. Reject a save from a worker that no longer owns the current generation.

The framework cannot be assumed to have solved every race. A LangGraph checkpoint issue reports silent data loss from concurrent writes. This is a practitioner incident, not a universal statement about the framework. Test contention against the exact checkpointer and storage configuration you deploy.

For parallel branches, decide whether their outputs are events, commutative values, or exclusive decisions. Events can be appended with stable identities. Commutative values need a tested reducer. Exclusive decisions need arbitration, ordering, or a single owner. A branch that writes a full parent state object should be treated as suspicious because it can copy stale sibling fields into its commit.

Record branch identity, base version, command identifier, current version, and conflict outcome in traces. These fields let engineers distinguish model errors from coordination failures without logging sensitive prompt content.

Select database isolation deliberately

Optimistic version checks solve a defined aggregate update. They do not eliminate every database anomaly. A decision may depend on several rows or a predicate such as "no active job exists for this account." In those cases, review the database's transaction guarantees.

PostgreSQL explains that transaction isolation levels permit different anomalies and that serialization failures require a retry of the complete transaction. Keep such transactions deterministic and short. Do not place a model call inside a transaction that may need to restart.

Use a unique constraint for invariants such as one active command per command identifier. Use row locking when a short, high-contention critical section truly needs one writer. Use serializable transactions when the decision spans a set of rows and the database can detect unsafe interleavings. Use version checks for ordinary aggregate updates where conflicts are expected to be uncommon.

Write this boundary down. Engineers need to know which system owns the version, which fields form the aggregate, and whether external updates pass through an outbox, a saga, or a separate conditional API.

Test the races deterministically

Ordinary load testing may never produce the critical ordering. Use barriers to pause each writer after its read, then release the commits in a chosen sequence.

  1. Start agent A and operator B from version 20.
  2. Pause both after they prepare commands.
  3. Commit B and assert the record becomes version 21.
  4. Commit A with expected version 20.
  5. Assert A receives a conflict and no field from version 21 disappears.
  6. Run the conflict classifier and assert the documented retry, merge, rejection, or escalation outcome.

Repeat this for an approval racing a proposal change, two parallel tool results, a revocation racing a write, two duplicate deliveries, and a checkpoint lease changing between compute and save. Verify the state, event stream, external side effects, and audit disposition after each run.

Add a hot-record test that forces several consecutive conflicts. Confirm the workflow stops at its budget and becomes visible to an operator. Add a worker-crash test after the conditional write commits but before the response is recorded. The stable command identifier should let the resumed workflow retrieve the original result rather than create another mutation.

Check negative outcomes, not only the final happy state. Assert that a stale approval never authorizes a changed consequential action, a rejected proposal cannot be replayed without current facts, and a conflict never becomes an unconditional overwrite through a fallback path.

Common mistakes

Last writer wins discards one intention based on timing. That can be acceptable for a disposable heartbeat, but it is not a safe conflict policy for business decisions.

A version field alone changes nothing. The comparison and mutation have to happen atomically. Retrying every conflict after a fresh read is also unsafe: it can create a model loop, increase cost, and hide a policy disagreement.

Version logic placed only in the agent layer is easy to bypass. Webhooks, admin tools, batch jobs, and direct service calls need the same check. Enforce it in the service or storage boundary that owns the record.

One merge function cannot safely handle every field. Text notes, tags, approvals, balances, ownership, and lifecycle states have different semantics. Put conflict rules next to domain commands and test each rule with competing actors.

Put the boundary into one workflow

Choose a workflow where an agent and at least one other actor can edit the same record. Add a version to the owning aggregate, change one write endpoint to require the expected version, and return a typed conflict containing the current version and changed-field summary. Keep the model call outside the commit.

Then write four conflict rules for that command: retry, merge, reject, and escalate. Not every rule must be reachable, but the code should make the allowed outcomes explicit. Run the barrier test with both commit orders and confirm that neither path silently erases the other actor's work.

The boundary works when the stale proposal is rejected, merged by an explicit rule, or shown to an operator. It must never return success after erasing a newer decision.

References


About Fire In Belly: Independent senior engineering from Tallinn, Estonia. We design and build AI workflow automation with the versioned mutation boundaries, conflict policies, and contention tests described above, at published fixed prices. Schedule a call to discuss your next project.