LLM Data Lineage: Tracing Outputs Back to Source Records

An AI workflow writes the wrong value into a customer record. The trace shows that each service returned successfully, and the audit log shows who approved the update. Neither record answers the harder question: which source snapshot, retrieved passages, prompt revision, model configuration, tool response, and deterministic transform produced that value? Without LLM data lineage, the team cannot reproduce the result or find every other output derived from the same bad input.

The fix is a lineage graph built from immutable references and explicit dependency edges. It should connect a business output to the exact evidence and execution versions behind it, while keeping sensitive payloads out of telemetry. This guide defines the event contract, capture points, retry rules, impact queries, and tests needed to make that graph useful during an incident.

Separate lineage from logs and traces

Logs describe events. Distributed traces connect timed operations within an execution. Audit records establish who attempted or approved an action. Lineage answers a different set of questions:

• What did this output depend on?
• Which activity generated each intermediate result?
• What other outputs used the same source version?
• Can the team assemble the same effective inputs again?
• Which derived records need correction or deletion?

Keep those records connected, but do not force one store to perform every job. OpenTelemetry's generative AI semantic conventions define spans, events, metrics, and attributes for model and agent operations. That makes them useful for latency, errors, token use, and execution correlation. A span may be sampled or retained for less time than a regulated business record, so it should not be the only record of derivation.

Use an immutable lineage event stream as the durable dependency record. Store a trace ID on every lineage event so an operator can open the corresponding execution trace. Store an audit event ID on approval and write activities so the operator can inspect identity and authorization. The lineage graph then joins the records without copying their full contents.

Design the lineage model around three concepts

The W3C PROV overview organizes provenance around entities, activities, and agents. That vocabulary maps cleanly to an internal AI workflow.

An entity is a versioned thing. Examples include a CRM record snapshot, a document chunk, a prompt template, a model response, a tool result, an approval decision, and the final business record. An activity is a transformation such as retrieval, prompt rendering, model inference, validation, or a database write. An agent is the person, service account, workflow definition, or provider responsible for an activity.

Three relationships carry most of the graph:

• An activity used an entity.
• An entity wasGeneratedBy an activity.
• An entity wasDerivedFrom another entity.

Do not model a mutable database row as one timeless entity. Create an identity for each version that the workflow observed. If the source system exposes a version, revision, ETag, or change sequence, include it. Otherwise, record a canonical content hash plus the observation time. The graph must distinguish the customer record read at 09:02 from the corrected record available at 09:20.

OpenLineage applies a similar event approach to jobs, runs, and datasets, with facets for extra metadata. Reuse that split when it fits your data platform, but add AI-specific entities rather than pretending a prompt, retrieval result, or approval is only a dataset.

Define one append-only event contract

A lineage event should be small enough to emit at every boundary and complete enough to reconstruct dependencies. This example uses references instead of raw business content:

{
  "event_id": "evt_01",
  "occurred_at": "2026-09-07T09:15:31Z",
  "workflow": {
    "name": "renewal-risk-review",
    "version": "wf_42",
    "run_id": "run_918"
  },
  "activity": {
    "id": "act_generate_summary_3",
    "type": "model_inference",
    "attempt": 1,
    "trace_id": "trace_771"
  },
  "used": [
    {"entity_id": "crm_account_24@v108", "role": "source"},
    {"entity_id": "prompt_renewal_summary@sha256:8c2", "role": "prompt"},
    {"entity_id": "retrieval_set_run_918_q2", "role": "evidence"},
    {"entity_id": "model_config_support_summary@v7", "role": "configuration"}
  ],
  "generated": [
    {"entity_id": "model_output_run_918_step_3", "role": "candidate_summary"}
  ],
  "agent": {"type": "service", "id": "workflow-renewal-review"},
  "payload_ref": "evidence://protected/run_918/step_3",
  "payload_hash": "sha256:44e"
}

The short values are illustrative identifiers, not a recommendation for hash length. Production hashes should use the security and collision properties required by your environment.

Use one event ID for idempotent ingestion. Use a stable run ID across the workflow. Give each activity a separate ID and attempt number. Every entity ID must resolve to immutable metadata or to a protected payload location that preserves the observed version. The event itself may carry classifications, tenant identity, retention class, and schema version, but not a prompt or customer record unless the lineage store has been designed to protect that data.

Capture lineage at each workflow boundary

Instrumentation should sit beside the code that accepts or commits data. Adding one callback around the model client misses retrieval, transforms, approvals, and business writes.

Record source reads as observed versions

At ingestion or query time, identify every source object the activity actually used. For a database read, capture table or resource identity, primary key, observed version, and selected field set. For files, capture repository or object identity, version, and content hash. For APIs, preserve the provider object ID, response version when available, and a protected snapshot reference.

Avoid identifying a source only by URL or row ID. Those locations can return different content later. Replay requires the observed version, not merely the place where current data lives.

Make retrieval sets first-class entities

A retrieval activity may inspect dozens of candidates and pass only five chunks to the model. Record the query plan as an input, the index and embedding versions as configuration, and the ordered selected set as a generated entity. Each selected item should point to its document and chunk version, score type, rank, and applicable access-control decision.

Lineage needs more than citation records. A citation may show what appeared in the final answer, while lineage needs to show what evidence entered the model context and which retrieval configuration produced it.

Version prompts and model configuration

Store the prompt template version separately from rendered prompt inputs. Record the provider model identifier, decoding settings, tool schema versions, and any routing policy that selected the model. MLflow Tracking documents runs, parameters, metrics, artifacts, and traces as separate records. Apply the same discipline here: treat configuration and generated artifacts as versioned dependencies, not text fields sprinkled through logs.

Do not claim that the same model request will always produce the same output. Replay has two useful modes. An evidence replay reassembles the exact effective inputs for inspection. A behavioral replay sends those inputs through a declared current or historical configuration and compares outcomes. Name the mode in tooling and incident reports so people do not confuse input reproducibility with deterministic generation.

Connect tools, approvals, and final writes

For a tool call, record the validated request entity, the tool and schema version, the authorization decision, and the response entity. For a human review, record the candidate shown, the evidence available, the reviewer identity reference, the decision, and any corrected value. For the final write, record the exact approved entity and the source-system version before and after the mutation.

That final edge turns a model trace into business lineage. It lets an impact query start from bad evidence and reach a ticket, invoice, account field, or notification that people actually acted on.

Handle retries, fan-out, and late events

Retries create new activities, not new workflow runs. Keep the same run ID, create a new activity ID, increment the attempt, and point the successful output to the attempt that generated it. A failed attempt remains useful evidence, but downstream entities must not depend on it unless its partial result was consumed.

Fan-out needs one activity per branch and a join activity that records every branch result it used. If a timeout causes the join to continue with three of four results, the generated entity must depend on those three and on a policy decision that allowed partial completion. Do not add the missing fourth dependency later merely because its event arrived.

Lineage ingestion should accept events out of order. Write append-only events first, then build graph projections asynchronously. Keep unresolved edges in a repair queue, expose their age, and block claims of complete lineage when required parents are missing. Event-time ordering should come from activity relationships and source versions, not from collector arrival time.

A current practitioner issue on recovering ordered LLM transformation provenance shows why sequence matters. A final text plus an unordered set of contributors cannot establish which transformation operated on which prior version. Preserve ordered activity and entity edges while the workflow still knows them.

Keep sensitive payloads out of telemetry

Lineage expands the number of systems that can reveal where sensitive data moved. Apply the same tenant, access, retention, and deletion boundaries used by the source system.

The event stream should normally contain identifiers, versions, hashes, classifications, and protected payload references. Store raw prompts, retrieved passages, tool responses, and model outputs in a content store with narrower access. Authorize graph traversal before resolving any payload reference. A person allowed to inspect one workflow run should not gain access to every upstream customer record through the lineage viewer.

Hashing does not anonymize predictable values. A hash is useful for integrity and equality checks, but an attacker can guess a small input space and compare hashes. Keep restricted values out of the event even when hashed, or use a keyed construction managed under the relevant security policy.

Deletion requires two operations. Remove or tombstone protected payloads according to policy, then retain only the minimum non-content lineage evidence permitted for operational and legal needs. Impact analysis should be able to find derived outputs before deletion removes the evidence required to correct them.

Build queries that operators can use

A lineage graph earns its cost by answering incident and maintenance questions quickly. Implement these four queries before adding a dashboard:

1. Explain an output. Starting from a business record version, return the generating activity, approval, model output, tool results, retrieval set, source snapshots, prompt version, model configuration, and trace links.
2. Find affected outputs. Starting from a bad document chunk, source record version, prompt revision, model configuration, or tool response, traverse forward to every generated business entity.
3. Assemble a replay packet. Return immutable metadata and authorized payload references for all dependencies, plus workflow code version and activity order.
4. Check completeness. Report missing required edges, unresolved entities, schema versions the reader cannot interpret, and payload references that expired before their dependents.

Set traversal limits and tenant filters in the query service rather than relying on callers. Large fan-out graphs can otherwise turn an incident query into a production outage or expose another tenant's dependencies.

Verify the lineage contract

Test lineage with fixtures that assert graph structure, not only event emission. A success-path fixture should start with two source versions, retrieve evidence, call a model, pass validation, receive approval, and write one business record. Assert that the output reaches every dependency through the expected activities.

Add failure fixtures for a model retry, a tool timeout with no consumed result, a partial fan-out, a corrected human decision, a duplicate event, an out-of-order event, an unresolved source, and an expired payload reference. For each fixture, run explain, impact, replay, and completeness queries.

Then perform a correction drill. Mark one source entity as invalid, run the forward impact query, and compare the returned business outputs with the fixture's known affected set. Reprocess one output under a new run ID, preserve a wasRevisionOf link to the old entity, and verify that the graph distinguishes correction history from the original derivation.

Track coverage as the share of final business writes with a complete path to required source, configuration, approval, and execution entities. Track unresolved-edge age separately. A high event count can coexist with unusable lineage if final writes have no dependency path.

Start with one high-consequence write

Choose one workflow that writes a durable business record or sends an external action. Define its required entity and activity types, add capture at each boundary, and implement the four operator queries. Run the correction drill before expanding instrumentation to other workflows.

Do not begin by collecting every prompt into a general telemetry index. Begin with the output that must be explained, then trace backward until every decision-relevant dependency has an immutable identity and an authorized path to its evidence.

References

1. W3C PROV overview supports the entity, activity, agent, usage, generation, and derivation model.
2. OpenLineage documentation supports the event-oriented job, run, dataset, and extensible facet structure.
3. OpenTelemetry GenAI semantic conventions supports the distinction between generative AI execution telemetry and durable lineage.
4. MLflow Tracking supports separate run, parameter, artifact, and trace records for reproducibility and inspection.
5. Ordered LLM provenance issue provides a practitioner example where ordered transformations must be recovered from a final generated artifact.

---