LLM Model Deprecation: Migrate Before Shutdown

An LLM model deprecation can stop a production workflow even when your application code has not changed. The obvious model name in one API client is rarely the whole dependency. Scheduled jobs, fallback routes, customer overrides, stored conversations, evaluation jobs, and provider aliases may still resolve to the retiring model. Change only the main configuration and the missed path fails on shutdown day.

Treat the notice as a migration deadline, not a search-and-replace task. Build a model dependency manifest, test the replacement against the workflow's business contract, cut traffic over through reversible configuration, and prove that calls to the retiring model have reached zero.

Distinguish deprecation from shutdown

Provider terminology matters because it determines what your system should do during the migration window. OpenAI defines deprecation as the process of retiring a model or endpoint, with a separate shutdown date after which it is no longer available. OpenAI also publishes recommended replacements and says generally available models normally receive at least six months of notice unless safety or compliance requires a shorter timeline.

Anthropic's model deprecation page likewise separates active, deprecated, and retired states. It recommends testing applications with the replacement well before retirement and provides a usage audit for finding deprecated-model calls. The exact notice policy differs by provider and model class, so store the provider's dates instead of applying one global assumption.

A deprecated model can still be callable. Do not convert the warning into an outage inside your own platform. A current Open Chat Studio issue reports that marking a model deprecated caused pipeline validation to fail immediately. Live chatbots stopped, and teams could not run tests or create the new pipeline version needed to migrate. The report covers one implementation. Production platforms still need separate behavior for warning and unavailable states.

Use four internal states:

1. active: new and existing assignments are allowed.
2. deprecated: existing assignments may run during the migration window, but new assignments are blocked.
3. retirement_due: the internal safety deadline has passed, so unresolved dependencies page an owner and may enter a controlled fallback.
4. retired: all calls are blocked because the provider no longer supports the model or your migration is complete.

Your internal safety deadline should come before the provider's shutdown. The gap leaves time to catch traffic from rare jobs and customer-specific paths.

Build the model dependency manifest

Searching the main repository for a model string finds only direct references. Production workflows often select models through environment variables, database settings, routing rules, aliases, and provider dashboards. Historical state can also pin a model that no current configuration mentions.

Create one manifest row for every execution path, not one row per code repository. A useful record looks like this:

workflow_id: invoice-triage
execution_path: scheduled-import
provider: example-provider
configured_model: production-model-alias
resolved_model: retiring-model-version
selection_source: tenant-policy
owner: finance-platform
risk_class: high
last_called_at: 2026-09-08T17:31:00Z
replacement_model: candidate-model-version
provider_shutdown_at: 2026-12-15T00:00:00Z
internal_cutover_at: 2026-11-15T00:00:00Z
migration_state: inventory_complete

Populate it from several sources:

• application code and deployment configuration;
• workflow database rows and tenant overrides;
• model routers, fallback chains, and feature flags;
• provider usage exports or audit pages;
• traces and billing records grouped by resolved model;
• scheduled evaluations, batch jobs, and test environments;
• persisted conversations or checkpoints that record a model;
• infrastructure templates and disaster recovery configuration.

Record both the configured value and the resolved model. An alias such as production-model can move without a code change. If telemetry records only the alias, you cannot prove which model handled a run or detect that a provider changed its target.

Microsoft's model lifecycle documentation shows why region and deployment type belong in the manifest. Replacement availability and overlap can vary by deployment mode and region. Google Cloud's deprecation schedule separately lists deprecation dates, shutdown dates, and migration destinations for affected generative AI features. A replacement named in a notice is not automatically deployable under every constraint your workflow has.

Set a manifest completeness rule before testing. For example, every model call observed in the last 90 days must map to an owned row, and every configured execution path must have either observed traffic or a documented reason it was dormant. Rare quarterly jobs need configuration evidence because a 90-day traffic window can miss them.

Freeze the replacement contract

The provider's recommended model is a candidate, not proof of compatibility. Write the workflow contract before comparing output. The contract should describe business behavior your application depends on:

• accepted input size and document types;
• required structured-output schema;
• allowed tools and valid argument shapes;
• refusal and escalation behavior;
• latency and cost limits;
• data region and retention controls;
• terminal states and side-effect rules;
• quality checks tied to the workflow outcome.

Separate hard invariants from measured preferences. A duplicate payment or unauthorized tool call is a hard failure. A modest latency or wording change may be acceptable if the business outcome still passes. Otherwise, a fluent answer can hide a broken control.

Pin the candidate by its most specific supported identifier. Do not evaluate a moving alias if a versioned model is available. Record the prompt, tool schema, model parameters, retrieval version, evaluator version, and application release used in each comparison. Otherwise a failed test cannot tell you what changed.

Replay production cases without side effects

Build a redacted replay set from representative production traces. Include ordinary cases, expensive cases, long inputs, refusals, each tool, malformed tool results, old conversation formats, retries, and past incidents. Sample across business risk, language, customer configuration, and input type. A random average can pass while one consequential slice breaks.

Run deterministic checks first. Validate response schemas locally. Reject unknown tools. Check required tool arguments. Confirm stop reasons and errors map to known internal states. Verify the candidate respects the context and output limits your workflow requires.

Then compare business outcomes. An invoice workflow might compare extracted totals, supplier identity, duplicate flags, approval route, and proposed tool action. Exact prose can differ as long as the decision and safety boundary match.

Replace write tools with dry-run implementations during replay. A model migration test must not send email, update a customer record, or create a payment. The dry-run tool should return realistic success and error payloads while recording the proposed action for comparison.

Keep stored conversation compatibility in its own test group. A new model may accept the current message format but fail when an old checkpoint contains provider-specific content blocks or tool-call identifiers. Decide whether those runs stay pinned to the old model, are transformed through a tested adapter, or are allowed to finish before cutover.

Cut over through reversible routing

Once replay passes, send copied production inputs to the candidate in shadow mode where policy allows it. The old model remains authoritative, and candidate tool actions remain inert. Compare schema validity, tool proposals, business outcomes, latency, errors, and cost by risk slice.

Move to a small live cohort only after the shadow checks pass. Use stable assignment by tenant, case, or conversation so one unit of work does not jump between models midstream. The existing AI workflow canary deployment process should own cohort selection, stop conditions, and promotion. The deprecation runbook adds a deadline and a dependency-closure requirement to that release process.

Make rollback a configuration change. Keep the old model, prompt bundle, adapter, and credentials available until the observation window closes. If the old model will disappear before a long-running workflow completes, pinning alone is not enough. Pause before the last safe checkpoint, finish early, or migrate the checkpoint through a tested compatibility path.

Handle partial migration explicitly. If one low-risk workflow passes but a regulated workflow lacks regional replacement capacity, do not mark the model migration complete. Track each execution path independently. Escalate the blocked path with its shutdown date, business owner, fallback options, and evidence still needed.

Prove retired-model traffic is zero

A green deployment does not prove the old model is unused. Verification needs both positive and negative evidence.

Positive evidence shows that eligible runs use the candidate and satisfy the workflow contract. Negative evidence shows that no call still resolves to the retiring model. Query provider usage, model gateway logs, and application traces by resolved model. Run the check across at least the longest relevant schedule interval, including weekends and batch windows.

Alert on any new call to a deprecated model after the internal cutover. Include workflow ID, execution path, owner, configuration source, and trace ID. Do not log prompt content just to identify the caller.

Test dormant paths before the shutdown date:

• trigger scheduled and event-driven workflows in a safe environment;
• exercise fallback routes by injecting controlled failures;
• resume old checkpoints and conversations;
• run disaster recovery configuration checks;
• inspect tenant overrides and newly restored backups;
• confirm test and evaluation jobs use supported models.

Only remove the old path when every manifest row is complete, candidate outcomes pass, unresolved in-flight work has a policy, and negative-use monitoring has stayed at zero for the chosen interval. Keep the migration evidence with the workflow's release record.

Handle common retirement failures

If the provider removes a model before your internal deadline, route only to a replacement that already passed the required contract. An untested automatic fallback can turn a visible outage into silent bad decisions. If no validated candidate exists, fail closed for high-risk side effects and send the case to the documented manual path.

If an alias moved unexpectedly, record the newly resolved version, stop treating prior evaluation as current, and rerun the affected gates. Alias drift is a model change even when configuration stayed constant.

If calls continue after cutover, use the manifest's selection_source to find the stale layer. A tenant database value needs a different fix from a queue payload or environment variable. Repair the source, replay the path, and restart the zero-use observation window.

If the candidate passes averages but fails one important slice, hold that slice on the old model only if the shutdown window permits it. Otherwise narrow the workflow, add human review, or choose another candidate. Do not bury the failure in an aggregate score.

Run a retirement-day drill

Before the real shutdown, block the old model in a staging gateway or controlled production cohort. Confirm that no workflow silently falls back to it, alerts reach the right owner, high-risk work fails safely, and the manual route can absorb expected volume. Restore normal routing after the drill and record every missed dependency as a manifest defect.

Export the last 90 days of model usage, group calls by resolved model and execution path, and map each group to an owner. Then compare that list with configuration and scheduled jobs. That first manifest will tell you whether the migration is a one-line change or a production program.

References

1. OpenAI API deprecations supports the deprecation, shutdown, replacement, and notice-period distinctions.
2. Anthropic model deprecations supports advance testing, retirement notices, replacement guidance, and usage auditing.
3. Google Cloud generative AI deprecations supports the dated deprecation, shutdown, and migration schedule pattern.
4. Microsoft Foundry model lifecycle supports lifecycle stages and replacement overlap differences across deployment conditions.
5. Open Chat Studio issue 4430 documents the practitioner-reported failure caused by treating deprecation as immediate invalidation.

---