BYOK LLM Platform: Proving Key Custody Before Rollout

Security reviews for internal agents rarely stall on model quality. They stall on three custody questions: who holds the provider credentials, who can decrypt stored prompts and traces, and what happens if either must be revoked today. A vendor slide that says "BYOK supported" usually collapses those into one checkbox and answers none of them.

This guide is the key-custody contract for one shared internal LLM or agent platform. It complements AI agent credential security, which owns keeping reusable secrets out of model context via a request broker, and LLM gateway deployment, which owns the shared control plane and egress closure. It also sits upstream of AI agents cost management: you cannot attribute or cap spend cleanly if every team still owns a flat provider key. Here the job is narrower: separate API-key BYOK from encryption-key BYOK, keep master credentials in a vault the platform only references, issue workspace-scoped virtual keys with budgets, prove plaintext never reaches agents, and rehearse revoke as an incident control.

Treat BYOK as two proofs, not one checkbox

Treat every BYOK checkbox as two separate proofs: API-key custody and encryption-key (CMK/EKM) custody. LockLLM's BYOK versus managed-keys guide names the split explicitly. API-key BYOK means the enterprise supplies its own OpenAI, Anthropic, Bedrock, or compatible credentials so billing, logging, and provider agreements stay under the enterprise account. Encryption-key BYOK, often called customer-managed keys (CMK), means the enterprise's KMS holds the master key that wraps data-at-rest keys; the platform must call back to that KMS to decrypt, and revocation stops decryption rather than relying on a contractual promise.

Augment's enterprise agent rollout guide frames the same split as the difference between unblocking a security review and merely routing traffic. Regulated teams need both: direct provider relationships for data-processing agreements and Zero Data Retention, plus customer-held encryption keys for lifecycle evidence. Accepting a single "BYOK" answer mixes those proofs and lets a platform pass review while still holding the only copy of the decryption material.

OpenAI Enterprise Key Management documents the encryption half in production terms. OpenAI generates a data encryption key (DEK), your cloud KMS wraps it with a key encryption key (KEK) you control, and OpenAI stores only the encrypted DEK as metadata. Your KEK never enters OpenAI systems. Supported external KMS targets are AWS KMS, Google Cloud KMS, and Azure Key Vault, with limited encrypt/decrypt grants to OpenAI identities. That is CMK/EKM. It does not decide who owns the provider API credential that places the call.

Write the review checklist as two columns. Column A: which workspace stores which provider credential, how it is referenced, and which virtual keys may use it. Column B: which KMS key encrypts prompts, traces, eval fixtures, and long-term memory at rest, who can grant decrypt, and how revoke is tested. If either column is empty, BYOK is incomplete.

Reject flat keys and managed-key double taxation

The failure mode most platforms inherit is a single highly privileged provider key pasted into every agent environment. LockLLM calls this the flat-key architecture. One leaked key becomes a financial denial-of-service against the enterprise quota, and audit logs cannot attribute which application burned the tokens. Managed-key platforms invert the problem: the vendor pools credentials, marks up tokens or buries them in seats, and the enterprise loses both negotiated rates and per-workload attribution.

Augment notes the complementary anti-pattern: giving every developer a personal provider key. Spend siloes, prepaid credits go unused, and there is no central place to enforce budgets or model allowlists. The stronger pattern is one (or a small set of) enterprise provider credentials, held centrally, with workspace-scoped virtual keys in front.

Managed keys remain fine for a short proof of concept with no regulated data. They are the wrong default once security asks for custody evidence, finance asks for direct provider invoices, or multiple internal products share the same model path. Document that decision explicitly so POC convenience does not silently become the production architecture.

Store master credentials as vault references

Store master provider credentials as vault references and issue virtual keys; never distribute the master key to teams or agents. Portkey Secret References describe the control-plane versus data-plane split that security teams actually want. The control plane stores only the reference configuration: manager type, secret path, and how to authenticate to AWS Secrets Manager, Azure Key Vault, or HashiCorp Vault. At request time the data plane fetches the secret, caches it briefly, and uses it. The dashboard never needs the plaintext value.

That pattern matters more than which gateway brand you pick. If the platform requires you to paste the OpenAI key into its admin UI and stores it as a first-class secret in the platform database, you have transferred custody even when marketing still says BYOK. Prefer integrations that resolve secret_mappings at runtime from infrastructure you already rotate and audit.

Warp's team-managed keys documentation shows the same custody intent for interactive and cloud-agent paths: secrets are sealed in the admin browser, re-encrypted with envelope encryption backed by Cloud KMS, never synced to member devices, and never delivered into cloud-agent environments. Member-local BYOK that lives only on a laptop cannot power server-side agents. Distinguish those modes in your own platform so "the engineer pasted a key in settings" is not mistaken for team-managed custody.

A minimal custody record looks like this:

credential_id: cred.openai.prod.2026q3
custody_mode: vault_reference          # not platform_plaintext
vault:
  manager: aws_secrets_manager
  secret_path: prod/llm/openai/api-key
  secret_key: OPENAI_API_KEY
  auth: assumed_role
provider: openai
environments: [production]
allowed_workspaces: [ws.platform, ws.support_triage]
virtual_keys:
  - vk.support_triage.prod
  - vk.eval.nightly
rotation_policy: 90d
last_rotated_at: 2026-09-01T00:00:00+03:00
break_glass: false

Keep break-glass credentials out of the default path. If you need an emergency key, store it under a separate reference with stricter approvals and a forced post-incident rotation.

Issue virtual keys with budgets, not master keys

Application code and agents should never see the master provider credential. They should see a gateway-issued virtual key bound to a workspace, environment, model allowlist, and budget. Augment describes virtual keys as the mechanism that maps a gateway token to one or more real provider keys while applications call a single endpoint. Fallbacks between OpenAI and Azure, or between regions, stay in gateway config rather than in every agent binary.

Budget the virtual key before you distribute it. Soft-alert before hard-stop. Separate experimentation pools from production pools so a prototype cannot starve a shipped workflow. Environment segmentation is non-negotiable: development, staging, and production must not share the same virtual key or the same vault reference. Otherwise load tests and production traffic become indistinguishable in provider logs and in your own attribution.

Close the bypass. If teams can still put a corporate card on a public API, the BYOK program is optional theater. Pair virtual-key distribution with the egress and gateway controls from the LLM gateway guide. Shadow keys are not only a security finding; they are evidence that your sanctioned path is too slow, too expensive, or missing a model.

Prove plaintext never reaches the agent

AI agent credential security already requires that the model never receives the reusable secret and that a broker signs destination-bound requests. BYOK does not relax that rule. It raises a sharper question: even when the enterprise "brings its own key," does any vendor hop still hold or log plaintext?

Ask vendors and your own platform team to draw the request path with decryption points marked. Acceptable answers look like: sealed upload, ciphertext at rest under CMK, decrypt only inside a short-lived gateway worker, inject into the provider TLS call, wipe memory, never write the key to logs or traces. Reject answers such as "we store your key encrypted with our KMS," "the agent runtime receives the key as an environment variable," or "cloud agents inherit the member's local key."

Warp documents one concrete boundary: team credentials decrypt only at the server-side inference boundary and never enter the cloud-agent worker environment. Copy that clarity into your architecture decision record. If your agents run tools that call providers directly, those tool runners need the same brokered injection path as the chat completion hop. Otherwise you have BYOK for the gateway and flat keys for the tools.

Rehearse revoke before you need it

Rehearse KMS revoke as an incident control before you need it, including the recovery path that restores service without pasting a new flat key. NIST SP 800-57 Part 1 treats keys as lifecycle objects with cryptoperiods, not as infinite config. OpenAI EKM depends on your KMS continuing to answer encrypt and decrypt; if you disable the grant or schedule key deletion, decryption stops. That is the point of CMK. It is also an availability risk if nobody has practiced recovery.

Augment calls out instant revocation mechanisms such as deleting imported key material or disabling a customer-managed key so further cryptographic operations fail until a deliberate restore. Build a drill around that:

1. Freeze new virtual-key issuance.
2. Disable or revoke the KMS grant used by the platform (or rotate the vault-referenced provider credential).
3. Confirm new completions fail closed and that ciphertext at rest remains unread.
4. Restore via the documented path: re-enable the grant, reimport material, or cut over to a pre-staged secondary credential reference.
5. Rotate anything that might have been exposed and write the incident timeline.

If the only recovery procedure is "paste a new key into the admin UI," you will recreate flat-key sprawl under pressure. Prefer dual references with controlled cutover, short data-plane cache TTLs so revoked secrets die quickly, and alerts when decrypt failure rate spikes.

Schedule ordinary rotation from the same playbook. Ninety days is a common starting cryptoperiod for many enterprise programs; frameworks differ on exact intervals, but auditors care that you have a written policy, evidence of execution, and CloudTrail or Key Vault diagnostic logs that show who used the key. Enable those logs before the first production agent ships.

Worked example: support-triage platform

A company runs one internal support-triage agent through a shared gateway. Security blocks production until custody is proven.

• API-key BYOK: one OpenAI production credential and one Anthropic fallback credential live in AWS Secrets Manager. Portkey-style secret references point at those paths. No application repository contains provider keys.
• Virtual keys: vk.support_triage.prod may call the approved chat models with a monthly budget and a hard stop. vk.support_triage.exp is a smaller pool for prompt experiments. Staging uses entirely separate references.
• CMK/EKM: conversation archives and eval fixtures are encrypted under a customer-managed KMS key in a bridge account. OpenAI EKM is enabled for the enterprise API project that holds customer content at the provider. Decrypt grants are limited to the gateway role and the archive service role.
• Agent runtime: the triage worker receives only the virtual key. Tool calls that need SaaS credentials go through the brokered path from credential security, not through environment variables.
• Drill: quarterly, platform disables the staging KMS grant, confirms fail-closed behavior, restores the grant, and records timing. Production drills use a secondary credential cutover rather than a hard revoke when customer impact risk is high.

Promote only when both BYOK columns are evidenced, virtual keys are budgeted, plaintext absence is documented, and at least one revoke-or-cutover drill has a dated record.

Promote-or-hold decision record

Hold the rollout if any of these remain true:

• The vendor or internal platform still stores provider secrets as platform-owned plaintext with no vault reference.
• "BYOK" was accepted without separate API-key and CMK/EKM evidence.
• Agents or tool runners can print, log, or inherit master credentials.
• Development and production share a credential or virtual key.
• There is no rehearsed revoke or cutover path that avoids emergency flat-key paste.
• Direct provider egress still bypasses the gateway, so custody and budgets are optional.

Promote when custody records exist for every provider credential, CMK/EKM wraps the sensitive stores that matter for your threat model, virtual keys enforce workspace budgets, plaintext-never-in-agent is an architecture invariant under test, and revoke drills have owners and dates. Then connect metering to cost management so the keys you finally trust also produce spend you can explain.

What to do next

List every path in your platform that can decrypt, inject, or print a provider credential or a customer-managed key grant. For each path, record custody mode, vault reference or KMS ARN, virtual keys that depend on it, and the last revoke drill. Close any path that still requires a pasted master key before you check the BYOK box on a security questionnaire.

References

1. OpenAI Enterprise Key Management (EKM) Overview - Official OpenAI documentation for customer-managed KEK envelope encryption with AWS KMS, GCP KMS, and Azure Key Vault.
2. NIST SP 800-57 Part 1 Revision 5 - Authoritative cryptographic key-management guidance for generation, storage, use, cryptoperiods, and destruction.
3. BYOK vs Managed AI Keys (LockLLM) - Separates API-key BYOK from encryption-key BYOK/CMK and documents flat-key blast radius.
4. BYOK for Enterprise Agent Rollouts (Augment) - Frames BYOK as the security-review unblocker and prefers virtual keys over per-developer sprawl.
5. Portkey Secret References - Gateway pattern for vault-referenced provider credentials without storing plaintext in the control plane.
6. Warp Team-Managed LLM API Keys and Endpoints - Admin-configured server-side keys that never sync to devices or cloud-agent environments.
7. Fire in Belly AI agent credential security - First-party baseline for keeping reusable secrets out of agent context.
8. Fire in Belly LLM gateway for internal AI workflows - First-party gateway and egress baseline this BYOK contract assumes.

---