Disclosure up front: I built a thing to solve this for us, and I'll link it at the bottom. Not trying to be sneaky — posting here because I want to hear how others are handling it.
We're a SOC-2 SaaS. Last quarter we tried to ship a Claude-backed feature to two enterprise customers. Both blocked it in security review. The objections weren't vague — they were specific:
- Prove zero retention end-to-end, not just "the model provider says so." They wanted our own logs to show prompts/responses aren't persisted anywhere we control.
- Per-tenant isolation. One customer's data cannot share a process, a cache, or a prompt context with another's, ever.
- An audit trail their own SOC 2 auditor could read without us writing a custom export.
We spent ~6 weeks trying to bolt this onto our existing proxy. It was miserable. Logging middleware kept accidentally capturing payloads. Tenant isolation via namespaces wasn't convincing to their reviewers — they wanted hard process boundaries. And our "audit trail" was a Datadog dashboard, which no auditor accepts.
So I ended up building it as a standalone proxy. Zero-retention by construction (payloads never hit disk, only hashed metadata for the audit log), per-tenant Docker containers so isolation is a kernel boundary not a config flag, and an audit export that maps to CC7/CC8 controls directly. Test suite is 159/159 green and it's running in front of our own product at vault.mindsparkstack.com.
If it's useful to anyone else stuck in the same review loop, it's here: https://mindsparkstack.com/vault?utm_source=reddit&utm_medium=organic&utm_campaign=vaultagent-launch — $999/mo, happy to answer technical questions in the thread.
More interested in the discussion though: how are you all getting LLM features past enterprise security? Are reviewers accepting provider-side zero-retention attestations, or are they demanding you prove it at your own perimeter like ours did? Curious whether this is a fintech/healthtech-specific pattern or everyone's hitting it now.