Disclosure up front: I build VaultAgent, so this is my project. Not trying to be sneaky — posting because the pattern I keep seeing is worth discussing even if you never touch my thing.
Context: we work with mid-market SaaS eng orgs (roughly 50-500 engineers) who are trying to ship LLM features while compliance is actively reviewing them. The consistent blocker isn't the model choice or the prompt layer. It's that the default AI gateway pattern is a shared multi-tenant proxy, and when the auditor asks three questions, it falls apart:
- Can you prove my tenant's prompts and completions were never co-resident with another customer's in logs or cache? On most shared proxies, the honest answer is 'sort of.'
- What's the retention window, and can you set it per-workload? Usually it's one global number the vendor picked.
- Can I get the raw, timestamped request/response log in a format my SOC-2 auditor will accept without a bespoke export script? Almost never out of the box.
I've watched teams lose 6-8 weeks of feature velocity on exactly this — not because the feature was risky, but because the logging and isolation story wasn't auditor-ready. The eng team ends up writing their own logging shim, then security rejects it, then it loops.
What actually unblocks these reviews in my experience: dedicated tenant isolation (not just a namespace), audit logs structured for SOC-2 evidence collection from day one, retention windows you control per project, and a human on support who responds in hours, not a Discord. That's the bet behind the Pro tier we just opened at $2500/mo — details here if useful: https://mindsparkstack.com/vault?utm_source=reddit&utm_medium=organic&utm_campaign=vaultagent-pro-launch
More interested in the discussion though — how are you handling the isolation + audit-log gap right now? Rolling your own? Living with a shared proxy and hoping? Curious what's actually working in your review cycles.