Keeping Company Data Private With Custom AI — And What To Do About It
How to ensure your company's documents stay private when building a custom AI assistant — the architectural choices that determine data control.
As the founder of Perceptra, a Mumbai digital growth studio, I work with real businesses on these challenges every week. This guide is written for owners and decision-makers, not engineers.
The data privacy problem generic AI creates
The three architectural options and their privacy implications
Option 1: Consumer AI products (ChatGPT, Claude.ai, Gemini). Documents pasted in are processed on the provider's servers. OpenAI, Anthropic, and Google each have data use policies for consumer products — but these policies generally include provisions for using conversation data to improve their models. Not appropriate for confidential business documents.
Option 2: Enterprise API with data processing agreements. Using the same AI models (GPT-4, Claude) via their enterprise API, with a signed Data Processing Agreement (DPA) that specifies your data will not be used for model training. This is significantly better than consumer products for data privacy, though your documents still transit through the provider's API infrastructure.
Option 3: Self-hosted LLM on your own infrastructure. Running an open-source LLM (LLaMA, Mistral, or similar) on your own servers means your documents never leave your infrastructure. Maximum data control; typically somewhat lower model capability than the frontier models (GPT-4o, Claude 3.5) at equivalent hardware cost.
What "private" actually means in each option
Consumer products: Not private. Documents may be used for training, accessible to provider employees for safety review, and subject to the provider's evolving terms of service.
Enterprise API with DPA: Better protected. Your data is not used for training under the DPA. Still transits provider infrastructure; the provider can still access data for breach investigation, legal compliance, and similar purposes.
Self-hosted: Fully private. Documents never leave infrastructure you control. You own the security responsibility entirely — backup, updates, access control.
The practical recommendation for most Mumbai businesses
For most business use cases (HR policies, internal procedures, product specifications), an enterprise API with a signed DPA is the right balance — meaningfully better data protection than consumer products, without the operational overhead of self-hosting.
For genuinely sensitive categories — client-privileged legal matter documents, financial client data under regulatory protection, health information — self-hosted infrastructure should be seriously evaluated.
What your custom AI assistant data privacy architecture should specify
- Where documents are stored (your servers, a private cloud, or a third-party vector database with appropriate DPA)
- Which LLM API is used, under what agreement
- Who has access to the knowledge base and query logs
- How long query logs are retained and for what purposes
- How document updates and deletions are handled
Frequently asked questions
No — it reduces some risks by keeping data within your controlled infrastructure, but compliance with data protection regulations depends on your overall data handling practices, the type of data stored, and whether the data subjects have been informed. If the knowledge base includes personal data about employees or clients, your data protection obligations apply to the AI system's handling of that data.
Key clauses to check: prohibition on using your data for model training; restrictions on employee access to your data; breach notification requirements; data deletion obligations at contract end. If your organisation handles particularly sensitive data, legal review of the DPA by a qualified privacy attorney is worthwhile.
Enterprise API agreements typically include retention limits on query data. Review your specific provider's terms. If retention of any kind is unacceptable for your data category, self-hosted infrastructure is the only option that eliminates this concern entirely.
Ready to Build
This For Your Business?
Book a strategy session. We scope your first project in 30 minutes, no jargon, no obligation.