AI can help organizations search knowledge, automate workflows, and build new digital products. But the privacy and security decision is broader than choosing a model: teams must also understand what enters the system, what it can access, and where its outputs go.
This guide offers a practical starting point for buyers and delivery teams. It focuses on generative AI, while many of the governance questions also apply to other AI systems.
Map the data before selecting a tool
Start with a specific business task and identify the minimum information needed to perform it. A support assistant, for example, may need an approved product guide but not a full customer database. Map the journey from source systems through prompts, file uploads, retrieval indexes, model providers, outputs, logs, and backups.
For each stage, record the data owner, sensitivity, access permissions, location, retention period, and reason for processing. Distinguish using data to answer a request from using it to train or improve a model. The UK Information Commissioner’s Office discusses why training and deployment purposes should be considered separately in its response on purpose limitation in the generative AI lifecycle.
Remove unnecessary fields and use representative test data where practical. Masking identifiers can reduce exposure, but it does not automatically make data anonymous. Check applicable requirements with qualified privacy or legal advisers; obligations depend on the jurisdictions, data, and use case.
Choose a deployment model deliberately
A hosted model API can reduce infrastructure work, but information sent to it crosses a provider boundary. A privately hosted or self-hosted model may give an organization more direct control over its environment, while shifting more responsibility for infrastructure, updates, monitoring, and access management to its team. Neither choice is inherently secure.
Compare options against the actual workflow, budget, performance, sensitivity, and operating capacity. Ask providers and implementation partners:
- Are prompts, files, outputs, or logs retained? For how long, and can retention settings be changed?
- Is customer data used for model training or service improvement? Which product-specific terms and settings apply?
- Where are data processed and stored, including by subcontractors? What transfer arrangements are relevant?
- What access, encryption, incident notification, deletion, and audit evidence are available?
- What happens to indexes, backups, and other copies when the service ends or data must be removed?
Request written answers and test the configuration rather than relying on a general product description. Deletion from a source system does not necessarily remove every downstream copy; agree how each copy will be handled.
Secure the application around the model
Generative AI adds risks to familiar application-security concerns. OWASP describes prompt injection, where instructions in user input or external content can redirect a model’s behavior. A retrieved document or email can therefore be both useful information and an untrusted input.
Apply authorization outside the model: limit what each user and service can retrieve or change, validate outputs before they reach other systems, and require human approval for consequential actions. Do not treat instructions in a prompt as an access-control boundary. If the application searches internal documents using retrieval-augmented generation, preserve source permissions in the retrieval layer and test cross-user access. OWASP also identifies risks in vector and embedding systems, including unauthorized retrieval.
Protect credentials, encrypt data in transit and at rest where appropriate, and restrict operational logs: logs can contain sensitive prompts or outputs. The possibility of sensitive information disclosure also makes output filtering and access checks worth testing, without assuming either is foolproof.
Set a launch gate and an operating plan
Before production, assign a business owner and agree on the risks that would stop a launch. Test with realistic scenarios: a user requesting another team’s document, a malicious instruction inside a retrieved file, a sensitive value appearing in an output, and an attempted action beyond the assistant’s permissions. Record results, fix failures, and retest.
After launch, monitor access and unusual activity without collecting more personal information than necessary. Set a review schedule for provider changes, permissions, retention, model behavior, and incident response. Define who can disable the integration, investigate an exposure, and notify affected parties when required. NIST’s Generative Artificial Intelligence Profile offers a voluntary framework for managing risks across the AI lifecycle.
Make responsibilities part of the partnership
Buyers evaluating a delivery partner should request a documented architecture and data-flow map, a control and test plan, and a handover covering ownership after launch. Agree who approves data sources, configures the provider, maintains integrations, reviews incidents, and pays for ongoing operation. Strategic partners and investors assessing an opportunity can use the same evidence to examine whether the proposed delivery model has credible operating responsibilities—not just a compelling demonstration.
Altweeb’s work across software development, technology consulting, and AI automation is relevant to organizations planning these delivery choices. The appropriate scope and controls should be agreed for each project rather than assumed.
Conclusion
A secure AI implementation starts with a defined purpose, limited data access, and clear provider responsibilities. Test those decisions before launch and revisit them as the system and its environment change.
To explore this further, see AI Automation for Business Optimization.
Add your perspective