Adding an AI assistant to your product? You’re in good company. Let’s answer the questions teams ask us most before they launch.
Treat model input and output as untrusted?
Anything the model reads — user messages, documents, web pages — can contain instructions. Anything it writes can contain unexpected content. Validate and handle both carefully.
Limit what the AI can reach?
Give the assistant access only to the data and tools it truly needs, scoped to the current user. Require confirmation before actions with real-world impact.
Test against a recognized list?
The OWASP Top 10 for LLM Applications is a good baseline for testing. Combine it with monitoring of usage and cost, and retest whenever prompts, tools or data sources change.
Questions to ask before launch?
What data can the assistant read? What actions can it take? Who reviews its output before it reaches a customer? What happens if it is asked to do something outside its purpose? Clear answers to these four questions prevent most issues.
Keep humans in the loop?
For refunds, account changes or anything with financial impact, require a human or explicit user confirmation. AI should assist decisions, not make irreversible ones on its own.
What you should do
- Document which data and tools your AI feature can reach
- Scope data access to the current user
- Require confirmation before high-impact actions
- Set usage and cost limits
- Monitor prompts and responses for abuse
What our team does
- Review your AI architecture and threat model
- Test against the OWASP Top 10 for LLM Applications
- Check data access and tool permissions
- Recommend practical guardrails
- Retest after changes
- Run a pre-launch AI threat-modeling session
- Test data access separately for each user role
- Advise on monitoring of AI usage and cost
Jargon buster
- LLM
- Large language model — the AI that powers chatbots like the one on your site.
- Prompt
- The instructions and text given to the AI.
- Guardrails
- Rules and checks that keep the AI from doing or saying the wrong thing.
AI moves fast. A short test before launch is far cheaper than a public mistake after it.
— The CRIS InfoTech security team