AI & LLM Security Testing
AI systems are creating a new layer of application risk. Test models, prompts, agents, tools and data flows as part of the modern attack surface.
AI features are increasingly wired into real applications — reading documents, calling internal APIs, querying knowledge bases and taking actions on a user’s behalf. That connectivity is what makes them useful, and it is also what makes them a security surface.
The distinguishing property is that instructions and data arrive through the same channel. A model reading a retrieved document cannot inherently tell the difference between content it should summarise and text written to be followed. Traditional application security models were not designed around that.
Risk concentrates where the model stops producing text and starts triggering actions.
- USERUserThe person or system making the request
- PROMPTPromptSystem instructions, user input and injected contextUntrusted content enters here
- MODELModelThe model and its configured behaviour
- AGENTAgentPlanning, autonomy and decisions about what to do nextText becomes action here
- TOOLSToolsFunctions, APIs and integrations the agent can call
- DATADataKnowledge sources, retrieval and records reachable through tools
- APPApplicationWhat the surrounding application does with the result
Conceptual view of a typical AI application. Assessment is scoped to the systems you authorize.
Focus areas
Where AI risk actually concentrates
A model that only returns text has a bounded blast radius. The picture changes once the same model can call a tool, query a database or send a message — the output stops being a suggestion and becomes an action.
So the practical question is rarely whether a model can be talked into saying something it should not. It is what the surrounding system permits when it does: which tools are reachable, whose data those tools can retrieve, and what happens downstream without a human in the loop.
Not every AI deployment carries meaningful risk. A narrow, read-only feature with no tool access is a very different proposition from an agent with write access to production systems, and assessment should be scoped accordingly.
How we assess AI systems
- 01MAPEstablish the request path: inputs, context sources, tools and downstream effects.
- 02TRUSTIdentify where untrusted content enters and what it can influence.
- 03PROBETest instruction handling, restrictions and boundary enforcement.
- 04TOOLSAssess what connected tools permit and how arguments are constrained.
- 05IMPACTEstablish what a successful influence attempt would actually achieve.
- 06REPORTFindings with reproduction detail and mitigation guidance.
Organizations this typically applies to.
Testing is scoped per engagement. Nothing here implies industry-specific certification or accreditation.
When Does an AI Feature Warrant Assessment?
A report your team can actually act on.
Exact deliverables and their format are confirmed during scoping.
What was assessed, what was found and what it means, written to be read by people who will not read the technical detail.
Each finding described with enough precision for an engineer to locate and understand it.
Reproduction detail and supporting evidence, so findings can be verified rather than taken on trust.
Severity considered against your environment, not only against a generic scoring table.
Practical direction on addressing each finding, including where a change belongs architecturally.
Verification that addressed findings no longer reproduce, within the agreed retest scope.
Adjacent parts of the attack surface.
Questions we get asked before an engagement.
No. The focus is your application — how it constructs prompts, what context it injects, which tools it exposes and what it does with the output. Foundation model behaviour is relevant context, but the controls you own are in the surrounding system.
Usually yes, and often more than expected. The provider secures the model; your application still decides what goes into the prompt, what data is retrieved, which tools are callable and what happens with the response. That is where assessable risk tends to live.
It is not currently a solved problem, and we would be cautious of anyone claiming otherwise. The practical approach is to reduce what a successful injection can achieve — constraining tool access, enforcing authorization outside the model, and keeping consequential actions behind explicit confirmation.
No. It depends on what the feature can reach. A read-only assistant over public content is low risk; an agent with access to customer data or the ability to act on internal systems warrants assessment. We would rather scope honestly than test something that does not need it.
Explore AI Security Testing
Tell us what your AI feature can read, call and act on. That determines whether assessment is worthwhile and, if it is, what it should cover.