Independent LLM security audit
Find out what your chatbot, RAG or agent can be talked into doing
An LLM security audit tests the application around the model: the prompt, the documents it retrieves, the tools it can call, the output it renders and the logs it leaves behind. This page sets out what is tested, what the report has to contain, and which EU AI Act duties that evidence answers.
Written by the OffSeq security team. The method, the report structure and the article mapping below are the ones used on paid engagements; reading them costs nothing.
§ 01 Definition
What an LLM security audit is, and what it is not
The phrase is used for at least four different things. On this site it means one: a security assessment of a deployed LLM application, carried out adversarially, with findings that can be reproduced from the transcript.
In scope
- The instruction boundary. Whether user text, retrieved documents, emails, tickets, web pages or tool descriptions can rewrite what the model has been told to do.
- The data the model can reach. Retrieval indexes, uploaded files, conversation memory, tenant boundaries, and what leaks back out through an answer.
- The actions the model can take. Function calls, MCP servers, code execution, outbound HTTP, email and ticket writes, and the identity each of them runs as.
- The way output is handled. Markdown, HTML and image rendering, link and URL construction, and downstream systems that trust model output as data.
- The surrounding platform. API authentication, vector database access control, secret handling, rate limits, and the logs an incident would be reconstructed from.
- The governance record. Whether the evidence produced can answer Articles 15, 26 and 50 of the AI Act, and whether the logs meet the six-month deployer floor.
Not in scope
- A model benchmark. We do not rank base models, measure reasoning quality or score accuracy on a public dataset.
- A bias or fairness audit. Discrimination testing is a separate discipline with separate expertise; an LLM security audit does not substitute for it.
- A brand-visibility check. Several products sell an "LLM audit" that measures how often a model mentions your company. That is marketing analytics, not security.
- A certification. No one certifies LLM applications today; prEN 18286, the quality management standard being drafted for AI Act purposes, is still in development.
- A one-off guarantee. Prompt injection has no patch. An audit measures exposure at a point in time and verifies the controls that hold when the model is fooled.
The distinction matters for procurement. If a proposal promises to "make the model safe" rather than to test what the application does when the model is manipulated, it is describing a product, not an audit.
§ 02 Applicability
Three deployments that need this, and why
The trigger is not the size of the model. It is the combination of data the assistant can reach and actions it can take on behalf of someone who did not authorize them.
- PROFILE A A customer-facing chatbot Anything a member of the public can type into. The attacker controls the input directly, the assistant usually holds a system prompt with business logic, and it often has a retrieval index of internal documentation behind it. Since 2 August 2026, providers of systems that interact directly with people must also make clear that the counterpart is an AI system.
- PROFILE B An internal copilot over confidential data A RAG assistant across a document store, a mailbox, a ticket system or a code repository. The attacker never talks to the model: they place instructions in a document the model will later read. This is the shape of EchoLeak and of the Slack AI retrieval attack, and it is the hardest class to detect from logs.
- PROFILE C Agents with tools and MCP servers Anything that can act: function calling, Model Context Protocol servers, coding agents, browser agents. Here a successful injection is not a disclosure, it is an unauthorized transaction. Third-party MCP servers add an unreviewed supply chain to an already trusted execution path.
§ 03 Coverage
What an audit actually tests
Eight control areas, each tied to a published identifier so the report can be checked against something other than our own opinion. Every finding is filed against one of them.
- LLM01 Prompt injection, direct and indirect Instructions are seeded into every channel the model ingests: the chat box, uploaded files, retrieved documents, HTML fetched from the web, ticket bodies, commit messages, and the descriptions of the tools themselves. Each attempt is repeated across runs, because temperature alone changes whether a guardrail fires.
- LLM02 · LLM07 Sensitive information and system prompt disclosure What the assistant will reveal about its own configuration, its retrieval sources, its tool inventory and other users. System prompt extraction matters less for the words than for what those words disclose: internal endpoints, allow-lists, business rules and, too often, credentials.
- LLM08 · LLM04 Retrieval poisoning and vector store weaknesses Whether a document a low-privileged user can write reaches the index, whether embeddings from separate tenants share a namespace, whether retrieval respects the same access control as the underlying store, and what the assistant answers when a poisoned passage outranks the true one.
- LLM06 · ASI02 Excessive agency, tool and MCP abuse The complete inventory of callable actions and the identity each one runs as. We test tool invocation from untrusted content, argument tampering, chained calls, approval fatigue, and whether a tool description can itself carry instructions. MCP servers are reviewed as dependencies, with their transport and authorization handling.
- LLM03 · ASI04 Supply chain and model provenance Where the model weights, adapters, embeddings, prompt templates, agent frameworks and MCP servers come from, how they are pinned, and who can change them without review. An AI bill of materials is the deliverable most teams find they cannot produce.
- LLM05 Output handling and rendering Model output is untrusted input to whatever consumes it. We test markdown and HTML rendering, image and link construction, and any downstream parser that treats a response as structured data. This is the channel that turns a disclosure into an exfiltration.
- LLM10 Unbounded consumption and abuse Cost and availability, treated as security properties: per-user and per-tenant limits, context inflation, recursive agent loops, and whether an unauthenticated visitor can drive inference spend. Agent loops make this materially worse than a classic rate-limit test.
- ART. 15 · 26 Governance and AI Act evidence The audit closes by mapping what was tested onto the obligations that apply. Article 15 requires high-risk systems to be resilient against attempts to alter their use, outputs or performance, and names data poisoning, model poisoning, adversarial examples and confidentiality attacks explicitly. Article 26 requires deployers to keep automatically generated logs for at least six months.
The OWASP Top 10 for LLM Applications 2025 and the OWASP Top 10 for Agentic Applications 2026 are published by the OWASP GenAI Security Project; MITRE ATLAS identifiers come from the public ATLAS data set. Both are cited so the coverage claim is auditable.
§ 04 Evidence log
What has actually gone wrong, with dates
None of these were model failures in the abstract. Each was an application that let untrusted content reach a model that held private data and could reach the outside world.
-
GitHub MCP toxic agent flow
An instruction placed in a public GitHub issue was read by an assistant connected to the GitHub MCP server, which then copied private repository content into a public pull request. The MCP server was not vulnerable; the agent architecture was.
-
EchoLeak, Microsoft 365 Copilot
A crafted email, never opened by the victim, was processed by Copilot retrieval and caused it to disclose data over the network. Microsoft scored the AI command injection at 9.3; NVD scored the same record at 7.5. Fixed server-side; no exploitation in the wild was reported.
-
Supabase MCP via a support ticket
A support ticket carrying instructions was read by an assistant whose database MCP connection ran with the service role, bypassing row-level security. It read an integration token table and wrote the secrets back into the ticket thread.
-
mcp-remote command injection
Connecting a client to an untrusted MCP server allowed operating-system command injection through a crafted authorization endpoint URL in the server response. A dependency in the agent supply chain, not a prompt problem.
-
Perplexity Comet agentic browser
Hidden text in a Reddit comment hijacked the browsing agent. The published proof of concept read the user's email address and a one-time code from webmail and posted them back as a reply. Brave recommended separating trusted from untrusted context and requiring explicit confirmation for sensitive actions.
-
ForcedLeak, Salesforce Agentforce
A Web-to-Lead form field carried the injection. CRM records were exfiltrated as image requests to a previously allow-listed domain that had expired and was re-registered for about five dollars. Patched by enforcing trusted URLs.
-
GTG-1002 AI-orchestrated intrusion campaign
Anthropic reported a state-sponsored group driving an agent with MCP tooling through reconnaissance, exploit generation, credential harvesting and exfiltration, with the model performing an estimated 80 to 90 percent of the campaign and humans at four to six decision points. Roughly thirty targets; a small number were breached.
§ 05 Method
How the engagement runs
Five phases. Each one produces an artifact that ends up in the report, so a reader can see what was covered and, just as importantly, what was not.
- Architecture and data-flow intake A working session that maps every input channel, every retrieval source, every tool and MCP server, memory, and every outbound path. Most of the eventual findings are visible on this diagram before a single payload is sent. Leaves behind A data-flow diagram with trust boundaries and the tool inventory
- Threat model The architecture is turned into a ranked list of applicable OWASP LLM and Agentic risks with their MITRE ATLAS techniques, and into the test plan. This is the step the bench on this page reproduces in miniature. Leaves behind A test plan tied to named risk identifiers, agreed before testing
- Adversarial testing Manual red teaming against the full payload taxonomy in the OWASP AI Testing Guide, supported by tooling of the garak and promptfoo class for breadth. Multi-turn, multi-language, multimodal and cross-tenant cases are run repeatedly, because a single pass measures luck rather than exposure. Leaves behind Reproducible transcripts, one per finding, with run counts
- Platform and integration review API authentication and authorization, vector database access control, MCP server configuration and transport, secret handling, egress rules, rate limits and the logging pipeline. This is ordinary application security work, and it is where the durable fixes live. Leaves behind Configuration findings with the exact setting and the fixed value
- Report, retest and mapping Findings with severity and reproduction steps, a coverage matrix against OWASP and ATLAS, the AI Act evidence pack, and an executive summary that states residual risk in plain language. A retest window is included; the retest letter records what was verified fixed. Leaves behind The report, the coverage matrix, the evidence pack, the retest letter
§ 06 Deliverables
What lands on your desk
A finding that cannot be reproduced is an opinion. Everything below is written so an engineer can re-run it and a regulator can read it.
- D1 Findings with reproduction transcripts Each finding carries the exact input, the channel it entered through, the model response, the run count, and the impact demonstrated. Severity reflects what the assistant could reach and do, not the novelty of the payload.
- D2 OWASP coverage matrix Every LLM01 to LLM10 and, for agentic systems, every ASI01 to ASI10 entry marked tested, not applicable or not tested, with the reason. The blanks are the part that makes the matrix useful.
- D3 MITRE ATLAS technique mapping Findings mapped to ATLAS technique identifiers so that detection engineering and threat intelligence can consume them alongside ATT&CK.
- D4 AI Act evidence pack A record of the robustness and cybersecurity testing performed against Article 15, a logging check against Articles 12 and 26 including the six-month retention floor, an Article 50 disclosure check, and input for the Article 6(4) classification memo.
- D5 Executive summary Two pages that a board can read: what was tested, what was found, what the residual risk is, and what changes when the recommendations are implemented. No severity inflation, no scores invented to fill a chart.
- D6 Retest and letter A retest of the fixed findings within the agreed window, and a letter stating what was verified. Useful for customers and insurers who ask for one, and honest about anything still open.
§ 07 Scope drivers
What decides the effort
There is no useful list price for this work, and any vendor publishing one is quoting for a scope they have not seen. These are the variables that actually move the number.
- S1 Number of applications and models One chatbot on one model is a different engagement from six assistants sharing a gateway, three model providers and a self-hosted fallback.
- S2 Feature surface Retrieval, tools, memory, browsing, code execution and multi-agent orchestration each add a class of tests. The bench below lists which ones your selection triggers.
- S3 Access level Black box takes longer and finds less. Grey or white box with the system prompt, the tool schemas and a staging environment produces materially better coverage for the same days.
- S4 Environment and tenancy Testing in production constrains destructive cases and needs a written rules-of-engagement annex. Multi-tenant deployments add cross-tenant isolation testing that single-tenant ones do not need.
- S5 Compliance driver An AI Act evidence pack, a DORA threat-led exercise or a customer security questionnaire each change the documentation burden, sometimes more than the testing itself.
- S6 Retest and cadence A single assessment is a snapshot. Model version changes, prompt changes, new retrieval sources and new tools each invalidate part of it, which is why retesting is scoped up front.
As a rough shape rather than a quotation: a single chatbot is a matter of days, an agent and MCP estate a matter of weeks. Drafting a scope takes about ten minutes and produces a brief you can send to any provider, not only to OffSeq.
§ 08 Interactive
The bench: threat model and obligation finder
Two working tools. The first turns an architecture into the risks and tests that apply to it. The second turns a role and a use case into the AI Act articles and dates that bind you. Nothing you select leaves your browser.
Interactive mode is not available. You can read the full reference content below. No answers are assessed and no result is calculated.
Both tools need JavaScript. The reference material behind them is below in full: the risk catalog the threat model draws from, and the regulatory dates the obligation finder uses.
OWASP Top 10 for LLM Applications 2025, the catalog every finding in an audit is filed against:
- LLM01:2025 Prompt Injection – user or retrieved content rewrites the model's instructions.
- LLM02:2025 Sensitive Information Disclosure – the assistant reveals data it should not.
- LLM03:2025 Supply Chain – models, adapters, frameworks and MCP servers of unverified provenance.
- LLM04:2025 Data and Model Poisoning – training, fine-tuning or retrieval data manipulated by an attacker.
- LLM05:2025 Improper Output Handling – model output consumed as trusted input downstream.
- LLM06:2025 Excessive Agency – more tools, permissions or autonomy than the task needs.
- LLM07:2025 System Prompt Leakage – configuration, rules and secrets recovered from the prompt.
- LLM08:2025 Vector and Embedding Weaknesses – retrieval poisoning and cross-tenant leakage.
- LLM09:2025 Misinformation – confident output relied on without verification.
- LLM10:2025 Unbounded Consumption – cost and availability abuse, including agent loops.
EU AI Act dates as they stand after Regulation (EU) 2026/1744, published in the Official Journal on 24 July 2026:
- 2 February 2025 – prohibited practices and the AI literacy duty apply.
- 2 August 2025 – general-purpose AI model obligations and the governance rules apply.
- 2 August 2026 – general application, including the Article 50 transparency duties.
- 2 December 2026 – the four-month marking grace for systems placed on the market before 2 August 2026 ends, and two further prohibited practices apply.
- 2 December 2027 – high-risk obligations for Annex III systems apply, moved from 2 August 2026.
- 2 August 2028 – high-risk obligations for Annex I product-embedded systems apply.
Threat model
The three ingredients present in every published agent incident:
- Present Not indicated Private data in reachSomething worth stealing is inside the context window
- Present Not indicated Untrusted content in contextContent an attacker can write reaches the model
- Present Not indicated Outbound channelThe model can move data or act outside the chat
Full path
A complete exfiltration path exists
All three ingredients are present at once, which is the exact shape of EchoLeak, the Supabase MCP case and ForcedLeak. Testing should start with an end-to-end exfiltration attempt, not with individual payloads.
Partial path
Two of three ingredients are present
One link is missing today. That link is usually added by the next feature request, so the architectural controls are worth putting in before it is, rather than after.
Single
One ingredient is present
Exposure is currently limited to what a manipulated answer can do on its own: misinformation, disclosure of the prompt, and abuse of inference cost. Still worth testing, at a smaller scope.
Model only
No architectural path indicated
On this selection the assistant is a text interface with no private data, no attacker-writable content and no way to act. The remaining risks are the ones that apply to every deployment: prompt disclosure, output handling and consumption.
13 risk areas apply to this architecture
Share this threat modelOpens the OffSeq scoping brief. Your selections stay in this page; copy the applicable identifiers into the brief if you want them included.
- LLM01:2025 Direct prompt injection Audit test The full payload taxonomy from OWASP AITG-APP-01: role play, context hijacking, obfuscation and token smuggling, multi-language, payload splitting, structured-output attacks and multi-turn escalation, each repeated across runs.
- LLM01:2025 Indirect injection through ingested content Audit test Instructions are planted in every channel the model reads without a human deciding to read it: a document in the index, a page it will fetch, an uploaded file, a ticket body, an MCP tool description. Delayed-trigger payloads are included.
- LLM07:2025 System prompt and configuration leakage Audit test Direct and oblique extraction of the system prompt, tool schemas, retrieval configuration and guardrail wording, followed by an assessment of what the recovered text discloses: endpoints, allow-lists, business rules, credentials.
- LLM02:2025 Sensitive information disclosure Audit test Whether the assistant will surface documents, fields or other users' data that the requesting identity has no right to, including through summarization, translation and quotation rather than direct retrieval.
- LLM08:2025 Retrieval poisoning and vector store weaknesses Audit test Who can write into the index, whether retrieval enforces the same access control as the source store, whether a planted passage can outrank the authoritative one, and whether embeddings from different tenants share a namespace.
- LLM04:2025 Data poisoning through user-supplied content Audit test Whether content submitted by a low-privileged user reaches a fine-tuning set, a feedback loop, a shared memory or an index that higher-privileged users query, and what review stands between submission and ingestion.
- LLM05:2025 Output handling and rendering exfiltration Audit test Markdown and HTML rendering, image and link construction, and any downstream consumer that parses the response. The question is whether a manipulated answer can cause a request to leave the browser or the server.
- LLM06:2025 · ASI02:2026 Excessive agency and tool misuse Audit test A complete inventory of callable actions, the identity each runs as, and whether untrusted content can trigger one. Argument tampering, chained calls and write actions with no human confirmation are tested individually.
- ASI01:2026 Agent goal hijack Audit test Whether the agent's objective itself can be rewritten mid-run by retrieved content, a subtask result or a tool response, and whether any checkpoint compares the current plan against the original human instruction.
- ASI03:2026 Identity and privilege abuse across tools Audit test Whether each tool call carries the end user identity or a shared service identity, how tokens are stored and scoped, and what a single compromised tool call can reach in the systems behind it.
- ASI04:2026 · LLM03:2025 Agentic supply chain and model provenance Audit test Provenance and pinning for models, adapters, embeddings, prompt templates, agent frameworks and MCP servers; who can change each without review; and whether an AI bill of materials can be produced at all.
- ASI05:2026 Unexpected code execution Audit test Sandbox boundaries, filesystem and network reach from executed code, whether configuration files inside a repository can be written by the agent, and whether generated code runs before a human sees it.
- ASI06:2026 Memory and context poisoning Audit test Whether a single conversation can write a durable instruction into memory, whether that memory is shared across users or sessions, and whether anything expires, is reviewed or can be attributed to its source.
- ASI09:2026 Human-agent trust exploitation Audit test How confirmations are presented, whether the approval dialog shows the real action and its real arguments, and how many approvals a normal working day produces before the human stops reading them.
- LLM02:2025 · LLM08:2025 Cross-tenant leakage Audit test Tenant separation in the vector store, the cache, the conversation history and the logs; and whether a tenant identifier is enforced server-side rather than carried in a prompt or a client-supplied parameter.
- LLM10:2025 Unbounded consumption Audit test Per-user, per-tenant and global limits; context inflation; recursive agent loops; and whether an unauthenticated visitor can drive inference spend or exhaust a shared quota.
- LLM09:2025 Misinformation and unsafe reliance Audit test Whether answers are grounded in retrievable sources, whether citations resolve, and what the interface does when the model has no basis for an answer. Assessed as a security property where the output drives a decision or a transaction.
Obligations
Profile Deployer · Customer-facing chatbot · SME
4 obligations already in force for this profile
0 more with a future application date
Maximum administrative fines under Article 99:
- EUR 35M / 7%Prohibited practices under Article 5
- EUR 15M / 3%Provider, deployer and Article 50 transparency breaches
- EUR 7.5M / 1%Incorrect, incomplete or misleading information to authorities
The audit produces the technical evidence listed against each obligation. It does not replace legal review of your classification.
- ART. 4 Support AI literacy among the people who operate the system Providers and deployers must take measures to support the development of AI literacy of their staff and others operating the system on their behalf, in proportion to their knowledge and the context of use. Regulation (EU) 2026/1744 replaced the original "shall ensure a sufficient level" wording with this softer duty, and it does not require any specific level for any individual.
- ART. 5 Do not operate a prohibited practice The prohibitions in Article 5 have applied since 2 February 2025 and carry the highest fine tier. Regulation (EU) 2026/1744 added two further prohibited practices that apply from 2 December 2026.
- ART. 3(3) Check whether the disclosure duty is actually yours A provider is anyone who places a system on the market or puts it into service under their own name or trademark. A company that presents a supplied assistant to its customers as its own is frequently the provider of that system, and inherits the Article 50(1) and 50(2) duties with it. Settle this before assuming the vendor carries them.
- ART. 50(1) Tell people they are interacting with an AI system Providers must design systems that interact directly with natural persons so that those persons are informed they are dealing with an AI system, unless that is obvious to a reasonably well-informed, observant and circumspect person. If you put a third-party assistant into service under your own name or trademark, you are the provider of that system.
- ART. 50(2) Mark generated content in a machine-readable format Providers of systems that generate synthetic audio, image, video or text must mark the output as artificially generated or manipulated, effectively and interoperably. Systems already on the market before 2 August 2026 had until 2 December 2026 to comply.
- ART. 50(3) Inform people exposed to emotion recognition or biometric categorization Deployers of an emotion recognition system or a biometric categorization system must inform the natural persons exposed to it of its operation, and process the personal data under the applicable data protection law.
- ART. 50(4) Disclose deepfakes and unreviewed public-interest text Deployers must disclose image, audio or video content constituting a deep fake, and must label AI-generated or manipulated text published to inform the public on matters of public interest where no human review or editorial responsibility applies.
- ART. 25 You are treated as the provider Putting your name or trademark on a high-risk system, making a substantial modification to it, or changing the intended purpose of a system, including a general-purpose one, so that it becomes high-risk, makes you the provider and moves the Article 16 obligations to you.
- ART. 6(2) The system falls in an Annex III high-risk area Annex III area 4 covers recruitment, selection, promotion, termination, task allocation and performance monitoring. Area 5 covers creditworthiness and credit scoring, and risk assessment and pricing for life and health insurance. A narrow derogation exists under Article 6(3), but a system that performs profiling of natural persons is always high-risk.
- ART. 6(4) Document the assessment if you conclude it is not high-risk A provider who considers that an Annex III system is not high-risk must document that assessment before the system is placed on the market or put into service, and is subject to the registration obligation in Article 49(2).
- ARTS. 9, 12, 15 Risk management, logging and cybersecurity for high-risk systems Article 15 requires high-risk systems to achieve an appropriate level of accuracy, robustness and cybersecurity and to be resilient against attempts by unauthorized third parties to alter their use, outputs or performance. It names data poisoning, model poisoning, adversarial examples and confidentiality attacks as the AI-specific vulnerabilities to address. Article 12 requires automatic event logging over the lifetime of the system.
- ART. 26 Deployer duties for a high-risk system Use the system according to the instructions, assign human oversight to competent people with authority, keep input data relevant and sufficiently representative where you control it, monitor operation and suspend use on risk, inform workers representatives before workplace use, and keep the automatically generated logs for at least six months where they are under your control.
- ART. 6(1) Product-embedded high-risk system A system used as a safety component of a product covered by the Union harmonization legislation in Annex I, where that product needs third-party conformity assessment, is high-risk under Article 6(1). Regulation (EU) 2026/1744 moved this date from 2 August 2027 to 2 August 2028.
- ART. 99 Proportionality for SMEs and small mid-caps Article 99 provides that fines for SMEs, including start-ups, are capped at the lower of the percentage or the fixed amount, and the Commission has confirmed that proportionality is taken into account for SMEs and small mid-caps when enforcing the transparency rules. Regulation (EU) 2026/1744 introduced the small mid-cap definition into the Act.
This is a reading aid, not legal advice, and it does not cover sectoral law, GDPR or national implementations. Verify every date against Regulation (EU) 2024/1689 and Regulation (EU) 2026/1744 before relying on it.
§ 09 Calendar
EU AI Act dates, after the Digital Omnibus
Most published guidance still says high-risk obligations began on 2 August 2026. That stopped being true on 27 July 2026. These are the dates as amended, each taken from the Official Journal text.
-
Regulation (EU) 2024/1689 enters into force
The AI Act was adopted on 13 June 2024 and published in the Official Journal on 12 July 2024.
In force
-
Prohibited practices and AI literacy
Chapters I and II apply: the Article 5 prohibitions and the Article 4 AI literacy duty.
In force
-
General-purpose AI models and governance
Obligations for providers of general-purpose AI models and the governance and penalty framework begin to apply. The General-Purpose AI Code of Practice was published on 10 July 2025.
In force
-
Digital Omnibus on AI enters into force
Regulation (EU) 2026/1744 of 8 July 2026, published in the Official Journal on 24 July 2026, enters into force on the third day after publication and amends the application dates below.
In force
-
General application and Article 50 transparency
The general date of application, unchanged by the Omnibus. Chatbots must tell people they are talking to an AI system, and generated content must be marked and labeled. The Commission put the enforcement ceiling for these rules at EUR 15 million or 3 percent of worldwide turnover.
In force
-
Marking grace ends; two further prohibitions apply
Providers of generative systems placed on the market before 2 August 2026 must comply with the Article 50(2) marking obligation by this date. Two prohibited practices added by the Omnibus also begin to apply.
Pending
-
Annex III high-risk obligations
Chapter III Sections 1 to 3 apply to systems classified as high-risk under Article 6(2) and Annex III, including recruitment, workforce management, creditworthiness and insurance pricing. Moved from 2 August 2026.
Pending
-
Annex I product-embedded high-risk obligations
The same obligations apply to systems classified as high-risk under Article 6(1) and Annex I, the safety components of regulated products. Moved from 2 August 2027.
Pending
Every date above is taken from the consolidated Official Journal texts of Regulation (EU) 2024/1689 and Regulation (EU) 2026/1744, cross-checked against the European Commission's implementation page. Where a secondary source disagrees, the Official Journal wins.
§ 10 Annexes
Four guides that go deeper
Long-form working notes on the parts of the audit that need more than a paragraph. Each one is sourced, dated and written to be checked.
- Annex 01 How to test an LLM application for prompt injection Map the inputs, map what the model can reach, then probe. The payload families that matter, the indirect channels most teams forget, and what a finding has to prove. Open annex
- Annex 02 OWASP Top 10 for LLM Applications 2025: what an auditor tests The ten risks, turned into a checklist: what is tested for each one, which ATLAS techniques it maps to, what evidence the report carries and what the durable fix is. Open annex
- Annex 03 EU AI Act for companies deploying chatbots, RAG and agents What actually binds a company that deploys an LLM: the amended dates, the Article 25 trap, Article 50 in practice, and which duties a security audit can produce evidence for. Open annex
- Annex 04 Securing AI agents and MCP: the OWASP Agentic Top 10 in practice What changes when the model can act: the ten agentic risks, the incidents that demonstrate each one, the MCP-specific tests, and the controls that survive a successful injection. Open annex
§ 11 Questions
Questions we are asked before every engagement
Short answers. The longer versions are in the annexes.
What is an LLM security audit?
How is it different from an ordinary penetration test?
Do we need this if we only use ChatGPT or Copilot at work?
When does the EU AI Act apply to us?
Did the high-risk deadline really move?
Is our HR or credit assistant high-risk?
What is prompt injection, and can it be fixed?
Does ISO/IEC 42001 make us AI Act compliant?
What do we actually receive?
How long does it take, and how often should we repeat it?
Sources
- Regulation (EU) 2024/1689 (Artificial Intelligence Act) Articles 3, 4, 6, 12, 15, 25, 26, 50, 99 and Annex III as cited on this page.
- Regulation (EU) 2026/1744 (Digital Omnibus on AI) Amended application dates: 2 December 2027 for Annex III, 2 August 2028 for Annex I, and the four-month Article 50(2) marking grace.
- AI Act implementation timeline and scope
- Transparency obligations under Article 50: frequently asked questions
- Safer and more transparent AI
- OWASP Top 10 for LLM Applications 2025
- OWASP Top 10 for Agentic Applications 2026
- OWASP AI Testing Guide, AITG-APP-01: Testing for Prompt Injection
- MITRE ATLAS technique and case-study data Source of the AML technique identifiers and the Slack AI and Morris II case studies cited above.
- Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (NIST AI 600-1)