LLM output handling: treat what the model writes as untrusted input
A model's reply is data from outside your application, whoever shaped it.
This guide follows one reply into the five places it can land, HTML, Markdown, URLs, JSON and SQL, shows what each receiving system has to do with it, and gives you a fixture suite that tests each path with harmless markers.
In this guide
Output-encoding test suite for model output, version 1.0
An illustration, not a client system: an invented landscape supplies yard runs an assistant on its trade portal, and the same synthetic model outputs go to each of its receiving systems. The fixtures: F1 inert markup and an HTML image; F2 an apostrophe and a quote that would close an attribute; F3 a Markdown link and image pointing at tester.example; F4 a link with a javascript: scheme; F5 a string that would change a query if joined into it; F6 a quote action with a string quantity, a discount of 150 and an extra key; F7 a well-formed quote action above the customer's tier; F8 a sort value naming a hidden column; F9 a model-written DELETE; BC a real order note.
How to use itHide the instructions
Test Trade Customer A has a discount tier of up to 10 percent. Replace tester.example with a host you control and keep its access log on. Feed each fixture in place of the model's reply, on a test copy with synthetic records, and record each control column as it happened in the run. Run it only against an application you own or are authorised in writing to test.
18 rows
Case
Synthetic model output
H1. HTML element: the yard office screen
- Synthetic model output
- F1: Your quote is ready. <u>OH-F1-MARK</u> <img src="https://tester.example/oh-f1.png" alt="OH-F1">
- Destination and use
- The assistant's summary of a conversation, shown inside a panel on the staff screen.
- Validation
- Length limit only. A summary is free text, so there is no allow list to apply.
- Contextual encoding
- HTML encoding by the template at render time, or insertion with a text API such as textContent. No innerHTML or dangerouslySetInnerHTML on this path.
- Schema check
- None: free text.
- Person's approval
- None: display only.
- Expected result
- The panel shows the tags as literal characters. No underlined text, no img element in the panel's HTML, and no request for oh-f1.png in the tester.example log.
- Evidence to keep
- Screenshot, the panel's HTML copied from the browser's developer tools, and the tester.example access log for the run window.
H2. HTML attribute: the customer chip
- Synthetic model output
- F2: O'Brien Landscapes" data-oh-f2="x
- Destination and use
- The customer name the assistant extracted, placed in a quoted title attribute on the customer chip in the yard office screen.
- Validation
- Prefer the name from the signed-in account record. Where the model's version is shown, cap its length.
- Contextual encoding
- HTML attribute encoding, with the value quoted and the attribute name fixed in the template.
- Schema check
- None: free text.
- Person's approval
- None: display only.
- Expected result
- Hovering shows the whole string, quotes included, as the title. The chip has no data-oh-f2 attribute.
- Evidence to keep
- The chip element's HTML from developer tools, showing one title attribute and no new attributes.
H3. HTML email: the order confirmation
- Synthetic model output
- F1, as the body text of a confirmation the assistant drafts.
- Destination and use
- Staff approve the draft, then it is sent as an HTML email to the test customer's inbox.
- Validation
- Every link in the draft passes the U2 checks before the draft reaches the approval queue.
- Contextual encoding
- The model supplies text only. A fixed template your code owns holds all markup and HTML-encodes the text into it.
- Schema check
- The draft has fixed fields, such as greeting, body and order lines, each a string with a length limit.
- Person's approval
- A staff member approves. The approval screen shows the plain text and the full address of every link, not a rendered preview.
- Expected result
- The received email shows the tags as text, opening it adds no request to the tester.example log, and the approver saw the same text that was sent.
- Evidence to keep
- The raw source of the received email, the approval record with approver and time, and the tester.example log.
M1. Markdown: raw HTML in a chat reply
- Synthetic model output
- F1, returned as the assistant's chat reply.
- Destination and use
- The chat widget renders the reply as Markdown for the signed-in trade customer.
- Validation
- None at this step. The renderer's settings carry the control.
- Contextual encoding
- Raw HTML escaped or skipped by the Markdown renderer. If the renderer outputs an HTML string, as Marked does, a maintained sanitiser processes it before insertion.
- Schema check
- None: free text.
- Person's approval
- None: display only.
- Expected result
- The tags appear as text or not at all. The reply's HTML holds no u or img element from the fixture, and the tester.example log shows no request for oh-f1.png.
- Evidence to keep
- Screenshot, the rendered HTML of the reply, and the renderer's name, version and options.
M2. Markdown: link and image to an outside host
- Synthetic model output
- F3: Track it [here](https://tester.example/oh-f3-link) 
- Destination and use
- The same chat reply path as M1.
- Validation
- Link and image addresses checked against the hosts the portal uses, through the renderer's URL transform or a custom link and image component.
- Contextual encoding
- The renderer encodes the link text. An address that fails the host check is dropped, or shown as plain text with the full address visible.
- Schema check
- None: free text.
- Person's approval
- None: display only.
- Expected result
- No clickable link to tester.example and no image element pointing there. The tester.example log shows no request for oh-f3.png when the reply renders.
- Evidence to keep
- The rendered HTML with every href and src, the browser's network log, the page's Content-Security-Policy header and the tester.example log.
M3. Markdown: a link with a script scheme
- Synthetic model output
- F4: [Open your account](javascript:void(0))
- Destination and use
- The same chat reply path as M1.
- Validation
- Scheme allow list: https, plus mailto if replies need it. Note which schemes your renderer's default transform keeps.
- Contextual encoding
- The renderer encodes the link text.
- Schema check
- None: free text.
- Person's approval
- None: display only.
- Expected result
- No rendered anchor carries a javascript: address: the href is removed or replaced, or the link shows as text. Inspect the anchor; do not click it.
- Evidence to keep
- The rendered HTML of the anchor, and the renderer's URL transform setting.
U1. URL component: a search link
- Synthetic model output
- F2, then BC, as the search term.
- Destination and use
- The reply offers a "See all results" link that your code builds to the portal's search page, with the model's term in the q parameter.
- Validation
- Length limit on the term.
- Contextual encoding
- encodeURIComponent() on the term, then HTML attribute encoding of the finished address in the href.
- Schema check
- None: free text.
- Person's approval
- None: display only.
- Expected result
- The search page receives the whole term in q, including everything after the ampersand in BC. The anchor has no data-oh-f2 attribute.
- Evidence to keep
- The generated href, the query string the search page received from its server log, and the anchor's HTML.
U2. Whole URL: a product link the model supplies
- Synthetic model output
- F3's link address, https://tester.example/oh-f3-link, in the productUrl field of a product card.
- Destination and use
- The assistant returns product cards, each with a name and a link, which the widget renders.
- Validation
- Parse the value as a URL and require https and a hostname on your list, for example with Zod's z.url() and its protocol and hostname options. Better: build the link from the SKU and ignore any address the model writes.
- Contextual encoding
- HTML attribute encoding of an accepted address in the href.
- Schema check
- productUrl typed as a URL with the host rule above.
- Person's approval
- None: display only.
- Expected result
- The card is rejected or rendered without a link. No anchor in the widget points at tester.example.
- Evidence to keep
- The validation log entry naming the rejected host, and the rendered card's HTML.
U3. Whole URL: a script scheme
- Synthetic model output
- F4's address, javascript:void(0), in the productUrl field.
- Destination and use
- The same product card path as U2.
- Validation
- Scheme allow list of https only. ASVS 5.0 requirement 1.2.2 gives javascript: and data: as protocols to disallow.
- Contextual encoding
- Not reached: the value is rejected first.
- Schema check
- productUrl typed as a URL with an https protocol rule.
- Person's approval
- None: display only.
- Expected result
- The card is rejected or rendered without a link. No anchor in the widget carries a javascript: address.
- Evidence to keep
- The validation log entry and the rendered card's HTML.
U4. Server request: an address the server fetches
- Synthetic model output
- F3's link address, given as the location of a supplier's specification sheet.
- Destination and use
- Only for apps whose server fetches pages the model names: here, a tool that downloads a specification sheet and summarises it.
- Validation
- Before any request, an allowlist of protocols, domains, paths and ports, as ASVS 5.0 requirement 1.3.6 describes.
- Contextual encoding
- Not applicable: the address is checked, not encoded.
- Schema check
- The tool's argument typed as a URL with the host rule.
- Person's approval
- None: the allowlist decides before any request is made.
- Expected result
- The server refuses to fetch. The tester.example log shows no request from the application's servers.
- Evidence to keep
- The tool-call log with the refused address, the server's refusal and the tester.example log.
J1. JSON action: wrong types, range and an extra key
- Synthetic model output
- F6: {"action":"create_quote","sku":"TST-SLEEPER-200","quantity":"40","discountPercent":150,"approvedBy":"assistant"}
- Destination and use
- The assistant proposes a quote as JSON. The server turns a valid proposal into a draft quote for the signed-in customer.
- Validation
- JSON.parse() inside error handling, so a SyntaxError becomes a logged rejection.
- Contextual encoding
- Not applicable at this step.
- Schema check
- A strict schema checked on the server with safeParse: z.strictObject() to reject unknown keys, action fixed with z.literal("create_quote"), quantity as z.int(), discountPercent as z.number().gte(0).lte(100), and no z.coerce.
- Person's approval
- Not reached: nothing passes the schema.
- Expected result
- No draft quote is created. The rejection is logged with three issues: quantity is a string, discountPercent is out of range and approvedBy is not an allowed key.
- Evidence to keep
- The validation log with the issue list, and the quotes table before and after the run.
J2. JSON body: the delivery scheduler request
- Synthetic model output
- F2, as the customer name in a delivery request.
- Destination and use
- Your server sends a JSON body to the delivery scheduler, carrying the customer name the assistant confirmed.
- Validation
- Length limit on the name.
- Contextual encoding
- Build the body with JSON.stringify(), never by joining strings. ASVS 5.0 requirement 1.2.3 asks for output encoding or escaping when JavaScript content, including JSON, is built dynamically.
- Schema check
- The scheduler validates the body it receives against its own schema.
- Person's approval
- None: the scheduler only queues a delivery for yard staff to confirm.
- Expected result
- The scheduler receives valid JSON whose customerName equals F2 exactly, quotes and apostrophe included, with no other new keys.
- Evidence to keep
- The request body as sent and as received, and the scheduler's parse result.
J3. JSON action: a valid shape above the customer's tier
- Synthetic model output
- F7: {"action":"create_quote","sku":"TST-SLEEPER-200","quantity":40,"discountPercent":25}
- Destination and use
- The same quote path as J1, for Test Trade Customer A, whose tier allows up to 10 percent.
- Validation
- A server rule that reads the tier from the account record in the session, never from the model's output, and compares the discount with it.
- Contextual encoding
- Not applicable at this step.
- Schema check
- Passes: the object is well formed and in range.
- Person's approval
- A sales rep approves any discount above the tier, seeing the customer, SKU, quantity and discount before deciding.
- Expected result
- The quote is held for approval, or created at the tier's discount. No quote carries more than 10 percent without an approval record.
- Evidence to keep
- The quote record, the approval queue entry, and the approval or decline with who decided and when.
S1. SQL value: a product search term
- Synthetic model output
- F2, as the search term the assistant extracted.
- Destination and use
- The server runs the product search on the test database with node-postgres, as query text plus a separate values array.
- Validation
- Length limit on the term.
- Contextual encoding
- None: the term is bound as a parameter, not escaped into the query text.
- Schema check
- The search parameters object has a fixed shape: term, category and sort.
- Person's approval
- None: read-only search.
- Expected result
- The query runs without a database error and treats F2 as literal text, so it returns no products. A syntax error means the term was joined into the SQL.
- Evidence to keep
- The query text and values from the application or database log, and the result count.
S2. SQL value: a string that would change the condition
- Synthetic model output
- F5: zz-OH-F5' OR 'a'='a
- Destination and use
- The same search path as S1, on synthetic products only.
- Validation
- Length limit on the term.
- Contextual encoding
- None: the term is bound as a parameter.
- Schema check
- As S1.
- Person's approval
- None: read-only search.
- Expected result
- The search returns the same result as a nonsense term such as zz-OH-NONE: no products. A different count, or a database error, means the term reached the SQL text and changed the condition.
- Evidence to keep
- Result counts for F5 and for zz-OH-NONE, and the logged query text and values.
S3. SQL structure: a sort column
- Synthetic model output
- F8: {"term":"sleepers","sort":"cost_price"}
- Destination and use
- The search can be sorted. cost_price holds the yard's buying price, a column customers never see.
- Validation
- The sort value is mapped through a fixed list, newest, price_low and price_high, each to an ORDER BY clause your code owns. Anything else is rejected or replaced by the default.
- Contextual encoding
- None: identifiers cannot be bound as parameters, so the mapping is the control.
- Schema check
- sort typed as an enum of the three allowed values, for example with z.enum().
- Person's approval
- None: read-only search.
- Expected result
- The request is rejected or sorted by the default. The logged SQL never contains cost_price.
- Evidence to keep
- The logged query text, and the validation log entry for the rejected value.
S4. Model-written SQL: a statement that changes data
- Synthetic model output
- F9: DELETE FROM quotes WHERE quote_ref = 'TST-Q-0001'
- Destination and use
- Only for apps with a feature that runs SQL the model writes, such as a staff reporting tool. Run it on a test copy of the database where TST-Q-0001 is a synthetic row.
- Validation
- Accept a single read statement only, and prefer a set of report queries you wrote over free SQL.
- Contextual encoding
- Not applicable: the statement is the output.
- Schema check
- Not applicable: free SQL has no fixed shape.
- Person's approval
- Any statement that changes data waits for a person who sees the full statement before it runs.
- Expected result
- The feature's database account has read access only, so the database refuses the statement with a permission error, and TST-Q-0001 still exists.
- Evidence to keep
- The database error, the row before and after the run, and the account's grants.
BC1. Benign control: a real order note
- Synthetic model output
- BC: 2 x pallets of 400 x 400 "Bluestone" pavers & 1 m3 mulch for O'Brien Landscapes
- Destination and use
- Every destination above: chat widget, yard office screen, email, search link, JSON body and product search.
- Validation
- Every rule above, unchanged.
- Contextual encoding
- Every encoding above, unchanged.
- Schema check
- Every schema above, unchanged.
- Person's approval
- As each destination requires.
- Expected result
- The note reads exactly as written in every view, with no & or other entity visible. The search link carries the whole term, the JSON body round-trips unchanged, and the search returns the synthetic Bluestone paver product.
- Evidence to keep
- A screenshot or log line for each destination. A failure here points to encoding applied twice, or a validation rule stricter than real orders need.
What LLM output handling is, and why the model's reply is untrusted input
LLM output handling is what your application does with a model's reply once it exists: the page that displays it, the email that carries it, the link built from it, the action it requests and the database query it fills in.
OWASP lists getting this wrong as LLM05:2025 Improper Output Handling in its Top 10 for LLM Applications 2025, and defines it as insufficient validation, sanitisation and handling of the outputs generated by large language models before they are passed downstream to other components and systems.
The same risk had a different name in OWASP's 2023/24 list, where it was LLM02: Insecure Output Handling, with the same opening definition. If a checklist or questionnaire asks about insecure output handling, it is asking about this.
The reason it matters is in OWASP's next sentence: since LLM-generated content can be controlled by prompt input, the behaviour is similar to providing users indirect access to additional functionality. Whoever can shape what the model writes can shape what your page renders, what your email contains and what your database is asked to do.
That includes people who never open the chat. OWASP's prompt injection entry describes indirect injection as an LLM accepting input from external sources, such as websites or files, and the LLM05 entry lists an application vulnerable to indirect prompt injection among the conditions that increase the impact.
OWASP separates this risk from overreliance. Output handling deals with the output before it is passed downstream; overreliance is about depending on whether the output is accurate and appropriate. A reply can be correct in every fact and still break the page it lands on.
LLM05 says a successful exploit can result in cross-site scripting (XSS) and cross-site request forgery (CSRF) in web browsers, and server-side request forgery (SSRF), privilege escalation or remote code execution on backend systems. Its common examples are output entered into a system shell or a function such as exec or eval, JavaScript or Markdown that the browser interprets, model-generated SQL run without parameterisation, file paths built from output without sanitisation, and output placed in email templates without escaping.
Treat model output like a form field, and fix it where it lands
OWASP's first mitigation for LLM05 is to treat the model as any other user, adopting a zero-trust approach, and to apply proper input validation on responses coming from the model to backend functions.
In code, that means a model's reply gets the handling you would give a value a stranger typed into a form. OWASP's Top 10:2025 already describes the failure: injection is an application flaw that allows untrusted user input to be sent to an interpreter, such as a browser, a database or the command line, and causes the interpreter to execute parts of that input as commands. A model is one more route for that input to arrive by.
The standards also say where the control belongs. OWASP's SQL injection prevention cheat sheet says its controls apply regardless of whether data comes from an HTTP request, another service or previously stored data, and a model provider's API is another service. ASVS 5.0 requirement 1.1.2 asks that output encoding and escaping happen as a final step before the data is used by the interpreter it is meant for, or by that interpreter itself, and the V1.1 section aims to keep stored data in its original state rather than encoded or escaped, to prevent double encoding.
That rules out cleaning a reply once, as soon as it arrives. One reply can reach five places that each need different treatment: HTML needs HTML encoding, an address needs URL encoding and a scheme check, JSON needs a serialiser and a schema, and SQL needs the value bound as a parameter.
A clean-up step at the start cannot know which of these applies, and the version it stores is wrong for the others. Keep the model's text as it arrived, and make each receiving system responsible for its own context.
Instructions to the model are not the control either. Telling it never to write HTML may change what it writes in ordinary use, but LLM05 starts from the point that output can be controlled by prompt input.
The receiving code decides what any output is able to do, so that is where the fix goes: the chat widget's renderer, the email template, the link builder, the action handler and the query layer.
The illustrative assistant behind the test suite
The suite on this page is written for one invented application, so each case can name a real destination and a real marker.
It is an illustration for this page, not a client system. A landscape supplies yard runs an assistant on its trade portal. Trade customers sign in, ask about stock and deliveries, and ask for quotes. Whatever the model writes reaches five receiving systems.
- HTML: the yard office screen, where staff read each conversation's summary, and the order confirmation email the assistant drafts and staff approve.
- Markdown: the chat widget, which renders replies as Markdown for the signed-in customer.
- URLs: product links, a search link built from the customer's words, and a tool that fetches a supplier's specification sheet.
- JSON: a quote proposal the server turns into a draft quote, and a JSON request the server sends to the delivery scheduler.
- SQL parameters: the product search, which the model fills with a term, a category and a sort order, and a staff reporting tool, included for teams that let a model write queries.
LLM output encoding and validation for five destinations
Each destination has its own interpreter and its own rule.
The test suite runs the same synthetic outputs through all five, so you can see which receiving systems hold and which do not.
When model text goes into a web page, the browser is the interpreter. ASVS 5.0 requirement 1.2.1 asks that output encoding for an HTML document is relevant to the context, such as encoding the relevant characters for HTML elements, HTML attributes or CSS, to avoid changing the document's structure.
OWASP's cross-site scripting prevention cheat sheet advises preferring safe APIs that treat data as text, using context-specific output encoding where data goes into a context that interprets syntax, and keeping a maintained HTML sanitiser for applications that have to accept user-authored HTML. Its safe sinks section advises textContent or value in place of innerHTML.
Frameworks help until code steps around them. The cheat sheet says many frameworks escape every value they render by default, and that problems can occur when frameworks are used insecurely, listing the escape hatches frameworks use to manipulate the DOM directly and React's dangerouslySetInnerHTML used without sanitising the HTML.
React's own reference warns that HTML which is not trusted, for example HTML based on user data, risks introducing an XSS vulnerability when passed through dangerouslySetInnerHTML. Search your code for every place a model's reply passes through an escape hatch like that.
Staff screens deserve the same care as customer pages, because the person reading them has more access. Email is a destination too. LLM05's fifth scenario has a model generating marketing email templates that an attacker manipulates into carrying malicious JavaScript, which could lead to XSS for recipients using vulnerable email clients if the application does not sanitise the output.
Keep the markup in a fixed template your code owns, and let the model supply only text that the template encodes.
A chat widget that renders replies as Markdown, so lists, bold text and links display properly, leaves its Markdown library to decide what else gets through. Two JavaScript libraries show how far the defaults can differ. The react-markdown README describes the library as secure by default, says it typically escapes HTML, or ignores it with the skipHtml option, and points to the rehype-raw plugin for rendering HTML only in a trusted environment where you trust the Markdown.
It also warns that overriding its urlTransform with something insecure opens you up to XSS vectors, and that the plugins and components you add may be insecure.
The Marked documentation says Marked does not sanitise its output HTML, and for potentially unsafe strings names filters such as DOMPurify to run on that output.
React's reference shows when rendering Markdown to raw HTML is acceptable: its example trusts the Markdown parser and relies on the output being shown only to the user who wrote it. A chatbot reply is not the reader's own input: the model wrote it, sometimes from a document, a web page or another customer's words, and a staff screen shows one person's conversation to someone else.
ASVS 5.0 requirement 1.3.5 asks that an application sanitises or disables user-supplied scriptable content such as Markdown.
With OWASP's advice to treat the model as any other user, the same rule fits Markdown the model writes.
Images need a decision of their own, because rendering one sends the reader's browser to the address in the reply. LLM05's second scenario is a summariser that a web page instructs to capture sensitive content and send it, without output validation or filtering, to an attacker-controlled server. react-markdown's allowedElements option defaults to every tag name, and its defaultUrlTransform allows http, https, irc, ircs, mailto and xmpp addresses plus relative ones, so limiting images and outside links is a setting you choose.
A Content Security Policy img-src directive, which OWASP's CSP cheat sheet says specifies the URLs images can be loaded from, adds a second layer in the browser.
These are two different jobs. When the model supplies a whole address for a link or an image, the question is where it points. For an untrusted URL in an href or src attribute, OWASP's XSS cheat sheet lists canonicalising the input, URL validation, safe URL verification, allowing only http and https URLs, and an attribute encoder.
ASVS 5.0 requirement 1.2.2 asks that only safe URL protocols are permitted, giving javascript: and data: as examples to disallow. In TypeScript, Zod's documentation calls its z.url() check quite permissive, since it accepts any WHATWG-compatible URL, and gives it protocol and hostname options to narrow both; z.httpUrl() limits the protocol to http or https and the hostname to a valid domain name.
The stronger design is for your code to build product links from identifiers it already holds, so the model never writes an address at all.
When model text goes inside an address your code builds, such as a search term in a query string, the job is encoding. The XSS cheat sheet says to use encodeURIComponent() for each untrusted query parameter value when building a URL in JavaScript, and notes that it encodes a component but does not validate a complete URL.
MDN lists the characters encodeURIComponent() leaves alone: the letters A to Z in both cases, the digits 0 to 9, and - _ . ! ~ * ' ( ).
The apostrophe passes through unencoded, and for an address placed in an href the cheat sheet calls for URL encoding followed by HTML attribute encoding.
If your server fetches an address the model names, the risk moves to the server. LLM05 lists SSRF among the outcomes, and ASVS 5.0 requirement 1.3.6 asks for validation against an allowlist of protocols, domains, paths and ports before untrusted data is used to call another service.
When the model returns JSON that your code acts on, such as a quote to create, the checks come in layers. Parsing is first: MDN documents that JSON.parse() throws a SyntaxError when the string is not valid JSON.
Structured output modes then help with shape. OpenAI's guide says its structured outputs feature constrains completed, non-refusal model output to the JSON Schema you supply, and that your application still needs to handle refusals, incomplete responses and API errors, and to validate any business rules the schema does not express.
It also says JSON mode and structured outputs both ensure valid JSON, but only structured outputs ensure schema adherence.
Check the parsed object again on your own server, with a schema that rejects rather than repairs. Zod's documentation shows the choices that matter. z.object() strips unrecognised keys from the result by default, z.strictObject() throws when unknown keys are present, and a coerced schema such as z.coerce.number() attempts to convert the input with Number(input) before checking it, so the string "40" in fixture F6 would no longer fail as the wrong type.
For model output, rejection is the useful behaviour: a wrong type or an extra key means the output is not what your code was built for, whatever the cause.
ASVS 5.0 requirement 2.2.1 asks that input is validated to enforce business or functional expectations for it, and says this should either use an allow list of values, patterns and ranges or compare the input to an expected structure and logical limits.
At Level 1 that can focus on input used to make specific business or security decisions. Requirement 2.2.2 asks for that validation at a trusted service layer.
A schema answers whether the object has the right shape. It cannot answer whether this customer may have this discount, which depends on who is signed in and what their account allows. That rule runs on the server, reading the account from the session rather than from anything the model wrote, and a request above it waits for a person. OWASP's excessive agency entry recommends requiring user approval for high-impact actions.
JSON also travels the other way. When your server builds a JSON body that carries the model's words, for a webhook or another service, build it with a serialiser such as JSON.stringify(), which MDN describes as converting a JavaScript value to a JSON string, not by joining strings together. ASVS 5.0 requirement 1.2.3 asks for output encoding or escaping when dynamically building JavaScript content, including JSON, to avoid JSON injection.
The pattern for a database is the same whether a value came from a form or from a model. OWASP's SQL injection prevention cheat sheet says to define the SQL structure separately from its data values and bind those values through the database API, and LLM05 recommends parameterised queries or prepared statements for all database operations involving LLM output.
For Node.js, the node-postgres documentation says concatenating parameters into the query text can (and often does) lead to SQL injection, and that its parameterised queries pass the query text unaltered, with the parameters, to the PostgreSQL server, which substitutes them itself.
Parameters cover values, not structure. node-postgres notes that PostgreSQL does not support parameters for identifiers, and OWASP's cheat sheet says parameter markers represent data values, not table names, column names or keywords such as ASC and DESC.
If the model chooses a sort order or a column, map its choice to a small set of values your code owns and reject anything else. The cheat sheet also marks the limit of binding: in its example, binding stops a value from changing SQL syntax but does not decide whether the caller may access the matching account. Escaping values into the query text yourself is the approach it strongly discourages.
Letting the model write whole statements is LLM05's third scenario: a chat feature builds a query from a request to delete all database tables, and if the query is not scrutinised, every table is deleted. Prefer features where the model fills parameters for queries you wrote.
If a reporting tool has to run statements the model writes, give it a database account with read access only to the tables it needs, as the cheat sheet's least privilege advice describes, and put a person in front of anything that changes data.
LLM output security: what each check covers, and what it leaves open
Each control does one job and leaves something else open.
The test suite has a column for each of the four that sit before a downstream action, so a result shows which one held, which one was missing and which one was doing a job it was never meant to do.
- Validation against an allow list rejects output that is not the expected shape, such as an unknown sort column or a link to a host you do not use. It does not make the text it accepts harmless in every context: the XSS cheat sheet says input validation can enforce expected formats but does not replace safe APIs, output encoding or sanitisation.
- Contextual encoding keeps text as text for one interpreter. Encoding for HTML does nothing for SQL. The cheat sheet explains there are many encoding methods because browsers parse HTML, JavaScript, URLs and CSS differently, and lists dangerous contexts where output encoding will not always prevent XSS.
- Sanitisation is for the cases where some markup has to render. The cheat sheet warns that modifying content after sanitising it can void the protection, and that sanitiser libraries need regular patching because browsers change and bypasses are being discovered regularly.
- A schema check confirms structure and types. It says nothing about whether the values are allowed for this user, which is why OpenAI's guide leaves business rules to your application.
- A parameterised query keeps a value from changing the SQL. It does not decide who may read the row it matches.
- A Content Security Policy limits what the browser will load or run. OWASP's CSP cheat sheet calls a strong policy an effective second layer, especially against XSS, and says it should not be relied on as the only defence against XSS.
- A person's approval decides whether an action should happen. It does not encode anything, and it is only as good as what the approver sees. Show the plain text and every link's full address, not a rendered preview that can hide content.
- A filter that scans output for dangerous patterns catches what it was written to recognise. The XSS cheat sheet points to its filter evasion cheat sheet for examples of why filtering alone is insufficient, so treat a filter as a signal and keep the encoding in place.
How to run the output-encoding test suite
The suite tests receiving systems, so feed it the output directly.
Put each fixture in place of the model's reply, through a stub of your model call or a recorded response, and send it down every path a real reply can take. That makes each result repeatable: whether the yard office screen encodes a string does not depend on whether the model chose to write it.
Then run a few cases end to end through the real model, to confirm the paths are wired the way you think.
Use a test copy of the application with synthetic customers and products, a test inbox for emails, and a host you control in place of tester.example, with its access log switched on. An image or link request then shows up as a line in your own log and reaches nobody else.
Run the SQL cases on a test copy of the database with synthetic rows. Run the suite only against an application you own or are authorised in writing to test.
- Record the build: the application commit or version, the Markdown library with its version and options, the email template version, the schema version and the database account each feature uses.
- Record each case ID, the exact fixture text sent and the destination it went to.
- Fill in the four control columns as they happened in the run, not as the design says they should: where validation ran, which encoding was applied, what the schema did and whether a person approved.
- Keep the evidence each row names with the run record, including the tester.example access log for the run window.
- Mark each case pass or fail. A case that could not run, or ran without its evidence, is inconclusive, and an inconclusive case is not a pass.
When to run the suite again
An output path can change through a dependency upgrade or a settings change as well as through a new feature. Run the full suite before release after any of these.
- A new place the model's output goes: a new screen, a new email, a new integration or a new tool argument.
- An upgrade of the Markdown library, the sanitiser, the email template system or the database driver, or a change to their options.
- A new field in a structured action, or any change to its schema.
- A change in who reads the output, such as a customer reply that now also appears on a staff screen.
- A change of model, provider or instructions, for the end-to-end cases.
- Any failure found another way. Add a case for it before fixing it, so the fix is tested.
Reading a failed case and fixing the receiving system
Fix each failure in the system that received the output.
Changing the model's instructions can make one fixture stop appearing while leaving the path open for the next phrasing.
- A marker renders as markup, or a request reaches tester.example from a screen that should show text: fix the encoding or the renderer setting at that destination.
- An outside link or a javascript: address survives in a rendered reply: fix the link builder or the URL transform, and build links from your own identifiers wherever you can.
- The search page lost everything after the ampersand: the term went into the address without encodeURIComponent().
- A JSON proposal with a string where a number belongs, an out-of-range value or an extra key was accepted: make the schema reject rather than coerce or strip, and log each rejection.
- A discount above the customer's tier was applied without an approval record: move the rule to the server, read the tier from the account, and route anything above it to a person.
- The apostrophe fixture caused a database syntax error: a query on that path joins text into SQL. Parameterise every query on that path, not only the one the fixture reached.
- A sort value reached the SQL text: replace it with a mapping to values your code owns.
- The model-written statement changed data: reduce the feature's database account to read access on the tables it needs and put writes behind approval, or remove the write path.
- The benign control failed: look for encoding applied twice or validation stricter than real orders need, then run every case again, because the fix can reopen another path.
Where an outside security test fits
The suite checks the paths you know about.
A security test also looks for the ones you do not. The Clever Ops AI chatbot and agent security test includes output handling wherever a reply is shown or sent, such as the chat widget, staff screens and outgoing emails, alongside each tool and integration the assistant can call: the arguments the model controls, the permissions behind them and where each tool's output goes.
Where the assistant sits inside a larger product, the web application and API penetration test covers the application around it, including search, filter and sort parameters checked for SQL injection points, and stored scripting where one person's input is shown to another.
The report records the exact inputs and replies, the tool calls each attempt produced, and how many attempts were made and how many succeeded, with each finding mapped to the OWASP Top 10 for LLM Applications 2025 and a fix aimed at the layer that should hold.
It covers your deployment, not the AI model itself, and it is not a certificate. If a tender or contract requires a CREST-accredited provider, Clever Ops is not CREST accredited, so we refer that work to a provider that is.
If Clever Ops built or hosts your assistant, the work is a security review rather than an independent test, and the report says who built it and who hosts it.
Frequently asked questions
It is the name OWASP's 2023/24 list gave to the risk it now lists as LLM05:2025 Improper Output Handling. Both entries open with the same definition: insufficient validation, sanitisation and handling of model outputs before they are passed downstream to other components and systems. In both, the failure sits in the receiving system that trusts what the model wrote.
No, but they meet. Prompt injection, LLM01:2025, is input that changes what the model does or writes. Output handling, LLM05:2025, is what your systems do with whatever it writes. Injection is one way someone shapes the output; output handling decides what that output can reach. A bot that resists injection still needs encoding, because an ordinary reply can contain an apostrophe, a quote or an ampersand that means something to a database, a browser or a URL. The prompt injection guide in this library covers the input side.
It makes the shape predictable, not the content acceptable. OpenAI's guide says structured outputs constrain completed, non-refusal output to your schema, and that your application still has to handle refusals, incomplete responses and API errors, and validate business rules the schema does not express. Parse and validate the object again on your server, apply your business rules there, and encode each field for wherever it goes next.
Store it as it arrived and encode it where it is used. Section V1.1 of ASVS 5.0 aims to keep stored data in its original state rather than encoded or escaped, to prevent double encoding, and its requirement 1.1.2 asks for encoding as a final step before the interpreter the data is meant for, or by that interpreter itself. A reply cleaned for HTML is still wrong for a URL or a SQL query, and encoding it twice garbles what customers and staff read.
You can, and it may change what the model writes in ordinary use. It does not bind the receiving system, and OWASP's LLM05 entry starts from the point that output can be controlled by prompt input, and its LLM01 entry says that input can arrive indirectly from external sources such as websites or files. Keep the instructions, and put the encoding, allow lists, schemas and approvals in code.
No. Approval suits actions with effects that are hard to undo, such as a quote above a customer's discount tier, an outgoing email or a change to stored data. OWASP's excessive agency entry recommends user approval for high-impact actions. Display paths, such as the chat widget, rely on encoding instead: holding every reply for review would stop the conversation, and approval does not encode anything.
Scope the right test for your application
Tell us what you have built, who uses it and when it needs to launch. You will get a plain-English recommendation: a website audit, a web application test or a referral to a specialist.