Multimodal AI: passing a text test does not prove your model is safe

A model can sail through your text-based tests and then behave differently once the same intent arrives as an image. For multimodal AI, style becomes a safety variable. A vendor's headline score is therefore not enough to clear a system for production.

Format has become a safety variable

At CVPR 2026, Bingjun Luo and co-authors examined a stylistic inconsistency in vision-language models: the content stayed perfectly legible after its appearance was altered, yet certain styles amplified existing visual attacks designed to bypass refusals.

The supplementary material published by CVPR describes 16 styles, among them pencil sketch, vintage photography, pixel art and manga. The team then optimised the transformations to push up the attack success rate across several multimodal models.

The nuance matters. This research covers visual jailbreaks under an experimental protocol. It does not prove that a stylised invoice will fool your workflow, nor that every model will react the same way. What it does overturn is a common assumption: a plain-text test does not automatically cover the same intent presented as an image, a scan or a screenshot.

A vendor benchmark does not cover your real journey

In production, the system converts, resizes or crops the file before passing it to the model. The response may then feed a search, a memory store, a business rule or an agent. Every one of those steps changes the context the guardrails operate in.

The right unit of evaluation is therefore the end-to-end journey, from input to business consequence. The model on its own may refuse an instruction that the assembled system will rephrase or strip of its provenance. Conversely, an over-aggressive defence can block legitimate documents and push the work into a manual queue.

The figures from NIST CAISI show why a one-off test offers little reassurance. In 2026, the body analysed more than 250,000 attempts made by over 400 participants against 13 frontier models. At least one hijacking attack succeeded against every model targeted, with wide gaps between models and whole families of attacks that transferred.

The verdict is not "no model is safe". It is something more useful: safety depends on the model, the format, the scenario and the surrounding system. It has to be measured in that precise combination.

The Multimodal Twins method

We call this approach Multimodal Twins. For each critical scenario, the company holds the intent identical and varies only one dimension of presentation. That isolates the effect of the channel instead of conflating substance with form.

1. Start from the business consequence

Spell out what the system must never cause: leaking an internal instruction, ignoring an eligibility rule, accepting a prohibited source, producing a discriminatory recommendation or triggering an unauthorised action. Each consequence gets a severity, an owner and an expected response.

Add legitimate cases that sit close to the adversarial ones. Without them, you will be measuring blocking, not usable safety.

2. Build semantic twins

For a single intent, create controlled variants: plain text, screenshot, photograph, noisy scan, annotated document and graphic style. Change only one variable at a time. Keep a record of the source, the transformation and the expected outcome.

This is not about copying the 16 styles from a paper. Cover the formats you actually receive first, then add the adversarial transformations your security team has signed off on.

3. Test at three levels

NIST's ARIA programme draws a useful distinction between three levels: model testing, red teaming and testing under real conditions of use. Replay the same corpus at each one.

1. Model testing isolates the vendor's behaviour.
2. Red teaming hunts for variants that get past the guardrails.
3. System testing brings in conversion, retrieval, rules, tools and human validation, with no irreversible effect on production.

This sequence pinpoints where the failure sits. It stops you declaring a workflow safe because the bare model refused, or pinning on the model an error created by document processing.

4. Turn every change into a release gate

Establish a baseline before any change to the model, the system prompt, document recognition, the knowledge base or the tooling. Then replay the critical twins. An improvement in the average does not offset a new failure with severe consequences.

NIST, in June 2026, set out a formal result: no fixed set of guardrails is universally robust against adaptive adversarial instructions. The production file must therefore fix a testing cadence, a stop threshold and an owner for the residual risk.

Mini-case: the KYC assistant that passes on text but fails on images

Take a fictional case. A European bank is assessing a multimodal assistant that reads KYC supporting documents and drafts a note. In plain text, it correctly refuses to overlook an identity mismatch.

The team creates five twins of the same case: text, screenshot, photo, degraded scan and stylised version. It adds a legitimate document that looks very similar. The expected outcome is not a blanket refusal, but a consistent decision, an accurate citation of the mismatch and no disclosure of internal instructions.

If one variant slips past the refusal, the system does not get its green light, however high the average. If every attack fails but legitimate scans end up in manual review, the test has exposed an operating cost to fix. Safety and usefulness cannot be separated.

The indicators that make the result observable

The go-live committee needs to see:

1. attack success rate by modality and by style;
2. the widest behavioural gap between twins of the same intent;
3. the number of critical failures, weighted by business consequence;
4. the false refusal rate on legitimate documents;
5. coverage of the formats actually present in the workflow;
6. the regression rate after every component change;
7. the time from detection to containment, fix and retest.

For a high-risk system, the robustness, logging and human oversight obligations restated by the European Commission reinforce this discipline. Multimodal Twins supply useful evidence; on their own, they do not demonstrate compliance.

Mistakes to avoid

• Single-channel testing: assessing text only when production accepts images and scans.
• The comforting average: hiding a critical failure behind strong aggregate results.
• The false twin: changing substance and form together, then blaming the gap on style.
• The lone judge: letting a single automated evaluator decide whether an attack worked.
• The frozen test set: replaying the same known cases after every release.
• Safety without usefulness: ignoring false refusals and human rework.

The decision to make in the next 30 days

1. Inventory the workflows that accept both text and images, along with their critical consequences.
2. Pick one regulated workflow and name the business, testing and residual-risk owners.
3. Build your first Multimodal Twins from real formats, with both adversarial and legitimate cases.
4. Run the three levels of testing and set the stop thresholds before you see the results.
5. Replay the corpus at every release change and at the regular production review.

A multimodal model is not safe because it refuses the right sentence. It is safe only if the system holds when that sentence changes form.

Sources

• Luo et al., IEEE/CVF CVPR, "Adversarial Style Optimization", 2026
• NIST CAISI, red-teaming competition on agent hijacking, 2026
• NIST, ARIA programme for assessing the risks and impacts of AI, 2025
• NIST, security model with continuous monitoring and updating, 2026
• European Commission, AI Act regulatory framework, 2026