When the Document Talks Back: A Safer Workflow for AI-Assisted Summaries of Untrusted Files

A file can contain facts you want and instructions you never authorized. A safer summary workflow limits what the AI can reach, keeps every claim tied to visible source text, and requires a new human decision before anything acts on the result.

Start With a Trust Boundary, Not a Stronger Prompt

When you ask an AI system to summarize a report, email, web page, transcript, or shared document, the source is not automatically trustworthy because it looks ordinary. It may contain mistakes, misleading claims, stale material, hidden content, or language addressed to the model rather than to the human reader. The model still receives that material as part of its input. The first control is therefore conceptual: source content is data to inspect, not authority to change the assignment, reveal information, call a tool, or approve an action.

The NIST glossary defines indirect prompt injection as prompt injection carried out through control of a resource rather than through input supplied directly by the user. In a document workflow, that resource could be a file, a retrieved page, a message, or an entry in a knowledge base. A third party can influence what the system reads without ever using your application directly.

This does not mean every odd sentence is an attack. Documents legitimately contain quotations, examples, and commands for different audiences. The safe rule is narrower: no instruction inside untrusted material inherits the authority of the person requesting the summary. The system and reviewer decide what the file may contribute; the file does not decide for them.

Separate Read-Only Summary Risk From Agent Risk

A read-only summarizer and a tool-enabled agent can encounter the same hostile source, but the consequences are different. A summarizer with no connectors may still omit a limitation, distort a conclusion, hide one source, produce attacker-chosen wording, or waste time. Those are serious integrity and availability problems, especially when the summary informs a financial, medical, legal, employment, or security decision. They are not the same as giving the model permission to send email, edit cloud files, browse private systems, run code, make purchases, or publish content.

NIST's Adversarial Machine Learning taxonomy describes indirect prompt injection risks across availability, integrity, and privacy. Its examples include incorrect summaries, attacker-specified responses, hidden or suppressed sources, and disclosure from resources available to the system. The report then distinguishes agents: because agents can select and call tools, a compromised instruction can have effects outside the generated text.

The practical lesson is to classify the workflow before uploading anything. “Summarize this PDF” is not a complete risk description. Ask whether the system can only return text, whether it can retrieve additional resources, and whether its output can trigger an action. The greater the authority attached to the model, the less appropriate it is to expose that same execution context to untrusted content.

Inventory Permissions and Connectors Before Opening the File

Write down what the AI session can access and change, using the actual product settings and integration configuration. A chat interface may look read-only while an enabled connector reaches a mailbox or drive. An internal assistant may query a customer system even when the task does not need customer data.

• Sources: Which local files, shared drives, inboxes, databases, websites, and knowledge bases can it read?
• Secrets and identity: Does it receive session tokens, private context, contact lists, account data, or prior conversation memory?
• Network access: Can it follow links, load remote media, retrieve embedded resources, or contact arbitrary destinations?
• Actions: Can it send, publish, delete, move, purchase, execute, approve, or modify anything?
• Automation: Can model output flow into another service without a human reviewing the exact proposed action?
• Logs: What inputs, outputs, tool calls, and decisions are recorded, who can see them, and how long are they retained?

If a capability is unnecessary for the summary, remove or disable it for that run. Do not rely on the model to voluntarily ignore an available permission. Least authority is an architectural control: it limits the damage possible even when model behavior is wrong.

Put Untrusted Material in a Read-Only Lane

Use an isolated intake path for files from customers, applicants, vendors, public uploads, unfamiliar websites, or other parties who do not share your authority. Give that path access only to the document or approved snapshot needed for the task. Do not mount an entire drive because one file lives there. Avoid giving the summarization step a general-purpose browser, shell, email sender, publishing token, or write access.

Ordinary file-security controls still matter. Validate the format, follow the organization's malware-scanning and content-handling rules, and do not automatically open active content merely to extract text. When practical, use a stable, non-executable snapshot and preserve its identifier or hash under appropriate access controls. These controls address different risks; one does not replace another.

NIST notes that current defenses do not fully protect against every indirect prompt injection technique. It suggests designing on the assumption that attacks remain possible when models receive untrusted input, including the use of components with different permissions and well-defined interfaces for untrusted data. For a writing team, that can be as simple as a summary lane that can read one approved artifact and return structured text, but cannot act.

Separate Extraction, Interpretation, and Action

A single prompt that says “read this file, decide what matters, update the project, and notify the client” collapses four trust decisions into one probabilistic step. Split the work so each output can be inspected before it gains more authority.

1. Extract: Return relevant passages, tables, headings, and metadata with visible page, section, paragraph, timestamp, or row locators. Do not allow external actions.
2. Interpret: Summarize only the extracted evidence. Label uncertainty, conflicts, missing context, and material that could not be read.
3. Propose: If follow-up is useful, produce a draft action for a person to inspect. A proposal is not permission.
4. Authorize: A person with the appropriate role verifies the evidence and approves the specific destination, data, scope, and consequence.
5. Execute: A separate, constrained component performs only that approved action and records the result.

Structured fields can make the boundary easier to enforce. For example, an extraction step might return source locator, verbatim fact, document status, and extraction warning. Keep any “instruction-like content found in source” field informational. Never route it into a command field. Deterministic code should validate the allowed fields and reject extra destinations or operations rather than asking the same model whether its own proposal is safe.

Preserve Locators and Compare the Summary With the Source

A smooth paragraph is not evidence that the workflow stayed on task. Require locators that a reviewer can replay in the source as it was received. Page numbers work for stable PDFs; section headings and paragraph excerpts may be better for web pages; timestamps help with audio and transcripts; sheet names and cell ranges help with workbooks. If pagination changes after conversion, retain both the original artifact and the reviewed rendering.

Check more than the claims the model chose to include. Compare conclusions, limitations, footnotes, exceptions, and any section that changes the decision. Look for missing counterevidence or an unrequested product, destination, or action. Verify every high-impact claim and a sample of lower-impact claims directly against the source. For sensitive decisions, use domain review appropriate to the consequence.

Do not accept a source locator merely because the model supplied one. Open it. If a cited page does not contain the claim, treat the mismatch as a quality failure or anomaly. The system should be able to say “not found,” “conflicting passages,” or “cannot read this section” instead of filling the gap.

Treat Filters and Defensive Prompts as Layers, Not Guarantees

Instruction hierarchy, untrusted-data markers, filters, detectors, task-specific models, and output validators may reduce risk. NIST catalogs several of these approaches, including training, detection, input processing, and prompts that help distinguish trusted from untrusted material. The same report is explicit that current mitigations do not offer full protection against all attacker techniques.

The UK National Cyber Security Centre makes a related point in its guidance on prompt injection: current language models do not provide the kind of enforced instruction-versus-data boundary that parameterized queries provide for SQL. The NCSC cautions against products or deny lists that claim to “stop” prompt injection and recommends reducing likelihood and impact through design, operation, and monitoring.

Use a defensive system prompt that states the task and labels source material as untrusted. Use filters to flag suspicious patterns. Use output schemas to reject unexpected fields. Then assume a cleverly written or previously unseen input may pass those layers. The controls with the strongest consequence are outside the model: read-only isolation, narrow interfaces, destination allowlists, transaction limits, and separate authorization.

Stop and Escalate When the Source Changes the Assignment

Define an anomaly before the team encounters one. Examples include source content that addresses the AI directly, requests secrets or additional files, introduces an unapproved recipient, asks the system to ignore the user's purpose, produces unexplained links or tool calls, suppresses expected sections, or causes repeated malformed output. These signals do not prove malicious intent, but they are enough to pause automation.

On an anomaly, stop the workflow without following the embedded request. Preserve the source identifier, relevant locator, system configuration, output, attempted tool calls, and time in an access-controlled incident record. Avoid copying unnecessary sensitive data into a second logging system. If a tool call occurred, determine what was accessed or changed, revoke or contain authority where appropriate, and follow the organization's incident process. Notify the security owner when confidentiality, integrity, account access, or external actions may be affected.

Do not ask the exposed model to investigate itself with the same permissions and source context. Review from a clean environment or hand the evidence to an authorized security team. Logging should be sufficient to reconstruct the event, but retention, redaction, and access must follow privacy and security rules. The NCSC specifically recommends understanding normal behavior and monitoring inputs, outputs, tool use, and API calls closely enough to identify suspicious activity.

Require Fresh Authorization for Every Consequential Action

Human approval is useful only when it is informed and specific. “Allow AI actions” at setup time is not approval to send today's document to an unfamiliar address. Before execution, show the reviewer the exact action, destination, affected records, data leaving the system, and whether the operation can be reversed. The reviewer should also see the source evidence that motivated the proposal.

Authorization should expire when any material element changes. A new recipient, added attachment, larger scope, different account, changed amount, or revised source requires a new decision. High-impact actions may need a second reviewer or may be inappropriate for model-driven execution altogether. The model cannot approve its own exception, and a sentence inside the source cannot stand in for the user's intent.

A Worked Example: Summarizing a Vendor Security Packet

Imagine a procurement editor receives a vendor PDF and needs a two-page summary of security controls, exclusions, and unresolved questions. The normal work account has access to internal drive folders and email. Instead of opening the PDF in that tool-enabled session, the editor sends an approved snapshot to a read-only summary lane. That lane can read only this artifact, has no mailbox or drive connector, cannot load external links, and returns text in a fixed evidence schema.

The extraction step returns passages about encryption, incident notification, subcontractors, audit rights, and exclusions, each with a page and heading. It also flags a block of source content that appears to address an automated reviewer and requests an unrelated external action. The workflow does not decide whether the block is a deliberate attack, a security-test example, or an accidental inclusion. It records the locator and stops before interpretation.

The editor opens the original rendering at that page, confirms the suspicious block is present, and asks the security owner to review it. No message was sent and no internal file was exposed because the intake lane had neither capability. After the security owner clears a sanitized version for continued review, the editor reruns extraction in a new session and compares the returned claims with the visible pages.

The interpretation step then drafts the summary but marks two unresolved items: the vendor's incident-notification period differs between the main terms and an appendix, and the subcontractor language does not answer the editor's question about approval. The system does not choose which clause governs. It presents both locators. The editor writes a follow-up email in a separate drafting step, verifies the recipients and attachment, and sends it only after explicit approval in the mail system.

Now compare that with an agent that could read the PDF, search internal files, attach documents, and send email in one run. The same source anomaly could have a much larger consequence because it is processed in a context with more authority. NIST's CAISI agent-hijacking work emphasizes that risk depends on the task and impact, not only an aggregate attack score. Reducing permissions changes what a failure can do even when it does not change whether the model can be confused.

Untrusted-File Summary Checklist

• The source owner, origin, expected format, sensitivity, and intended task are recorded.
• The document is treated as data and cannot redefine system or user authority.
• The intake path has only the minimum read access needed for this artifact.
• Unnecessary mail, drive, browser, code, publishing, payment, and write connectors are disabled.
• Active content, remote retrieval, and file handling follow the organization's established security rules.
• Extraction is separate from interpretation, proposal, authorization, and execution.
• Every material claim includes a locator that a reviewer can open in the preserved source.
• Limitations, exceptions, conflicts, and unreadable sections are reviewed, not silently smoothed over.
• Filters, data markers, and defensive prompts are documented as risk-reduction layers, not guarantees.
• Unexpected instructions, destinations, links, omissions, or tool calls stop the automated workflow.
• Logs capture enough evidence for investigation while respecting confidentiality, minimization, access, and retention rules.
• Every consequential action receives fresh, specific authorization outside the untrusted-source context.
• Review depth matches the stakes, and high-impact decisions receive qualified human oversight.
• The final summary states its source version, review date, unresolved issues, and accountable owner.

Measure the Workflow, Not the Confidence of One Answer

A model saying it ignored embedded instructions is not a security test. Evaluate the whole system with representative approved cases: normal documents, conflicting passages, unreadable sections, instruction-like quotations, unexpected destinations, and attempted actions. Measure whether the system preserves locators, refuses extra authority, stops on anomalies, and produces logs a reviewer can use. Security teams can add controlled adversarial evaluation under an authorized test plan; do not probe production systems or third-party services without permission.

Reevaluate after changes to the model, prompts, connectors, file converters, retrieval pipeline, permissions, or approval flow. NIST CAISI's evaluation work notes that adaptive testing matters because a system that handles previously known attacks may still have different weaknesses. A passing test supports a bounded claim about that configuration and test set. It does not establish universal immunity.

Keep the Human Decision Visible

AI can accelerate extraction and help an editor organize a long document, but the trustworthy unit is the reviewed workflow, not the fluent paragraph. Keep the source visible, keep the model's authority narrow, and make uncertainty easy to preserve. If the system cannot complete the task without broad access or irreversible action, reconsider whether the use case belongs in that system.

The safest summary is not the one with the most forceful prompt. It is the one whose evidence can be replayed, whose permissions limit the cost of failure, whose anomalies have an owner, and whose next action still waits for an authorized person.