AI & agent security deep dive

Generative AI does not create an entirely new set of security problems so much as it amplifies existing ones at machine speed. Permissions that were technically wrong but practically harmless become a search result. A user who would never have found a document now has it summarised for them. And an agent with a standing credential can be talked into using it by text hidden inside a file it was asked to read.

flowchart TB
  subgraph IDP["Wrapped by identity — every agent gets a Microsoft Entra Agent ID, a sponsor and Conditional Access"]
    subgraph STACK["The four places a control can go"]
      direction TB
      DATA["Data · Purview and SharePoint Advanced Management — data risk assessments<br/>sensitivity labels · DLP for Copilot · Restricted Content Discovery and Access Control"]
      GATE["Gateway · AI Gateway in Azure API Management — token limits and quotas<br/>semantic caching · content safety policy · token telemetry"]
      MODEL["Model and agent · Microsoft Foundry guardrails and controls — Prompt Shields<br/>groundedness · PII · protected material · blocklists · spotlighting"]
      RUN["Runtime · Microsoft Defender Security for AI — real-time protection over<br/>tool invocations · Defender for AI Services detections"]
      DATA --> GATE
      GATE --> MODEL
      MODEL --> RUN
    end
  end

The threats you are being asked to control

1 · The data layer: stop the assistant seeing what it should not

This is the single highest-value piece of work before deploying Copilot, and it is entirely conventional data governance done properly.

Finding the oversharing

• Purview DSPM data risk assessments — an assessment runs automatically each week over the top 100 most-used SharePoint sites, and you can add custom assessments. It surfaces sites with sensitive data, risky sharing links and broad permissions, with remediation actions attached.
• SharePoint Advanced Management data access governance reports — sharing links, "Everyone except external users" (EEEU) usage, permission state for sites, OneDrive and files, and sensitivity labels applied to files.
• Content management assessment — finds sites with oversized audiences, EEEU, broken inheritance, inappropriate sharing, and sites that are inactive or ownerless.
• Site access review — delegates the review of those reports to the site owners who actually know whether the access is justified.

Containing it

Monitoring what the AI is actually doing

In Purview, Activity explorer has an AI activities view showing AI interactions, the sensitive information types they contained, visits to generative AI websites, and DLP rule matches during those interactions. Workloads are grouped as Copilot experiences and agents, Enterprise AI apps (such as ChatGPT Enterprise) and Other AI apps. To see interactions with third-party AI sites at all, devices must be onboarded to Purview — which in practice means enrolled in Intune and onboarded to Defender for Endpoint.

2 · The model layer: Foundry guardrails and controls

In Microsoft Foundry, a guardrail is a named container of controls, and you assign it to model deployments and to agents.

POST https://<resource>.services.ai.azure.com/openai/v1/responses
Authorization: Bearer <token>
Content-Type: application/json

# Select a named guardrail for this request. If the name does not match an
# existing guardrail, the call fails with InvalidContentFilterPolicy.
x-policy-id: contoso-public-facing

{
  "model": "gpt-5",
  "input": "Summarise the attached policy document.",
  "instructions": "You are a helpful assistant. \"\"\" <documents> ...untrusted content... </documents> \"\"\""
}

Spotlighting

Spotlighting is extra protection against document attacks: it tags untrusted input documents by base64-encoding them, so the model treats their content as lower-trust data rather than instructions. It is off by default, has no direct cost, and is available only for models used through the Chat Completions API — but the encoding increases the token count, which can push a long document over the input limit or raise cost. Delimit untrusted content with """ <documents> … </documents> """ and JSON-escape it so the safety system can parse it.

Foundry also offers a guided guardrail set-up for agents: a short questionnaire — who uses the agent, where its inputs come from, whether it touches sensitive data or PII, whether it calls external tools, whether it takes consequential actions, whether it generates code — that turns the answers into a recommended control set.

3 · The gateway layer: AI Gateway in API Management

Foundry has a built-in AI Gateway, backed by Azure API Management and configured from Foundry portal → Manage → AI Gateway. It governs ingress to a single Foundry resource; for anything broader you run API Management standalone.

<inbound>
  <!-- Cap each consumer's token consumption. Exceeding the per-minute rate
       returns 429; exceeding the period quota returns 403. -->
  <llm-token-limit counter-key="@(context.Subscription.Id)"
                   tokens-per-minute="5000"
                   estimate-prompt-tokens="true"
                   remaining-tokens-variable-name="remainingTokens" />

  <!-- Emit token counts as a custom metric, dimensioned per consumer. -->
  <llm-emit-token-metric namespace="genai">
    <dimension name="Subscription ID" value="@(context.Subscription.Id)" />
  </llm-emit-token-metric>

  <!-- Screen prompts with Azure AI Content Safety before they reach the model. -->
  <llm-content-safety backend-id="content-safety-backend" shield-prompt="true">
    <categories output-type="EightSeverityLevels">
      <category name="SelfHarm" threshold="4" />
      <category name="Violence" threshold="4" />
    </categories>
  </llm-content-safety>

  <!-- Reuse a semantically similar previous completion instead of calling the model. -->
  <llm-semantic-cache-lookup score-threshold="0.05"
                             embeddings-backend-id="embeddings-backend"
                             embeddings-backend-auth="system-assigned" />
</inbound>

Limits apply at the project level, so each project gets its own tokens-per-minute and quota. Creating the gateway needs Contributor or Owner to create the API Management instance (or API Management Service Contributor to reuse one), plus a Foundry owner-level role on the Foundry resource. The gateway also registers custom agents and MCP tools hosted anywhere into the Foundry inventory for governance.

4 · The identity layer: Microsoft Entra Agent ID

Agent ID extends Microsoft Entra to AI agents, so an agent is a governed non-human identity rather than a shared app registration. Agent identity blueprints act as templates, creating individual agent identities in a parent–child relationship so policy applies consistently across large fleets. It supports OAuth 2.0, the Model Context Protocol (MCP) and agent-to-agent (A2A) protocols, and third-party agents from platforms such as AWS Bedrock can be brought in through the Entra ID Auth SDK sidecar or workload identity federation.

Agents created in Copilot Studio automatically receive an agent identity, with the maker recorded as the sponsor. Microsoft's own agents — the Conditional Access Optimization Agent, for example — carry one too, so their queries are attributed to an agent rather than to a person.

flowchart TB
  Q["You want Conditional Access to govern what an agent can reach"] --> A{"Which OAuth flow is the agent using?"}
  A -->|"On-behalf-of — a user signed in, and<br/>the agent acts as that user"| U["Target users and groups<br/>The user is the subject of the token exchange,<br/>so selecting agent identities would not match"]
  A -->|"Client credentials — the agent acts<br/>as itself, with no user present"| AG["Target agent identities<br/>Scale with custom security attributes,<br/>and use agent risk as a condition"]

5 · The runtime layer: Defender real-time protection

Guardrails screen text. Real-time protection screens actions — it inspects agent activity throughout the agentic loop and can stop a risky tool invocation before it executes.

flowchart TB
  U["A user or another agent sends a prompt"] --> GW{"AI Gateway in API Management<br/>token limits"}
  GW -->|"Over the tokens-<br/>per-minute limit"| R429["429 Too Many Requests"]
  GW -->|"Over the total token<br/>quota for the period"| R403["403 Forbidden"]
  GW -->|"Within both"| IN{"Foundry guardrail, input controls<br/>hate · violence · sexual · self-harm<br/>user prompt attack · document attack · PII"}
  IN -->|"A control fires"| BLK["Filtered — the annotation records<br/>detected and filtered"]
  IN -->|"Clean"| M["The model responds, then the output controls run —<br/>the same categories plus protected material,<br/>groundedness and custom blocklists"]
  M --> DEF{"The agent invokes a tool, and Defender<br/>real-time protection evaluates the<br/>invocation before it runs"}
  DEF -->|"A custom rule matches"| STOP["Blocked, and recorded<br/>as a behaviour"]
  DEF -->|"Allowed"| DONE["The tool runs; the default rule<br/>still records the behaviour"]

Default rule versus custom rules

• The default rule audits every agent: matching activity is recorded as a behaviour, and nothing is stopped. This is how you gain visibility before enforcing.
• Custom rules block matching actions before they execute, and also record the behaviour. Scope them narrowly to high-confidence threats and specific agents.
• Behaviours land in the BehaviorInfo table in advanced hunting — what happened, why it was risky, and which agent, user and tool were involved — so they can drive custom detections and downstream automation.
• Near-real-time detections surface as alerts only in audit mode; once a blocking rule covers an agent, alerts are not generated for it.

Turning it on for Copilot Studio

1. In the Microsoft Defender portal, open Security for AI → Get started and select the Copilot Studio step.
2. Toggle Real-time protection on.
3. Copy the Power Platform integration URL and send it to your Power Platform administrator, who completes onboarding on their side using a Microsoft Entra application.
4. Take the App ID back from them, paste it into the Defender portal and save. The Copilot Studio step then shows Connected.
5. Connect the Microsoft 365 connector as well — without it, blocking still works but no alerts or incidents appear in the Defender portal.

Makers see the result in Copilot Studio as a Protection status column on the Agents page, reading Protected, Needs review or Unknown, broken down into threat detection, authentication, policy and content moderation.

6 · Posture and monitoring for AI

Blast radius

"Analyse blast radius for security risks related to Entra Agent ID by using Defender XDR" is the objective's phrase for a simple question: if this agent is subverted, what can it reach? You answer it by joining the agent's identity and its granted permissions to the exposure graph — the agent inventory in the Defender portal for the identity and its connections, and Microsoft Security Exposure Management's attack paths and critical asset classification for what lies downstream. The remediation is almost always the same two moves: narrow the agent's permissions, and put Conditional Access in front of them.

7 · Managing agents in the Microsoft 365 admin center

The administrative counterpart to all of the above. In the Microsoft 365 admin center you manage the agent inventory across Microsoft 365 — who owns each agent, who it is available to, its lifecycle, and which agents makers are permitted to create and publish at all. Power Platform administrators set the corresponding policies for Copilot Studio, including data loss prevention policies over connectors, which is why the Copilot Studio protection status surfaces Policies as its own category.

Scenario-to-answer lookup

Next: Exam tips & caveats, for the final week.