Implement generative AI and agentic solutions

2.1 Build generative applications by using Foundry

Connecting an application to a Foundry project

This is an explicit exam objective, and the answer is short: one endpoint, one credential, one client.

from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient

project = AIProjectClient(
    endpoint="https://<resource>.services.ai.azure.com/api/projects/<project>",
    credential=DefaultAzureCredential(),
)
openai = project.get_openai_client()          # plain model calls
agent_client = project.get_openai_client(agent_name="policy-assistant")  # bound to an agent

Deploying and consuming models

Once a model is deployed, its deployment name is what you pass as model=. The same call shape works across LLMs, small models, code models and multimodal models — which is exactly why the exam can ask you to swap models without changing code.

# Text
r = openai.responses.create(model="gpt-5-mini", input="Explain RAG in two sentences.")

# Multimodal: text + image in one input item
r = openai.responses.create(
    model="gpt-5",
    input=[{
        "role": "user",
        "content": [
            {"type": "input_text", "text": "What safety issues can you see on this site?"},
            {"type": "input_image", "image_url": "https://example.com/site.jpg"},
        ],
    }],
)

# Streaming, for perceived latency
with openai.responses.stream(model="gpt-5-mini", input=prompt) as stream:
    for event in stream:
        if event.type == "response.output_text.delta":
            print(event.delta, end="")

Implementing retrieval-augmented generation

flowchart LR
  subgraph ING["Ingestion — offline"]
    D["Source documents"] --> CH["Chunk<br/>+ enrich"] --> EMB["Embed"] --> IDX["Index<br/>text + vectors"]
  end
  subgraph RUN["Query time — online"]
    Q["User question"] --> RET["Retrieve<br/>hybrid + semantic rerank"]
    IDX --> RET
    RET --> CTX["Top-k grounding<br/>chunks + citations"]
    CTX --> P["Prompt:<br/>instructions + context + question"]
    P --> M["Model"] --> ANS["Grounded answer<br/>with citations"]
  end

Three ways to do RAG in Foundry, in increasing order of control:

from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient
from azure.ai.projects.models import PromptAgentDefinition

project = AIProjectClient(
    endpoint="https://my-foundry.services.ai.azure.com/api/projects/my-project",
    credential=DefaultAzureCredential(),
)

# A prompt agent is pure configuration: model + instructions + tools.
agent = project.agents.create_version(
    agent_name="policy-assistant",
    definition=PromptAgentDefinition(
        model="gpt-5-mini",
        instructions=(
            "You are an HR policy assistant for Contoso. "
            "Answer ONLY from the retrieved policy documents and always cite the "
            "document name. If the documents do not cover the question, say so "
            "and suggest contacting HR — never guess."
        ),
        tools=[
            {
                "type": "file_search",
                "vector_store_ids": ["vs_hr_policies"],
            }
        ],
    ),
)
print(agent.id, agent.name, agent.version)

Workflows, tool-augmented flows and multi-step reasoning

Foundry workflows preview orchestrate multiple agents in a repeatable process, with branching, variables and human steps, authored in the portal or as YAML in VS Code. Three starting templates map directly to the orchestration patterns:

For deterministic, long-running or event-driven business processes, Azure Logic Apps is the complementary answer: Foundry hosts and grounds the agent, Logic Apps triggers it from events or schedules, waits on long-running approvals and exposes connectors as tools.

Evaluating models and applications

The outline calls out four things to evaluate for: fabrications, relevance, quality and safety. Those map onto evaluator families directly.

from azure.ai.evaluation import (
    GroundednessEvaluator, RelevanceEvaluator, evaluate,
)

model_config = {
    "azure_endpoint": AZURE_ENDPOINT,
    "azure_deployment": "gpt-5-mini",   # the judge model
}

results = evaluate(
    data="golden_questions.jsonl",       # query, context, response, ground_truth
    evaluators={
        "groundedness": GroundednessEvaluator(model_config),  # catches fabrications
        "relevance": RelevanceEvaluator(model_config),
    },
    azure_ai_project=PROJECT_ENDPOINT,   # publishes results to the portal
)

# Fail the CI job on regression rather than eyeballing a dashboard.
assert results["metrics"]["groundedness.groundedness"] >= 4.0

2.2 Build agents by using Foundry

Roles, goals, conversation tracking and tool schemas

An agent definition is four decisions, and the exam tests each separately.

sequenceDiagram
  autonumber
  participant U as User
  participant R as Agent runtime
  participant M as Model
  participant T as Tool

  U->>R: responses.create(conversation, input)
  R->>R: Guardrail — user input
  R->>M: instructions + conversation items + tool schemas
  M-->>R: tool_call(name, arguments)
  R->>R: Guardrail — tool call (agents, preview)
  R->>T: Invoke tool
  T-->>R: Tool result
  R->>R: Guardrail — tool response (agents, preview)
  R->>M: Conversation + tool result
  M-->>R: Final message
  R->>R: Guardrail — output
  R-->>U: Response + citations, persisted as items

Function calling

Function calling is client-side: the model decides that a function should be called and with which arguments; your code executes it and returns the result. That is why function calling cannot live inside a toolbox — the execution happens on your side.

# A function tool is a JSON Schema the model can choose to call.
# The schema IS the prompt for the tool: names and descriptions decide
# whether the model picks it correctly.
create_ticket = {
    "type": "function",
    "name": "create_support_ticket",
    "description": (
        "Create a support ticket in ServiceNow. Use only after the user has "
        "confirmed the summary. Do not use for questions that documentation answers."
    ),
    "parameters": {
        "type": "object",
        "properties": {
            "title": {"type": "string", "description": "Short one-line summary"},
            "severity": {
                "type": "string",
                "enum": ["low", "medium", "high", "critical"],
                "description": "Business impact, not user frustration",
            },
            "description": {"type": "string"},
        },
        "required": ["title", "severity", "description"],
        "additionalProperties": False,
    },
}

response = openai.responses.create(
    model="gpt-5-mini",
    input="Raise a high severity ticket: the checkout page is returning 500s.",
    tools=[create_ticket],
)

for item in response.output:
    if item.type == "function_call":
        args = json.loads(item.arguments)
        result = servicenow.create(**args)
        # Feed the result back in, referencing the call id.
        response = openai.responses.create(
            model="gpt-5-mini",
            previous_response_id=response.id,
            input=[{
                "type": "function_call_output",
                "call_id": item.call_id,
                "output": json.dumps(result),
            }],
        )
print(response.output_text)

Structured outputs

When downstream code consumes the output, constrain generation to a schema rather than parsing prose. This is also the answer to "produce structured JSON output" in Skill 4.

from pydantic import BaseModel

class Ticket(BaseModel):
    title: str
    severity: str
    affected_service: str
    customer_impacting: bool

# Structured outputs constrain generation to the schema, so downstream code
# never has to parse free text or defend against a stray markdown fence.
response = openai.responses.parse(
    model="gpt-5-mini",
    input=email_body,
    text_format=Ticket,
)
ticket = response.output_parsed
print(ticket.severity, ticket.customer_impacting)

Tools and toolboxes

A toolbox curates tools once and exposes them through a single managed MCP-compatible endpoint, with central authentication, guardrails, observability and versioning. Agents point at the toolbox, so you can add, remove or reconfigure tools — or promote a new default version — without changing or redeploying agent code. Toolboxes are Foundry-homed but not Foundry-bound: any MCP-compatible runtime (Agent Framework, LangGraph, your own code) can consume one.

Tools vs skills is a distinction the outline implies: tools define what an agent can do; skills define how it performs a multi-step task. Skills are versioned and immutable, and agents discover them automatically through MCP resources at startup.

Conversation memory

# Server-side conversation: history is maintained for you.
conversation = openai.conversations.create()

r1 = openai.responses.create(conversation=conversation.id, input="Book me a room in Lisbon.")
r2 = openai.responses.create(conversation=conversation.id, input="Make it two nights.")
# r2 knows what "it" refers to.

# Lightweight alternative: chain by response id, no conversation object.
r1 = openai.responses.create(model="gpt-5-mini", input="Summarise this contract.")
r2 = openai.responses.create(model="gpt-5-mini", previous_response_id=r1.id, input="Now list the risks.")

Semantic memory is the layer above: a memory provider extracts durable facts and summaries from conversations, stores them by scope (typically a stable user or tenant id your application owns) and retrieves the relevant ones in later sessions. Extraction is asynchronous — if the next step depends on a newly written memory, you must wait for it. Keep exact transcripts in a history provider if you need complete records; memory deliberately stores facts, not the conversation.

Multi-agent orchestration

flowchart TB
  subgraph SEQ["Sequential"]
    A1["Extract"] --> A2["Validate"] --> A3["Summarise"]
  end
  subgraph CON["Concurrent"]
    C0["Fan out"] --> C1["Legal"] & C2["Finance"] & C3["Technical"]
    C1 & C2 & C3 --> C4["Aggregate"]
  end
  subgraph HAND["Handoff"]
    H0["Triage"] -->|"transfers ownership"| H1["Refunds"]
    H0 -->|"transfers ownership"| H2["Orders"]
  end
  subgraph TOOLS["Agent-as-tool"]
    T0["Primary agent<br/>keeps ownership"] --> T1["Specialist<br/>returns a result"]
    T1 --> T0
  end

Autonomy, safeguards and approval controls

"Build autonomous or semi-autonomous workflows with safeguards and approval flow controls" is an objective in its own right. The mechanisms:

• Approval-required tools — mark a tool so the workflow pauses and emits an approval request before it executes. This is the standard answer for refunds, purchases, deletions and anything irreversible.
• Autonomous mode with turn limits — let an agent proceed without the user for a bounded number of turns, so it cannot loop indefinitely.
• Human-in-the-loop request/response — a request port that pauses the workflow for free-form human input, not just approve/reject.
• Checkpointing — persist workflow state so an approval that arrives hours or days later can resume the run. Essential for long-running processes.
• Guardrails on tool call and tool response — scan what the agent sends to tools and what comes back.
• Scoped agent identity and egress controls — limit what the agent can reach at all.

from typing import Annotated
from agent_framework import tool

# The workflow pauses and emits an approval request before this ever runs.
@tool(approval_mode="always_require")
def process_refund(order_number: Annotated[str, "Order number to refund"]) -> str:
    """Process a refund for the given order."""
    return f"Refund processed for {order_number}."

Monitoring agents and error analysis

• Tracing shows every model call, tool invocation and decision in one timeline — the only practical way to debug "why did it do that?".
• Agent evaluators — intent resolution, tool call accuracy, task adherence, task completion — turn behaviour into numbers.
• Monitoring dashboards track latency, token consumption, error rates and quality scores in production.
• Cluster analysis groups failing evaluation cases so you fix a class of problem rather than one row.
• Agent optimizer preview evaluates behaviour and automatically generates better instructions, skills, tool descriptions and model selections.

2.3 Optimise and operationalise

Prompt engineering

Techniques the exam names, and what each is actually for:

Model parameters

Reflection, chain-of-thought evaluation and self-critique

An explicit objective. These are patterns you implement, not switches you flip:

• Self-critique — the model produces an answer, then a second pass criticises it against explicit criteria and revises. Costs an extra call or two; measurably improves quality on hard tasks.
• Reflection loop — generate → evaluate → revise, repeated until a quality bar is met or an iteration cap is hit. Always set the cap.
• Chain-of-thought evaluation — score the reasoning trace, not just the final answer, to find where a multi-step task went wrong.
• LLM-as-judge — a separate model (the judge, configured in the evaluator) scores outputs. This is how the built-in AI-assisted evaluators work.
• Agent optimizer preview — the managed version: it evaluates behaviour and rewrites instructions and tool descriptions for you.

Observability: tracing, tokens, safety, latency

from azure.monitor.opentelemetry import configure_azure_monitor

# Traces go to the project's Application Insights connection.
connection_string = project.telemetry.get_application_insights_connection_string()
configure_azure_monitor(connection_string=connection_string)

# Opt in to recording prompt and completion content in traces.
# Off by default: the content may contain customer data.
os.environ["AZURE_TRACING_GEN_AI_CONTENT_RECORDING_ENABLED"] = "true"

Orchestrating multiple models and hybrid systems

The last bullet of the skill: "orchestrate multiple models, flows, or hybrid LLM and rules engines". The patterns worth naming:

• Model routing / cascading — a cheap small model handles the easy majority and escalates hard cases to a larger model. The standard answer to "cut cost without losing quality".
• Specialist models per stage — embeddings for retrieval, a reasoning model for planning, a fast model for the user-facing reply.
• Hybrid LLM + rules — deterministic business rules decide anything regulated or numeric; the model handles language. If a question involves eligibility, pricing or compliance thresholds, the rules engine decides and the model explains — never the reverse.
• Gateway-level orchestration — Azure API Management as an AI gateway for load balancing across deployments and regions, token limits and quota enforcement.