Implement and manage an analytics solution
The platform-engineering third of the exam: configuring the workspace, moving code safely between environments, locking data down at six different layers, and choosing the right orchestration tool.
- 1.1 Configure workspace settings — Spark, domain, OneLake, Apache Airflow
- 1.2 Lifecycle management — version control, database projects, deployment pipelines
- 1.3 Security & governance — workspace/item/row/column/object/folder access, DDM, labels, endorsement, audit logs, OneLake security
- 1.4 Orchestrate processes — tool choice, schedules and event triggers, parameters and dynamic expressions
1.1 Configure Microsoft Fabric workspace settings
Workspace settings are reached from the workspace → Workspace settings. The four groups the exam names explicitly are Spark, domain, OneLake and Apache Airflow.
Spark workspace settings
| Setting | What it controls | Why it matters |
|---|---|---|
| Runtime version | The bundled Spark / Python / Delta versions | Pinning a runtime keeps library compatibility stable across notebooks |
| Starter pool | Pre-warmed, Microsoft-managed Spark nodes | Session starts in seconds instead of minutes — the default |
| Custom pool | Node size, min/max nodes, autoscale, dynamic executor allocation | Right-sizing compute for a specific workload; required when you need larger nodes |
| Environment | Libraries (PyPI/conda/.whl/.jar), Spark properties, resource files | Reusable dependency bundle attached to notebooks and Spark job definitions |
| Native execution engine | Vectorized C++ execution for supported operators | Large speed-up; unsupported operators silently fall back to standard Spark |
| High concurrency mode | Multiple notebooks share one Spark session | Cuts CU burn and start-up time when several users work at once |
| Autologging / job admission | MLflow autologging; optimistic job admission | Affects how many concurrent jobs the capacity will admit |
Caveat — starter pool vs custom pool
Starter pools win on start-up latency because the nodes are already warm, but you cannot change their node size. Creating a custom pool gives you control over node size and autoscale at the cost of a cold start (a few minutes) while nodes provision. "Notebook takes ~3 minutes to start" → they are on a custom pool; switch to a starter pool. "Job OOMs on the default nodes" → custom pool with a larger node size.
Domain workspace settings
Domains group workspaces by business area to enable federated governance — the central Fabric admin delegates authority to domain admins who know their own data.
- Created and managed by the Fabric admin in the admin portal → Domains; workspaces are then assigned to a domain (individually, in bulk, or by workspace admin).
- Subdomains allow a second level of grouping under a domain.
- Domain admins manage the domain's settings, image and default label; domain contributors can assign their own workspaces into it.
- Domains drive discovery — data in the OneLake catalog / data hub can be browsed and filtered by domain.
- Certain governance defaults (such as a default sensitivity label or certification policy) can be scoped per domain.
OneLake workspace settings
| Setting | Effect |
|---|---|
| OneLake access (ADLS Gen2 API) | Whether tools that speak the ADLS Gen2 API — Azure Storage Explorer, OneLake file explorer, third-party clients — can address the workspace's data directly |
| Users can access data stored in OneLake with apps external to Fabric | Turning this off forces all access through Fabric items and blocks direct external connections |
Symptom to recognise
"Storage Explorer / a third-party tool suddenly gets access denied against OneLake, but everything works inside Fabric." That is the external-access workspace (or tenant) setting being disabled — not a permissions problem on the item.
Apache Airflow workspace settings New in the July 2026 outline
Apache Airflow job is Fabric's managed Airflow offering — you author DAGs in Python and Fabric runs them, giving you Airflow's dependency graph, retries, sensors and operator ecosystem alongside native Fabric items.
Configure the runtime at Workspace settings → Data Factory → Apache Airflow Runtime Settings:
| Setting | Options / meaning |
|---|---|
| Pool type | Starter pool (default, pre-configured) or Custom pool |
| Compute node size | Small for simple DAGs, Large for complex or production DAGs |
| Enable autoscale | Lets the Airflow pool scale nodes up and down with demand |
| Extra nodes | Raises concurrent DAG capacity — each additional node provides capacity for 3 more workers |
| Airflow version / requirements | Runtime version plus Python package requirements for the environment |
Airflow job vs Data pipeline — when to pick which
Choose a Data pipeline for Fabric-native orchestration with a visual designer, native activities and built-in triggers — it is the default answer. Choose an Apache Airflow job when the requirement mentions existing Airflow DAGs to migrate, Python-as-code DAG authoring, complex cross-system dependencies, or the Airflow operator ecosystem. Airflow jobs support workspace identity for authentication and can run Fabric items from a DAG.
1.2 Implement lifecycle management
Azure DevOps or GitHub"] GIT --> PR["Branch + pull request
(code review)"] PR --> MAIN["main branch"] DEVWS -->|"deployment pipeline"| TESTWS["Test workspace"] TESTWS -->|"deployment pipeline"| PRODWS["Prod workspace"]
Version control with Git integration
- Supported providers: Azure DevOps Repos and GitHub (including GitHub Enterprise). Other Git hosts are not natively supported.
- A workspace connects to one repository, one branch and one folder at a time. Switching branches is a workspace-level action.
- Core actions: Commit (workspace → Git), Update (Git → workspace), and Undo uncommitted changes. The source-control pane shows per-item status.
- Items serialize to a folder per item containing its definition files (for example
.platformplus notebook / pipeline / model definition files). - Branch out to a new workspace creates an isolated workspace on a feature branch — the standard pattern for parallel development.
Caveat — Git syncs definitions, never data
Git integration versions item definitions (notebook code, pipeline JSON, semantic model metadata, Lakehouse/Warehouse schema metadata). It does not version the rows in your tables or the files in Files/. The same is true of deployment pipelines. Any answer implying "promote the data with Git" is wrong.
Database projects
A SQL database project is the schema-as-code approach for Fabric Warehouses: the object definitions live as .sql files in source control rather than existing only in the deployed database.
| Capability | Detail |
|---|---|
| Author | VS Code with the SQL Database Projects extension, or Visual Studio / SSDT |
| Build output | A dacpac describing the desired schema state |
| Schema compare | Diff the project against the deployed Warehouse in either direction |
| Publish / deploy | Apply the project's schema to the target; automatable with SqlPackage in CI/CD |
| Import | Reverse-engineer an existing Warehouse into a new project to get it under source control |
When a database project is the answer
Look for "declarative", "schema as code", "review schema changes in a pull request", "detect drift between environments", or "deploy the warehouse schema from a build pipeline". Deployment pipelines promote the item; a database project gives you object-level control and diffing of the schema inside it.
Deployment pipelines
| Concept | Detail |
|---|---|
| Stages | Up to 10; the classic shape is Development → Test → Production. Each stage is bound to one workspace. |
| Deployment rules | Per-stage overrides so promoted items point at the right environment — data source rules, parameter rules, connection rules. Set on the target stage. |
| Selective deployment | Deploy a subset of items instead of everything. |
| Auto-binding | Dependencies between deployed items are automatically re-pointed to the target stage's copies. |
| Comparison view | Shows which items differ between stages before you deploy. |
| Backwards deployment | Possible when the target stage has no items, e.g. hydrating a new Dev from Prod. |
| Automation | Deployment pipeline REST APIs / Fabric CLI let you drive promotion from Azure DevOps or GitHub Actions. |
Caveat — permissions and the rules trap
To deploy you need access to both the source and target workspaces (Contributor or higher on the target; Admin to manage the pipeline itself). And deployment rules live on the target stage — the single most common wrong answer is configuring the rule on the source. Without rules, a promoted pipeline or semantic model keeps pointing at the Dev data source.
1.3 Configure security and governance
Fabric security is layered. Work top-down: a user must clear every layer to see a row.
(admin portal)"] --> C["Capacity settings"] C --> W["Workspace roles
Admin · Member · Contributor · Viewer"] W --> I["Item permissions
Read · ReadData · ReadAll · Write · Reshare"] I --> D["Data-level controls"] D --> RLS["RLS — rows"] D --> CLS["CLS — columns"] D --> OLS["OLS — objects in semantic models"] D --> DDM["DDM — masked values"] D --> FLS["OneLake data access roles — folders"]
Workspace-level access controls
| Role | Can do | Cannot do |
|---|---|---|
| Admin | Everything: manage the workspace, add/remove members, delete the workspace, manage all items | — |
| Member | Create/edit/delete items, share items and grant others access, publish apps | Delete the workspace or change its settings |
| Contributor | Create/edit/delete items, run notebooks and pipelines | Share items or manage members |
| Viewer | View items and read data they are granted; run queries against SQL endpoints | Create, edit, delete, or see item definitions/code |
Caveat — Member vs Contributor
The dividing line is resharing. Both can build; only Member (and Admin) can grant other people access. "Engineers must build pipelines but must not be able to share data outward" → Contributor. Also note Viewer does not automatically mean data access to everything: Viewer can query through the SQL endpoint, but Lakehouse file access and some item data still require explicit permission.
Item-level access controls
Sharing an individual item grants a specific permission set. For Lakehouses and Warehouses the three read permissions are the ones to distinguish:
| Permission | Grants |
|---|---|
| Read | See the item exists and connect to its endpoints — not the underlying data by itself |
| ReadData | Read data through the SQL analytics endpoint / Warehouse, subject to RLS, CLS and DDM |
| ReadAll | Read the raw files and tables in OneLake (Spark, shortcuts, external tools) — bypasses SQL-layer RLS/CLS/DDM |
| Write | Modify the item and its data |
| Reshare | Pass the granted permissions on to others |
| Execute / Build | Run the item (jobs) or build new content on a semantic model |
ReadAll is a security hole if you rely on T-SQL controls
RLS, CLS and DDM are enforced by the SQL engine. A user with ReadAll reads the Delta files directly through Spark or OneLake and therefore sees everything, unmasked and unfiltered. If a scenario requires row filtering that holds for all engines, the answer is OneLake security, not T-SQL RLS — or you must withhold ReadAll.
Row-, column-, object- and folder/file-level controls
| Control | Granularity | Where it is defined | Mechanism |
|---|---|---|---|
| RLS | Rows | Warehouse, SQL analytics endpoint | CREATE FUNCTION predicate + CREATE SECURITY POLICY |
| RLS (semantic model) | Rows | Semantic model | DAX filter expression on a model role |
| CLS | Columns | Warehouse, SQL analytics endpoint | GRANT / DENY SELECT on named columns |
| OLS | Whole tables/columns, hidden entirely | Semantic model only | Model role with None on the object |
| Folder/file | Folders in a Lakehouse | OneLake | OneLake data access roles |
-- 1. Predicate function: returns a row when access is allowed
CREATE FUNCTION dbo.fn_SecurityPredicate(@Region AS nvarchar(50))
RETURNS TABLE
WITH SCHEMABINDING
AS
RETURN SELECT 1 AS accessResult
WHERE @Region = USER_NAME()
OR USER_NAME() = 'analytics-admin@contoso.com';
GO
-- 2. Bind it to the table with a security policy
CREATE SECURITY POLICY dbo.SalesRegionFilter
ADD FILTER PREDICATE dbo.fn_SecurityPredicate(Region) ON dbo.Sales
WITH (STATE = ON);-- Grant only the non-sensitive columns; Salary is never selectable
GRANT SELECT ON dbo.Employees(EmployeeName, Department, JobTitle)
TO [analyst@contoso.com];
-- DENY always wins over GRANT, even via role membership
DENY SELECT ON dbo.Employees(Salary) TO [analyst@contoso.com];Caveat — OLS is semantic-model-only
Object-level security exists only in semantic models, and its distinguishing property is that restricted objects become completely invisible — they vanish from the field list rather than erroring. CLS in a Warehouse, by contrast, leaves the column visible in metadata and raises a permission error on select. "The table must not even appear to those users" → OLS.
OneLake security
OneLake data access roles secure data at the OneLake layer, so the rule is enforced regardless of which engine reads it — Spark, T-SQL endpoint, shortcut consumer or external tool.
- Defined on a Lakehouse under Manage OneLake data access: create a role, select the folders/tables it covers, assign users or groups.
- Permissions are additive across roles a user belongs to, and inherit down into subfolders.
- A DefaultReader-style role typically maps existing workspace access; you then add narrower roles for restricted folders.
- Newer OneLake security capabilities extend this to row- and column-level rules defined once at the OneLake layer and enforced across engines.
Decision rule
Need the restriction to hold only for SQL consumers? T-SQL RLS/CLS/DDM is fine and is the simplest answer. Need it to hold for Spark and external tools too? That is OneLake security. Check the newest Microsoft docs for the current preview/GA status of the row- and column-level OneLake capabilities before relying on them.
Dynamic data masking
DDM masks values at query time in the result set. The stored data is unchanged.
| Function | Behaviour | Example output |
|---|---|---|
default() | Full mask by data type | xxxx for strings, 0 for numerics, 1900-01-01 for dates |
email() | First character, then a fixed pattern | mXXX@XXXX.com |
partial(prefix,padding,suffix) | Keeps a prefix and suffix, pads the middle | partial(0,"XXXX-XXXX-XXXX-",4) → XXXX-XXXX-XXXX-1234 |
random(low,high) | Random number in range (numeric columns) | 42 |
ALTER TABLE dbo.Customers
ALTER COLUMN Email nvarchar(200) MASKED WITH (FUNCTION = 'email()');
ALTER TABLE dbo.Customers
ALTER COLUMN CreditCard varchar(19) MASKED WITH (FUNCTION = 'partial(0,"XXXX-XXXX-XXXX-",4)');
-- Let a specific principal see real values
GRANT UNMASK TO [fraud-team@contoso.com];DDM is obfuscation, not a security boundary
Anyone with UNMASK, elevated database permissions, or ReadAll file access sees the real values. A determined user can also infer masked values through WHERE predicates, since filtering happens on the underlying data. For genuine protection use CLS (deny the column outright), OneLake security, or encryption — DDM is for casual over-the-shoulder exposure.
Sensitivity labels and endorsement
| Feature | Purpose | Key behaviour |
|---|---|---|
| Sensitivity labels | Classification and protection, from Microsoft Purview Information Protection | Applied to any Fabric item; inherit downstream (Lakehouse → semantic model → report) and flow into exports (Excel, PDF, PBIX) where encryption is enforced |
| Promoted | "This is ready for others to use" | Any item owner / workspace Member or Contributor can promote their own item |
| Certified | "This meets the organization's quality bar" | Only principals on the tenant/domain certification allow-list can certify |
| Master data | "This is the authoritative source for this entity" | Also restricted to designated certifiers |
Caveat — labels vs endorsement
They answer different questions. A sensitivity label says how confidential data is and can enforce protection. An endorsement says how trustworthy/authoritative it is and enforces nothing. Also: promotion is self-service, certification is gated — that distinction is a frequent question.
Fabric audit logs
- Fabric activity flows into the Microsoft 365 unified audit log, searchable in Microsoft Purview → Audit.
- Fabric admins can also use the admin portal audit/activity view and the Get Activity Events admin API or Office 365 Management Activity API for programmatic export.
- Captured events include item create/read/update/delete, sharing and permission changes, exports and downloads, and data access.
- Retention follows the Microsoft 365 licence — 180 days under Audit (Standard) with longer retention available under Audit (Premium)/E5 add-ons. Verify current retention against Purview docs.
Which log answers which question?
"Who accessed or shared this item?" → audit log (Purview). "Why is the capacity throttling and which item burned the CUs?" → Capacity Metrics app. "Why did last night's run fail?" → monitoring hub. Don't reach for the audit log to troubleshoot performance.
1.4 Orchestrate processes
Dataflow Gen2 vs pipeline vs notebook
| Dimension | Dataflow Gen2 | Notebook | Data pipeline |
|---|---|---|---|
| Role | Transformation | Transformation | Orchestration |
| Persona | Analyst / citizen developer | Data engineer / scientist | Data engineer |
| Language | M (Power Query), visual | PySpark, Spark SQL, Scala, R | Visual designer + expression language |
| Scales to | Small–medium volumes | Very large volumes (Spark) | Depends on the activities it calls |
| Connectivity | 150+ Power Query connectors | Whatever Spark can read | Copy activity connector library |
| Can call others? | No | Yes (other notebooks) | Yes — notebooks, dataflows, stored procs |
Caveat — the framing that decides it
If the stem describes one transformation, choose Dataflow Gen2 (no-code/analyst) or a notebook (code/scale). If it describes several steps that must run in order, with conditions, retries or looping, the answer is a data pipeline — which then calls the dataflow or notebook. Pipelines orchestrate; they are not themselves a transformation engine.
Schedules and event-based triggers
| Trigger | Fires on | Typical use |
|---|---|---|
| Schedule | A recurrence (by the minute, hour, day, week) with a start/end and time zone | Nightly or hourly batch loads |
| Event-based (OneLake events) | File/folder created, deleted or renamed in OneLake | Process files as soon as they land |
| Event-based (Azure Blob / external) | Blob storage events surfaced through the Real-Time hub | React to arrivals in ADLS/Blob |
| Job events | A Fabric job succeeds, fails or completes | Chain workloads across items; alert on failure |
| Manual / API | Run now, REST API, or an upstream pipeline | Ad-hoc runs and testing |
How event triggers are wired
Fabric event triggers are built on the Real-Time hub and Activator: the event source publishes into the hub, and an Activator rule starts the pipeline. That is why "run the pipeline when a file lands in OneLake" is an event trigger and not a tight polling schedule.
Orchestration patterns, parameters and dynamic expressions
read watermark / file list"] LOOKUP --> FE["ForEach
iterate, optionally in parallel"] FE --> COPY["Copy data
source to bronze"] COPY --> IF{"If condition
rows copied greater than 0"} IF -->|yes| NB1["Notebook
bronze to silver"] IF -->|no| SKIP["Skip / log"] NB1 --> NB2["Notebook
silver to gold"] NB2 --> SP["Stored procedure
update watermark"] SP --> REF["Semantic model refresh"]
Control-flow activities worth knowing
| Activity | Use |
|---|---|
| Lookup | Read a single value or row set (a watermark, a config table) to drive later activities |
| ForEach | Iterate a collection; isSequential off runs items in parallel up to a batch count |
| If condition / Switch | Branch on an expression |
| Until | Loop until a condition is true — polling patterns |
| Set / Append variable | Accumulate state within a run |
| Invoke pipeline | Call a child pipeline — the modular orchestration pattern |
| Web / Webhook | Call REST endpoints; Webhook waits for a callback |
| Fail | Deliberately fail a run with a custom message |
Dynamic expressions
Pipeline expressions start with @. Know how to read these:
// Pipeline parameter and variable
@pipeline().parameters.InputFolder
@variables('RowCount')
// System variables
@pipeline().RunId
@pipeline().TriggerTime
// Output of a previous activity
@activity('LookupWatermark').output.firstRow.LastLoadedDate
@activity('CopyToBronze').output.rowsCopied
// String building and dates
@concat(pipeline().parameters.Prefix, '_',
formatDateTime(utcnow(), 'yyyyMMdd'), '.parquet')
@formatDateTime(addDays(utcnow(), -1), 'yyyy-MM-dd')
// Condition for an If activity
@greater(int(activity('CopyToBronze').output.rowsCopied), 0)
// Current item inside a ForEach
@item().TableNameNotebook orchestration
# Run another notebook and capture its exit value (timeout in seconds)
result = notebookutils.notebook.run("transform_silver", 300,
{"load_date": "2026-07-21"})
# Run many notebooks with a dependency graph, in parallel where possible
notebookutils.notebook.runMultiple(dag)
# Return a value to the caller (pipeline or parent notebook)
notebookutils.notebook.exit("rows_loaded=18432")Parameter cells and notebookutils
A notebook receives parameters through a cell tagged as the parameter cell — values passed by the pipeline's Notebook activity override the defaults in that cell. Note also that mssparkutils has been superseded by notebookutils in Fabric; you may still see the old name in older material and in exam distractors.
Caveat — %run vs notebookutils.notebook.run
%run inlines another notebook into the same Spark session, sharing variables — good for shared helper functions. notebookutils.notebook.run() executes it as a separate job with its own session, passes parameters and returns an exit value — that is the orchestration primitive. For parallel execution with dependencies, runMultiple() with a DAG beats a chain of sequential calls.
Scenario quick reference
| Scenario | Deciding constraint | Answer |
|---|---|---|
| Promote items Dev → Test → Prod | Environment promotion | Deployment pipeline |
| Promoted pipeline still points at the Dev database | Per-stage binding | Deployment rule on the target stage |
| Review notebook changes in a pull request | Code review | Git integration |
| Warehouse schema under source control with drift detection | Schema as code | Database project |
| Engineers build but must not share data outward | No resharing | Contributor role |
| Each sales rep sees only their own region | Row filtering, SQL consumers | RLS security policy |
| Row filtering that also holds for Spark readers | Cross-engine | OneLake security |
| Hide the Salary column from analysts | Column restriction | CLS — GRANT/DENY on columns |
| Table must not even appear in the field list | Invisibility | OLS in the semantic model |
| Show only the last 4 digits of a card number | Presentation masking | DDM partial() |
| Restrict access to one folder of a Lakehouse | Folder granularity | OneLake data access role |
| Mark a semantic model as the authoritative source | Trust signal | Certified endorsement (allow-listed users only) |
| Prove who exported a report last month | Forensics | Purview audit log |
| Run a job the moment a file lands in OneLake | Event-driven | Event-based trigger via Real-Time hub |
| Migrate existing Airflow DAGs into Fabric | Airflow ecosystem | Apache Airflow job |
| Notebook takes minutes to start every morning | Cold start | Starter pool (instead of custom pool) |
| Same libraries needed across many notebooks | Dependency reuse | Environment |