Listen to this Post
Introduction:
As organizations race to operationalize machine learning, the rush to production often leaves critical security backdoors wide open. Azure Machine Learning (Azure ML) batch inference pipelines deployed on Microsoft Fabric, while powerful, introduce a new attack surface where a seemingly minor misconfiguration in storage accounts or CI/CD workflows can lead to complete system compromise. Recent research reveals that an attacker with only write access to a pipeline’s linked storage account can escalate privileges, execute arbitrary code under Azure ML’s managed identity, and extract secrets from Key Vaults—effectively bypassing all your official security controls.
Learning Objectives:
- Understand the architecture and exploitation vectors of Azure ML batch inference pipelines integrated with Microsoft Fabric.
- Implement least-privilege access controls and secure CI/CD pipelines to prevent code injection and privilege escalation.
- Learn to harden OneLake and Azure ML storage configurations using role-based access control (RBAC), Azure Key Vault, and private endpoints.
- The Vulnerability: Exploiting Azure ML’s Invoker Scripts and Storage Account Gap
The foundation of this exploitation lies in how Azure ML handles pipeline components. For each pipeline component, Azure ML automatically generates and stores invoker scripts within a linked Storage Account. If an attacker gains write access to that Storage Account—whether through compromised credentials, misconfigured shared access signatures (SAS), or an insider threat—they can modify these scripts. When the pipeline subsequently executes, the modified script runs under the compute instance’s managed identity. If that identity is highly privileged (e.g., Contributor or Owner on the subscription), the attacker can execute arbitrary code, extract secrets from Key Vault, and pivot across the cloud environment.
How to Identify and Mitigate the Vulnerability:
- Audit Storage Account Permissions: Run the following Azure CLI command to list all role assignments for a storage account and identify any over-privileged accounts:
az role assignment list --resource-group <resource-group> --scope /subscriptions/<subscription-id>/resourceGroups/<rg>/providers/Microsoft.Storage/storageAccounts/<storage-account> --output table
- Enforce Least Privilege: Remove any write permissions for non-administrative users. Only the Azure ML service principal and a dedicated, highly secure CI/CD agent should have write access.
- Disable SSO for Compute Instances: Microsoft has acknowledged that SSO, when enabled (a previous default), could broaden the escalation path. In future API versions, SSO will be disabled by default. For now, explicitly disable it via ARM templates or policy.
- Use System-Assigned Managed Identities with Caution: Prefer user-assigned managed identities with strictly scoped permissions rather than system-assigned identities, which inherit the creator’s broad privileges.
- Hardening the CI/CD Pipeline for Azure ML and Microsoft Fabric
A secure CI/CD pipeline is your first line of defense against code injection and supply chain attacks. The integration of Azure ML with Microsoft Fabric via Data Factory activities must be treated with the same rigor as any production software deployment.
Step-by-Step Secure Pipeline Configuration:
- Use a Dedicated Service Principal: Create a service principal in Microsoft Entra ID with the least privileges necessary (e.g., `Contributor` on the Azure ML workspace, `Storage Blob Data Contributor` on the linked storage account).
- Store Secrets in Azure Key Vault: Never hardcode credentials in pipeline YAML files. Use Azure CLI to reference secrets:
Retrieve secret from Key Vault and set as environment variable export AZURE_CLIENT_SECRET=$(az keyvault secret show --name "my-sp-secret" --vault-name "my-keyvault" --query value -o tsv)
- Integrate JFrog Artifactory as a Secure Proxy: To prevent supply chain attacks, route all Python package requests through JFrog Artifactory. This caches and scans every package before it reaches the training environment. Configure `pip.conf` as follows:
[bash] index-url = https://<username>:<access-token>@mycompany.jfrog.io/artifactory/api/pypi/pypi-virtual/simple trusted-host = mycompany.jfrog.io
- Enforce Immutable Artifacts: Treat every model and pipeline component as an immutable artifact. Tag each build with a version and push it to a secure registry like JFrog Artifactory:
Enable BuildKit and set variables export DOCKER_BUILDKIT=1 export ARTIFACTORY_HOST=mycompany.jfrog.io export TAG=1.0.0 Build and push docker build --platform linux/amd64 -t ${ARTIFACTORY_HOST}/docker-virtual/azureml-training:${TAG} -f Dockerfile . docker push ${ARTIFACTORY_HOST}/docker-virtual/azureml-training:${TAG} - Implement Compliance-as-Code Gates: Integrate policy checks directly into the pipeline. For example, use Azure Policy to ensure that no storage account allows public access, and fail the build if a violation is detected.
-
Securing the Architecture: OneLake Shortcuts, Data Access, and Network Controls
Azure Machine Learning cannot directly access data stored in Fabric’s OneLake. Instead, it relies on OneLake shortcuts that point to Azure Data Lake Storage (ADLS) Gen2. This architectural handoff creates a crucial security boundary.
Step-by-Step Hardening Guide:
- Configure OneLake Shortcuts with Service Principals: When creating an ADLS Gen2 shortcut in Fabric, ensure the connection uses a service principal with read-only access to the specific container, not a user account.
- Enable Workspace Outbound Access Protection (Preview): This feature restricts outbound connections from Fabric IQ items to only explicitly approved external sources. By default, all outbound connections are blocked.
– To configure, a workspace admin must create data connection rules that specify which cloud sources graph can connect to. Only those approved sources and the destination lakehouse are allowed; all other outbound connections, including mirrored databases, are blocked.
3. Implement Row-Level and Column-Level Security in OneLake: Fabric Spark now integrates with OneLake security to enforce row-level security (RLS) and column-level security (CLS) policies consistently. Define these policies once on the lakehouse, and Spark transparently filters results at query time.
– Example RLS policy creation (OneLake security role):
-- This SQL-like syntax defines a filter that restricts users to seeing only their own sales data CREATE SECURITY POLICY SalesFilter ADD FILTER PREDICATE rls.fn_securitypredicate(SalesRep) ON dbo.Sales;
– Enforcement: When a user runs a query like SELECT FROM lakehouse.sales, Spark requests the effective access from OneLake, which returns allowed columns and row-filter metadata. Spark then executes the query in a privileged system context that filters the rows, returning only the authorized subset to the user. The workspace Admin, Member, and Contributor roles bypass these filters; they apply only to Viewers and users granted access through OneLake security roles.
4. Defensive Commands: Auditing and Monitoring
Continuous auditing of your Azure ML and Fabric environment is non-negotiable. Use these commands to monitor for suspicious activity.
Linux/macOS:
- Check for over-privileged storage account access:
az role assignment list --assignee <user-or-sp-id> --scope /subscriptions/<sub-id>/resourceGroups/<rg>/providers/Microsoft.Storage/storageAccounts/<sa> --include-inherited
- Monitor Key Vault access logs:
az monitor activity-log list --resource-id /subscriptions/<sub-id>/resourceGroups/<rg>/providers/Microsoft.KeyVault/vaults/<kv> --query "[?contains(operationName,'Microsoft.KeyVault/vaults/secrets/read')]"
Windows (PowerShell):
- Get Azure ML batch endpoint details:
Get-AzMLWorkspaceBatchEndpoint -ResourceGroupName <rg> -WorkspaceName <ws> | Format-List
- Retrieve storage account keys (ensure you have proper authorization):
Get-AzStorageAccountKey -ResourceGroupName <rg> -AccountName <sa>
Azure CLI (Cross-Platform):
- List all batch deployments and their associated storage:
az ml batch-deployment list --endpoint-name <ep> --resource-group <rg> --workspace-name <ws> --query "[].{name:name, storage:storageAccount}" --output table - Set a diagnostic setting to send pipeline execution logs to Log Analytics:
az monitor diagnostic-settings create --resource /subscriptions/<sub>/resourceGroups/<rg>/providers/Microsoft.MachineLearningServices/workspaces/<ws> --name "aml-pipeline-logs" --logs '[{"category": "PipelineRuns","enabled": true}]' --workspace <log-analytics-workspace-id>
What Undercode Say:
- The Storage Account is the New Attack Vector: Treat your Azure ML storage account with the same security rigor as your production servers. Any write access to it is a potential privilege escalation path.
- Shift-Left Security for AI Pipelines: Integrate vulnerability scanning and compliance checks into your CI/CD pipeline before code ever reaches the training environment. Policy-as-code gates are your best defense against supply chain attacks.
- OneLake Security is More Than Just Encryption: Leverage OneLake’s native RLS and CLS capabilities to enforce fine-grained access control directly at the data layer. This protects against both external attackers and over-privileged internal users.
- Transparency is Not a Fix: Microsoft’s acknowledgment that storage access equals compute instance access is a feature, not a bug. Build your zero-trust architecture around this reality; never assume a compromised storage account is low-impact.
Prediction:
As more organizations adopt MLOps and platforms like Microsoft Fabric, we will see a sharp rise in attacks targeting the AI supply chain—specifically, the storage and code repositories that feed ML pipelines. Expect to see tooling emerge that automatically scans for over-permissive storage accounts and injects malicious model dependencies. Defenders will respond with strict immutability policies for models and training data, and with real-time anomaly detection on pipeline execution logs. The battle for AI security will be won or lost in the CI/CD pipeline and the storage layer, not just at the network perimeter.
▶️ Related Video (74% Match):
🎯Let’s Practice For Free:
IT/Security Reporter URL:
Reported By: Giannis Arvanitopoulos – Hackers Feeds
Extra Hub: Undercode MoN
Basic Verification: Pass ✅
