Azure data science pipeline with Azure Machine Learning Service and Azure Databricks

Machine learning pipeline

Enterprise security readiness Model

Authentication

• Setup Authentication for Azure ML resources and workflows
• Detailed instructions are documented here

Authorization

You can create multiple workspaces, and each workspace can be shared by multiple people. When you share a workspace, you can control access to it by assigning these roles to users:

• Owner
• Contributor
• Reader

Network Security

• Use Storage account for your workspace Instructions on Connecting storage account to workspace virtual network
• Use Keyvault instance with your workspace Instructions
• Use a Machine learning Compute Instructions
• Use Azure Databricks Instructions
• Use Azure Firewall Instructions

Data Encryption

• Encryption at rest
• Encryption in transit
• Using Azure KeyVault

Secure webservices deployed through Azure ML with SSL

Detailed instructions to secure webservices with SSL

Monitoring Azure Machine Learning

Azure Machine Learning logs monitoring data using Azure Monitor, which is a full stack monitoring service in Azure. Azure Monitor provides a complete set of features to monitor your Azure resources. It can also monitor resources in other clouds and on-premises.

Monitoring Azure ML pipeline for Administrators Instructions

As a data scientist or developer to monitor information specific to your model training runs see below links

Start, monitor, and cancel training runs

Log metrics for training runs

Track eexperiments with MLflow

Visualize runs with TensorBoard

Regenerate storage account access keys

Deployment Model

The above conceptual security model are realized in Azure through the following architecture components:

Layer	Component	Security Capabilities
Platform Configuration	Azure Policy	Enforce Azure SKUs. Mandatory PaaS service settings
Platform Configuration	Network Security Groups	Ingress and egress network topology and controls. Network level segmentation.
ML Authentication	Key / Token authentication	Allow only authenticated network connections. Forward client certificates.
ML Security	Azure Premium	HTTP triggers from private VNET w/ access to resources in a private VNET
ML Platform Security	Azure virtual networks	A virtual network acts as a security boundary, isolating your Azure resources from the public internet.
ML Platform Security	Application Insights	Monitor network connections. Customized alerts. Application telemetry.
Data Authorization	Data authorization service	Custom service for evaluating application API calls

Restrict Access to container registry from VNET via firewall rules

• Instructions here

Azure datascience Machine learning pipeline Model