Monitor models in Azure Machine Learning

Rating & reviews (0 reviews)
Study notes

Application Insights
  • application performance management service in Microsoft Azure that enables the capture, storage, and analysis of telemetry data from applications
  • monitor telemetry from many kinds of application, including applications that are not running in Azure
The necessary package is already included in Azure Machine Learning Web services, so you can use it to capture and review telemetry from models published with Azure Machine Learning.
When you create an Azure Machine Learning workspace an Azure Application Insights resource is created in the same resource group as your workspace
You can select a specific Azure Application Insights resource to associate with it if you like.

When deploying a new real-time service, you can enable Application Insights in the deployment configuration for the service

dep_config = AciWebservice.deploy_configuration(cpu_cores = 1,
memory_gb = 1,
enable_app_insights=True)

or
service = ws.webservices['my-svc']
service.update(enable_app_insights=True)


Application Insights automatically captures any information written to the standard output and error logs, and provides a query capability to view data in these logs.
Capture telemetry data for Application insights
def init():
global model
model = joblib.load(Model.get_model_path('my_model'))
def run(raw_data):
data = json.loads(raw_data)['data']
predictions = model.predict(data)
log_txt = 'Data:' + str(data) + ' - Predictions:' + str(predictions)
print(log_txt)
return predictions.tolist()

To analyze captured log data, you can use the Log Analytics query interface for Application Insights in the Azure portal.

References:
Monitor models with Azure Machine Learning - Training | Microsoft Learn