Study Notes
Types of compute targets in Machine Learning:
- Local compute
Run the experiment on the same compute target as the code used to initiate the experiment
Physical laptop or a virtual machine such as an Azure Machine Learning compute instance
Geat choice during development and testing with low to moderate volumes of data. - Compute clusters
For experiment workloads with high scalability requirements, you can use Azure Machine Learning compute clusters.
This is a cost-effective way to run experiments that need to handle large volumes of data or use parallel processing to distribute the workload and reduce the time it takes to run. - Attached compute
If you already use an Azure-based compute environment for data science, such as a virtual machine or an Azure Databricks cluster, you can attach it to your Azure Machine Learning workspace and use it as a compute target for certain types of workload.
Using compute targets
- Submitted experiment.
- Run will be queued while the COMPUTE_TARGET is started, and the specified environment created.
- The run will be processed on the compute environment.
from azureml.core import Environment, ScriptRunConfig
from azureml.core.compute import ComputeTarget
compute_name = "MY_CLUSTER_NAME"
training_cluster = ComputeTarget(workspace=ws, name=compute_name)
training_env = Environment.get(workspace=ws, name='MY_ENVIRONMENT_NAME')
script_config = ScriptRunConfig(source_directory='my_dir',
script='script.py',
environment=training_env,
compute_target=training_cluster)
References:
Introduction to compute targets - Training | Microsoft Learn