Study notes
All about a run is stored in an Environment.
Generic term.
Environments are encapsulated by the Environment class; which you can use to create environments and specify runtime configuration for an experiment.
In Python - Machine learning an environment stores:
Python version
Conda or pip packages
Way to create an environment:
1. From file
from azureml.core import Environment
py_env = Environment.from_conda_specification(name='ENV_NAME',
file_path='./conda.yml')
py_env = Environment.from_conda_specification(name='ENV_NAME',
file_path='./conda.yml')
conda.yml
name: py_env
dependencies:
- numpy
- pandas
- scikit-learn
- pip:
- azureml-defaults
from azureml.core import Environment
env = Environment.from_existing_conda_environment(name='ENV_NAME',
conda_environment_name='py_env')
3. By specifying pacages
from azureml.core import Environment
from azureml.core.conda_dependencies import CondaDependencies
env = Environment('training_environment')
deps = CondaDependencies.create(conda_packages=['scikit-learn','pandas','numpy'],
pip_packages=['azureml-defaults'])
env.python.conda_dependencies = deps
When you run a Python script as an experiment in Azure Machine Learning, a Conda environment is created.
It defines the execution context for the script.
Azure Machine Learning provides a default environment that includes many common packages;
- azureml-defaults package that contains the libraries necessary for working with an experiment run
- popular packages l(pandas, numpy, etc
The conda dependencies are installed first, followed by the pip dependencies. Make sure pip package is included inconda dependencies
Example - wil be written a file experiment_env.yml
%%writefile $experiment_folder/experiment_env.yml
name: experiment_env
dependencies:
# The python interpreter version.
# Currently Azure ML only supports 3.8.2 and later.
- python=3.8.2
- scikit-learn
- ipykernel
- matplotlib
- pandas
- pip
- pip:
- azureml-defaults
- pyarrow
Use env file to create the enviroment
rom azureml.core import Environment
# Create a Python environment for the experiment (from a .yml file)
experiment_env = Environment.from_conda_specification("experiment_env", experiment_folder + "/experiment_env.yml")
# Let Azure ML manage dependencies
experiment_env.python.user_managed_dependencies = False
# Print the environment details
print(experiment_env.name, 'defined.')
print(experiment_env.python.conda_dependencies.serialize_to_string())
experiment_env defined.
name: experiment_env
dependencies:
# The python interpreter version.
..
..
Finally, register the environment for further use:
With the environment registered, you can reuse it for any scripts that have the same requirements.
# Register the environment
# The environment is registered with the name you assigned when you first created it
experiment_env.register(workspace=ws)
experiment_env.register(workspace=ws)
List registered environments
from azureml.core import Environment
envs = Environment.list(workspace=ws)
for env in envs:
print("Name",env)
envs = Environment.list(workspace=ws)
for env in envs:
print("Name",env)
References:
Introduction to environments - Training | Microsoft Learn
Exercise - Work with Compute Contexts - Training | Microsoft Learn