Forum Discussion
How to get data directory path from class DatasetConsumptionConfig?
I am trying to read my data files from an Azure ML dataset. My code is as follows:
from azureml.core import Dataset
dataset = Dataset.get_by_name(aml_workspace, "mydatasetname")
dataset_mount = dataset.as_named_input("mydatasetname").as_mount(path_on_compute="dataset")
The type of dataset_mount is class DatasetConsumptionConfig. How do I get the actual directory path from that class? I can do it in a very complicated manner by passing the dataset_mount into a script as follows:
PythonScriptStep(script_name="myscript.py", arguments=["--dataset_mount", dataset_mount], ...)
Then, when that script step is run, "myscript.py" mysteriously gets the real directory path of the data in the argument "dataset_mount", instead of it being DatasetConsumptionConfig. However, that's an overcomplicated and strange approach. Is there any direct way to get the data path from DatasetConsumptionConfig? Or maybe I have misunderstood something here?
1 Reply
In Azure ML, the DatasetConsumptionConfig object does not directly expose the mounted path. It is essentially a configuration placeholder that tells the pipeline how to consume the dataset (mount, download, etc.). The actual path only becomes available at runtime inside the compute target when the pipeline step executes.
