How to Orchestrate ADF Pipelines as Selectable Steps in a Configurable Job

I am working on building a dynamic job orchestration mechanism using Azure Data Factory (ADF). I have multiple pipelines in ADF, and each pipeline represents a distinct step in a larger job.

I would like to implement a solution where I can dynamically select or deselect individual pipeline steps (i.e., ADF pipelines) as part of a job. The idea is to configure a job by checking/unchecking steps, and then execute only the selected ones in sequence or based on dependencies.

Available resources for this solution:

• Azure Data Factory (ADF)
• Azure SQL Managed Instance (SQL MI)
• Any other relevant Azure-native service (if needed)

Could you please suggest a solution that meets the following requirements:

• Dynamically configure which pipelines (steps) to include in a job.
• Add or remove steps without changing hardcoded logic in ADF.
• Ensure scalability and maintainability of the orchestration logic.
• Keep the solution within the scope of ADF, SQL MI, and potentially other Azure-native services (no external apps or third-party orchestrators).

Any design pattern, architecture recommendations, or examples would be greatly appreciated.

Thanks!