About orchestration

Orchestration in DataGOL allows you to define and manage the execution order of multiple pipelines. This is crucial when you have dependencies between pipelines, where one pipeline needs to complete successfully before another can begin. Without orchestration, scheduling pipelines to run independently might lead to failures or unexpected results if a dependent pipeline runs before its required data is ready.

Orchestration ensures a controlled and sequential execution of your data pipelines, less prone to undesirable data, and guarantees data availability for downstream processes like reporting and analysis.