📄️ About pipelines
Data pipeline is a structured workflow that facilitates the movement and transformation of data from various data sources into your central warehouse and also in another data source. A pipeline can be compared to a conveyor belt that picks up data, performs specific actions on it, and then deposits it in its designated location.
📄️ Creating Standard pipeline
1. On the DataGOL Home page, from the left navigation panel, click Lakehouse > Pipelines.
📄️ Creating Custom pipeline
1. On the DataGOL Home page, from the left navigation panel, click Lakehouse > Pipelines.
📄️ Creating Dedup pipeline
1. On the DataGOL Home page, from the left navigation panel, click Lakehouse > Pipelines.
📄️ Pipeline Sync Modes
While creating a pipelilne, you can choose any of the following sync mode options:
📄️ Managing Pipeline
You can manage and monitor the pipelines from the Pipelines page.
📄️ Pipeline Details
Click the link of a pipeline in the Pipeline list for a detailed view.