Managing Pipeline

You can manage and monitor the pipelines from the Pipelines page.

• The main Pipelines page display a list of all created pipelines. Click Run to run a pipeline directly. You can view the details of the pipeline such as the data source, destination, owner, and status of the run from this page.

• Click the more details icon corresponding to a pipeline in the list to do the following:

  ||

  • View details of the pipeline.

  • Add users to the pipeline. Only Lakehouse members can be added.

  • Clone the pipeline

  • View Lineage of the pipeline data.

  • Delete the pipeline.

• Filters: Enables you to filter the list of pipelines based on various criteria (created by, type, status, tags).

• Running Pipelines: You can manually trigger a pipeline run from the Summary page or wait for its scheduled execution.

• Filtering Streams: You can apply filters to specific streams within a pipeline to extract only relevant data based on certain conditions. This is useful for tenant-specific data or focusing on particular subsets of information.