Sourcify
Effortlessly find and manage open-source dependencies for your projects.

A unified control plane for building, scaling, and observing AI and data pipelines.

Dagster is a data orchestrator designed to unify the development, scaling, and monitoring of AI and data pipelines. It provides a declarative programming model focused on data assets, enabling data engineers to define pipelines as code. Built-in lineage and observability tools facilitate faster issue detection and resolution, ensuring data quality and reliability. Dagster supports integrations with popular data tools like dbt, Databricks, Snowflake, and BigQuery, streamlining data movement, transformation, and model training. The platform’s architecture allows for flexible deployment options, including cloud-based and on-premise setups, with features like role-based access control and audit logs to support enterprise security and compliance requirements. Dagster's asset-based framework simplifies complex data workflows, making it easier for teams to build and maintain data platforms at scale.
Dagster is a data orchestrator designed to unify the development, scaling, and monitoring of AI and data pipelines.
Explore all tools that specialize in ml workflow management. This domain focus ensures Dagster delivers optimized results for this specific requirement.
Explore all tools that specialize in track data lineage. This domain focus ensures Dagster delivers optimized results for this specific requirement.
Tracks the flow of data through pipelines, providing a visual representation of dependencies between datasets and transformations.
Defines pipelines as a series of software-defined assets, simplifying pipeline management and improving code reusability.
Monitors key pipeline metrics such as freshness, performance, and cost, providing real-time insights into pipeline health.
Utilizes AI to analyze pipeline logs and identify potential issues, streamlining the debugging process.
Provides detailed cost breakdowns for pipeline execution, enabling teams to optimize resource utilization and reduce expenses.
Install Dagster via pip or Docker.
Define data assets using the `@asset` decorator in Python.
Create a `defs.py` file to define your Dagster repository.
Use Dagster's UI to visualize pipeline lineage and dependencies.
Configure connections to data sources and destinations (e.g., Snowflake, BigQuery).
Implement data quality checks using Dagster's built-in features or integrations with dbt.
Set up alerting and monitoring to track pipeline health and performance.
Deploy Dagster to your cloud or on-premise infrastructure.
All Set
Ready to go
Verified feedback from other users.
"Users praise Dagster for its asset-based framework, robust lineage capabilities, and ease of integration with modern data tools."
Post questions, share tips, and help other users.
Effortlessly find and manage open-source dependencies for your projects.

End-to-end typesafe APIs made easy.

Page speed monitoring with Lighthouse, focusing on user experience metrics and data visualization.

Topcoder is a pioneer in crowdsourcing, connecting businesses with a global talent network to solve technical challenges.

Explore millions of Discord Bots and Discord Apps.

Build internal tools 10x faster with an open-source low-code platform.

Open-source RAG evaluation tool for assessing accuracy, context quality, and latency of RAG systems.

AI-powered synthetic data generation for software and AI development, ensuring compliance and accelerating engineering velocity.