https://dagster.io/ logo
#ask-ai
Title
# ask-ai
d

David de Meij

03/06/2024, 1:54 PM
is there a way to connect dagster jobs from different repositories?
s

Scout

03/06/2024, 1:54 PM
Based on the documentation, Dagster allows for the structuring of a project with multiple code locations, which can be useful for separating conflicting dependencies where each definition has its own package requirements and deployment specs. However, the documentation does not explicitly state that jobs from different repositories can be directly connected. In the context of Dagster, repositories were traditionally the mechanism for organizing assets, schedules, sensors, and jobs, and there could be many repositories per code location. This could imply some level of isolation between repositories. The
create_repository_using_definitions_args
function is mentioned as a stopgap for those who want to use the new
Definitions
API but still want multiple logical groups of assets in the same code location. Issues and discussions from the Dagster community suggest that there have been feature requests and discussions around the ability to view multiple groups/jobs in a unified view and support asset groups across locations/repos. This indicates that there is interest in better ways to organize and potentially connect jobs across different repositories or groups. However, without explicit documentation stating that jobs from different repositories can be connected, it's not clear whether this is currently supported as a built-in feature in Dagster. If you need to connect jobs from different repositories, you might need to look into workarounds or consider restructuring your code to be within the same repository or code location, if possible. For a definitive answer or guidance on how to achieve this, if it's possible, you would likely need to consult the Dagster community or maintainers directly, as the documentation does not provide a clear solution to this specific use case.
3 Views