The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

Hey real general quick question -- can I have tasks that are reactive to system events (in addition to gRPC) or am I scripting the CLI and using calendar only?

You would have to write something to glue the events to GraphQL calls but sure?

I want a client to connect a gRPC server, pull in data, and execute using that input.

I could also have a write to a data store and want to trigger a workflow off of that, for example a write event on AWS that fired an EventBridge event that triggered a Lambda that used Dagster.

Dagster is not itself an event engine, you would need something else doing that part of thing.

A custom client that would establish a gRPC connection to our swarm.

And then you use the GraphQL API to kick off runs, or similar

That's the API Dagit exposes to launch pipeline runs?

Ahh, thank you, I'm wondering about Dagster alone.

Our use case is coordinating a data pipeline across a distributed compute infra. It seems like maybe Dagster is not a fit? That's what I'm trying to validate.

We want the compute instances to be actioning tasks in the dag.

When you want to launch a run, you use the API to do that

Okay, thank you, I will RTFM and ask a better question :slightly_smiling_face:

I think the `sensors` feature that is coming in `0.10.0` may be what you are looking for

<@UH3RM70A2> is there some reading I can do on that? I'd like to know more about `sensors`

<@U01FGAAQX4K> we also have a high level overview of the sensors feature in the last 1/3rd or so of our most recent Community Meeting: <https://www.youtube.com/watch?v=6nKCd2TaXik>

More detailed docs are coming very soon as we finalize things for the upcoming release