Just figured I posted my question in the wrong cha...
# ask-community
Just figured I posted my question in the wrong channel 😓
Hi everyone, I'm new to Dagster and we're trying to bring it onboard at SharperShape. I feel I'm getting the gist of it and I find it amazingly helpful compared to our previous in-house scheduler, however, I would like to ask more experienced users about one particular use-case.
We work with GIS data and data collected from sensors in the real world tends to be noisy and require many tweaks to get it working. That said, it is common in our pipeline to have steps that can be skipped on demand. I'm wondering if, in Dagster's case, we would have to define multiple dags with permutations of the skipped steps or if that possibility is already contemplated.
Other than that, is there a way to dynamically interconnect jobs in a completely custom dag using just the GUI?
Hi Fabio. Glad you're finding Dagster helpful! It is possible to skip certain steps--Dagster skips execution of steps that didn't receive an input. Here are some docs: https://docs.dagster.io/concepts/ops-jobs-graphs/graphs#with-conditional-branching But in terms of configuring at runtime whether certain steps will execute (e.g. via UI), that's currently not possible.
🌈 1