Fabio Picchi
04/12/2023, 8:59 AMHi everyone, I'm new to Dagster and we're trying to bring it onboard at SharperShape. I feel I'm getting the gist of it and I find it amazingly helpful compared to our previous in-house scheduler, however, I would like to ask more experienced users about one particular use-case.
We work with GIS data and data collected from sensors in the real world tends to be noisy and require many tweaks to get it working. That said, it is common in our pipeline to have steps that can be skipped on demand. I'm wondering if, in Dagster's case, we would have to define multiple dags with permutations of the skipped steps or if that possibility is already contemplated.
Other than that, is there a way to dynamically interconnect jobs in a completely custom dag using just the GUI?
claire
04/13/2023, 9:18 PM