The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

yeah my favorite part is that it’s so complicated they made a whole visualization tool where you can watch the candy move around

lol, is there really? i'm not surprised to be honest

yeah check this out—<http://rxmarbles.com/#combineLatest> you can drag the input candy around and it actually updates the output candy live

hey folks—been playing around with the config editor a bunch, what do we think about auto-populating the config template a bit further? I’d really like to be able to add a solid, and upon autocompleting the solid name, get a complete solid config block I can fill in - it’s super hard to remember the names without the type explorer also on the page

image.png

there are going to be cases of “sparse” config

where we import libraries of “resources” that schematize their entire config space

in which case scaffolding would not be useful

hmm - that’s true… I don’t think we want to autocomplete the blocks for all config, just for solids where once you specify one var you’re required to provide the entire block?

```
(dagster) [schrockn@mbp ~/code/dagster/python_modules/dagster/dagster/dagster_examples (schrockn/execution-from-dagit)]$ dagster pipeline scaffold_config pandas_hello_world
context:
  default:
    config:
      log_level: ''
execution:
  serialize_intermediates: true
expectations:
  evaluate: true
solids:
  load_num_csv:
    config:
      path: path/to/something
```

so this is the current result of scaffodling the config

imagine a much much more complicated config section that is system-specified

like under “execution” there is only “serialize_intermediates”

but i can imagine dozens of optiosn with complicated nesting as we move forward

but i agree that we need some aspect of this