https://dagster.io/ logo
#dagster-feedback
Title
# dagster-feedback
c

Colin Sullivan

01/02/2022, 5:42 PM
Hello, I'm trying to put together a POC to show to my team about how we could imagine migrating exiting code bases to dagster, which looks so so promising for us. The trouble I'm having is thinking about the
context
argument for ops. It feels odd to use the context object in the body of an op function to access a value, especially since the body of the job function doesn't pass the context argument explicitly. I’d rather express those values as parameters to the op function and use the body of the job function to pass explicitly. This would make testing more straightforward to avoid needing to mock a context object and simplify migrating existing projects. Similarly, I feel like I'd prefer to write a schema for an entire Job, rather than on a per op basis. Then have job function take a context or run_config argument, that defines the necessary values to pass to each op in the body. Am I missing the magic that explains why I ought to resist the urge to structure a pipeline like this? Is it possible to access the run_config in the body of the job function? And can I define the schema for an entire job?
2 Views