https://dagster.io/ logo
Title
g

Guillaume Jeusel

07/03/2021, 6:40 AM
Hello 👋 - Just starting to evaluate Dagster. I would like to take actions on the mocked resource in my ‘unittest’ ModeDefinition before executing the pipeline in my unittest (pytest). The purpose is to define the returned value of an API mocked call, hence be able to test the post-processing (only mocking the boundaries with the outside world). However, the magicmock (using ModeDefinition.mock_resource) seems to be only accessible by calling the actual
resource.resource_fn(None)
. Am I missing something ? Also, ideally, before any other test starts, this mock should be instantiated again (avoid side effects). Maybe one solution might be to permit adding mode definitions to a PipelineDefinition after instantiation ? That way, I would have some fixture adding the “unittest” mode on my pipelines, with resources mocked value being other fixtures usable inside my tests. As I don’t really see the point of having a ‘unittest’ mode defined in my production code. Altough I do see the benefit of having a mode mocking specific resources to avoid changing any system state and just allow the user to test the “Extract” and “Transform” part.
s

schrockn

07/04/2021, 4:35 PM
Hey Guillaume thanks for the question! We completely agree with your assessment of the ergonomic problems here. Our next release (coming out on Thursday) has core changes that will make this use case far more elegant and straightforward.
❤️ 1
👍 1
s

sandy

07/16/2021, 12:56 AM
Hey Guillaume - as Nick mentioned, our latest release contains a rework of Dagster's core APIs that should help address the problem you raised. Here's what it looks like to use the new APIs with resources that are supplied in a test: https://docs.dagster.io/guides/dagster/graph_job_op#a-pipeline-with-a-test-mode We're looking for feedback before we make these our main APIs, so I'd be curious to hear any thoughts you have