https://dagster.io/ logo
b

Ben Torvaney

04/01/2021, 1:12 PM
An additional (possibly dumb) question - does Dagit/dagster support mode alteration un the UI/CLI? For instance if I wanted to run a pipeline under a specific mode, but with one resource changed? Or do I have to create a new mode to support that use-case?
a

alex

04/01/2021, 3:00 PM
currently you have to make a new
Mode
b

Ben Torvaney

04/01/2021, 3:43 PM
Thanks, Alex 👍
s

sandy

04/01/2021, 4:51 PM
@Ben Torvaney - out of curiosity, what's a situation where it would be helpful to be able to do this in Dagit?
b

Ben Torvaney

04/01/2021, 6:10 PM
Well at the moment we have some pipelines for local development and testing which use local/mocked resources. Although occasionally it can be useful to pull in “real” data for one resource. Creating a new mode temporarily is fine, I was just wondering if there was another way
s

sandy

04/01/2021, 7:44 PM
That makes sense. If you want to be able to vary that behavior from Dagit / CLI, one approach would be to make a "composite" resource that does mock vs. real stuff depending on what config option you give it. E.g.
Copy code
@resource(config_schema={"environment": str}):
def my_resource(init_context):
    if init_context.resource_config["environment"] == "prod":
        return MyProdResource()
    else:
        return MyDevResource()
1
2 Views