i'd love some guidance on how i should think about...
# dagster-feedback
i'd love some guidance on how i should think about op
vs op
. i find myself wanting to write python, where everything is an argument to the op function, with sensible defaults. for example, i'm trying to write a function that lists objects in a bucket --
-- that are under a certain
. here's how i've written it now:
Copy code
@op(config_schema={"bucket": Field(str, default_value="my-bucket")})
def get_object_files(context, object_id):
    bucket = s3.Bucket(context.op_config["bucket"])
    for object in bucket.objects.filter(Prefix=f"{object_id}/"):
        ... do something
but should
be an input? why not? should i make it an input and have a
op that generates the value? Should
be a config, so that users can parameterize it from the launchpad? any thoughts would be appreciated!
hi @Stephen Bailey, good question. when I'm writing library ops, my rule of thumb to decide between input / config is "if it's likely that someone will want to change this value at runtime, make it an input, otherwise, make it config". I've found that the experience of wiring up stuff like a setup op to pass in a constant feels a bit clunkier than just writing
get_object_files.configured({"bucket": "some-cool-bucket"})
and using that in a job. Another way to put it is that there are (kinda) three times that you might want to modify a value used by an op: when you're setting up your job (i.e. underneath a job decorator), in the launchpad, and at runtime. Config is generally better for the first 2, but doesn't work at all for the final one.
and when you say, at runtime, you're talking about specifically from within the context of a job run, based on the output of another op?
yep exactly
❤️ 1
makes sense!