The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

is it possible to use `__post_init_post_parse__` in pydantic resources? my use case is that i want to construct a client property on the resource, but only want to supply the secrets. here's what i expect to be able to do:

```from dagster import asset, materialize, ConfigurableResource

class FooResource(ConfigurableResource):
    api_key = "foo"

    def __post_init_post_parse__(self):
        self.client = get_client(self.api_key)

@asset
def thing(context, foo: FooResource):
    foo.client.do_something()


result = materialize([thing], resources={"foo": FooResource(api_key="test")})
print(result)```

The spaghetti I’ve been cooking on my resources that works reasonably well is something like this:
```class UtilsS3IOManager(ConfigurableIOManager):
    class Config:
        extra = "allow"
        frozen = False

    bucket: str

    @property
    def s3(self):
        if not hasattr(self, "_s3"):
            self._s3 = construct_s3_client()    
        return self._s3```

oi, with the following
```class DatadogApiResource(ConfigurableResource):
    api_key: str
    app_key: str

    @property
    def client(self):
        if not hasattr(self, "_client"):
            self._client = self.get_client()
        return self._client```
i get this:
```dagster._core.errors.DagsterInvalidInvocationError: 'DatadogApiResource' is a Pythonic resource and does not support manipulating undeclared attribute '_client' as it inherits from 'pydantic.BaseModel' without extra="allow". If trying to maintain state on this resource, consider building a separate, stateful client class, and provide a method on the resource to construct and return the stateful client.```

You need  the `class Config` block with the `extra` and `frozen` params as I sent to make sure the Resources aren’t frozen

that’s why I referred to it as spaghetti, it’s probably not what the Dagster team would recommend :smile:

You can use PrivateAttr from pydantic for defining internal variables, looks slightly nicer tha hasattr. <https://dagster.slack.com/archives/C01U954MEER/p1682319379488309?thread_ts=1682226946.013319&amp;channel=C01U954MEER&amp;message_ts=1682319379.488309|https://dagster.slack.com/archives/C01U954MEER/p1682319379488309?thread_ts=1682226946.013319&amp;channel=C01U954MEER&amp;message_ts=1682319379.488309>

just my two cents, using the <https://docs.python.org/3/library/functools.html#functools.cached_property|functools.cached_property> seems to work pretty well and then you don't have to modify the class yourself

<@U02M08FP55E> this is much nicer! Will change mine to follow this as well

Thanks all for your feedback - we’re considering changing our recommendations around state management in resources and providing corresponding lifecycle methods. An RFC for this <https://github.com/dagster-io/dagster/pull/13938|change is up here> - the PR description goes over the rationale + new patterns. If you have any input please leave it there!