Chaitya
04/03/2023, 8:01 PMTypeError: "Config" is immutable and does not support item assignment
@dataclass
class Config():
some: int
params: str
class DagsterConfig(Config):
some: int
params: str
# some logic that populates this object
config: Config = load_config()
@asset
def some_asset(config: DagsterConfig):
return something
materialize([asset], run_config={"ops": {"asset": config.asdict()}})
sandy
04/04/2023, 12:04 AMben
04/04/2023, 12:08 AMdataclass
here? dagster.Config
wraps pydantic.BaseModel
which acts similarly to a base Python dataclass.
You should be able to do e.g.
class DagsterConfig(Config):
some: int
params: str
config: DagsterConfig = load_config()
@asset
def some_asset(config: DagsterConfig):
return "foo"
materialize([asset], run_config=RunConfig(ops={"some_asset": config}))
Here, RunConfig
will marshall DagsterConfig
into the right dict format to pass into the assetChaitya
04/04/2023, 12:16 AMload_config
requires that the config datatype be defined as a dataclass directlyben
04/04/2023, 12:24 AMfrom dataclasses import dataclass
from dagster import Config
from pydantic import create_model
from pydantic.dataclasses import dataclass as pydantic_dataclass
@dataclass
class MyConfig:
foo: str
bar: int
MyDagsterConfig = create_model(
"MyDagsterConfig", __base__=(pydantic_dataclass(MyConfig).__pydantic_model__, Config)
)
config = MyDagsterConfig(foo="foo", bar=2)
print(config.foo)
which converts base dataclass -> pydantic and then uses multiple inheritance w/ Config to create a new model classfrom dataclasses import dataclass
from dagster import Config, RunConfig, asset, materialize
from pydantic import create_model
from pydantic.dataclasses import dataclass as pydantic_dataclass
@dataclass
class MyConfig:
foo: str
bar: int
MyDagsterConfig = create_model(
"MyDagsterConfig", __base__=(pydantic_dataclass(MyConfig).__pydantic_model__, Config)
)
@asset
def some_asset(config: MyDagsterConfig):
return config.foo
config = MyConfig(foo="foo", bar=2)
result = materialize(
[some_asset], run_config=RunConfig(ops={"some_asset": config.__dict__})
)
print(result.asset_value("some_asset"))
Chaitya
04/04/2023, 3:07 PM