https://dagster.io/ logo
#ask-community
Title
# ask-community
b

Brian Stroh

05/25/2023, 6:37 PM
Hello, I'm trying to find a good way to pass the same set of parameters to all assets within my job. RunConfig looks like it's meant to explicitly supply different Config parameters for each asset within the job, and it feels hacky to build a dictionary comprehension just to populate my Config for each asset with the same exact parameter values. Here is a simple example where I'm just trying to have one of my Config values use a default
datetime
.
Copy code
from datetime import datetime
from dagster import DagsterType, Config

DagsterDatetime = DagsterType(
    name="DagsterDatetime",
    description="Standard library `datetime.datetime` type as a DagsterType",
    type_check_fn=lambda _, obj: isinstance(obj, datetime),
)

class MyAssetConfig(Config):
    """Configuration parameters for my assets"""

    valuation_date: DagsterDatetime = datetime(year=2023, month=4, day=30)
    prior_valuation_date: str = "3/31/2023"
    model_folder: str = "myfolder/"
When I run
dagit
, pydantic gives me this error:
Copy code
...
    class MyAssetConfig(Config):
  File "pydantic\main.py", line 204, in pydantic.main.ModelMetaclass.__new__
  File "pydantic\fields.py", line 488, in pydantic.fields.ModelField.infer
  File "pydantic\fields.py", line 419, in pydantic.fields.ModelField.__init__
  File "pydantic\fields.py", line 539, in pydantic.fields.ModelField.prepare
  File "pydantic\fields.py", line 801, in pydantic.fields.ModelField.populate_validators
  File "pydantic\validators.py", line 718, in find_validators
The above exception occurred during handling of the following exception:
TypeError: issubclass() arg 1 must be a class
  File "pydantic\validators.py", line 709, in pydantic.validators.find_validators
Interestingly, it looks a lot like this pydantic issue that resurfaced two days ago, but I'm still using
typing==4.5.0
. This looks like a pretty basic typing issue, and I apologize if I just don't have enough exposure to how to implement these types in dagster. I'm using
dagster==1.3.4
. Does anyone know how to pass parameters to asset Config that don't have basic
int
,
str
or
bool
types?
👀 1
b

ben

05/26/2023, 12:00 PM
Hi Brian, This question raises the unfortunate distinction between two type systems - the Dagster Type system which can be used to specify the input and output types for ops and assets (and including custom types) and the config type system. These Dagster Types aren’t meant to be use with the config system, which uses basic Python/Pydantic types. The set of types is pretty limited - you can use primitives,
Lists
Dicts
, or
Config
-subclassed dataclasses. In this case you may just want to use a
str
or
float
for
valuation_date
since we don’t yet support
datetime
as a config type. We should be providing a better error message here.
2 Views