https://dagster.io/ logo
Join Slack
Powered by
hi team - what is the proper syntax for using Enum...
# ask-community
d
hi team - what is the proper syntax for using Enums with the new Config-style of defining op config schemas?
i have the following:
Copy code
from enum import Enum, auto

from pydantic import BaseModel, Field
from dagster import Config as DagsterConfig, op

class CompareOperator(Enum):
    LT = "LT"

class ColumnFilter(DagsterConfig):
    column: str = Field(description="column")
    threshold: float = Field(description="threshold")
    compare_operator: CompareOperator
    
    class Config:
        use_enum_values = True
        
ColumnFilter(column="col", threshold=0.1, compare_operator="LT")  # Resolves just fine

@op
def my_op(context, config: ColumnFilter):
    print(config)
but the op definition yields the following error:
Copy code
DagsterInvalidConfigDefinitionError: Error defining config. Original value passed: <enum 'CompareOperator'>. <enum 'CompareOperator'> cannot be resolved.
This value can be a:
    - Field
    - Python primitive types that resolve to dagster config types
        - int, float, bool, str, list.
    - A dagster config type: Int, Float, Bool, Array, Optional, Selector, Shape, Permissive, Map
    - A bare python dictionary, which is wrapped in Field(Shape(...)). Any values
      in the dictionary get resolved by the same rules, recursively.
    - A python list with a single entry that can resolve to a type, e.g. [int]
j
cc @ben
d
@jamie / @ben soft bump if anyone has any ideas. am sure i could revert to using op’s 
config_schema
again, but it’s a tad messier and would prefer to use the pydantic approach if possible.
k
I've got the same puzzle: Giving the enum type as below gives a reasonable error that dagster only allows the types available in the original config_schema, including Field:
Copy code
class MyConfig(Config):
    key: MyEnum
I then tried that approach:
Copy code
from dagster import (
    Enum as DagsterEnum,
    Field
)

class MyConfig(Config):
    key: Field(DagsterEnum.from_python_enum(MyEnum))
This however gives a pydantic validation error on the dagster Field
Copy code
RuntimeError: error checking inheritance of Field(<dagster._config.config_type.Enum object at 0x0000021B87AD2440>, default=@, is_required=True) (type: Field)
...
  File "pydantic\main.py", line 198, in pydantic.main.ModelMetaclass.__new__
  File "pydantic\fields.py", line 506, in pydantic.fields.ModelField.infer
  File "pydantic\fields.py", line 436, in pydantic.fields.ModelField.__init__
  File "pydantic\fields.py", line 557, in pydantic.fields.ModelField.prepare
  File "pydantic\fields.py", line 831, in pydantic.fields.ModelField.populate_validators
  File "pydantic\validators.py", line 760, in find_validators
The above exception occurred during handling of the following exception:
TypeError: issubclass() arg 1 must be a class
  File "pydantic\validators.py", line 751, in pydantic.validators.find_validators
so, in short, no luck either. Shame as its a neat feature, but I guess its early days
b
Hi David & kelvyn, apologies for the delayed response! Right now Enums aren’t well supported with the Pythonic config system, I have a PR out to fix this which I’m hoping to get in for tomorrow’s release.
👏 2
k
brilliant. I wish we could pump out changes as quick ! - though, admittedly, that's more often down to me than anyone else.
d
just saw that the latest 1.2.4 release containing the fix was dropped. can confirm my sample code above works now. cheers guys!
b
🙌 great to hear!
let us know if you run into any other snags w/ the config changes
💯 1
d
this is unrelated to configs, but ben would you know if there have been any recent changes to how 
context.pdb.set_trace()
works? on older versions of dagster, im quite confident i was able to properly set breakpoints for local dagit instances using that line before, but now i immediately get the following error:
Copy code
The above exception was caused by the following exception:
bdb.BdbQuit
  File "/home/david.tong/miniconda3/envs/dagster3/lib/python3.8/site-packages/dagster/_core/execution/plan/utils.py", line 54, in op_execution_error_boundary
    yield
  File "/home/david.tong/miniconda3/envs/dagster3/lib/python3.8/site-packages/dagster/_utils/__init__.py", line 439, in iterate_with_context
    next_output = next(iterator)
  File "/home/david.tong/miniconda3/envs/dagster3/lib/python3.8/site-packages/dagster/_core/execution/plan/compute_generator.py", line 122, in _coerce_solid_compute_fn_to_iterator
    result = invoke_compute_fn(
  File "/home/david.tong/miniconda3/envs/dagster3/lib/python3.8/site-packages/dagster/_core/execution/plan/compute_generator.py", line 116, in invoke_compute_fn
    return fn(context, **args_to_pass) if context_arg_provided else fn(**args_to_pass)
  File "/home/david.tong/repos/dagster_workflows/dagster_workflows/dagster_modules/some_job/ops/new_op.py", line 25, in run_some_job_large
    config = load_yaml("/home/david.tong/repos/dagster_workflows/nav_config.yaml")
  File "/home/david.tong/repos/dagster_workflows/dagster_workflows/dagster_modules/some_job/ops/new_op.py", line 25, in run_some_job_large
    config = load_yaml("/home/david.tong/repos/dagster_workflows/nav_config.yaml")
  File "/home/david.tong/miniconda3/envs/dagster3/lib/python3.8/bdb.py", line 88, in trace_dispatch
    return self.dispatch_line(frame)
  File "/home/david.tong/miniconda3/envs/dagster3/lib/python3.8/bdb.py", line 113, in dispatch_line
    if self.quitting: raise BdbQuit
j
Hey @David Tong do you mind asking that question in the main support channel? it’ll get more eyes from the dagster team, and likely be helpful to other users too!
👍 1
17 Views
Open in Slack
PreviousNext
Powered by