Hello team! it that possible to configure mode in ...
# announcements
Hello team! it that possible to configure mode in dagster configurations files?
It is not. The schema of a config file is actually determined from mode.
aha. thanks. I was just thinking of how to implement modes to sensors
to make it use the same connections as we use in pipelines
can you give me some more context on how you want to use modes within sensors?
before reading the docs I was trying to make it this way:
@sensor(pipeline_name="structure_pipeline", ModeDefinition("local_dev", resource_defs={"in_folder": local_in_folder}), ModeDefinition("prod", resource_defs={"in_folder": prod_in_folder}), ) def my_directory_sensor(context): in_dir = context.resources.in_folder for filename in os.listdir(in_dir): filepath = os.path.join(in_dir, filename) if os.path.isfile(filepath): yield RunRequest( run_key=filename, run_config={"solids": {"process_file": {"config": {"filename": filename}}}}, )
Gotcha. So you would like resources and modes available in sensors so that you can test the sensors?
makes sense!
we don’t support this right now but something we will definitely look into
thanks for bringing this up
HI @Viacheslav Nefedov - can I ask one follow-up question, where were you imagining that you would supply the config for the resource / specify which mode to use? Maybe when starting the sensor?
actually the first idea was to separate environment settings from code so my devops could manage the settings not touching the code
so we were trying to move all the ModeDefinitions from pipeline files
@daniel mode seems for me to be environment related thing, so I was looking for placing it in workspace.yaml or in repo.py
as it doesn't a working option for now so I was moving all the definition to files that are being imported from pipeline files and defined mode there
but this doesn't work with sensors. Meaning I haven't found a way to select a proper ModeDefinition with custom code yet :)