19913d85-f662-4518-a9a1-7adc1793dfd4.png){kind=small}
Channels
#announcements
Title
v
Viacheslav Nefedov
03/08/2021, 12:22 PMHello team! it that possible to configure mode in dagster configurations files?
s
schrockn
03/08/2021, 4:14 PMIt is not. The schema of a config file is actually determined from mode.
v
Viacheslav Nefedov
03/08/2021, 4:16 PMaha. thanks. I was just thinking of how to implement modes to sensors
to make it use the same connections as we use in pipelines
s
schrockn
03/08/2021, 4:19 PMcan you give me some more context on how you want to use modes within sensors?
v
Viacheslav Nefedov
03/08/2021, 4:24 PMbefore reading the docs I was trying to make it this way:
@sensor(pipeline_name="structure_pipeline",
ModeDefinition("local_dev", resource_defs={"in_folder": local_in_folder}),
ModeDefinition("prod", resource_defs={"in_folder": prod_in_folder}),
)
def my_directory_sensor(context):
in_dir = context.resources.in_folder
for filename in os.listdir(in_dir):
filepath = os.path.join(in_dir, filename)
if os.path.isfile(filepath):
yield RunRequest(
run_key=filename,
run_config={"solids": {"process_file": {"config": {"filename": filename}}}},
)
s
schrockn
03/08/2021, 4:25 PMGotcha. So you would like resources and modes available in sensors so that you can test the sensors?
v
Viacheslav Nefedov
03/08/2021, 4:25 PMyep
s
schrockn
03/08/2021, 4:25 PMmakes sense!
we don’t support this right now but something we will definitely look into
thanks for bringing this up
d
daniel
03/08/2021, 4:28 PMHI @Viacheslav Nefedov - can I ask one follow-up question, where were you imagining that you would supply the config for the resource / specify which mode to use? Maybe when starting the sensor?
v
Viacheslav Nefedov
03/08/2021, 4:28 PMactually the first idea was to separate environment settings from code so my devops could manage the settings not touching the code
so we were trying to move all the ModeDefinitions from pipeline files
@daniel mode seems for me to be environment related thing, so I was looking for placing it in workspace.yaml or in repo.py
as it doesn't a working option for now so I was moving all the definition to files that are being imported from pipeline files and defined mode there
but this doesn't work with sensors. Meaning I haven't found a way to select a proper ModeDefinition with custom code yet :)