Stephen Bailey
12/12/2022, 11:48 AMDefinitions
way of defining the code location. What's more, it paves the way for enabling Dagster to promote a default (optional) project structure, if sensors and schedules were able to be loaded from package directories in the same way that assets are. Here's what I'd love to see.
1. a new function called build_default_definitions
from dagster import Definitions, load_assets_from_package_name, load_sensors_from_pacakge_name, load_schedules_from_pacakge_name, load_jobs_from_pacakge_name
def build_default_definitions(
asset_package_name=".assets",
sensor_package_name=".sensors",
schedules_package_name=".schedules",
jobs_package_name=".jobs"
):
return Definitions(
assets=load_assets_from_package_name(asset_package_name),
sensors=load_sensors_from_pacakge_name(sensor_package_name),
jobs=load_jobs_from_pacakge_name(jobs_package_name),
schedules=load_schedules_from_pacakge_name(schedules_package_name),
)
then, in a user's auto-generated project __init__.py
from dagster import build_default_definitions
def = build_default_definitions()
This would make alternative project structures opt-in: users would never have to worry about whether a sensor, asset or job they added is getting sucked into the code location ; if they put it in the right folder, it just happens.