Jaewoo Park
08/22/2023, 3:20 PM/opt/dagster/dagster_home
doesnt actually hold anything. Does helm deployed dagster generate a dagster.yaml? If not if we wanted to get the DagsterInstance how would you go about doing so?Quentin Gaborit
08/22/2023, 3:30 PMdagster.yml
is generated by the Helm chart, the templates for object of the files being located at https://github.com/dagster-io/dagster/tree/master/helm/dagster/templates/helpers/instanceQuentin Gaborit
08/22/2023, 3:32 PMDAGSTER_HOME
needs to match the dagsterHome:
defined in the Helm chart globalJaewoo Park
08/22/2023, 3:32 PMdagster.yaml
not found error. Ah yeah it does seem like it templates a dagster.yaml hmmben
08/22/2023, 4:39 PMdagster.yaml
file is supposed to be created from a config mapping which is mounted as a volume in the deployment. Are you able to see this config mapping in your cluster?ben
08/22/2023, 4:39 PMJaewoo Park
08/22/2023, 4:50 PMexecute_job
from within another job.ben
08/22/2023, 4:58 PMcontext.instance
(rather than DagsterInstance.get()
). From there you could call execute_job
, however this is going to try to execute the job synchronously in the op.
You may instead want to queue/submit the job so it runs through normal channels (e.g. in a separate pod). You can do this using the GraphQL API or by attaching a sensor: https://github.com/dagster-io/dagster/discussions/7160#discussioncomment-2473920Jaewoo Park
08/22/2023, 5:54 PM