https://dagster.io/ logo
k

Ken

06/09/2020, 11:45 PM
Hello Guys, I am having an issue running solidSubset with "Launch Execution". Details Posted in Comment Sections. My Goal is to run solidSubset on a GKE Launch-Execution. But it is failing when I pass inputs values for non-first-solid.
My Pipeline has 2 solids: [image 1] print_user_information -> solid_sleep My goal is to be able to run launch-execute only "solid_sleep" on GKE. Where am I doing wrong? print_user_information: • Input: Optional<string> • Output: Int solid_sleep: • Input: Int In the dagit playground, when I do "Launch Execution" with this config, it works ( creates new tab showing the GKE Job launch ) [Image 2]
Copy code
solids:
  print_user_information:
    inputs:
      cmd: 
        value: "echo hello world"
But when I updated the solidSubset to run only "solid_sleep", and do "Launch Execution", it fails [Image 3 + 4] Config:
Copy code
solids:
  solid_sleep:
    inputs:
      sleep_time: 2
However If I do "Start Execution" Instead, it works [Image 5] I opened the dev console and found some stack trace. posted in the comment section. [Note 6] ( adding images and notes .... done.)
Image 1:
Image 2 : successful launch for full pipeline
Image 3 + 4: Config error :
Image 5: Start-execution subset only solid 2 successful
Note 6: Stack trace:
Copy code
{
  "__typename": "PythonError",
  "message": "dagster.core.errors.DagsterInvalidConfigError: Error in config for pipeline test_pipeline\n    Error 1: Undefined field \"inputs\" at path root:solids:solid_sleep. Expected: \"{ }\".\n",
  "stack": [
    "  File \"/usr/local/lib/python3.6/dist-packages/dagster_graphql/implementation/utils.py\", line 14, in _fn\n    return fn(*args, **kwargs)\n",
    "  File \"/usr/local/lib/python3.6/dist-packages/dagster_graphql/implementation/execution/launch_execution.py\", line 29, in launch_pipeline_execution\n    return _launch_pipeline_execution(graphene_info, execution_params)\n",
    "  File \"/usr/local/lib/python3.6/dist-packages/dagster_graphql/implementation/execution/launch_execution.py\", line 65, in _launch_pipeline_execution\n    step_keys_to_execute=step_keys_to_execute,\n",
    "  File \"/usr/local/lib/python3.6/dist-packages/dagster_graphql/implementation/external.py\", line 200, in get_execution_plan_index_or_raise\n    external_pipeline=external_pipeline, environment_dict=environment_dict, mode=mode,\n",
    "  File \"/usr/local/lib/python3.6/dist-packages/dagster_graphql/implementation/context.py\", line 203, in create_execution_plan_index\n    step_keys_to_execute=step_keys_to_execute,\n",
    "  File \"/usr/local/lib/python3.6/dist-packages/dagster/core/execution/api.py\", line 42, in create_execution_plan\n    environment_config = EnvironmentConfig.build(pipeline_def, environment_dict, mode=mode)\n",
    "  File \"/usr/local/lib/python3.6/dist-packages/dagster/core/system_config/objects.py\", line 85, in build\n    environment_dict,\n"
  ]
}
m

max

06/10/2020, 12:17 AM
hi @Ken i think this is fixed on master, where the distinction between start and launch is going away
we'll be releasing 0.8.0 on thursday so if you'd prefer to wait for a release, this should be resolved by that release
🎉 2
k

Ken

06/10/2020, 12:19 AM
Oh that's awesome Max, thank you! Yes, I will wait until Thursday. Currently, we are using 0.7.12. Can't wait for the new 0.8.0, lots of cool features coming in. Thanks again for the great work!!