Hello, I've been using dagster for a while (great job guys by the way!), however I'm hitting a roadblock: • The backfill using the CLI appears to work properly • The backfill using dagit doesn't do anything (no log, no action after clicking the button) Has anyone here already experienced this issue? Does anyone can tell me how to have debug/verbose mode enabled when using dagit? Many thanks! Tested on 0.9.21 and 0.9.22.post0 Previously the backfill with the UI on 0.9.5 worked properly

Hi - does the button spin after you press it?

@sandy Hi, well it spins and then goes back to its previous state:

Very strange. You can go to <dagit url>/flags to turn on debug logging in the browser console

Was hoping for an error, but I don't see anything outrageous in the console

Sorry this is giving you trouble - is there any way you'd be able to post code to reproduce this locally? (Filed an issue to improve our verbose logging to be more helpful in cases like this in the future: https://github.com/dagster-io/dagster/issues/3438)

@daniel I'll investigate more on my end, and if I can't find anything I'll reduce my code to the bare minimum for a reproducible case - This feature would indeed be very nice!

@Adonis another thing that @dish on our team suggested that might work - if you're able to use chrome devtools to capture the Websocket request and response that goes out when you press the launch button and send that our way, that would also be very useful for us to see what's happening

@daniel Here it is:

Ah, OK, so I do see an InvalidStepError in there, which we just added better error handling for a couple of days ago actually (but that wouldn't be in 0.9.22 yet). Two things are surprising to me: • that the same command is passing in the CLI, I'd expect that to also give you some kind of InvalidStepError • That you're not seeing some kind of catchall red box that says "An unexpected error occurred"

Indeed, given https://github.com/dagster-io/dagster/commit/4253bc949afb071557df4613c1d418e6f503b55f I'd have expected that red box as well

OK, I think the red box might be in 0.9.22pre0. So that could solve that mystery

Yeah I'm obtaining this through the CLI directly:

$ dagster pipeline backfill -p total_groene_mient_etl_gm -w ./dagster/workspace.yaml  --location scp_data_app.repos.groene_mient --partitions 2020-04-01
$ PYTHONPATH=. dagster pipeline backfill -p total_groene_mient_etl_gm -w ./dagster/workspace.yaml  --location scp_data_app.repos.groene_mient --partitions 2020-04-01

     Pipeline: total_groene_mient_etl_gm
Partition set: monthly_total_groene_mient_etl_gm
   Partitions: 
                2020-04-01

Do you want to proceed with the backfill (1 partitions)? [y/N]: y
Launching runs... 
Launched backfill job `lpugyymr`

$ dagster - DEBUG - total_groene_mient_etl_gm - 5b832854-a8c9-4631-922d-c4399a16f276 - 13471 - ENGINE_EVENT - Starting initialization of resources [asset_store, measurement_metadata_repo, measurement_publisher, measurement_repo].
2020-12-17 18:19:25 - dagster - DEBUG - total_groene_mient_etl_gm - 5b832854-a8c9-4631-922d-c4399a16f276 - 13471 - ENGINE_EVENT - Finished initialization of resources [asset_store, measurement_metadata_repo, measurement_publisher, measurement_repo].
2020-12-17 18:19:25 - dagster - DEBUG - total_groene_mient_etl_gm - 5b832854-a8c9-4631-922d-c4399a16f276 - 13471 - PIPELINE_START - Started execution of pipeline "total_groene_mient_etl_gm".
2020-12-17 18:19:27 - dagster - DEBUG - total_groene_mient_etl_gm - 5b832854-a8c9-4631-922d-c4399a16f276 - 13471 - ENGINE_EVENT - Executing steps using multiprocess executor: parent process (pid: 13471)
2020-12-17 18:19:27 - dagster - DEBUG - total_groene_mient_etl_gm - 5b832854-a8c9-4631-922d-c4399a16f276 - 13471 - total.extract_total_house_energy_demand.compute - ENGINE_EVENT - Launching subprocess for total.extract_total_house_energy_demand.compute
2020-12-17 18:19:27 - dagster - DEBUG - total_groene_mient_etl_gm - 5b832854-a8c9-4631-922d-c4399a16f276 - 13471 - total.extract_total_house_energy_supply.compute - ENGINE_EVENT - Launching subprocess for total.extract_total_house_energy_supply.compute
2020-12-17 18:19:27 - dagster - DEBUG - total_groene_mient_etl_gm - 5b832854-a8c9-4631-922d-c4399a16f276 - 13471 - total.extract_total_house_heating_consumption.compute - ENGINE_EVENT - Launching subprocess for total.extract_total_house_heating_consumption.compute

and, sorry, I just verified, I think the red box change is coming in a couple weeks in 0.10.0, not 0.9.21 or 0.9.22pre0. So that explains that.

@daniel Sorry for the delay, here is the JSON from the WS before an InvalidStepError:

{
  "id": "33",
  "type": "start",
  "payload": {
    "variables": {
      "backfillParams": {
        "selector": {
          "partitionSetName": "monthly_total_schoonschip_etl_sc",
          "repositorySelector": {
            "repositoryName": "schoonschip",
            "repositoryLocationName": "scp_data_app.repos.schoonschip"
          }
        },
        "partitionNames": [
          "2020-11-01"
        ],
        "reexecutionSteps": [
          "total.compute",
          "extract_total_house_battery_consumption.compute",
          "transform_total_house_battery_consumption.compute",
          "load_total_house_battery_consumption.compute",
          "extract_total_house_battery_production.compute",
          "transform_total_house_battery_production.compute",
          "load_total_house_battery_production.compute",
          "extract_total_house_battery_soc.compute",
          "transform_total_house_battery_soc.compute",
          "load_total_house_battery_soc.compute",
          "extract_total_house_heating_consumption.compute",
          "transform_total_house_heating_consumption.compute",
          "load_total_house_heating_consumption.compute",
          "extract_total_house_pv_production.compute",
          "transform_total_house_pv_production.compute",
          "load_total_house_pv_production.compute"
        ],
        "fromFailure": false,
        "tags": []
      }
    },
    "extensions": {},
    "operationName": "LaunchPartitionBackfill",
    "query": "mutation LaunchPartitionBackfill($backfillParams: PartitionBackfillParams!) {\n  launchPartitionBackfill(backfillParams: $backfillParams) {\n    __typename\n    ... on PartitionBackfillSuccess {\n      backfillId\n      __typename\n    }\n    ... on PartitionSetNotFoundError {\n      message\n      __typename\n    }\n    ... on PythonError {\n      message\n      stack\n      __typename\n    }\n  }\n}\n"
  }
}

Thanks! For some reason it doesn’t seem to like one of those steps that’s being passed in the reexecutionSteps key (which is almost certainly a bug). If it’s possible to post the code for the pipeline I think we should be able to get to the bottom of this

@daniel Fair enough, though as I mentioned before, I need to strip down the code to a small reproducible version of this issue, since it was made generic to serve for several of our use cases. It would be quite unreadable if I simply copy and pasted it

Sure -it's possible we'd be able to find the bug just from inspecting the original code (even if it's big or we can't execute it locally). But if you're willing to strip it down for us that's great too - either option works!