The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

:bug: Hey All! I think I found a bug in the dagaster-databricks connector, but wanted to confirm before I open up an issue on github.

Seems like there’s an issue adding custom tags to new databricks clusters at runtime since it tries to append to a read only list.

Error in comments :)

```2022-12-16 17:32:33 -0500 - dagster - ERROR - your_job - 6cdfbe88-3160-4352-9390-7bef393d6358 - 86261 - assetname - STEP_FAILURE - Execution of step "asset name" failed.

RuntimeError: Cannot modify ReadOnlyList

Stack Trace:
  File "/PATH/.venv/lib/python3.8/site-packages/dagster/_core/execution/plan/execute_plan.py", line 266, in dagster_event_sequence_for_step
    for step_event in check.generator(step_events):
  File "PATH/.venv/lib/python3.8/site-packages/dagster_databricks/databricks_pyspark_step_launcher.py", line 220, in launch_step
    databricks_run_id = self.databricks_runner.submit_run(self.run_config, task)
  File "PATH/.venv/lib/python3.8/site-packages/dagster_databricks/databricks.py", line 166, in submit_run
    tags.append({"key": "__dagster_version", "value": dagster.__version__})
  File "/PATH/.venv/lib/python3.8/site-packages/dagster/_utils/__init__.py", line 234, in __readonly__```

removing `run_config.clsuter.new._"custom_tags": [{"key": "evaluation", "value": "true"}],_` is a work around, but I’d expect this to either apply the tag or raise some config / resource validation error.