Adding @Todd de Quincey here to talk about the deployment error he's running into on Serverless cc: @ben @alex

Hi all, Thanks for starting the thread, Jarred. So the reproduction steps are as follows: 1. Created a local dagster project using

project_fully_featured

(Note: the primary branch is set as

master

by default) 2. Changed the primary branch to

main

3. No other changes made at all to the local repo 4. Pushed the code to github (personal account, as this is a quick PoC) 5. Created a Dagster Cloud account and using the serverless option, attempted to link Dagster Cloud to my repo 6. Noted that the

dagster_cloud.yaml

file was missing, so I copied this in from the Quickstart example 7. Updated the yaml file as per the below (I think this correct?) 8. Github actions run, but are failing with the below error 9. Thought that perhaps this was simply a config related issue, so I cloned the Quickstart repo and attempted to hook Dagster cloud up to it, but the Dagster cloud UI is throwing errors, and I now can’t change back to the old repo either

locations:
  - location_name: midnite_dagster
    code_source:
      package_name: project_fully_featured

#13 1.970 Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/site-packages (from jupyter-client>=6.1.5->nbclient<0.6.0,>=0.5.0->nbconvert->dagit==1.0.1->-r requirements.txt (line 2)) (23.2.1)
#13 1.972 Requirement already satisfied: tornado>=6.0 in /usr/local/lib/python3.8/site-packages (from jupyter-client>=6.1.5->nbclient<0.6.0,>=0.5.0->nbconvert->dagit==1.0.1->-r requirements.txt (line 2)) (6.2)
#13 2.612 Installing collected packages: dagster, dagster-graphql, dagit
#13 2.613   Attempting uninstall: dagster
#13 2.614     Found existing installation: dagster 1.0.3
#13 2.771     Uninstalling dagster-1.0.3:
#13 2.919       Successfully uninstalled dagster-1.0.3
#13 3.671   Attempting uninstall: dagster-graphql
#13 3.672     Found existing installation: dagster-graphql 1.0.3
#13 3.703     Uninstalling dagster-graphql-1.0.3:
#13 3.733       Successfully uninstalled dagster-graphql-1.0.3
#13 3.855   Attempting uninstall: dagit
#13 3.857     Found existing installation: dagit 1.0.3
#13 3.878     Uninstalling dagit-1.0.3:
#13 3.923       Successfully uninstalled dagit-1.0.3
#13 4.145 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
#13 4.145 dagster-spark 0.16.3 requires dagster==1.0.3, but you have dagster 1.0.1 which is incompatible.
#13 4.145 dagster-slack 0.16.3 requires dagster==1.0.3, but you have dagster 1.0.1 which is incompatible.
#13 4.145 dagster-pyspark 0.16.3 requires dagster==1.0.3, but you have dagster 1.0.1 which is incompatible.
#13 4.145 dagster-postgres 0.16.3 requires dagster==1.0.3, but you have dagster 1.0.1 which is incompatible.
#13 4.145 dagster-pandas 0.16.3 requires dagster==1.0.3, but you have dagster 1.0.1 which is incompatible.
#13 4.145 dagster-dbt 0.16.3 requires dagster==1.0.3, but you have dagster 1.0.1 which is incompatible.
#13 4.145 dagster-aws 0.16.3 requires dagster==1.0.3, but you have dagster 1.0.1 which is incompatible.
#13 4.146 Successfully installed dagit-1.0.1 dagster-1.0.1 dagster-graphql-1.0.1
#13 4.146 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: <https://pip.pypa.io/warnings/venv>
#13 DONE 4.4s

#14 [8/8] RUN dagster-cloud --version
#0 0.064 /bin/sh: 1: dagster-cloud: not found
#14 ERROR: process "/bin/sh -c dagster-cloud --version" did not complete successfully: exit code: 127
------
 > [8/8] RUN dagster-cloud --version:
#0 0.064 /bin/sh: 1: dagster-cloud: not found
------
Dockerfile:21
--------------------
  19 |     
  20 |     # Make sure dagster-cloud is installed. Fail early here if not.
  21 | >>> RUN dagster-cloud --version
  22 |     
  23 |     # placeholder section for adding env vars below
--------------------
error: failed to solve: process "/bin/sh -c dagster-cloud --version" did not complete successfully: exit code: 127
Error: buildx failed with: error: failed to solve: process "/bin/sh -c dagster-cloud --version" did not complete successfully: exit code: 127

Created a local dagster project using project_fully_featured (Note: the primary branch is set as master by default)

This was with

dagster project from-example

I assume? As per the build output, the

dagster-cloud

dependency is not present so needs to get added to

setup.py

or a

requirements.txt

cc @yuhan

Thought that perhaps this was simply a config related issue, so I cloned the Quickstart repo and attempted to hook Dagster cloud up to it, but the Dagster cloud UI is throwing errors, and I now can’t change back to the old repo either

This we’ll have to look in to cc @ben

This was with

dagster project from-example

I assume?

Correct presumption!