Dagster Jarred
08/14/2022, 5:23 PMTodd de Quincey
08/16/2022, 6:26 AMproject_fully_featured
(Note: the primary branch is set as master
by default)
2. Changed the primary branch to main
3. No other changes made at all to the local repo
4. Pushed the code to github (personal account, as this is a quick PoC)
5. Created a Dagster Cloud account and using the serverless option, attempted to link Dagster Cloud to my repo
6. Noted that the dagster_cloud.yaml
file was missing, so I copied this in from the Quickstart example
7. Updated the yaml file as per the below (I think this correct?)
8. Github actions run, but are failing with the below error
9. Thought that perhaps this was simply a config related issue, so I cloned the Quickstart repo and attempted to hook Dagster cloud up to it, but the Dagster cloud UI is throwing errors, and I now can’t change back to the old repo either
locations:
- location_name: midnite_dagster
code_source:
package_name: project_fully_featured
#13 1.970 Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/site-packages (from jupyter-client>=6.1.5->nbclient<0.6.0,>=0.5.0->nbconvert->dagit==1.0.1->-r requirements.txt (line 2)) (23.2.1)
#13 1.972 Requirement already satisfied: tornado>=6.0 in /usr/local/lib/python3.8/site-packages (from jupyter-client>=6.1.5->nbclient<0.6.0,>=0.5.0->nbconvert->dagit==1.0.1->-r requirements.txt (line 2)) (6.2)
#13 2.612 Installing collected packages: dagster, dagster-graphql, dagit
#13 2.613 Attempting uninstall: dagster
#13 2.614 Found existing installation: dagster 1.0.3
#13 2.771 Uninstalling dagster-1.0.3:
#13 2.919 Successfully uninstalled dagster-1.0.3
#13 3.671 Attempting uninstall: dagster-graphql
#13 3.672 Found existing installation: dagster-graphql 1.0.3
#13 3.703 Uninstalling dagster-graphql-1.0.3:
#13 3.733 Successfully uninstalled dagster-graphql-1.0.3
#13 3.855 Attempting uninstall: dagit
#13 3.857 Found existing installation: dagit 1.0.3
#13 3.878 Uninstalling dagit-1.0.3:
#13 3.923 Successfully uninstalled dagit-1.0.3
#13 4.145 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
#13 4.145 dagster-spark 0.16.3 requires dagster==1.0.3, but you have dagster 1.0.1 which is incompatible.
#13 4.145 dagster-slack 0.16.3 requires dagster==1.0.3, but you have dagster 1.0.1 which is incompatible.
#13 4.145 dagster-pyspark 0.16.3 requires dagster==1.0.3, but you have dagster 1.0.1 which is incompatible.
#13 4.145 dagster-postgres 0.16.3 requires dagster==1.0.3, but you have dagster 1.0.1 which is incompatible.
#13 4.145 dagster-pandas 0.16.3 requires dagster==1.0.3, but you have dagster 1.0.1 which is incompatible.
#13 4.145 dagster-dbt 0.16.3 requires dagster==1.0.3, but you have dagster 1.0.1 which is incompatible.
#13 4.145 dagster-aws 0.16.3 requires dagster==1.0.3, but you have dagster 1.0.1 which is incompatible.
#13 4.146 Successfully installed dagit-1.0.1 dagster-1.0.1 dagster-graphql-1.0.1
#13 4.146 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: <https://pip.pypa.io/warnings/venv>
#13 DONE 4.4s
#14 [8/8] RUN dagster-cloud --version
#0 0.064 /bin/sh: 1: dagster-cloud: not found
#14 ERROR: process "/bin/sh -c dagster-cloud --version" did not complete successfully: exit code: 127
------
> [8/8] RUN dagster-cloud --version:
#0 0.064 /bin/sh: 1: dagster-cloud: not found
------
Dockerfile:21
--------------------
19 |
20 | # Make sure dagster-cloud is installed. Fail early here if not.
21 | >>> RUN dagster-cloud --version
22 |
23 | # placeholder section for adding env vars below
--------------------
error: failed to solve: process "/bin/sh -c dagster-cloud --version" did not complete successfully: exit code: 127
Error: buildx failed with: error: failed to solve: process "/bin/sh -c dagster-cloud --version" did not complete successfully: exit code: 127
alex
08/16/2022, 2:30 PMCreated a local dagster project using project_fully_featured (Note: the primary branch is set as master by default)This was with
dagster project from-example
I assume?
As per the build output, the dagster-cloud
dependency is not present so needs to get added to setup.py
or a requirements.txt
cc @yuhan
Thought that perhaps this was simply a config related issue, so I cloned the Quickstart repo and attempted to hook Dagster cloud up to it, but the Dagster cloud UI is throwing errors, and I now can’t change back to the old repo eitherThis we’ll have to look in to cc @ben
Todd de Quincey
08/16/2022, 3:36 PMThis was withCorrect presumption!I assume?dagster project from-example