Dagster Jarred08/08/2022, 9:01 PM
David Jayatillake08/09/2022, 5:06 PM
Remco Loof08/11/2022, 2:27 PM
Dagster Jarred08/14/2022, 5:23 PM
yunyao Iris pan08/16/2022, 1:03 AM
Matthew Seal08/18/2022, 8:22 PM
Sterling Paramore08/23/2022, 3:54 PM
after adding the code to the image in the
. I was thinking I could: 1. Build the standard image using
2. Tag the image created something like
dagster-cloud serverless build
3. Create a new Dockerfile that looks like:
4. Tag that custom image with the same tag as the standard image and push that to the dagster serverless registry. Think that will work? Any gotchas?
FROM dagsterstandard RUN cd meltano && meltano install
Jules Huisman (Quantile)08/24/2022, 6:48 AM
Leo Qin08/25/2022, 12:10 AM
Sterling Paramore08/30/2022, 10:34 PM
Sterling Paramore08/31/2022, 4:03 PM
My suspicion is that it ran out of local disk space for storing temporary logs in a sqlite database. How much disk space should we have available?
This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: (sqlite3.OperationalError) disk I/O error
Adrien09/05/2022, 1:27 PM
dagster-cloud serverless deploy --location-name "moo" --package-name moo
... seems to be stemming from this error:
CalledProcessError: Command '['docker', 'build', PosixPath('.'), '-t', '<http://657821118200.dkr.ecr.us-west-2.amazonaws.com/serverless-agent-efb6f0bb-fc57-39b9-b07e-8b5704af5a51:moo|657821118200.dkr.ecr.us-west-2.amazonaws.com/serverless-agent-efb6f0bb-fc57-39b9-b07e-8b5704af5a51:moo>', '-f', '-', '--platform', 'linux/amd64']' returned non-zero exit status 1.
... and I can't find this
$ dagster-cloud serverless deploy --location-name "moo" --package-name moo Docker version 20.10.17, build 100c70180f Sending build context to Docker daemon 143.5MB Step 1/11 : FROM python:3.8-slim ---> bdd3315885d4 Step 2/11 : COPY *dagster_cloud_pre_install.sh /dagster_cloud_pre_install.sh COPY failed: no source files were specified
file anywhere. Anyone has any idea what might be happening?
Adrien09/07/2022, 10:53 AM
packages. I was wondering if we missed such an image? We did a little bit of digging on Docker Hub & the like but couldn't find anything 🤷
Kevin Martin09/07/2022, 12:21 PM
Kevin Martin09/07/2022, 12:25 PM
Heber Aguilar Calvo09/12/2022, 2:20 PM
github repository proposed by Dagster, but now we would like to connect Dagster with another github repository. I looked in the UI but I did not find how to change repos. Do you know how we can do that? @Stéfanny Robert
Sterling Paramore09/13/2022, 10:39 PM
Ricky Kim09/14/2022, 1:56 PM
David Jayatillake09/18/2022, 9:31 PM
Adrien09/20/2022, 3:32 PM
Zach P09/21/2022, 9:59 PM
This seems like a simple issue of it not picking up my
DagsterCloudHTTPError: 404 Client Error: Not Found for url: <https://dagster.cloud//prod/graphql>: Not Found Error: No serverless registry information found - your serverless deployment may still be activating.
correctly, but I’ve set it to what I believe it should be and it doesn’t seem to be working. I didnt use the template, but have copied and edited the .github, workspace.yaml, and dagster-cloud.yaml over. Do I perhaps need to set
, if so what do I include in the URL (EG: does it need to be https://dagster.cloud/myOrg/prod , or perhaps without prod or https? Currently I have
Zach P09/22/2022, 8:29 PM
YH09/23/2022, 2:11 PM
Zach P09/26/2022, 5:16 PM
Further up above, we see that the env_var dictionary that is constructed also adds this extra whitespace to AWS_ACCESS_KEY_ID and not to any other fields.
Dockerfile:44 -------------------- 42 | ENV DAGSTER_CLOUD_URL=*** 43 | ENV ACTIONS_STEP_DEBUG=true 44 | >>> ENV AWS_ACCESS_KEY_ID= *** 45 | -------------------- ERROR: failed to solve: Syntax error - can't find = in "***". Must be of the form: name=value Error: buildx failed with: ERROR: failed to solve: Syntax error - can't find = in "***". Must be of the form: name=value
I’ve checked and my secret itself has no whitespace in it. Any assistance here would be great 🙂, also perhaps confirmation regarding if serverless users should be passing envs in this manner, and if not, what the ‘best practice’ alternative is
##[debug] "AWS_SECRET_ACCESS_KEY": "***", ##[debug] "DAGSTER_CLOUD_URL": "***", ##[debug] "ACTIONS_STEP_DEBUG": "true", ##[debug] "AWS_ACCESS_KEY_ID": " *** "
Jules Huisman (Quantile)09/29/2022, 9:44 AM
It also doesn't run our pipeline, and doesn't notify us that the pipeline was not executed. So the only way we know is by inspecting the dashboard.
dagster._core.errors.DagsterUserCodeUnreachableError: dagster._core.errors.DagsterUserCodeUnreachableError: Failure loading server endpoint for prod:change_data: botocore.exceptions.ClientError: An error occurred (ThrottlingException) when calling the RegisterTaskDefinition operation (reached max retries: 10): Rate exceeded
David Merritt10/04/2022, 7:39 PM
David Merritt10/04/2022, 10:23 PM
James Hale10/05/2022, 10:55 PM
fails when attempting to upload the generated image (it appears). Any suggestions for how to resolve this? Attempting to run:
dagster-cloud deploy serverless ...
dagster-cloud serverless deploy --location-name tasks --deployment prod
CalledProcessError: Command 'echo [REDACTED] | docker login --username AWS --password-stdin <http://657821118200.dkr.ecr.us-west-2.amazonaws.com/serverless-agent-ed093e88-dd8e-34b6-a653-53ad81fae8a1|657821118200.dkr.ecr.us-west-2.amazonaws.com/serverless-agent-ed093e88-dd8e-34b6-a653-53ad81fae8a1>' returned non-zero exit status 1.
James Hale10/13/2022, 9:43 PM