set the channel description: lets talk about orchestration without customer managed infrastructure

I saw in the video that even with Serverless the developer switched to vscode to do development. Do you foresee Dagster Cloud having an inbuilt IDE to use to keep the developer in one place should they so choose?

Hi all, small question; do the agents have a specific IP range that I can whitelist to access my RDS instance?

Adding @Todd de Quincey here to talk about the deployment error he's running into on Serverless cc: @ben @alex

Hi, how can I deploy codes in the existing github example directory in server-less deployment mode? For example, I want to try out the “project_fully_featured” example using dagster cloud

I noticed that the org invite only lasts for 15 min before it expires. It'd be a lot more convenient if that invite key lasted a couple hours so folks in different timezones could easily connect when they log in.

I need to customize the container that gets deployed to dagster cloud a bit. I need it to run

meltano install

after adding the code to the image in the

Dockerfile

. I was thinking I could: 1. Build the standard image using

dagster-cloud serverless build

2. Tag the image created something like

dagsterstandard

3. Create a new Dockerfile that looks like:

FROM dagsterstandard
RUN cd meltano && meltano install

4. Tag that custom image with the same tag as the standard image and push that to the dagster serverless registry. Think that will work? Any gotchas?

Is there a plan on the roadmap to define the region where you want your compute to exist? Currently all our data needs to make a round trip to the US, which slows certain processes which need to make a lot of requests.

hello - how does serverless pricing work with regard to sensors? Are you charged for the runtime of the sensor (ie every 30 seconds, unless otherwise configured), or only if the sensor detects work to kick off?

Started getting this error today during a serverless deploy. Not sure how to interpret it. Is it something wrong with ECS?

I go this error while running a large Meltano job:

This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: (sqlite3.OperationalError) disk I/O error

My suspicion is that it ran out of local disk space for storing temporary logs in a sqlite database. How much disk space should we have available?

Hi everyone 👋 Taking serverless for a spin and running into an error when running the command:

dagster-cloud serverless deploy --location-name "moo" --package-name moo

CalledProcessError: Command '['docker', 'build', PosixPath('.'), '-t', '<http://657821118200.dkr.ecr.us-west-2.amazonaws.com/serverless-agent-efb6f0bb-fc57-39b9-b07e-8b5704af5a51:moo|657821118200.dkr.ecr.us-west-2.amazonaws.com/serverless-agent-efb6f0bb-fc57-39b9-b07e-8b5704af5a51:moo>', '-f', '-',
'--platform', 'linux/amd64']' returned non-zero exit status 1.

... seems to be stemming from this error:

$ dagster-cloud serverless deploy --location-name "moo" --package-name moo

Docker version 20.10.17, build 100c70180f
Sending build context to Docker daemon  143.5MB
Step 1/11 : FROM python:3.8-slim
 ---> bdd3315885d4
Step 2/11 : COPY *dagster_cloud_pre_install.sh /dagster_cloud_pre_install.sh
COPY failed: no source files were specified

... and I can't find this

dagster_cloud_pre_install.sh

file anywhere. Anyone has any idea what might be happening?

Hi again 👋 We're working on a Docker image that'd make it easy to deploy our repositories to Dagster Cloud, from our pipelines. Ideally it'd basically be a small image, it'd have an installation of Docker, Python3.x, and the relevant

dagster-*

packages. I was wondering if we missed such an image? We did a little bit of digging on Docker Hub & the like but couldn't find anything 🤷

Hey all! I set up a serverless deployment, but ran into issues where it seemed like the Github connection was not finalizing, and whenever I was routed back to the deployment setup screen, the button to set up Github was still in place. When I looked a couple days later, it was there, but I couldn’t set up the sample project - I think either because I didn’t give permission beyond one (empty) repository, or because the account I connected with wasn’t an organization owner. Do either of those make sense? I did get it working using a completely different Github account, but is there a way to back that out and reconfigure the deployment with the initial account the right way? (I have another question, but I’ll post that separately.)

So, we have a serverless deployment now, with the example code. I’m trying to wrap my head around the relationship btw the deployment and a Github repo and a Dagster workspace / repo. If we have code in multiple Github repos, do we have to have multiple deployments for them? If not, how do we get them loaded as workspaces / repos under one deployment?

Hello everyone, We're free trialing Dagster Serverless Cloud version. We used the

dagster_starter_pack

github repository proposed by Dagster, but now we would like to connect Dagster with another github repository. I looked in the UI but I did not find how to change repos. Do you know how we can do that? @Stéfanny Robert

Would be nice to be able to test out sensor functionality in a branch deployment without forcing them to start out enabled.

Hi. We are trialing Dagster Serverless, and noticed that sometimes after deploying to prod, workspace and assets show not the latest commit just pushed to prod, but workspace and assets used to be defined in an old commit. If I rerun the Gitlab (we're using Gitlab) pipeline to redeploy, it shows correct workspace and assets, but I was wondering if this is a known behaviour.

I’m writing part 2 of Dabbling with Dagster! Right now if I were to make changes locally in VSCode (as far as I understand I can’t change the code in my repo in Dagster Cloud), I would need to run Dagster locally to test them. The other way would be to commit them to my PR branch and then Dagster Cloud branch deployments would reflect the changes I’ve made, but there is quite a bit of lag doing this between making my change and seeing if it works. I would end up committing a fair bit of junk to check it works.

Hi 👋 We're leveraging Dagster Cloud Serverless for most of our computes, but we have some workloads that are required to run on our infrastructure to account for governance and / or private networking requirements, so we're also keen to leverage a Hybrid Cloud solution. I was wondering if there was a "best practice" on how to best do this? Ideally we'd have a single graph, with some jobs/ops running in serverless, and some other running in our hybrid setup, is that possible? Thanks! 🙏

👋 Hey folks I’m going back to actually make sure the github CI/CD for dagster cloud serverless is actually working and I’m getting an error I think is realted to how I’ve set it up but I’m not 100% sure. My error is:

DagsterCloudHTTPError: 404 Client Error: Not Found for url: 
<https://dagster.cloud//prod/graphql>: Not Found
Error: No serverless registry information found - your serverless deployment may still be activating.

This seems like a simple issue of it not picking up my

ORGANIZATION_ID

correctly, but I’ve set it to what I believe it should be and it doesn’t seem to be working. I didnt use the template, but have copied and edited the .github, workspace.yaml, and dagster-cloud.yaml over. Do I perhaps need to set

DAGSTER_CLOUD_URL

, if so what do I include in the URL (EG: does it need to be https://dagster.cloud/myOrg/prod , or perhaps without prod or https? Currently I have

DAGSTER_CLOUD_API_TOKEN

and

ORGANIZATION_ID

set.

Question regarding IAM roles for serverless. Is there a way to set up a role to be assumed by dagster serverless? For branch deployments this doesn’t seem to hard since you can use github oidcs -> assume role during CICD, however, I cant find much information on how to set this up for non-branch deployments. Currently, looks like the only documentation on this is by setting the secret & access tokens to an iam user at the repo level. This isn’t really what we’d prefer. Would assuming a role in a custom image be sufficient? How long do those persist for?, or is there something else that would either be easier or block this from working? 🤔

Hi, I'm evaluating dagster-cloud and am wondering how I can configure dagster to load different AWS accounts for deployments: branch: • aws-stg-profile-a • aws-stg-profile-b prod: • aws-prod-profile-a • aws-prod-profile-b Would it be a matter of setting it up via GitHub actions'

configure-aws-credentials

?

Hey guys, still working on some IAM issues I’m having to enable our branch and main deployments in serverless. I’m wondering if there may be a bug. I’ve created some IAM roles that are assumed as a resource during runtime, however, I’m having issues passing the base user to the deployments. To attempt to do this, I’m setting the repos AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in the repo secrets to a low-priv user “dagster_user”. However, this ends up failing with the following error:

Dockerfile:44
--------------------
  42 |     ENV DAGSTER_CLOUD_URL=***
  43 |     ENV ACTIONS_STEP_DEBUG=true
  44 | >>> ENV AWS_ACCESS_KEY_ID= *** 
  45 |     
--------------------
ERROR: failed to solve: Syntax error - can't find = in "***". Must be of the form: name=value
Error: buildx failed with: ERROR: failed to solve: Syntax error - can't find = in "***". Must be of the form: name=value

Further up above, we see that the env_var dictionary that is constructed also adds this extra whitespace to AWS_ACCESS_KEY_ID and not to any other fields.

##[debug]  "AWS_SECRET_ACCESS_KEY": "***",
##[debug]  "DAGSTER_CLOUD_URL": "***",
##[debug]  "ACTIONS_STEP_DEBUG": "true",
##[debug]  "AWS_ACCESS_KEY_ID": " *** "

I’ve checked and my secret itself has no whitespace in it. Any assistance here would be great 🙂, also perhaps confirmation regarding if serverless users should be passing envs in this manner, and if not, what the ‘best practice’ alternative is

Are there others that experience the following error:

dagster._core.errors.DagsterUserCodeUnreachableError: dagster._core.errors.DagsterUserCodeUnreachableError: Failure loading server endpoint for prod:change_data: botocore.exceptions.ClientError: An error occurred (ThrottlingException) when calling the RegisterTaskDefinition operation (reached max retries: 10): Rate exceeded

It also doesn't run our pipeline, and doesn't notify us that the pipeline was not executed. So the only way we know is by inspecting the dashboard.

Hello - I am having an error on the initial activation of my repo for serverless. Can anyone help?

Question: is the serverless container/agent always running such that I can send an API call to run a one-off job/graph at any point with no spin up? Or is there some spinup init time before the job will run?

Hi there - I've been running into an issue attempting to deploy to Dagster serverless for the first time.

dagster-cloud deploy serverless ...

fails when attempting to upload the generated image (it appears). Any suggestions for how to resolve this? Attempting to run:

dagster-cloud serverless deploy --location-name tasks --deployment prod

Error:

CalledProcessError: Command 'echo [REDACTED] | docker login --username AWS --password-stdin <http://657821118200.dkr.ecr.us-west-2.amazonaws.com/serverless-agent-ed093e88-dd8e-34b6-a653-53ad81fae8a1|657821118200.dkr.ecr.us-west-2.amazonaws.com/serverless-agent-ed093e88-dd8e-34b6-a653-53ad81fae8a1>' returned non-zero exit status 1.

We're having a number of issues with our serverless deployment at the moment: 1. Runs are failing immediately - see error in thread 2. Some pages in the UI are not loading (e.g., deployment overview and schedules) - see error in thread

cross-post from #dagster-support https://dagster.slack.com/archives/C01U954MEER/p1665673706094059