https://dagster.io/ logo
#dagster-cloud
Title
# dagster-cloud
t

Takahiro Tsuboi

08/16/2023, 3:52 AM
Hello, I want to use AWS lambda as an execution environment for job execution. Currently using ECS, but it takes about 3 minutes to secure resources such as ENI even though we just want to execute dbt. I want to use ECS for jobs that consume a lot of memory, and lambda for jobs that execute dbt. There used to be a similar project called Dagma, but it seems to have been removed now.
👍 1
p

prha

08/16/2023, 5:05 PM
Hi Takahiro. We don’t have support for an AWS lambda executor. Please upvote this issue, (or better yet create a separate new issue specifically for AWS Lambda) and we’ll consider it in our next roadmapping session
t

Takahiro Tsuboi

08/17/2023, 2:10 AM
thanks. I would be very happy to have it added to the roadmap. Serverless computing exists in the dagster cloud, but I think it is difficult to use in many cases because it often gives strong privileges to the execution environment.
p

prha

08/17/2023, 2:18 AM
Do the jobs you’re looking to execute on lambda typically run within the lambda runtime limits?
t

Takahiro Tsuboi

08/17/2023, 2:33 AM
Yes, it is a light job that should only take a couple of minutes.
10 Views