My team is evaluating implementing either Airflow on MWAA and Dagster-core on AWS — primarily for dbt, but one of the concerns I've heard is that Dagster might not be able to trigger or work with AWS Lambda functions. Does Dagster have any sort of support for these? Similar to what Airflow has here: https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/operators/lambda.html Is directly using Boto3 a viable solution? Or is there some pitfall I'm missing here?

You can write code within an asset or op via boto3 that trigger lambdas functions.

is there a performance tradeoff of doing this vs how Airflow handles Lambda?

The airflow Lambda operator just uses boto3 under the hood, so I'd think the performance would be the same

MWAA does the job you’ll probably need it to do, but it’s quite inflexible. Some issues that I’ve had with it over the past 12 months: • rigid Airflow versions which AWS are very slow at times to update. This also means that if there is a fix in a minor Airflow version (e.g. 2.5.x), you’ll either never get it or have to wait months and months for an upgrade • The upgrade of the MWAA environment between versions is cumbersome. It’s not like just updating a pip version. It’s very involved and IIRC, you need to spin up a new env, and do a migration process (snapshot meta DB, copy over etc etc etc). Not elegant at all • There is a local dev env, but it’s not great at all. Very bloated and I manually had to patch a number of bugs. • Using env vars is horrible. You need to use a bastardized approach of using an Airflow plugin. • No CI/CD out of the box. But it’s a very simple setup so not hard to implement. On the plus side though, it’s very low ops and personally I find the price reasonable-ish. I’ve also found it very reliable.