https://dagster.io/ logo
s

sandy

08/05/2022, 3:27 PM
<!channel> - Announcing... ☎️📞👋 Dagster 1.0: Hello 👋📞☎️

Hello from the other side

! Data teams who want to access what’s unique about Dagster now have a fully stable foundation to build on. 1.0 doesn’t include any seismic changes - rather, it marks that we’ve put the finishing touches on Dagster’s core abstractions. @schrockn committed Dagster’s first line of code four years ago. Since then, Dagster has had 463 releases and grown from an experimental programming framework to a production-grade orchestrator with software-defined assets, schematized config, rich logging, sensors, thirty three integration libraries... and more. Over 200 people have contributed code to Dagster, and its Slack community has grown to 4,300 members. Through all this, we held off on declaring Dagster 1.0, because we believe iteration results in better software - if you stick with the first approach that occurs to you, you’ll likely end up far away from the global optimum. Over that time, we made tweaks to Dagster’s core APIs that substantially improved them. Eventually, we found we weren’t making breaking changes. Last year, we arrived at stable compute abstractions - ops, graphs, jobs, schedules, and sensors. This year, we did the same with the asset layer - software-defined assets, materializations, and asset partitions. With all the core abstractions now stable, it was time to make things official and declare 1.0. 1.0 includes some other significant changes and improvements too: • Example revamp - All Dagster examples are revamped with a consistent project layout, descriptive names, and more helpful README files. • Repository-wide default executors and loggers - if specified, these will be used for any jobs which do not explicitly set an
executor_def
or
logger_defs
. • Run status sensors can now yield
RunRequests
-
allowing you to kick off a job in response to the status of another job. • Input managers - when loading an upstream asset or op output as an input, you can now set custom loading behavior using the
input_manager_key
argument to AssetIn and In. • Global asset lineage graph - back and better than before. The graph clusters together assets that are in the same group, and the query bar allows you to visualize a custom slice of your asset graph. • dagster project CLI - contains commands for bootstrapping Dagster projects from scratch or from examples, with our recommended project structure. • Legacy and deprecated API removal - many deprecated APIs have been removed. Take a look at the migration guide for details. • Local-to-prod guide - a new guide on how config, resources, and environment variables fit together to make jobs and assets portable across environments. • Library versioning - Dagster’s integration libraries are now at version 0.16 and are compatible with Dagster 1.0. We’ll mark them 1.x after we audit their APIs in the same way we’ve audited the core APIs. ResourcesFull release notesMigration guide And thank you to everyone who helped us name this release!
👌 54
🚶 19
doge 12
❤️ 20
🎉 35
daggy cowboy 11
baby yoda 7
☎️ 7
pog 3
📱 7
🎉 61
blob hero 14
🎅 12
🦍 9
attention hello 7
🗣️ 8
🌈 2
big dag eyes 12
🐣 7
1️⃣ 3
s

schrockn

08/05/2022, 3:29 PM
🪦🪦🪦 RIP
@solid
@pipeline
🪦🪦🪦
⚱️ 6
blob salute 5
🪦 1
l

Luca Molari

08/05/2022, 4:15 PM
Now I can no longer be fired for having chosen it over airflow blob cheer
dead 3
😂 1
🌈 9
blob cheer 1
pog 8
dagster spin 1
m

Max Wong

08/07/2022, 8:18 AM
just noticed latest version for some of dagster packages are listed as
0.16.1
, is this expected?
s

schrockn

08/07/2022, 12:01 PM
Hey @Max Wong yes this is expected. The following language is in the changelog, but we need to surface it in a more prominent spot: “”Dagster’s integration libraries haven’t yet achieved the same API maturity as Dagster core. For this reason, all integration libraries will remain on a pre-1.0 (0.16.x) versioning track for the time being. However, 0.16.x library releases remain fully compatible with Dagster 1.x. In the coming months, we will graduate integration libraries one-by-one to the 1.x versioning track as they achieve API maturity.“”
🎉 1
👍 1
s

Stefan Adelbert

08/08/2022, 10:28 AM
@Max Wong That caught me out today. Where did you take that screenshot from? Is there a "right" place for a peasant like me to view the latest versions for all dagster packages?
m

Max Wong

08/08/2022, 10:33 AM
@Stefan Adelbert the screenshot is from dependabot alerts. As for checking latest package version, pypi should get you covered
👍 1