https://dagster.io/ logo
#ask-community
Title
# ask-community
c

clay

04/07/2023, 12:46 PM
Don't know if this is more of a python or Dagster question, but.... is there an easy way with Dagster, when using more than one code location, to make a custom function available to all code location and their modules/sub-modules? Or do I have to write and install a package?
🤖 1
d

Daniel Gafni

04/07/2023, 12:52 PM
If your code locations live in the same monorepo, you can install a package from local path. Otherwise, you can install the common package from
git+https
or
git+ssh
. I advise to publish it a private PyPI properly tho, this way different code locations don't need to be in sync with the common package version all the time. Installing from
git
would allow this too but it's a little harder to configure for docker builds etc
❤️ 1
c

clay

04/07/2023, 12:53 PM
They don't live in the same monorepo, unfortunately... they're all in separate repos and deployed in docker containers.
d

Daniel Gafni

04/07/2023, 12:54 PM
^ added comments about
git+...
installtion
c

clay

04/07/2023, 12:54 PM
Thank you, Daniel!
I was hoping Dagster had some type of official extension system that I didn't know about. 🙂
a

Andras Somi

04/07/2023, 12:56 PM
For what it’s worth we are doing the
git
way from Daniel’s solutions with some code that is used in different services across our systems. It’s doable but currently requires manually updating Dockerfiles with the latest version of the package, so it’s not as nice as pinning some version range in a requirements.txt. But it’s usable still.
👍 1
d

Daniel Gafni

04/07/2023, 12:57 PM
you can use
git+
with requirements.txt as well as with Poetry
shouldn't be a problem with that
a

Andras Somi

04/07/2023, 12:59 PM
Yes, that’s right. I think we had some other reasons (unrelated to this question) why we went for adding it in the Dockerfile, so just ignore the dockerfile part.
But if you want to control what version package you install from the repo via
git+
(eg. from
release/...
branches) you’ll end up manually updating your requirements.txt too.
d

Daniel Gafni

04/07/2023, 1:02 PM
yes