19913d85-f662-4518-a9a1-7adc1793dfd4.png){kind=small}
Channels
announcements
dagster-airbyte
dagster-airflow
dagster-bigquery
dagster-cloud
dagster-cube
dagster-dask
dagster-dbt
dagster-de
dagster-ecs
dagster-feedback
dagster-kubernetes
dagster-machine-learning
dagster-noteable
dagster-releases
dagster-serverless
dagster-showcase
dagster-snowflake
dagster-spatial
dagster-support
dagster-wandb
dagstereo
data-platform-design
events
faq-read-me-before-posting
gigs-freelance
github-discussions
introductions
jobs
random
tools
豆瓣酱帮
Title
c
clay
03/22/2023, 5:01 PMso.. in this example, I can't see the .sql file with glob.glob or by just trying to read it as a file. If I ssh into the Docker container, I see the sql file in there.... but I guess I'm not registering it as an asset so it is not being put into the container that's launched when I do a run?
j
jamie
03/22/2023, 5:26 PMhmm interesting. I’m not very well versed in how the container for an individual run is created and what’s included. If you were to run this locally on your computer does the file get found?
c
clay
03/22/2023, 5:27 PMNo, the file is available and visible inside of the code location container but not when the asset is materialized. The logging of glob.glob in the asset shown here returns an empty list
j
jamie
03/22/2023, 5:27 PMok and that happens if you run locally (without docker) too?
c
clay
03/22/2023, 5:27 PMSo I'm thinking the .sql file is not copied into the container that is created when the asset is materialized
In vanilla python, I can see the file
j
jamie
03/22/2023, 5:28 PMok cool that makes sense, just wanted to confirm before i loop some more folks in
c
clay
03/22/2023, 5:28 PMI would guess you could test with a txt file or any other file?
It's just a question, I suppose, of "Why can't I see/find this file that is in the same directory when materializing the asset?"
At the end of the day, it's not a huge deal, but it would really help me to declutter my dagster assets file
If it makes a difference, I'm mounting a volume as the code location. The container name for the code location is
analysisanalysis_imageanalysis_image[DockerRunLauncher] Launching run in a new container 99b2920fa7d82f5cd8751f9a571a217cce4c769db91d78cc255264428c4990fe with image analysis_image
So, I would think that the instance of
analysis_image(and therefore, the .sql file would be available)
Daniel had pointed me to this, which works: https://docs.dagster.io/_apidocs/utilities#dagster.file_relative_path
So, thanks, Daniel!
Not sure where you comment went. 🙂