JP05/14/2023, 11:26 PM
Taylor Murphy05/15/2023, 2:33 PM
JP05/15/2023, 4:53 PM
function. I do not know, if this is by design or not, but it is not installed with
. Maybe the blog assumes this module to be available globally and in $PATH? One will be be presented with following error in terminal's stdout
No jobs will be visible in http://127.0.0.1:3000/locations/dag_melt_project/jobs To fix this, one would
..dag-melt/dagster/dag-venv/lib/python3.10/site-packages/dagster/_core/workspace/context.py:589: UserWarning: Error loading repository location dag_melt_project:FileNotFoundError: [Errno 2] No such file or directory: 'meltano'
source ~/dag-melt/dagster/dag-venv/bin/activate && pip install meltano && deactivate
2. One would not be able to run the job that now is available with fix in #1 because of following error
I do not know, if this is a breaking change in Meltano (because the tutorial tells the user to install the latest version or something else).. anyhow, I have not been able to work around this yet. Honestly would not think anyone to continue with a tutorial at this point.
dagster._check.CheckError: Value in dict mismatches expected type for key MELTANO_CLI_LOG_CONFIG. Expected value of type <class 'str'>. Got value /Users/janispuris/dag-melt/dagster/dag-venv/lib/python3.10/site-packages/dagster_meltano/logging.yaml of type <class 'pathlib.PosixPath'>.
Taylor Murphy05/15/2023, 5:49 PM
JP05/15/2023, 5:50 PM
change line 34 from
"MELTANO_CLI_LOG_CONFIG": Path(__file__).parent / "logging.yaml",
The meltano job runs without issues..
"MELTANO_CLI_LOG_CONFIG": str(Path(__file__).parent / "logging.yaml"),
for keys and values, however
does not return a
Path(__file__).parent / "logging.yaml"
, but a
object, hence the error.. I've checked this on Python 3.11, 3.10 and 3.9 🤷 From what I can tell from git blame, this property was introduced back in Feb 18, 2023 as part of New dagster_meltano package #15 and therefore I'm clueless as to how this was working at the time of writing the blog post 😞 Would you suggest me to open github issue on this?
where it was allowing non-string entries to exist in the dict, which was "fine" until a change in
package in dagster/python_modules/dagster/dagster/_check/__init__.py at master · dagster-io/dagster · GitHub I suspect previously the dict values were not validated, but now they are, hence the error. Specifically this function file:
It is little too late to go further down the rabbit hole on this. I'll open a PR on
sometime tomorrow on 🤷
Taylor Murphy05/15/2023, 6:58 PM
Fraser Marlow05/15/2023, 8:15 PM
Fraser Marlow05/18/2023, 12:53 AM
and everything ran smoothly. Thanks @JP for your help on identifying and troubleshooting this one!
dagster, version 1.2.6 meltano, version 2.16.1