Ben Latz
08/24/2022, 2:08 PMZach
08/24/2022, 8:54 PMgeoHeil
08/25/2022, 10:29 AMgeoHeil
08/25/2022, 11:08 AMdagster-cloud --help
no longer lists it as an optiongeoHeil
08/25/2022, 8:14 PMYang
08/26/2022, 5:03 PMTraceback (most recent call last):
File "/opt/homebrew/Caskroom/miniforge/base/envs/py39_env_x86/lib/python3.9/site-packages/urllib3/connectionpool.py", line 703, in urlopen
httplib_response = self._make_request(
File "/opt/homebrew/Caskroom/miniforge/base/envs/py39_env_x86/lib/python3.9/site-packages/urllib3/connectionpool.py", line 398, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/opt/homebrew/Caskroom/miniforge/base/envs/py39_env_x86/lib/python3.9/http/client.py", line 1285, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/opt/homebrew/Caskroom/miniforge/base/envs/py39_env_x86/lib/python3.9/http/client.py", line 1331, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/opt/homebrew/Caskroom/miniforge/base/envs/py39_env_x86/lib/python3.9/http/client.py", line 1280, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/opt/homebrew/Caskroom/miniforge/base/envs/py39_env_x86/lib/python3.9/http/client.py", line 1040, in _send_output
self.send(msg)
File "/opt/homebrew/Caskroom/miniforge/base/envs/py39_env_x86/lib/python3.9/http/client.py", line 980, in send
self.connect()
File "/opt/homebrew/Caskroom/miniforge/base/envs/py39_env_x86/lib/python3.9/site-packages/docker/transport/unixconn.py", line 30, in connect
sock.connect(self.unix_socket)
FileNotFoundError: [Errno 2] No such file or directory
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/homebrew/Caskroom/miniforge/base/envs/py39_env_x86/lib/python3.9/site-packages/requests/adapters.py", line 489, in send
resp = conn.urlopen(
File "/opt/homebrew/Caskroom/miniforge/base/envs/py39_env_x86/lib/python3.9/site-packages/urllib3/connectionpool.py", line 787, in urlopen
retries = retries.increment(
File "/opt/homebrew/Caskroom/miniforge/base/envs/py39_env_x86/lib/python3.9/site-packages/urllib3/util/retry.py", line 550, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/opt/homebrew/Caskroom/miniforge/base/envs/py39_env_x86/lib/python3.9/site-packages/urllib3/packages/six.py", line 769, in reraise
raise value.with_traceback(tb)
File "/opt/homebrew/Caskroom/miniforge/base/envs/py39_env_x86/lib/python3.9/site-packages/urllib3/connectionpool.py", line 703, in urlopen
httplib_response = self._make_request(
File "/opt/homebrew/Caskroom/miniforge/base/envs/py39_env_x86/lib/python3.9/site-packages/urllib3/connectionpool.py", line 398, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/opt/homebrew/Caskroom/miniforge/base/envs/py39_env_x86/lib/python3.9/http/client.py", line 1285, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/opt/homebrew/Caskroom/miniforge/base/envs/py39_env_x86/lib/python3.9/http/client.py", line 1331, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/opt/homebrew/Caskroom/miniforge/base/envs/py39_env_x86/lib/python3.9/http/client.py", line 1280, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/opt/homebrew/Caskroom/miniforge/base/envs/py39_env_x86/lib/python3.9/http/client.py", line 1040, in _send_output
self.send(msg)
File "/opt/homebrew/Caskroom/miniforge/base/envs/py39_env_x86/lib/python3.9/http/client.py", line 980, in send
self.connect()
File "/opt/homebrew/Caskroom/miniforge/base/envs/py39_env_x86/lib/python3.9/site-packages/docker/transport/unixconn.py", line 30, in connect
sock.connect(self.unix_socket)
urllib3.exceptions.ProtocolError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))
Joshua Smart-Olufemi
08/26/2022, 5:18 PMStephen Bailey
08/29/2022, 2:23 PMKirk Stennett
08/29/2022, 7:47 PMFrank Lu
08/30/2022, 8:23 PMcloud_workspace.yaml
and locations.yaml
?? We currently use locations.yaml
for main deployment and cloud_workspace.yaml
for branch deployment. Is there anyway we can just use the same file?? It seems pretty redundant. What’s the best practice??Qwame
08/31/2022, 4:31 PMreturn
in my ops. I know in the in-built io manager, these are pickled using the file system. How does it work in dagster cloud? What's the default IO manager there and what gets stored in the postgres database? Other than event log and job Metadata, is it possible to have info like output of ops go there? Need answers to these questions to help our adoption. Thanks!Shawn O’Hare
08/31/2022, 6:48 PMprod
) deploy? It's not clear to me what the developer workflow is now supposed to look like given the viewer
role only exists now as part of the Enterprise plan. Our basic intuition here is
1. Maintain two separate Standard accounts, corresponding to a dev
and prod
environment and simply not grant much access to prod
. The obvious flaw here is then most people would be unable to actually view or debug issues that happen in prod, as they simply wouldn't have access.
2. Encourage more purely local development, e.g., viewing job changes in a local dagit instance. We don't have enough experience to know if this is practical, i.e., if the runtimes are sufficiently similar.Frank Lu
09/07/2022, 6:14 PMZach P
09/08/2022, 3:06 AMgeoHeil
09/08/2022, 11:00 AMZach
09/08/2022, 4:30 PMDavid Fernández Calle
09/09/2022, 9:41 AMException: No pod names in job after it started
Stack Trace:
File "/usr/python/__pypackages__/3.9/lib/dagster/_core/execution/plan/utils.py", line 47, in solid_execution_error_boundary
yield
File "/usr/python/__pypackages__/3.9/lib/dagster/_utils/__init__.py", line 430, in iterate_with_context
next_output = next(iterator)
File "/usr/python/__pypackages__/3.9/lib/dagster/_core/execution/plan/compute_generator.py", line 73, in _coerce_solid_compute_fn_to_iterator
result = fn(context, **kwargs) if context_arg_provided else fn(**kwargs)
File "/usr/python/__pypackages__/3.9/lib/dagster/_annotations.py", line 108, in inner
return target(*args, **kwargs)
File "/usr/python/__pypackages__/3.9/lib/dagster_k8s/ops/k8s_job_op.py", line 220, in k8s_job_op
raise Exception("No pod names in job after it started")
Code:
from dagster import job
from dagster_k8s import k8s_job_op
my_k8s_op_1 = k8s_job_op.configured(
{
"image": "myrepo/test:test",
"command": ["run_job.sh"],
"args": [],
"env_vars": ["ENVIRONMENT=stage"],
"resources": {
"requests": {
"memory": "512Mi",
"cpu": "500m"
},
"limits": {
"memory": "2048M",
"cpu": "1000m"
}
},
"pod_spec_config": {"service_account_name": "my-custom-access"},
"service_account_name": "my-dagster-cloud-agent",
},
name="my_test_job",
)
@job()
def my_k8s_job_1():
my_k8s_op_1()
Yevhen Samoilenko
09/09/2022, 12:56 PMFrank Lu
09/12/2022, 6:33 PMSterling Paramore
09/12/2022, 6:58 PMZach
09/12/2022, 9:05 PMZach P
09/14/2022, 8:25 PMJason
09/15/2022, 9:23 PMbranch deployment
I have branch that's "stuck" in loading status. I've deleted the branch deployment and then redeployed but it goes back into suck. Also, I've tested a new branch (based on main) and that loads fine. One other note, when the branch originally got stuck the gh action shows a time out error so I think that's messed things up going forward.
Anyone knows how to unstick it? I sure creating a completely new branch with the same code would work, I just don't want to do it 🙂Ben Latz
09/16/2022, 9:07 PMPrratek Ramchandani
09/19/2022, 3:46 PMException: Timed out waiting for server eltjobs-7542980173014a5e58097494d2e0622716468f34-370ded.dagster-cloud:4000
when loading assets from our dbt project. i think the fix would be to set this local_startup_timeout
param:
1. is that correct?
2. where do i set that? we somehow ended up with a locations.yaml
, dagster_cloud.yaml
, and a dagster.yaml
. i'm also not sure how to know which of these is being used by our cloud deployment - the dagster.yaml
for example isn't referenced anywhere in our codebasegeoHeil
09/20/2022, 10:32 AMLevan
09/20/2022, 9:17 PMdagster._check.CheckError: Failure condition: Tried to access partition key for input 'source' of step 'TEST', but the step input has a partition range: '2022-09-04' to '2022-09-11'.
Any ideas what could go wrong?geoHeil
09/21/2022, 9:50 AMXuan Mai Ho
09/21/2022, 12:01 PMBuild and deploy to Dagster Cloud serverless
step:
#15 28.12 g++ -std=c++11 -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DZMQ_HAVE_CURVE=1 -DZMQ_USE_TWEETNACL=1 -DZMQ_USE_EPOLL=1 -DZMQ_IOTHREADS_USE_EPOLL=1 -DZMQ_POLL_BASED_ON_POLL=1 -Ibundled/zeromq/include -Ibundled -I/usr/local/include/python3.8 -c buildutils/initlibzmq.cpp -o build/temp.linux-x86_64-cpython-38/buildutils/initlibzmq.o
#15 28.12 error: command 'g++' failed: No such file or directory
#15 28.12 [end of output]
#15 28.12
#15 28.12 note: This error originates from a subprocess, and is likely not a problem with pip.
#15 28.12 ERROR: Failed building wheel for pyzmq
#15 28.12 Successfully built wkflw promise
#15 28.12 Failed to build pyzmq
#15 28.12 ERROR: Could not build wheels for pyzmq, which is required to install pyproject.toml-based projects
#15 ERROR: process "/bin/sh -c if [ -f \"requirements.txt\" ]; then pip install -r requirements.txt; fi" did not complete successfully: exit code: 1
------
> [ 9/11] RUN if [ -f "requirements.txt" ]; then pip install -r requirements.txt; fi:
#15 28.12 creating build/temp.linux-x86_64-cpython-38/bundled/zeromq/src
#15 28.12 g++ -std=c++11 -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DZMQ_HAVE_CURVE=1 -DZMQ_USE_TWEETNACL=1 -DZMQ_USE_EPOLL=1 -DZMQ_IOTHREADS_USE_EPOLL=1 -DZMQ_POLL_BASED_ON_POLL=1 -Ibundled/zeromq/include -Ibundled -I/usr/local/include/python3.8 -c buildutils/initlibzmq.cpp -o build/temp.linux-x86_64-cpython-38/buildutils/initlibzmq.o
#15 28.12 error: command 'g++' failed: No such file or directory
#15 28.12 [end of output]
#15 28.12
#15 28.12 note: This error originates from a subprocess, and is likely not a problem with pip.
#15 28.12 ERROR: Failed building wheel for pyzmq
#15 28.12 Successfully built wkflw promise
#15 28.12 Failed to build pyzmq
#15 28.12 ERROR: Could not build wheels for pyzmq, which is required to install pyproject.toml-based projects
------
Dockerfile:24
--------------------
23 | # Install the rest of dependencies in case there is a requirements.txt
24 | >>> RUN if [ -f "requirements.txt" ]; then \
25 | >>> pip install -r requirements.txt; \
26 | >>> fi
27 |
--------------------
ERROR: failed to solve: process "/bin/sh -c if [ -f \"requirements.txt\" ]; then pip install -r requirements.txt; fi" did not complete successfully: exit code: 1
Error: buildx failed with: ERROR: failed to solve: process "/bin/sh -c if [ -f \"requirements.txt\" ]; then pip install -r requirements.txt; fi" did not complete successfully: exit code: 1
Builds were 👌 yesterdayLevan
09/23/2022, 6:51 PM