Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Connection errors with Chroma template #466

Closed
mgburns opened this issue Apr 5, 2024 · 3 comments
Closed

Connection errors with Chroma template #466

mgburns opened this issue Apr 5, 2024 · 3 comments

Comments

@mgburns
Copy link

mgburns commented Apr 5, 2024

I'm attempting to deploy the Chroma template (introduced in #434) and connect it to a Flask app.

Connection code:

import chromadb

db = chromadb.HttpClient(host=os.getenv('CHROMA_DB_HOST', 'chroma.railway.internal'))
Flask service stack trace
[2024-04-05 13:42:46 +0000] [11] [INFO] Worker exiting (pid: 11)

[2024-04-05 13:42:46 +0000] [7] [INFO] Shutting down: Master

[2024-04-05 13:42:46 +0000] [7] [INFO] Reason: Worker failed to boot.

[2024-04-05 13:43:13 +0000] [7] [INFO] Starting gunicorn 20.0.4

[2024-04-05 13:43:13 +0000] [7] [INFO] Listening at: http://0.0.0.0:6731/ (7)

[2024-04-05 13:43:13 +0000] [7] [INFO] Using worker: sync

[2024-04-05 13:43:13 +0000] [11] [INFO] Booting worker with pid: 11

[2024-04-05 13:43:19 +0000] [11] [ERROR] Exception in worker process

Traceback (most recent call last):

File "/opt/venv/lib/python3.8/site-packages/urllib3/connection.py", line 198, in _new_conn

sock = connection.create_connection(

File "/opt/venv/lib/python3.8/site-packages/urllib3/util/connection.py", line 60, in create_connection

for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):

File "/root/.nix-profile/lib/python3.8/socket.py", line 918, in getaddrinfo

for res in _socket.getaddrinfo(host, port, family, type, proto, flags):

socket.gaierror: [Errno -2] Name or service not known

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

File "/opt/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 793, in urlopen

response = self._make_request(

File "/opt/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 496, in _make_request

conn.request(

File "/opt/venv/lib/python3.8/site-packages/urllib3/connection.py", line 400, in request

self.endheaders()

File "/root/.nix-profile/lib/python3.8/http/client.py", line 1251, in endheaders

self._send_output(message_body, encode_chunked=encode_chunked)

File "/root/.nix-profile/lib/python3.8/http/client.py", line 1011, in _send_output

self.send(msg)

File "/root/.nix-profile/lib/python3.8/http/client.py", line 951, in send

self.connect()

File "/opt/venv/lib/python3.8/site-packages/urllib3/connection.py", line 238, in connect

self.sock = self._new_conn()

File "/opt/venv/lib/python3.8/site-packages/urllib3/connection.py", line 205, in _new_conn

raise NameResolutionError(self.host, self, e) from e

urllib3.exceptions.NameResolutionError: <urllib3.connection.HTTPConnection object at 0x7fa4a5e09190>: Failed to resolve 'chroma.railway.internal' ([Errno -2] Name or service not known)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

File "/opt/venv/lib/python3.8/site-packages/requests/adapters.py", line 486, in send

resp = conn.urlopen(

File "/opt/venv/lib/python3.8/site-packages/urllib3/connectionpool.py", line 847, in urlopen

retries = retries.increment(

File "/opt/venv/lib/python3.8/site-packages/urllib3/util/retry.py", line 515, in increment

raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]

urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='chroma.railway.internal', port=8000): Max retries exceeded with url: /api/v1/tenants/default_tenant (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x7fa4a5e09190>: Failed to resolve 'chroma.railway.internal' ([Errno -2] Name or service not known)"))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "/opt/venv/lib/python3.8/site-packages/chromadb/api/client.py", line 438, in _validate_tenant_database

self._admin_client.get_tenant(name=tenant)

File "/opt/venv/lib/python3.8/site-packages/chromadb/api/client.py", line 486, in get_tenant

return self._server.get_tenant(name=name)

File "/opt/venv/lib/python3.8/site-packages/chromadb/telemetry/opentelemetry/__init__.py", line 127, in wrapper

return f(*args, **kwargs)

File "/opt/venv/lib/python3.8/site-packages/chromadb/api/fastapi.py", line 197, in get_tenant

resp = self._session.get(

File "/opt/venv/lib/python3.8/site-packages/requests/sessions.py", line 602, in get

return self.request("GET", url, **kwargs)

File "/opt/venv/lib/python3.8/site-packages/requests/sessions.py", line 589, in request

resp = self.send(prep, **send_kwargs)

File "/opt/venv/lib/python3.8/site-packages/requests/sessions.py", line 703, in send

r = adapter.send(request, **kwargs)

File "/opt/venv/lib/python3.8/site-packages/requests/adapters.py", line 519, in send

raise ConnectionError(e, request=request)

requests.exceptions.ConnectionError: HTTPConnectionPool(host='chroma.railway.internal', port=8000): Max retries exceeded with url: /api/v1/tenants/default_tenant (Caused by NameResolutionError("<urllib3.connection.HTTPConnection object at 0x7fa4a5e09190>: Failed to resolve 'chroma.railway.internal' ([Errno -2] Name or service not known)"))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "/opt/venv/lib/python3.8/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker

worker.init_process()

File "/opt/venv/lib/python3.8/site-packages/gunicorn/workers/base.py", line 119, in init_process

self.load_wsgi()

File "/opt/venv/lib/python3.8/site-packages/gunicorn/workers/base.py", line 144, in load_wsgi

self.wsgi = self.app.wsgi()

File "/opt/venv/lib/python3.8/site-packages/gunicorn/app/base.py", line 67, in wsgi

self.callable = self.load()

File "/opt/venv/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 49, in load

return self.load_wsgiapp()

File "/opt/venv/lib/python3.8/site-packages/gunicorn/app/wsgiapp.py", line 39, in load_wsgiapp

return util.import_app(self.app_uri)

File "/opt/venv/lib/python3.8/site-packages/gunicorn/util.py", line 358, in import_app

mod = importlib.import_module(module)

File "/root/.nix-profile/lib/python3.8/importlib/__init__.py", line 127, in import_module

return _bootstrap._gcd_import(name[level:], package, level)

File "<frozen importlib._bootstrap>", line 1014, in _gcd_import

File "<frozen importlib._bootstrap>", line 991, in _find_and_load

File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked

File "<frozen importlib._bootstrap>", line 671, in _load_unlocked

File "<frozen importlib._bootstrap_external>", line 843, in exec_module

File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed

I set up a simple repro here:

Help!

Copy link

github-actions bot commented Apr 5, 2024

Hello, @mgburns! Thanks for your submission.

Our team will respond soon. If you need more immediate help, try our Forum or our Discord. Thanks!

@brody192
Copy link

brody192 commented Apr 5, 2024

Hey, Gitub issues aren't to be used for support, please open a help thread in either of the Forum or Discord links the bot has sent above.

@mgburns
Copy link
Author

mgburns commented Apr 5, 2024

@brody192 Apologies -- my assumption was that this is either a bug with the template or a lack of documentation, which I had thought was appropriate here:

This repository handles GitHub focused discussions about Railway templates. We handle Template bounties, feature requests, and bug reports here.

I'll move the discussion to the forum and report back if it is in fact a bug.

@mgburns mgburns closed this as completed Apr 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants