Not working with the OpenAI Assistants API

[melvinmt]

Report

The Pezzo Proxy does not work with Assistants API calls. It returns a 404 error because the request URL is malformed (e.g., the v1threads part in https://proxy.pezzo.ai/openai/v1threads/thread_xxxxx/runs HTTP/1.1). Even when correcting the request URL, it does not properly pass on the OpenAI-Beta assistants=v2 header. This could either be an issue with the OpenAI SDK itself or with Pezzo. We would expect Pezzo to transparently pass on the request to OpenAI even if it can’t deal with the request itself.

Expected behavior

We expected Pezzo to properly forward the request to OpenAI without altering the URL structure or headers, allowing the Assistants API to function correctly.

Steps to reproduce the problem

Setup the Environment:
• Ensure you have the Pezzo Proxy configured to route OpenAI API requests.
• Ensure you have access to the OpenAI Assistants API.
2. Make an API Call via Pezzo:
• Use the OpenAI SDK to make a call to the Assistants API.
• Include the OpenAI-Beta assistants=v2 header in your request.
3. Observe the Error:
• The request should return a 404 error due to the malformed URL (v1threads part in the URL).
4. Correct the URL and Resend the Request:
• Manually correct the request URL and resend it.
• Observe that even with the corrected URL, the OpenAI-Beta assistants=v2 header is not properly passed on.

Logs (if applicable)

POST /openai/v1/threads/thread_xxxx/runs HTTP/1.1
Host: proxy.pezzo.ai
Accept-Encoding: gzip, deflate
Connection: keep-alive
Accept: application/json
Content-Type: application/json
User-Agent: _ModuleClient/Python 1.30.5
X-Stainless-Lang: python
X-Stainless-Package-Version: 1.30.5
X-Stainless-OS: MacOS
X-Stainless-Arch: arm64
X-Stainless-Runtime: CPython
X-Stainless-Runtime-Version: 3.11.9
Authorization: Bearer sk-proj-xxxxx
X-Pezzo-Api-Key: pez_xxxx
X-Pezzo-Project-Id: clwkxxxx
X-Pezzo-Environment: local
X-Pezzo-Cache-Enabled: true
OpenAI-Beta: assistants=v2
X-Stainless-Async: false
X-Stainless-Stream-Helper: threads.runs.create_and_stream
X-Stainless-Custom-Event-Handler: true
Content-Length: 169

{"assistant_id": "asst_xxxxxxxxx", "temperature": 0, "tool_choice": {"type": "code_interpreter"}, "stream": true, "tools": [{"type": "code_interpreter"}]}

HTTP/1.1 400 Bad Request
Date: Tue, 18 Jun 2024 19:31:37 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 230
Connection: keep-alive
X-Powered-By: Express

{"error":{"message":"You must provide the 'OpenAI-Beta' header to access the Assistants API. Please try again by setting the header 'OpenAI-Beta: assistants=v2'.","type":"invalid_request_error","param":null,"code":"invalid_beta"}}

Pezzo version

No response

How do you use Pezzo?

Local Development Setup

[arielweinberger]

@melvinmt are you willing to make a contribution? should be fairly straightforward in apps/proxy!

[ranst91]

@melvinmt do you know what is the URL that is originally being sent to pezzo?
looking at the code that does this, I'd expect it to add a / if anything. Trying to understand home come you ended up with none.