Exception requests.exceptions.JSONDecodeError is raised in APIError constructor

[darttie]

Describe the bug
My code called sh.sheet1.batch_get([gsrange], Dimension.cols) which sends request to google sheets. It looks like some error response was returned, but the following exception was raised when APIError was trying to be constructed:

Traceback (most recent call last):
  File "helper.py", line 37, in get_capabilities_list
    capabilities = sh.sheet1.batch_get([gsrange], Dimension.cols)
  File ".local/lib/python3.10/site-packages/gspread/worksheet.py", line 1096, in batch_get
    response = self.client.values_batch_get(
  File ".local/lib/python3.10/site-packages/gspread/http_client.py", line 255, in values_batch_get
    r = self.request("get", url, params=params)
  File ".local/lib/python3.10/site-packages/gspread/http_client.py", line 128, in request
    raise APIError(response)
  File ".local/lib/python3.10/site-packages/gspread/exceptions.py", line 45, in __init__
    self.error: Mapping[str, Any] = response.json()["error"]
  File ".local/lib/python3.10/site-packages/requests/models.py", line 978, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

To Reproduce
Unfortunately the bug prevents me from knowing what error actually happened. Usually this step succeeds, but once it has failed.

1. call sh.sheet1.batch_get(['G1:S1'], Dimension.cols)

Expected behavior
Correct APIError is returned

Environment info:

• Ubuntu
• Python 3.10.12
• gspread Version: 6.1.2

[dan-ladd]

I noticed this intermittently starting on 2024-08-07 around 15:00 UTC. I was getting it on fetch_sheet_metadata()

venv/lib/python3.9/site-packages/gspread/http_client.py", line 303, in fetch_sheet_metadata
->
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

[lavigne958]

Hi, thank you for raising this issue, I believe this is a problem with Google servers, we receive a valid JSON error message on every error for as many years as I can remember on this project.

possibly it could be a network error too where you don't receive the error response (the connection closes early, etc).

In order to investigate your issue and help further similar situations and possibly provide a solution to #1491 we can change the way we handle errors to:

1. try to extract HTTP payload as JSON
2. if it succeed:
3. we extract the informations as we already do
4. if it fails:
   1. we build an empty Response object
   2. we get the pure, raw text payload (which could be empty, we don't know)
   3. be set the response and the internal Dict object with response_code=-1 and the error message with whatever text we extracted

This way we don't fail because of weird/empty/malformed JSON response
We still can print/return some error in the same format we do all over the project
We can solve the issue with the mentioned PR above that returns some HTML error message.

@alifeee how do you feel about this ?