Crashes instead of displaying "Could not find any answers for this topic" when it wasn't able to find any relavant information.

[Speedway1]

For topics that return no results, it crashed rather than put out a meaningful error.

From the logs:

Of course, the information I've gleaned from your data is quite limited, so additional subtopics or a rephrasing of these may be more suitable depending on the depth and focus of the report.
📋Subtopics: []
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/uvicorn/protocols/websockets/websockets_impl.py", line 244, in run_asgi
result = await self.app(self.scope, self.asgi_receive, self.asgi_send) # type: ignore[func-returns-value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in call
return await self.app(scope, receive, send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in call
await super().call(scope, receive, send)
File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/applications.py", line 123, in call
await self.middleware_stack(scope, receive, send)
File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/middleware/errors.py", line 151, in call
await self.app(scope, receive, send)
File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 65, in call
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
raise exc
File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
await app(scope, receive, sender)
File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/routing.py", line 756, in call
await self.middleware_stack(scope, receive, send)
File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/routing.py", line 776, in app
await route.handle(scope, receive, send)
File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/routing.py", line 373, in handle
await self.app(scope, receive, send)
File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/routing.py", line 96, in app
await wrap_app_handling_exceptions(app, session)(scope, receive, send)
File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
raise exc
File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
await app(scope, receive, sender)
File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/routing.py", line 94, in app
await func(session)
File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/fastapi/routing.py", line 348, in app
await dependant.call(**values)
File "/data/misc/gpt-researcher/backend/server.py", line 53, in websocket_endpoint
report = await manager.start_streaming(task, report_type, report_source, websocket)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/misc/gpt-researcher/backend/websocket_manager.py", line 57, in start_streaming
report = await run_agent(task, report_type, report_source, websocket)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/misc/gpt-researcher/backend/websocket_manager.py", line 75, in run_agent
report = await researcher.run()
^^^^^^^^^^^^^^^^^^^^^^
File "/data/misc/gpt-researcher/backend/report_type/detailed_report/detailed_report.py", line 36, in run
subtopics = await self._get_all_subtopics()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/misc/gpt-researcher/backend/report_type/detailed_report/detailed_report.py", line 62, in _get_all_subtopics
return subtopics.dict()["subtopics"]
^^^^^^^^^^^^^^
AttributeError: 'list' object has no attribute 'dict'
INFO: connection closed

[Speedway1]

Actually there may be something deeper here. It may be that there's an assumption (based on using powerful models like GPT-4o) that the text analysis model can write good JSON which isn't necessarily true. For example I have just used The Pitbull for the text analysis (very good, efficient and powerful), but it produced bad JSON (see below).

We probably should have a specialist LLM for text to JSON encoding (ie "Smart", "Fast" and "Coding" LLM). For example codestral would make short work of the encoding task.

Here's the Huggingface to the LLM: https://huggingface.co/fblgit/UNA-ThePitbull-21.4B-v2

This is the research question: "What is the latest news from the war in Ukraine taking into account all factors, including politics in 3rd party entities, affecting it?"

Here'e the output from the failure:

Finalized research step.
💸 Total Research Costs: $0.006954480000000001
🤔 Generating subtopics...

🤖 Calling una-thepitbull-21.4b-v2_q5_km:latest...

Exception in parsing subtopics : Invalid json output: :Sure, here's the report on the recent developments in Ukraine and the key factors affecting it based on your provided research data. I've organized the subtopics in a logical order for easy readability:

{
"subtopics": [
{
"task": "Latest6 n news from the war in Ukraine"
},
{
"task": "Factors including politics in 3rd party entities affecting the war"
},
{
"task": "Peace summit and its outcomes"
}
]
}
📋Subtopics: []
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/uvicorn/protocols/websockets/websockets_impl.py", line 244, in run_asgi
result = await self.app(self.scope, self.asgi_receive, self.asgi_send) # type: ignore[func-returns-value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in call
return await self.app(scope, receive, send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in call
await super().call(scope, receive, send)
File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/applications.py", line 123, in call
await self.middleware_stack(scope, receive, send)
File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/middleware/errors.py", line 151, in call
await self.app(scope, receive, send)
File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 65, in call
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
raise exc
File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
await app(scope, receive, sender)
File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/routing.py", line 756, in call
await self.middleware_stack(scope, receive, send)
File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/routing.py", line 776, in app
await route.handle(scope, receive, send)
File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/routing.py", line 373, in handle
await self.app(scope, receive, send)
File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/routing.py", line 96, in app
await wrap_app_handling_exceptions(app, session)(scope, receive, send)
File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
raise exc
File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
await app(scope, receive, sender)
File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/starlette/routing.py", line 94, in app
await func(session)
File "/data/misc/gpt-researcher_env/lib/python3.11/site-packages/fastapi/routing.py", line 348, in app
await dependant.call(**values)
File "/data/misc/gpt-researcher/backend/server.py", line 53, in websocket_endpoint
report = await manager.start_streaming(task, report_type, report_source, websocket)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/misc/gpt-researcher/backend/websocket_manager.py", line 57, in start_streaming
report = await run_agent(task, report_type, report_source, websocket)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/misc/gpt-researcher/backend/websocket_manager.py", line 75, in run_agent
report = await researcher.run()
^^^^^^^^^^^^^^^^^^^^^^
File "/data/misc/gpt-researcher/backend/report_type/detailed_report/detailed_report.py", line 36, in run
subtopics = await self._get_all_subtopics()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/misc/gpt-researcher/backend/report_type/detailed_report/detailed_report.py", line 62, in _get_all_subtopics
return subtopics.dict()["subtopics"]
^^^^^^^^^^^^^^
AttributeError: 'list' object has no attribute 'dict'

[assafelovic]

Hey @Speedway1 this is definitely an issue with output inconsistency in LLMs. GPT Researcher is fully tested using GPT but would love the communities support in improving it for other LLMs as well. Which LLM are you using?

[Speedway1]

Yes, agreed, I was doing further tests and came to the same conclusion.

I was using "pitbull" https://huggingface.co/fblgit/UNA-ThePitbull-21.4B-v2

But then I swapped over to command-r which works much better but even with command-r this issue comes up occasionally.

Will try with some other models and report back here.

[Speedway1]

Hey @Speedway1 this is definitely an issue with output inconsistency in LLMs. GPT Researcher is fully tested using GPT but would love the communities support in improving it for other LLMs as well. Which LLM are you using?

I have done a fix that handles the string error properly. Not sure how to do the pull requests to submit in git so will give the code extract here. The code is in agent.py (gpt_researcher/master/agent.py).

The fix is to initialse "sub_queries" as an array:

         # Initialise sub_queries as an empty list
         sub_queries = []

        # Generate Sub-Queries including original query
        sub_queries.extend(
               await get_sub_queries(query=query, agent_role_prompt=self.role,
                                            cfg=self.cfg, parent_query=self.parent_query,
                                            report_type=self.report_type, cost_callback=self.add_costs
               )
        )

Note that we then use "sub_queries.extend" instead of "sub_queries =" to add the rows.

[assafelovic]

Can you explain the rational for this? Why not just catch errors on get_sub_queries and return an empty list if needed?

[Speedway1]

It would need to catch the exceptions to do that but yes, that's a great solution.

This way avoids the exception in the first place. But of course other exceptions could occur.