Cached model and parallel requests

[AlexBlack772]

I'm working on a server that can handle multiple concurrent requests.

Initializing the Synthesizer class takes a long time, so I initialize the class once and then reuse it for each request.

This works fine if the requests are made in series. But when I make several parallel requests, I get errors

"stack expects each tensor to be equal size, but got [1, 69] at entry 0 and [1, 71] at entry 2"

Is there any way to reduce time of initialization of Synthesizer class and still avoid errors during parallel requests?

[KyuubiYoru]

Maybe a Synthesizer pool would work, if you have Hardware the can handle multiple Synthesizer.
How many request do you need to handle?

[AlexBlack772]

About 2-4 RPS maximum.

[KyuubiYoru]

A Synthesizer can only handle one request at a time. For multiple parallel requests, you need to create multiple Synthesizer instances. Instead of creating a Synthesizer for each request, you can create a Synthesizer pool and reuse the instances.

[1973sir]

Hello,have you solved this problem?Would be nice if you share the code,I'm facing the same problem.

[the-homeless-god]

+1