The renew grace period is too small to renew locks for a batch of Service Bus Queue messages

[tj---]

• Package Name: azure-servicebus
• Package Version: 7.4.0 / 7.12.2
• Operating System: macOS
• Python Version: 3.8.13

Describe the bug
If we take a batch of messages from the Service Bus Queue, it takes a long time to renew the locks (since there is a sleep of 0.5s between each update). The renewal happens very close to the end of expiry (10s) - so it is a very critical window.
The only solution for a large batch (say of size 400) is to have a ridiculously large number of threads that can renew the locks together.

To Reproduce
Steps to reproduce the behavior:

1. Create a queue (with a lock lifetime of 4m) and push some messages in it.
2. Put the Queue name and Connection string in this sample code and run it. https://gist.github.com/tj---/cf9cec837febea4f9903d987019ba651 [Alternate: with hack]
3. Observe the error logs start showing up after ~4 minutes (only a few locks will be renewed, renewal of the rest will fail)

Expected behavior

1. The renewal of locks should be permitted via a configurable renew_period parameter and soon (eg: I would start renewing the locks 1-2 minutes in advance)
2. It should be possible to increase the lock lifetime at the Queue itself (which is currently a max of 5m)

Screenshots
If applicable, add screenshots to help explain your problem.

1. Logs (with issue):
   redacted_issue_log.txt

2. Logs (With hack in place - increased the _renew_period to 90s and max_workers = 2):
   redacted_ok_log.txt

Additional context
Original reference: #35717 (comment)

[github-actions]

Thank you for your feedback. Tagging and routing to the team member best able to assist.

[kristapratico]

@tj--- thanks for the detailed issue, the team will investigate and respond as soon as possible.