Allow consumers to choose BoundedChannelFullMode

[Mpdreamz]

This PR exposes BoundedChannelFullMode

This allows consumers to decide certain trade offs themselves.

Here are four example all pushing a million documents to the channel in a tight loop (extreme case of overproducing).

BoundedChannelFullMode.Wait is still the default but in these cases where we extremely overproduce it can allocate more then expected see: https://github.com/dotnet/runtime/blob/fadd8313653f71abd0068c8bf914be88edb2c8d3/src/libraries/System.Threading.Channels/src/System/Threading/Channels/BoundedChannel.cs#L609

A downside of BoundedChannelFullMode.DropWrite is that TryWrite on a BoundedChannel will always return true: https://github.com/dotnet/runtime/blob/fadd8313653f71abd0068c8bf914be88edb2c8d3/src/libraries/System.Threading.Channels/src/System/Threading/Channels/BoundedChannel.cs#L428 and there won't be any notifications of items that failed to write.

On a dataset of a million documents using the following buffer options:

default scenario

Inbound	Outbound	Concurrency	FullMode	Indexed All
100_000 (default)	1_000 (default)	default (>1)	WAIT	✅

default scenario with DropWrite

Inbound	Outbound	Concurrency	FullMode	Indexed All
100_000 (default)	1_000 (default)	default (>1)	DROPWRITE	💥 (only 710_001 docs made it)

constraint scenario

Inbound	Outbound	Concurrency	FullMode	Indexed All
1_000	200	default (>1)	WAIT	✅

While successful it takes quite a bit longer to index all million items.

constraint scenario with DropWrite

Inbound	Outbound	Concurrency	FullMode	Indexed All
1_000	200	default (>1)	DROPWRITE	💥 (only 264_601 docs made it)

[Mpdreamz]

The defaults scenarios still have 'high' gen2 allocations but they are all owned by the TlsOverPerCoreLockedStacksArrayPool backing the System.Channels BoundedBuffer.

Unsure if there is anything we can do to control it.

See e.g this proposal to .NET dotnet/runtime#53895

[stevejgordon]

LGTM