[Bug]: BuildManager fails to dispose of cache plugin after exception thrown during BeginBuild

[AArnott]

Issue Description

BuildManager fails to dispose of the plugin after the exception described in #10615, leading to all future build attempts that use the plugin to fail because the semaphore hasn't been released.

Steps to Reproduce

See #10615 for repro.

Expected Behavior

Subsequent builds fail with this message appearing only once:

Another instance of MSBuildCache is already running in this build. This is typically due to a misconfiguration of the plugin settings, in particular different plugin settings across projects.

Actual Behavior

Subsequent builds fail with an error message appearing twice:

Another instance of MSBuildCache is already running in this build. This is typically due to a misconfiguration of the plugin settings, in particular different plugin settings across projects.
Another instance of MSBuildCache is already running in this build. This is typically due to a misconfiguration of the plugin settings, in particular different plugin settings across projects.

Analysis

The cache plugin is created on this callstack:

>	Microsoft.MSBuildCache.Local.dll!Microsoft.MSBuildCache.MSBuildCachePluginBase<Microsoft.MSBuildCache.PluginSettings>.TryAcquireLock(Microsoft.MSBuildCache.PluginSettings settings, Microsoft.Build.Experimental.ProjectCache.PluginLoggerBase logger) Line 853	C#
 	Microsoft.MSBuildCache.Local.dll!Microsoft.MSBuildCache.MSBuildCachePluginBase<Microsoft.MSBuildCache.PluginSettings>.BeginBuildInnerAsync(Microsoft.Build.Experimental.ProjectCache.CacheContext context, Microsoft.Build.Experimental.ProjectCache.PluginLoggerBase logger, System.Threading.CancellationToken cancellationToken) Line 229	C#
 	Microsoft.MSBuildCache.Local.dll!Microsoft.MSBuildCache.MSBuildCachePluginBase<Microsoft.MSBuildCache.PluginSettings>.TimeAndLogAsync.AnonymousMethod__0() Line 1143	C#
 	Microsoft.MSBuildCache.Local.dll!Microsoft.MSBuildCache.MSBuildCachePluginBase<Microsoft.MSBuildCache.PluginSettings>.TimeAndLogAsync<int>(Microsoft.Build.Experimental.ProjectCache.PluginLoggerBase logger, System.Func<System.Threading.Tasks.Task<int>> innerAsync, string context, string memberName) Line 1111	C#
 	Microsoft.MSBuildCache.Local.dll!Microsoft.MSBuildCache.MSBuildCachePluginBase<Microsoft.MSBuildCache.PluginSettings>.BeginBuildAsync(Microsoft.Build.Experimental.ProjectCache.CacheContext context, Microsoft.Build.Experimental.ProjectCache.PluginLoggerBase logger, System.Threading.CancellationToken cancellationToken) Line 215	C#
 	Microsoft.Build.dll!Microsoft.Build.Experimental.ProjectCache.ProjectCacheService.CreateAndInitializePluginAsync(Microsoft.Build.Experimental.ProjectCache.ProjectCacheDescriptor projectCacheDescriptor, Microsoft.Build.Graph.ProjectGraph projectGraph, Microsoft.Build.BackEnd.BuildRequestConfiguration buildRequestConfiguration, System.Collections.Generic.ICollection<string> requestedTargets, System.Threading.CancellationToken cancellationToken) Line 258	C#
 	Microsoft.Build.dll!Microsoft.Build.Experimental.ProjectCache.ProjectCacheService.GetProjectCachePluginAsync.AnonymousMethod__1() Line 168	C#
 	mscorlib.dll!System.Lazy<System.Threading.Tasks.Task<Microsoft.Build.Experimental.ProjectCache.ProjectCacheService.ProjectCachePlugin>>.CreateValue()	Unknown
 	mscorlib.dll!System.Lazy<System.Threading.Tasks.Task<Microsoft.Build.Experimental.ProjectCache.ProjectCacheService.ProjectCachePlugin>>.LazyInitValue()	Unknown
 	Microsoft.Build.dll!Microsoft.Build.Experimental.ProjectCache.ProjectCacheService.GetProjectCachePluginAsync(Microsoft.Build.Experimental.ProjectCache.ProjectCacheDescriptor projectCacheDescriptor, Microsoft.Build.Graph.ProjectGraph projectGraph, Microsoft.Build.BackEnd.BuildRequestConfiguration buildRequestConfiguration, System.Collections.Generic.ICollection<string> requestedTargets, System.Threading.CancellationToken cancellationToken) Line 164	C#
 	Microsoft.Build.dll!Microsoft.Build.Experimental.ProjectCache.ProjectCacheService.InitializePluginsForVsScenario.AnonymousMethod__1(Microsoft.Build.Experimental.ProjectCache.ProjectCacheDescriptor projectCacheDescriptor) Line 151	C#
 	mscorlib.dll!System.Threading.Tasks.Parallel.ForWorker.AnonymousMethod__1()	Unknown
 	mscorlib.dll!System.Threading.Tasks.Task.InnerInvokeWithArg(System.Threading.Tasks.Task childTask)	Unknown
 	mscorlib.dll!System.Threading.Tasks.Task.ExecuteSelfReplicating.AnonymousMethod__0(object <p0>)	Unknown
 	mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)	Unknown
 	mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)	Unknown
 	mscorlib.dll!System.Threading.Tasks.Task.ExecuteWithThreadLocal(ref System.Threading.Tasks.Task currentTaskSlot)	Unknown
 	mscorlib.dll!System.Threading.Tasks.Task.ExecuteEntry(bool bPreventDoubleExecution)	Unknown
 	mscorlib.dll!System.Threading.Tasks.ThreadPoolTaskScheduler.TryExecuteTaskInline(System.Threading.Tasks.Task task, bool taskWasPreviouslyQueued)	Unknown
 	mscorlib.dll!System.Threading.Tasks.TaskScheduler.TryRunInline(System.Threading.Tasks.Task task, bool taskWasPreviouslyQueued)	Unknown
 	mscorlib.dll!System.Threading.Tasks.Task.InternalRunSynchronously(System.Threading.Tasks.TaskScheduler scheduler, bool waitForCompletion)	Unknown
 	mscorlib.dll!System.Threading.Tasks.Parallel.ForWorker<object>(int fromInclusive, int toExclusive, System.Threading.Tasks.ParallelOptions parallelOptions, System.Action<int> body, System.Action<int, System.Threading.Tasks.ParallelLoopState> bodyWithState, System.Func<int, System.Threading.Tasks.ParallelLoopState, object, object> bodyWithLocal, System.Func<object> localInit, System.Action<object> localFinally)	Unknown
 	mscorlib.dll!System.Threading.Tasks.Parallel.ForEachWorker<Microsoft.Build.Experimental.ProjectCache.ProjectCacheDescriptor, object>(System.Collections.Generic.IEnumerable<Microsoft.Build.Experimental.ProjectCache.ProjectCacheDescriptor> source, System.Threading.Tasks.ParallelOptions parallelOptions, System.Action<Microsoft.Build.Experimental.ProjectCache.ProjectCacheDescriptor> body, System.Action<Microsoft.Build.Experimental.ProjectCache.ProjectCacheDescriptor, System.Threading.Tasks.ParallelLoopState> bodyWithState, System.Action<Microsoft.Build.Experimental.ProjectCache.ProjectCacheDescriptor, System.Threading.Tasks.ParallelLoopState, long> bodyWithStateAndIndex, System.Func<Microsoft.Build.Experimental.ProjectCache.ProjectCacheDescriptor, System.Threading.Tasks.ParallelLoopState, object, object> bodyWithStateAndLocal, System.Func<Microsoft.Build.Experimental.ProjectCache.ProjectCacheDescriptor, System.Threading.Tasks.ParallelLoopState, long, object, object> bodyWithEverything, System.Func<object> localInit, System.Action<object> localFinally)	Unknown
 	mscorlib.dll!System.Threading.Tasks.Parallel.ForEach<Microsoft.Build.Experimental.ProjectCache.ProjectCacheDescriptor>(System.Collections.Generic.IEnumerable<Microsoft.Build.Experimental.ProjectCache.ProjectCacheDescriptor> source, System.Threading.Tasks.ParallelOptions parallelOptions, System.Action<Microsoft.Build.Experimental.ProjectCache.ProjectCacheDescriptor> body)	Unknown
 	Microsoft.Build.dll!Microsoft.Build.Experimental.ProjectCache.ProjectCacheService.InitializePluginsForVsScenario.AnonymousMethod__0() Line 145	C#
 	mscorlib.dll!System.Threading.Tasks.Task.Execute()	Unknown
 	mscorlib.dll!System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)	Unknown
 	mscorlib.dll!System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext executionContext, System.Threading.ContextCallback callback, object state, bool preserveSyncCtx)	Unknown
 	mscorlib.dll!System.Threading.Tasks.Task.ExecuteWithThreadLocal(ref System.Threading.Tasks.Task currentTaskSlot)	Unknown
 	mscorlib.dll!System.Threading.Tasks.Task.ExecuteEntry(bool bPreventDoubleExecution)	Unknown
 	mscorlib.dll!System.Threading.ThreadPoolWorkQueue.Dispatch()	Unknown

I suspect an exception thrown from this isn't caught properly such that the failed plugin will still be disposed of.

Versions & Configurations

Dev17.12 (35228.240.main)

[AR-May]

@dfederm could you please take a look on this issue?

[dfederm]

Yep, I'll look at some point. I worked with Andrew on this and am just using this issue for tracking so I don't forget.