Preventing MSBuild from replacing my binary dlls

[michaelsamoilenko]

My projects get some dlls as binary references. We have build folder and some dlls are copied to that folder and we reference that dlls in our projects. Example:

<Reference Include="System.Text.Json">
      <SpecificVersion>False</SpecificVersion>
      <Private>False</Private>
    </Reference>

At the same time our projects get some dlls from nuget packages. So our projects have references to some packages.
The problem is that default MSBuild behavior will replace some of our binary referenced dlls with some stuff which it got from nuget package. I'll explain:

• A.dll is binary reference, it's in a build folder
• B.dll is in nuget package and it references A.dll . A.dll is also stored as nuget package in nuget package cache
  when we run MSBuild it copies B.dll from nuget package to build folder and overrides our A.dll with that one from nuget package
  and overrides our A.dll with that one.
  We want to disable this. We want MSBuild to keep our dlls and not to do any replacement

The possible workaround is to use ExcludeAssets for B.dll. The problem is that by doing that we only resolve current problems but we don't protect us from having troubles in the future. If somebody adds new nuget package and we don't do ExludeAssets the same problem will reappear.

We need some solution that will work always when user adds some new nuget package and that package inside it references to some other package which dll is actually the one we already have as binary reference.

Any thoughts?

[rainersigwald]

Can you share a binary log (⚠️ note information-disclosure concerns) of this happening? I would generally expect the direct project reference to "win" over a transitive reference, so I think we'll need to understand the details here.

[michaelsamoilenko]

hey, we need some time to gather information.

[michaelsamoilenko]

@rainersigwald , sorry it took a while for us to prepare.
Just a bit of background to this request. Our team is building a small component of a legacy monolithic software application which has all binaries deployed to a single folder.

This single folder already contains a predetermined version of certain 3rd party dependencies (e.g. System.Memory). We need to build against exactly this version, rather than any version resolved as a PackageReference because ultimately, at runtime, the version referenced as a binary reference will be what is loaded. A custom AssemblyResolver is used to avoid any version mismatches. We understand that this is far from best practice, but unfortunately this setup is beyond our control.

To give a somewhat contrived example, in our project we might reference System.Text.Json as a PackageReference. Our single deployment folder already contains one of it's dependencies, System.Memory. We attempt to make this work by adding both the binary reference to System.Memory in our project, as well as the PackageReference to System.Text.Json. We need the binary reference to take precedence over any transitive references resolved for the dependencies of System.Text.Json.

This is not what happens. Our investigation so far shows that ResolvePackageAssets correctly resolves both the binary reference to System.Memory as well as the reference in the package. This is later de-duplicated in ResolveLockFileReferences, with the reference from the package winning. The build then overwrites the "pinned" binary in the output folder with the one from the package.

We haven't been able to find a simple way to make sure the binary reference takes precedence. The best attempt we've had is explicitly adding transitive binary references to every project where this is needed, followed by a custom target which will eliminate duplicate references from a package. The source for our custom target is attached below:

<Target Name="PreferBinaryReferences" AfterTargets="ResolvePackageAssets" Condition="'$(ExcludeTopLevelReferences)' != 'true'">
    <JoinItems Right="@(ResolvedCompileFileDefinitions)" RightKey="FileName" RightMetadata="*" Left="@(Reference)" LeftKey="" LeftMetadata="*">
      <Output TaskParameter="JoinResult" ItemName="_DuplicateReferences"/>
    </JoinItems>

    <RemoveDuplicates Inputs="@(_DuplicateReferences)">
      <Output TaskParameter="Filtered" ItemName="_ReferencesToRemove"/>
    </RemoveDuplicates>

    <ItemGroup>
      <_ReferencesByHintPath Include="@(_ReferencesToRemove -> '%(HintPath)')"/>
      <ResolvedCompileFileDefinitions Remove="@(ResolvedCompileFileDefinitions)" Condition="'%(Filename)' == '@(_ReferencesByHintPath -> '%(FileName)')'"/>
      <RuntimeCopyLocalItems Remove="@(RuntimeCopyLocalItems)" Condition="'%(Filename)' == '@(_ReferencesByHintPath -> '%(FileName)')'"/>
      <RuntimeTargetsCopyLocalItems Remove="@(RuntimeTargetsCopyLocalItems)" Condition="'%(Filename)' == '@(_ReferencesByHintPath -> '%(FileName)')'"/>
    </ItemGroup>
  </Target>

Also I'm sending a zip file with binlog and the whole sample project where we have reproduced an issue
ReproSampleProjects.zip

[rainersigwald]

(I haven't looked deeply yet) Have you considered NuGet transitive pinning the packages that have defined versions in your output?

[michaelsamoilenko]

We have considered this, but two problems:

• It puts a burden on our team to monitor and keep the transitive pinning up-to-date with whatever is defined centrally (dozens of teams contribute to this with little communication)
• We define files centrally. Not packages, not versions, but actual binary files. Working out the package version given just a binary is not always easy, and on occasion we've seen incompatible dependencies. We somehow get away with this at runtime.

[michaelsamoilenko]

@rainersigwald , any luck of finding a simpler solution?

[michaelsamoilenko]

bump 😄