Having a Project dependency in incremental Target

[NogginBops]

Hello! I have some questions regarding incremental build targets.
I'm trying to hook up a code generation project to another project. I have one project that generates code, and I want the second project to automatically generate code if it is out of date.
I've managed to build and run the generator project during the build on the main project.
I'm trying to use the incremental feature as the code generation step is quite slow, so I've managed to make the target not run if the generated files are up to date compared to a specification file, and it works.
But now I want any changes to the generator project to trigger the generation in the main project.
So I basically need to include an entire project in the Input attribute, but I don't know how to do that or if it's even possible?

My targets are setup like this:

<ItemGroup>
  <Spec Include="..\spec.xml" />
  <GeneratedFiles Include="files to generate" Exclude="manually edited files" />
</ItemGroup>

<!--  'DispatchToInnerBuilds' is so that this target is only run once in contrast to task that run after 'PreBuildEvent' which get run multiple times for each target framework  -->
<Target Name="Download spec" BeforeTargets="DispatchToInnerBuilds">
  <!-- Here we check if spec.xml exists, if it doesn't we download the file. -->
  <DownloadFile SourceUrl="url" DestinationFolder=".." SkipUnchangedFiles="true" Condition="Exists('../spec.xml') == 'false'" />
</Target>

<!-- We build this project once for the entire build, then we can call into the exe directly to avoid having multiple processes trying to build the project at the same time and failing. -->
<Target Name="Build bindings generator" BeforeTargets="DispatchToInnerBuilds">
  <Exec Command="dotnet build ./src/Generator.Bind/Generator.Bind.csproj --framework netcoreapp3.1" WorkingDirectory="../../" StandardOutputImportance="low" StandardErrorImportance="low" />
</Target>

<!-- After we've downloaded the spec file we can generate the bindings if needed. By setting Inputs and Outputs msbuild decides when we need to generate the bindings. -->
<Target Name="Generate bindings" AfterTargets="Download spec" BeforeTargets="CoreCompile;CopyFilesToOutputDirectory" Inputs="@(Spec)" Outputs="@(GeneratedFiles)">
  <Exec Command="echo Generating bindings..." />

  <Exec Command=".\src\Generator.Bind\bin\$(Configuration)\netcoreapp3.1\Bind.exe" WorkingDirectory="../../" />
  
  <Exec Command="echo Generated bindings!" />

  <ItemGroup>
    <Compile Include="@(GeneratedFiles)" Condition="'%(Compile.Identity)' != '%(GeneratedFiles.GeneratedFiles)'"/>
    <!-- For clean to work properly -->
    <FileWrites Include="@(GeneratedFiles)" />
  </ItemGroup>
</Target>

Any other tips on how to improve this project file setup would also be greatly appreciated. There are probably tons of features that I don't even know about.

[rainersigwald]

But now I want any changes to the generator project to trigger the generation in the main project.

One way that many projects think about this is not "were there any changes in the generator project" but "has the output of the generator project changed?" Would that work for you? If so, add the generator executable and any support files like DLLs or configuration to the inputs of the GenerateBindings target.

[rainersigwald]

  <Exec Command="echo Generating bindings..." />

You can accomplish this goal with lower overhead and cleaner logging by using the Message task:

-  <Exec Command="echo Generating bindings..." />
+  <Message Text="Generating bindings..." />

You may also wish to add Importance="high", if you want to see the messages in a normal build output (the default is Normal and doesn't make it to the console when using lower log verbosities).

[NogginBops]

The fact that <Message Text="message" /> doesn't show anything in the log by default caused me much headache when trying to learn this. Only thing I got to work reliable at the time was echo.
I modified the project to use <Message> and it works with Importance="high", thanks!

[rainersigwald]

We have docs on a fairly closely related scenario: https://learn.microsoft.com/visualstudio/msbuild/tutorial-custom-task-code-generation. That uses a task to do the code generation, instead of calling a command-line tool, but much of the setup is similar so reading that will likely be helpful. As always with our docs, feedback is welcome!

[rainersigwald]

It's almost always a bad idea to exec dotnet build within a build--it can create race conditions, the logging is mostly lost, and it's just generally clunky.

Here, you can replace that target with a ProjectReference to the generator project, with some special properties to ensure that it builds "normally" instead of "how it would need to to be referenced here".

I made some modifications to your logic before starting this but this is the diff I applied:

diff --git a/ConsumesGeneratedFiles/ConsumesGeneratedFiles.csproj b/ConsumesGeneratedFiles/ConsumesGeneratedFiles.csproj
index 3ab3a7d..51f9003 100644
--- a/ConsumesGeneratedFiles/ConsumesGeneratedFiles.csproj
+++ b/ConsumesGeneratedFiles/ConsumesGeneratedFiles.csproj
@@ -12,12 +12,13 @@
   <GeneratedFiles Include="..\Generated.cs" />
 </ItemGroup>
 
-<!--  'DispatchToInnerBuilds' is so that this target is only run once in contrast to task that run after 'PreBuildEvent' which get run multiple times for each target framework  -->
-
-<!-- We build this project once for the entire build, then we can call into the exe directly to avoid having multiple processes trying to build the project at the same time and failing. -->
-<Target Name="Build bindings generator" BeforeTargets="DispatchToInnerBuilds">
-  <Exec Command="dotnet build Generator/Generator.csproj --framework netcoreapp3.1" WorkingDirectory=".." StandardOutputImportance="low" StandardErrorImportance="low" />
-</Target>
+<ItemGroup>
+  <ProjectReference Include="..\Generator\Generator.csproj"
+                    SkipGetTargetFrameworkProperties="true"
+                    UndefineProperties="TargetFramework"
+                    ReferenceOutputAssembly="false"
+                    ExcludeAssets="all" />
+</ItemGroup>
 
 <!-- After we've downloaded the spec file we can generate the bindings if needed. By setting Inputs and Outputs msbuild decides when we need to generate the bindings. -->
 <Target Name="Generate bindings" AfterTargets="Download spec" BeforeTargets="CoreCompile;CopyFilesToOutputDirectory" Inputs="@(Spec)" Outputs="@(GeneratedFiles)">

Relevant metadata:

• SkipGetTargetFrameworkProperties means "don't worry about TargetFramework compatibility" since we're just going to run it.
• UndefineProperties ensures that the current TargetFramework doesn't flow to the reference, so it can be anything.
• ReferenceOutputAssembly means "don't put this assembly in the compiler's inputs for this project" since it's just going to generate code.
• ExcludeAssets ensures that any NuGet package references in the generator don't apply to the consuming project.

[NogginBops]

Thanks! This works great.
I tried doing the same thing with a postprocess project and it seems like it didn't really work and I got error MSB4057: The target "GetTargetPath" does not exist in the project (#4303 related?)

Here is the relevant parts of the csproj:
main csproj (the same one I described above)

<ItemGroup>
    <!-- Add a project reference here so that the Generator.Rewrite project is built, 
    but make sure that we don't pass framework configs to it and exlude all assets from the final build. -->
    <ProjectReference Include="..\Generator.Rewrite\Generator.Rewrite.csproj"
                      SkipGetTargetFrameworkProperties="true"
                      UndefineProperties="TargetFramework"
                      ReferenceOutputAssembly="false"
                      ExcludeAssets="all" />
</ItemGroup>

<!-- After we have compiled the dll we run the rewriter -->
  <Target Name="Rewrite" BeforeTargets="PostBuildEvent">
    <Message Text="Rewriting src/OpenTK.Graphics/bin/$(Configuration)/$(TargetFramework)" Importance="high" />
    <Exec Command=".\src\Generator.Rewriter\bin\$(Configuration)\netcoreapp3.1\Rewrite.exe -a ./src/OpenTK.Graphics/bin/$(Configuration)/$(TargetFramework)/OpenTK.Graphics.dll" WorkingDirectory="../../" />
    <Message Text="Rewrote src/OpenTK.Graphics/bin/$(Configuration)/$(TargetFramework)" Importance="high" />
  </Target>

rewriter.csproj

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFrameworks>netcoreapp3.1</TargetFrameworks>
    <AllowUnsafeBlocks>true</AllowUnsafeBlocks>
    <RootNamespace>OpenTK.Rewrite</RootNamespace>
    <AssemblyName>Rewrite</AssemblyName>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="CommandLineParser" Version="2.7.82" />
    <PackageReference Include="Mono.Cecil" Version="0.11.1" />
  </ItemGroup>

  <Import Project="..\..\props\common.props" />
  <Import Project="..\..\props\nuget-common.props" />
</Project>

I tried changing <TargetFrameworks> to <TargetFramework> in the rewriter csproj with no success. I also tried changing UndefineProperties="TargetFramework" to UndefineProperties="TargetFrameworks" which didn't work either.

[rainersigwald]

Hm, a single-targeted (TargetFramework instead of TargetFrameworks) rewriter.csproj shouldn't have that problem. Would it be possible for you to share a binlog of the failure? Please read the warnings about how much information you share if you post a binlog before doing so.

[NogginBops]

Here is some more information on the issue I managed to figure out.

The issue only appears when building in visual studio.
If I do dotnet build path/to/project_that_references_the_project.csproj in a console everything works fine.

If I try to build within visual studio I get the following error message:

2>------ Build started: Project: OpenTK.Graphics, Configuration: Debug Any CPU ------
2>C:\...\src\Generator.Rewrite\Generator.Rewrite.csproj : error MSB4057: The target "GetTargetPath" does not exist in the project.
2>Done building project "Generator.Rewrite.csproj" -- FAILED.

I'm using VS2022 17.3.4

[NogginBops]

Hmm, it seems like Generator.Rewrite.csproj was using TargetFrameworks instead of TargetFramework. Changing that fixed this issue, now I just have to fixed the rest of the issues, which hopefully won't be a problem.