[clang][perf-training] Support excluding LLVM build from PGO training

[petrhosek]

Using LLVM build itself for PGO training is convenient and a great starting point but it also has several issues:

• LLVM build implicitly depends on tools other than CMake and C/C++ compiler and if those tools aren't available in PATH, the build will fail.
• LLVM build also requires standard headers and libraries which may not always be available in the default location requiring an explicit sysroot.
• Building a single configuration (-DCMAKE_BUILD_TYPE=Release) only exercises the -O3 pipeline and can pesimize other configurations.
• Building for the host target doesn't exercise all other targets.
• Since LLVMSupport is a static library, this doesn't exercise the linker (beyond what the CMake itself does).

Rather than using LLVM build, ideally we would provide a more minimal, purpose built corpus. While we're working on building such a corpus, provide a CMake option that lets vendors disable the use LLVM build for PGO training.

[llvmbot]

@llvm/pr-subscribers-clang

Author: Petr Hosek (petrhosek)

Changes

Using LLVM build itself for PGO training is convenient and a great starting point but it also has several issues:

• LLVM build implicitly depends on tools other than CMake and C/C++ compiler and if those tools aren't available in PATH, the build will fail.
• LLVM build also requires standard headers and libraries which may not always be available in the default location requiring an explicit sysroot.
• Building a single configuration (-DCMAKE_BUILD_TYPE=Release) only exercises the -O3 pipeline and can pesimize other configurations.
• Building for the host target doesn't exercise all other targets.
• Since LLVMSupport is a static library, this doesn't exercise the linker (beyond what the CMake itself does).

Rather than using LLVM build, ideally we would provide a more minimal, purpose built corpus. While we're working on building such a corpus, provide a CMake option that lets vendors disable the use LLVM build for PGO training.

---

Full diff: https://github.com/llvm/llvm-project/pull/126876.diff

3 Files Affected:

• (modified) clang/utils/perf-training/CMakeLists.txt (+6)
• (modified) clang/utils/perf-training/lit.cfg (+3)
• (modified) clang/utils/perf-training/lit.site.cfg.in (+1)

diff --git a/clang/utils/perf-training/CMakeLists.txt b/clang/utils/perf-training/CMakeLists.txt
index 4aed086563ee9..0c1cdd9a1fb60 100644
--- a/clang/utils/perf-training/CMakeLists.txt
+++ b/clang/utils/perf-training/CMakeLists.txt
@@ -6,6 +6,12 @@ set(CLANG_PGO_TRAINING_DATA "${CMAKE_CURRENT_SOURCE_DIR}" CACHE PATH
 set(CLANG_PGO_TRAINING_DATA_SOURCE_DIR OFF CACHE STRING "Path to source directory containing cmake project with source files to use for generating pgo data")
 set(CLANG_PGO_TRAINING_DEPS "" CACHE STRING "Extra dependencies needed to build the PGO training data.")
 
+option(CLANG_PGO_TRAINING_USE_LLVM_BUILD "Use LLVM build for generating PGO data" ON)
+
+llvm_canonicalize_cmake_booleans(
+  CLANG_PGO_TRAINING_USE_LLVM
+)
+
 if(LLVM_BUILD_INSTRUMENTED)
   configure_lit_site_cfg(
     ${CMAKE_CURRENT_SOURCE_DIR}/lit.site.cfg.in
diff --git a/clang/utils/perf-training/lit.cfg b/clang/utils/perf-training/lit.cfg
index adefc7893ac44..3f6089b7139a7 100644
--- a/clang/utils/perf-training/lit.cfg
+++ b/clang/utils/perf-training/lit.cfg
@@ -27,6 +27,9 @@ config.clang = lit.util.which('clang', config.clang_tools_dir).replace('\\', '/'
 config.name = 'Clang Perf Training'
 config.suffixes = ['.c', '.cc', '.cpp', '.m', '.mm', '.cu', '.ll', '.cl', '.s', '.S', '.modulemap', '.test']
 
+if not config.use_llvm_build:
+    config.excludes = ['llvm-support']
+
 cc1_wrapper = '%s %s/perf-helper.py cc1' % (config.python_exe, config.perf_helper_dir)
 
 use_lit_shell = os.environ.get("LIT_USE_INTERNAL_SHELL")
diff --git a/clang/utils/perf-training/lit.site.cfg.in b/clang/utils/perf-training/lit.site.cfg.in
index 9d279d552919a..da81ec21a28a6 100644
--- a/clang/utils/perf-training/lit.site.cfg.in
+++ b/clang/utils/perf-training/lit.site.cfg.in
@@ -11,6 +11,7 @@ config.python_exe = "@Python3_EXECUTABLE@"
 config.cmake_exe = "@CMAKE_COMMAND@"
 config.llvm_src_dir ="@CMAKE_SOURCE_DIR@"
 config.cmake_generator ="@CMAKE_GENERATOR@"
+config.use_llvm_build = @CLANG_PGO_TRAINING_USE_LLVM_BUILD@
 
 # Let the main config do the real work.
 lit_config.load_config(config, "@CLANG_SOURCE_DIR@/utils/perf-training/lit.cfg")

[petrhosek]

@tstellar We're trying to build a more minimal corpus that would address the issues I listed with @ilovepi. In the meantime, it'd be helpful to have this option since the current implementation actually breaks on our builders (which don't have host toolchain and sysroot installed).

[tstellar]

LGTM.

[tstellar]

@petrhosek I also wanted to mention, if you are interested in upstreaming this minimal corpus, I would support making it the default instead of the current llvm-support build.

[ilovepi]

@tstellar We're still developing the corpus ATM, but contributing it upstream is something we've discussed and would like to do.