This avoids relying on finding these dependencies in $PATH while
running the tests. Some tests may set PATH="" while executing them,
which previously caused those tests to fail in this build setup.
The same is not an issue with MSVC builds with
-DCMAKE_MSVC_RUNTIME_LIBRARY=MultiThreadedDLL, because those
DLLs are also available in global system directories (including
DLLs such as msvcp140.dll and vcruntime140.dll).
This matches what 28851edf164a337c334755ae33fd58f03cffd5a2 in
llvm-project did.
When setting up the built libcxx files for testing, the built
files are installed into a temporary test root, and the install of
them litters the build log with mostly uninteresting output.
This is required for building libffi. With ubuntu-22.04 images,
the libltdl-dev package was available out of the box.
Also explicitly install swig, which we rely on - this also seems
to have been available implicitly.
This fixes building on Ubuntu 24.04 (the one currently used by
the github actions ubuntu-latest tag), by including a fix for texinfo
7.0.
Not bumping to ffmpeg 7.x, as some tests there can be flaky at times.
These days, it seems to be installed out of the box already on the
runner images, and installing it triggers very visible warnings
about it already being installed.
Currently, the "macos-latest" runners aren't really the latest
version - on the contrary, they're running the oldest HW/OS.
Currently, there are macos-12, macos-13 and macos-14, and macos-latest
are the macos-12 ones.
The macos-14 ones run on Apple Silicon.
Previously, the macos-12 runners have usually managed to run the build
job in less than 3 hours. However, lately, the jobs often end up
taking much longer, over 5 hours, and have ended up exceeding the
6 hour timeout occasionally, even if they still also some times run
in the expected time of less than 3 hours.
Switch to the faster macos-14 runners (which can complete the build
job in around 1 h 40 min), to avoid these timeouts.
6b6b1240e2 tried to build a
multiplatform image, by building two images separately and pushing
them one at a time, with the same name (but with a different platform).
This doesn't work as expected; the last push for the image name
replaces earlier pushes. So depending on which push finishes
last, we'd end up with either an amd64 or arm64 image, but not
one that supports both.
Instead, push two separate images just with push-by-digest, and
then create the actual tag with "docker buildx imagetools create".
Alternatively, we could build all platform variants of the main
image with Dockerfile.toolchain, simplifying the setup significantly.
The contents of such a build is marginally different though, so
keep the amd64 image as it was, for now, compiled in that environment.
The total number of exported symbols in this build configuration has
now reached the limit for the number of exported symbols in a DLL,
with the latest version of llvm-project from git.
This isn't an issue when building with Clang, as it marks many
symbols as excluded from auto-export.
Previously, when echoing the definition of BUILD_DATE to
$GITHUB_OUTPUT, that doesn't make it available as an environment
variable. (And if it would be echoed to $GITHUB_ENV, it still
wouldn't take effect until the next step within the same job either.)
Therefore, this previously set BUILD_DATE to $(date -d "" +%s),
which seems to produce a timestamp corresponding to the start of
the current day.
This sets SOURCE_DATE_EPOCH to the commit timestamp, toolchains
should pick up timestamps from this variable.
The same sources ideally would produce the same binary, see
https://reproducible-builds.org.
Set the timestamps within the tar archives to the start of the
CI pipeline.
Sort the archive contents.
The generated tar archives contain the uid/gid of the builder,
if unpacked as root on another system those will denote nothing
or an unexpected user/group.
The expectation would be uid/gid = 0.
Use a simple enough tar format as well.
The Mac version of tar (bsdtar?) does not support those flags,
for CI the separate gnu-tar is installed.
These were missed in ad30b74173.
The mismatch between upload-artifact and download-artifact in
ad30b74173 makes the later steps
fail to download the artifacts from previous steps.
This makes the built toolchain usable outside of MSYS2 too.
For running tests, this has another very subtle effect.
If the toolchain is built in the MSYS2 clang64 environment, the
compiler binary depends on libc++.dll. This DLL is found in $PATH,
in /clang64/bin.
When building/running tests, the current working directory will be
e.g. llvm-mingw/test/x86_64. At some point while building/running
tests, the test makefile will copy the llvm-mingw toolchain's
runtime DLLs into this directory in order to be able to execute those
newly built binaries. This also includes another, different libc++.dll.
When a libc++.dll exists in the current directory, this is preferred
over the one found further back in $PATH in /clang64/bin. As this
file is copied in parallel with other build jobs that might be
doing compilation, those compilation processes may end up loading
this other DLL. If fully copied it probably shouldn't be an issue,
but if not fully copied yet, who knows what might happen.
By making sure that there is a libc++.dll in the same directory as
the toolchain executables (rather than expecting it to be found
further back in $PATH), we ensure that this DLL is loaded and used
instead of the one in the current directory.
This should fix stray errors when running tests in the clang64
environment.
This skips test that use a lot of memory. In particular, there's
one test that would use up to 6.8 GB of memory, while the GitHub
Actions runners only have 8 GB of memory available.
This parameter was added in 122064a6303eb9c06e0af231f5a4ce145d9a2e67
in the llvm-project main branch, and backported to 17.x in
aa03fba83b17bbfcd9ed869efed2818b503875d6.
This allows seeing the names of tests as they are executed, making
the intermediate log output on github actions more useful. It
also makes it easier to correlate test issues between tests as
they are executed (e.g. if one test causes another one to run out
of memory).
Also run with --time-tests, allowing identifying slow tests.
This was already applied in the separate libcxx test job added in
7483eae1821766a0f84dbd96f5619058f7065008; this backports the change
into the test job in the main build file.
Ninja isn't documented as one of the tools provided by the CI runner
images, but it used to be available implicitly installed before
in the windows-2022 images. Since the 20230918.1.0 image update,
it no longer is available out of the box; see
https://github.com/actions/runner-images/issues/8343.
Jeremy Drake