Commit Graph
876 Commits
Author SHA1 Message Date
Martin Storsjö 8536b740e3 build-llvm: Use the new LLVM_NATIVE_TOOL_DIR option for pointing out preexisting native tools
This allows setting one variable pointing at all tools, instead
of setting multiple individual options for all tools.
2023-01-29 01:13:48 +02:00
Martin Storsjö c49460aa86 github: Add setup for running libcxx's tests on the built toolchain
Keep this disabled by default (since it takes a fair amount of time
and has good test coverage upstream), but including it in disabled
form so that it's easy to enable temporarily for testing occasionally.

Libcxx does test the mingw configuration in their own CI (with an older
pinned release of the compiler), so this is mostly redundant, and
primarily relevant for testing that the very latest compiler (and
python setup) works for the tests.
2023-01-29 01:13:39 +02:00
Martin Storsjö dfd4c31f2e test: Simplify handling of OpenMP
It is available on all architectures by default now. Keep the
HAVE_OPENMP conditional within the test makefile to allow custom
testing without having it available.
2023-01-29 01:09:17 +02:00
Martin Storsjö 6b8725333a build-openmp: Build the OpenMP library on all architectures
Since a356782426f5bf54a00570e1f925345e5fda7b2e in llvm-project,
OpenMP can be built for mingw/aarch64, since
fb947c358661b3bd3d5e1fa776ec74cc0f308854 it should work correctly
in all currently tested cases, and since
97958c9bb83cec44b6ce13e732b53de0171a5d43 it can be built for
mingw/armv7.
2023-01-29 01:09:17 +02:00
Martin Storsjö c31664aba4 Dockerfile.dev: Merge the ssp tests up with an earlier block 2023-01-29 01:09:17 +02:00
Martin Storsjö fb67e16120 build-all: Stop requiring curl to be available
Curl (or wget) isn't required for fetching libssp any longer,
so don't require it while running build-all.sh.

(It's still required by build-cross-tools.sh, for build-make.sh
though.)
2023-01-29 01:09:17 +02:00
Martin Storsjö 4ba7adbb90 run-tests: Check for the error messages from ssp routines
Since using the ssp routines from mingw-w64 instead of those from
GCC's libssp, these messages are included in the piped output of
the process, so we can check for the right messages.
2023-01-29 01:09:17 +02:00
Martin Storsjö abc36e1f7c run-tests: Stop trying to copy libssp-0.dll
We no longer build that DLL.
2023-01-29 01:09:17 +02:00
Martin Storsjö 4ee7cc97de build-libssp: Stop building the real libssp from GCC
As mingw-w64 now provides these routines, we no longer need to build
the original library from GCC.

As building with -fstack-protector-strong still links with -lssp
(and -lssp_nonshared), we still need to provide libraries with these
names, but they can be empty dummy libraries.
2023-01-29 01:09:17 +02:00
Martin Storsjö ef9806fc0a run-tests: Stop explicitly linking against -lssp for the fortification test
Since libmingwex.a provides these routines now, we don't need to
specify -lssp manually.
2023-01-29 01:09:17 +02:00
Martin Storsjö 36f8c1359e Switch the default to enabling CFGuard in all individual build-* scripts 2023-01-29 01:09:17 +02:00
Alvin Wong ce554e93c5 Revert "Enable CFGuard for scheduled builds which uses latest llvm and mingw-w64"
This reverts commit e76e935c7d.

This is no longer necessary, since the default currently is to
enable CFGuard.
2023-01-29 01:09:17 +02:00
Alvin Wong 9d352374f2 Enable CFGuard by default 2023-01-29 01:09:17 +02:00
Martin Storsjö d07bb8889b build-python: Bump to Python 3.10.9 2023-01-29 01:09:17 +02:00
Martin Storsjö 26ae975a6a build-mingw-w64: Bump to the current version from January 28th
Notable changes:
- Support for CFGuard
- SSP routines are bundled as part of libmingwex, rendering the separate
  libssp from GCC unnecessary

Plus lots and lots of various other minor fixes and improvements.
2023-01-29 01:09:17 +02:00
Martin Storsjö e06702461f build-llvm: Update to llvm 16.0.0 RC1
Notable changes are:
- Emit -exclude-symbols: directives for hidden symbol visibility (supported
  by LLD 15 and binutils ld 2.40). This allows reducing the number of
  exported symbols in e.g. libLLVM-*.dll.
- Support for CFGuard on mingw, thanks to Alvin Wong!
- LLD now picks up libraries named <name>.lib when linking with -l<name>,
  even if -static was specified
- Fixed SEH unwind info on AArch64 if using the AArch64 feature
  PAC (Pointer Authentication Code) - previously, unwinding would fail if
  running on a system that actually does use the feature. (The consistency
  of AArch64 SEH unwind info was also improved for other cases.)
- Fixed stack alignment on AArch64, for stack frames large enough to
  require stack probing
- Fixed llvm-objcopy corrupting some cases of object files with empty sections
- Improvements to LLDB, where it now uses the regular symbol list of DLLs
  (including non-exported symbols)
- OpenMP now can be built and works on armv7 and aarch64 (in mingw
  configurations)

Notable for simplifying how to cross compile LLVM itself:
- The preexisting build machine tools can be specified with one single option
  LLVM_NATIVE_TOOL_DIR instead of passing a number of variables like
  LLVM_TABLEGEN, CLANG_TABLEGEN etc.
2023-01-29 01:08:43 +02:00
Martin Storsjö 49950ae6be build-lldb-mi: Skip building lldb-mi, as it's incompatible with current llvm versions
Since the switch to building in C++17 mode in llvm 16, lldb-mi is
incompatible, until lldb-mi itself switches to building with C++17 too.

A fix has been suggested upstream in
https://github.com/lldb-tools/lldb-mi/pull/102, but there hasn't
been any action on it so far.
2023-01-29 01:06:10 +02:00
Martin Storsjö 241a1cd148 test: Prepare the test scripts for testing native Windows tools via WSL
Use the TOOLEXT variable in run-tests.sh in the same way as in the
Makefile. The Makefile sets this variable automatically if invoked
by mingw32-make, but if running run-tests.sh in WSL with native
Windows compilers, then we need to set TOOLEXT manually when calling
run-tests.sh.

Allow skipping the cfguard failure tests when run in such a
configuration, since Linux make in WSL can't easily run cmd.exe
like in MSYS2.
2023-01-21 23:48:24 +02:00
Martin Storsjö 9d96e5745f Include the llvm-size tool like other binutils-like tools
This was requested in
https://github.com/mstorsjo/llvm-mingw/issues/320.
2023-01-12 14:11:03 +02:00
Martin Storsjö bd190c04c3 build-llvm: Remove the --symlink-projects option
This option was mostly relevant for making it possible to share
ccache between builds in different directories, due to specific
logic in CMake's Ninja generator, which used relative paths for
sources as long as they were below the main source directory
(llvm-project/llvm). This option wasn't usable on Windows.

Since CMake 3.21.0, this option no longer worked for the intended
purpose - that version of CMake changed the Ninja generator to
always use absolute paths to sources, for consistency with other
generators and to simplify things. (See commit
c564a3e3fff52ef811291c5ba8fb07a5a1b47f97 in CMake's git history.)

It is still possible to achieve ccache sharing across build
directories anyway, by setting the ccache option CCACHE_BASEDIR,
which makes ccache rewrite absolute paths to relative ones within
that tree, and by setting the LLVM CMake option
LLVM_USE_RELATIVE_PATHS_IN_FILES=ON, which adds -ffile-prefix-map
options to the compiler, to remap source paths as they are exposed
in the generated code.

(Strictly speaking, LLVM_USE_RELATIVE_PATHS_IN_FILES is only needed
if debug info is enabled. On its own, without CCACHE_BASEDIR, that
option produces compiler output that is identical across build
directories - but the absolute paths end up in the preprocessor output
in source file/line comments, which makes ccache fail to reuse the cache.)
2023-01-10 12:01:06 +02:00
Martin Storsjö 04c623fe8b crt-test: Improve the check for testing the >64 bit range for strtold
The previous condition failed to identify that macOS on aarch64
also has long double equal to regular double.
2022-12-01 12:13:20 +02:00
Martin Storsjö 21c279aab5 crt-test: Skip the NAN preservation tests on platforms other than mingw
This part of the tests does pass on glibc on x86_64 and on MSVC,
but fails on macOS (for some functions) and on glibc on aarch64
(for one function).

This is a somewhat overly pedantic test, but is useful for checking
the NAN preservation behaviour when modifying implementations.

Also print the sign bit explicitly - not all printf implementations
distinguish between NAN and -NAN when printed.
2022-12-01 12:07:32 +02:00
Martin Storsjö 6a37ec900b github: Test building and running the OpenMP tests on Windows
These tests should pass with the latest version from git (but
don't pass with the current pinned 15.0.0 version).
2022-11-29 22:40:47 +02:00
Martin Storsjö 01597c1d72 github: Include the version file in the cross-built toolchains too 2022-11-14 00:26:44 +02:00
Martin Storsjö a5a6488198 github: Test a known-good version of ffmpeg
The latest version in git might have some tests failing occasionally
(like it does currently).
2022-11-07 23:10:21 +02:00
Martin Storsjö 486dcf3227 build-compiler-rt: Avoid finding an unrelated LLVMConfig.cmake
Since 23c665371a98c7a16765d12bf2db19b0626b7181 in llvm-project
(Nov 2, 2022), compiler-rt uses CMake's find_package() for
finding potentially installed LLVM files, instead of using
llvm-config.

If the surrounding distribution does have an unrelated install
of LLVM packages, compiler-rt's CMake files would find this, and
add include directories from it into the include paths used
when building compiler-rt.

The ORC runtimes include libc++ headers, and those error out
on these unexpected include topologies (when libc++'s cstddef
includes stddef.h and expects to get libc++'s stddef.h, but ends
up getting a different stddef.h included). (Without this,
it wouldn't have been noticed that the build ends up using a
mismatched sysroot's headers.)

This fixes building in msys2 in the clang64 environment, where
the base toolchain packages end up including LLVM packages,
including LLVMConfig.cmake.

This fixes the same issue as
3b96433cdf did, for the new form of
compiler-rt cmake files.
2022-11-03 11:49:05 +02:00
Martin Storsjö e934cba58e test: Include the runtime DLLs even if not running tests
This makes the produced binaries ready for running separately
afterwards, which was regressed in
e16ba0772f.
2022-11-02 10:37:58 +02:00
Martin Storsjö e16ba0772f run-tests: Split out building and running of tests into a Makefile
This allows building and running the tests in parallel.
2022-10-29 23:22:56 +03:00
Martin Storsjö 710f2f055a run-tests: Rename HAS_CFGUARD to HAVE_CFGUARD
This picks a different naming style for flags like this.
2022-10-29 23:22:56 +03:00
Martin Storsjö 0d1f6acfdc prepare-cross-toolchain: Copy the share subdirectory in the cross roots
This makes the cross built two-stage toolchains match the single-stage
cross toolchains in this aspect.
2022-10-29 23:16:37 +03:00
Martin Storsjö a7130b4396 github: Set a ulimit timeout when cross-building ffmpeg 2022-10-29 23:07:43 +03:00
Martin Storsjö 5e0fd8fe82 prepare-cross-toolchain: Copy all runtime DLLs to the main bin directory
We used to copy libc++ and libunwind, since those were needed
to actually run the toolchain - but others, like the asan runtime,
may be needed for the user to be able to run the executables that
are built. If this bin directory is in the user's $PATH, they should
be picked up automatically.

For distribution, and for architectures other than the default,
the user still needs to use the binaries from the arch specific
directories.
2022-10-27 22:17:04 +03:00
Martin Storsjö 9750d128ad Revert "github: Update the environment used for doing msys2 builds"
This reverts commit 55db22d242.

The base tarball for setup-msys2 should now be new enough to
include the patched compiler in the clang64 environment, so this
no longer requires updating the environment on each build.
2022-10-27 22:17:04 +03:00
Martin Storsjö 64dffd28a5 run-tests: Merge TESTS_CPP_LOAD_DLL back into TESTS_CPP
This was split in 591a5c2ddc since
we couldn't/shouldn't test these for UWP targets.

When the testing for UWP targets was reduced in
551b046498, there was no longer any
distinction between TESTS_CPP_LOAD_DLL and TESTS_CPP, so merge
them back.
2022-10-27 00:40:06 +03:00
Martin Storsjö c541aa87b8 run-tests: Fix a missed rename/split of RUN_X86
This was missed in 3eb7f64593.
2022-10-27 00:40:06 +03:00
Martin Storsjö 3eb7f64593 run-tests: Allow overriding the command for running separately for i686 and x86_64 2022-10-25 11:09:38 +03:00
Martin Storsjö 5c375afc11 prepare-cross-toolchain: Include other compiler-rt files too (headers and auxillary files) 2022-10-21 18:00:21 +03:00
Martin Storsjö 36c3834a4b prepare-cross-toolchain: Add comments explaining what is done 2022-10-21 18:00:21 +03:00
Martin Storsjö c915fc9212 build-cross-tools: Check for required tools
This matches what's checked in build-all.sh.
2022-10-21 18:00:20 +03:00
Martin Storsjö aa5216db99 build-all: Allow building only tools, but no runtimes 2022-10-21 18:00:20 +03:00
Martin Storsjö 349877296b Allow overriding what archs to build for, or what version to target, for MACOS_REDIST builds 2022-10-21 18:00:20 +03:00
Martin Storsjö 53e803c424 github: Update use of actions/checkout and actions/upload-artifact from v2 to v3
The use of actions/download-artifact was already at v3.

This should avoid warnings about use of Node 12 which now is
deprecated.
2022-10-21 18:00:20 +03:00
Martin Storsjö 7c66ebe60d release-macos: Avoid picking up libzstd from e.g. homebrew
As macOS itself doesn't ship with libzstd, avoid picking it up from
third party intallations.

CMake even looks for things in directories outside of $PATH;
for Darwin targets, CMake includes /opt/homebrew in
CMAKE_SYSTEM_PREFIX_PATH which gets searched automatically.
2022-10-19 13:04:27 +03:00
Martin Storsjö 71b3bcb278 crt-test: Wrap parameters to tested string functions to avoid optimizing them
When some of the parameters to standard C functions are known
constants, the compiler can optimize out the call to the actual
function.

When built with fortification enabled, the compiler previously
didn't actually end up calling the __<func>_chk functions in
most cases, as it could optimize out much of the calls. Now all
tested function calls should end up emitted as intended, making
this actually do functional testing of the __<func>_chk function
implementations.
2022-10-14 14:34:02 +03:00
Martin Storsjö c81a93e86c crt-test: Fix tests with MSVC after changing the F() macro
The literal NAN in MSVC is a negative NAN, while it is a positive one
in other environments. (strtod("NAN", NULL) did return a positive NAN
though.) In most tests where we test that NANs are preserved, the exact
sign doesn't matter much as long as it matches, but for some tests, it
does matter. Change to using variables with known representation
in the tests where it matters.

Also add a LL suffix to make a tricky constant more correct.
In MSVC, the variable initialization
    long long i = -2147483648;
actually sets the variable to the positive value 2147483648 (both
in MSVC 2019 and 2022).
2022-10-14 14:33:57 +03:00
Martin Storsjö 9be66e9d22 crt-test: Fix the macros for avoiding compiler optimizations
The macros F() and L() etc were meant for providing constants for
the tests, without the compiler seeing them as constants, to
avoid optimizing out the entire functions that are to be tested.

However in practice, the compiler is also able to optimize out
e.g. a call to strtol("42", NULL, 0), so this didn't have the
intended effect.

Instead make the macros wrap through a no-op function, but which
is indirected via a global function pointer. The compiler can't
reason about global variables keeping their values (they could be
modified by e.g. a constructor in a different object file, before
running the main function). This avoids needing to pass all values
through strtod/strtol etc.

Change some constants like 123456789012345, which are seen by
the compiler as integer constants (which aren't exactly representable
in floating point form) into 123456789012345.0, which the compiler
won't complain about being inexact.
2022-10-14 14:33:51 +03:00
Martin Storsjö c076463631 readme: Prefer the -Wl,--pdb= and -Wl,--pdb=<filename> forms for PDB output
Using two dashes for this option name is more idiomatic for
long GNU options (using only one dash only is supported for options
where it doesn't conflict with single-character options; it does
work for "-pdb" but "--pdb" is more correct).

Stop documenting the separate argument form "-Wl,--pdb,<filename>".
If noone would use this form, one could consider switching the tools
over to interpret "-Wl,--pdb" as a boolean option (optional_argument
instead of required_argument in getopt_long) as a less awkward syntax
for "-Wl,--pdb=".
2022-10-12 15:28:45 +03:00
Martin Storsjö 22b1ff128b Consistently use the --pdb= form in scripts
Stop using the separate argument form (-Wl,-pdb,<filename>), and
prefer using the option form with an implicit output name.
2022-10-12 15:28:45 +03:00
Martin Storsjö 21d1cf6f18 readme: Remove the note about PDB output not working for armv7/aarch64
PDB output now does work for armv7 (earlier it used to error out),
and the PDB output does work properly on aarch64.

(The recently added tests in run-lldb-tests.sh passes successfully on
aarch64. Debugging on armv7 with LLDB has got a few known issues though,
both with DWARF and with PDB, while WinDbg and/or Visual Studio might
work better.)
2022-10-12 15:27:48 +03:00
Martin Storsjö a291ea689e build-cross-tools: Add options for disabling various parts of the build 2022-10-05 00:09:05 +03:00