POSIX sh (and `bash`) impose a restriction on environment variable name
format and disallow hypheps in the names. Normally it's not a problem
as nothing usually tries to refer nyphenated names.
One exception is `nix develop` (https://github.com/NixOS/nix/issues/6848):
$ nix develop -f. gcc -L
gcc-wrapper> ...-get-env.sh: line 70: expand-response-params: bad substitution
Note that bash usually uses explicitly created `expandResponseParams`
variant of the same variable.
To work the problem around let's avoid environment variable export and move
it to `passthru` for `cc` (used ina few places) and remove it completely for
`binutils` (does not seem to be used at all).
Noticed this bug when was trying to bootstrap m4 on darwin. That fixes
line 163: no such file or directory error
That does not solve all problems staging has on darwin.
This enables users to make use of clang's multi-platform/target support
without having to go through full cross system setup. This is especially useful
for generating bpf object files, I'm not even usre what would a no-userland
cross compile system tuple even look like to even try going that route.
Fixes#176128
Otherwise, these warnings are emitted:
command-line option '-Wformat=1' is valid for C/C++/ObjC/ObjC++ but not for Fortran
command-line option '-Wformat-security' is valid for C/C++/ObjC/ObjC++ but not for Fortran
'-Werror=' argument '-Werror=format-security' is not valid for Fortran
Fixes part of #27218
Take a "nixSupport" argument that is an attrset of lists to append
to "$out/nix-support/${name}" where name is the name of the
attribute. This attrset is available from the passthru of the
wrapped compiler.
This is an alternative to imperatively issuing flags with
extraBuildCommands and makes it possible to append or filter
flags with an override.
In https://github.com/NixOS/nixpkgs/issues/158042 I noticed order
mismatch as a bootstrap build failure when building x86_64-linux
against glibc-2.35 in nixpkgs (bootstrap libs has glibc-2.27):
expand-response-params> ld: /nix/store/p4s4jf7aq6v6z9iazll1aiqwb34aqxq9-bootstrap-tools/lib/crt1.o: in function `_start':
expand-response-params> /build/glibc-2.27/csu/../sysdeps/x86_64/start.S:101: undefined reference to `__libc_csu_fini'
expand-response-params> ld: /build/glibc-2.27/csu/../sysdeps/x86_64/start.S:102: undefined reference to `__libc_csu_init'
expand-response-params> collect2: error: ld returned 1 exit status
Here crt1.o from glibc-2.27 links against libc.so.6 from glibc-2.35.
This happens because ordering of `-L` (influences `libc.so` lookup) and
`-B` (influences `crt1.o` lookup) flags differs:
expand-response-params> -B/...-bootstrap-tools/lib
expand-response-params> -B/...-glibc-2.35/lib/
...
expand-response-params> -L/...-glibc-2.35/lib
expand-response-params> -L/...-bootstrap-tools/lib
The change makes consistent ordering of `-L`/`-B` and allows getting to
stage4 for `glibc-2.35` target.
So far we've ignored response files in arguments, and did not
check linkType against expanded parameters. This means if
we have `-static` in a @reponse-file, linkType will not be
set to `-static` as we never check against the expanded arguments
from response files.
fixes e.g.:
pkgsMusl.libfsm
pkgsMusl.libiscsi
pkgsMusl.nsjail
pkgsMusl.pv
match strings have whitespace on either side, which wasn't
matching leading/trailing arguments previously
fixes:
pkgsMusl.bulletml
pkgsMusl.proot
pkgsMusl.python3
Debian explains this issue well in the dpkg-buildflags manpage:
-fPIE
Can be linked into any program, but not a shared library (recommended).
-fPIC
Can be linked into any program and shared library.
On projects that build both programs and shared libraries you might need to
make sure that when building the shared libraries -fPIC is always passed last
(so that it overrides any previous -PIE) to compilation flags such as CFLAGS.
(from https://manpages.debian.org/bullseye/dpkg-dev/dpkg-buildflags.1.en.html#hardening)
In newer versions of mingw, programs compiled with FORTIFY_SOURCE need
to link to libssp or they will have link-time errors.
gmp has been broken since @pstn updated mingw-64 in c60a0b0447
GPRbuild is a multi language build system developed by AdaCore which
is mostly used for build Ada-related projects using GNAT.
Since GPRbuild is used to build itself and its dependency library
XML/Ada we first build a bootstrap version of it using the provided
bash build script bootstrap.sh as the gprbuild-boot derivation.
gprbuild-boot is then used to build xmlada and the proper gprbuild
derivation.
GPRbuild has its own search path mechanism via GPR_PROJECT_PATH which
we address via a setupHook. It currently works quite similar to the
pkg-config one: It accumulates all inputs into GPR_PROJECT_PATH,
GPR_PROJECT_PATH_FOR_BUILD etc. However this is quite limited at the
moment as we don't have a gprbuild wrapper yet which understands the
_FOR_BUILD suffix. However, we'll need to address this in the future
as it is currently basically impossible to test since the distinction
only affects cross-compilation, but it is not possible to build a GNAT
cross-compiler in nixpkgs at the moment (I'm working on changing that,
however).
Another issue we had to solve was GPRbuild not finding the right GNAT
via its gprconfig tool: GPRbuild has a knowledge base with compiler
definitions which run some checks and collect info about binaries
which are in PATH. In the end the first compiler in PATH that supports
the desired language is selected.
We want GPRbuild to discover our wrapped GNAT since the unwrapped one
is incapable of producing working binaries since it won't find the
crt*.o objects distributed with libc. GPRbuild however needs to find
the Ada runtime distributed with GNAT which is not part of the wrapper
derivation, so it will skip the wrapper and select the unwrapped GNAT.
Symlinking the unwrapped's lib directory into the wrapper fixes this
problem, but breaks linking in some cases (e. g. when linking against
OMP from gcc, the runtime variant will shadow the problem dynamic lib
from buildInputs). Additionally it uses gnatls as an indicator it has
found GNAT which is not part of the wrapper.
The solution we opted to adopt here is to install a custom compiler
description into gprbuild's knowledge base which properly detects the
nixpkgs GNAT wrapper: It uses gnatmake to detect GNAT instead of
gnatls and discovers the runtime via a symlink we add to
`$out/nix-support`. This additional definition is enough to properly
detect GNAT, since the plain wrapped gcc detection works out of the
box. It may, however, be necessary to add special definitions for
other languages in the future where gprbuild also needs to discover
the runtime.
One future improvement would be to install libgpr into a separate
output or split it into a separate derivation (which would require to
link gprbuild statically always since otherwise we end up with a
cyclical dependency).
This will begin the process of breaking up the `useLLVM` monolith. That
is good in general, but I hope will be good for NetBSD and Darwin in
particular.
Co-authored-by: sterni <sternenseemann@systemli.org>
This PR adds a new aarch64 android toolchain, which leverages the
existing crossSystem infrastructure and LLVM builders to generate a
working toolchain with minimal prebuilt components.
The only thing that is prebuilt is the bionic libc. This is because it
is practically impossible to compile bionic outside of an AOSP tree. I
tried and failed, braver souls may prevail. For now I just grab the
relevant binaries from https://android.googlesource.com/.
I also grab the msm kernel sources from there to generate headers. I've
included a minor patch to the existing kernel-headers derivation in
order to expose an internal function.
Everything else, from binutils up, is using stock code. Many thanks to
@Ericson2314 for his help on this, and for building such a powerful
system in the first place!
One motivation for this is to be able to build a toolchain which will
work on an aarch64 linux machine. To my knowledge, there is no existing
toolchain for an aarch64-linux builder and an aarch64-android target.
Also begin to start work on cross compilation, though that will have to
be finished later.
The patches are based on the first version of
https://reviews.llvm.org/D99484. It's very annoying to do the
back-porting but the review has uncovered nothing super major so I'm
fine sticking with what I've got.
Beyond making the outputs work, I also strove to re-sync the packages,
as they have been drifting pointlessly apart for some time.
----
Other misc notes, highly incomplete
- lvm-config-native and llvm-config are put in `dev` because they are
tools just for build time.
- Clang no longer has an lld dep. That was introduced in
db29857eb3, but if clang needs help
finding lld when it is used we should just pass it flags / put in the
resource dir. Providing it at build time increases critical path
length for no good reason.
----
A note on `nativeCC`:
`stdenv` takes tools from the previous stage, so:
1. `pkgsBuildBuild`: `(?1, x, x)`
2. `pkgsBuildBuild.stdenv.cc`: `(?0, ?1, x)`
while:
1. `pkgsBuildBuild`: `(?1, x, x)`
2. `pkgsBuildBuild.targetPackages`: `(x, x, ?2)`
3. `pkgsBuildBuild.targetPackages.stdenv.cc`: `(?1, x, x)`
In a typical build environment the toolchain will use the value of the
MACOSX_DEPLOYMENT_TARGET environment variable to determine the version
of macOS to support. When cross compiling there are two distinct
toolchains, but they will look at this single environment variable. To
avoid contamination, we always set the equivalent command line flag
which effectively disables the toolchain's internal handling.
Prior to this change, the MACOSX_DEPLOYMENT_TARGET variable was
ignored, and the toolchains always used the Nix platform
definition (`darwinMinVersion`) unless overridden with command line
arguments.
This change restores support for MACOSX_DEPLOYMENT_TARGET, and adds
nix-specific MACOSX_DEPLOYMENT_TARGET_FOR_BUILD and
MACOSX_DEPLOYMENT_TARGET_FOR_TARGET for cross compilation.
The check for including the C++ standard library headers was nested inside the
check for linking with the C++ standard library. As a result, the `-nostdlib`
flag incorrectly implied `-nostdinc++`, which made it virtually impossible to
partially link C++ objects.
Fixes build failures with clang:
clang-7: error: unknown argument: '-fPIC -target'
clang-7: error: no such file or directory: '@<(printf %qn -O2'
clang-7: error: no such file or directory: 'x86_64-apple-darwin'
Introduced by 60c5cf9cea in #112449
