First, we need check against the host platform, not the build platform.
That's simple enough.
Second, we move away from exahustive finite case analysis (i.e.
exhaustively listing all platforms the package builds on). That only
work in a closed-world setting, where we know all platforms we might
build one. But with cross compilation, we may be building for arbitrary
platforms, So we need fancier filters. This is the closed world to open
world change.
The solution is instead of having a list of systems (strings in the form
"foo-bar"), we have a list of of systems or "patterns", i.e. attributes
that partially match the output of the parsers in `lib.systems.parse`.
The "check meta" logic treats the systems strings as an exact whitelist
just as before, but treats the patterns as a fuzzy whitelist,
intersecting the actual `hostPlatform` with the pattern and then
checking for equality. (This is done using `matchAttrs`).
The default convenience lists for `meta.platforms` are now changed to be
lists of patterns (usually a single pattern) in
`lib/systems/for-meta.nix` for maximum flexibility under this new
system.
Fixes#30902
This is a temporary workaround to make `nix-env -qa` and `nix search` ignore
broken packages as they they did before this patchset.
This patch should be reverted after `nix` gets a proper fix for this.
See NixOS/nix#1771.
This option makes `meta.evaluate` into a close approximation of the result of
evaluating `.outPath` by checking all the dependencies recursively at a cost of
2x slowdown. Note that actually evaluating `.outPath` costs some
5x-7x more because `.outPath` also computes all the hashes.
I hope this will be a temporary measure. If there is consensus around
issue #33599, then we can follow an explicit `dontCheck`, but default to
not checking during cross builds when none is given.
This pushes check-meta evaluation to derivation evaluation step, leaving all other
attributes accessible.
Before this commit:
> $ HOME=/homeless-shelter NIX_PATH=nixpkgs=$(pwd) nix-instantiate --eval --strict ./default.nix -A xen --argstr system aarch64-linux
> Package ‘xen-4.5.5’ in pkgs/applications/virtualization/xen/generic.nix:226 is not supported on ‘aarch64-linux’, refusing to evaluate.
as expected
> $ HOME=/homeless-shelter NIX_PATH=nixpkgs=$(pwd) nix-instantiate --eval --strict ./default.nix -A xen.name --argstr system aarch64-linux
> Package ‘xen-4.5.5’ in pkgs/applications/virtualization/xen/generic.nix:226 is not supported on ‘aarch64-linux’, refusing to evaluate.
> $ HOME=/homeless-shelter NIX_PATH=nixpkgs=$(pwd) nix-instantiate --eval --strict ./default.nix -A xen.meta.description --argstr system aarch64-linux
> Package ‘xen-4.5.5’ in pkgs/applications/virtualization/xen/generic.nix:226 is not supported on ‘aarch64-linux’, refusing to evaluate.
which is unfortunate since its impossible to use packages in autogenerated
documentation on all platforms.
After this commit:
> $ HOME=/homeless-shelter NIX_PATH=nixpkgs=$(pwd) nix-instantiate --eval --strict ./default.nix -A xen --argstr system aarch64-linux
still fails
> $ HOME=/homeless-shelter NIX_PATH=nixpkgs=$(pwd) nix-instantiate --eval --strict ./default.nix -A xen.name --argstr system aarch64-linux
> "xen-4.5.5"
> $ HOME=/homeless-shelter NIX_PATH=nixpkgs=$(pwd) nix-instantiate --eval --strict ./default.nix -A xen.meta.description --argstr system aarch64-linux
> "Xen hypervisor and related components (vanilla)"
- All deps go on the PATH
- CC and Bintools wrappers with their host != depender's host still get their
setup hooks run.
- Environment hooks get applied to all packages
This isn't so elegent, but eases the transition on a very significant
PR.
We now have the information to properly determine the role the
cc-wrapper dependency has, by taking advantage of `offset`. No longer
use the soon-to-be-deprecated crossConfig environment variable, the
temp hack used before this change.
4 far-reaching changes: Smaller PATH, New vars, different propagation
logic, and different hook logic
Smaller PATH
------------
`buildInputs` no longer go on the PATH at build time, as they cannot be
run when cross compiling and we don't want to special case. Simply make
a `nativeBuildInput` too if one needs them on the PATH. Fixes#21191.
Many new depedendency variables
-------------------------------
See the stdenv chapter of the nixpkgs manual. I pulled out the existing
documentation of dependency specification into a new section, and added
language for these two (and their propagated equivalents) along side
the others'.
More complex propagation logic
------------------------------
Before a propagated*XXX*Input always acted as if it was specified
directly as a *XXX*Input downstream. That's simple enough, but violates
the intended roles of each sort of dep, which has functional and not
just stylistic consequences.
The new algorithm is detailed in the manual, and ensures everything
ends up in the right place. I tried to give both an informal and formal
description, but I suspect in practice it will not make much sense
until one tries cross compiling, after which it will immediately make
sense as the only sane option.
Simplified hook logic
---------------------
Rather than `envHook` and `crossEnvHook`, whose behavior differs
depending on whether we are cross compiling or not, there is now one
hook per sort (or rather non-propagated and propagated pair of sorts)
of dependency. These new hooks have the same meaning regardless of
cross compilation. See the setup hook section of stdenv chapter of the
Nixpkgs manual for more details.
- tracing seems annoying enough
- we get errors for all packages instead of aborting on the first one
- easier to differentiate from unwanted packages (broken, unfree, etc.)
This continues #23374, which always kept around both attributes, by
always including both propagated files: `propgated-native-build-inputs`
and `propagated-build-inputs`. `nativePkgs` and `crossPkgs` are still
defined as before, however, so this change should only barely
observable.
This is an incremental step to fully keeping the dependencies separate
in all cases.
I find the separation of concerns, accumulating, then processing, easier
to follow. Also, with my yet-to-be-merged cross work, the accumulation
part will become more complex.
This requires some small changes in the stdenv, then working around the
weird choice LLVM made to hardcode @rpath in its install name, and then
lets us remove a ton of annoying workaround hacks in many of our Go
packages. With any luck this will mean less hackery going forward.
Why 6? It seems a decently high number, giving us room for more degrees
of debugging before the `set -x` sledgehammer without incurring a
mass-rebuild.
Environment variable filter in substituteAll was not precise and produced
undefined and invalid variable names. Vladimír Čunát tried to fix that in [1],
but `env -0` did not work during Darwin bootstrap, so [2] reverted this change
and replaced an error due to invalid variables with a warning. Recently in #28057
John Ericson added `set -u` to `setup.sh` and undefined variables made the setup
fail during e.g. `nix-build -A gnat` with `setup: line 519: !varName: unbound
variable`.
[1] 62fc8859c1
[2] 81df035429
This becomes necessary if more wrappers besides cc-wrapper start
supporting hardening flags. Also good to make the warning into an
error.
Also ensure interface is being used right: Not as a string, not just in
bash.
Older bash version, like those in the bootstrap tools and on macOS,
currently confuse variables defined as an empty array with undefined
variables. `${foo+"${foo[@]}"}` will prevent `set -u` problems with
empty arrays and older without making a single '' in the empty case.
Care is taken to `set +u` when running hooks so as to not break existing
packages.
