This commit adds a test for the newly added support for
__structuredAttrs in autoPatchelf(hook). It copied a reasonably
small-closure binary derivation that makes use of autoPatchelf, stripped
it down for the purpose of the test, and check that autoPatchelf
correctly set the interpreter and runpath whether __structuredAttrs is
set to true or not.
Due to the check soon depending on the base branch (see `--base`),
the CI check can't reasonably share all code with a local check.
We can still make a script to run it locally, just not sharing all code.
This derivation verifies that all jobset attributes can be evaluated
under tryEval without producing any non-catchable errors or causing
infinite recursion.
Previously, not passing `--base` would enforce the most strict checks.
While there's currently no actual violation of these stricter checks,
this does not match the previous behavior.
This won't matter once CI passes `--base`, the code handling the
optionality can be removed then.
This implements the option for a gradual migration to stricter checks.
For now this is only done for the check against empty non-auto-called
callPackage arguments, but in the future this can be used to ensure all
new packages make use of `pkgs/by-name`.
This is implemented by adding a `--base <BASE_NIXPKGS>` flag, which then
compares the base nixpkgs against the main nixpkgs version, making sure
that there are no regressions.
The `--version` flag is removed. While it was implemented, it was never
used in CI, so this is fine.
This prepares the code base for the removal of the `--version` flag, to
be replaced with a flag that can specify a base version to compare the
main Nixpkgs against, in order to have gradual transitions to stricter
checks.
This refactoring does:
- Introduce the `version` module that can house the logic to increase
strictness, with a `version::Nixpkgs` struct that contains the
strictness conformity of a single Nixpkgs version
- Make the check return `version::Nixpkgs`
- Handle the behavior of the still-existing `--version` flag with `version::Nixpkgs`
- Introduce an intermediate `process` function to handle the top-level
logic, especially useful in the next commit
Convenience function to run another validation over a successful validation result.
This will be usable in more locations in future commits, making the code
nicer.
This makes it such that these two errors can both be thrown for a single
package:
- The attribute value not being a derivation
- The attribute not being a proper callPackage
The tests had to be adjusted to only throw the error they were testing
for
cudaPackages.cuda_compat: ignore missing libs provided at runtime
cudaPackages.gpus: Jetson should never build by default
cudaPackages.flags: don't build Jetson capabilities by default
cudaPackages: re-introduce filter for pre-existing CUDA redist packages in overrides
cudaPackages: only recurseIntoAttrs for the latest of each major version
cudaPackages.nvccCompatabilities: use GCC 10 through CUDA 11.5 to avoid a GLIBC incompatability
cudaPackages.cutensor: acquire libcublas through cudatoolkit prior to 11.4
cudaPackages.cuda_compat: mark as broken on aarch64-linux if not targeting Jetson
cudaPackages.cutensor_1_4: fix build
cudaPackages: adjust use of autoPatchelfIgnoreMissingDeps
cudaPackages.cuda_nvprof: remove unecessary override to add addOpenGLRunpath
cudaPackages: use getExe' to avoid patchelf warning about missing meta.mainProgram
cudaPackages: fix evaluation with Nix 2.3
cudaPackages: fix platform detection for Jetson/non-Jetson aarch64-linux
python3Packages.tensorrt: mark as broken if required packages are missing
Note: evaluating the name of the derivation will fail if tensorrt is not present,
which is why we wrap the value in `lib.optionalString`.
cudaPackages.flags.getNixSystem: add guard based on jetsonTargets
cudaPackages.cudnn: use explicit path to patchelf
cudaPackages.tensorrt: use explicit path to patchelf
While using very old compilers is a fair usecase, it induces a maintenance churn as
we collect more and more LLVM versions for the LLVM maintainers.
Especially when we need to backport uniform changes to the whole tree,
furthermore, it consumes and waste CI resources.
- passing it as expression gives large error messages
which are not very readable
- this commits puts the file in nix-store
and patches the final program to have access to
the path to the file as env.
- We simply pass this file to nix-instantiate
We have several cross-compilation bugs that show up if
hostPlatform!=buildPlatform yet
hostPlatform.config==buildPlatform.config.
These bugs have appeared and disappeared as we've fiddled with the
definition of equality for platform objects. This commit adds a
clear-cut case where they are *not* equal and never will be, so we
can test it.
Currently the tool prints problems right as it is checking the code
without an intermediate error representation. However for various reasons
it would be beneficial to have an intermediate error type:
- It makes the code cleaner, having all errors in one place
- It allows printing the error in different ways, e.g. for a future
--json mode
This commit prepares for an incremental refactoring for an intermediate
error/problem representation. Most notable is that we want to be able to collect
multiple errors/problems and not just exit on the first one.
We introduce the type alias CheckResult and CheckError (later renamed to
NixpkgsProblem), where CheckError allows collecting multiple
CheckErrors using the utility function flatten_check_results (later
renamed to check_result::sequence)
The write_check_result function is only temporarily introduced to help
refactoring, it's removed again in later commits.
Implement new 'buildTeXEnv' to leverage multi-output packages and provide
__overrideTeXConfig/withPackages for modifying the configuration or adding
packages. The override mechanism is prefixed until stabilized.
Allows detecting whether attributes are overridden in all-packages.nix.
In a future commit we'll use this to detect empty arguments being set in
all-packages.nix and complain about that.
When specifying the `builder` attribute in `stdenv.mkDerivation`, this
will be effectively transformed into
builtins.derivation {
builder = stdenv.shell;
args = [ "-e" builder ];
}
This also means that `default-builder.sh` is never sourced and as a
result it's not guaranteed that `$NIX_ATTRS_SH_FILE` is set to a correct
location[1].
Also, we need to source `.attrs.sh` to source `$stdenv`. So, the
following is done now:
* If `$NIX_ATTRS_SH_FILE` points to a correct location, then use it.
Directly using `.attrs.sh` is problematic for `nix-shell(1)` usage
(see previous commit for more context), so prefer the environment
variable if possible.
* Otherwise, if `.attrs.sh` exists, then use it. See [1] for when this
can happen.
* If neither applies, it can be assumed that `__structuredAttrs` is
turned off and thus nothing needs to be done.
[1] It's possible that it doesn't exist at all - in case of Nix 2.3 or
it can point to a wrong location on older Nix versions with a bug in
`__structuredAttrs`.
This fixes a bug where `patchShebangs` crashes when trying to patch
files that contain only a shebang (e.g. `#!/bin/bash`) (and nothing
else) and do not end with a newline.
Such file can be produced using `printf "#!/bin/bash" > example` or
`echo -n "#!/bin/bash" > example`.
I don't understand why one would want to create such files, as they do
literally nothing, but the chromium tarball we are using started
shipping some 🫠
Full reproducer:
```nix
with import <nixpkgs> { };
stdenv.mkDerivation {
dontUnpack = true;
name = "patch-shebangs-no-trailing-newline-reproducer";
postPatch = ''
printf "#!/bin/bash" > reproducer
chmod +x reproducer
patchShebangs reproducer
'';
}
```
```
❯ nix-build reproducer.nix
this derivation will be built:
/nix/store/vmbshdkdk4a0bayw3wi21wvxyhzpcsy2-patch-shebangs-no-trailing-newline-reproducer.drv
building '/nix/store/vmbshdkdk4a0bayw3wi21wvxyhzpcsy2-patch-shebangs-no-trailing-newline-reproducer.drv'...
patching sources
patching script interpreter paths in reproducer
/nix/store/vr6wwdxkmyy44sg0gwxi10b8fc5zhwz0-stdenv-linux/setup: line 144: pop_var_context: head of shell_variables not a function context
error: builder for '/nix/store/vmbshdkdk4a0bayw3wi21wvxyhzpcsy2-patch-shebangs-no-trailing-newline-reproducer.drv' failed with exit code 1;
last 3 log lines:
> patching sources
> patching script interpreter paths in reproducer
> /nix/store/vr6wwdxkmyy44sg0gwxi10b8fc5zhwz0-stdenv-linux/setup: line 144: pop_var_context: head of shell_variables not a function context
For full logs, run 'nix log /nix/store/vmbshdkdk4a0bayw3wi21wvxyhzpcsy2-patch-shebangs-no-trailing-newline-reproducer.drv'.
```
We seem to have enough tests to run into this now:
error: creating symlink from '/private/tmp/nix-build-nixpkgs-check-by-name.drv-0/source/test-tmp/var/nix/gcroots/profiles' to '/private/tmp/nix-build-nixpkgs-check-by-name.drv-0/source/test-tmp/var/nix/profiles': File exists
On Darwin, /tmp is sometimes a symlink to /private/tmp, which couldn't
be handled before:
error: access to canonical path '/private/var/folders/xp/9_ry6h9x6l9gh_g32qspz0_40000gp/T/.tmpFbcNO0' is forbidden in restricted mode
This both fixes that and adds a test to make sure it can't happen again
This reverts commit d33a309f2c9f85c28cb897e623503f210057a77a.
commit caused ofborg-eval to fail with
```
$ nix-build pkgs/top-level/release.nix -A tests.cc-wrapper.gcc10Stdenv.aarch64-darwin
error: evaluation aborted with the following error message: 'cannot find attribute `tests.cc-wrapper.gcc10Stdenv''
```
on the 3 other platfomrs the attr is there
ignore `llvmPackages_latest`, it's an alias and it'll be a duplicate.
ignore `llvmPackages_git`, it's not supposed to be used in nixpkgs and
may be broken.
This introduces the `pkgs/by-name` directory as proposed by RFC 140.
Included are:
- The implementation to add packages defined in that directory to the
top-level package scope
- Contributer documentation on how to add packages to it
- A GitHub Actions workflow to check the structure of it on all PRs
most tests use debian-devscripts' hardening-check, so only work on
ELF systems and can only detect a limited subset of flags.
some extra tests actually execute fortify-protected programs and
should be slightly more universally applicable.
- Better filesystem case-sensitivity heuristic
We shouldn't assume that Linux is always case-sensitive.
- Don't include case-sensitive filename in tree
Was used for tests, but this broke channel updates because there's a
check to make sure there's no case-sensitive files!
https://hydra.nixos.org/build/233371356/nixlog/1
This commit does the following two things:
1. Changes the Haskell package used in the extraDependencies test from
releaser to conduit. The reason for this is that conduit is more
of a "core" package in Haskell, and it is more likely to always be
working. (If conduit is not compiling, then large chunks of Hackage
won't be working.)
2. Tighten up the GHCi test to actually use the package from
extraDependencies. It appears that GHCi can fail to import the
package from extraDependencies, and that doesn't cause GHCi to
return an error code, which means the test isn't actually testing
anything.
This new change makes it so that if the package from
extraDependencies is actually not included for some reason, then
this shellFor test should fail.
```
nix-repl> (pkgs.htop.overrideAttrs { pname = "hello-overriden"; }).pname
error:
… while evaluating a branch condition
at /nix/store/phn5cahwacv9wjgalygw62x8l4xbl6x3-source/lib/customisation.nix:86:7:
85| in
86| if builtins.isAttrs result then
| ^
87| result // {
… while calling the 'isAttrs' builtin
at /nix/store/phn5cahwacv9wjgalygw62x8l4xbl6x3-source/lib/customisation.nix:86:10:
85| in
86| if builtins.isAttrs result then
| ^
87| result // {
(stack trace truncated; use '--show-trace' to show the full trace)
error: attempt to call something which is not a function but a set
at /nix/store/phn5cahwacv9wjgalygw62x8l4xbl6x3-source/pkgs/stdenv/generic/make-derivation.nix:58:21:
57| f = self: super:
58| let x = f0 super;
| ^
59| in
```
This commit adds `pkgs.test.cross.sanity`, which is meant to be a
carefully curated list of builds/packages that tend to break when
refactoring our cross-compilation infrastructure.
It should strike a balance between being small enough to fit in
a single eval (i.e. not so large that hydra-eval-jobs is needed)
so we can ask @ofborg to check it, yet should have good examples
of things that often break. So, no buckshot `mapTestOnCross`
calls here.
Makes overrideAttrs usable in the same way that `override` can be used.
It allows the first argument of `overrideAttrs` to be either a function
or an attrset, instead of only a function:
hello.overrideAttrs (old: { postBuild = "echo hello"; })
hello.overrideAttrs { postBuild = "echo hello"; }
Previously only the first example was possible.
Co-authored-by: adisbladis <adisbladis@gmail.com>
Co-authored-by: matthewcroughan <matt@croughan.sh>
This change adds a flag to update shebang paths that point to the Nix
store. This is particularly useful when a cross-compiled package uses
same script at compile-time and run-time, but the interpreter must be
changed since hostPlatform != buildPlatform.
With a recent hackage update, turtle stopped compiling on ghc94. This
commit changes the tests.haskell.incremental test to use the temporary
package instead of turtle.
We shouldn't try compiling to aarch64-unknown-linux-gnu from
aarch64-unknown-linux-gnu, because that tends to confuse things and is
not representative of actual cross compilation usage.
This commit adds a derivation `gcc-stageCompare` to
`pkgs/test/stdenv/default.nix`.
It is important to always build this derivation whenever building
`stdenv`! Because we are using a Nix-driven bootstrap instead of
gcc's built-in `--enable-bootstrap`, the `gcc` derivation no longer
performs the post-self-compilation sanity check. You must build
this derivation in order to perform that sanity check.
The major benefit of this new approach is that the sanity check
(which involves a third compilation of gcc) can be performed
*concurrently* with all packages that depend on `stdenv`, rather
than serially. Since `stdenv` has very little derivation-level
parallelism it cannot take advantage of more than one or perhaps two
builders. If you have three or more builders this commit will
reduce the time-to-rebuild-stdenv by around 20% (one of three gcc
rebuilds is removed from the critical path, and stdenv's build time
is dominated by roughly 3*gcc + 1*binutils + 1*bison-test-suite).
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
Set the executable bit before running the check phase, so that the check
phase can run the script to test its behaviour.
This aligns with what `concatTextFile` is doing.
Also use explicit `if` statements so that we don't silently ignore
`chmod` failures.
