3
0
Fork 0
forked from mirrors/nixpkgs
Commit graph

256 commits

Author SHA1 Message Date
aszlig b25b6e0c75
stdenv: Improve ELF detection for isELF
The isELF function only checks whether ELF is contained within the first
4 bytes of the file, which is a bit fuzzy and will also return
successful if it's a text file starting with ELF, for example:

  ELF headers
  -----------

  Some text here about ELF headers...

So instead, we're now doing a precise match on \x7fELF.

Signed-off-by: aszlig <aszlig@nix.build>
Acked-by: @Ericson2314
Closes: https://github.com/NixOS/nixpkgs/pull/47244
2018-09-25 06:55:18 +02:00
Matthew Bauer ba5717a6f5 stdenv: fix HOST_PATH change
a4630c65ca was incorrect in assuming $SHELL would be a path to the
bash derivation. In fact $SHELL will be a path to the bash executable.

Unfortunately this did not fix the original issue. So instead, we just
have to reuse initialPath can be added like PATH is.

Sorry for the inconvenience! I hadn’t thought through the effects of
the last commit.

/cc @copumpkin @ericson2314
2018-09-17 14:18:06 -05:00
Matthew Bauer a4630c65ca stdenv: add shell to HOST_PATH for backwards compatibility
To avoid breaking things, we need to make sure SHELL goes into
HOST_PATH. This reflects my changes to patch-shebangs to make it cross
compilation ready. When a script is patched from the Nix store it now
looks to HOST_PATH to get the targeted machine’s executables.
Unfortunately, this only works in native builds.
2018-09-16 15:58:21 -05:00
Jan Malakhovski c63ca0a431 stdenv: implement enableParallelChecking option
Works similarly to `enableParallelBuilding`, but is set by default when
`enableParallelBuilding` is set. In my experience most packages that build
fine in parallel also check fine in parallel.
2018-09-05 01:14:39 +00:00
James Deikun bd63de114e stdenv/build-support: support .tbz and .txz tarballs 2018-07-30 15:30:16 -04:00
Matthew Bauer cd91d1ad9f setup.sh: add HOST_PATH
HOST_PATH contains the path of the host package. This will include the
packages listed in buildInputs & depsHostHost. Use this to find
runtime commands that the host needs.

For instance to find the runtime version of perl,

$ PATH="$HOST_PATH" command -v perl
/nix/store/...-perl-5.28.0-aarch64-unknown-linux-android/bin/perl

This path should not be executed directly (it will break for cross
compilation). Only use it to find the location of executables that
will be run by your host system. Your build tools will, as always, be
available on the default PATH.
2018-07-22 23:14:49 -04:00
Matthew Bauer 37273afd27
Merge pull request #43224 from volth/patch-179
[staging] substitute(): --subst-var was silently coercing to "" if the variable does not exist.
2018-07-21 15:48:14 -04:00
aszlig 739c835515
stdenv-setup: Remove superfluous check for /bin/sh
The line was essentially checking whether /bin/sh exists and is
executable and if that's the case, the isScript function returns
successfully.

When asking the author of this line on IRC it seems that even they can't
remember or imagine what this was supposed to be.

In summary: Whenever /bin/sh doesn't exist during a build, *any* file
given to isScript is reported as being a script even if it isn't.

This is kinda counter-intuitive and not something what somebody would
expect from a function called "isScript".

Signed-off-by: aszlig <aszlig@nix.build>
Cc: @edolstra
2018-07-09 20:43:34 +02:00
volth 0de0ce5893 substitute(): --subst-var was silently coercing to "" if the variable does not exist. 2018-07-08 16:32:03 +00:00
John Ericson 5e17335bd7 Merge remote-tracking branch 'upstream/staging' into strictDeps 2018-05-14 23:33:03 -04:00
John Ericson 330ca731e8 treewide: Get rid of all uses of crossConfig
The hack of using `crossConfig` to enforce stricter handling of
dependencies is replaced with a dedicated `strictDeps` for that purpose.
(Experience has shown that my punning was a terrible idea that made more
difficult and embarrising to teach teach.)

Now that is is clear, a few packages now use `strictDeps`, to fix
various bugs:

 - bintools-wrapper and cc-wrapper
2018-05-14 23:30:37 -04:00
John Ericson 4f7cdd35d5
Merge pull request #40139 from obsidiansystems/modular-setup-hooks
treewide: Modular setup hooks
2018-05-07 15:32:10 -04:00
John Ericson 34a3233a2e stdenv: Support concatenating setup hooks from multiple parts. 2018-05-07 14:43:22 -04:00
Jan Malakhovski 50af975d85 stdenv: implement checkTarget and installCheckTarget autodetection 2018-04-25 19:53:25 +00:00
Jan Malakhovski e9e06888ed stdenv: generic/setup.sh: cleanup installPhase 2018-04-25 19:53:25 +00:00
Jan Malakhovski ad98c36f1b stdenv: generic/setup.sh: simplify buildPhase Makefile check 2018-04-25 19:53:24 +00:00
Shea Levy b24ce2ae63
Handle sourceRoots with leading dashes 2018-02-15 11:50:24 -05:00
Shea Levy f83b6e1130
unpackPhase: Handle sources starting with a hyphen 2018-01-24 21:58:57 -08:00
adisbladis c2316114bc
stdenv: Kill off ensureDir 2018-01-09 11:14:48 +08:00
John Ericson 469fd89832 stdenv-setup: Ease the transition with native builds
- All deps go on the PATH

 - CC and Bintools wrappers with their host != depender's host still get their
   setup hooks run.

 - Environment hooks get applied to all packages

This isn't so elegent, but eases the transition on a very significant
PR.
2017-12-30 22:04:23 -05:00
John Ericson a036473a0a {bintools,cc}-wrapper: Fix setup hook to respect the role of the cc-compiler
We now have the information to properly determine the role the
cc-wrapper dependency has, by taking advantage of `offset`. No longer
use the soon-to-be-deprecated crossConfig environment variable, the
temp hack used before this change.
2017-12-30 22:04:21 -05:00
John Ericson 7f3ca3e21a stdenv: Fix handling of dependencies and hooks
4 far-reaching changes: Smaller PATH, New vars, different propagation
logic, and different hook logic

Smaller PATH
------------

`buildInputs` no longer go on the PATH at build time, as they cannot be
run when cross compiling and we don't want to special case. Simply make
a `nativeBuildInput` too if one needs them on the PATH. Fixes #21191.

Many new depedendency variables
-------------------------------

See the stdenv chapter of the nixpkgs manual. I pulled out the existing
documentation of dependency specification into a new section, and added
language for these two (and their propagated equivalents) along side
the others'.

More complex propagation logic
------------------------------

Before a propagated*XXX*Input always acted as if it was specified
directly as a *XXX*Input downstream. That's simple enough, but violates
the intended roles of each sort of dep, which has functional and not
just stylistic consequences.

The new algorithm is detailed in the manual, and ensures everything
ends up in the right place. I tried to give both an informal and formal
description, but I suspect in practice it will not make much sense
until one tries cross compiling, after which it will immediately make
sense as the only sane option.

Simplified hook logic
---------------------

Rather than `envHook` and `crossEnvHook`, whose behavior differs
depending on whether we are cross compiling or not, there is now one
hook per sort (or rather non-propagated and propagated pair of sorts)
of dependency. These new hooks have the same meaning regardless of
cross compilation. See the setup hook section of stdenv chapter of the
Nixpkgs manual for more details.
2017-12-30 22:04:21 -05:00
John Ericson da19c34d0f stdenv setup: Always use both propagated files
This continues #23374, which always kept around both attributes, by
always including both propagated files: `propgated-native-build-inputs`
and `propagated-build-inputs`. `nativePkgs` and `crossPkgs` are still
defined as before, however, so this change should only barely
observable.

This is an incremental step to fully keeping the dependencies separate
in all cases.
2017-11-21 10:44:44 -05:00
John Ericson 6a5cda5131 stdenv setup: Run setup hooks and other processing after accumulating deps
I find the separation of concerns, accumulating, then processing, easier
to follow. Also, with my yet-to-be-merged cross work, the accumulation
part will become more complex.
2017-11-15 18:51:06 -05:00
John Ericson f6fcb9bc0d stdenv: Turn on set -x if NIX_DEBUG >= 6
Why 6? It seems a decently high number, giving us room for more degrees
of debugging before the `set -x` sledgehammer without incurring a
mass-rebuild.
2017-09-26 11:24:19 -04:00
John Ericson 127a5f3357 treewide: Use (( "${NIX_DEBUG:-0}" >= 1) )) consistently 2017-09-26 11:24:19 -04:00
Eelco Dolstra 0061fae2e6 genericBuild: Communicate the current build phase to Nix
This allows the progress bar to show e.g.

  [1/9/59 built] building bison-3.0.4 (configurePhase): checking for strdup... yes
2017-09-07 22:24:27 +02:00
Eelco Dolstra 6b3cef2246 Remove tracePhases
This has not been used in a long time.
2017-09-07 22:15:37 +02:00
Orivej Desh f4044c1ccc stdenv-setup: list environment variables with awk 2017-09-03 12:57:08 +00:00
Orivej Desh a09d9e7cd4 stdenv-setup: fix substituteAll with set -eu
Environment variable filter in substituteAll was not precise and produced
undefined and invalid variable names.  Vladimír Čunát tried to fix that in [1],
but `env -0` did not work during Darwin bootstrap, so [2] reverted this change
and replaced an error due to invalid variables with a warning.  Recently in #28057
John Ericson added `set -u` to `setup.sh` and undefined variables made the setup
fail during e.g. `nix-build -A gnat` with `setup: line 519: !varName: unbound
variable`.

[1] 62fc8859c1
[2] 81df035429
2017-09-03 12:57:08 +00:00
John Ericson 81194eef45 stdenv-setup: Use set -u as much as possible
Older bash version, like those in the bootstrap tools and on macOS,
currently confuse variables defined as an empty array with undefined
variables. `${foo+"${foo[@]}"}` will prevent `set -u` problems with
empty arrays and older without making a single '' in the empty case.

Care is taken to `set +u` when running hooks so as to not break existing
packages.
2017-08-23 15:57:56 -04:00
Tuomas Tynkkynen 7320fa9d45 Revert "stdenvs: Distinguish between extraBuildInputs and extraNativeBuildInputs"
This reverts commit eeabf85780.

This change suddenly makes tons of stdenv internals visible in
nativeBuildInputs of every derivation, which doesn't seem desirable.
E.g:

````
nix-repl> hello.nativeBuildInputs
[ «derivation /nix/store/bcfkyf6bhssxd2vzwgzmsbn7b5b9rpxc-patchelf-0.9.drv»
  «derivation /nix/store/4wnshnz9wwanpfzcrdd76rri7pyqn9sk-paxctl-0.9.drv»
  << snip 10+ lines >>
  «derivation /nix/store/d35pgh1lcg5nm0x28d899pxj30b8c9b2-gcc-wrapper-6.4.0.drv»
]
````
2017-08-18 13:21:56 +03:00
John Ericson eeabf85780 stdenvs: Distinguish between extraBuildInputs and extraNativeBuildInputs
Additionally, instead of pulling them from `setup.sh`, route them via
Nix. This gets us one step closer to making stdenv be a plain attribute
set instead of a derivation.
2017-08-15 18:24:54 -04:00
Linus Heckemann 17753fa005 stdenv: fix typo in setup.sh 2017-08-09 17:33:02 +01:00
John Ericson 42f35503b5 cc-wrapper: Make hygienic
See the added comments for what exactly has been done.
2017-08-07 03:05:50 -04:00
John Ericson 820e4021d3 stdenv-setup: Remove any declare -g
This is invalid before bash-4.2, affecting bash used impurely in
nix-shell on MacOS.
2017-07-26 09:11:18 -04:00
John Ericson ea7d13cf1a stdenv-setup and misc hooks: Work with bash-3.4 for MacOS nix-shell
This is a temporary measure until this impurity is removed from Nix.
2017-07-26 09:08:01 -04:00
John Ericson f6f40e3fe5 stdenv-setup and misc pkgs: Revert to space-deliminated propagated-* files
We cannot switch to line-delimited yet, because certain Nix commands do
not read in the entire file, but just the first line.
2017-07-26 09:07:55 -04:00
John Ericson 34c0ba498c stdenv-setup: Add quotes that don't do anything for consistency.
@vcunat and others rightly point out that it's easier to quote always,
than learn Bash's idiosyncrasies enough to know when it doesn't make a
difference.

This reverts commit 2743078f66, which
removes quotes that don't do anything, and then goes further adding
even more quotes.
2017-07-25 14:36:00 -04:00
Eelco Dolstra 6669a3b477
stdenv: Remove log nesting
Nix/Hydra no longer support pretty printing of logs, so this is no
longer useful.
2017-07-24 14:30:46 +02:00
John Ericson 30a1420414 stdenv-setup: Pull out and explain 3-part printing of commands
@Dezgeg made the good point that the reasons for doing this were not at
all intuitive.
2017-07-13 16:31:39 -04:00
John Ericson 2743078f66 stdenv-setup: Remove useless quotes
foo=$1 surprisingly doesn't need quotes in Bash. Word splits are only
syntactic in string variable (not array var!) assignments.
2017-07-13 14:59:53 -04:00
John Ericson 273a4c1c78 stdenv-setup: Combine [[ .. ]] && [[ .. ]] into one [[ .. && .. ]]
Also remove useless quotes on same line
2017-07-13 14:57:16 -04:00
John Ericson 5d693c84d2 stdenv-setup: Clean up 'substitute()' for style and error handling
It now blows up on null byte in file (rather than silently truncating),
and invalid arguments (rather than silently skipping).
2017-07-12 17:47:20 -04:00
John Ericson 5d4efb2c81 stdenv-setup: Misc improvements as directed by ShellCheck
I took some liberties with the flags-echoing code to make it more
concise and correct. Also, a few warnings in findInputs and friends I
skipped because I am going to rewrite those anyways.

Thanks @grahamc for telling me about this great linter!
2017-07-12 15:31:10 -04:00
John Ericson 8d76effc17 stdenv-setup: Make the package accumulators associative arrays instead of strings
This is generally cleaner: less eval, less worrying about separators,
and probably also faster. I got the idea from that python wrapper
script.
2017-07-12 15:30:56 -04:00
John Ericson 3cb745d5a6 stdenv: Store one package per line in nix-support/propagated-*
This makes those files a bit easier to read. Also, for what it's worth,
it brings us one baby step closer to handling spaces in store paths.

Also, I optimized handling of many transitive deps with read. Probably,
not very beneficial, but nice to enforce the pkg-per-line structure.
Doing so let me find much dubious code and fix it.

Two misc notes:

 - `propagated-user-env-packages` also needed to be adjusted as
   sometimes it is copied to/from the propagated input files.

 - `local fd` should ensure that file descriptors aren't clobbered
   during recursion.
2017-07-10 13:32:13 -04:00
John Ericson e57a220f81 stdenv, swift: Use local fd in is* bash functions for hygiene 2017-07-07 11:40:07 -04:00
John Ericson a14cf06182 stdenv: Harden hook runners
Instead of eval, use a "nameref" to get the name of the array and
iterate with that. Also, make the for-loop parameter a local variable,
too.
2017-07-07 11:35:09 -04:00
Eelco Dolstra 94d164dd7f
Add a setup hook for detecting $TMPDIR references in RPATHs and wrapper scripts 2017-05-04 20:23:57 +02:00