3
0
Fork 0
forked from mirrors/nixpkgs
Commit graph

69 commits

Author SHA1 Message Date
Alyssa Ross 76405e3077
compressFirmwareXz: fix with empty lib/firmware
Fixes: 8aa8e0ce7f ("nixos/udev: compress all firmware if supported")
2022-05-26 07:08:50 +00:00
Alyssa Ross 8aa8e0ce7f
nixos/udev: compress all firmware if supported
This should be a significant disk space saving for most NixOS
installations.  This method is a bit more complicated than doing it in
the postInstall for the firmware derivations, but this way it's
automatic, so each firmware package doesn't have to separately
implement its compression.

Currently, only xz compression is supported, but it's likely that
future versions of Linux will additionally support zstd, so I've
written the code in such a way that it would be very easy to implement
zstd compression for those kernels when they arrive, falling back to
xz for older (current) kernels.

I chose the highest possible level of compression (xz -9) because even
at this level, decompression time is negligible.  Here's how long it took
to decompress every firmware file my laptop uses:

	i915/kbl_dmc_ver1_04.bin                  	2ms
	regulatory.db                             	4ms
	regulatory.db.p7s                         	3ms
	iwlwifi-7265D-29.ucode                    	62ms
	9d71-GOOGLE-EVEMAX-0-tplg.bin             	22ms
	intel/dsp_fw_kbl.bin                      	65ms
	dsp_lib_dsm_core_spt_release.bin          	6ms
	intel/ibt-hw-37.8.10-fw-22.50.19.14.f.bseq	7ms

And since booting NixOS is a parallel process, it's unlikely (but
difficult to measure) that the time to user interaction was held up at
all by most of these.

Fixes (partially?) #148197
2022-05-13 14:36:34 +00:00
Janne Heß e5995b2235
makeInitrdNG: Strip more and remove output
This strips all elf files as far as possible and removes a lot of
unnecessary output. Also wrap in the binaries instead of relying on
$PATH.
2022-05-01 17:06:14 +02:00
Artturi 98ff3e401c
Merge pull request #165892 from tpwrules/fix-initrd-dirlinks
make-initrd: fix reproducibility problems
2022-04-21 20:51:35 +03:00
Janne Heß b9bf28fd70
nixos/stage-1-init: Pass all parameters to the builder
This is for compatiblity with the old builder to get compat with uboot
support and to prepend microcode updates to the initrd
2022-04-16 20:36:18 +01:00
Thomas Watson 7fd6cea253 make-initrd: fix reproducibility problems
cpio includes the number of directory hard links in archives it creates.
Some filesystems, like btrfs, do not count directory hard links the same
way as more common filesystems like ext4 or tmpfs, so archives built
when /tmp is on such a filesystem do not reproduce. This patch replaces
cpio with bsdtar, which does not have this issue. The specific
invocation is from this page:
https://reproducible-builds.org/docs/archives/
2022-04-14 19:06:30 -05:00
Will Fancher d193ef8a57 make-initrd-ng: init 2022-03-22 07:02:22 -04:00
Thomas Watson d8bb89b9ff make-initrd: fix reproducibility problems with hard links 2022-01-29 18:07:11 -05:00
Linus Heckemann b0fc6e8ff9 make-initrd: fix #132059 2021-08-08 07:53:59 +02:00
Ben Siraphob 02ee14b9d5 treewide: stdenvNoCC.lib -> lib 2021-01-27 13:01:51 +07:00
John Ericson 9c213398b3 lib: Clean up how linux and gcc config is specified
Second attempt of 8929989614589ee3acd070a6409b2b9700c92d65; see that
commit for details.

This reverts commit 0bc275e634.
2021-01-23 10:01:28 -05:00
Jonathan Ringer 0bc275e634
Revert "lib: Clean up how linux and gcc config is specified"
This is a stdenv-rebuild, and should not be merged
into master

This reverts commit 8929989614.
2021-01-22 14:07:06 -08:00
John Ericson 8929989614 lib: Clean up how linux and gcc config is specified
The `platform` field is pointless nesting: it's just stuff that happens
to be defined together, and that should be an implementation detail.

This instead makes `linux-kernel` and `gcc` top level fields in platform
configs. They join `rustc` there [all are optional], which was put there
and not in `platform` in anticipation of a change like this.

`linux-kernel.arch` in particular also becomes `linuxArch`, to match the
other `*Arch`es.

The next step after is this to combine the *specific* machines from
`lib.systems.platforms` with `lib.systems.examples`, keeping just the
"multiplatform" ones for defaulting.
2021-01-21 22:44:09 -05:00
Dominik Xaver Hörl 49130f93b7 nixos/modules-closure.sh: don't fail if firmware is missing
Since fdf32154fc, we no longer allow
missing modules in the initrd. Unfortunately since before this commit,
the modules-closure script would also fail on missing firmware, which
is a very common case (e.g. xhci-pci.ko.xz lists renesas_usb_fw.mem as
dependent firmware). Fix this by only issuing a warning instead.
2021-01-11 17:26:33 +01:00
Linus Heckemann 14fbf575ec make-initrd: various improvements
- Generate a link to the initramfs file with an appropriate file
  extension, guessed based on the compressor by default
- Use correct metadata in u-boot images if generated, up to now this
  was hardcoded to gzip and would silently generate an erroneous image
  if another compressor was specified
- Document all the parameters
- Improve cross-building compatibility, by allowing passing either a
  string as before, or a function taking a package set and returning the
  path to a compressor in the "compressor" argument of the
  function.
- Support more compression algorithms
- Place compressor executable function and arguments in passthru, for
  reuse when appending initramfses

Co-Authored-By: Dominik Xaver Hörl <hoe.dom@gmx.de>
2020-12-17 11:37:04 +01:00
Linus Heckemann 3a3c9c9548 makeInitrd: include dotfiles at root 2020-12-17 11:10:37 +01:00
Linus Heckemann 85e0ae7827 makeInitrd: don't assume uImage => arm
mips for example might use uImages too
2020-12-17 11:10:37 +01:00
Linus Heckemann a343ff7e14 makeInitrd: make uinitrd behaviour optional 2020-12-17 11:10:36 +01:00
Arthur Gautier 70bc1a3f49 makeModulesClosuse: read modules from corrent kernel version
Before this commit, the firmware information would be loaded from the
currently running kernel, not from the kernel to be loaded.

This commit ensures the correct kernel version and modules are read.
2020-08-27 10:48:18 -07:00
Arthur Gautier ee8572d6b3 makeModulesClosure: fixup firmware extraction
After a recent upgrade of modinfo, its output is now incorrect for
builtin modules. This commit filters out the output until a fix is made
available upstream
2020-08-27 10:45:23 -07:00
CrystalGamma b155b1dafb makeModulesClosure: handle builtin modules better
The previous code discarded entire dependency trees if the first entry in the dependency list compiled by `modprobe --show-depends` is a builtin and otherwise handled its output in a rather hackish way.
2020-07-16 20:44:07 +02:00
Markus S. Wamser 34b04a6364 modules-shrunk: preserve module priorities from build
depmod looks for files modules.order and modules.builtin which are
generated at kernel build time but were previously not passed to
the modules-shrunk derivation
2019-12-07 23:04:17 +01:00
Eelco Dolstra 328796d734
makeInitrd: Use stdenvNoCC 2019-10-11 17:12:09 +02:00
Arian van Putten b086b342bb build-support/make-initrd: Don't derive derivation name from file name
not all valid file names are valid derivation names. This can cause
troubles when, for example, trying to place systemd template unit
files, which contain an '@' in their name,  in an initrd.

Fixes #53987
2019-05-29 16:07:05 +02:00
Linus Heckemann 3b16f345b5 make-initrd: allow specifying name
Also moves the argument defaults out of all-packages.nix and into
make-initrd itself.
2019-04-13 23:20:58 +02:00
John Ericson 0828e2d8c3 treewide: Remove usage of remaining redundant platform compatability stuff
Want to get this out of here for 18.09, so it can be deprecated
thereafter.
2018-08-30 17:20:32 -04:00
aszlig de581b99ca
kernel: Fix running kernels *with* modules
Pull request #38470 added support for running/building kernels without
modules. This got merged in 38e04bbf29 but
unfortunately while this works perfectly on kernels without modules it
also makes sure that *every* kernel gets no modules.

So all of our VM tests fail since that merge with something like this:

machine# loading module loop...
machine# modprobe: FATAL: Module loop not found in directory /lib/modules/4.14.33
machine# loading module vfat...
machine# modprobe: FATAL: Module vfat not found in directory /lib/modules/4.14.33
machine# loading module nls_cp437...
machine# modprobe: FATAL: Module nls_cp437 not found in directory /lib/modules/4.14.33
machine# loading module nls_iso8859-1...
machine# modprobe: FATAL: Module nls_iso8859-1 not found in directory /lib/modules/4.14.33
machine# loading module fuse...
machine# modprobe: FATAL: Module fuse not found in directory /lib/modules/4.14.33
machine# loading module dm_mod...
machine# modprobe: FATAL: Module dm_mod not found in directory /lib/modules/4.14.33

I shortly tested this against the "misc" VM test and the test is working
again.

In the long term (and I currently don't have time for this) it would be
better to also have a VM test which tests a kernel without modules.

Signed-off-by: aszlig <aszlig@nix.build>
Cc: @roberth, @7c6f434c
2018-04-12 15:43:53 +02:00
Robert Hensing 30bff42231 linux module handling: support kernels without modules 2018-04-05 17:00:00 +02:00
Jan Malakhovski 7079e744d4 Merge branch 'master' into staging
Resolved the following conflicts (by carefully applying patches from the both
branches since the fork point):

   pkgs/development/libraries/epoxy/default.nix
   pkgs/development/libraries/gtk+/3.x.nix
   pkgs/development/python-modules/asgiref/default.nix
   pkgs/development/python-modules/daphne/default.nix
   pkgs/os-specific/linux/systemd/default.nix
2018-03-10 20:38:13 +00:00
Vladimír Čunát a373fe8322
makeInitrd: explain why we don't use closureInfo
/cc #36268.
2018-03-05 13:04:55 +01:00
Eelco Dolstra 165b32d386
Revert "makeInitrd: Use closureInfo"
This reverts commit 776a5e6ebf.

Fixes #36268.
2018-03-05 12:49:59 +01:00
Shea Levy 6a32291523
makeModulesClosure: Fix cross-compilation 2018-02-28 15:01:32 -05:00
Eelco Dolstra 776a5e6ebf
makeInitrd: Use closureInfo 2018-02-23 16:41:31 +01:00
Nikolay Amiantov 56e0943b08 makeModulesClosure: support firmware
Link it in stage 1.
2018-02-16 00:11:07 +02:00
Tuomas Tynkkynen 0d9f2f0bb4 platforms.nix: Clean up more 'uboot' legacy
For a while now, the only thing the 'uboot' attribute does is to tell
whether to add ubootTools to kernel/initrd builds. That can be
determined with platform.kernelTarget == "uImage" just as well.
2017-11-05 17:06:59 +02:00
Tuomas Tynkkynen b50693d16c kernel, initrd: Remove legacy ubootChooser 2017-11-05 15:11:12 +02:00
Eelco Dolstra 5939af52b3
pathsFromGraph: Remove obsolete printManifest feature 2017-10-25 15:38:11 +02:00
John Ericson deb8fae4c0 make-initrd: Don't use stdenv.cross 2017-06-28 18:21:05 -04:00
Joachim Fasting 5a8147479e make-initrd: create reproducible initrds
To achieve reproducible results, `cpio` archive members are added in
sorted order and inodes renumbered.

The `cpio-clean.pl` script is made obsolete by setting mtimes via
`touch` & using `cpio --reproducible`.  Suggested by @dezgeg in
https://github.com/NixOS/nixpkgs/pull/21273#issuecomment-268116605.

Note that using `--reproducible` means that initial ramdisk creation now
requires at least `cpio` version 2.12 (released in 2015).
2016-12-20 14:00:04 +01:00
Tuomas Tynkkynen 5076c0a5d3 modules-closure: Use stdenvNoCC 2016-10-20 02:39:29 +03:00
Nikolay Amiantov 6d6f6b4748 makeModulesClosure: don't set pipefail as it's already set in $stdenv/setup 2016-08-14 22:38:33 +03:00
Eelco Dolstra 030838ba25 makeModulesClosure: Small cleanup 2016-02-01 18:19:23 +01:00
Nikolay Amiantov 9c431b77ff make-initrd: store all files with root as owner 2015-12-28 12:07:11 +03:00
William A. Kennington III f2655e4fa0 makeInitrd: Support prepending other initrds 2015-03-25 14:14:46 -07:00
Alexander Kjeldaas 81172a77e6 Document paths-from-graph.pl somewhat. 2014-06-28 20:44:22 -05:00
Mathijs Kwik b21853f255 Fix initrd breaking by recent repeatable-builds changes
See the comments at f67015cae4
for more information.

Please note: this makes initrd unrepeatable again, but most people will prefer that above an unbootable system.
2014-04-12 00:06:30 +02:00
Alexander Kjeldaas f67015cae4 Make initrd and the kernel builds repeatable. 2014-04-05 08:41:06 +02:00
Eelco Dolstra 4c77c6e232 makeModulesClosure: Use kmod instead of module-init-tools 2013-08-07 22:46:11 +02:00
Shea Levy acd4299e75 Merge branch 'master' into stdenv-updates
Conflicts:
	pkgs/applications/networking/browsers/chromium/default.nix
	pkgs/top-level/all-packages.nix

Merge conflicts seemed trivial, but a look from viric and aszlig would be nice.
2013-02-21 11:13:30 -05:00
Shea Levy 79c3f992b4 makeInitrd: Make the compressor configurable 2013-02-19 11:49:31 -05:00