Create a many-layered Docker Image.
Implements much less than buildImage:
- Doesn't support specific uids/gids
- Doesn't support runninng commands after building
- Doesn't require qemu
- Doesn't create mutable copies of the files in the path
- Doesn't support parent images
If you want those feature, I recommend using buildLayeredImage as an
input to buildImage.
Notably, it does support:
- Caching low level, common paths based on a graph traversial
algorithm, see referencesByPopularity in
0a80233487993256e811f566b1c80a40394c03d6
- Configurable number of layers. If you're not using AUFS or not
extending the image, you can specify a larger number of layers at
build time:
pkgs.dockerTools.buildLayeredImage {
name = "hello";
maxLayers = 128;
config.Cmd = [ "${pkgs.gitFull}/bin/git" ];
};
- Parallelized creation of the layers, improving build speed.
- The contents of the image includes the closure of the configuration,
so you don't have to specify paths in contents and config.
With buildImage, paths referred to by the config were not included
automatically in the image. Thus, if you wanted to call Git, you
had to specify it twice:
pkgs.dockerTools.buildImage {
name = "hello";
contents = [ pkgs.gitFull ];
config.Cmd = [ "${pkgs.gitFull}/bin/git" ];
};
buildLayeredImage on the other hand includes the runtime closure of
the config when calculating the contents of the image:
pkgs.dockerTools.buildImage {
name = "hello";
config.Cmd = [ "${pkgs.gitFull}/bin/git" ];
};
Minor Problems
- If any of the store paths change, every layer will be rebuilt in
the nix-build. However, beacuse the layers are bit-for-bit
reproducable, when these images are loaded in to Docker they will
match existing layers and not be imported or uploaded twice.
Common Questions
- Aren't Docker layers ordered?
No. People who have used a Dockerfile before assume Docker's
Layers are inherently ordered. However, this is not true -- Docker
layers are content-addressable and are not explicitly layered until
they are composed in to an Image.
- What happens if I have more than maxLayers of store paths?
The first (maxLayers-2) most "popular" paths will have their own
individual layers, then layer #(maxLayers-1) will contain all the
remaining "unpopular" paths, and finally layer #(maxLayers) will
contain the Image configuration.
The `overrideScope` bound by `makeScope` (via special `callPackage`)
took an override in the form `super: self { … }`. But this is
dangerously close to the `self: super { … }` form used by *everything*
else, even other definitions of `overrideScope`! Since that
implementation did not even share any code either until I changed it
recently in 3cf43547f4, this inconsistency
is almost certainly an oversight and not intentional.
Unfortunately, just as the inconstency is hard to debug if one just
assumes the conventional order, any sudden fix would break existing
overrides in the same hard-to-debug way. So instead of changing the
definition a new `overrideScope'` with the conventional order is added,
and old `overrideScope` deprecated with a warning saying to use
`overrideScope'` instead. That will hopefully get people to stop using
`overrideScope`, freeing our hand to change or remove it in the future.
For technical reasons, we cannot easily add a warning to top-level
definitions, so 2a6e4ae49a and
e51f736076 reverted the deprecation. But
we can still remove mention of the would-be deprecated definitions to
steer people towards using the preferred alternatives.
Because dates are an impurity, by default buildImage will use a static
date of one second past the UNIX Epoch. This can be a bit frustrating
when listing docker images in the CLI:
$ docker image list
REPOSITORY TAG IMAGE ID CREATED SIZE
hello latest 08c791c7846e 48 years ago 25.2MB
If you want to trade the purity for a better user experience, you can
set created to now.
pkgs.dockerTools.buildImage {
name = "hello";
tag = "latest";
created = "now";
contents = pkgs.hello;
config.Cmd = [ "/bin/hello" ];
}
and now the Docker CLI will display a reasonable date and sort the
images as expected:
$ docker image list
REPOSITORY TAG IMAGE ID CREATED SIZE
hello latest de2bf4786de6 About a minute ago 25.2MB
This package providesa completion input method for faster typing.
See https://mike-fabian.github.io/ibus-typing-booster
Detailed instructions how to activate this IBus engine on your desktop
can be found in the upstream docs: https://mike-fabian.github.io/ibus-typing-booster/documentation.html
A simple VM with the Gnome3 desktop and activated `ibus' looks like
this:
```nix
{
emojipicker = { pkgs, ... }: {
services.xserver = {
enable = true;
desktopManager.gnome3.enable = true;
desktopManager.xterm.enable = false;
};
users.extraUsers.vm = {
password = "vm";
isNormalUser = true;
};
i18n.inputMethod.ibus.engines = [
pkgs.ibus-engines.typing-booster
];
i18n.inputMethod.enabled = "ibus";
virtualisation.memorySize = 2048;
};
}
```
Fixes #38721
A new python script has been added to replace the aged viml-based
updater. The new updater has the following advantages:
- use rss feeds to check for updates quicker
- parallel downloads & better caching
- uses proper override mechanism instead of text substitution
- update generated files in-place instead of having to insert updated plugins manually
Automatically reading `dependencies` from the plugins directory has been
not re-implemented.
This has been mostly been used by Mark Weber's plugins, which seem to
no longer receive regular updates.
This could be implemented in future as required.
This aims to make the `weechat` package even more configurable. It
allows to specify scripts and commands using the `configure` function
inside a `weechat.override` expression.
The package can be configured like this:
```
with import <nixpkgs> { };
weechat.override {
plugins = { availablePlugins, ... }: {
plugins = builtins.attrValues availablePlugins;
init = ''
/set foo bar
/server add freenode chat.freenode.org
'';
scripts = [ "/path/to/script.py" ];
};
}
```
All commands are passed to `weechat --run-command "/set foo bar;/server ..."`.
The `plugins' attribute is not necessarily required anymore, if it's
sufficient to add `init' commands, the `plugins' will be
`builtins.attrValues availablePlugins' by default.
Additionally the result contains `weechat` and `weechat-headless`
(introduced in WeeChat 2.1) now.
I don't know when we can/should remove them, but this at least gets
people to stop using them. The preferred alternatives also date back to
17.09 so writing forward-compatable code without extra conditions is
easy.
Beginning with these as they are the least controversial.
Since #44522 it's possible to specify custom certificates for the Citrix
receiver. As it took me some time to create a proper setup Citrix can
behave fairly unexpected.
I mostly covered two aspects:
* Don't install Citrix with `nix run`: when `citrix.desktop` is linked
to $XDG_CONFIG_DIRS, it's possible to start a session directly from the
browser when loading `.ica` files which makes the usage *way* easier.
* It's possible to add custom certificates using the Citrix wrapper. A
new store path with the original derivation and the certificates will be
created and therefore no rebuild of the package is needed when adding
new certs.
The server is not verified over the git:// transfer protocol. If you
clone a repository over git://, you should check if the latest commit's
hash is correct.
On the other hand, https:// will always verify the server automatically,
using certificate authorities.
Use example package `zerobin` instead of `bepasty-server` which
is no longer part of python-packages.
This fixes the examples for current nixpkgs versions.
1. Use the same approach like in the overlay example:
Override `python` instead of `pythonPackages` so that
`python.pkgs` refers to the new package set like `pythonPackages`.
This also fixes a bug in the original example where
`pkgs.fetchgit` was not in scope.
Add an extra example to illustrate how to override just a
package set.
2. Fix mix-up between `super` and `self` in the explanation text.
Also, simplify the explanation.
This makes the command ‘nix-env -qa -f. --arg config '{skipAliases =
true;}'’ work in Nixpkgs.
Misc...
- qtikz: use libsForQt5.callPackage
This ensures we get the right poppler.
- rewrites:
docbook5_xsl -> docbook_xsl_ns
docbook_xml_xslt -> docbook_xsl
diffpdf: fixup
This script is used to automatically fix issues within xml documentation
files.
The script is *for now* intended to be used ad-hoc, and the commits to
be examined.
A future discussion will define whether:
* This commit and scripts are kept.
* The script is extended for common use.
The biggest issue right now with the script is that it *could* in theory
destroy a valid space-less varlistentry.
The script could, in practical use, be changed and extended to normalize
some parts of the XML files, mainly:
* A common quoting style for attributes
* Fix-up some weird formatting automatically that xmlformat doesn't
catch
This fixes a regression introduced in 4b06383.
[dezgeg squashed in to fit the changes introduced in "db: Use more
conventional outputs, also split bin"]
Relevant section: 9.5.2.3. How to install a compiler with libraries, hoogle and documentation indexes
Since version 5 `hoogle server`s --local flag solves the problem with links from
`http:` to `file:` URIs:
hoogle server --local -p 8080
As suggested in https://github.com/NixOS/nixpkgs/pull/39416#discussion_r183845745
the versioning attributes in `lib` should be consistent to
`nixos/version` which implicates the following changes:
* `lib.trivial.version` -> `lib.trivial.release`
* `lib.trivial.suffix` -> `lib.trivial.versionSuffix`
* `lib.nixpkgsVersion` -> `lib.version`
As `lib.nixpkgsVersion` is referenced several times in `NixOS/nixpkgs`,
`NixOS/nix` and probably several user's setups. As the rename will cause
a notable impact it's better to keep `lib.nixpkgsVersion` as alias with
a warning yielded by `builtins.trace`.
Note that a bunch of non-python packages use this attribute already.
Some of those are clearly unaware of the fact that this attribute does
not exists in stdenv because they define it but don't to add it to
their `bulidInputs` :)
Also note that I use `buildInputs` here and only handle regular
builds because python and haskell builders do it this way and I'm not
sure how to properly handle the cross-compilation case.
Setting haskell.packageOverrides like so:
haskell = super.haskell // {
packageOverrides = self: super: {
my-package = ...;
my-other-package = ...;
};
};
causes all compiler-specific package sets to be overridden with those
overrides.
- Rectifies diverging CSS by combining
nixos/nixpkgs docs CSS
- Moves our custom Highlight.js loader in to
the hljs package
- Switches the nixos docs to use SVG
callouts too
Nobody has stepped up to keep maintaining this and it's several
years old, and the last strict Java 7 dependency, as it won't work
with newer versions without an update.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
- Add example for setting up nix-shell, improve rust docs
- Rust docs: add gcc rust dependencies and fix carnix commands
- Fix a typo with the carnix command.
* trying to build emscriptenPackages not all fail
* reading the console.log it turns out python executable is not in place and that is why emconfigure didnt work
* backup commit
* much more targets are compiling now
* added common revisioning
* revision bump to 1.37.36 (not tested)
* fixed xmllint
* forcing unit testing, will implement the tests after i get home
* json_c test working
* added tests
* tiny fixes
* added documentation
Resolved the following conflicts (by carefully applying patches from the both
branches since the fork point):
pkgs/development/libraries/epoxy/default.nix
pkgs/development/libraries/gtk+/3.x.nix
pkgs/development/python-modules/asgiref/default.nix
pkgs/development/python-modules/daphne/default.nix
pkgs/os-specific/linux/systemd/default.nix
This involved:
* Installing miniperl as $dev/bin/perl
* Setting miniperl to take INC from
lib/perl5/{site_perl/,}cross_perl/${version} as well as
lib/perl5/{site_perl/,}/${version}/${runtimeArch}, in that
order. miniperl taking from runtimeArch is not really correct, but
it works in some pure-perl cases (e.g. Config.pm) and can be
overridden with the cross_perl variant.
* Installing perl-cross's stubs into
$dev/lib/perl5/cross_perl/${version}
* Patching MakeMaker.pm to gracefully degrade (very slightly) if B.pm
can't be loaded, which it can't in cross-compilation.
* Passing the right build-time and runtime perls to Makefile.PL
Existing "mips64el" should be "mipsel".
This is just the barest minimum so that nixpkgs can recognize them as
systems - although required for building individual derivations onto
MIPS boards, it is not sufficient if you want to actually build nixos on
those targets
I originally wrote this for packaging proprietary games in Vuizvui[1]
but I thought it would be generally useful as we have a fair amount of
proprietary software lurking around in nixpkgs, which are a bit tedious
to maintain, especially when the library dependencies change after an
update.
So this setup hook searches for all ELF executables and libraries in the
resulting output paths after install phase and uses patchelf to set the
RPATH and interpreter according to what dependencies are available
inside the builder.
For example consider something like this:
stdenv.mkDerivation {
...
nativeBuildInputs = [ autoPatchelfHook ];
buildInputs = [ mesa zlib ];
...
}
Whenever for example an executable requires mesa or zlib, the RPATH will
automatically be set to the lib dir of the corresponding dependency.
If the library dependency is required at runtime, an attribute called
runtimeDependencies can be used to list dependencies that are added to
all executables that are discovered unconditionally.
Beside this, it also makes initial packaging of proprietary software
easier, because one no longer has to manually figure out the
dependencies in the first place.
[1]: https://github.com/openlab-aux/vuizvui
Signed-off-by: aszlig <aszlig@nix.build>
Closes: #34506
I hope this will be a temporary measure. If there is consensus around
issue #33599, then we can follow an explicit `dontCheck`, but default to
not checking during cross builds when none is given.
This accounts for all the new dependencies and propagation logic changes
I'm about to add.
Fixes #1915---with this change I think the distinction is finally clear
enough.
This removes some stale code that was a no-op for some time and adds
some docs/examples to help people with explicitly and consistently
choosing versions of some emacs packages (to help with problems
similar to #27083).
Also add a wrapper generator that allows adding the plugins back
conveniently and corresponding documentation in the package notes
section of the nixpkgs manual.
The biggest benefit is that we no longer have to update the registry
package. This means that just about any cargo package can be built by
nix. No longer does `cargo update` need to be feared because it will
update to packages newer then what is available in nixpkgs.
Instead of fetching the cargo registry this bundles all the source code
into a "vendor/" folder.
This also uses the new --frozen and --locked flags which is nice.
Currently cargo-vendor only provides binaries for Linux and
macOS 64-bit. This can be solved by building it for the other
architectures and uploading it somewhere (like the NixOS cache).
This also has the downside that it requires a change to everyone's deps
hash. And if the old one is used because it was cached it will fail to
build as it will attempt to use the old version. For this reason the
attribute has been renamed to `cargoSha256`.
Authors:
* Kevin Cox <kevincox@kevincox.ca>
* Jörg Thalheim <Mic92@users.noreply.github.com>
* zimbatm <zimbatm@zimbatm.com>
Commit 8537cf0f81
("CONTRIBUTING.md: suggest "nixos/<module>" prefix for NixOS changes")
only changed CONTRIBUTING.md file and forgot about the Nixpkgs manual.
(I didn't know this information was stored in two places.)
The main motivation for this is to have something to google for LD=$CC.
Eventually, this should probably be moved to another section, but we
can deal with that later.
1. Call `nix-build` with `--no-out-link` to avoid cluttering the source dir.
2. Re-add `patchShebangs`, since `buildCommand` doesn't imply a patch phase. (It was my fault to remove this in the first place, sorry!)
This allows one to always override the call to `buildPythonPackage`.
In the following example we create an environment where we have the `blaze` package using an older version of `pandas`. We override first the Python interpreter and pass `packageOverrides` which contains the overrides for packages in
the package set.
```
with import <nixpkgs> {};
(let
python = let
packageOverrides = self: super: {
pandas = super.pandas.overridePythonPackage(old: rec {
version = "0.19.1";
name = "pandas-${version}";
src = super.fetchPypi {
pname = "pandas";
inherit version;
sha256 = "08blshqj9zj1wyjhhw3kl2vas75vhhicvv72flvf1z3jvapgw295";
};
});
};
in pkgs.python3.override {inherit packageOverrides;};
in python.withPackages(ps: [ps.blaze])).env
```
1. 'wrapper' has been renamed to 'wrappedRuby', so use this instead.
2. mkDerivation isn't called with a 'src' attribute, so skip the 'unpackPhase' to avoid an error.
3. Simplify the build command. 'mkdir' and 'patchShebangs' don't need to be called explicitly.
So that helper scripts can be easily sourced in interactive shell
configuration. `autojump` package was already present and had the same
requirements for findind a `share` folders, so I took an inspiration
there.
I beleive this is a better alternative to:
- https://github.com/NixOS/nixpkgs/pull/25080
- https://github.com/NixOS/nixpkgs/pull/27058
Replacing `$out/share/shell` with `$bin/share/fzf` was necessary to
prevent dependency loop in produced derivations.
Packages get --host and --target by default, but can explicitly request
any subset to be passed as needed. See docs for more info.
rustc: Avoid hash breakage by using the old (ignored)
dontSetConfigureCross when not cross building
Improve beam docs:
* correct spelling
* update per pandoc changes
* capitalize titles
* capitalize BEAM throughout and use "the BEAM" when referring to the virtual machine.
* tweak grammar and phrasing
* reformat build-tools-rebar3 section
* add more links
* re-wrap <para>s
Also update <programlisting>s
* normalize whitespace
* don't double quote homepage
* use $ in all shell snippets
The documentation got a bit stale compared to actual contents of
nixpkgs. This commit focuses on updating existing docs, not on making
sure all details of beam packages are covered.
This section gives some details on how to setup an "environment"
without having to go through NixOS (although it could be used there
too). I’ve tried to make it straightforward and have a kind of
"tutorial" feel. Not sure if that’s appropriate for the manual, so any
recommended changes would be helpful.
This gives some basics on configuring Emacs within Nix. The
configuration is fairly long just to give a good idea of what’s going
on. I can trim out some of it if it’s not necessary. Note that there
is already a section for Emacs in the NixOS manual. However, this is
aimed at avoiding using modules altogether to make things easier for
non-NixOS users. This configuration should work on NixOS anyway,
however.
Fixes #24243
Fixes #19956
Each bootstrapping stage ought to just depend on the previous stage, but
poorly-written compilers break this elegence. This provides an easy-enough
way to depend on the next stage: targetPackages. PLEASE DO NOT USE IT
UNLESS YOU MUST!
I'm hoping someday in a pleasant future I can revert this commit :)
* Update python.md
this makes it clear how to alter `attributes` by using `packageOverrides`
* Update python.md
* Update python.md
* Update python.md
* Update python.md
* Update python.md
* Update python.md
Fix code syntax highlighting by specifying language in every code block
and adding some context to Nix code blocks to make them valid
expressions. Use the same markup style for all code blocks. Reformat
some code blocks.
fixes #23535
- Describe the new `libsForQt5.callPackage` interface
- Emphasize that Qt dependencies must be imported unqualified
- Describe the new `kdeWrapper` wrapper generator
If a package's meta has `knownVulnerabilities`, like so:
stdenv.mkDerivation {
name = "foobar-1.2.3";
...
meta.knownVulnerabilities = [
"CVE-0000-00000: remote code execution"
"CVE-0000-00001: local privilege escalation"
];
}
and a user attempts to install the package, they will be greeted with
a warning indicating that maybe they don't want to install it:
error: Package ‘foobar-1.2.3’ in ‘...default.nix:20’ is marked as insecure, refusing to evaluate.
Known issues:
- CVE-0000-00000: remote code execution
- CVE-0000-00001: local privilege escalation
You can install it anyway by whitelisting this package, using the
following methods:
a) for `nixos-rebuild` you can add ‘foobar-1.2.3’ to
`nixpkgs.config.permittedInsecurePackages` in the configuration.nix,
like so:
{
nixpkgs.config.permittedInsecurePackages = [
"foobar-1.2.3"
];
}
b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
‘foobar-1.2.3’ to `permittedInsecurePackages` in
~/.config/nixpkgs/config.nix, like so:
{
permittedInsecurePackages = [
"foobar-1.2.3"
];
}
Adding either of these configurations will permit this specific
version to be installed. A third option also exists:
NIXPKGS_ALLOW_INSECURE=1 nix-build ...
though I specifically avoided having a global file-based toggle to
disable this check. This way, users don't disable it once in order to
get a single package, and then don't realize future packages are
insecure.
This fixes the "sliding window" principle:
0. Run packages: build = native; host = foreign; target = foreign;
1. Build packages: build = native; host = native; target = foreign;
2. Vanilla packages: build = native; host = native; target = native;
3. Vanilla packages: build = native; host = native; target = native;
n+3. ...
Each stage's build dependencies are resolved against the previous stage,
and the "foreigns" are shifted accordingly. Vanilla packages alone are
built against themsevles, since there are no more "foreign"s to shift away.
Before, build packages' build dependencies were resolved against
themselves:
0. Run packages: build = native; host = foreign; target = foreign;
1. Build packages: build = native; host = native; target = foreign;
2. Build packages: build = native; host = native; target = foreign;
n+2. ...
This is wrong because that principle is violated by the target
platform staying foreign.
This will change the hashes of many build packages and run packages, but
that is OK. This is an unavoidable cost of fixing cross compiling.
The cross compilation docs have been updated to reflect this fix.
If the flag enableIntegerSimple is true GHC will be build with the GPL-free but
slower integer-simple library instead of the faster but GPLed integer-gmp
library.
The attribute `pkgs.haskell.compiler.integer-simple."${ghcVersion}"` provides a
GHC compiler build with `integer-simple`.
Similarly, the attribute `pkgs.haskell.packages.integer-simple."${ghcVersion}"`
provides a package set supporting `integer-simple`.
Closes https://github.com/NixOS/nixpkgs/pull/22121.
Closes https://github.com/NixOS/nixpkgs/issues/5493.
The Nix search path is the established mechanism for specifying the
location of Nix expressions, so let's use it instead of adding another
environment variable.
Do so in the stdenv section where the other two are discussed. This can be
done without brining up cross-compilation by talking about build-time vs
run-time.
This patch add a new argument to Nixpkgs default expression named "overlays".
By default, the value of the argument is either taken from the environment variable `NIXPKGS_OVERLAYS`,
or from the directory `~/.nixpkgs/overlays/`. If the environment variable does not name a valid directory
then this mechanism would fallback on the home directory. If the home directory does not exists it will
fallback on an empty list of overlays.
The overlays directory should contain the list of extra Nixpkgs stages which would be used to extend the
content of Nixpkgs, with additional set of packages. The overlays, i-e directory, files, symbolic links
are used in alphabetical order.
The simplest overlay which extends Nixpkgs with nothing looks like:
```nix
self: super: {
}
```
More refined overlays can use `super` as the basis for building new packages, and `self` as a way to query
the final result of the fix-point.
An example of overlay which extends Nixpkgs with a small set of packages can be found at:
https://github.com/nbp/nixpkgs-mozilla/blob/nixpkgs-overlay/moz-overlay.nix
To use this file, checkout the repository and add a symbolic link to
the `moz-overlay.nix` file in `~/.nixpkgs/overlays` directory.
