A tricky thing about FreeBSD is that there is no stable ABI across
versions. That means that putting in the version as part of the config
string is paramount.
We have a parsed represenation that separates name versus version to
accomplish this. We include FreeBSD versions 12 and 13 to demonstrate
how it works.
The code of `lib.closePropagation` was internally using a
recursion on the dependencies and returns all the derivation directly or
indirectly referenced by buildInputs.
`lib.closeProgation` is implemented in pure nix and uses an unique
function for list which is quadratic and does "true" equality, which
needs deep set comparison.
Instead, we use the `builtins.genericClosure` which is implemented as a
builtin and uses a more efficient sorting feature.
Note that `genericClosure` needs a `key` to discriminate the values, we
used the `outPath` which is unique and orderable.
On benchmarks, it performs up to 15x time faster on a benchmark related
to haskellPackages.ghcWithPackages.
Personally, I think that warnings such as
warning: The option `services.redis.enable' defined in `/home/ma27/Projects/nixpkgs/test.nix@node-vm' has been renamed to `services.redis.servers..enable'.
are fairly confusing because of the `..` and it's more correct to
actually quote that. With this change the warning now looks like this:
warning: The option `services.redis.enable' defined in `/home/ma27/Projects/nixpkgs/test.nix@node-vm' has been renamed to `services.redis.servers."".enable'.
While implementing that I realized that you'd have
a similar problem whenever you use attribute-names that aren't
identifiers, e.g.
services.nginx.virtualHosts."example.org".locations."/".invalid = 23;
now results in the following error:
error: The option `interactive.nodes.vm.services.nginx.virtualHosts."example.org".locations."/".invalid' does not exist. Definition values:
- In `/home/ma27/Projects/nixpkgs/test.nix@node-vm': 23
Of course there are some corner-cases where this won't work: when
generating the manual, you display submodules like this:
services.nginx.virtualHosts.<name>
Since `<name>` isn't a value, but an indicator for a submodule, it must
not be quoted. This also applies to the following identifiers:
* `*` for `listOf submodule`
* `<function body>` for `functionTo`
This might not be correct if you actually have a submodule with an
attribute name called `<name>`, but I think it's an improvement over the
current situation and for this you'd probably need to make even more
complex changes to the module system.
The motivation is to have a single identifier for that. Useful for the
next commit where I'll try to escape option-parts correctly (options can
be any kind of strings, but unless these are Nix identifiers, they must
be quoted).
Since `<function body>` (or `<name>`/`*`) are special identifiers in
error messages and the manual, we need a unique way to mark an option
part as function call because these are not to be quoted.
Move already implemented functionality to the upper level so
it could be used in a more generic way.
Signed-off-by: Ivan Nikolaenko <ivan.nikolaenko@unikie.com>
This brings two benefits:
1. The complete list of collisions is printed in the whenever any colliding
attribute is accessed.
2. The sets are intersected using a C++ primitive, which runs in O(n) time
(intersecting pre-sorted lists) with small constants rather than interpreted
Nix code.
Thanks to @toonn for prompting this improvement.
```
nix-repl> pkgsCross.arm-embedded.stdenv.hostPlatform.emulatorAvailable pkgsCross.arm-embedded.buildPackages
false
nix-repl> pkgsCross.aarch64-multiplatform.stdenv.hostPlatform.emulatorAvailable pkgsCross.aarch64-multiplatform.buildPackages
true
```
will be useful for stuff like handling https://github.com/NixOS/nixpkgs/issues/187109
deprecate literalDocBook by adding a warning (that will not fire yet) to
its uses and other docbook literal strings by adding optional warning
message to mergeJSON.
The comment in lib/systems/default.nix for uname.processor indicates that it
should match `uname -p`. I tried that command and found that it reports
`unknown` on all of these machines:
- `x86_64-linux`
- `aarch64-linux`
- `mips64el-linux`
- `powerpc64le-linux`
The command `uname -m` reports the expected value on all of the above.
I think the comment is wrong. So I fixed it.
This attr provides the location of each definition.
This is particularly useful for introspecting options of type
`attrsOf`. E.g., it allows finding the location of a systemd
service definition by parsing
`options.systemd.services.definitionsWithLocations`.
This is particularly useful for disabling modules defined in a flake.
Example:
disabledModules = [ "${flake}/modules/mymodule.nix" ];
Previously, absolute string paths were internally prepended with `modulesPath`,
which caused the module filtering to fail.
Recent commit 59356f11c1 ("perlPackages: Ensure all packages have a
license", 2022-08-22) added a license field to Perl packages where the
license was missing. The above mentioned packages got assigned
`unfreeRedistributable` license, which is not precise and makes all
packages depending on them unbuildable without `NIXPKGS_ALLOW_UNFREE`.
The packages actually have a license which SPDX calls
BSD-4-Clause-Shortened (https://spdx.org/licenses/BSD-4-Clause-Shortened.html).
In this commit, we add this license to the list of allowed licenses
and change the license field of the mentioned packages.
Closes#188103
In Nixpkgs, we assume that the "config" field is a canonicalized GNU
triple. I noticed that non-canonical values were being used here,
because the pkgsCross.mips64el-linux-gnu triples did not contain the
vendor field, but the pkgsCross.mips64el-linux-gnu.pkgsStatic did.
Here, I've run all the MIPS triples in lib.systems.examples through
config.sub to canonicalize them. I think this will avoid nasty
surprises in future.
Tested by building Nix and the bootstrap files for
pkgsCross.mips64el-linux-gnu.
More nixpkgs code such as `boot.initrd.systemd.emergencyAccess` defines
options that takes hashed passwords, so move the type definition from
modules/ into lib/.
The type definition itself stays unchanged.
`m` must always be an attrset at this point. It is basically always
evaluated. This will make it throw when any of the attrs is accessed,
rather than just `config`. We assume that this will improve the error
message in more scenarios.
This has been deprecated for a long time, and it's doubtful it had any
users to start with. And having an undisablable warning when
enumarating platforms is not good.
These servers apparently no longer exist, since September 2, 2021[1].
If somebody needs this for non-Scaleway machines, they should suggest
its reintroduction with a different name.
[1]: https://news.ycombinator.com/item?id=27192757
Very confusingly, the `isPowerPC` predicate in
`lib/systems/inspect.nix` does *not* match `powerpc64le`!
This is because `isPowerPC` is defined as
isPowerPC = { cpu = cpuTypes.powerpc; };
Where `cpuTypes.powerpc` is:
{ bits = 32; significantByte = bigEndian; family = "power"; };
This means that the `isPowerPC` predicate actually only matches the
subset of machines marketed under this name which happen to be 32-bit
and running in big-endian mode which is equivalent to:
with stdenv.hostPlatform; isPower && isBigEndian && is32bit
This seems like a sharp edge that people could easily cut themselves
on. In fact, that has already happened: in
`linux/kernel/common-config.nix` there is a test which will always
fail:
(stdenv.hostPlatform.isPowerPC && stdenv.hostPlatform.is64bit)
A more subtle case of the strict isPowerPC being used instead of the
moreg general isPower accidentally are the GHC expressions:
Update pkgs/development/compilers/ghc/8.10.7.nix
Update pkgs/development/compilers/ghc/8.8.4.nix
Update pkgs/development/compilers/ghc/9.2.2.nix
Update pkgs/development/compilers/ghc/9.0.2.nix
Update pkgs/development/compilers/ghc/head.nix
Since the remaining legitimate use sites of isPowerPC are so few, remove
the isPowerPC predicate completely. The alternative expression above is
noted in the release notes as an alternative.
Co-authored-by: sternenseemann <sternenseemann@systemli.org>
canExecute is like isCompatible, but also checks that the Kernels are
_equal_, i.e. that both platforms use the same syscall interface. This
is crucial in order to actually be able to execute binaries for the
other platform.
isCompatible is dropped, since it has changed semantically and there's
no use case left in nixpkgs.
Since we (exclusively) use isCompatible to gauge whether platform a can
execute binaries built for platform b, mode switching CPUs are not to be
considered compatible for our purposes: Switching the mode of a CPU
usually requires a reset. At the very least we can't execute a mix of
executables for the two modes which would usually be the case in nixpkgs
where we may want to execute buildInputs for the hostPlatform in
addition to nativeBuildInputs for the buildPlatform.
Since the list only gates the platforms the nixpkgs flake exposes
packages to build on, the `hydra` label made little sense. It was also
only used for this purpose, so the `tier*` attributes were largely
unnecessary.
To reflect the intention more accurately, we expose
`lib.systems.flakeExposed` and use it to gate flake.nix's system list.
Currently the only way to set the description for a submodule type is to
use `freeformType`. This is not ideal as it requires setting a
freeform type, and evaluates the submodule config unnecessarily.
Instead, add a `description` argument to `submoduleWith`.
This reverts commit PR #167947.
Flakes aren't standardised and the `lib` namespace shouldn't be
polluted with utilities that serve only experimental uses.
This patch causes the autodetection code in lib/systems/platforms.nix
to return {} if it cannot detect the platform and one of the
platform.nix-detection-provided attributes (linux-kernel, gcc, and
rustc) are accessed, rather than silently assuming the "pc" platform
as was previously done.
It is definitely safe to assume that code using these attributes is
prepared to deal with `gcc` and `rustc` not being defined, because
many of the working entries in this file don't define it.
Regarding `linux-kernel` the situation is less certain, but some code
(`lib/systems/default.nix` for example) is already designed to deal
with that attribute being missing. At worst it would result in an
"attribute not found" error.
While adding mips64el bootstrap support to nixpkgs, the silent
assumption that mips64el routers are actually Intel PCs caused
significant frustration. This commit removes that assumption in order
to save people who port nixpkgs to new platforms in the future from
this frustration.
For other platforms like Intel and ARM, we can do
e.g. lib.platforms.aarch64 to get only the 64-bit ARM platorms, but
until now there were no equivalents for RISC-V.
Prior to this commit, nixpkgs would assume that every little-endian
mips32 system was a "fuloong2f_n32".
Not only are there plenty of mips32 chips other than the fuloong, but
the fuloong is actually a mips64 chip! Note that the "n32" ABI is
(confusingly) an ABI for 64-bit mips chips (like the "x32" ABI for
amd64 chips -- both are ABIs which use 32-bit pointers on an
otherwise-64-bit system).
This error causes far-ranging problems. One of them was particularly
difficult to track down: it caused GCC to select 128-bit `long double`
types, which is invalid for the mips32 ABI. This isn't noticed until
you try to build musl-libc, which is careful to check for these things.
Prior to this commit,
nix-build . -A pkgsCross.mipsel-linux-gnu.pkgsStatic.hello
would fail. With this commit and #170736, it succeeds.
There is only one ABI for 32-bit MIPS chips. Before mips64, it didn't
really have a name.
The 64-bit MIPS ABI comes in two flavors, "n64" and "n32". It is
commonplace to refer to the old 32-bit ABI as "o32" (MIPS and SGI
documents do this).
However, when configuring gcc, one must use --with-abi=32, not
--with-abi=o32.
Let's keep GCC happy with this commit.
Probably being the most prominent document demonstrating the problem,
configuration.nix(5) describes various types in plural, e.g.
- ` Type: list of strings`
- ` Type: list of systemd options`
However, there are other cases where appending "s" to the inner type
effectively changes the type, e.g.
- ```
Type: list of string matching the pattern
[a-zA-Z0-9@%:_.\-]+[.](service|socket|device|mount|automount|swap|target|path|timer|scope|slice)s
```
This should've read "list of string[s]..." but instead changes the
regular expression.
Simply drop the best-effort plural in favour of correctness and
simplicity rather than adding more grammar related logic/trying to fix
this.
* lib/strings: optimise hasInfix function
* lib/strings: optimise hasInfix further using regex
* rstudio: call hasInfix with a string
* lib/strings: remove let from hasInfix
Co-authored-by: pennae <82953136+pennae@users.noreply.github.com>
Co-authored-by: pennae <82953136+pennae@users.noreply.github.com>
This commit creates flakes.nix, which is a library containing functions
which relate to interacting with flakes. It also moves related functions
from trivial.nix into it.
This is essentially a copy of the function of the same name, from
flake-compat. callLocklessFlake is useful when trying to utilise a
flake.nix without a lock file, often for when you want to create a
subflake from within a parent flake.
Co-authored-by: Tom Bereknyei <tomberek@gmail.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Closes#168327
The issue reported there can be demonstrated with the following
expression:
→ nix-instantiate --eval -E "with import ./. {}; pkgs.lib.options.showDefs [ { file = \"foo\"; value = pkgs.rust.packages.stable.buildRustPackages; } ]"
error: attempt to call something which is not a function but a string
at /home/ma27/Projects/nixpkgs/lib/trivial.nix:442:35:
441| isFunction = f: builtins.isFunction f ||
442| (f ? __functor && isFunction (f.__functor f));
| ^
443|
Basically, if a `__functor` is in an attribute-set at depth-limit,
`__functor` will be set to `"<unevaluated>"`. This however breaks
`lib.isFunction` which checks for a `__functor` by invoking `__functor`
with `f` itself.
The same issue - "magic" attributes being shadowed by `withRecursion` -
also applies to others such as
`__pretty`/`__functionArgs`/`__toString`.
Since these attributes have a low-risk of causing a stack overflow
(because these are flat attr-sets or even functions), ignoring them in
`withRecursion` seems like a valid solution.
This allows other module system consumers to
disable these docs via option merging.
For instance arion uses asciidoc instead of
docbook so that would look awful.
This commit adds an `isPower64` predicate to the two existing
predicates for this architecture (`isPower` and `isPowerPC`).
Note that `isPowerPC` matches only 32-bit machines, whereas `isPower`
matches both 64-bit and 32-bit machines. Prior to this commit there
was no single `isXXX` predicate for `powerpc64le`.
Documents the _module.args option, motivated by many usages in Flakes,
especially with the deprecation of extraArgs
(78ada83361)
The documentation rendering for this option had to be handled a bit
specially, since it's not declared in nixos/modules like all the other
NixOS options.
Co-Authored-By: pennae <github@quasiparticle.net>
Co-Authored-By: Robert Hensing <robert@roberthensing.nl>
This uses the levenshtein distance to look through all possible
arguments to find ones that are close to what was requested:
error: Function in /home/infinisil/src/nixpkgs/pkgs/tools/text/ripgrep/default.nix
called without required argument "fetchFromGithub",
did you mean "fetchFromGitHub" or "fetchFromGitLab"?
With https://github.com/NixOS/nix/pull/3468 (in current nixUnstable) the error
message becomes even better, adding line location info
Adds some functions related to string similarity:
- lib.strings.commonPrefixLength
- lib.strings.commonSuffixLength
- lib.strings.levenshtein
- lib.strings.levenshteinAtMost
types.optionSet has been deprecated for almost 10 years now
(0e333688ce)! A removal
was already attempted in 2019
(27982b408e), but it was promptly
reinstantiated since some third-party uses were discovered
(f531ce75e4).
It's finally time to remove it for good :)
The current logic assumes that everything that isn't a derivation is a
store path, but it can also be something that's *coercible* to a store
path, like a flake input.
Unnecessary uses of `lib.toDerivation` result in errors in pure evaluation
mode when `builtins.storePath` is disabled.
Also document what a `package` is.
MIPS has a large space of {architecture,abi,endianness}; this commit
adds all of them to lib/systems/platforms.nix so we can be done with
it.
Currently lib/systems/inspect.nix has a single "isMips" predicate,
which is a bit ambiguous now that we will have both mips32 and mips64
support, with the latter having two ABIs. Let's add four new
predicates (isMips32, isMips64, isMips64n32, and isMips64n64) and
treat the now-ambiguous isMips as deprecated in favor of the
more-specific predicates. These predicates are used mainly for
enabling/disabling target-specific workarounds, and it is extremely
rare that a platform-specific workaround is needed, and both mips32
and mips64 need exactly the same workaround.
The separate predicates (isMips64n32 and isMips64n64) for ABI
distinctions are, unfortunately, useful. Boost's user-scheduled
threading (used by nix) does does not currently supports mips64n32,
which is a very desirable ABI on routers since they rarely have
more than 2**32 bytes of DRAM.
This reverts commit 6b077c47ff.
Thanks Infinisil for discovering this problem:
> After a lot of trial and error, trying to prove why fixupOptionType should
> be used here or not, I figured it out: It's needed for the sake of file
> locations in error messages.
... where a bare submodule is an option that has a type like
`submoduleWith x`, as opposed to `attrsOf (submoduleWith x)`.
This makes migration unnecessary when introducing a freeform type
in an existing option tree.
Closes#146882
This ensures that the module file locations are propagated to the
freeform type, which makes it so that submodules in freeform types now
have their declaration location shown in the manual, fixing
https://github.com/NixOS/nixpkgs/issues/132085.
In addition, this also newly allows freeformTypes to be declared
multiple times and all declarations being merged together according to
normal option merging.
This also removes some awkwardness regarding the type of `freeformType`
This type correctly merges multiple option types together while also
annotating them with file information. In a future commit this will be
used for `_module.freeformType`
Makes all options rendered in the manual throw an error if they don't
have a type specified.
This is a follow-up to #76184
Co-Authored-By: Silvan Mosberger <contact@infinisil.com>
Change comment type so than nixdoc picks them up into Nixpkgs manual.
Also improve phrasing a bit and move stuff around so that it is formatted better.
This is a squashed commit. These are the original commit messages:
lib/option: Improve comment
better comment
Update documentation
Updated nixos/doc/manual/development/options-declarations.md with info on mkEnableOption and mkPackageOption.
Updated the comment on mkEnableOption in lib/options.nix
remove trailing whitespace
nixos/doc/option-declarations: Update IDs & formatting
nixos/docs/option-declarations: Escape angle brackets
Build DB from MD
(Amended) Fix typo
Co-authored-by: pennae <82953136+pennae@users.noreply.github.com>
(Amended) Build DB from MD (again)
This is a squashed commit. These are the original commit messages:
lib/options: Add mkPackageOption
lib/options: Add missing semicolon
lib/options.nix: Make mkPackageOption more complicated
lib/options: Fix indent. & spacing
lib/options.nix: Remove example and align comment
lib/options: ravenous overuse of arguments
lib/options: Format better
lib/options: Add default examplePath decl
lib/options: Make better mkPackageOption function
lib/options: Remove trailing whitespace
lib/options: Improve mkPackageOptions
lib/options: Remove pkgs prefixing
Co-authored-by: pennae <82953136+pennae@users.noreply.github.com>
lib/options: Slim down mkPackageOption further
lib/options: mkPackageOption: Add "pkgs." to example
lib/options: mkPackageOption: Make name & pkgs single arguments
lib/options: mkPackageOption: Swap name & pkgs
lib/options: Remove unnecessary import
Co-authored-by: pennae <82953136+pennae@users.noreply.github.com>
Allow a \n character at the end of the string and remove it during the
merge function.
An option of this type will resolve to the value "foo" whether it is set
to "foo" or "foo\n".
This is useful when using 'builtins.readFile' or ''-strings, which might
add an unintended newline (for example, bash trim the final newline from
a subshell).
`assert` has the annoying property that it dumps a lot of code at the
user without the built in capability to display a nicer message. We have
worked around this using `assertMsg` which would *additionally* display
a nice message. We can do even better: By using `throw` we can make
evaluation fail before assert draws its conclusions and prevent it from
displaying the code making up the assert condition, so we get the nicer
message of `throw` and the syntactical convenience of `assert`.
Before:
nix-repl> python.override { reproducibleBuild = true; stripBytecode = false; }
trace: Deterministic builds require stripping bytecode.
error: assertion (((lib).assertMsg (reproducibleBuild -> stripBytecode)) "Deterministic builds require stripping bytecode.") failed at /home/lukas/src/nix/nixpkgs/pkgs/development/interpreters/python/cpython/2.7/default.nix:45:1
After:
nix-repl> python.override { reproducibleBuild = true; stripBytecode = false; }
error: Deterministic builds require stripping bytecode.
Add a new type, inheriting 'types.str' but checking whether the value
doesn't contain any newline characters.
The motivation comes from a problem with the
'users.users.${u}.openssh.authorizedKeys' option.
It is easy to unintentionally insert a newline character at the end of a
string, or even in the middle, for example:
restricted_ssh_keys = command: keys:
let
prefix = ''
command="${command}",no-pty,no-agent-forwarding,no-port-forwarding,no-X11-forwarding
'';
in map (key: "${prefix} ${key}") keys;
The 'prefix' string ends with a newline, which ends up in the middle of
a key entry after a few manipulations.
This is problematic because the key file is built by concatenating all
the keys with 'concatStringsSep "\n"', with result in two entries for
the faulty key:
''
command="...",options...
MY_KEY
''
This is hard to debug and might be dangerous. This is now caught at
build time.
most modules can be evaluated for their documentation in a very
restricted environment that doesn't include all of nixpkgs. this
evaluation can then be cached and reused for subsequent builds, merging
only documentation that has changed into the cached set. since nixos
ships with a large number of modules of which only a few are used in any
given config this can save evaluation a huge percentage of nixos
options available in any given config.
in tests of this caching, despite having to copy most of nixos/, saves
about 80% of the time needed to build the system manual, or about two
second on the machine used for testing. build time for a full system
config shrank from 9.4s to 7.4s, while turning documentation off
entirely shortened the build to 7.1s.
- Eta reduce `mapAttrsRecursiveCond`, `foldAttrs`, `getAttrFromPath`.
- Modify `matchAttrs` to use `elemAt` instead of `head (tail xs)` to access
elements.
- Modify `matchAttrs` to use `any id` instead of `foldr and true`.
- Eta reduce formal arguments of `recursiveUpdate'.
- Access elements in `recursiveUpdateUntil` using `elemAt` and `head`
directly instead of `head (tail xs)` which copies a singleton unnecessarily.
(`elemAt` is used instead of `last` to save a primitive call to `length`,
this is possible because the 2-tuple structure is guranteed)
- Use `length` instead of comparison to empty list to save a copy.
the foldl is equivalent to a zip with concat. list concatenation in nix
is an O(n) operation, which makes this operation extremely inefficient
when large numbers of modules are involved.
this change reduces the number of list elements by 7 million on the
system used to write this, total memory spent on lists by 58MB, and
total memory allocated on the GC heap by almost 100MB (with a similar
reduction in GC heap size). it's also slightly faster.
While it is a fact of life that aarch64-darwin is built on Hydra, it has
never formally been elevated from the Tier 7 state it was originally
assigned in RFC 0046. Since platform Tier status is not only
descriptive, but also normative, a consensus to commit to supporting
aarch64-darwin would need to be reached.
`builtins.currentSystem` is not available in pure eval. For this
particular test, we don't really care since it's all about generating
.drv files.
Fixes the following error:
$ nix flake check
warning: unknown flake output 'lib'
error: attribute 'currentSystem' missing
at /nix/store/8wvnlbjxlr90kq2qa6d9zjpj8rqkilr5-source/lib/tests/misc.nix:499:73:
498| let
499| deriv = derivation { name = "test"; builder = "/bin/sh"; system = builtins.currentSystem; };
| ^
500| in {
(use '--show-trace' to show detailed location informat
Move function spdxLicense, internally used in yarn2nix
to lib/meta.nix, and
rename to getLicenseFromSpdxId
A similar function is implemented in poetry2nix,
but the one originally in yarn2nix seems beter.
since it falls back to an license-like attrset
for mismatched case
instead of a plain string
