Now type checks the resulting function values and allows mkMerge and co.
Also indicates that the type check is done in the function body
Co-Authored-By: Robert Hensing <robert@roberthensing.nl>
Looks like these got left behind in the
kernelArch -> linuxArch migration.
Fixes:
* pkgsCross.powernv.linuxHeaders
* pkgsCross.riscv64.linuxHeaders
* pkgsCross.riscv32.linuxHeaders
and dependees
Immensely helpful when you want to see the changes a function makes to
its value as it passes through.
Example:
```
$ nix-instantiate --strict --eval -E '(with import ./lib; traceFnSeqN 2 "id" (x: x) { a.b.c = 3; })'
trace: {
fn = "id";
from = {
a = {
b = {…};
};
};
to = {
a = {
b = {…};
};
};
}
{ a = { b = { c = 3; }; }; }
```
This reverts commit 4ff1ab5a56.
We need this to type options like:
services.xserver.windowManager.xmonad.extraPackages that specify functions that
take an attribute set containing packages / plugins and return a list containing
a selection of the values in this set.
The reason we need a dedicated type for this is to have the correct merge
behaviour. Without the functionTo type merging multiple function option
definitions results in an evaluation error. The functionTo type merges
definitions by returning a new function that applies the functions of all the
definitions to the given input and merges the result.
(cherry picked from commit 7ed41ff5e7)
The `platform` field is pointless nesting: it's just stuff that happens
to be defined together, and that should be an implementation detail.
This instead makes `linux-kernel` and `gcc` top level fields in platform
configs. They join `rustc` there [all are optional], which was put there
and not in `platform` in anticipation of a change like this.
`linux-kernel.arch` in particular also becomes `linuxArch`, to match the
other `*Arch`es.
The next step after is this to combine the *specific* machines from
`lib.systems.platforms` with `lib.systems.examples`, keeping just the
"multiplatform" ones for defaulting.
For renames like
mkAliasOptionModule [ "services" "compton" ] [ "services" "picom" ]
where the target is an option set (like services.picom) instead of a single
option (like services.picom.enable), previously the renamed option type
was unset, leading to it being `types.unspecified`.
This changes it to be `types.submodule {}` instead, which makes more
sense.
Since 40e7be1 all ARM platforms that didn't have a parsed cpu version
(e.g. arm-none-eabi) would be handled as armv7l-hf-multiplatform which
did break building arm-trusted-platform packages for some targets (e.g.
rk3399).
Using pcBase as fallback, instead of armv7l-hf-multiplatform,
corresponds with the behaviour we had before 40e7be1.
We recently switched to more explicit GPL license names in line
with the SPDX change and GNU Foundation recommendations:
https://www.gnu.org/licenses/identify-licenses-clearly.html
This followed up older change to use the recommended SPDX ID
18a5e8c36b
but using the `-only` variant for these deprecated licenses too
makes it harder to check for them automatically.
Let’s switch to the appropriate SPDX ID again.
Previously the .enable option was used to encode the condition as well,
which lead to some oddness:
- In order to encode an assertion, one had to invert it
- To disable a check, one had to mkForce it
By introducing a separate .check option this is solved because:
- It can be used to encode assertions
- Disabling is done separately with .enable option, whose default can be
overridden without a mkForce
Previously this option was thought to be necessary to avoid infinite
recursion, but it actually isn't, since the check evaluation isn't fed
back into the module fixed-point.
This implements assertions/warnings supported by the module system directly,
instead of just being a NixOS option (see
nixos/modules/misc/assertions.nix).
This has the following benefits:
- It allows cleanly redoing the user interface. The new
implementation specifically allows disabling assertions or
converting them to warnings instead.
- Assertions/warnings can now be thrown easily from within
submodules, which previously wasn't possible and needed workarounds.
Initially https://github.com/NixOS/nixpkgs/pull/82897 prevented
non-visible options from being rendered in the manual, but
visible-but-internal options were still being recursed into. This fixes
this, aligning the recurse condition here with the one in
make-options-doc/default.nix
The last use of `kernelMajor` in Nixpkgs was removed in 2018.
Even then, I'm not positive it was actually in an exercised code path.
AFAIUI this is now totally redundant and useless as it really was meant
for the 2.4 -> 2.6 transition.
split comes from builtins, not lib.
error: attribute 'split' missing, at /nix/path/nixpkgs/lib/sources.nix:4:4
(use '--show-trace' to show detailed location information)
I think there was a silent (i.e. semantic) merge conflict between PR #101139 and
PR #100456. This commit should fix the error, which manifests as follows:
error: undefined variable 'boolToString' at /home/kkini/src/nixpkgs/lib/types.nix:552:42
Nix can perform static scope checking, but whenever code is inside
a `with` expression, the analysis breaks down, because it can't
know statically what's in the attribute set whose attributes were
brought into scope. In those cases, Nix has to assume that
everything works out.
Except it doesnt. Removing `with` from lib/ revealed an undefined
variable in an error message.
If that doesn't convince you that we're better off without `with`,
I can tell you that this PR results in a 3% evaluation performance
improvement because Nix can look up local variables by index.
This adds up with applications like the module system.
Furthermore, removing `with` makes the binding site of each
variable obvious, which helps with comprehension.
Add a friendly function to easily return a flattened list of files
within a directory.
This is useful if you want to easily iterate or concatSep the list of
files all found within a directory.
(i.e. when constructing Java's CLASSPATH)
Style improvements
Co-authored-by: Silvan Mosberger <github@infinisil.com>
Previously if `_file` was specified by a module:
trace: warning: The type `types.string' of option `foo' defined in `/nix/store/yxhm2il5yrb92fldgriw0wyqh2kk9qyc-bug.nix' is deprecated. See https://github.com/NixOS/nixpkgs/pull/66346 for better alternative types.
With this change:
trace: warning: The type `types.string' of option `foo' defined in `/home/infinisil/src/nixpkgs/bug.nix' is deprecated. See https://github.com/NixOS/nixpkgs/pull/66346 for better alternative types.
If multiple definitions are passed, this evaluates them all as if they
were the only one, for a better error message. In particular this won't
show module-internal properties like `_type = "override"` and co.
- These symbols can be confusing for those not familiar with them
- There's no harm in making these more obvious
- Terminals may not print them correctly either
Also changes the function argument printing slightly to be more obvious
This new type has unsurprising merge behavior: Only attribute sets are
merged together (recursively), and only if they don't conflict.
This is in contrast to the existing types:
- types.attrs is problematic because later definitions completely
override attributes of earlier definitions, and it doesn't support
mkIf and co.
- types.unspecified is very similar to types.attrs, but it has smart
merging behavior that often doesn't make sense, and it doesn't support
all types
The vision here is that configuration tools can generate .json or .toml
files, which can be plugged into an existing configuration.
Eg:
{ lib, ... }:
{
imports = [
(lib.modules.importJSON ./hardware-configuration.json)
];
}
Jasper has been marked insecure for a while, and upstream has not
been responsive to CVEs for over a year.
Fixes#55388.
Signed-off-by: David Anderson <dave@natulte.net>
Previously the only way to deprecate a type was using
theType = lib.warn "deprecated" (mkOptionType ...)
This caused the warning to be emitted when the type was evaluated, but
the error didn't include which option actually used that type.
With this commit, types can specify a deprecationMessage, which when
non-null, is printed along with the option that uses the type
> NOTE: This function is not performant and should be avoided.
It's not used at all in-tree now, so we can remove it completely after
any remaining users are given notice.
An easy-to-make mistake when declaring e.g. a submodule is the accidental
confusion of `options` and `config`:
types.submodule {
config = {
foo = mkOption { /* ... */ };
};
}
However the error-message
The option `[definition 1-entry 1].foo' defined in `<expr.nix>' does not exist.
is fairly unhelpful because it seems as the options are declared at the
first sight. In fact, it took a colleague and me a while to track down such
a mistake a few days ago and we both agreed that this should be somehow caught
to save the time we spent debugging the module in question.
At first I decided to catch this error in the `submodules`-type directly
by checking whether `options` is undeclared, however this becomes fairly
complicated as soon as a submodule-declaration e.g. depends on existing
`config`-values which would've lead to some ugly `builtins.tryExec`-heuristic.
This patch now simply checks if the option's prefix has any options
defined if a point in evaluation is reached where it's clear that the
option in question doesn't exist. This means that this patch doesn't
change the logic of the module system, it only provides a more detailed
error in certain cases:
The option `[definition 1-entry 1].foo' defined in `<expr.nix>' does not exist.
However it seems as there are no options defined in [definition 1-entry 1]. Are you sure you've
declared your options properly? This happens if you e.g. declared your options in `types.submodule'
under `config' rather than `options'.
The refactoring in fd75dc8765
introduced a mistake in the error message that doesn't show the full
context anymore. E.g. with this module:
options.foo.bar = lib.mkOption {
type = lib.types.submodule {
baz = 10;
};
default = {};
};
You'd get the error
The option `baz' defined in `/home/infinisil/src/nixpkgs/config.nix' does not exist.
instead of the previous
The option `foo.bar.baz' defined in `/home/infinisil/src/nixpkgs/config.nix' does not exist.
This commit undoes this regression
Submodules that have a freeform type set behave as if that was the type
of the option itself (for values that don't have an option). Since the
submodules options are shown as separate entries in the manual, it makes
sense to show the freeform type as the submodule type.
For programs that have a lot of (Nix-representable) configuration options,
a simple way to represent this in a NixOS module is to declare an
option of a type like `attrsOf str`, representing a key-value mapping
which then gets generated into a config file. However with such a type,
there's no way to add type checking for only some key values.
On the other end of the spectrum, one can declare a single separate
option for every key value that the program supports, ending up with a module
with potentially 100s of options. This has the benefit that every value
gets type checked, catching mistakes at evaluation time already. However
the disadvantage is that the module becomes big, becomes coupled to the
program version and takes a lot of effort to write and maintain.
Previously there was a middle ground between these two
extremes: Declare an option of a type like `attrsOf str`, but declare
additional separate options for the values you wish to have type
checked, and assign their values to the `attrsOf str` option. While this
works decently, it has the problem of duplicated options, since now both
the additional options and the `attrsOf str` option can be used to set a
key value. This leads to confusion about what should happen if both are
set, which defaults should apply, and more.
Now with this change, a middle ground becomes available that solves above
problems: The module system now supports setting a freeform type, which
gets used for all definitions that don't have an associated option. This
means that you can now declare all options you wish to have type
checked, while for the rest a freeform type like `attrsOf str` can be
used.
This fundamentally changes how the module evaluation internally
handles definitions without an associated option.
Previously the values of these definitions were discarded and only
the names were propagated. This was fine because that's all that's
needed for optionally checking whether all definitions have an
associated option with _module.check.
However with the upcoming change of supporting freeform modules,
we *do* need the values of these.
With this change, the module evaluation cleanly separates definitions
that match an option, and ones that do not.
`toHex` converts the given positive integer to a string of the hexadecimal
representation of that integer. For example:
```
toHex 0 => "0"
toHex 16 => "10"
toHex 250 => "FA"
```
`toBase base i` converts the positive integer `i` to a list of it
digits in the given `base`. For example:
```
toBase 10 123 => [ 1 2 3 ]
toBase 2 6 => [ 1 1 0 ]
toBase 16 250 => [ 15 10 ]
```
The previous hash was too short and caused evaluation-time errors like:
invalid SRI hash 'sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA='
Additionally, since the fact that this is broken implies that nobody
could have been using it, "SRI" is a bit of a vague and obscure term,
`fakeSriHash` would be somewhat of a mouthful, and the relevant fetcher
parameters are just called `hash`, rename it to `fakeHash`.
This is used in in the manual generation for option identifiers that can
be linked. This, unike what the example describes, doesn't preserve
quotes which is needed for these identifiers to be valid.
This reverts commit 124cccbe3b.
124cccbe3b
broke the build of NixOS manual.
It does not make sense to be as strict as with attributes since we
are not limited by the CLI's inability to handle numbers.
Placeholders should not be quoted either as they are not part of Nix
syntax but a meta-level construct.
fix: Adding libtool to allow darwin compiles
Libtool seems to be required for mongodb to compile on darwin.
fix: Marking MongoDB as broken on aarch64
fix: Adding libtools to the pkg imports
Update mongodb to 4.0.4
Currently, not providing `name` to `cleanSourceWith` will use the name
of the imported directory. However, a common case is for this to be the
top level of some repository. In that case, the name will be the name of
the checkout on the current machine, which is not necessarily
reproducible across different settings, and can lead to e.g. cache
misses in CI.
This is documented in the comment on `cleanSourceWith`, but this does
not stop it being a subtle trap for users.
There are different tradeoffs in each case:
1. If `cleanSourceWith` defaults to `"source"`, then we may end up with a
user not knowing what directory a source store path corresponds to.
However, it being called "unnamed" may give them a clue that there is a
way for them to name it, and lead them to the definition of the
function, which has a clear `name` parameter.
2. If `cleanSoureWith` defaults to the directory name, then a user may face
occasional loss of caching, which is hard to notice, and hard to track
down. Tracking it down likely requires use of more advanced tools like
`nix-diff`, and reading the source of a lot of nix code.
I think the downside of the status quo is worse.
This is really another iteration of
https://github.com/NixOS/nix/issues/1305: that led to adding the `name`
argument in the first place, this just makes us use a better default
`name`.
The _module option is added as an internal option set, and it messes up
the results of module evaluations, requiring people to manually filter
_modules out.
If people depend on this, they can still use config._module from inside
the modules, exposing _module as an explicitly declared user option. Or
alternatively with the _module attribute now returned by evalModules.
newlib is the default for most tools when no kernel is provided. Other
exist, but this seems like a safe default.
(cherry picked from commit 8009c20711)
This helps with troubleshooting exceptions in config values, which were hard
to track down for options with many definitions.
The trace will look like:
error: while evaluating the attribute 'config.foo' at undefined position:
[...]
while evaluating the option `foo':
[...]
while evaluating definitions from `/home/user/mymod.nix':
while evaluating 'dischargeProperties' at /home/user/nixpkgs/lib/modules.nix:464:25, called from /home/user/nixpkgs/lib/modules.nix:392:137:
while evaluating the attribute 'value' at /home/user/nixpkgs/lib/modules.nix:277:44:
Value error!
where the `/home/user/mymod.nix` module is
{ lib, ... }: {
options.foo = lib.mkOption {
type = lib.types.lines;
};
config.foo = builtins.throw "Value error!";
}
In 87a19e9048 I merged staging-next into master using the GitHub gui as intended.
In ac241fb7a5 I merged master into staging-next for the next staging cycle, however, I accidentally pushed it to master.
Thinking this may cause trouble, I reverted it in 0be87c7979. This was however wrong, as it "removed" master.
This reverts commit 0be87c7979.
I merged master into staging-next but accidentally pushed it to master.
This should get us back to 87a19e9048.
This reverts commit ac241fb7a5, reversing
changes made to 76a439239e.
Let’s call them by what they are, option names.
`generators.mkValueStringDefault` is a better value string renderer
than plain `toString`.
Also add docs to all options.
The semantic difference between `encode` and `to` is not apparent.
Users are likely to confuse both functions (which leads to unexpected
error messages about the wrong types). Like in `generators.nix`, all
functions should be prefixed by `to`.
Furthermore, converting to a string depends on the target context. In
this case, it’s a POSIX shell, so we should name it that (compare
`escapeShellArg` in `strings.nix`).
We can later add versions that escape for embedding in e.g. python
scripts or similar.
lib/cli is very similar to generators, so it should follow largely the
same interface. Similar to how generators isn’t exported, we should
also namespace cli by default (plus “cli” is only three characters to
type).
Before c9214c394b and
9d396d2e42 if .git is symlink the version
would gracefully default to no git revision. With those changes an
exception is thrown instead.
This introduces a new function `pathIsGitRepo` that checks if
`commitIdFromGitRepo` fails without error so we don't have to
reimplement this logic again and can fail gracefully.
Not all modules use name attribute as the name of the submodule, for example,
environment.etc uses target. We will need to maintain a list of exceptions.
lib.commitIdFromGitRepo now resolves the refs from the
parent repository in case the supplied path is a file
containing the path to said repository. this adds support
for git-worktree and things alike. see gitrepository-layout(5).
this also:
- adds a new boolean function lib.pathIsRegularFile to
check whether a path is a regular file
- patches lib.revisionWithDefault and
the revision and versionSuffix attributes in
config.system.nixos in order to support git-worktrees
The standard attrsOf is strict in its *values*, meaning it's impossible to
access only one attribute value without evaluating all others as well.
lazyAttrsOf is a version that doesn't have that problem, at the expense
of conditional definitions not properly working anymore.
Without this change, accessing `mergedValue` from `mergeDefinitions` in
case there are no definitions will throw an error like
error: evaluation aborted with the following error message: 'This case should never happen.'
This change makes it throw the appropriate error
error: The option `foo' is used but not defined.
This is fully backwards compatible.
Fix the broken test in https://github.com/NixOS/nixpkgs/pull/77416
Apparently hydra uses `nix-build lib/tests/release.nix` to run all
tests, where IFD isn't allowed. Fortunately we can get around this with
builtins.toFile, which doesn't require IFD, but still can test the
properties we want.
This fixes imports from the store not being possible, which was caused by
https://github.com/NixOS/nixpkgs/pull/76857
E.g. such a case:
imports = [ "${home-manager}/nixos" ];
With this change, disabledModules applies recursively, meaning if you
have a module "foo.nix" with
imports = [ ./bar.nix ];
then setting
disabledModules = [ "foo.nix" ];
will disable both "foo.nix" and "bar.nix", whereas previously only
"foo.nix" would be disabled.
This change along with https://github.com/NixOS/nixpkgs/pull/61570 allows
modules to be fully disabled even when they have some `mkRenamedOption`
imports.
Previously when this function was called without a value coercible to a
string it would throw an error instead of returning false. Now it does.
As a result this now allows the use of a type like `either path attrs`
without it erroring out when a definition is an attribute set.
The warning about there not being a isPath primop was removed because
this is not the case anymore, there is builtins.isPath. But also there
always was `builtins.typeOf x == "path"` that could've been used
instead. However the path type now stands for more than just path types,
but absolute paths in general.
If I understand correctly, the problem isn't so much that you're assigning to
that top-level attribute, but that the assignment to the attribute (or any
child of the attribute) introduces the 'config' object and prevents 'lifting'
all settings to a generated 'config' object.
This reverts commit eec83d41e3.
This broke hydra evaluation because with this commit submodule values
are allowed to be paths, however the certmgr module uses `either
(submodule ...) path` in its type, meaning it already used paths for
something else which would now be interpreted as a submodule.
This adds a new utility to intelligently convert Nix records to
command line options to reduce boilerplate for simple use cases and to
also reduce the likelihood of malformed command lines
`pipe` is a useful operator for creating pipelines of functions.
It works around the usual problem of e.g. string operations becoming
deeply nested functions.
In principle, there are four different ways this function could be
written:
pipe val [ f1 .. fn ]
pipe val [ fn .. f1 ]
compose [ f1 .. fn ] val
compose [ fn .. f1 ] val
The third and fourth form mirror composition of functions, they would
be the same as e.g. `(f1 << f2 << f3 .. << fn) val`.
However, it is not clear which direction the list should have (as one
can see in the second form, which is the most absurd.
In order not to confuse users, we decide for the most “intuitive”
form, which mirrors the way unix pipes work (thus the name `pipe`).
The flow of data goes from left to right.
Co-Authored-By: Silvan Mosberger <infinisil@icloud.com>
Remove the "version" parameter in order to make it more widely
available.
Starts making some kernel configuration helpers available.
The intent is to be able to better build and check the linux kernel
configuration.
