Closes#168327
The issue reported there can be demonstrated with the following
expression:
→ nix-instantiate --eval -E "with import ./. {}; pkgs.lib.options.showDefs [ { file = \"foo\"; value = pkgs.rust.packages.stable.buildRustPackages; } ]"
error: attempt to call something which is not a function but a string
at /home/ma27/Projects/nixpkgs/lib/trivial.nix:442:35:
441| isFunction = f: builtins.isFunction f ||
442| (f ? __functor && isFunction (f.__functor f));
| ^
443|
Basically, if a `__functor` is in an attribute-set at depth-limit,
`__functor` will be set to `"<unevaluated>"`. This however breaks
`lib.isFunction` which checks for a `__functor` by invoking `__functor`
with `f` itself.
The same issue - "magic" attributes being shadowed by `withRecursion` -
also applies to others such as
`__pretty`/`__functionArgs`/`__toString`.
Since these attributes have a low-risk of causing a stack overflow
(because these are flat attr-sets or even functions), ignoring them in
`withRecursion` seems like a valid solution.
Consider a derivation a value to be serialized.
nix-repl> lib.generators.toGitINI { hello = { drv = pkgs.hello; }; }
error: evaluation aborted with the following error message: 'generators.mkValueStringDefault: attrsets not supported: <derivation /nix/store/533q15q67sl6dl0272dyi7m7w5pwkkjh-hello-2.10.drv>'
Fixes#137390
As suggested in #131205.
Now it's possible to pretty-print a value with `lib.generators` like
this:
with lib.generators;
toPretty { }
(withRecursion { depthLimit = 10; } /* arbitrarily complex value */)
Also, this can be used for any other pretty-printer now if needed.
When having e.g. recursive attr-set, it cannot be printed which is
solved by Nix itself like this:
$ nix-instantiate --eval -E 'let a.b = 1; a.c = a; in builtins.trace a 1'
trace: { b = 1; c = <CYCLE>; }
1
However, `generators.toPretty` tries to evaluate something until it's
done which can result in a spurious `stack-overflow`-error:
$ nix-instantiate --eval -E 'with import <nixpkgs/lib>; generators.toPretty { } (mkOption { type = types.str; })'
error: stack overflow (possible infinite recursion)
Those attr-sets are in fact rather common, one example is shown above, a
`types.<type>`-declaration is such an example. By adding an optional
`depthLimit`-argument, `toPretty` will stop evaluating as soon as the
limit is reached:
$ nix-instantiate --eval -E 'with import ./Projects/nixpkgs-update-int/lib; generators.toPretty { depthLimit = 2; } (mkOption { type = types.str; })' |xargs -0 echo -e
"{
_type = \"option\";
type = {
_type = \"option-type\";
check = <function>;
deprecationMessage = null;
description = \"string\";
emptyValue = { };
functor = {
binOp = <unevaluated>;
name = <unevaluated>;
payload = <unevaluated>;
type = <unevaluated>;
wrapped = <unevaluated>;
};
getSubModules = null;
getSubOptions = <function>;
merge = <function>;
name = \"str\";
nestedTypes = { };
substSubModules = <function>;
typeMerge = <function>;
};
}"
Optionally, it's also possible to let `toPretty` throw an error if the
limit is exceeded.
Previously, if a derivation without a `drvPath` was handled, an error
would be thrown:
nix-repl> lib.generators.toPretty {} { type = "derivation"; }
error: attribute 'drvPath' missing, at /home/infinisil/src/nixpkgs/lib/generators.nix:251:24
With this commit it doesn't anymore:
nix-repl> lib.generators.toPretty {} { type = "derivation"; }
"<derivation ???>"
This matches what `nix repl` outputs:
nix-repl> { type = "derivation"; }
«derivation ???»
This reverts commit d9a7d03da8.
Reason for this is that it actually doesn't migitate the issue on nix
stable for another reason: builtins.tryEval doesn't prevent the error
generated by builtins.functionArgs from halting evaluation:
> builtins.tryEval (builtins.functionArgs builtins.functionArgs)
error: 'functionArgs' requires a function, at (string):1:19
Thus it seems that there is no workaround to make
lib.generators.toPretty work with nix stable and primops since there is
no way to distinguish between primops and lambdas in nix.
An high level example case of this problem occuring can be found below:
nix-repl> lib.generators.toPretty {} (lib.concatStringsSep "\n")
error: 'functionArgs' requires a function, at /home/lukas/src/nix/nixpkgs/lib/trivial.nix:334:42
However this does not happen on other partially applied functions:
nix-repl> lib.generators.toPretty {} (lib.concatMapStringsSep "\n")
"<function>"
The issue, as it turns out is that while builtins are functions,
builtins.functionArgs throws if is passed a builtin or a partially
applied builtin:
nix-repl> lib.generators.toPretty {} builtins.toString
error: 'functionArgs' requires a function, at /home/lukas/src/nix/nixpkgs/lib/trivial.nix:334:42
nix-repl> lib.generators.toPretty {} (builtins.foldl' (a: b: a + b))
error: 'functionArgs' requires a function, at /home/lukas/src/nix/nixpkgs/lib/trivial.nix:334:42
I'm pretty sure this qualifies as a nix bug and should be filed
accordingly, but we can work around it in lib.generators.toPretty by
using tryEval and falling back to {} which functionArgs _should_ return
for builtins.
The nix behavior is inconsistent to say the least:
nix-repl> builtins.functionArgs builtins.functionArgs
error: 'functionArgs' requires a function, at (string):1:1
nix-repl> builtins.typeOf builtins.functionArgs
"lambda"
builtins.functionArgs (a: 1 + a)
{ }
nix-repl> builtins.typeOf (a: 1 + a)
"lambda"
- These symbols can be confusing for those not familiar with them
- There's no harm in making these more obvious
- Terminals may not print them correctly either
Also changes the function argument printing slightly to be more obvious
Nix now supports floats & we can pretty easily map them to Plist’s
<real></real> type. Note that I am unsure how this affects older
version of Nix that may or may not have builtins.isFloat available.
Make sure this satisfies minver.nix’s "1.11" requirement.
So far, `mkValueString` defaulted to `toString`,
which is a bad match for most configuration file formats,
especially because how booleans are formatted.
This also improves error messages for unsupported types.
Add a test to codify the formatting.
Among other things, this will allow *2nix tools to output plain data
while still being composable with the traditional
callPackage/.override interfaces.
This means the generation of values can now be influenced, even down from e.g.
an INI generator.
Breaks the interface of `mkKeyValueDefault` to match its interface to other
generator functions. It might me sensible to rename `mkKeyValue` and
`mkKeyValueSet` to conform to the `toX`-style of generator functions.
This does break the API of being able to import any lib file and get
its libs, however I'm not sure people did this.
I made this while exploring being able to swap out docFn with a stub
in #2305, to avoid functor performance problems. I don't know if that
is going to move forward (or if it is a problem or not,) but after
doing all this work figured I'd put it up anyway :)
Two notable advantages to this approach:
1. when a lib inherits another lib's functions, it doesn't
automatically get put in to the scope of lib
2. when a lib implements a new obscure functions, it doesn't
automatically get put in to the scope of lib
Using the test script (later in this commit) I got the following diff
on the API:
+ diff master fixed-lib
11764a11765,11766
> .types.defaultFunctor
> .types.defaultTypeMerge
11774a11777,11778
> .types.isOptionType
> .types.isType
11781a11786
> .types.mkOptionType
11788a11794
> .types.setType
11795a11802
> .types.types
This means that this commit _adds_ to the API, however I can't find a
way to fix these last remaining discrepancies. At least none are
_removed_.
Test script (run with nix-repl in the PATH):
#!/bin/sh
set -eux
repl() {
suff=${1:-}
echo "(import ./lib)$suff" \
| nix-repl 2>&1
}
attrs_to_check() {
repl "${1:-}" \
| tr ';' $'\n' \
| grep "\.\.\." \
| cut -d' ' -f2 \
| sed -e "s/^/${1:-}./" \
| sort
}
summ() {
repl "${1:-}" \
| tr ' ' $'\n' \
| sort \
| uniq
}
deep_summ() {
suff="${1:-}"
depth="${2:-4}"
depth=$((depth - 1))
summ "$suff"
for attr in $(attrs_to_check "$suff" | grep -v "types.types"); do
if [ $depth -eq 0 ]; then
summ "$attr" | sed -e "s/^/$attr./"
else
deep_summ "$attr" "$depth" | sed -e "s/^/$attr./"
fi
done
}
(
cd nixpkgs
#git add .
#git commit -m "Auto-commit, sorry" || true
git checkout fixed-lib
deep_summ > ../fixed-lib
git checkout master
deep_summ > ../master
)
if diff master fixed-lib; then
echo "SHALLOW MATCH!"
fi
(
cd nixpkgs
git checkout fixed-lib
repl .types
)
With `builtins.functionArgs` we can get some information if the first argument
is an attrset and whether the contained fields have default values. Encode that
into the pretty-printed lambda.
They both reference the toJSON builtin, so we get semantic identifiers
that express the intent of the generation.
Both should be able to map each nix value (minus functions) to the
destination config files.
Includes two invocation unit tests.
Many configurations are INI-style files. Attribute sets can be mapped
rather painlessly to the INI format.
This adds a function toINI inside a new generators library section.
Also, unit tests for the default values are provided.
