mirrors/nixpkgs
1
0
Fork 1
mirror of https://github.com/NixOS/nixpkgs.git synced 2024-11-18 03:30:45 +00:00
Code Issues Wiki Activity
489203 commits 266 branches 124 tags 6 GiB
Commit graph

875 commits

Author SHA1 Message Date
Robert Hensing f01fec4099 nixos/testing/network.nix: Fix specialisations onlyShorthand 2022-09-24 17:38:09 +01:00
Robert Hensing 9886db059a nixos/testing: Embrace callTest 
My conception of its input was wrong. It is quite a useful construct,
even if its name is a bit weird.
 2022-09-24 17:38:09 +01:00
Robert Hensing 5297d584bc nixos/lib/eval-config: Document the use of baseModules 2022-09-24 17:38:09 +01:00
Robert Hensing 57ac9dd3ae nixos/lib/build-vms.nix: Remove 
It has been replaced by the modular test framework in nixos/lib/testing.

If you are looking for a way to produce a VM-test-like configuration
outside of the test framework, use the nixos/lib/testing/nixos-test-base.nix
NixOS module, possibly in combination with { _module.args.nodes = .....; }.
 2022-09-24 17:38:09 +01:00
Robert Hensing 24d1d74e4e nixos/testing: Extract nixos-test-base.nix NixOS module 2022-09-24 17:38:09 +01:00
Robert Hensing 38fb09e427 testing-python.nix: Replace makeTest implementation 2022-09-24 17:38:08 +01:00
Robert Hensing b0c781cc41 nixos/testing: Move entrypoint to nixos/lib + doc 2022-09-24 17:38:08 +01:00
Robert Hensing 9e4277b970 nixos/testing/network.nix: Fix for specialisations and specialArgs.name 
Before this, it relied on being able to `imports` from the `name`
module argument, which wasn't really necessary and required a
potentially quite breaking change. We can revert that now.
 2022-09-21 10:55:12 +01:00
Robert Hensing a958a4aa00 nixos/testing: Add pkgs parameter 
This parameter is for packages to use in VMs, unlike hostPkgs.
 2022-09-21 10:55:11 +01:00
Robert Hensing 124b0c4abc nixos/testing/network.nix: Avoid deprecated .config 2022-09-21 10:55:11 +01:00
Robert Hensing b3de22483c nixos/testing-python.nix: Add evalTest 
This is a decomposition of the testing-python.nix and build-vms.nix
files into modules.

By refactoring the glue, we accomplish the following:

 - NixOS tests can now use `imports` and other module system features.
    - Network-wide test setup can now be reusable; example:
       - A setup with all VMs configured to use a DNS server
       - Split long, slow tests into multiple tests that import a
         common module that has most of the setup.
    - Type checking for the test arguments
    - (TBD) "generated" options reference docs
 - Aspects that had to be wired through all the glue are now in their
   own files.
    - Chief example: interactive.nix.
    - Also: network.nix

In rewriting this, I've generally stuck as close as possible to the
existing code; copying pieces of logic and rewiring them, without
changing the logic itself.

I've made two exceptions to this rule

 - Introduction of `extraDriverArgs` instead of hardcoded
   interactivity logic.

 - Incorporation of https://github.com/NixOS/nixpkgs/pull/144110
   in testScript.nix.

I might revert the latter and split it into a new commit.
 2022-09-21 10:55:11 +01:00
Robert Hensing 5b52d552cb nixos/make-options-doc: Explain docbook to markdown migration 2022-09-17 13:38:23 +01:00
pennae 767485a0de lib/options: deprecate docbook text and literalDocBook 
deprecate literalDocBook by adding a warning (that will not fire yet) to
its uses and other docbook literal strings by adding optional warning
message to mergeJSON.
 2022-09-10 18:23:13 +02:00
Jacek Galowicz 8ae3e986a7
Merge pull request #189999 from sbruder/test-driver-formatting-closed-port 
nixos/test-driver: fix formatting of closed port
 2022-09-06 14:48:13 +02:00
Simon Bruder 9caf5f6ba9
nixos/test-driver: fix formatting of closed port 
Co-authored-by: a-kenji <aks.kenji@protonmail.com>
 2022-09-06 14:32:56 +02:00
Alyssa Ross 488412a1db
nixos/test-driver: add wait_for_unit() timeout arg 
For example, the wait_for_unit() call in the Moodle test times out for
myself and others[1], so it would be good to be able to increase it to
something less likely to be hit by a test that would otherwise pass.

[1]: https://github.com/NixOS/nixpkgs/pull/177052#issue-1266336706
 2022-09-02 18:06:03 +00:00
pennae 3bddcf5f90
Merge branch 'master' into option-docs-md 2022-09-01 16:10:09 +02:00
pennae 9547123258 nixos/*: convert internal option descriptions to MD 
we'll have to do it eventually, may as well be now.
 2022-08-31 16:32:54 +02:00
pennae ef176dcf7e nixos/*: automatically convert option descriptions 
conversions were done using https://github.com/pennae/nix-doc-munge
using (probably) rev f34e145 running

    nix-doc-munge nixos/**/*.nix
    nix-doc-munge --import nixos/**/*.nix

the tool ensures that only changes that could affect the generated
manual *but don't* are committed, other changes require manual review
and are discarded.
 2022-08-31 16:32:53 +02:00
pennae 5a643387ec nixos/*: normalize more docbook for auto-conversion 2022-08-31 16:32:53 +02:00
pennae 9c3c13b50d nixos/make-options-doc: add inline roles for varname/envar 
both of these render distinctly from plain literals in the manpage, and
manpages even semantically distinguish between the two.
 2022-08-31 16:21:10 +02:00
Silvan Mosberger 6eb620ace7
Merge pull request #180222 from YorikSar/qemu-vm-darwin-pkgs 
nixos/qemu-vm: Allow to build and run VMs on Darwin
 2022-08-30 21:54:22 +02:00
pennae 65fd6f0774 nixos/make-options-doc: eat newlines in MD admonitions 
leaving some newlines around after an admonition was closed causes the
newline rule to match, which in turn inserts literallayout newlines into
te xml output. that's not what we want.
 2022-08-27 19:18:29 +02:00
Yuriy Taraday e0fd131310 nixos/qemu: Add QEMU arguments for HVF on Darwin 
Only support the same architechture for now, no emulation.

Related to #108984.
 2022-08-25 17:00:14 +04:00
Yuriy Taraday 8a25d56a6d nixos/qemu: nixpkgs-fmt nixos/lib/qemu-common.nix 2022-08-25 17:00:11 +04:00
Ivan Shapovalov ac18c55b56 make-system-tarball: use pixz -t 
Pass `-t` to pixz to prevent it from appending an index to the end of
the uncompressed stream, confusing tools such as `machinectl import-tar`.

Fixes: #187816
 2022-08-22 05:28:36 +03:00
Christian Kögler 5e7662d848
Merge pull request #167692 from peperunas/onsuccess 
nixos/systemd: Add onSuccess module option for units
 2022-08-19 09:17:14 +02:00
Robert Schütz 243053e521 python310Packages.mistune: 0.8.4 -> 2.0.4 
Remove mistune_0_8 because it's insecure.
 2022-08-15 06:53:01 +00:00
pennae af98bacbe0 Revert "nixos/docs: cache mergeJSON md conversion on baseOptionsJSON" 
This reverts commit 52b0ad17e3.

we only needed this because mergeJSON was slow, but in the interim we
found a better solution to the slowness.
 2022-08-05 17:13:47 +02:00
pennae 645cfa59ac nixos/make-option-docs: add xref support to markdown conversion 2022-08-03 22:01:14 +02:00
pennae 7a091b2686 nixos/make-options-doc: reuse markdown instance in mergeJSON 
this doesn't construct a new (expensive) parser for every option, making
rendering about 30x faster.
 2022-08-03 22:00:24 +02:00
pennae 6b13dd0e9e
Merge pull request #183491 from pennae/automatic-md-conversions 
treewide: automatically md-convert option descriptions
 2022-08-02 02:15:30 +02:00
Silvan Mosberger fe2c9ae814
Merge pull request #177012 from hercules-ci/nixos-modular-system 
`flake.lib.nixosSystem`: Allow `nixpkgs.system` to be set modularly; improve error message
 2022-08-01 15:05:16 +02:00
Alyssa Ross 5330c0a1af
treewide: use isAarch where appropriate 2022-07-30 16:18:27 +00:00
pennae 52b0ad17e3 nixos/docs: cache mergeJSON md conversion on baseOptionsJSON 
with ever more options being markdown rather than docbook the conversion
time is starting to become a significant factor of doc build time.
luckily we can pre-convert all nixos option docs to MD and cache the
result of this conversion, then merge the already-converted json file
with user option docs. we leave options.json unconverted to keep it as
close to the actual nix code as possible.
 2022-07-28 23:20:02 +02:00
pennae 18be724a58 nixos/make-options-doc: give MD conversion error locations 
during docs conversion it can be very useful to know exactly *where* the
error the script complained about is. the name of the option should be
sufficient since option merging is rather rare, and won't merge doc
attributes anyway.
 2022-07-28 20:03:02 +02:00
Philippe Schaaf 5ae6580474 add hub mode comment 
Signed-off-by: Philippe Schaaf <philippe.schaaf@secunet.com>
 2022-07-21 16:34:09 +02:00
Philippe Schaaf f6a290932e use vde switch in hubmode by default 
Within a dual VM test-setup a strange behaviour was observed.
The two VMs are connected via one vde_switch instance
(instancevirtualisation.vlans = [ 1 ]; IMO a bad attribute name for
switch instances, has nothing to do with VLANs in sense of 802.1Q).

A ping on the base interface (eth1) works, but not on VLAN
subinterfaces (vlan1@eth1). A tcpdump of eth1 includes the ARP requests
tagged with the subinterfaces VLAN ID, but responses seems not to pass
the vde_switch. This works fine if performed on the base interface.

Putting the vde_switch in hub mode results in flooding
traffic to all vde_switch ports. This results in a expected behaviour
and a ping on a VLAN subinterface works as expected.

Signed-off-by: Philippe Schaaf <philippe.schaaf@secunet.com>
 2022-07-21 11:38:03 +02:00
Robert Hensing 711e653a65 nixos/eval-config: Allow system to be set modularly when arg is null 2022-07-10 13:35:54 +02:00
Martin Weinelt c0d1de14b3
Merge pull request #179291 from dotlambda/utillinux-alias 
treewide: don't use utillinux alias
 2022-07-02 00:05:54 +02:00
Robert Hensing 7c81905344 nixos/make-options-doc: Support Nix-provided declaration locations 
Feature was introduced in https://github.com/NixOS/nixpkgs/pull/174460,
but wasn't supported in `mergeJSON.py` yet.
 2022-06-27 22:07:20 +02:00
Robert Hensing 8bff3fef40 nixos/make-options-doc: Support block quotes 
Our tooling would trip without the inner <para>, despite the docbook
docs suggesting that <para> occurs in <blockquote> and vice versa.
 2022-06-27 17:41:32 +02:00
Robert Hensing e04aa1bcd9 nixos/make-options-doc: Escape inline code and code blocks 2022-06-27 17:41:32 +02:00
Robert Hensing f900ed1749 nixos/make-options-doc: Support newline md node 
This occurs in the ast generated for blockquotes.
 2022-06-27 17:41:32 +02:00
Robert Hensing bccc3e747b nixos/make-options-doc: Fix exception handler for arity /= 1 methods 2022-06-27 17:41:32 +02:00
Robert Hensing aff2dbbc82 make-options-doc: Make variablelist id configurable 
I've tried XInclude set-xml-id first, but our tooling did not pick up on it.
 2022-06-27 17:41:32 +02:00
Robert Schütz 40990c3586 treewide: don't use utillinux alias 2022-06-27 04:55:23 +00:00
Robert Hensing fa9c83ca7f
Merge pull request #174460 from hercules-ci/module-docs-Nix-driven-location-links 
make-options-doc: Support Nix-provided declaration links
 2022-06-22 15:48:46 +02:00
Lassulus 773cbfbd28
Merge pull request #151633 from wucke13/sd-image-experiment 
honor `sdImage.compressImage` also in the build process
 2022-06-22 14:48:21 +02:00
Robert Hensing e2c261f2c0
Merge pull request #176146 from pennae/module-docs-markdown 
treewide: markdown option docs
 2022-06-21 13:16:02 +02:00
Jörg Thalheim f5ca531054
Merge pull request #158602 from hercules-ci/systemd-unit-mixed-list-error 
nixos/systemd: Custom error when mixing list/non-list defs
 2022-06-19 07:05:04 +01:00
Robert Hensing 2c8bbf33fd nixos/test-driver: Support mypy through regular mechanisms 2022-06-17 12:22:57 +02:00
Robert Hensing cee66a8cd5 make-options-doc: Support Nix-provided declaration links 
Previously, the location logic was hardcoded, supporting only
Nixpkgs and NixOps properly, leaving other uses of the module
system without good location support.
 2022-06-15 00:45:05 +02:00
pennae 320aa2a791 treewide: attempt at markdown option docs 2022-06-12 12:44:38 +02:00
Robert Hensing 9970c0b874
Merge pull request #174470 from hercules-ci/module-docs-visibility-override 
make-options-doc: Allow transformOptions to override visibility
 2022-06-10 14:27:58 +02:00
Robert Hensing bad676c7ed
Merge pull request #174441 from tljuniper/172325-nixostest-override-python-pkgs-additional-param 
nixos/test-driver: additional Python packages in test driver
 2022-06-05 20:46:01 +02:00
Bobby Rong 9d2a8e3190
nixos/test-driver: fix type hint for send_chars 2022-06-04 12:05:47 +08:00
Robert Hensing 793180cf55
Merge branch 'master' into 172325-nixostest-override-python-pkgs-additional-param 2022-06-03 14:22:13 +02:00
Michael Schneider fd4ebd8990 add opt out flag for type checking 2022-06-03 13:37:04 +02:00
Michael Schneider 814027378b nixos/test-driver: Typecheck TestScript 2022-06-02 10:05:45 +02:00
Robert Hensing 53326189ba
Merge pull request #174829 from blaggacao/fix/cleanup-nixpkgs-initial-system 
nixos: remove effect-less nixpgks.initialSystem
 2022-05-31 16:26:25 +02:00
tljuniper a99736e399 nixos/test-driver: add option to add extra python packages to test code 2022-05-29 11:22:23 +02:00
Alyssa Ross 1e396f9802
nixos/testing-python.nix: fix typo in warning 
"pyton" -> "python"
 2022-05-27 13:00:25 +00:00
David Arnold 646e214e11
nixos: remove effect-less nixpgks.initialSystem 
- initialSystem was keeping track of the evaluating system
- it had been used by `nesting.children`
- since, 20.09, `nesting.children` has been replaced with named
  specializations

It appears that this option was left over and not cleand up properly.
 2022-05-26 20:00:05 -05:00
Robert Hensing 75bc6da237 make-options-doc: Filter options after transformOptions 
This allows the user-supplied function to change the visibility
of options.
 2022-05-25 12:50:07 +02:00
Robert Hensing 90344e9f40
Merge pull request #171163 from hercules-ci/nixpkgs-config-doc 
Add generated `nixpkgs.config` doc to Nixpkgs manual
 2022-05-19 20:51:38 +02:00
Robert Hensing 0b02135d3b nixosOptionsDoc: refactor 
Thanks to Infinisil for pointing this out.
 2022-05-16 22:53:23 +02:00
Robert Hensing 062bc5e74a lib.types.functionTo: Add pseudo-attr to generated docs 2022-05-13 09:01:05 +02:00
Robert Hensing ae172a2bb4 treewide: nixosTest -> testers.nixosTest 2022-05-09 14:16:06 +02:00
Robert Hensing 28f99aad31 nixos/testing-python.nix: Set meta.mainProgram 2022-05-05 12:33:57 +02:00
Robert Hensing bb2c5a3684 nixosOptionsDoc: Make appendix tag optional 2022-05-01 21:51:19 +02:00
Patrick Hilhorst 976f00963f
nixos/test-driver: rewrite wait_before_entry to wait 2022-04-29 15:36:03 +02:00
Will Fancher 69d8047516 nixos: Fix up systemd shutdown ramfs 2022-04-28 14:12:14 -04:00
Patrick Hilhorst 1db0dcdfec
nixos/test-driver: add wait_before_entry to polling_condition 2022-04-27 16:23:33 +02:00
Patrick Hilhorst 57beeedbdf
nixos/test-driver: allow multiple entry of polling condition 2022-04-27 16:21:10 +02:00
Anna Gillert dbc95f15b8 nixos/test-driver: Avoid shell injection in machine.execute() 2022-04-21 10:54:00 +02:00
Anna Gillert f7e89a59da nixos/test-driver: fix missing shellopts in execute 
Without this fix, setting the shellopts in `machine.execute` is
inconsitent.  When no timeout is used, shellopts `set -euo pipefail` are
applied to the command as expected. When a timeout is specified, the
shellopts are not applied to the command itself (which is called inside
a `sh -c` that doesn't inherit the shellopts) but rather to the
`timeout` command, leading to the following full command:

```bash
(set -euo pipefail; timeout 900 sh -c 'cmd') | (base64 --wrap 0; echo)\n
```

With this fix, this is the command we get:

```bash
timeout 900 sh -c 'set -euo pipefail; false | true') | (base64 --wrap 0; echo)\n
```
 2022-04-21 10:54:00 +02:00
Graham Christensen d65eff4fb6
Merge pull request #167902 from DeterminateSystems/openstack-zfs-cleanup 
openstack-image: init, make-single-disk-zfs-image: init
 2022-04-16 19:44:53 -04:00
Jacek Galowicz 6b414c5b66
Merge pull request #168609 from rnhmjoj/pr-test-driver 
nixos/test-driver: highlight driver log lines
 2022-04-14 13:09:36 +02:00
rnhmjoj ca1b1f6dc0
nixos/test-driver: highlight driver log lines 
There is a whole lot of noise in a NixOS test log due to journal,
commands, and test driver messages all being mixing together.
With this commit the test driver messages are highlighted so you don't
have to squint too much to see where a subtest starts and ends or what
was the last command being run.

Here's an excerpt from a highlighted log:

    machine # [    6.647826] dhcpcd[668]: eth0: adding default route via 10.0.2.2
    machine # I've had a great day!
    (finished: must succeed: curl -sfL https://example.com/how-are-you? | tee /dev/stderr, in 0.19 seconds)
    (finished: subtest: Privoxy can filter https requests, in 0.22 seconds)
    subtest: Blocks are working
    machine: waiting for TCP port 443
    machine # Connection to localhost (::1) 443 port [tcp/https] succeeded!
    (finished: waiting for TCP port 443, in 0.03 seconds)
    machine: must fail: curl -f https://example.com/ads 1>&2
    machine # [    6.730608] nscd[742]: 742 monitored file `/etc/resolv.conf` was written to
    machine # [    6.733071] privoxy[814]: 2022-04-14 09:41:01.695 7fa995c39640 Actions: +block{Fake ads} ...
    machine # curl: (22) The requested URL returned error: 403
    machine # [    6.753098] systemd[1]: Stopping Name Service Cache Daemon...
    machine # [    6.755632] systemd[1]: nscd.service: Deactivated successfully.
    machine # [    6.757069] systemd[1]: Stopped Name Service Cache Daemon.
    (finished: must fail: curl -f https://example.com/ads 1>&2, in 0.07 seconds)
    machine: must succeed: curl -f https://example.com/PRIVOXY-FORCE/ads 1>&2
    machine # [    6.765099] systemd[1]: Starting Name Service Cache Daemon...
    machine # [    6.781519] nscd[925]: 925 monitoring file `/etc/passwd` (1)
    machine # [    6.844105] systemd[1]: Reached target Network is Online.
    machine # [    6.848860] privoxy[814]: 2022-04-14 09:41:01.805 7fa995c39640 Actions: +block{Fake ads} ...
    machine # [    6.856068] nscd[925]: 925 monitoring file `/etc/nsswitch.conf` (8)
    machine # [    6.858229] systemd[1]: Reached target Multi-User System.
    machine # [    6.858934] nscd[925]: 925 monitoring directory `/etc` (2)
    machine # [    6.861315] systemd[1]: Startup finished in 2.374s (kernel) + 4.451s (userspace) = 6.826s.
    machine # Hot Nixpkgs PRs in your area. Click here!
    machine # [    6.873055] nscd[925]: 925 monitoring directory `/etc` (2)
    machine # [    6.874489] nscd[925]: 925 monitoring file `/etc/nsswitch.conf` (8)
    machine # [    6.875599] nscd[925]: 925 monitoring directory `/etc` (2)
    (finished: must succeed: curl -f https://example.com/PRIVOXY-FORCE/ads 1>&2, in 0.12 seconds)
    (finished: subtest: Blocks are working, in 0.22 seconds)
    subtest: Temporary certificates are cleaned
    machine: must succeed: test $(ls /run/privoxy/certs | wc -l) -gt 0
    (finished: must succeed: test $(ls /run/privoxy/certs | wc -l) -gt 0, in 0.02 seconds)
    machine: must succeed: date -s "$(date --date '12 days')"
    (finished: must succeed: date -s "$(date --date '12 days')", in 0.02 seconds)
    machine # [    6.959589] systemd[1]: Started Logrotate Service.
    machine # [    6.966685] systemd[1]: Starting Cleanup of Temporary Directories...
    machine # [    6.974783] systemd[1]: logrotate.service: Deactivated successfully.
    machine # [    7.004493] systemd[1]: systemd-tmpfiles-clean.service: Deactivated successfully.
    machine # [    7.005457] systemd[1]: Finished Cleanup of Temporary Directories.
    machine: must succeed: test $(ls /run/privoxy/certs | wc -l) -eq 0
    (finished: must succeed: test $(ls /run/privoxy/certs | wc -l) -eq 0, in 0.01 seconds)
    (finished: subtest: Temporary certificates are cleaned, in 0.15 seconds)
    (finished: run the VM test script, in 7.44 seconds)
    test script finished in 7.48s
    cleanup
    kill machine (pid 8)
 2022-04-14 11:54:21 +02:00
Sandro a39694b41f
Merge pull request #168194 from SuperSandro2000/xserver-excludePackages 
Xserver exclude packages
 2022-04-13 11:37:32 +02:00
Sandro Jäckel 28539842d8
nixos/utils: move removePackagesByName to here from gnome 2022-04-11 15:42:49 +02:00
Lassulus f0848eeb05
Merge pull request #167316 from helsinki-systems/feat/opportunistic-kvm 
nixos/lib/qemu-common: Opportunistically use KVM
 2022-04-10 18:54:59 +01:00
Graham Christensen 8a5bdce566 make-single-disk-zfs-image: init 2022-04-07 15:28:28 -04:00
Graham Christensen a8f41adbb7 amazon-image: use make-multi-disk-zfs-image 2022-04-07 15:28:28 -04:00
Giulio De Pasquale fbc8dbb075 nixos/systemd: Add onSuccess module option for units 2022-04-07 13:25:52 +02:00
Robert Hensing c705953267
Merge pull request #165540 from Infinisil/module-args-docs 
lib/modules: Document `_module.args`
 2022-04-05 21:51:46 +02:00
Silvan Mosberger 25de2935ef lib/modules: Document _module.args 
Documents the _module.args option, motivated by many usages in Flakes,
especially with the deprecation of extraArgs
(78ada83361)

The documentation rendering for this option had to be handled a bit
specially, since it's not declared in nixos/modules like all the other
NixOS options.

Co-Authored-By: pennae <github@quasiparticle.net>
Co-Authored-By: Robert Hensing <robert@roberthensing.nl>
 2022-04-05 18:26:40 +02:00
lassulus ed18db011f make-disk-image: add systemdMinimal to PATH 
this is needed to setup /tmp inside the chroot. which is needed by
systemd-boot to sucessfully install
 2022-04-05 11:57:53 +02:00
Max Hausch a2465483e8
nixos/lib/qemu-common: Opportunistically use KVM 
This will try to use KVM if available and fallback to using TCG if not.
Useful for nested virtualisation of arm on x86.
 2022-04-05 11:49:42 +02:00
Janne Heß beb02229fb
nixos/systemd-unit-options: Fix indentation 2022-04-04 13:53:58 +01:00
Janne Heß 4b9efea255
nixos/stage-1-systemd: Implement job scripts 2022-04-04 11:44:31 +01:00
Janne Heß 7cdc4dd5d1
Merge pull request #164943 from ElvishJerricco/systemd-initrd-reuse-systemd-module 
initrd: Opt-in bare bones systemd-based initrd
 2022-04-03 15:53:02 +02:00
Janne Heß b7c62b8238
nixos/systemd-initrd: Remove unit options that don't work 2022-04-01 11:58:30 +02:00
Janne Heß 1e5261f31c
nixos/systemd-lib: Use module composition 2022-04-01 09:57:19 +02:00
Robert Hensing ecbf5ae27a nixosTest: Simplify doc by deprecating syntax sugar 2022-03-28 14:11:58 +02:00
Silvan Mosberger 5a67e9db61
Merge pull request #133532 from Infinisil/systemd-unit-dirs 2022-03-25 19:05:35 +01:00
Silvan Mosberger c70a466d21 nixos/systemd: Allow creation of unit directories 
This patch allows creation of files like
/etc/systemd/system/user-.slice.d/limits.conf with

    systemd.units."user-.slice.d/limits.conf" = {
      text = ''
        [Slice]
        CPUAccounting=yes
        CPUQuota=50%
      '';
    };

which previously threw an error

Also renames the systemd-unit-path test to sytsemd-misc, and extends it to
test that `systemd.units` can handle directories. In this case we make
sure that resource limits specified in user slices apply.
 2022-03-25 17:47:44 +01:00
Janne Heß 7d0e21c77e
nixos/test-runner: Allow writing to qemu stdin 2022-03-25 12:08:21 +01:00
Florian Klink fc91cdb5bc nixos/lib/systemd-lib.nix: move comment back down to packages 
https://github.com/NixOS/nixpkgs/pull/164943#discussion_r833215903
 2022-03-24 19:03:09 +01:00
Florian Klink e3083decc4 systemd-initrd, systemd-lib: drop initrdServiceToUnit 
We can just use serviceToUnit here.
 2022-03-24 18:57:16 +01:00
Robert Hensing ce5a33e62b
Merge pull request #164660 from ncfavier/tests-restrict-arguments 
nixos/testing: restrict arguments to makeTest
 2022-03-24 17:01:47 +01:00
Will Fancher 9828446608 systemd-initrd: Fix Environment= and PATH 2022-03-22 21:28:51 -04:00
Will Fancher 2d4ebf1259 initrd: Optional systemd-based initrd 2022-03-22 21:28:43 -04:00
Viktor Kleen 6d766ae8b7
nixos/test-driver: deduplicate VLANs 
Naively deduplicate VLANs in the python driver for NixOS tests. The
current implementation accidentally works, since the VLan class mutates
the environment. On construction it sets QEMU_VDE_SOCKET_${id} and this
environment variable gets overwritten once a second VLAN with the same
id is constructed. Because the NIC flags passed to qemu just use the
QEMU_VDE_SOCKET_${id} environment variable, this implicitly chooses a
single vde_switch process for each VLAN.

However, this leads to unusable vde_switch processes being spawned in
each test run and as a side effect makes it impossible to access the
correct VLan objects in the interactive test driver. It also makes it
remarkably hard to understand why the current implementation ever
worked.
 2022-03-22 13:16:28 +00:00
Will Fancher 25113740a5 nixos: systemd-lib: Make generateUnits general with default args 2022-03-22 07:02:23 -04:00
Robert Hensing 6c469679f6 Merge remote-tracking branch 'upstream/master' into tests-restrict-arguments 2022-03-21 23:17:17 +01:00
Will Fancher 52c98fc3e9 nixos: systemd: Split unit types into separate module 2022-03-21 10:25:27 -04:00
Naïm Favier 8f57dc38d9
fixup! nixos/testing: restrict arguments to makeTest 2022-03-19 11:51:39 +01:00
Naïm Favier 79a234567c
nixos/testing: restrict arguments to makeTest 
Disallow passing arbitrary arguments to makeTest since they are not
used; this can help catch mistakes.
 2022-03-18 02:44:11 +01:00
Robert Hensing 0395086d0c
Merge pull request #162271 from Infinisil/warn-no-type 
Throw an error for options without a type
 2022-03-16 22:58:45 +01:00
Bob van der Linden 9bc093b30a
nixos: systemd: split off helper functions into systemd-lib 2022-03-15 21:52:13 +01:00
pennae aa7b129708
Merge pull request #154113 from pennae/systemd-escaping 
nixos: add functions and documentation for escaping systemd Exec* directives
 2022-03-13 19:57:32 +00:00
pennae 40a35299fa nixos: add functions and documentation for escaping systemd Exec* directives 
it's really easy to accidentally write the wrong systemd Exec* directive, ones
that works most of the time but fails when users include systemd metacharacters
in arguments that are interpolated into an Exec* directive. add a few functions
analogous to escapeShellArg{,s} and some documentation on how and when to use them.
 2022-03-12 00:48:52 +01:00
Janne Heß 0c766a100e lib/options: Throw error for options without a type 
Makes all options rendered in the manual throw an error if they don't
have a type specified.

This is a follow-up to #76184

Co-Authored-By: Silvan Mosberger <contact@infinisil.com>
 2022-02-28 22:51:41 +01:00
talyz b549e7bbb5
genJqSecretsReplacementSnippet: Fix error handling 
With the previous change that enabled error propagation through
`inherit_errexit`, the script would fail if `errexit` was set, but
`inherit_errexit` was not. This is due to `shopt -p` exiting with an
error if the option is disabled. To work around this, use the exit
code instead of the text value returned by `shopt -p`.

Fixes #160869.
 2022-02-23 12:48:51 +01:00
Daniel Fullmer 24fb872079 nixos/test-driver: fix regression when using legacy start command 
The aarch64-linux versions of the boot.uefiUsb and boot.uefiCdrom tests
were broken by b0fc9da879.
That commit was a refactor which omitted the qemuBinary option, which was
previously available in the legacy start command. This restores that
option and fixes the tests previously mentioned.
 2022-02-17 18:12:24 -08:00
Janne Heß fa3c756621
Merge pull request #157329 from helsinki-systems/feat/nixos-reload-triggers 
nixos/systemd: Implement reload triggers
 2022-02-11 23:59:26 +01:00
Janne Heß b5b3ee4f78
nixos/systemd: Add reloadTriggers to services 2022-02-09 15:14:37 +01:00
Robert Hensing f86de05ca6 nixos/systemd: Custom error when mixing list/non-list defs 2022-02-08 12:50:06 +01:00
Jonathan Ringer 46fd0afcb3
Merge remote-tracking branch 'origin/master' into staging-next 
Conflicts:
	pkgs/development/python-modules/pysdl2/default.nix
	pkgs/top-level/aliases.nix
 2022-02-06 18:36:59 -08:00
Robert Hensing 8403e02a5d
Merge pull request #126769 from ncfavier/nixosSystem-lib 
nixos: move default module location logic to `eval-config.nix`
 2022-02-06 22:49:17 +01:00
Jonathan Ringer c6c7162876
Merge remote-tracking branch 'origin/staging-next' into staging 2022-02-03 21:34:17 -08:00
Jonathan Ringer 9a16965c75
Merge remote-tracking branch 'origin/staging-next' into staging 
Conflicts:
	pkgs/development/python-modules/awesomeversion/default.nix
	pkgs/development/python-modules/md2gemini/default.nix
	pkgs/development/python-modules/mill-local/default.nix
	pkgs/development/python-modules/pyatv/default.nix
	pkgs/development/python-modules/pyezviz/default.nix
	pkgs/development/python-modules/pyinsteon/default.nix
	pkgs/development/python-modules/pylitterbot/default.nix
	pkgs/development/python-modules/pynuki/default.nix
	pkgs/development/python-modules/pypck/default.nix
	pkgs/development/python-modules/types-requests/default.nix
	pkgs/top-level/aliases.nix
 2022-02-03 21:29:24 -08:00
Kim Lindberger 415b9c3b5a
Merge pull request #157519 from talyz/gitlab-error-handling 
nixos/gitlab: Implement better script error handling
 2022-02-03 22:19:03 +01:00
Lassulus b80607af67
Merge pull request #154804 from Mic92/tests-prompt 
nixos/tests: add prompt to shell_interact()
 2022-02-02 18:50:46 +01:00
github-actions[bot] aee8ca6639
Merge staging-next into staging 2022-02-01 00:02:21 +00:00
Guillaume Girol d6f45aa6c3
Merge pull request #152372 from symphorien/systemd-validate-after 
nixos/systemd: validate the values of systemd.services.<name>.after
 2022-01-31 20:57:13 +00:00
talyz 95069d76d3
genJqSecretsReplacementSnippet: Propagate secret file read errors 
If an error occurs while trying to read a secret file, we want that
error to propagate to the main shell context. That means we have to
set the `inherit_errexit` option, which allows errors from subshells
to propagate to the outer shell. Also, the subshell cannot run as part
of another command, such as `export`, since that will simply ignore
the subshell exit status and only respect `export`s exit status; first
assigning the value to a variable and then exporting it solves issue.
 2022-01-31 12:44:54 +01:00
Jan Tojnar 2da5ce4ab3 Merge branch 'staging-next' into staging 
; Conflicts:
;	pkgs/development/python-modules/fakeredis/default.nix
 2022-01-28 14:05:11 +01:00
Naïm Favier 5ab62e17b5
nixos: move default module location logic to eval-config.nix 2022-01-27 22:13:35 +01:00
Daniel Olsen 68e9cd0f7e nixos/lib: Use SingleLineStr in systemd description 2022-01-27 12:56:36 -08:00
Dmitry Kalinkin 0693fd77f7
Merge branch 'staging-next' into staging 
Conflicts:
	nixos/doc/manual/from_md/release-notes/rl-2205.section.xml
	nixos/doc/manual/release-notes/rl-2205.section.md
	pkgs/development/python-modules/aioesphomeapi/default.nix
	pkgs/development/python-modules/mat2/default.nix
	pkgs/development/python-modules/pydevccu/default.nix
	pkgs/development/python-modules/pywlroots/default.nix
	pkgs/development/python-modules/rokuecp/default.nix
 2022-01-27 00:54:10 -05:00
Patrick Hilhorst 23c87e7991
nixos/testing-python: actually copy build artefacts 
This broke in #156858.
 2022-01-26 22:26:19 +01:00
Marijan Petričević 411bb3c35f add --no-interactive flag 2022-01-26 11:27:01 +01:00
Marijan Petričević 44830dc048 use lib.optionalString 2022-01-26 10:46:46 +01:00
Marijan Petričević 75de3397fd nixos/tests: fix #146169 2022-01-26 10:46:45 +01:00
Marijan Petričević 446c21fdc7 factor out tmp-dir checks 2022-01-25 11:31:25 +01:00
Marijan Petričević 03c90974a7 add tmp_dir access check 2022-01-25 11:14:57 +01:00
Marijan Petričević e17fcbc966 introduce writeable_dir argparse type 2022-01-25 10:21:33 +01:00
Marijan Petričević 7765670c8a make output_directory an optional parameter 2022-01-25 01:18:52 +01:00
Marijan Petričević f9b5f9dba7 nixos/test-driver: use an argument instead of the out env-var 2022-01-25 01:17:29 +01:00
Alyssa Ross 1a60dd2adc
Merge remote-tracking branch 'nixpkgs/staging-next' into staging 
Conflicts:
	pkgs/development/python-modules/cupy/default.nix
	pkgs/development/python-modules/staticjinja/default.nix
 2022-01-24 15:39:34 +00:00
Samuel Dionne-Riel 18ebd1c330
Merge pull request #153409 from tpwrules/fix-nonx86-iso-reproducibility 
installer/cd-dvd/iso-image: avoid leaking build timestamps on non-x86
 2022-01-23 21:15:26 -05:00
github-actions[bot] aeb6db8171
Merge staging-next into staging 2022-01-20 18:01:55 +00:00
Alyssa Ross 464d6108b0 Revert "qemu, runInLinuxVM: change default cpu to qemu64" 
This reverts commit af180d554b.

The bug we were working around with this change has been fixed in QEMU
6.2.0.
 2022-01-20 15:29:33 +00:00
Robert Hensing 98ae5a9298
Merge pull request #153211 from hercules-ci/minimal-nixos 
Add minimal NixOS entrypoint
 2022-01-20 14:42:46 +01:00
Jörg Thalheim f472e52aa6 nixos/tests: add prompt to shell_interact() 
Example session:

>>> start_all()
>>> machines[0].shell_interact()
client: waiting for the VM to finish booting
client: connected to guest root shell
client: (connecting took 0.00 seconds)
(finished: waiting for the VM to finish booting, in 0.05 seconds)
client: Terminal is ready (there is no initial prompt):
$ ls -la
total 39
drwxrwxrwt 11 root root 4096 Jan 15 06:06 .
drwxr-xr-x 17 root root 4096 Jan 15 06:06 ..
drwxrwxrwt  2 root root 4096 Jan 15 06:06 .font-unix
drwxrwxrwt  2 root root 4096 Jan 15 06:06 .ICE-unix
drwx------  2 root root    2 Jan 12 20:19 shared
drwx------  3 root root 4096 Jan 15 06:06 systemd-private-b8f21699ea684491926859758de41975-nscd.service-txgYVZ
drwx------  3 root root 4096 Jan 15 06:06 systemd-private-b8f21699ea684491926859758de41975-systemd-logind.service-U3GmlL
drwxrwxrwt  2 root root 4096 Jan 15 06:06 .Test-unix
drwxrwxrwt  2 root root 4096 Jan 15 06:06 .X11-unix
drwxr-xr-x  2 root root    2 Jan 15 06:06 xchg
drwxrwxrwt  2 root root 4096 Jan 15 06:06 .XIM-unix
 2022-01-15 07:07:06 +01:00
Robert Hensing 2bf5958169
Merge pull request #151082 from hercules-ci/nixos-cleanup-vmWithBootLoader 
nixos: turn vmWithBootLoader into option (`nixos-rebuild build-vm`)
 2022-01-14 18:49:27 +01:00
Robert Hensing 3168017b90 nixos/lib: Clarify that nixos.evalModules impl is NOT experimental 2022-01-10 12:36:57 +01:00
Robert Hensing d3f956aba3 nixos/lib: Add featureFlags, use it for minimal modules 2022-01-10 12:36:57 +01:00
Robert Hensing 25caf736d5 nixos/eval-config: Layer on top of nixos/eval-config-minimal 2022-01-10 12:26:59 +01:00
Robert Hensing e31e096b66 nixos/lib: Move evalModules into its own file 2022-01-10 12:26:59 +01:00
Zhaofeng Li 9e97ef794a lib/qemu-common: Add serial device name for RISC-V 2022-01-09 15:29:52 -08:00
github-actions[bot] b40a01817b
Merge master into staging-next 2022-01-08 18:01:00 +00:00
Jacek Galowicz 048fd95f10
Merge pull request #146905 from Synthetica9/failure_mode 
nixos/test-driver: add polling_condition
 2022-01-08 18:28:25 +01:00
Jonathan Ringer 86f9a32c4c
Merge remote-tracking branch 'origin/master' into staging-next 
Conflicts:
	pkgs/development/python-modules/hydra/default.nix
 2022-01-07 10:31:29 -08:00
Patrick Hilhorst 0c3f1cf420
Merge pull request #153273 from Synthetica9/passthru-driverInteractive 2022-01-07 16:04:43 +01:00
Robert Hensing fbd038eca2 nixos/lib: init (experimental) 2022-01-07 01:21:07 +01:00
Thomas Watson 450ce00ec4 installer/cd-dvd/iso-image: avoid leaking build timestamps on non-x86 2022-01-05 12:05:15 -06:00
github-actions[bot] 89a0a771b6
Merge master into staging-next 2022-01-05 00:01:52 +00:00
Patrick Hilhorst d4dc638d77
nixos/test-driver: also passthru driverInteractive 2022-01-04 14:52:37 +01:00
Patrick Hilhorst 793a2f50f1
nixos/test-driver: remove unused imports, add pylint unused-import check 2022-01-02 23:12:21 +01:00
Patrick Hilhorst a2f5092867
nixos/test-driver: simplify logic, reduce interaction surface 2022-01-02 22:52:17 +01:00
Patrick Hilhorst 7830f000c5
nixos/test-driver: simplify coopmulti 2022-01-02 22:20:04 +01:00
pennae 50954ad1c5 nixos/make-options-doc: treat missing descriptions as errors by default 
this partially solves the problem of "missing description" warnings of the
options doc build being lost by nix build, at the cost of failing builds that
previously ran. an option to disable this behaviour is provided.
 2022-01-02 19:46:13 +01:00
pennae 1301bdb185 nixos/make-options-doc: turn relatedPackages into links 
link to search.nixos.org instead of pulling package metadata out of pkgs. this
lets us cache docs of a few more modules and provides easier access to package
info from the HTML manual, but makes the manpage slightly less useful since
package description are no longer rendered.
 2022-01-02 19:46:13 +01:00
pennae b92a47c87c nixos/make-options-doc: add type annotations to mergeJSON.py 2022-01-02 19:46:13 +01:00
pennae fc614c37c6 nixos/documentation: split options doc build 
most modules can be evaluated for their documentation in a very
restricted environment that doesn't include all of nixpkgs. this
evaluation can then be cached and reused for subsequent builds, merging
only documentation that has changed into the cached set. since nixos
ships with a large number of modules of which only a few are used in any
given config this can save evaluation a huge percentage of nixos
options available in any given config.

in tests of this caching, despite having to copy most of nixos/, saves
about 80% of the time needed to build the system manual, or about two
second on the machine used for testing. build time for a full system
config shrank from 9.4s to 7.4s, while turning documentation off
entirely shortened the build to 7.1s.
 2022-01-02 19:46:13 +01:00
Patrick Hilhorst ac6c06c549
nixos/test-driver: bump version 2022-01-01 23:17:37 +01:00
Patrick Hilhorst 4e1556ed4d
nixos/test-driver: add polling_condition 2022-01-01 23:17:32 +01:00
github-actions[bot] 129083edcf
Merge master into staging-next 2021-12-29 06:01:11 +00:00
Julien Moutinho 8b842173d0 nixos/make-options-doc: fix invalid ':' in XML NCName (non-colonized name) 2021-12-28 22:18:16 -05:00
Guillaume Girol 2a37dd8ab3 nixos/systemd: validate the values of systemd.services.<name>.after 
and similar option. Notably check that they end with one of .service,
.target, etc.
 2021-12-27 12:00:00 +00:00
wucke13 121e2f7e15 honor sdImage.compressImage in intermediate build steps 2021-12-21 21:31:44 +01:00
Robert Hensing 9fd9c617a9 nixos/lib/eval-config.nix: Return all of evalModules return attrs 
We were exposing everything pointwise anyway.
If any new attrs are added, there's a good chance we'll want to
expose them anyway.
 2021-12-17 14:07:47 +01:00
Robert Hensing 8fd49c116b nixos/default.nix: Use extendModules 2021-12-17 13:17:37 +01:00
github-actions[bot] a292b5fe68
Merge staging-next into staging 2021-12-14 12:02:00 +00:00
pennae a70b1eb630 nixos/lib/make-options-doc: fix with nix 2.3 2021-12-14 03:41:09 +01:00
Dmitry Kalinkin 07a8ae0c5a
Merge branch 'staging-next' into staging 
Conflicts:
	pkgs/development/libraries/log4cplus/default.nix
 2021-12-11 14:01:19 -05:00
Jacek Galowicz b6bf1ca717
Merge pull request #149329 from marijanp/test-driver-restructuring 
nixos/test-driver: make the test-driver a python package
 2021-12-10 18:41:49 +00:00
Marijan Petričević ab693de868 nixos/test-driver: make the test-driver a python package 2021-12-10 12:27:45 +00:00
Silvan Mosberger 2f2b60dd87 lib/nixos/eval-config.nix: Fix extraArgs 
Fixes a mistake in https://github.com/NixOS/nixpkgs/pull/148315 that
caused https://github.com/NixOS/nixpkgs/issues/148343#issuecomment-990881216
 2021-12-10 12:55:30 +01:00
Silvan Mosberger 15c41e1d54
Merge pull request #147265 from pennae/option-docs-build 
put all option docs build flavors on equal footing
 2021-12-08 21:27:02 +01:00
Artturi 3fe92b45cb
Merge pull request #149665 from Artturin/squashfscores 2021-12-08 20:20:32 +02:00
Artturin eea6baad50 make-squashfs: use $NIX_BUILD_CORES or 48 cores if above 48 
by default all cores are used

hoping this will fix the hydra i686 squashfs build issues as all the
failures were using 64 cores

Parallel mksquashfs: Using 64 processors
Creating 4.0 filesystem on ..., block size 1048576.
FATAL ERROR: mangle2:: xz compress failed with error code 5
 2021-12-08 19:23:31 +02:00
ajs124 eee45bb295
Merge pull request #146815 from ElvishJerricco/systemd-utils-expressions 
Move systemd-lib.nix and systemd-unit-options.nix into utils
 2021-12-08 15:07:28 +00:00
Silvan Mosberger 490d46f044
Merge pull request #148315 from hercules-ci/nixos-evalModules-legacy-cleanup 
NixOS/evalModules legacy cleanup
 2021-12-07 18:30:52 +01:00
pennae 027f7e1b7f nixos/lib/make-options-doc: generate options.xml from options.json 
to do this we must replace derivations with attrsets in make-options-doc, since
xml can represent derivations differently from attrset but json cannot. this
also given asciidoc and mddoc the ability to handle derivation differently,
which they previously didn't have.
 2021-12-06 16:12:32 +01:00
pennae 9b97a2ea88 nix/lib/make-options-doc: remove nix-level sorting 
there are no remaining users of sorted option lists except the docbook build,
which sorts its input separately.
 2021-12-06 16:12:32 +01:00
pennae 4670400309 nixos/lib/make-options-doc: generate asciidoc/md in derivations 
use the json file derivation we already have to also generate the asciidoc and
md options docs instead of formatting the options in nix. docbook docs are
already produced in derivations.

the new script produce the exact same output as the old in-nix generation.
 2021-12-06 16:12:30 +01:00
Artturin 60422ba2ea nixos/test-driver: add 10ms delay to send_key 
attempt to fix https://github.com/NixOS/nixpkgs/issues/147294
 2021-12-03 20:04:56 +02:00
Robert Hensing 78ada83361 nixos/eval-config: Deprecate extraArgs and check parameters 2021-12-02 18:23:43 +00:00
Robert Hensing 0b5aea2b27 nixos/eval-config: Remove a rec 2021-12-02 18:23:43 +00:00
Robert Hensing 59c4a35aab nixos/eval-config: Avoid evalModules args and check parameters 2021-12-02 18:23:43 +00:00
Jacek Galowicz 58371472fe
Merge pull request #146512 from DeterminateSystems/better-visibility 
nixos/test-driver: more context when step finishes, give more functions nested labels
 2021-12-02 09:26:21 +00:00
Jacek Galowicz 57dc1085dc
Merge pull request #146271 from DeterminateSystems/better-timeouts 
nixos/test-driver: add execute_with_timeout, add (functional) timeouts to more functions
 2021-12-01 19:01:05 +00:00
Cole Helbling af765f3abd nixos/test-driver: give more functions nested labels 
This will make it easier to trace through the test execution without
having to scroll through the entire kernel output.
 2021-12-01 10:29:56 -08:00
Cole Helbling c6ee63259a nixos/test-driver: more context when step finishes 
When displaying the amount of time some step took, with no other
context, it becomes nigh impossible (especially in longer tests) to see
when specific steps finished.
 2021-12-01 10:29:56 -08:00
Robert Hensing 8a129f8cf0
Merge pull request #144094 from hercules-ci/nixos-specialisations-use-extendModules 
nixos/specialisation: Rephrase in terms of extendModules, noUserModules
 2021-12-01 11:03:36 +01:00