alioth/nixpkgs
3
0
Fork 0
forked from mirrors/nixpkgs
Code Releases Activity
495377 commits 217 branches 116 tags 4 GiB
Commit graph

220 commits

Author SHA1 Message Date
Jörg Thalheim 4f992e8da0
Merge pull request #237840 from lilyinstarlight/fix/systemd-initrd-vconsole-test 
nixos/tests/systemd-initrd-vconsole: fix test and improve reliability
 2023-06-15 19:41:08 +01:00
Lily Foster f1f0d4fbdd
nixos/test-driver: fix timeout option for wait_for_console_text 2023-06-15 06:47:49 -04:00
Thomas Baggaley 491efa0b3e test-driver: respect timeout in wait_until_fails 
- `wait_until_fails` was not passing through its `timeout` argument to
  the internal `retry` function, hence was always using 900 seconds (the
  default timeout for `retry`) rather than the user-specified value.
 2023-06-13 05:11:33 +01:00
Alyssa Ross 507ff39251 nixos/test-driver: fix formatting 
This caused the test driver to fail to build.

Fixes: 406de94b41 ("nixos/test-driver: add `timeout` option for `wait_for_console_text`")
 2023-05-28 12:29:26 +00:00
Jacek Galowicz b4b45ee6d2
Revert "nixos/lib/test-driver: enable EFI variable reads at runtime" 2023-05-28 13:05:59 +02:00
Jacek Galowicz d9626034f1
Merge pull request #234513 from NixOS/test-driver/wait_for_console_timeout 
nixos/test-driver: add `timeout` option for `wait_for_console_text`
 2023-05-28 09:57:00 +02:00
Jacek Galowicz be62469b22
Merge pull request #229027 from NixOS/qemu-vm/read-efi-var 
nixos/lib/test-driver: enable EFI variable reads at runtime
 2023-05-28 09:51:29 +02:00
Raito Bezarius d1104e2109 nixos/test-driver: add timeout option for wait_for_console_text (variant 2) 2023-05-28 00:07:43 +02:00
Raito Bezarius 406de94b41 nixos/test-driver: add timeout option for wait_for_console_text 
Previously, `wait_for_console_text` would block indefinitely until there were lines
shown in the buffer.

This is highly annoying when testing for things that can just hang for some reasons.

This introduces a classical timeout mechanism via non-blocking get on the Queue.
 2023-05-27 23:55:52 +02:00
Raito Bezarius f1aee66f92 nixos/lib/test-driver: enable EFI variable reads at runtime 
This is useful whenever you want to diagnose the current state of UEFI
variables, to assert that bootloaders or boot programs (systemd-stub)
did their job correctly and set their variables accordingly.

In the future, it can enable inspecting SecureBoot keys also.
 2023-05-27 22:44:58 +02:00
Jacek Galowicz 3580ac6c65
Merge pull request #234427 from alyssais/create_machine 
nixos/test-driver: undeprecate create_machine
 2023-05-27 17:02:11 +02:00
Alyssa Ross e33c2a5e4c nixos/test-driver: add missing spaces to warning 2023-05-27 15:00:00 +00:00
Alyssa Ross 845576aac4
nixos/test-driver: undeprecate create_machine 
This warning was added a year and a half ago, but still no test in
NixOS directly instantiates the machine class, presumably because it's
not actually possible for a test to do so without losing
functionality.  For example, there's no way for a NixOS test to access
the output directory that create_machine passes to the Machine
constructor.

This warning is therefore just contributing to alert fatigue for
users, who are unable to follow its advice.  Once it's actually
possible to do what it suggests, the warning can be reintroduced.
 2023-05-27 12:56:04 +00:00
figsoda 202699c918 nixos/tests: fix typos 2023-05-19 22:31:04 -04:00
Jacek Galowicz b7dfa5082a
Merge pull request #228220 from R-VdP/test_driver_guest_shell_timeout 
nixos-test-driver: include a timeout for the recv call, do not assume sh == bash
 2023-05-09 13:05:28 +02:00
r-vdp 4147b878bc
nixos-test-driver: include a timeout for the recv call, do not assume sh == bash 2023-04-26 01:05:53 +02:00
Alyssa Ross d6e84a4574
nixosTest: remove hostname limitations 2023-04-25 08:55:30 +00:00
Alyssa Ross 27f97953e0
nixos/test-driver: accept non-\w* filenames 
What the code was trying to do was helpfully add a directory and
extension if none were specified, but it did this by checking whether
the filename was composed of a very limited character set that didn't
even include dashes.

With this change, the intention of the code is clearer, and I can put
dashes in my screenshot names.
 2023-03-22 15:54:06 +00:00
Astro 64a4ae6f37 nixos/test-driver: remove allow_reboot state from Machine, make it a start() kwarg 2023-03-16 01:50:15 +01:00
Astro 9fc85a85d5 nixos/test-driver: implement Machine.reboot() 2023-03-16 01:41:32 +01:00
Astro 95d121e798 nixos/test-driver: fix allow_reboot 2023-03-15 12:48:01 +01:00
rnhmjoj 067d688b16
nixos/test-driver: handle decoding errors in Machine.execute 
The output of a command is not guaranteed to be valid UTF-8, so the
decoding can fail raising UnicodeDecodeError. If this happens during a
`succeeds` the check will be erroneously marked failed.

This changes the error handling to the "replace" mode, where invalid
codepoints are replaced with � (REPLACEMENT CHARACTER U+FFFD) and the
decoding can go on.
 2023-02-07 08:47:14 +01:00
rnhmjoj f2929eb949
nixos/test-driver: drop logging from Machine.send_monitor_command 
Several machine operations, like `send_chars` and `send_key`, are
implemented by calling `send_monitor_command`, possibly multiple times.
This generates a huge amount of unnecessary noise in the log, because
`send_monitor_command` is a low-level operation and an implementation
detail.

Here's an excerpt from a highlighted log before and afte the change.

Before:

    subtest: Can generate a PGP key
    machine: sending keys 'gpg --gen-key\n'
    machine: sending monitor command: sendkey g
    machine: waiting for monitor prompt
    (finished: waiting for monitor prompt, in 0.00 seconds)
    (finished: sending monitor command: sendkey g, in 0.00 seconds)
    machine: sending monitor command: sendkey p
    machine: waiting for monitor prompt
    (finished: waiting for monitor prompt, in 0.00 seconds)
    (finished: sending monitor command: sendkey p, in 0.00 seconds)
    machine: sending monitor command: sendkey g
    machine: waiting for monitor prompt
    (finished: waiting for monitor prompt, in 0.00 seconds)
    (finished: sending monitor command: sendkey g, in 0.00 seconds)
    machine: sending monitor command: sendkey spc
    machine: waiting for monitor prompt
    (finished: waiting for monitor prompt, in 0.00 seconds)
    (finished: sending monitor command: sendkey spc, in 0.00 seconds)
    machine: sending monitor command: sendkey 0x0C
    machine: waiting for monitor prompt
    (finished: waiting for monitor prompt, in 0.00 seconds)
    (finished: sending monitor command: sendkey 0x0C, in 0.00 seconds)
    machine: sending monitor command: sendkey 0x0C
    machine: waiting for monitor prompt
    (finished: waiting for monitor prompt, in 0.00 seconds)
    (finished: sending monitor command: sendkey 0x0C, in 0.00 seconds)
    machine: sending monitor command: sendkey g
    machine: waiting for monitor prompt
    (finished: waiting for monitor prompt, in 0.00 seconds)
    (finished: sending monitor command: sendkey g, in 0.00 seconds)
    machine: sending monitor command: sendkey e
    machine: waiting for monitor prompt
    (finished: waiting for monitor prompt, in 0.00 seconds)
    (finished: sending monitor command: sendkey e, in 0.00 seconds)
    machine: sending monitor command: sendkey n
    machine: waiting for monitor prompt
    (finished: waiting for monitor prompt, in 0.00 seconds)
    (finished: sending monitor command: sendkey n, in 0.00 seconds)
    machine: sending monitor command: sendkey 0x0C
    machine: waiting for monitor prompt
    (finished: waiting for monitor prompt, in 0.00 seconds)
    (finished: sending monitor command: sendkey 0x0C, in 0.00 seconds)
    machine: sending monitor command: sendkey k
    machine: waiting for monitor prompt
    (finished: waiting for monitor prompt, in 0.00 seconds)
    (finished: sending monitor command: sendkey k, in 0.00 seconds)
    machine: sending monitor command: sendkey e
    machine: waiting for monitor prompt
    (finished: waiting for monitor prompt, in 0.00 seconds)
    (finished: sending monitor command: sendkey e, in 0.00 seconds)
    machine: sending monitor command: sendkey y
    machine: waiting for monitor prompt
    (finished: waiting for monitor prompt, in 0.00 seconds)
    (finished: sending monitor command: sendkey y, in 0.00 seconds)
    machine: sending monitor command: sendkey ret
    machine: waiting for monitor prompt
    (finished: waiting for monitor prompt, in 0.00 seconds)
    (finished: sending monitor command: sendkey ret, in 0.00 seconds)
    (finished: sending keys 'gpg --gen-key\n', in 0.15 seconds)

After:

    subtest: Can generate a PGP key
    machine: sending keys 'gpg --gen-key\n'
    (finished: sending keys 'gpg --gen-key\n', in 0.15 seconds)
 2023-02-07 08:40:54 +01:00
Jörg Thalheim 29db54c373 nixos/tests: extend shell_interact to accept alternative socat addresses 
`shell_interact()` is currently not nice to use.  If you try to cancel
the socat process, it will also break the nixos test. Furthermore
ptpython creates it's own terminal that subprocesses are running in,
which breaks some of the terminal features of socat.
Hence this commit extends `shell_interact` to allow also to connect to
arbitrary servers i.e. tcp servers started by socat.
 2023-01-27 13:09:25 +01:00
Guillaume Girol 33afbf39f6 treewide: switch to nativeCheckInputs 
checkInputs used to be added to nativeBuildInputs. Now we have
nativeCheckInputs to do that instead. Doing this treewide change allows
to keep hashes identical to before the introduction of
nativeCheckInputs.
 2023-01-21 12:00:00 +00:00
Jacek Galowicz 869545857f
Merge pull request #208354 from bjornfor/nixos-test-driver-quote-shell-args 
nixos/test-driver: quote some shell command lines
 2022-12-30 22:41:04 +01:00
Bjørn Forsman c876f0c146 nixos/test-driver: quote some shell command lines 
This makes them robust against paths with whitespace.
 2022-12-30 13:19:59 +01:00
Bjørn Forsman 4c45c3f8f2 nixos/test-driver: use ASCII single quotes everywhere 
A few places used Unicode U+2018/U+2019 left/right single quotes (but
not always correctly balanced). Let's just use plain ASCII single quotes
everywhere.
 2022-12-30 13:00:19 +01:00
Bjørn Forsman ce5644f658 nixos/test-driver: use f-strings instead of .format() 
For readability.

Suggested-by: @tfc
 2022-12-29 23:51:35 +01:00
Bjørn Forsman dba49a43a0 nixos/test-driver: add optional address arg to wait_for_{open,closed}_port 
This is useful for testing servers configured to listen on specific
addresses.
 2022-12-29 13:50:17 +01:00
Jacek Galowicz 9938dec682
Merge pull request #170636 from Synthetica9/wait_before_entry 
nixos/test-driver: add wait_before_entry
 2022-12-23 19:46:23 +01:00
Zhaofeng Li 8b2d34fa5e test-driver: Allow configuring delay for send_{key,chars} 2022-11-19 06:55:25 +01:00
Jacek Galowicz 8ae3e986a7
Merge pull request #189999 from sbruder/test-driver-formatting-closed-port 
nixos/test-driver: fix formatting of closed port
 2022-09-06 14:48:13 +02:00
Simon Bruder 9caf5f6ba9
nixos/test-driver: fix formatting of closed port 
Co-authored-by: a-kenji <aks.kenji@protonmail.com>
 2022-09-06 14:32:56 +02:00
Alyssa Ross 488412a1db
nixos/test-driver: add wait_for_unit() timeout arg 
For example, the wait_for_unit() call in the Moodle test times out for
myself and others[1], so it would be good to be able to increase it to
something less likely to be hit by a test that would otherwise pass.

[1]: https://github.com/NixOS/nixpkgs/pull/177052#issue-1266336706
 2022-09-02 18:06:03 +00:00
Philippe Schaaf 5ae6580474 add hub mode comment 
Signed-off-by: Philippe Schaaf <philippe.schaaf@secunet.com>
 2022-07-21 16:34:09 +02:00
Philippe Schaaf f6a290932e use vde switch in hubmode by default 
Within a dual VM test-setup a strange behaviour was observed.
The two VMs are connected via one vde_switch instance
(instancevirtualisation.vlans = [ 1 ]; IMO a bad attribute name for
switch instances, has nothing to do with VLANs in sense of 802.1Q).

A ping on the base interface (eth1) works, but not on VLAN
subinterfaces (vlan1@eth1). A tcpdump of eth1 includes the ARP requests
tagged with the subinterfaces VLAN ID, but responses seems not to pass
the vde_switch. This works fine if performed on the base interface.

Putting the vde_switch in hub mode results in flooding
traffic to all vde_switch ports. This results in a expected behaviour
and a ping on a VLAN subinterface works as expected.

Signed-off-by: Philippe Schaaf <philippe.schaaf@secunet.com>
 2022-07-21 11:38:03 +02:00
Robert Hensing 2c8bbf33fd nixos/test-driver: Support mypy through regular mechanisms 2022-06-17 12:22:57 +02:00
Robert Hensing bad676c7ed
Merge pull request #174441 from tljuniper/172325-nixostest-override-python-pkgs-additional-param 
nixos/test-driver: additional Python packages in test driver
 2022-06-05 20:46:01 +02:00
Bobby Rong 9d2a8e3190
nixos/test-driver: fix type hint for send_chars 2022-06-04 12:05:47 +08:00
Robert Hensing 793180cf55
Merge branch 'master' into 172325-nixostest-override-python-pkgs-additional-param 2022-06-03 14:22:13 +02:00
Michael Schneider 814027378b nixos/test-driver: Typecheck TestScript 2022-06-02 10:05:45 +02:00
tljuniper a99736e399 nixos/test-driver: add option to add extra python packages to test code 2022-05-29 11:22:23 +02:00
Patrick Hilhorst 976f00963f
nixos/test-driver: rewrite wait_before_entry to wait 2022-04-29 15:36:03 +02:00
Patrick Hilhorst 1db0dcdfec
nixos/test-driver: add wait_before_entry to polling_condition 2022-04-27 16:23:33 +02:00
Patrick Hilhorst 57beeedbdf
nixos/test-driver: allow multiple entry of polling condition 2022-04-27 16:21:10 +02:00
Anna Gillert dbc95f15b8 nixos/test-driver: Avoid shell injection in machine.execute() 2022-04-21 10:54:00 +02:00
Anna Gillert f7e89a59da nixos/test-driver: fix missing shellopts in execute 
Without this fix, setting the shellopts in `machine.execute` is
inconsitent.  When no timeout is used, shellopts `set -euo pipefail` are
applied to the command as expected. When a timeout is specified, the
shellopts are not applied to the command itself (which is called inside
a `sh -c` that doesn't inherit the shellopts) but rather to the
`timeout` command, leading to the following full command:

```bash
(set -euo pipefail; timeout 900 sh -c 'cmd') | (base64 --wrap 0; echo)\n
```

With this fix, this is the command we get:

```bash
timeout 900 sh -c 'set -euo pipefail; false | true') | (base64 --wrap 0; echo)\n
```
 2022-04-21 10:54:00 +02:00
rnhmjoj ca1b1f6dc0
nixos/test-driver: highlight driver log lines 
There is a whole lot of noise in a NixOS test log due to journal,
commands, and test driver messages all being mixing together.
With this commit the test driver messages are highlighted so you don't
have to squint too much to see where a subtest starts and ends or what
was the last command being run.

Here's an excerpt from a highlighted log:

    machine # [    6.647826] dhcpcd[668]: eth0: adding default route via 10.0.2.2
    machine # I've had a great day!
    (finished: must succeed: curl -sfL https://example.com/how-are-you? | tee /dev/stderr, in 0.19 seconds)
    (finished: subtest: Privoxy can filter https requests, in 0.22 seconds)
    subtest: Blocks are working
    machine: waiting for TCP port 443
    machine # Connection to localhost (::1) 443 port [tcp/https] succeeded!
    (finished: waiting for TCP port 443, in 0.03 seconds)
    machine: must fail: curl -f https://example.com/ads 1>&2
    machine # [    6.730608] nscd[742]: 742 monitored file `/etc/resolv.conf` was written to
    machine # [    6.733071] privoxy[814]: 2022-04-14 09:41:01.695 7fa995c39640 Actions: +block{Fake ads} ...
    machine # curl: (22) The requested URL returned error: 403
    machine # [    6.753098] systemd[1]: Stopping Name Service Cache Daemon...
    machine # [    6.755632] systemd[1]: nscd.service: Deactivated successfully.
    machine # [    6.757069] systemd[1]: Stopped Name Service Cache Daemon.
    (finished: must fail: curl -f https://example.com/ads 1>&2, in 0.07 seconds)
    machine: must succeed: curl -f https://example.com/PRIVOXY-FORCE/ads 1>&2
    machine # [    6.765099] systemd[1]: Starting Name Service Cache Daemon...
    machine # [    6.781519] nscd[925]: 925 monitoring file `/etc/passwd` (1)
    machine # [    6.844105] systemd[1]: Reached target Network is Online.
    machine # [    6.848860] privoxy[814]: 2022-04-14 09:41:01.805 7fa995c39640 Actions: +block{Fake ads} ...
    machine # [    6.856068] nscd[925]: 925 monitoring file `/etc/nsswitch.conf` (8)
    machine # [    6.858229] systemd[1]: Reached target Multi-User System.
    machine # [    6.858934] nscd[925]: 925 monitoring directory `/etc` (2)
    machine # [    6.861315] systemd[1]: Startup finished in 2.374s (kernel) + 4.451s (userspace) = 6.826s.
    machine # Hot Nixpkgs PRs in your area. Click here!
    machine # [    6.873055] nscd[925]: 925 monitoring directory `/etc` (2)
    machine # [    6.874489] nscd[925]: 925 monitoring file `/etc/nsswitch.conf` (8)
    machine # [    6.875599] nscd[925]: 925 monitoring directory `/etc` (2)
    (finished: must succeed: curl -f https://example.com/PRIVOXY-FORCE/ads 1>&2, in 0.12 seconds)
    (finished: subtest: Blocks are working, in 0.22 seconds)
    subtest: Temporary certificates are cleaned
    machine: must succeed: test $(ls /run/privoxy/certs | wc -l) -gt 0
    (finished: must succeed: test $(ls /run/privoxy/certs | wc -l) -gt 0, in 0.02 seconds)
    machine: must succeed: date -s "$(date --date '12 days')"
    (finished: must succeed: date -s "$(date --date '12 days')", in 0.02 seconds)
    machine # [    6.959589] systemd[1]: Started Logrotate Service.
    machine # [    6.966685] systemd[1]: Starting Cleanup of Temporary Directories...
    machine # [    6.974783] systemd[1]: logrotate.service: Deactivated successfully.
    machine # [    7.004493] systemd[1]: systemd-tmpfiles-clean.service: Deactivated successfully.
    machine # [    7.005457] systemd[1]: Finished Cleanup of Temporary Directories.
    machine: must succeed: test $(ls /run/privoxy/certs | wc -l) -eq 0
    (finished: must succeed: test $(ls /run/privoxy/certs | wc -l) -eq 0, in 0.01 seconds)
    (finished: subtest: Temporary certificates are cleaned, in 0.15 seconds)
    (finished: run the VM test script, in 7.44 seconds)
    test script finished in 7.48s
    cleanup
    kill machine (pid 8)
 2022-04-14 11:54:21 +02:00
Janne Heß 7d0e21c77e
nixos/test-runner: Allow writing to qemu stdin 2022-03-25 12:08:21 +01:00