3
0
Fork 0
forked from mirrors/nixpkgs
Commit graph

385 commits

Author SHA1 Message Date
florianjacob 8b07500163
nixos/lib: use removePrefix in escapeSystemdPath 2020-03-14 03:13:04 +00:00
Michele Guerini Rocco 7b15d6cee4
Merge pull request #81241 from thefloweringash/nesting-system
nixos/activation: propagate system to nested configurations
2020-03-13 09:58:10 +01:00
Maximilian Bosch 6d14bac048
nixos/python-test-driver: allow single char as hostName and fix misleading error-message
In case of invalid chars, the error-message references "perl variables"
which is not the case here as the python-based framework is used.
2020-03-13 01:06:12 +01:00
Andrew Childs ce416779bb nixos/activation: use eval-config's system argument for nesting
This avoids a possible surprise if the user is using `nixpkgs.system`
and `nesting.children`. `nesting.children` is expected to ignore all
parent configuration so we shouldn't propagate the user-facing option
`nixpkgs.system`. To avoid doing so, we introduce a new internal
option for holding the value passed to eval-config.nix, and use that
when recursing for nesting.
2020-03-05 20:28:31 +09:00
Jacek Galowicz d772206c49 nixosTests: Make VLAN argument optional in python test driver 2020-02-14 19:05:03 +01:00
worldofpeace 95e41ecd8f
Merge pull request #79659 from worldofpeace/cleanup-python-test-lib
testing: fix runInMachineWithX/runInMachine
2020-02-14 12:38:57 -05:00
worldofpeace aa4ba502d6
Merge pull request #79335 from flokli/deprecate-perl-vm-tests
nixos/testing: add deprecation notice for Perl VM tests
2020-02-09 21:08:11 -05:00
Florian Klink 0945178b3c nixos/testing: add deprecation notice for Perl VM tests
Most VM tests have been migrated to use the python test driver
(introduced in #71684), the migration is tracked in #72828 (which also
thankfully uncovered and fixed many currently broken tests)

While increasing the acceptance and adoption of NixOS integration tests
by using a more popular language, there was also nobody willing to do
larger refactors in the currently very convoluted test infrastructure.

We plan to remove the perl infrastructure between the 20.03 and 20.09
release, to be able to do these refactorings.

Some people might be using Perl tests in their internal CI, so print a
warning for 20.03, and give users time to move to the python testing
infrastructure.
2020-02-09 23:53:59 +01:00
worldofpeace 5507e09618 testing: fix runInMachineWithX 2020-02-09 16:27:01 -05:00
worldofpeace 88f76812f2 testing-python: readd auto displayManager
we import it for the runInMachineWithX
2020-02-09 16:23:35 -05:00
worldofpeace fa9af83e96 testing-python: fix runInMachine
The test script's were unported.
It's unclear whether the preBuild or
postBuild will work as expect, due to
the linting of the test scripts.
2020-02-09 16:21:52 -05:00
Samuel Dionne-Riel cebb0b49d4 Merge pull request #79368 from sorki/ext4resize
nixos/lib/make-ext4-fs: fall back to resize2fs -M if exact resize fails
2020-02-09 14:35:08 -05:00
Richard Marko db894cb8ce nixos/lib/make-ext4-fs: fall back to resize2fs -M if exact resize fails
See also
- https://bugs.launchpad.net/ubuntu/+source/e2fsprogs/+bug/1415077/comments/4
- https://github.com/NixOS/nixpkgs/pull/62262
2020-02-09 17:18:21 +01:00
Jacek Galowicz 798fcaafa4 nixos/test: Add copy_from_host and copy_from_host_via_shell to python
test driver
2020-02-07 13:24:16 +01:00
John Ericson 14fbd4146d
Merge pull request #59855 from volth/qemu-cpu-passthru
runInLinuxVM, test-driver: pass host's cpu type to guest vm
2020-02-01 22:59:37 -05:00
Silvan Mosberger 80a2740991
Merge pull request #78265 from Synthetica9/https-homepages
treewide: fix redirected urls
2020-01-27 15:00:53 +01:00
Jacek Galowicz f63ef280df nixosTests: Drop invalid utf characters in command output 2020-01-26 17:11:59 +01:00
Patrick Hilhorst 593e11fd94
treewide: fix redirected urls
According to https://repology.org/repository/nix_unstable/problems, we have a
lot of packages that have http links that redirect to https as their homepage.
This commit updates all these packages to use the https links as their
homepage.

The following script was used to make these updates:

```

curl https://repology.org/api/v1/repository/nix_unstable/problems \
    | jq '.[] | .problem' -r \
    | rg 'Homepage link "(.+)" is a permanent redirect to "(.+)" and should be updated' --replace 's@$1@$2@' \
    | sort | uniq > script.sed

find -name '*.nix' | xargs -P4 -- sed -f script.sed -i
```
2020-01-22 11:26:22 +01:00
Florian Klink 764aba4c1b
Merge pull request #77587 from singron/fix-shared
test-driver.py: Share the shared dir between VMs
2020-01-19 22:07:31 +01:00
Eric Culp 6795a4002c test-driver.py: Use /tmp/shared for copy_from_vm
The docstring says it uses a directory shared among all vms, although
that doesn't seem necessary for the functionality. However, it does need
to be consistent between the guest and host.
2020-01-19 10:33:46 -08:00
Florian Klink ed0b5b6133
Merge pull request #77662 from marijanp/verbose-python-test-driver
nixos/test: added verbose output for failed tests
2020-01-17 13:50:49 +01:00
volth 2bd296a7e9 runInLinuxVM, test-driver: pass host's cpu type to guest vm
'kvm64' is the most generic CPU, which does not support SSE4.2, AVX and other ISA extentions.
2020-01-15 23:35:00 +00:00
Erik Arvstedt 5bdb653baf test-driver.py: fix decoding of VM output
The codec format 'unicode_escape' was introduced in 52ee102 to handle
undecodable bytes in boot menus.

This made the problem worse as unicode chars outside of iso-8859-1
produce garbled output and valid utf-8 strings (such as "\x" ) trigger
decoding errors.

Fix this by using the default 'utf-8' codec and by explicitly ignoring
decoding errors.
2020-01-14 15:52:44 -08:00
Erik Arvstedt caa435fd1d test-driver.py: specify coreutils dependency
Otherwise the driver script fails when coreutils are not in PATH.
2020-01-14 15:52:44 -08:00
Marijan Petričević 61c61f80e5 nixos/test: added verbose output for failed tests 2020-01-14 09:33:10 +01:00
Eric Culp 21f567bdd9 test-driver.py: Share the shared dir between VMs
This changes the python test driver to match the behavior of the perl
test driver.  I.e. the directory mounted into /tmp/shared should be the
same for all machines.

This probably fixes many tests, but I found this while investigating
failures in nixos/tests/ceph-multi-node.nix.
2020-01-12 15:49:17 -08:00
Maximilian Bosch b72661749b
nixos/python-test-driver: add an option to disable python linter
While it's a good idea to automate the linting of the python code used
for our tests, I think that it can be quite distracting when hacking on
a NixOS test.

I figured that it might be more convenient to add an option as a
shortcut for this to avoid that everyone needs to dig into the test
driver again.
2019-12-22 19:27:43 +01:00
Jan Tojnar 8dc5ff7dcf
nixos/displayManager: deprecate separate options for default wm/dm
The upstream session files display managers use have no concept of sessions being composed from
desktop manager and window manager. To be able to set upstream session files as default
session, we need a single option. Having two different ways to set default session would be confusing,
though, so we decided to deprecate the old method.

We also created separate script for each session, just like we already had a separate desktop
file for each one, and started using displayManager.sessionPackages mechanism to make the
session handling more uniform.
2019-12-15 04:16:20 +01:00
Bernardo Meurer 70c5a78062
nixos: compress make-ext4-fs with zstd 2019-12-13 16:18:04 -08:00
Jacek Galowicz e05ae69904 nixos/test: Use retry() in all looping functions that need timeouts 2019-12-03 16:59:43 +01:00
Florian Klink da3a320d13
Merge pull request #74077 from 7c6f434c/add-test-driver-py-copy-from-vm
Add test driver py copy from vm
2019-12-01 13:56:23 +01:00
Domen Kožar 2c8c19972d
Merge pull request #73966 from d-goldin/nixosoptions_xml_strctx
make-options-doc: fix string context issues
2019-11-25 13:20:00 +01:00
Michael Raskin ad38a08ecb test-driver.py: add Machine.copy_from_vm 2019-11-25 11:35:48 +01:00
Michael Raskin ef5bc381f9 test-driver.py: Fix the shared directory reference 2019-11-25 11:33:47 +01:00
Robin Gloster ba03a1b731
nixos/tests: fail on failing subtests
This was the behaviour in the perl test driver too and a regression.

Also cleaned up unused imports
2019-11-25 01:08:06 +01:00
Dima 3c15d578d4 make-options-doc: fix string context issues
When using `documentation.nixos.includeAllModules = true;` with external
modules, the string context might contain dependencies to derivations
and so `toFile` refuses to evaluate;

```
error: in 'toFile': the file 'options.xml' cannot refer to derivation outputs, at
[...]/nixpkgs/nixos/lib/make-options-doc/default.nix:89:16
```

This is not an issue when using `writeText` (instead of manually
stripping the context).
2019-11-23 15:40:45 +01:00
Florian Klink 758efb9348
Merge pull request #73190 from flokli/ceph-tests-python
WIP: nixos/ceph: port tests to python
2019-11-21 20:13:41 +01:00
Jacek Galowicz 4dba4db1db nixos/test: let python driver evaluate test code in global scope so the
user can define and use functions
2019-11-20 19:08:18 +01:00
Jacek Galowicz 72dd1c6a74 nixos/test: If qemu monitor returns 0, simply return (like perl script does) 2019-11-20 17:38:20 +01:00
Jacek Galowicz 67e2042861 nixos/test: Fix child environment of python driver 2019-11-20 17:37:12 +01:00
worldofpeace 4583e293fd
Merge pull request #73251 from worldofpeace/remove-slim
nixos/slim: remove
2019-11-15 15:56:48 +00:00
worldofpeace ce26b3eaf0 nixos/slim: remove
The SLIM project is abandoned and their last release was in 2013.
Because of this it poses a security risk to systems, no one is working
on it or picked up maintenance. It also lacks compatibility with systemd
and logind sessions. For users, there liikely isn't anything like slim
that's as lightweight in terms of dependencies.
2019-11-11 17:10:41 -05:00
Jörg Thalheim 03e6ca15e2
test-driver: add mypy support
It's a good idea to expand this in future to test code as well,
so we get type checking there as well.
2019-11-11 13:49:48 +00:00
Florian Klink a8e9a93493 nixos/tests: fix succeed() with multiple commands
we previously immediately returned the first commands output, and didn't
execute any of the other commands.

Now, return the last commands output.
This should be documented in the method docstring.
2019-11-11 13:45:11 +01:00
Leonhard Markert c7becac738
nixos/tests: correct arguments to get_unit_info 2019-11-08 15:01:42 +01:00
Marijan Petricevic 5854aaf680 nixos/tests: improve get_unit_info expressiveness on failure 2019-11-08 11:20:54 +01:00
worldofpeace 1e7ddf233a
Merge pull request #72943 from marijanp/port-test-driver-python
Port remaining test-driver functions to python
2019-11-07 20:23:27 +00:00
worldofpeace 6ed6d1af6a
Merge pull request #72947 from tfc/boot-test-port-fix
Boot test port fix
2019-11-07 17:32:43 +00:00
Jacek Galowicz 52ee1026b0 nixos/test: Handle undecodable bytes
This threw exceptions in boot menus
2019-11-07 10:00:39 +01:00
worldofpeace 3780b9e69c
Merge pull request #72835 from tfc/nixos-integration-test-ports
Nixos integration test ports
2019-11-07 01:05:36 +00:00
Marijan 9915a8ca45 nixos/tests: implemented python test-driver version of forwardPort 2019-11-06 22:43:37 +01:00
Marijan 01f79dfc9e nixos/tests: implemented python test-driver version of dumpTTYContents 2019-11-06 22:43:37 +01:00
Jacek Galowicz cf138b4e6b nixos/tests: Fix subprocess launch call for VM monitor
This crashed in the create-script case
2019-11-06 21:48:08 +01:00
Daniel Schaefer afbfef93a7 nixos/tests: Ignore shutdown/crash if not booted
Condition seems to be inverted. Crash and shutdown only make sense, when
the machine is booted; i.e. we return immediately otherwise.
In the Perl test driver this is:

    return unless $self->{booted};
2019-11-06 16:06:43 +01:00
Jacek Galowicz 45b339b736 nixos/tests: Implement python test-driver version of getWindowNames and waitForWindow 2019-11-05 18:07:05 +01:00
Jacek Galowicz 9c54658226 nixos/tests: Implement python test-driver version of requireActiveUnit 2019-11-05 18:07:00 +01:00
Florian Klink ac97edf013 nixos/test: use ptpython as repl 2019-11-04 23:50:27 +01:00
Jacek Galowicz 3a28fefe7d nixos/test: Port test driver to python
Thanks @blitz and @jtraue for help with implementing machine methods
2019-11-04 23:50:27 +01:00
Chuck ac65940537 nixos/make-iso9660-image: Fix doc typo 2019-10-18 11:58:28 -07:00
Chuck 4623f0a2c6 nixos/make-iso9660-image: Fix storeContents documentation
`packages` was renamed to `storeContents` in
668c146e33, but this comment
describing its purpose was not updated to match.
2019-10-18 11:58:05 -07:00
Eelco Dolstra 39c0e489d3
Revert "make-disk-image: replace nixos-install by nix-env calls"
This reverts commit e9bf955fd6. We use
nixos-install to ensure that make-disk-image produces the same result
as a regular installation (9802da517f)
and to reduce code duplication. If there is something broken in
nixos-install, it should be fixed there.
2019-10-07 17:21:28 +02:00
Christian Kögler e9bf955fd6 make-disk-image: replace nixos-install by nix-env calls
nixos-install can only be called from a machine
2019-09-29 19:57:19 +02:00
Sarah Brofeldt 7ca5b39125 nixos/lib/utils: Make the set recursive again, unbreak eval 2019-09-08 13:31:18 +02:00
Florian Klink 2f3b9cd52c
Merge pull request #66274 from talyz/gitlab
nixos/gitlab: Add support for secure secrets and more
2019-09-07 12:52:44 -07:00
talyz 64358cb0e9 nixos/utils: Handle arbitrary secrets in JSON output files
Introduce new functions which allows modules to define options where,
if the input is an attrset and the output is JSON, the user can define
arbitrary secrets.
2019-09-06 16:56:25 +02:00
volth 08f68313a4 treewide: remove redundant rec 2019-08-28 11:07:32 +00:00
Frederik Rietdijk 5061fe0c2c Merge staging-next into staging 2019-08-28 08:26:42 +02:00
volth 35d68ef143 treewide: remove redundant quotes 2019-08-26 21:40:19 +00:00
danbst d80cd26ff9 Merge branch 'master' into flip-map-foreach 2019-08-18 18:00:25 +03:00
Domen Kožar dcd50c0ea0
pkgs.lib -> lib 2019-08-12 11:46:53 +02:00
Domen Kožar 6cf861c617
make-options-doc: add asciidoc variant 2019-08-10 20:11:04 +02:00
Domen Kožar 3a93fcfd1e
make-options-doc: expose Nix set 2019-08-10 14:24:11 +02:00
Domen Kožar 5cfd034af0
Extract NixOS options documentation generation to a function
Motivation is to support other repositories containing nixos
modules that would like to generate options documentation:

- nix-darwin
- private repos
- arion
- ??
2019-08-08 16:18:09 +02:00
Nikola Knezevic d0ef94258d Make hostname in tests overridable
The original form effectively forbade any NixOS configuration that is under
test to explicitly set the hostname.
2019-08-07 10:31:13 +02:00
Danylo Hlynskyi 7585496eff
Merge branch 'master' into flip-map-foreach 2019-08-05 14:09:28 +03:00
danbst 0f8596ab3f mass replace "flip map -> forEach"
See `forEach`-introduction commit.
```
rg 'flip map ' --files-with-matches | xargs sed -i 's/flip map /forEach /g'
```
2019-08-05 14:03:38 +03:00
danbst 91bb646e98 Revert "mass replace "flip map -> foreach""
This reverts commit 3b0534310c.
2019-08-05 14:01:45 +03:00
Andrew Childs d2144755a4 nixos-test-driver: allow configuration of net frontend and backend
When IPXE tests were added, an option was added for configuring only
the frontend, and the backend configuration was dropped entirely. This
caused most installer tests to fail.
2019-07-22 13:44:27 +03:00
Nikolay Amiantov 81d35a9d7e nixos-test-driver: support netRomFile
Needed for UEFI PXE netboot testing.
2019-07-15 19:33:26 +03:00
danbst 3b0534310c mass replace "flip map -> foreach"
See `foreach`-introduction commit.
```
rg 'flip map ' --files-with-matches | xargs sed -i 's/flip map /foreach /g'
```
2019-07-14 13:46:10 +03:00
worldofpeace 3f4a353737 treewide: use dontUnpack 2019-07-01 04:23:51 -04:00
Samuel Dionne-Riel a0aff40f65 make-ext4-fs: Allows populating with custom files
This will allow adding /boot files to the rootfs.
2019-06-16 17:47:30 -04:00
Arian van Putten cbc45b5981 nixos: Make nesting.children work in NixOS tests
We differentiate between modules and baseModules in  the
VM builder for NixOS tests. This way, nesting.children, eventhough
it doesn't inherit from parent, still has enough config to
actually complete the test. Otherwise, the qemu modules
would not be loaded, for example, and a nesting.children
statement would not evaluate.
2019-05-29 12:50:49 +02:00
Arian van Putten d50b434234 nixos: Make 'nesting.clone' work in NixOS tests
Because nesting.clone calls 'eval-config.nix' manually,
without the 'extraArgs' argument that provides the 'nodes'
argument to nixos modules in nixos tests, evaluating
of 'nesting.clone' definitions would fail with the following error

while evaluating the module argument `nodes' in "<redacted>"
while evaluating the attribute '_module.args.nodes' at undefined position:
attribute 'nodes' missing, at <redacted./nixpkgs/lib/modules.nix:163:28

by not using 'extraArgs' but a nixos module instead, the nodes parameter
gets propagated to the 'eval-config.nix' call that  'nesting.clone'
makes too -  getting rid of the error.

See  https://github.com/NixOS/nixpkgs/blob/master/nixos/modules/system/activation/top-level.nix#L13-L23
See  https://github.com/NixOS/nixpkgs/blob/master/nixos/lib/build-vms.nix#L27
See  https://github.com/NixOS/nixpkgs/issues/20886#issuecomment-495952149
2019-05-26 00:20:21 +02:00
Arian van Putten 2e75a7b516 nixos: doc: optionally include all modules in manual generation
Before this change `man 5 configuration.nix` would only show options of modules in
the `baseModules` set, which consists only of the list of modules in
`nixos/modules/module-list.nix`

With this change applied and `documentation.nixos.includeAllModules` option enabled
all modules included in `configuration.nix` file will be used instead.

This makes configurations with custom modules self-documenting. It also means
that importing non-`baseModules` modules like `gce.nix` or `azure.nix`
will make their documentation available in `man 5 configuration.nix`.

`documentation.nixos.includeAllModules` is currently set to `false` by
default as enabling it usually uncovers bugs and prevents evaluation.
It should be set to `true` in a release or two.

This was originally implemented in #47177, edited for more configurability,
documented and rebased onto master by @oxij.
2019-03-05 09:41:40 +00:00
Matthew Bauer 878965731f
nixos/lib/testing.nix: config defaults to {}
Fixes #51858

56e12aae54 ends up passing config to pkgs. Unfortunately this might be null and pkgs/top-level/default.nix assumes it is an attrset. To fix this, we just make the default for config = {}. Thanks to @kristoff3r for tracking this down.

/cc @domenkozar
2019-02-06 20:04:24 -05:00
Travis Athougies a66ef3aa3d Add options to build disk image function (#50239)
* add options to build disk image function

* Revert suffix changes
2019-01-24 01:47:55 +02:00
Samuel Dionne-Riel 50555a6d35
Merge pull request #54330 from samueldr/fix/disable-tests-shell-access
Revert "Add ssh backdoor to VM tests infrastructure."
2019-01-19 14:20:15 -05:00
Samuel Dionne-Riel 3aab228d09 Revert "Add ssh backdoor to VM tests infrastructure."
This reverts commit d6e3db44cf.

See #53935 for explanations. In short, it may be causing issues with
tests on the build infrastructure.
2019-01-19 13:24:39 -05:00
Danylo Hlynskyi 23a13b562c
kernel config: add support for CONFIG_SQUASHFS_ZSTD (#52967)
Also, allow override `make-squashfs.nix` compression parameters.
2019-01-17 15:24:44 +02:00
Silvan Mosberger 497e6f1705
Merge pull request #51661 from eonpatapon/testing-hosts
test: set machines fqdn in /etc/hosts
2019-01-12 20:41:13 +01:00
Samuel Dionne-Riel 2646a64fbc
Merge pull request #53827 from samueldr/feature/data-in-logs
tests: Logs timing in tests
2019-01-12 12:54:56 -05:00
Samuel Dionne-Riel 3b68ddb6fe
Merge pull request #53828 from samueldr/feature/double-alarm-time
tests: Wait for shell for twice as long (10m)
2019-01-12 12:08:14 -05:00
Samuel Dionne-Riel b28b37eb00 tests: Wait for shell for twice as long (10m)
See #49441 for an earlier attempt, which was subsequently reverted. I am
assuming that doubling the time will be sufficient if the machine is
overloaded since so many of the tests already pass at 5 minutes, while
still not holding back failures for needlessly long.
2019-01-11 22:40:19 -05:00
Samuel Dionne-Riel 5d93e2c01c test-driver: Logs time taken for nests 2019-01-11 22:36:31 -05:00
Samuel Dionne-Riel 1fe0018df8 test-driver: Adds time it took to connect to guest in logs
This will make it possible to track whether the time is generous or not
when ran on hydra.
2019-01-11 22:36:31 -05:00
Matthew Bauer 04373fd3cc
Merge pull request #52594 from matthewbauer/fix-51025
make-disk-image: use filterSource instead of cleanSource
2019-01-07 16:29:58 -06:00
Matthew Bauer f05d8f31ec make-disk-image: use filterSource instead of cleanSource
cleanSource does not appear to work correctly in this case. The path
does not get coerced to a string, resulting in a dangling symlink
produced in channel.nix.  Not sure why, but this
seems to fix it.

Fixes #51025.

/cc @elvishjericco
2019-01-07 16:28:50 -06:00
Frederik Rietdijk 9618abe87c Merge master into staging-next 2019-01-04 21:13:19 +01:00
Matthew Bauer 74312c7ef5
Merge pull request #52760 from akru/master
lib/make-ext4-fs: more efficient store maker
2019-01-03 15:07:27 -06:00