It's time again, I guess :>
Main motivation is to stop being pinged about software that I maintained
for work now that I'm about to switch jobs. There's no point in pinging
me to review/test updates or to debug issues in e.g. the Atlassian stack
or on mailman since I use neither personally.
But there's also a bunch of other stuff that I stopped using personally. While
at it I realized that I'm still maintainer of a few tests & modules related to
packages I stopped maintaining in the past already.
The `nix.*` options, apart from options for setting up the
daemon itself, currently provide a lot of setting mappings
for the Nix daemon configuration. The scope of the mapping yields
convience, but the line where an option is considered essential
is blurry. For instance, the `extra-sandbox-paths` mapping is
provided without its primary consumer, and the corresponding
`sandbox-paths` option is also not mapped.
The current system increases the maintenance burden as maintainers have to
closely follow upstream changes. In this case, there are two state versions
of Nix which have to be maintained collectively, with different options
avaliable.
This commit aims to following the standard outlined in RFC 42[1] to
implement a structural setting pattern. The Nix configuration is encoded
at its core as key-value pairs which maps nicely to attribute sets, making
it feasible to express in the Nix language itself. Some existing options are
kept such as `buildMachines` and `registry` which present a simplified interface
to managing the respective settings. The interface is exposed as `nix.settings`.
Legacy configurations are mapped to their corresponding options under `nix.settings`
for backwards compatibility.
Various options settings in other nixos modules and relevant tests have been
updated to use structural setting for consistency.
The generation and validation of the configration file has been modified to
use `writeTextFile` instead of `runCommand` for clarity. Note that validation
is now mandatory as strict checking of options has been pushed down to the
derivation level due to freeformType consuming unmatched options. Furthermore,
validation can not occur when cross-compiling due to current limitations.
A new option `publicHostKey` was added to the `buildMachines`
submodule corresponding to the base64 encoded public host key settings
exposed in the builder syntax. The build machine generation was subsequently
rewritten to use `concatStringsSep` for better performance by grouping
concatenations.
[1] - https://github.com/NixOS/rfcs/blob/master/rfcs/0042-config-option.md
This should NOT be backported to 20.09!
When 21.03 is released, the DB changes are about a year old and
operators had two release cycles for the upgrade. At this point it
should be fair to remove the compat layer to reduce the complexity of
the module itself.
Also removed `pkgs.hydra-flakes` since flake-support has been merged
into master[1]. Because of that, `pkgs.hydra-unstable` is now compiled
against `pkgs.nixFlakes` and currently requires a patch since Hydra's
master doesn't compile[2] atm.
[1] https://github.com/NixOS/hydra/pull/730
[2] https://github.com/NixOS/hydra/pull/732
Upgrades Hydra to the latest master/flake branch. To perform this
upgrade, it's needed to do a non-trivial db-migration which provides a
massive performance-improvement[1].
The basic ideas behind multi-step upgrades of services between NixOS versions
have been gathered already[2]. For further context it's recommended to
read this first.
Basically, the following steps are needed:
* Upgrade to a non-breaking version of Hydra with the db-changes
(columns are still nullable here). If `system.stateVersion` is set to
something older than 20.03, the package will be selected
automatically, otherwise `pkgs.hydra-migration` needs to be used.
* Run `hydra-backfill-ids` on the server.
* Deploy either `pkgs.hydra-unstable` (for Hydra master) or
`pkgs.hydra-flakes` (for flakes-support) to activate the optimization.
The steps are also documented in the release-notes and in the module
using `warnings`.
`pkgs.hydra` has been removed as latest Hydra doesn't compile with
`pkgs.nixStable` and to ensure a graceful migration using the newly
introduced packages.
To verify the approach, a simple vm-test has been added which verifies
the migration steps.
[1] https://github.com/NixOS/hydra/pull/711
[2] https://github.com/NixOS/nixpkgs/pull/82353#issuecomment-598269471
During the last update, `hydra-notify` was rewritten as a daemon which
listens to postgresql notifications for each build[1]. The module
uses the `hydra-notify.service` unit from upstream's Hydra module and
the VM test ensures that email notifications are sent properly.
Also updated `hydra-init.service` to install `pg_trgm` on a local
database if needed[2].
[1] c7861b85c4
[2] 8a0a5ec3a3
We ship `https://cache.nixos.org` as binary cache by default which
automatically substitutes the test derivation used inside the Hydra
test. However it needs to be built locally to confirm that
`hydra-queue-runner` works properly.
Also inherited the platform name for the test derivation from `system`
to ensure that the build can be tested on each supported platform.
ZHF #68361
Hydra should support multiple Nix versions (and currently contains fixes
to work with Nix 2.0 and higher).
Further Nix versions can be added to the `hydraPkgs` expression in the
test case which lists all supported Nix versions for Hydra.
This bumps Hydra to the latest revision available. As Hydra doesn't have
a release model (and therefore no tags) ATM, the derivation will pin
against the actual git revision and the date of the commit in the
derivation name.
Additionally the following changes have been made:
* Dropped `postUnpack` phase. It is useful when working with the Hydra
source (and no dirty changes shall be used in `release.nix`, but is has
no use in `nixpkgs`).
* Added myself as maintainer to have more folks available in case of
future breakage.
* Implemented support for Nix 2.0 and `unstable` (currently 2.1):
Since 1672bcd230447f1ce0c3291950bdd9a662cee974 in NixOS/nix the
evaluator differentiates between `settings` and `evalSettings`.
Previously `restrictEval` in `hydra-eval-jobs.cc` has been set in
`settings`, this doesn't work anymore in Nix 2.1 and is therefore
incompatible to Nix 2.0 on an API level.
To resolve this, the flag `isGreaterNix20` parses the version string
of `pkgs.nix` and applies a patch if nix.version<=2.0.
Furthermore the Hydra build with Nix 2.1 requires `boost` as build input
which is not needed for Nix 2.0. To avoid unnecessary increase in the
closure size this library will only used as build input for
nix.version>2.0.
* Fixed the NixOS test for `hydra`:
disabled binary cache to allow sandbox builds (otherwise it would
query `cache.nixos.org` during the Hydra build inside the test).
Additionally the trivial.nix jobset required simplification (as done
in NixOS/hydra, e.g. tests/api-test.nix) as bash is not available in
the build sandbox as builder (even when adding pkgs.bash to
systemPackages).
The easiest workaround to confirm a the functionality of a jobset
without importing nixpkgs is to use the default shell /bin/sh which
is mounted from `pkgs.busybox` into the build env
(https://github.com/NixOS/nixpkgs/pull/44841#discussion_r209751972) in the
VM and a named pipe to create $out.
Closes #44044
A script is used to create a project, and configure a jobset. This
jobset fetches a local file containing a trivial Nix expression. The
test script makes sure this derivation has been successfully built by
Hydra.
