3
0
Fork 0
forked from mirrors/nixpkgs
Commit graph

223 commits

Author SHA1 Message Date
pennae d98322834b nixos/*: fix docbook deprecation notices
mostly no rendering changes except in buildkite, which used markdown
where docbook was expected without marking up its markdown.
2022-09-10 18:23:13 +02:00
pennae 9547123258 nixos/*: convert internal option descriptions to MD
we'll have to do it eventually, may as well be now.
2022-08-31 16:32:54 +02:00
pennae ef176dcf7e nixos/*: automatically convert option descriptions
conversions were done using https://github.com/pennae/nix-doc-munge
using (probably) rev f34e145 running

    nix-doc-munge nixos/**/*.nix
    nix-doc-munge --import nixos/**/*.nix

the tool ensures that only changes that could affect the generated
manual *but don't* are committed, other changes require manual review
and are discarded.
2022-08-31 16:32:53 +02:00
pennae 51a11254a7 nixos/*: literalDocBook -> literalMD
no change to rendered output
2022-08-27 19:18:29 +02:00
pennae 6039648c50 nixos/*: automatically convert option docs 2022-08-19 22:40:58 +02:00
pennae 7e7d68a250 nixos/*: mark pre-existing markdown descriptions as mdDoc 2022-08-19 22:40:58 +02:00
pennae b51f8036c2 nixos/*: use properly indented strings for option docs
using regular strings works well for docbook because docbook is not as
whitespace-sensitive as markdown. markdown would render all of these as
code blocks when given the chance.
2022-08-19 22:40:58 +02:00
pennae b7327e966b nixos/*: normalize links with #TEXT=href
the xslt pass will re-add this text, and not having href and text be
equal is required to automatically migrate these links to markdown.
2022-08-19 22:40:58 +02:00
Vincent Haupert 786f72c32e nixos/github-runner: start Runner.Listener directly in ExecStart=
The `runsvc.sh` script wraps a JavaScript script which starts
`Runner.Listener` and also handles failures. This has the downside that
the service _always_ exits with status code 0, i.e., success. This
causes frequent service restarts when running in ephemeral mode with a
faulty config as Systemd always sees a success exit status. To prevent
this, this commit changes the service config to call `Runner.Listener`
directly. The JavaScript wrapper stops the process with a SIGINT, hence,
the Systemd unit now sends a SIGINT to stop the service.
2022-08-15 13:13:58 -04:00
Vincent Haupert 987a4b4231 nixos/github-runner: add support for ephemeral runners
Adds the module option `ephemeral`. If set to true, configures the
runner registration with the `--ephemeral` option. This causes the
runner to exit after processing a single job, to de-register itself, and
to delete its configuration. Afterward, systemd restarts the service
which triggers a new ephemeral registration with a clean state.
2022-08-15 13:13:58 -04:00
Vincent Haupert 3f075e5bb1 nixos/github-runner: add PAT support
This commit introduces support for runner registrations through a
personal access token (PAT). To use a PAT instead of a registration
token, place an appropriately scoped PAT in `tokenFile`. If the file
contains a PAT, the configuration script queries a new runner
registration token. Using a runner registration token directly continues
to work as before.
2022-08-15 13:13:58 -04:00
Vincent Haupert 65542a6348 nixos/github-runner: use state instead of runtime dir as RUNNER_ROOT
Using the runtime directory as `RUNNER_ROOT` is wrong. We should always
use the state directory like we already do when invoking the runner
configure script. Otherwise, the runner constructs the wrong path for
some files (.credentials, .runner, ...).
2022-08-15 13:13:58 -04:00
pennae 93c57a9884
Merge pull request #185056 from pennae/option-docs-md
nixos/*: more option docs conversions
2022-08-05 17:36:49 +02:00
Sandro 3125eef570
Merge pull request #182713 from SuperSandro2000/hydra-runuser-2 2022-08-04 12:08:44 +02:00
pennae 61e93df189 nixos/*: automatically convert option docs to MD
once again using nix-doc-munge (69d080323a)
2022-08-03 22:46:41 +02:00
pennae 3aebb4a2be nixos/*: normalize link format
make (almost) all links appear on only a single line, with no
unnecessary whitespace, using double quotes for attributes. this lets us
automatically convert them to markdown easily.

the few remaining links are extremely long link in a gnome module, we'll
come back to those at a later date.
2022-08-03 21:57:46 +02:00
pennae 6b13dd0e9e
Merge pull request #183491 from pennae/automatic-md-conversions
treewide: automatically md-convert option descriptions
2022-08-02 02:15:30 +02:00
pennae 2e751c0772 treewide: automatically md-convert option descriptions
the conversion procedure is simple:

 - find all things that look like options, ie calls to either `mkOption`
   or `lib.mkOption` that take an attrset. remember the attrset as the
   option
 - for all options, find a `description` attribute who's value is not a
   call to `mdDoc` or `lib.mdDoc`
 - textually convert the entire value of the attribute to MD with a few
   simple regexes (the set from mdize-module.sh)
 - if the change produced a change in the manual output, discard
 - if the change kept the manual unchanged, add some text to the
   description to make sure we've actually found an option. if the
   manual changes this time, keep the converted description

this procedure converts 80% of nixos options to markdown. around 2000
options remain to be inspected, but most of those fail the "does not
change the manual output check": currently the MD conversion process
does not faithfully convert docbook tags like <code> and <package>, so
any option using such tags will not be converted at all.
2022-07-30 15:16:34 +02:00
misuzu 9b1db3db0e nixos/gitlab-runner: undeprecate configFile option 2022-07-28 17:16:55 +02:00
misuzu b4028126f1 nixos/gitlab-runner: add settings option 2022-07-28 17:16:55 +02:00
Sandro 87c66cbb56
nixos/hydra: fix runuser in init 2022-07-24 18:05:10 +02:00
Bjørn Forsman 16108ff74a nixos/jenkins-job-builder: set serviceConfig.Type = "oneshot"
This change allows detecting configuration errors during
switch-to-configuration instead of them being reported asynchronously
*after* switch-to-configuration has exited.

(And update the NixOS test accordingly.)
2022-07-23 13:30:53 +02:00
Vincent Haupert 539b61ea37 nixos/github-runner: fix capset syscall filtering
capset(2) is a single system call, not a set of multiple system calls.
2022-07-21 16:08:15 +02:00
Alex Martens c34749dd63 nixos/github-runner: fix systemd defaults for common workflows 2022-07-17 22:02:57 -07:00
Bjørn Forsman 0080a93cdf nixos/jenkins-job-builder: create secret file with umask 0077
IOW, don't make it world readable.
2022-07-17 15:24:48 +02:00
Bjørn Forsman 50eaf82b6f nixos/jenkins-job-builder: fix jenkins authentication
The current authentication code is broken against newer jenkins:

  jenkins-job-builder-start[1257]: Asking Jenkins to reload config
  jenkins-start[789]: 2022-07-12 14:34:31.148+0000 [id=17]        WARNING hudson.security.csrf.CrumbFilter#doFilter: Found invalid crumb 31e96e52938b51f099a61df9505a4427cb9dca7e35192216755659032a4151df. If you are calling this URL with a script, please use the API Token instead. More information: https://www.jenkins.io/redirect/crumb-cannot-be-used-for-script
  jenkins-start[789]: 2022-07-12 14:34:31.160+0000 [id=17]        WARNING hudson.security.csrf.CrumbFilter#doFilter: No valid crumb was included in request for /reload by admin. Returning 403.
  jenkins-job-builder-start[1357]: curl: (22) The requested URL returned error: 403

Fix it by using `jenkins-cli` instead of messing with `curl`.

This rewrite also prevents leaking the password in process listings. (We
could probably do it without `replace-secret`, assuming `printf` is a
shell built-in, but this implementation should be safe even with shells
not having a built-in `printf`.)

Ref https://github.com/NixOS/nixpkgs/issues/156400.
2022-07-16 12:30:41 +02:00
Sandro Jäckel 90761632ae
nixos/hydra: use runuser like hydra flake 2022-07-07 22:37:51 +02:00
Sebastien Bourdeauducq 6cecab9cd8 hydra: create runcommand-logs directory 2022-06-06 11:36:58 -07:00
Rick van Schijndel 3a7f7c1153 hydra-unstable -> hydra_unstable 2022-05-10 23:32:02 +02:00
Ivan Kozik 59a76614f3 treewide: chown user:group instead of user.group to fix warnings from coreutils 9.1 2022-05-05 22:05:18 +00:00
Matthieu Coudron c1e318d0c5 services.gitlab-runner: support runner description
The description for the runner in the UI is by default sthg like
"npm_nixos_d0544ed48909" i.e., the name of the attribute.

I wanted to have a more user-friendly description and added a
description to the service.
Seems like gitlab-runner doesn't like having both fields set:
"Cannot use two forms of the same flag: description name"
so I used one or the other.
2022-04-26 11:43:46 +02:00
Lassulus c29e523824
Merge pull request #163030 from newAM/github-runner-harden
nixos/github-runner: systemd service hardening
2022-04-20 16:30:23 +01:00
P. R. d. O eeb3206a29 nixos/jenkinsSlave: install java
Installs Java into the Jenkins agent and allows specifying the JDK/JRE package to use. This is necessary as Jenkins verifies if the agent contains Java installed through the java -fullversion command, which if not, the connection will fail.
2022-04-01 22:11:18 +02:00
Alex Martens 334b30c464 nixos/github-runner: systemd service hardening 2022-04-01 12:22:10 -07:00
Lucas Hoffmann f3bda2f2c0 docs: replace defunct git://github.com urls with https:// 2022-03-25 11:21:42 +01:00
Profpatsch 75b08e3624 modules/github-runner: Improve description of url
We spent a whole afternoon debugging this, because upstream has very
bad software quality and the error messages were incredibly
misleading.

So let’s document it for the sanity of other people.

Btw, I think the implementation of our module is pretty brittle,
especially the part about diffing tokens to check whether they
changed. We should rather just request a new builder registration
every time, it’s not that much overhead, and always set `replace` so
it is idempotent.
2022-03-11 10:26:39 +01:00
Vincent Haupert 7c2319905d nixos/github-runner: use --disableupdate flag
Use the new `--disableupdate` configure flag instead of our patch.
2022-01-28 09:46:12 +01:00
polykernel 4a9d9928dc nixos/nix-daemon: use structural settings
The `nix.*` options, apart from options for setting up the
daemon itself, currently provide a lot of setting mappings
for the Nix daemon configuration. The scope of the mapping yields
convience, but the line where an option is considered essential
is blurry. For instance, the `extra-sandbox-paths` mapping is
provided without its primary consumer, and the corresponding
`sandbox-paths` option is also not mapped.

The current system increases the maintenance burden as maintainers have to
closely follow upstream changes. In this case, there are two state versions
of Nix which have to be maintained collectively, with different options
avaliable.

This commit aims to following the standard outlined in RFC 42[1] to
implement a structural setting pattern. The Nix configuration is encoded
at its core as key-value pairs which maps nicely to attribute sets, making
it feasible to express in the Nix language itself. Some existing options are
kept such as `buildMachines` and `registry` which present a simplified interface
to managing the respective settings. The interface is exposed as `nix.settings`.

Legacy configurations are mapped to their corresponding options under `nix.settings`
for backwards compatibility.

Various options settings in other nixos modules and relevant tests have been
updated to use structural setting for consistency.

The generation and validation of the configration file has been modified to
use `writeTextFile` instead of `runCommand` for clarity. Note that validation
is now mandatory as strict checking of options has been pushed down to the
derivation level due to freeformType consuming unmatched options. Furthermore,
validation can not occur when cross-compiling due to current limitations.

A new option `publicHostKey` was added to the `buildMachines`
submodule corresponding to the base64 encoded public host key settings
exposed in the builder syntax. The build machine generation was subsequently
rewritten to use `concatStringsSep` for better performance by grouping
concatenations.

[1] - https://github.com/NixOS/rfcs/blob/master/rfcs/0042-config-option.md
2022-01-26 21:04:50 -05:00
Graham Christensen 06edb74413
Merge pull request #148785 from pennae/more-option-doc-staticizing
treewide: more defaultText for options
2021-12-17 11:14:08 -05:00
Bobby Rong 94144484c2
Merge pull request #148164 from veehaitch/nixos-github-runner-148024-v2
nixos/github-runner: refactor tokens handling
2021-12-17 16:28:21 +08:00
pennae fb0e5be843 treewide: add defaultText for options with simple interpolation defaults
adds defaultText for all options that use `cfg.*` values in their
defaults, but only for interpolations with no extra processing (other
than toString where necessary)
2021-12-09 01:13:48 +01:00
pennae 2512455639 nixos/*: add trivial defaultText for options with simple defaults 2021-12-02 22:35:04 +01:00
Vincent Haupert 3cf9508c72 nixos/github-runner: refactor tokens handling
This commit changes how we deal with the current token, i.e., the token
which may exist from a previous runner registration, and the configured
token, i.e., the path set for the respective NixOS configuration option.

Until now, we copied the configured and the current token (if any) to
the runtime directory to compare them. The path of the current token may
reference a file which is only accessible to specific users (even only
root). Therefore, we ran the copying of credentials with elevated
privileges by prefixing the `ExecStartPre=` script with a `+` (see
systemd.service(5)). In this script, we also changed the owner of the
files to the service user. Apparently, however, the user/group pair
sometimes did not exist because we use `DynamicUser=`.

To address this issue, we no longer change the owner of the file.
Instead, we change the file permissions to 0666 to allow the runner
configuration script (runs with full sandboxing) to read-write the file.
Due to the current permissions of the runtime directory (0755), this
would expose the token. Therefore, we process the tokens in the state
directory, which is only accessible to the service user.

If a new token file exists in the state directory, the configuration
script should trigger a new runner registration. Afterward, it deletes
the new token file. The token is still available using the path of the
current token which is inaccessible within the service's sandbox.
2021-12-01 16:15:43 +01:00
Robert Hensing 3a98abe8bd nixos/hercules-ci-agent: Update module 2021-11-19 11:05:04 +01:00
Naïm Favier 2ddc335e6f
nixos/doc: clean up defaults and examples 2021-10-04 12:47:20 +02:00
Andreas Fuchs 15d14ad150
github-runner: Allow configuring the package to use (#134661)
This addresses #120263 in part, by allowing users to override the
github-runner derivation that is bound to turn non-functional via the
self-update mechanism. (And it'll allow using a buildFHSUserEnv-based
derivation, if someone ends up building that!)
2021-08-24 11:23:39 +02:00
Daniel Nagy 79a86e7ef5
treewide: Port type adaptations 2021-08-11 00:45:08 +02:00
Pascal Bach ee50c21488
Merge pull request #111768 from misuzu/gitlab-runner-warnings
nixos/gitlab-runner: warn about possible secrets leak
2021-08-02 21:41:50 +02:00
Sandro d5bd34ebf2
treewide: convert phases that contain ":" to dont* = true (#130500) 2021-07-18 23:42:48 +02:00
Franz Pletz f73efb9fb3
Merge pull request #121841 from Pamplemousse/jenkins-cli
jenkins: Create the `jenkins-cli` command
2021-07-06 14:45:29 +00:00