mirror of
https://github.com/NixOS/nixpkgs.git
synced 2024-12-25 03:17:13 +00:00
230 lines
9.2 KiB
Markdown
230 lines
9.2 KiB
Markdown
|
|
## Maintainer Workflow
|
|
|
|
This is the documentation for periodically merging the `haskell-updates` branch
|
|
into the `master` branch. This workflow is performed by members in the
|
|
[@NixOS/haskell](https://github.com/orgs/NixOS/teams/haskell) team.
|
|
Each member of the team takes a two week period where they are in charge of
|
|
merging the `haskell-updates` branch into `master`.
|
|
|
|
The goal of this workflow is to regularly merge the `haskell-updates` branch
|
|
into the `master` branch, while making sure there are no evaluation errors or
|
|
build errors that get into `master`.
|
|
|
|
The workflow generally proceeds in three main steps:
|
|
|
|
1. create the initial `haskell-updates` PR, and update Stackage and Hackage snapshots
|
|
1. wait for contributors to fix newly broken Haskell packages
|
|
1. merge `haskell-updates` into `master`
|
|
|
|
We describe each of these steps in a separate section.
|
|
|
|
### Initial `haskell-updates` PR
|
|
|
|
In this section we create the PR for merging `haskell-updates` into `master`.
|
|
|
|
1. Make sure the `haskell-updates` branch is up-to-date with `master`.
|
|
|
|
1. Update the Stackage Nightly resolver used by Nixpkgs and create a commit:
|
|
|
|
```console
|
|
$ ./maintainers/scripts/haskell/update-stackage.sh --do-commit
|
|
```
|
|
|
|
1. Update the Hackage package set used by Nixpkgs and create a commit:
|
|
|
|
```console
|
|
$ maintainers/scripts/haskell/update-hackage.sh --do-commit
|
|
```
|
|
|
|
1. Regenerate the Haskell package set used in Nixpkgs and create a commit:
|
|
|
|
```console
|
|
$ maintainers/scripts/haskell/regenerate-hackage-packages.sh --do-commit
|
|
```
|
|
|
|
1. Push these commits to the Nixpkgs repository.
|
|
|
|
1. Open a PR on Nixpkgs merging `haskell-updates` into `master`.
|
|
|
|
Use the following message body:
|
|
|
|
```markdown
|
|
### This Merge
|
|
|
|
This PR is the regular merge of the `haskell-updates` branch into `master`.
|
|
|
|
This branch is being continually built and tested by hydra at https://hydra.nixos.org/jobset/nixpkgs/haskell-updates.
|
|
|
|
I will aim to merge this PR **by 2021-TODO-TODO**. If I can merge it earlier, there might be successor PRs in that time window. As part of our rotation @TODO will continue these merges from 2021-TODO-TODO to 2021-TODO-TODO.
|
|
|
|
### haskellPackages Workflow Summary
|
|
|
|
Our workflow is currently described at `pkgs/development/haskell-modules/HACKING.md`.
|
|
|
|
The short version is this:
|
|
* We regularly update the Stackage and Hackage pins on `haskell-updates` (normally at the beginning of a merge window).
|
|
* The community fixes builds of Haskell packages on that branch.
|
|
* We aim at at least one merge of `haskell-updates` into `master` every two weeks.
|
|
* We only do the merge if the `mergeable` job is succeeding on hydra.
|
|
* If a maintained package is still broken at the time of merge, we will only merge if the maintainer has been pinged 7 days in advance. (If you care about a Haskell package, become a maintainer!)
|
|
|
|
---
|
|
|
|
This is the follow-up to #TODO.
|
|
```
|
|
|
|
Make sure to replace all TODO with the actual values.
|
|
|
|
### Notify Maintainers and Fix Broken Packages
|
|
|
|
After you've done the previous steps, Hydra will start building the new and
|
|
updated Haskell packages. You can see the progress Hydra is making at
|
|
https://hydra.nixos.org/jobset/nixpkgs/haskell-updates. This Hydra jobset is
|
|
defined in the file [release-haskell.nix](../../top-level/release-haskell.nix).
|
|
|
|
#### Notify Maintainers
|
|
|
|
When Hydra finishes building all the updated packages for the `haskell-updates`
|
|
jobset, you should generate a build report to notify maintainers of their
|
|
newly broken packages. You can do that with the following commands:
|
|
|
|
```console
|
|
$ maintainers/scripts/haskell/hydra-report.hs get-report
|
|
$ maintainers/scripts/haskell/hydra-report.hs ping-maintainers
|
|
```
|
|
|
|
The `hyda-report.hs ping-maintainers` command generates a Markdown document
|
|
that you can paste in a GitHub comment on the PR opened above. This
|
|
comment describes which Haskell packages are now failing to build. It also
|
|
pings the maintainers so that they know to fix up their packages.
|
|
|
|
It may be helpful to pipe `hydra-report.hs ping-maintainers` into `xclip`
|
|
(XOrg) or `wl-copy` (Wayland) in order to post on GitHub.
|
|
|
|
This build report can be fetched and re-generated for new Hydra evaluations.
|
|
It may help contributors to try to keep the GitHub comment updated with the
|
|
most recent build report.
|
|
|
|
Maintainers should be given at least 7 days to fix up their packages when they
|
|
break. If maintainers don't fix up their packages with 7 days, then they
|
|
may be marked broken before merging `haskell-updates` into `master`.
|
|
|
|
#### Fix Broken Packages
|
|
|
|
After getting the build report, you can see which packages and Hydra jobs are
|
|
failing to build. The most important jobs are the `maintained` and `mergeable`
|
|
jobs. These are both defined in
|
|
[`release-haskell.nix`](../../top-level/release-haskell.nix).
|
|
|
|
`mergeable` is a set of the most important Haskell packages, including things
|
|
like Pandoc and XMonad. These packages are widely used. We would like to
|
|
always keep these building.
|
|
|
|
`maintained` is a set of Haskell packages that have maintainers in Nixpkgs.
|
|
We should be proactive in working with maintainers to keep their packages
|
|
building.
|
|
|
|
Steps to fix Haskell packages that are failing to build is out of scope for
|
|
this document, but it usually requires fixing up dependencies that are now
|
|
out-of-bounds.
|
|
|
|
#### Mark Broken Packages
|
|
|
|
Packages that do not get fixed can be marked broken with the following
|
|
commands. First check which packages are broken:
|
|
|
|
```console
|
|
$ maintainers/scripts/haskell/hydra-report.hs get-report
|
|
$ maintainers/scripts/haskell/hydra-report.hs mark-broken-list
|
|
```
|
|
|
|
This shows a list of packages that reported a build failure on `x86_64-linux` on Hydra.
|
|
|
|
Next, run the following command:
|
|
|
|
```console
|
|
$ maintainers/scripts/haskell/mark-broken.sh --do-commit
|
|
```
|
|
|
|
This first opens up an editor with the broken package list. Some of these
|
|
packages may have a maintainer in Nixpkgs. If these maintainers have not been
|
|
given 7 days to fix up their package, then make sure to remove those packages
|
|
from the list before continuing. After saving and exiting the editor, the
|
|
following will happen:
|
|
|
|
- packages from the list will be added to
|
|
[`configuration-hackage2nix/broken.yaml`](configuration-hackage2nix/broken.yaml)
|
|
- `hackage-packages.nix` will be regenerated
|
|
- the
|
|
[`configuration-hackage2nix/transitive-broken.yaml`](configuration-hackage2nix/transitive-broken.yaml)
|
|
file will be updated
|
|
- `hackage-packages.nix` will be regenerated again
|
|
- everything will be committed
|
|
|
|
### Merge `haskell-updates` into `master`
|
|
|
|
Now it is time to merge the `haskell-updates` PR you opened above.
|
|
|
|
Before doing this, make sure of the following:
|
|
|
|
- All Haskell packages that fail to build are correctly marked broken or
|
|
transitively broken.
|
|
- The `maintained` and `mergeable` jobs are passing on Hydra.
|
|
- The maintainers for any maintained Haskell packages that are newly broken
|
|
have been pinged on GitHub and given at least a week to fix their packages.
|
|
This is especially important for widely-used packages like `cachix`.
|
|
|
|
When you've double-checked these points, go ahead and merge the `haskell-updates` PR.
|
|
After merging, **make sure not to delete the `haskell-updates` branch**, since it
|
|
causes all currently open Haskell-related pull-requests to be automatically closed on GitHub.
|
|
|
|
### Additional Info
|
|
|
|
Here are some additional tips that didn't fit in above.
|
|
|
|
- It is possible to start a new Hydra evaluation by logging into Hydra with
|
|
your GitHub or Google account.
|
|
|
|
- You should occasionally merge the `master` branch into the
|
|
`haskell-updates` branch.
|
|
|
|
In an ideal world, when we merge `haskell-updates` into `master`, it would
|
|
cause few Hydra rebuilds on `master`. Ideally, the `nixos-unstable`
|
|
channel would never be prevented from progressing because of needing to
|
|
wait for rebuilding Haskell packages.
|
|
|
|
In order to make sure that there are a minimal number of rebuilds after
|
|
merging `haskell-updates` into `master`, `master` should occasionally be
|
|
merged into the `haskell-updates` branch.
|
|
|
|
This is especially important after `staging-next` is merged into `master`, since
|
|
there is a high chance that this will cause all the Haskell packages to
|
|
rebuild.
|
|
|
|
However, as we are working on cleaning up `haskell-updates`, `master` will
|
|
continually progress. It may not always be possible to keep the
|
|
`haskell-updates` branch fully up-to-date with `master` without causing
|
|
mass-rebuilds on the `haskell-updates` jobset.
|
|
|
|
- Make sure never to update the Hackage package hashes in
|
|
[`pkgs/data/misc/hackage/`](../../../pkgs/data/misc/hackage/), or the
|
|
pinned Stackage Nightly versions on the release branches (like
|
|
`release-21.05`).
|
|
|
|
This means that the
|
|
[`update-hackage.sh`](../../../maintainers/scripts/haskell/update-hackage.sh)
|
|
and
|
|
[`update-stackage.sh`](../../../maintainers/scripts/haskell/update-stackage.sh)
|
|
scripts should never be used on the release branches.
|
|
|
|
However, changing other files in `./.` and regenerating the package set is encouraged.
|
|
This can be done with
|
|
[`regenerate-hackage-packages.sh`](../../../maintainers/scripts/haskell/regenerate-hackage-packages.sh)
|
|
as described above.
|
|
|
|
## Contributor Workflow
|
|
|
|
(TODO: this section is to describe the type of workflow for non-committers to
|
|
contribute to `haskell-updates`)
|