This breaks the builder when a nix-shell or keepBuildTree is used. The
issue occurs because paths to cargo lockfiles are read with NIX_BUILD_TOP,
which is not reliable.
This breaks a nix-shell because NIX_BUILD_TOP simply is not set, causing
an invalid path to be used. This can be worked around using
NIX_BUILD_TOP=$PWD, but that obviously is not great.
This breaks keepBuildTree because it changes the working directory to a
different path than NIX_BUILD_TOP. Since the lockfiles are copied based
on the working directory, but read based on NIX_BUILD_TOP, this causes
the hook to not be able to find them.
This was solved by both reading these files based on the working directory,
using absolute paths to avoid having to traverse back in the directory tree.
Fixes: #138554
Rust binaries are unconditionally linked to libiconv on Darwin (see https://github.com/rust-lang/libc/issues/2870). We already add it as a dependency in `buildRustPackage`, so let's go a step further and propagate it.
`toTargetArch` in `pkgs/build-support/rust/lib/default.nix` is used to
set `CARGO_CFG_TARGET_ARCH`. This environment variable is supposed to
be the `<arch>` portion of an LLVM-style platform name:
```
<arch><sub>-<kernel>-<libc><abi>
```
Note that the pointer-width (the "64" in "x86_64" and "mips64") is
part of `<arch>`, but the endianness (the `_be` in `aarch64_be`) is
*not*.
Unfortunately at the moment nixpkgs' parsed `cpuType` has no way to
query for the three subparts (name, pointer-width, and
subarch/endianness), nor any way to ask for just the first two parts.
For now, this commit simply fixes the problem in the two cases that
matter: `mips64el` and `powerpc64le`, which I believe are the only two
platforms supported by both rust and nixpkgs which have a
"subarchitecture".
Tell rust if we want our binaries linked statically or dynamically.
Otherwise the compiler will always produce statically linked binaries for musl
targets, as this is the default.
We are replicating one mechanism behind `-Z build-std`.
There isn't yet crate2nix support for this, but one can (and I do) add
the missing stdlib deps (for this feature to pick up) with overrides.
With Rust 1.61, it is necessary to link to external static/dynamic libaries
when building the rlib that uses them, rather than when linking the final
binary. In fact, it is no longer necessary to specify the libraries to link
when building the final binary, but the library search path flags must still
be included.
The old logic flow had the structure
if ( … ) {
if ( … ) {
…
} else {
…
}
} else {
…
}
which is quite hard to follow in Nix. Instead we ensure that no if
expression is inside a then branch.
This change is zero rebuild, as no logic was changed.
This parameter is being set to `$NIX_BUILD_CORES` by default. This is a
standard practice but there's a suspicion that this can produce broken
builds. For some details see
https://github.com/cargo2nix/cargo2nix/issues/184 . As a
work-around/test, it'd be good if codegen-units can be set to something
constant, such as `1`. This PR allows it.
Note that the default of `$NIX_BUILD_CORES` is preserved so this MR
causes no change in default behaviour and no rebuilds.
These features are internal-only, have special characters that bash
doesn't support in variable names, and aren't normally given
environment variables by cargo as far as I can tell.
Some crates do not have a Cargo.toml at the top-level, but only in
nested directories. Before this change importCargoLock used to fail with:
error: manifest path `/nix/store/some-store-path/Cargo.toml` does not exist
- `toRustTarget` and friends pulled out from rust tools into rust
library. Since they don't depend on any packages they can be more
widely useable.
- `build-rust-package` gets its own directory
- `fetch-cargo-tarball` gets its own directory
In restricted mode (and therefore with flakes) `builtins.readFile` may not be the result of `builtins.toFile`,
making it impossible to use a generated lockFile (with or without IFD),
and thereby causing evaluation to fail if `system != builtins.currentSystem` on Hydra
so the jobs are not delegated to eligible build machines that support that system.
This is done in a way that avoids rebuilds.
I currently do not have much time to work on nixpkgs. Remove
myself as a maintainer from a bunch of packages to avoid that
people are waiting on me for a review.
near the end of 2019, the default Cargo.lock format was changed to
[[package]]
checksum = ...
This is what importCargoLock assumes. If the crate had not been `cargo
update`'d with a more recent toolchain than the one with the new
format as default, importCargoLock would fail when trying to access
pkg.checksum.
I ran into such a case (shamefully, in my own crate) and it took me a
while to figure out what was going on, so here is an assert with a
more user friendly message and a hint.
According to rustc implementation[1], `-C incremental=no` enables
incremental builds with directory name `no`. This patch removes the
`-C incremental` argument to disable incremental builds.
[1]: ee86f96ba1/compiler/rustc_session/src/options.rs (L918-L919)
This change introduces the cargoLock argument to buildRustPackage,
which can be used in place of cargo{Sha256,Hash} or cargoVendorDir. It
uses the importCargoLock function to build the vendor
directory. Differences compared to cargo{Sha256,Hash}:
- Requires a Cargo.lock file.
- Does not require a Cargo hash.
- Retrieves all dependencies as fixed-output derivations.
This makes buildRustPackage much easier to use as part of a Rust
project, since it does not require updating cargo{Sha256,Hash} for
every change to the lock file.
This function can be used to create an output path that is a cargo
vendor directory. In contrast to e.g. fetchCargoTarball all the
dependent crates are fetched using fixed-output derivations. The
hashes for the fixed-output derivations are gathered from the
Cargo.lock file.
Usage is very simple, e.g.:
importCargoLock {
lockFile = ./Cargo.lock;
}
would use the lockfile from the current directory.
The implementation of this function is based on Eelco Dolstra's
import-cargo:
https://github.com/edolstra/import-cargo/blob/master/flake.nix
Compared to upstream:
- We use fetchgit in place of builtins.fetchGit.
- Sync to current cargo vendoring.
Also begin to start work on cross compilation, though that will have to
be finished later.
The patches are based on the first version of
https://reviews.llvm.org/D99484. It's very annoying to do the
back-porting but the review has uncovered nothing super major so I'm
fine sticking with what I've got.
Beyond making the outputs work, I also strove to re-sync the packages,
as they have been drifting pointlessly apart for some time.
----
Other misc notes, highly incomplete
- lvm-config-native and llvm-config are put in `dev` because they are
tools just for build time.
- Clang no longer has an lld dep. That was introduced in
db29857eb3, but if clang needs help
finding lld when it is used we should just pass it flags / put in the
resource dir. Providing it at build time increases critical path
length for no good reason.
----
A note on `nativeCC`:
`stdenv` takes tools from the previous stage, so:
1. `pkgsBuildBuild`: `(?1, x, x)`
2. `pkgsBuildBuild.stdenv.cc`: `(?0, ?1, x)`
while:
1. `pkgsBuildBuild`: `(?1, x, x)`
2. `pkgsBuildBuild.targetPackages`: `(x, x, ?2)`
3. `pkgsBuildBuild.targetPackages.stdenv.cc`: `(?1, x, x)`
The `checkType` argument of buildRustPackage was not used anymore
since the refactoring of `buildRustPackage` into hooks. This was
an oversight that is fixed by this change.
The check type can also be passed directly to cargoCheckHook using the
`cargoCheckType` environment variable.
API change:
`cargoParallelTestThreads` suggests that this attribute sets the
number of threads used during tests, while it is actually a boolean
option (use 1 thread or NIX_BUILD_CORES threads). In the hook, this
is replaced by a more canonical name `dontUseCargoParallelTests`.
