3
0
Fork 0
forked from mirrors/nixpkgs
Commit graph

277 commits

Author SHA1 Message Date
Vladimír Čunát f9f6f41bff Merge branch 'master' into closure-size
TODO: there was more significant refactoring of qtbase and plasma 5.5
on master, and I'm deferring pointing to correct outputs to later.
2015-12-31 09:53:02 +01:00
aszlig a5bc11f9eb
nixos/vm-tests: Remove msize mount option
This seems to be the root cause of the random page allocation failures
and @wizeman did a very good job on not only finding the root problem
but also giving a detailed explanation of it in #10828.

Here is an excerpt:

  The problem here is that the kernel is trying to allocate a contiguous
  section of 2^7=128 pages, which is 512 KB. This is way too much:
  kernel pages tend to get fragmented over time and kernel developers
  often go to great lengths to try allocating at most only 1 contiguous
  page at a time whenever they can.

  From the error message, it looks like the culprit is unionfs, but this
  is misleading: unionfs is the name of the userspace process that was
  running when the system ran out of memory, but it wasn't unionfs who
  was allocating the memory: it was the kernel; specifically it was the
  v9fs_dir_readdir_dotl() function, which is the code for handling the
  readdir() function in the 9p filesystem (the filesystem that is used
  to share a directory structure between a qemu host and its VM).

  If you look at the code, here's what it's doing at the moment it tries
  to allocate memory:

    buflen = fid->clnt->msize - P9_IOHDRSZ;

    rdir = v9fs_alloc_rdir_buf(file, buflen);

  If you look into v9fs_alloc_rdir_buf(), you will see that it will try
  to allocate a contiguous buffer of memory (using kzalloc(), which is a
  wrapper around kmalloc()) of size buflen + 8 bytes or so.

  So in reality, this code actually allocates a buffer of size
  proportional to fid->clnt->msize. What is this msize? If you follow
  the definition of the structures, you will see that it's the
  negotiated buffer transfer size between 9p client and 9p server. On
  the client side, it can be controlled with the msize mount option.

  What this all means is that, the reason for running out of memory is
  that the code (which we can't easily change) tries to allocate a
  contiguous buffer of size more or less equal to "negotiated 9p
  protocol buffer size", which seems to be way too big (in our NixOS
  tests, at least).

After that initial finding, @lethalman tested the gnome3 gdm test
without setting the msize parameter at all and it seems to have resolved
the problem.

The reason why I'm committing this without testing against all of the
NixOS VM test is basically that I think we can only go better but not
worse than the current state.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2015-12-14 17:26:24 +01:00
Luca Bruno a412927924 Merge remote-tracking branch 'origin/master' into closure-size 2015-11-25 21:37:30 +01:00
Eelco Dolstra 75e41b0210 Add Fedora 23 2015-11-25 16:18:15 +01:00
Eelco Dolstra e4eee41ad0 Add Ubuntu 15.10 2015-11-25 15:40:08 +01:00
Vladimír Čunát 333d69a5f0 Merge staging into closure-size
The most complex problems were from dealing with switches reverted in
the meantime (gcc5, gmp6, ncurses6).
It's likely that darwin is (still) broken nontrivially.
2015-11-20 14:32:58 +01:00
aszlig 82d88b5bcb
vmTools.commonDebPackages: Add "mawk".
While debugging an issue with running NixOps tests, I found out that the
output from debClosureGenerator is not deterministic.

The reason behind this is the way how Provides and Replaces fields are
handled. I haven't yet found out what's the exact issue, but so far
packages "Provides" are more or less picked at random.

So, running the NixOps Hetzner tests we get either mawk, original-awk or
gawk altering on every invocation.

While for the test it isn't poisionous whether wi have mawk or gawk,
having original-awk certainly is, because live-build only works with
mawk or gawk.

The best solution would obviously be to make debClosureGenerator
deterministic, but in the case of "Provides: awk", we can safely pick
mawk by default, because the latter has a "Priority: required" in its
package description.

This also has the advantage that we can safely cherry-pick this to
release-15.09 because it's very unlikely that we'll break the
debClosureGenerator by adding a dependency to commonDebPackages.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2015-10-26 20:56:42 +01:00
Eelco Dolstra 7260c04266 debian: Update to 7.9, 8.2 2015-10-08 11:20:26 +02:00
Eelco Dolstra 106738b196 Give more memory for the disk image builder
http://hydra.nixos.org/build/26480662
2015-10-06 15:03:26 +02:00
Vladimír Čunát 5227fb1dd5 Merge commit staging+systemd into closure-size
Many non-conflict problems weren't (fully) resolved in this commit yet.
2015-10-03 13:33:37 +02:00
Eelco Dolstra 69829dd4ff Debian: Update to 8.1 2015-06-12 11:22:59 +02:00
Eelco Dolstra 99d2e80d97 In VM builds, connect stdin to /dev/null
This prevents programs from hanging while trying to read from stdin.
2015-06-04 20:08:56 +02:00
Benjamin Staffin 1db5bafc94 Typo fix! s/wheezy/jessie/ 2015-05-29 11:16:33 +02:00
Benjamin Staffin c7ac0ed3eb Add Debian 8.0
Had to add file type logic in debClosureGenerator because Packages.bz2
is not available for Jessie - only Packages.xz is.
2015-05-29 10:49:20 +02:00
Eelco Dolstra 178c52e7ad Add Ubuntu 15.04 2015-05-22 13:21:02 +02:00
Eelco Dolstra fef86de618 Make module loading work in VMs
Also, execute stage 2 as PID 1.
2015-05-13 17:36:58 +02:00
Eelco Dolstra b97362989f Provide /run in VMs 2015-05-13 17:36:29 +02:00
Eelco Dolstra c3f6c8c582 Mount /dev/shm in VMs 2015-05-07 15:31:36 +02:00
Vladimír Čunát 3b9ef2c71b fix "libc}/lib" and similar references
Done mostly without any verification.
I didn't bother with libc}/include, as the path is still correct.
2015-05-05 11:52:08 +02:00
Eelco Dolstra 3a31043b43 fillDiskWithRPMs: Pass --extract-over-symlinks to cpio
http://hydra.nixos.org/build/20003320
2015-02-27 14:00:11 +01:00
Eelco Dolstra 28ea9050fa Remove ancient RedHat 9 / SuSE 9 images
These are 12 years old and I don't think anybody is using them.
2015-02-12 17:55:29 +01:00
Eelco Dolstra da73688e3d fillDiskWithRPMs: Shut up some warnings 2015-02-12 17:55:28 +01:00
Eelco Dolstra 3786f08d25 Add Fedora 21 2015-02-10 11:32:43 +01:00
Eelco Dolstra 1b1e015e6e Update Debian to 7.8 2015-01-15 14:51:55 +01:00
Eelco Dolstra 7358b8dfda Fix a warning in initrd generation
${pkgs.busybox}/bin and ${pkgs.busybox}/sbin are now the same
directory, so don't copy both.
2015-01-12 17:52:30 +01:00
Eelco Dolstra 872f8c0431 Remove RHEL 7 RC
It no longer exists on the RedHat FTP site.
2015-01-12 15:13:21 +01:00
Eelco Dolstra 02df7cdaa0 Fix CentOS 6.5 URLs 2015-01-12 15:13:05 +01:00
Eelco Dolstra b8e33a5dcb Ubuntu 14.10: Update hashes 2014-12-05 16:02:59 +01:00
Eelco Dolstra 99a2afd784 Update to Debian 7.7 2014-11-20 11:11:21 +01:00
Eelco Dolstra e5513ac4dc Add Ubuntu 14.10 2014-11-20 11:11:08 +01:00
Eelco Dolstra fda5c17a6f vmTools: Fix permissions on /tmp and /dev/* 2014-11-18 14:49:39 +01:00
Shea Levy 3fe41ca763 Debain 7.6-amd64 tarball was updated in place 2014-10-20 15:26:51 -04:00
Shea Levy 652030e85d debian packages tarball was updated in place 2014-10-20 15:23:44 -04:00
Mathijs Kwik e80cbedbae VM builds: Use default kernel
(this was set to 3.10 when 3.4 was our default)
See commit d846e97656.
2014-09-23 09:17:01 +02:00
Eelco Dolstra 05e81e0d9f debian: Update to 6.0.10 2014-08-10 15:17:00 +02:00
Eelco Dolstra 1245ca3ff7 debian: Update to 7.6 2014-07-14 14:35:09 +02:00
Eelco Dolstra 139608dd34 Update Ubuntu Packages.bz2 hashes 2014-05-20 13:39:06 +02:00
Eelco Dolstra 0ff68e81b9 debian: Update to 7.5
[Note from Austin: I think @edolstra forgot to merge this to master.]

(cherry picked from commit 02b056c5b1 on
release-14.04)
2014-05-03 10:55:26 -05:00
Austin Seipp 2be1b4c034 build-support/vm: add Ubuntu 14.04 Trusty Tahir
Signed-off-by: Austin Seipp <aseipp@pobox.com>
2014-05-03 10:51:32 -05:00
Rob Vermaas eaedf303a0 Add RHEL7 to vm functions. 2014-05-01 15:13:35 +02:00
Eelco Dolstra 1bae93cb82 debian: Update to 6.0.9 2014-04-25 10:43:26 +02:00
Eelco Dolstra f73ff0529e runInLinuxVM: Set the clock properly 2014-03-18 15:02:09 +01:00
aszlig 88ea17284e
vmTools: Hook in runInWindowsVM function.
Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-02-26 06:21:48 +01:00
Petr Rockai 97721af469 VMs: Add Fedora 17 to the list of distros (only had 16 and 18). 2014-02-15 12:57:21 +01:00
Eelco Dolstra 8ba1fdef00 debian: Update to 7.4 2014-02-14 20:29:43 +01:00
Shea Levy b529a34b4a Add Fedora 20
Signed-off-by: Shea Levy <shea@shealevy.com>
2014-01-19 20:50:46 -05:00
Rob Vermaas 1625743902 Fix Fedora8 vm builds. 2014-01-15 21:32:38 +01:00
Rob Vermaas ed9520bfb5 Allow specifying extra qemu flags using QEMU_OPTS. Replace CentOS 6.3 with CentOS 6.5 (6.3 is removed from site). 2014-01-14 22:51:26 +01:00
Eelco Dolstra 3abcd7e268 debian: Update to 7.3 2014-01-06 15:51:00 +01:00
Eelco Dolstra 4747796a9e Build RPMs for i686 rather than i386
The default target (i386-linux) causes flags like "-march i386" to be
added, which breaks on recent Fedora releases (18 and up), resulting
in errors like:

  /usr/lib/gcc/i686-redhat-linux/4.7.2/../../../../include/c++/4.7.2/ext/atomicity.h:48: undefined reference to `__atomic_fetch_add_4'

So set the target to i686-linux.

http://hydra.nixos.org/build/6567357
2013-10-23 12:55:07 +02:00
Eelco Dolstra 391de89913 Debian 7: Update to 7.2
Also rename "debian70" to "debian7" to reflect the Wheezy's new
versioning scheme.
2013-10-23 11:30:51 +02:00
Eelco Dolstra d6f97c3601 Debian 6: Update to 6.0.8 2013-10-23 11:17:20 +02:00
Eelco Dolstra 27e91e0044 Add Fedora 19 2013-10-23 11:15:45 +02:00
Eelco Dolstra 34fcf33c0b Add Ubuntu 13.10 2013-10-23 11:08:12 +02:00
Eelco Dolstra d846e97656 VM builds: Use Linux 3.10
3.4 apparently gives corrupt 9pfs data on x86.  See e.g.

  http://hydra.nixos.org/build/5661036

where reading /nix/store/kfldnrrsq0lbbv13gjxdfb3vb1sbaz88-vm-run-stage2
gives garbage.  Strangely it doesn't happen here:

  http://hydra.nixos.org/build/5658978

Ah well.
2013-08-07 14:48:22 +02:00
Eelco Dolstra 4342a32deb VM builds: Reduce kernel verbosity 2013-08-01 14:35:31 +02:00
Eelco Dolstra d077851b7d VM builds: Use the default kernel 2013-08-01 14:35:31 +02:00
Eelco Dolstra b5fcb5b67d Use the qemu-kvm wrapper 2013-07-31 14:53:35 +02:00
aszlig 69dccda3c0
VM builds: Update debian Wheezy image to v7.1.
Wheezy has been released on June 15th and on all mirrors the SHA256 hash
of Packages.bz2 has changed to reflect the new release, so let's update.

Here is the release announcement from Debian:

http://www.debian.org/News/2013/20130615

It also seems that the versioning scheme has changed in version 7.x, so
they seem to have switched to a two digit versioning scheme. This means,
that the attribute name "debian70..." should really be something like
"debian7...", but I'm keeping the attribute as-is to not break
references.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2013-07-30 05:59:59 +02:00
aszlig cd4b5e1a35
fillDiskWithDebs: Create fake start-stop-daemon.
This is needed in order to prevent services from starting while
populating the image with the contents of the .deb files. The procedure
used here is exactly the same as used in debootstrap.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2013-07-30 05:59:40 +02:00
Eelco Dolstra 59aca26975 * Use 'or'. 2013-07-15 14:33:27 +02:00
Rob Vermaas da7db90068 Add CentOS 6.4 to vm images 2013-07-05 17:52:42 +02:00
Rob Vermaas 51a22a8f8a Add loopback network device and localhost entry to /etc/hosts for VM tests. 2013-07-05 15:02:43 +02:00
Eelco Dolstra 9f5f30a7e6 Remove runInGenericVM
It's not used anywhere and probably doesn't work anymore.
2013-07-05 00:17:04 +02:00
Eelco Dolstra 0e386d0c13 VM builds: Use 9p/virtfs instead of CIFS
9p (with caching enabled) is much faster than CIFS and doesn't require
Samba or virtual networking.  For instance, building GNU Hello with
CIFS takes ~323s on my laptop, but with 9p it takes 54s.

More measurements will be needed to see if "cache=fscache" is really
faster than "cache=loose" (the former seems to be a little bit
faster).
2013-07-05 00:17:04 +02:00
Eelco Dolstra 9efe759dd8 VM builds: Increase the default memory size to 512 MB 2013-07-05 00:17:04 +02:00
Eelco Dolstra 2321f2d55c VM builds: Panic on OOM 2013-07-05 00:17:04 +02:00
Eelco Dolstra 5f8571405b VM builds: Use ext4 instead of ext2 2013-07-05 00:17:03 +02:00
Eelco Dolstra ed37a6b465 VM builds: Use qemu instead of the obsolete qemu-kvm 2013-07-05 00:17:03 +02:00
Eelco Dolstra 5cfa68ab50 Reinstate sec=none
Seems to be required, despite the subsequent sec=ntlm. But the NixOS
VM tests work fine without this flag :-S

http://hydra.nixos.org/build/5451901
2013-07-03 14:38:56 +02:00
Eelco Dolstra 75e34854f6 Remove redundant cifs option 2013-07-02 17:03:29 +02:00
Shea Levy dd42dd480b runInLinuxImage: Fix derivation overriding.
This only ever worked because runInLinuxVM happened to call
overrideDerivation, which itself erroneously passed arbitrarily-added
attributes to the new call to derivation.

Hopefully this time Eelco won't have to revert my change ;)

Signed-off-by: Shea Levy <shea@shealevy.com>
2013-05-15 01:15:16 -04:00
Eelco Dolstra 9f9620f566 Doh 2013-05-13 22:04:33 +02:00
Eelco Dolstra 278b255388 Restore unintentionally deleted line 2013-05-13 18:15:02 +02:00
Eelco Dolstra 9df603b300 Use Linux 3.9 for VM builds
This seems to fix CIFS hangs like: http://hydra.nixos.org/build/4916655
2013-05-13 18:12:12 +02:00
Eelco Dolstra 6fb4c62f25 Fix some old Fedora builds
http://hydra.nixos.org/build/4949447
2013-05-13 17:47:20 +02:00
Eelco Dolstra 5815d18460 Add Fedora 18 2013-05-08 14:08:59 +02:00
Eelco Dolstra e7480d9cb7 Add Debian 7.0 2013-05-07 11:19:46 +02:00
Eelco Dolstra 1073187f78 Add Ubuntu 13.04 2013-05-03 14:01:32 +02:00
Eelco Dolstra fb600a5d99 Debian VM image: Update to 6.0.7 2013-03-15 12:48:35 +01:00
Eelco Dolstra b643316a42 VM builds: Use BusyBox
http://hydra.nixos.org/build/3510928
2012-12-18 16:57:11 +01:00
Lluís Batlle i Rossell 62fa1b8782 Adding ext4 to vmTools. Otherwise, nixos build-vm fails with 'useBootLoader'. 2012-12-16 21:43:35 +01:00
Eelco Dolstra 7f115621ca Add Ubuntu 12.10 2012-12-04 20:06:07 +01:00
Eelco Dolstra 85650db656 Update Debian Squeeze to 6.0.6 2012-12-04 19:58:21 +01:00
Eelco Dolstra b29d424115 Get deb-closure.pl to work with the latest dpkg 2012-12-04 19:56:31 +01:00
Eelco Dolstra 2ab46949cf Add universe repo for older Ubuntu releases 2012-09-13 17:15:58 -04:00
Eelco Dolstra fc35bed470 Fix incorrect version of Ubuntu Maverick (10.10) 2012-09-13 17:15:58 -04:00
Eelco Dolstra 20de8c8086 Make the "universe" repository available to the Ubuntu image generator
Also fix Ubuntu 12.04 name from "oneiric" to "precise".
2012-08-27 13:53:07 -04:00
Eelco Dolstra bacc6ab790 Remove trailing whitespace 2012-08-27 13:53:07 -04:00
Eelco Dolstra 395718e927 Create /dev/random and /dev/urandom in VMs
WWW::Curl needs this to prevent "Fatal: no entropy gathering module
detected".
2012-08-27 13:53:07 -04:00
Eelco Dolstra a54734e087 vmTools: Provide hook to add more Samba shares 2012-08-16 10:55:13 -04:00
Rob Vermaas 0cbfd8ff04 * update hashes for debian queeze
svn path=/nixpkgs/trunk/; revision=34316
2012-06-01 17:53:53 +00:00
Eelco Dolstra b6917d3b73 * Added Ubuntu 12.04.
svn path=/nixpkgs/trunk/; revision=34055
2012-05-11 02:04:14 +00:00
Eelco Dolstra 9aa30ba705 * Pass -cpu kvm64 to keep GMP from barfing.
svn path=/nixpkgs/trunk/; revision=33848
2012-04-19 18:44:02 +00:00
Eelco Dolstra 8b0bc7a745 * Add modules required by Linux 3.2.
svn path=/nixpkgs/trunk/; revision=33742
2012-04-11 07:45:12 +00:00
Ludovic Courtès 9d125d64bf VM: Upgrade to Debian 5.0.10.
svn path=/nixpkgs/trunk/; revision=33008
2012-03-12 08:13:10 +00:00
Ludovic Courtès 0a433261c5 VM: Make sure `smbd' is in QEMU's $PATH.
svn path=/nixpkgs/trunk/; revision=32987
2012-03-11 14:23:32 +00:00
Ludovic Courtès 5d159f8a06 runInGenericVM: Use whatever the current system is.
svn path=/nixpkgs/trunk/; revision=32970
2012-03-10 11:44:47 +00:00
Ludovic Courtès 2f01e58e61 vmTools: Use stdenv's glibc in `initrd-utils'.
svn path=/nixpkgs/trunk/; revision=32878
2012-03-08 09:40:45 +00:00
Eelco Dolstra 5e4c7aa8f3 * This substitution causes bash to hang. Urgh. Bash's pattern
substitutions aren't very reliable.

svn path=/nixpkgs/trunk/; revision=32830
2012-03-06 18:49:44 +00:00