3
0
Fork 0
forked from mirrors/nixpkgs
Nix Packages collection
Go to file
Andreas Rammhold 322eb3596b cpython: add enableNoSemanticInterposition flag
This flags enables the -fno-semantic-interposition compiler flag for all
our cpython builds by default. It can be disabled by overriding the
enableNoSemanticInterposition attribute.

This CFLAG has the impact that our python code runs up to 23% faster
than previously. See the pyperformance benchmark results below.

The only downside that exists is that we can no longer override
libpython symbols via LD_PRELOAD when calling the python interpreter. If
a user needs they can disable the newly introduced flag and use that
instead. However those situations should be rare.

The Fedora Project did the equivalent change in their Fedora 32 release
(and haven't reverted it yet). They wrote a nice [wiki page] about the
ups and downs of this change. I recommend reading that.

Benchmark results of running Python 3.9 before and after this change:

+-------------------------+---------------+--------------------+--------------+------------------------+
| Benchmark               | py39.nix.json | py39-nsip.nix.json | Change       | Significance           |
+=========================+===============+====================+==============+========================+
| 2to3                    | 666 ms        | 642 ms             | 1.04x faster | Significant (t=12.73)  |
+-------------------------+---------------+--------------------+--------------+------------------------+
| chameleon               | 15.0 ms       | 14.6 ms            | 1.03x faster | Significant (t=4.70)   |
+-------------------------+---------------+--------------------+--------------+------------------------+
| chaos                   | 198 ms        | 182 ms             | 1.09x faster | Significant (t=13.73)  |
+-------------------------+---------------+--------------------+--------------+------------------------+
| crypto_pyaes            | 185 ms        | 175 ms             | 1.06x faster | Significant (t=9.70)   |
+-------------------------+---------------+--------------------+--------------+------------------------+
| deltablue               | 12.3 ms       | 11.2 ms            | 1.10x faster | Significant (t=14.45)  |
+-------------------------+---------------+--------------------+--------------+------------------------+
| django_template         | 85.1 ms       | 82.0 ms            | 1.04x faster | Significant (t=6.61)   |
+-------------------------+---------------+--------------------+--------------+------------------------+
| dulwich_log             | 102 ms        | 101 ms             | 1.01x faster | Not significant        |
+-------------------------+---------------+--------------------+--------------+------------------------+
| fannkuch                | 670 ms        | 634 ms             | 1.06x faster | Significant (t=15.87)  |
+-------------------------+---------------+--------------------+--------------+------------------------+
| float                   | 182 ms        | 176 ms             | 1.03x faster | Significant (t=7.32)   |
+-------------------------+---------------+--------------------+--------------+------------------------+
| go                      | 393 ms        | 366 ms             | 1.07x faster | Significant (t=17.63)  |
+-------------------------+---------------+--------------------+--------------+------------------------+
| hexiom                  | 15.8 ms       | 14.9 ms            | 1.06x faster | Significant (t=13.81)  |
+-------------------------+---------------+--------------------+--------------+------------------------+
| json_dumps              | 19.3 ms       | 18.7 ms            | 1.03x faster | Significant (t=7.46)   |
+-------------------------+---------------+--------------------+--------------+------------------------+
| json_loads              | 38.4 us       | 37.9 us            | 1.01x faster | Not significant        |
+-------------------------+---------------+--------------------+--------------+------------------------+
| logging_format          | 15.0 us       | 14.2 us            | 1.05x faster | Significant (t=6.32)   |
+-------------------------+---------------+--------------------+--------------+------------------------+
| logging_silent          | 328 ns        | 305 ns             | 1.07x faster | Significant (t=8.85)   |
+-------------------------+---------------+--------------------+--------------+------------------------+
| logging_simple          | 13.8 us       | 13.0 us            | 1.06x faster | Significant (t=10.77)  |
+-------------------------+---------------+--------------------+--------------+------------------------+
| mako                    | 25.2 ms       | 24.7 ms            | 1.02x faster | Not significant        |
+-------------------------+---------------+--------------------+--------------+------------------------+
| meteor_contest          | 133 ms        | 130 ms             | 1.03x faster | Significant (t=3.59)   |
+-------------------------+---------------+--------------------+--------------+------------------------+
| nbody                   | 222 ms        | 201 ms             | 1.10x faster | Significant (t=26.85)  |
+-------------------------+---------------+--------------------+--------------+------------------------+
| nqueens                 | 161 ms        | 152 ms             | 1.06x faster | Significant (t=8.84)   |
+-------------------------+---------------+--------------------+--------------+------------------------+
| pathlib                 | 28.4 ms       | 26.3 ms            | 1.08x faster | Significant (t=11.85)  |
+-------------------------+---------------+--------------------+--------------+------------------------+
| pickle                  | 13.8 us       | 13.1 us            | 1.06x faster | Significant (t=10.84)  |
+-------------------------+---------------+--------------------+--------------+------------------------+
| pickle_dict             | 32.7 us       | 26.6 us            | 1.23x faster | Significant (t=32.43)  |
+-------------------------+---------------+--------------------+--------------+------------------------+
| pickle_list             | 4.39 us       | 4.34 us            | 1.01x faster | Not significant        |
+-------------------------+---------------+--------------------+--------------+------------------------+
| pickle_pure_python      | 782 us        | 738 us             | 1.06x faster | Significant (t=15.71)  |
+-------------------------+---------------+--------------------+--------------+------------------------+
| pidigits                | 184 ms        | 181 ms             | 1.01x faster | Not significant        |
+-------------------------+---------------+--------------------+--------------+------------------------+
| pyflate                 | 1.02 sec      | 959 ms             | 1.07x faster | Significant (t=21.13)  |
+-------------------------+---------------+--------------------+--------------+------------------------+
| python_startup          | 34.3 ms       | 34.0 ms            | 1.01x faster | Not significant        |
+-------------------------+---------------+--------------------+--------------+------------------------+
| python_startup_no_site  | 15.5 ms       | 15.3 ms            | 1.01x faster | Not significant        |
+-------------------------+---------------+--------------------+--------------+------------------------+
| raytrace                | 912 ms        | 849 ms             | 1.07x faster | Significant (t=11.21)  |
+-------------------------+---------------+--------------------+--------------+------------------------+
| regex_compile           | 273 ms        | 261 ms             | 1.05x faster | Significant (t=5.82)   |
+-------------------------+---------------+--------------------+--------------+------------------------+
| regex_dna               | 188 ms        | 187 ms             | 1.00x faster | Not significant        |
+-------------------------+---------------+--------------------+--------------+------------------------+
| regex_effbot            | 3.37 ms       | 3.66 ms            | 1.09x slower | Significant (t=-16.12) |
+-------------------------+---------------+--------------------+--------------+------------------------+
| regex_v8                | 29.8 ms       | 29.7 ms            | 1.01x faster | Not significant        |
+-------------------------+---------------+--------------------+--------------+------------------------+
| richards                | 119 ms        | 113 ms             | 1.05x faster | Significant (t=5.71)   |
+-------------------------+---------------+--------------------+--------------+------------------------+
| scimark_fft             | 625 ms        | 592 ms             | 1.06x faster | Significant (t=19.64)  |
+-------------------------+---------------+--------------------+--------------+------------------------+
| scimark_lu              | 273 ms        | 253 ms             | 1.08x faster | Significant (t=16.68)  |
+-------------------------+---------------+--------------------+--------------+------------------------+
| scimark_monte_carlo     | 186 ms        | 170 ms             | 1.10x faster | Significant (t=14.70)  |
+-------------------------+---------------+--------------------+--------------+------------------------+
| scimark_sor             | 330 ms        | 310 ms             | 1.07x faster | Significant (t=11.89)  |
+-------------------------+---------------+--------------------+--------------+------------------------+
| scimark_sparse_mat_mult | 9.03 ms       | 8.36 ms            | 1.08x faster | Significant (t=17.71)  |
+-------------------------+---------------+--------------------+--------------+------------------------+
| spectral_norm           | 247 ms        | 232 ms             | 1.06x faster | Significant (t=14.64)  |
+-------------------------+---------------+--------------------+--------------+------------------------+
| sqlalchemy_declarative  | 194 ms        | 185 ms             | 1.04x faster | Significant (t=4.80)   |
+-------------------------+---------------+--------------------+--------------+------------------------+
| sqlalchemy_imperative   | 28.2 ms       | 27.4 ms            | 1.03x faster | Significant (t=3.80)   |
+-------------------------+---------------+--------------------+--------------+------------------------+
| sqlite_synth            | 4.89 us       | 4.73 us            | 1.03x faster | Significant (t=10.60)  |
+-------------------------+---------------+--------------------+--------------+------------------------+
| sympy_expand            | 833 ms        | 813 ms             | 1.02x faster | Significant (t=4.82)   |
+-------------------------+---------------+--------------------+--------------+------------------------+
| sympy_integrate         | 33.0 ms       | 31.6 ms            | 1.05x faster | Significant (t=6.47)   |
+-------------------------+---------------+--------------------+--------------+------------------------+
| sympy_str               | 490 ms        | 477 ms             | 1.03x faster | Significant (t=7.58)   |
+-------------------------+---------------+--------------------+--------------+------------------------+
| sympy_sum               | 254 ms        | 245 ms             | 1.04x faster | Significant (t=7.82)   |
+-------------------------+---------------+--------------------+--------------+------------------------+
| telco                   | 11.6 ms       | 11.3 ms            | 1.03x faster | Significant (t=5.62)   |
+-------------------------+---------------+--------------------+--------------+------------------------+
| tornado_http            | 175 ms        | 172 ms             | 1.02x faster | Not significant        |
+-------------------------+---------------+--------------------+--------------+------------------------+
| unpack_sequence         | 51.4 ns       | 56.2 ns            | 1.09x slower | Significant (t=-2.40)  |
+-------------------------+---------------+--------------------+--------------+------------------------+
| unpickle                | 20.3 us       | 19.8 us            | 1.02x faster | Significant (t=4.09)   |
+-------------------------+---------------+--------------------+--------------+------------------------+
| unpickle_list           | 5.41 us       | 5.75 us            | 1.06x slower | Significant (t=-26.56) |
+-------------------------+---------------+--------------------+--------------+------------------------+
| unpickle_pure_python    | 544 us        | 524 us             | 1.04x faster | Significant (t=6.47)   |
+-------------------------+---------------+--------------------+--------------+------------------------+
| xml_etree_generate      | 154 ms        | 148 ms             | 1.04x faster | Significant (t=7.98)   |
+-------------------------+---------------+--------------------+--------------+------------------------+
| xml_etree_iterparse     | 130 ms        | 129 ms             | 1.01x faster | Not significant        |
+-------------------------+---------------+--------------------+--------------+------------------------+
| xml_etree_parse         | 178 ms        | 179 ms             | 1.00x slower | Not significant        |
+-------------------------+---------------+--------------------+--------------+------------------------+
| xml_etree_process       | 123 ms        | 118 ms             | 1.04x faster | Significant (t=10.22)  |
+-------------------------+---------------+--------------------+--------------+------------------------+

[wiki page]: https://fedoraproject.org/wiki/Changes/PythonNoSemanticInterpositionSpeedup
2021-07-05 12:34:02 -07:00
.github workflows: fix documentation for periodic 24 hours merge 2021-07-01 20:16:36 +09:00
doc python3: 3.8 -> 3.9 2021-07-05 12:34:02 -07:00
lib strings.nix: Fix overly monomorphic type signature comments 2021-07-04 20:52:48 +02:00
maintainers Merge pull request #129005 from pimeys/planetscale 2021-07-05 03:22:20 +01:00
nixos Merge master into staging-next 2021-07-05 18:01:11 +00:00
pkgs cpython: add enableNoSemanticInterposition flag 2021-07-05 12:34:02 -07:00
.editorconfig Merge pull request #110395 from zowoq/gemset 2021-01-22 09:31:07 +01:00
.gitattributes
.gitignore
.version 21.11 is Porcupine! 2021-05-22 18:14:06 -07:00
CONTRIBUTING.md CONTRIBUTING.md: Move to repo root, where it is more visible. 2021-06-26 04:51:38 +02:00
COPYING COPYING: 2020 -> 2021 2021-01-03 00:49:09 +00:00
default.nix
flake.nix flake.nix: add aarch64-darwin 2021-05-17 00:27:02 +09:00
README.md CONTRIBUTING.md: Move to repo root, where it is more visible. 2021-06-26 04:51:38 +02:00

NixOS logo

Code Triagers badge Open Collective supporters

Nixpkgs is a collection of over 60,000 software packages that can be installed with the Nix package manager. It also implements NixOS, a purely-functional Linux distribution.

Manuals

  • NixOS Manual - how to install, configure, and maintain a purely-functional Linux distribution
  • Nixpkgs Manual - contributing to Nixpkgs and using programming-language-specific Nix expressions
  • Nix Package Manager Manual - how to write Nix expressions (programs), and how to use Nix command line tools

Community

Other Project Repositories

The sources of all official Nix-related projects are in the NixOS organization on GitHub. Here are some of the main ones:

  • Nix - the purely functional package manager
  • NixOps - the tool to remotely deploy NixOS machines
  • nixos-hardware - NixOS profiles to optimize settings for different hardware
  • Nix RFCs - the formal process for making substantial changes to the community
  • NixOS homepage - the NixOS.org website
  • hydra - our continuous integration system
  • NixOS Artwork - NixOS artwork

Continuous Integration and Distribution

Nixpkgs and NixOS are built and tested by our continuous integration system, Hydra.

Artifacts successfully built with Hydra are published to cache at https://cache.nixos.org/. When successful build and test criteria are met, the Nixpkgs expressions are distributed via Nix channels.

Contributing

Nixpkgs is among the most active projects on GitHub. While thousands of open issues and pull requests might seem a lot at first, it helps consider it in the context of the scope of the project. Nixpkgs describes how to build tens of thousands of pieces of software and implements a Linux distribution. The GitHub Insights page gives a sense of the project activity.

Community contributions are always welcome through GitHub Issues and Pull Requests. When pull requests are made, our tooling automation bot, OfBorg will perform various checks to help ensure expression quality.

The Nixpkgs maintainers are people who have assigned themselves to maintain specific individual packages. We encourage people who care about a package to assign themselves as a maintainer. When a pull request is made against a package, OfBorg will notify the appropriate maintainer(s). The Nixpkgs committers are people who have been given permission to merge.

Most contributions are based on and merged into these branches:

  • master is the main branch where all small contributions go
  • staging is branched from master, changes that have a big impact on Hydra builds go to this branch
  • staging-next is branched from staging and only fixes to stabilize and security fixes with a big impact on Hydra builds should be contributed to this branch. This branch is merged into master when deemed of sufficiently high quality

For more information about contributing to the project, please visit the contributing page.

Donations

The infrastructure for NixOS and related projects is maintained by a nonprofit organization, the NixOS Foundation. To ensure the continuity and expansion of the NixOS infrastructure, we are looking for donations to our organization.

You can donate to the NixOS foundation by using Open Collective:

License

Nixpkgs is licensed under the MIT License.

Note: MIT license does not apply to the packages built by Nixpkgs, merely to the files in this repository (the Nix expressions, build scripts, NixOS modules, etc.). It also might not apply to patches included in Nixpkgs, which may be derivative works of the packages to which they apply. The aforementioned artifacts are all covered by the licenses of the respective packages.