mirror of
https://github.com/NixOS/nixpkgs.git
synced 2024-09-11 15:08:33 +01:00
0fccd5bba4
The documentation got a bit stale compared to actual contents of nixpkgs. This commit focuses on updating existing docs, not on making sure all details of beam packages are covered.
457 lines
17 KiB
XML
457 lines
17 KiB
XML
<section xmlns="http://docbook.org/ns/docbook"
|
||
xmlns:xlink="http://www.w3.org/1999/xlink"
|
||
xml:id="sec-beam">
|
||
|
||
<title>Beam Languages (Erlang, Elixir & LFE)</title>
|
||
<section xml:id="beam-introduction">
|
||
<title>Introduction</title>
|
||
<para>
|
||
In this document and related Nix expressions we use the term
|
||
<emphasis>Beam</emphasis> to describe the environment. BEAM is
|
||
the name of the Erlang Virtial Machine and, as far as we know,
|
||
from a packaging perspective all languages that run on BEAM are
|
||
interchangable. The things that do change, like the build
|
||
system, are transperant to the users of the package. So we make
|
||
no distinction.
|
||
</para>
|
||
</section>
|
||
<section xml:id="beam-structure">
|
||
<title>Structure</title>
|
||
<para>
|
||
All Beam-related things are available via top-level
|
||
<literal>beam</literal> attribute, which includes:
|
||
</para>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<para>
|
||
<literal>interpreters</literal>: contains a set of compilers running
|
||
on Beam, including multiple Erlang/OTP versions
|
||
(<literal>beam.interpreters.erlangR19</literal>, etc), Elixir
|
||
(<literal>beam.interpreters.elixir</literal>) and LFE
|
||
(<literal>beam.interpreters.lfe</literal>).
|
||
</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para>
|
||
<literal>packages</literal>: contains a set of package sets, each
|
||
compiled with a specific Erlang/OTP version, e.g.
|
||
<literal>beam.packages.erlangR19</literal>.
|
||
</para>
|
||
</listitem>
|
||
</itemizedlist>
|
||
<para>
|
||
Default Erlang compiler is defined by
|
||
<literal>beam.interpreters.erlang</literal> and aliased as
|
||
<literal>erlang</literal>. Default package set with Beam packages is
|
||
defined by <literal>beam.packages.erlang</literal> and aliased at
|
||
top-level as <literal>beamPackages</literal>.
|
||
</para>
|
||
<para>
|
||
If you want to create a package set built with a custom Erlang version,
|
||
use lambda <literal>beam.packagesWith</literal>, which accepts an
|
||
Erlang/OTP derivative and produces a package set similar to
|
||
<literal>beam.packages.erlang</literal>.
|
||
</para>
|
||
<para>
|
||
Many Erlang/OTP distributions available in
|
||
<literal>beam.interpreters</literal> have their versions with ODBC and/or
|
||
Java enabled. For example there's
|
||
<literal>beam.interpreters.erlangR19_odbc_javac</literal> which
|
||
corresponds to <literal>beam.interpreters.erlangR19</literal>.
|
||
</para>
|
||
<para>
|
||
We also provide <literal>beam.packages.erlang.callPackage</literal>, which
|
||
simplifies writing Beam package definitions, by injecting all packages from
|
||
<literal>beam.packages.erlang</literal> into top-level context.
|
||
</para>
|
||
</section>
|
||
<section xml:id="build-tools">
|
||
<title>Build Tools</title>
|
||
<section xml:id="build-tools-rebar3">
|
||
<title>Rebar3</title>
|
||
<para>
|
||
By default Rebar3 wants to manage it's own dependencies. This is perfectly
|
||
acceptable in the normal non-Nix setup. In the Nix world it is not. To
|
||
support this we have created two versions of rebar3,
|
||
<literal>rebar3</literal> and <literal>rebar3-open</literal>. The
|
||
<literal>rebar3</literal> version has been patched to remove the ability
|
||
to download anything from it. If you are not running it a nix-shell or a
|
||
nix-build then its probably not going to work for you.
|
||
<literal>rebar3-open</literal> is the normal, un-modified rebar3. It
|
||
should work exactly as would any other version of rebar3. Any Erlang
|
||
package should rely on <literal>rebar3</literal> and thats really what you
|
||
should be using too. See <literal>buildRebar3</literal> below.
|
||
</para>
|
||
</section>
|
||
<section xml:id="build-tools-other">
|
||
<title>Mix & Erlang.mk</title>
|
||
<para>
|
||
Both Mix and Erlang.mk work exactly as you would expect. There
|
||
is a bootstrap process that needs to be run for both of
|
||
them. However, that is supported by the
|
||
<literal>buildMix</literal> and <literal>buildErlangMk</literal> derivations.
|
||
</para>
|
||
</section>
|
||
</section>
|
||
|
||
<section xml:id="how-to-install-beam-packages">
|
||
<title>How to install Beam packages</title>
|
||
<para>
|
||
Beam packages are not registered in the top level simply because they are
|
||
not relevant to the vast majority of Nix users. They are installable using
|
||
the <literal>beam.packages.erlang</literal> attribute set and aliased as
|
||
<literal>beamPackages</literal>. This attribute points at packages built by
|
||
the default Erlang/OTP version in Nixpkgs as defined by
|
||
<literal>beam.interpreters.erlang</literal>.
|
||
|
||
You can list the avialable packages in the
|
||
<literal>beamPackages</literal> with the following command:
|
||
</para>
|
||
|
||
<programlisting>
|
||
$ nix-env -f "<nixpkgs>" -qaP -A beamPackages
|
||
beamPackages.esqlite esqlite-0.2.1
|
||
beamPackages.goldrush goldrush-0.1.7
|
||
beamPackages.ibrowse ibrowse-4.2.2
|
||
beamPackages.jiffy jiffy-0.14.5
|
||
beamPackages.lager lager-3.0.2
|
||
beamPackages.meck meck-0.8.3
|
||
beamPackages.rebar3-pc pc-1.1.0
|
||
</programlisting>
|
||
<para>
|
||
To install any of those packages into your profile, refer to them by
|
||
their attribute path (first column):
|
||
</para>
|
||
<programlisting>
|
||
$ nix-env -f "<nixpkgs>" -iA beamPackages.ibrowse
|
||
</programlisting>
|
||
<para>
|
||
The attribute path of any Beam packages corresponds to the name
|
||
of that particular package in Hex or its OTP Application/Release name.
|
||
</para>
|
||
</section>
|
||
<section xml:id="packaging-beam-applications">
|
||
<title>Packaging Beam Applications</title>
|
||
<section xml:id="packaging-erlang-applications">
|
||
<title>Erlang Applications</title>
|
||
<section xml:id="rebar3-packages">
|
||
<title>Rebar3 Packages</title>
|
||
<para>
|
||
There is a Nix function called <literal>buildRebar3</literal> (defined
|
||
in <literal>beam.packages.erlang.buildRebar3</literal> and aliased at
|
||
top-level). We use this function to make a derivation that understands
|
||
how to build the rebar3 project. For example, the expression we use to
|
||
build the <link
|
||
xlink:href="https://github.com/erlang-nix/hex2nix">hex2nix</link>
|
||
project follows.
|
||
</para>
|
||
<programlisting>
|
||
{stdenv, fetchFromGitHub, buildRebar3, ibrowse, jsx, erlware_commons }:
|
||
|
||
buildRebar3 rec {
|
||
name = "hex2nix";
|
||
version = "0.0.1";
|
||
|
||
src = fetchFromGitHub {
|
||
owner = "ericbmerritt";
|
||
repo = "hex2nix";
|
||
rev = "${version}";
|
||
sha256 = "1w7xjidz1l5yjmhlplfx7kphmnpvqm67w99hd2m7kdixwdxq0zqg";
|
||
};
|
||
|
||
beamDeps = [ ibrowse jsx erlware_commons ];
|
||
}
|
||
</programlisting>
|
||
<para>
|
||
This derivation is callable with
|
||
<literal>beam.packages.erlang.callPackage</literal> (see above). If you
|
||
want to call this package using normal <literal>callPackage</literal>,
|
||
you should refer to dependency packages via
|
||
<literal>beamPackages</literal>, e.g.
|
||
<literal>beamPackages.ibrowse</literal>.
|
||
</para>
|
||
<para>
|
||
The only visible difference between this derivation and
|
||
something like <literal>stdenv.mkDerivation</literal> is that we
|
||
have added <literal>beamDeps</literal> to the derivation. If
|
||
you add your Beam dependencies here they will be correctly
|
||
handled by the system.
|
||
</para>
|
||
<para>
|
||
If your package needs to compile native code via Rebar's port
|
||
compilation mechenism. You should add <literal>compilePort =
|
||
true;</literal> to the derivation.
|
||
</para>
|
||
</section>
|
||
<section xml:id="erlang-mk-packages">
|
||
<title>Erlang.mk Packages</title>
|
||
<para>
|
||
Erlang.mk functions almost identically to Rebar. The only real
|
||
difference is that <literal>buildErlangMk</literal> is called
|
||
instead of <literal>buildRebar3</literal>
|
||
</para>
|
||
<programlisting>
|
||
{ buildErlangMk, fetchHex, cowlib, ranch }:
|
||
buildErlangMk {
|
||
name = "cowboy";
|
||
version = "1.0.4";
|
||
src = fetchHex {
|
||
pkg = "cowboy";
|
||
version = "1.0.4";
|
||
sha256 =
|
||
"6a0edee96885fae3a8dd0ac1f333538a42e807db638a9453064ccfdaa6b9fdac";
|
||
};
|
||
beamDeps = [ cowlib ranch ];
|
||
|
||
meta = {
|
||
description = ''Small, fast, modular HTTP server written in
|
||
Erlang.'';
|
||
license = stdenv.lib.licenses.isc;
|
||
homepage = "https://github.com/ninenines/cowboy";
|
||
};
|
||
}
|
||
</programlisting>
|
||
</section>
|
||
<section xml:id="mix-packages">
|
||
<title>Mix Packages</title>
|
||
<para>
|
||
Mix functions almost identically to Rebar. The only real
|
||
difference is that <literal>buildMix</literal> is called
|
||
instead of <literal>buildRebar3</literal>
|
||
</para>
|
||
<programlisting>
|
||
{ buildMix, fetchHex, plug, absinthe }:
|
||
buildMix {
|
||
name = "absinthe_plug";
|
||
version = "1.0.0";
|
||
src = fetchHex {
|
||
pkg = "absinthe_plug";
|
||
version = "1.0.0";
|
||
sha256 =
|
||
"08459823fe1fd4f0325a8bf0c937a4520583a5a26d73b193040ab30a1dfc0b33";
|
||
};
|
||
beamDeps = [ plug absinthe ];
|
||
|
||
meta = {
|
||
description = ''A plug for Absinthe, an experimental GraphQL
|
||
toolkit'';
|
||
license = stdenv.lib.licenses.bsd3;
|
||
homepage = "https://github.com/CargoSense/absinthe_plug";
|
||
};
|
||
}
|
||
</programlisting>
|
||
<para>
|
||
Alternatively one can use <literal>buildHex</literal> as a shortcut for the above:
|
||
</para>
|
||
<programlisting>
|
||
{ buildHex, buildMix, plug, absinthe }:
|
||
buildHex {
|
||
name = "absinthe_plug";
|
||
version = "1.0.0";
|
||
sha256 =
|
||
"08459823fe1fd4f0325a8bf0c937a4520583a5a26d73b193040ab30a1dfc0b33";
|
||
builder = buildMix;
|
||
beamDeps = [ plug absinthe];
|
||
|
||
meta = {
|
||
description = ''A plug for Absinthe, an experimental GraphQL
|
||
toolkit'';
|
||
license = stdenv.lib.licenses.bsd3;
|
||
homepage = "https://github.com/CargoSense/absinthe_plug";
|
||
};
|
||
}
|
||
</programlisting>
|
||
</section>
|
||
</section>
|
||
</section>
|
||
<section xml:id="how-to-develop">
|
||
<title>How to develop</title>
|
||
<section xml:id="accessing-an-environment">
|
||
<title>Accessing an Environment</title>
|
||
<para>
|
||
Often, all you want to do is be able to access a valid
|
||
environment that contains a specific package and its
|
||
dependencies. we can do that with the <literal>env</literal>
|
||
part of a derivation. For example, lets say we want to access an
|
||
erlang repl with ibrowse loaded up. We could do the following.
|
||
</para>
|
||
<programlisting>
|
||
~/w/nixpkgs ❯❯❯ nix-shell -A beamPackages.ibrowse.env --run "erl"
|
||
Erlang/OTP 18 [erts-7.0] [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]
|
||
|
||
Eshell V7.0 (abort with ^G)
|
||
1> m(ibrowse).
|
||
Module: ibrowse
|
||
MD5: 3b3e0137d0cbb28070146978a3392945
|
||
Compiled: January 10 2016, 23:34
|
||
Object file: /nix/store/g1rlf65rdgjs4abbyj4grp37ry7ywivj-ibrowse-4.2.2/lib/erlang/lib/ibrowse-4.2.2/ebin/ibrowse.beam
|
||
Compiler options: [{outdir,"/tmp/nix-build-ibrowse-4.2.2.drv-0/hex-source-ibrowse-4.2.2/_build/default/lib/ibrowse/ebin"},
|
||
debug_info,debug_info,nowarn_shadow_vars,
|
||
warn_unused_import,warn_unused_vars,warnings_as_errors,
|
||
{i,"/tmp/nix-build-ibrowse-4.2.2.drv-0/hex-source-ibrowse-4.2.2/_build/default/lib/ibrowse/include"}]
|
||
Exports:
|
||
add_config/1 send_req_direct/7
|
||
all_trace_off/0 set_dest/3
|
||
code_change/3 set_max_attempts/3
|
||
get_config_value/1 set_max_pipeline_size/3
|
||
get_config_value/2 set_max_sessions/3
|
||
get_metrics/0 show_dest_status/0
|
||
get_metrics/2 show_dest_status/1
|
||
handle_call/3 show_dest_status/2
|
||
handle_cast/2 spawn_link_worker_process/1
|
||
handle_info/2 spawn_link_worker_process/2
|
||
init/1 spawn_worker_process/1
|
||
module_info/0 spawn_worker_process/2
|
||
module_info/1 start/0
|
||
rescan_config/0 start_link/0
|
||
rescan_config/1 stop/0
|
||
send_req/3 stop_worker_process/1
|
||
send_req/4 stream_close/1
|
||
send_req/5 stream_next/1
|
||
send_req/6 terminate/2
|
||
send_req_direct/4 trace_off/0
|
||
send_req_direct/5 trace_off/2
|
||
send_req_direct/6 trace_on/0
|
||
trace_on/2
|
||
ok
|
||
2>
|
||
</programlisting>
|
||
<para>
|
||
Notice the <literal>-A beamPackages.ibrowse.env</literal>.That
|
||
is the key to this functionality.
|
||
</para>
|
||
</section>
|
||
<section xml:id="creating-a-shell">
|
||
<title>Creating a Shell</title>
|
||
<para>
|
||
Getting access to an environment often isn't enough to do real
|
||
development. Many times we need to create a
|
||
<literal>shell.nix</literal> file and do our development inside
|
||
of the environment specified by that file. This file looks a lot
|
||
like the packaging described above. The main difference is that
|
||
<literal>src</literal> points to project root and we call the
|
||
package directly.
|
||
</para>
|
||
<programlisting>
|
||
{ pkgs ? import "<nixpkgs"> {} }:
|
||
|
||
with pkgs;
|
||
|
||
let
|
||
|
||
f = { buildRebar3, ibrowse, jsx, erlware_commons }:
|
||
buildRebar3 {
|
||
name = "hex2nix";
|
||
version = "0.1.0";
|
||
src = ./.;
|
||
beamDeps = [ ibrowse jsx erlware_commons ];
|
||
};
|
||
drv = beamPackages.callPackage f {};
|
||
|
||
in
|
||
drv
|
||
</programlisting>
|
||
<section xml:id="building-in-a-shell">
|
||
<title>Building in a shell (for mix projects)</title>
|
||
<para>
|
||
We can leverage the support of the Derivation, regardless of
|
||
which build Derivation is called by calling the commands themselves.
|
||
</para>
|
||
<programlisting>
|
||
# =============================================================================
|
||
# Variables
|
||
# =============================================================================
|
||
|
||
NIX_TEMPLATES := "$(CURDIR)/nix-templates"
|
||
|
||
TARGET := "$(PREFIX)"
|
||
|
||
PROJECT_NAME := thorndyke
|
||
|
||
NIXPKGS=../nixpkgs
|
||
NIX_PATH=nixpkgs=$(NIXPKGS)
|
||
NIX_SHELL=nix-shell -I "$(NIX_PATH)" --pure
|
||
# =============================================================================
|
||
# Rules
|
||
# =============================================================================
|
||
.PHONY= all test clean repl shell build test analyze configure install \
|
||
test-nix-install publish plt analyze
|
||
|
||
all: build
|
||
|
||
guard-%:
|
||
@ if [ "${${*}}" == "" ]; then \
|
||
echo "Environment variable $* not set"; \
|
||
exit 1; \
|
||
fi
|
||
|
||
clean:
|
||
rm -rf _build
|
||
rm -rf .cache
|
||
|
||
repl:
|
||
$(NIX_SHELL) --run "iex -pa './_build/prod/lib/*/ebin'"
|
||
|
||
shell:
|
||
$(NIX_SHELL)
|
||
|
||
configure:
|
||
$(NIX_SHELL) --command 'eval "$$configurePhase"'
|
||
|
||
build: configure
|
||
$(NIX_SHELL) --command 'eval "$$buildPhase"'
|
||
|
||
install:
|
||
$(NIX_SHELL) --command 'eval "$$installPhase"'
|
||
|
||
test:
|
||
$(NIX_SHELL) --command 'mix test --no-start --no-deps-check'
|
||
|
||
plt:
|
||
$(NIX_SHELL) --run "mix dialyzer.plt --no-deps-check"
|
||
|
||
analyze: build plt
|
||
$(NIX_SHELL) --run "mix dialyzer --no-compile"
|
||
|
||
</programlisting>
|
||
<para>
|
||
If you add the <literal>shell.nix</literal> as described and
|
||
user rebar as follows things should simply work. Aside from the
|
||
<literal>test</literal>, <literal>plt</literal>, and
|
||
<literal>analyze</literal> the talks work just fine for all of
|
||
the build Derivations.
|
||
</para>
|
||
</section>
|
||
</section>
|
||
</section>
|
||
<section xml:id="generating-packages-from-hex-with-hex2nix">
|
||
<title>Generating Packages from Hex with Hex2Nix</title>
|
||
<para>
|
||
Updating the Hex packages requires the use of the
|
||
<literal>hex2nix</literal> tool. Given the path to the Erlang
|
||
modules (usually
|
||
<literal>pkgs/development/erlang-modules</literal>). It will
|
||
dump a file called
|
||
<literal>hex-packages.nix</literal>. That file will contain all
|
||
the packages that use a recognized build system in Hex. However,
|
||
it can't know whether or not all those packages are buildable.
|
||
</para>
|
||
<para>
|
||
To make life easier for our users, it makes good sense to go
|
||
ahead and attempt to build all those packages and remove the
|
||
ones that don't build. To do that, simply run the command (in
|
||
the root of your <literal>nixpkgs</literal> repository). that follows.
|
||
</para>
|
||
<programlisting>
|
||
$ nix-build -A beamPackages
|
||
</programlisting>
|
||
<para>
|
||
That will build every package in
|
||
<literal>beamPackages</literal>. Then you can go through and
|
||
manually remove the ones that fail. Hopefully, someone will
|
||
improve <literal>hex2nix</literal> in the future to automate
|
||
that.
|
||
</para>
|
||
</section>
|
||
</section>
|