tvl-depot/doc/manual/overview.xml

<chapter id='chap-overview'>
  <title>Overview</title>

  <para>
    This chapter provides a guided tour of Nix.
  </para>


  
  <!--######################################################################-->

  <sect1>
    <title>Basic package management</title>

    <para>
      Let's start from the perspective of an end user.  Common operations at
      this level are to install and remove packages, ask what packages are
      installed or available for installation, and so on.  These are operations
      on the <emphasis>user environment</emphasis>: the set of packages that a
      user <quote>sees</quote>.  In a command line Unix environment, this means
      the set of programs that are available through the <envar>PATH</envar>
      environment variable.  (In other environments it might mean the set of
      programs available on the desktop, through the start menu, and so on.)
    </para>

    <para>
      The terms <quote>installation</quote> and <quote>uninstallation</quote>
      are used in this context to denote the act of adding or removing packages
      from the user environment.  In Nix, these operations are dissociated from
      the physical copying or deleting of files.  Installation requires that
      the files constituting the package are present, but they may be present
      beforehand.  Likewise, uninstallation does not actually delete any files;
      this is done automatically by running a garbage collector.
    </para>

    <para>
      User environments are manipulated through the <command>nix-env</command>
      command.  The query operation can be used to see what packages are
      currently installed.
    </para>

    <screen>
$ nix-env -q
MozillaFirebird-0.7
sylpheed-0.9.7
pan-0.14.2</screen>

    <para>
      (<option>-q</option> is actually short for <option>--query
        --installed</option>.)  The package names are symbolic: they don't have
      any particular significance to Nix (as they shouldn't, since they are not
      unique&mdash;there can be many derivations with the same name).  Note that
      these packages have many dependencies (e.g., Mozilla uses the
      <literal>gtk+</literal> package) but these have not been installed in the
      user environment, though they are present on the system.  Generally,
      there is no need to install such packages; only packages containing
      programs should be installed.
    </para>
      
    <para>
      To install packages, a <emphasis>Nix expression</emphasis> is required
      that tells Nix how to build that package.  There is a <ulink
        url='https://svn.cs.uu.nl:12443/dist/trace/trace-nixpkgs-trunk.tar.bz2'>set 
        of standard of Nix expressions</ulink> for many common packages.
      Assuming that you have downloaded and unpacked these, you can view the
      set of available packages:
    </para>

    <screen>
$ nix-env -qaf pkgs/system/i686-linux.nix
gettext-0.12.1
sylpheed-0.9.7
aterm-2.0
gtk+-1.2.10
apache-httpd-2.0.48
pan-0.14.2
...</screen>

    <para>
      The Nix expression in the file <filename>i686-linux.nix</filename> yields
      the set of packages for a Linux system running on x86 hardware.  For
      other platforms, copy and modify this file for your platform as
      appropriate. [TODO: improve this]
    </para>

    <para>
      It is also possible to see the <emphasis>status</emphasis> of available
      packages, i.e., whether they are installed into the user environment
      and/or present in the system:
    </para>

    <screen>
$ nix-env -qasf pkgs/system/i686-linux.nix
-P gettext-0.12.1
IP sylpheed-0.9.7
-- aterm-2.0
-P gtk+-1.2.10</screen>

    <para>
      This reveals that the <literal>sylpheed</literal> package is already
      installed, or more precisely, that exactly the same instantiation of
      <literal>sylpheed</literal> is installed.  This guarantees that the
      available package is exactly the same as the installed package with
      regard to sources, dependencies, build flags, and so on.  Similarly, we
      see that the <literal>gettext</literal> and <literal>gtk+</literal>
      packages are present but not installed in the user environment, while the
      <literal>aterm</literal> package is not installed or present at all (so,
      if we were to install it, it would have to be built or downloaded first).
    </para>

    <para>
      The install operation is used install available packages from a Nix
      environment.  To install the <literal>pan</literal> package (a
      newsreader), you would do:
    </para>

    <screen>
$ nix-env -if pkgs/system/i686-linux.nix pan</screen>

    <para>
      Since installation may take a long time, depending on whether any
      packages need to be built or downloaded, it's a good idea to make
      <command>nix-env</command> run verbosely by using the <option>-v</option>
      (<option>--verbose</option>) option.  This option may be repeated to
      increase the level of verbosity.  A good value is 3
      (<option>-vvv</option>).
    </para>

    <para>
      In fact, if you run this command verbosely you will observe that Nix
      starts to build many packages, including large and fundamental ones such
      as <literal>glibc</literal> and <literal>gcc</literal>.  I.e., you are
      performing a source installation.  This is generally undesirable, since
      installation from sources may require large amounts of disk and CPU
      resources.  Therefore a <quote>binary</quote> installation is generally
      preferable.
    </para>

    <para>
      Rather than provide different mechanisms to create and perform
      the installation of binary packages, Nix supports binary deployment
      <emphasis>transparently</emphasis> through a generic mechanism of
      <emphasis>substitute expressions</emphasis>.  If an request is made to
      build some Nix expression, Nix will first try to build any substitutes
      for that expression.  These substitutes presumably perform an identical
      build operation with respect to the result, but require less resources.
      For instance, a substitute that downloads a pre-built package from the
      network requires less CPU and disk resources, and possibly less time.
    </para>

    <para>
      Nix's use of cryptographic hashes makes this entirely safe.  It is not
      possible, for instance, to accidentally substitute a build of some
      package for a Solaris or Windows system for a build on a SuSE/x86 system.
    </para>

    <para>
      While the substitute mechanism is a generic mechanism, Nix provides two
      standard tools called <command>nix-pull</command> and
      <command>nix-push</command> that maintain and use a shared cache of
      prebuilt derivations on some network site (reachable through HTTP).  If
      you attempt to install some package that someone else has previously
      built and <quote>pushed</quote> into the cache, and you have done a
      <quote>pull</quote> to register substitutes that download these prebuilt
      packages, then the installation will automatically use these.
    </para>

    <para>
      For example, to pull from our <ulink
        url='http://losser.st-lab.cs.uu.nl/~eelco/nix-dist/'>cache</ulink> of
      prebuilt packages (at the time of writing, for SuSE Linux/x86), use the
      following command:
    </para>

    <screen>
$ nix-pull
http://catamaran.labs.cs.uu.nl/dist/nix/nixpkgs-<replaceable>version</replaceable>/
obtaining list of Nix archives at http://catamaran.labs.cs.uu.nl/dist/nix/nixpkgs-<replaceable>version</replaceable>...
...</screen>

    <para>
      If <command>nix-pull</command> is run without any arguments, it will pull
      from the URLs specified in the file
      <filename><replaceable>prefix</replaceable>/etc/nix/prebuilts.conf</filename>.
    </para>

    <para>
      Assuming that the <literal>pan</literal> installation produced no errors,
      it can be used immediately, that is, it now appears in a directory in the
      <envar>PATH</envar> environment variable.  Specifically,
      <envar>PATH</envar> includes the entry
      <filename><replaceable>prefix</replaceable>/var/nix/links/current/bin</filename>, 
      where
      <filename><replaceable>prefix</replaceable>/var/nix/links/current</filename> 
      is just a symlink to the current user environment:
    </para>

    <screen>
$ ls -l /nix/var/nix/links/
...
lrwxrwxrwx  1 eelco ... 15 -> /nix/store/1871...12b0-user-environment
lrwxrwxrwx  1 eelco ... 16 -> /nix/store/59ba...df6b-user-environment
lrwxrwxrwx  1 eelco ... current -> /nix/var/nix/links/16</screen>

    <para>
      That is, <filename>current</filename> in this example is a link to
      <filename>16</filename>, which is the current user environment.  Before
      the installation, it pointed to <filename>15</filename>.  Note that this
      means that you can atomically roll-back to the previous user environment
      by pointing the symlink <filename>current</filename> at
      <filename>15</filename> again.  This also shows that operations such as
      installation are atomic in the Nix system: any arbitrarily complex
      set of installation, uninstallation, or upgrade actions eventually boil
      down to the single operation of pointing a symlink somewhere else (which
      can be implemented atomically in Unix).
    </para>

    <para>
      What's in a user environment? It's just a set of symlinks to the files
      that constitute the installed packages.  For instance:
    </para>

    <screen>
$ ls -l /nix/var/nix/links/16/bin
lrwxrwxrwx  1 eelco ... MozillaFirebird -> /nix/store/35f8...4ae6-MozillaFirebird-0.7/bin/MozillaFirebird
lrwxrwxrwx  1 eelco ... svn -> /nix/store/3829...fb5d-subversion-0.32.1/bin/svn
...</screen>

    <para>
      Note that, e.g., <filename>svn</filename> =
      <filename>/nix/var/nix/links/current/bin/svn</filename> =
      <filename>/nix/var/nix/links/16/bin/svn</filename> =
      <filename>/nix/store/59ba...df6b-user-environment/bin/svn</filename> =
      <filename>/nix/store/3829...fb5d-subversion-0.32.1/bin/svn</filename>.
    </para>

    <para>
      Naturally, packages can also be uninstalled:
    </para>

    <screen>
$ nix-env -e pan</screen>

    <para>
      This means that the package is removed from the user
      environment.  It is <emphasis>not</emphasis> yet removed from
      the system.  When a package is uninstalled from a user
      environment, it may still be used by other packages, or may
      still be present in other user environments.  Deleting it under
      such conditions would break those other packages or user
      environments.  To prevent this, packages are only
      <quote>physically</quote> deleted by running the Nix garbage
      collector, which searches for all packages in the Nix store that
      are no longer <quote>reachable</quote> from outside the store.
      Thus, uninstalling a package is always safe: it cannot break
      other packages.
    </para>

    <para>
      Upgrading packages is easy.  Given a Nix expression that
      contains newer versions of installed packages (that is, packages
      with the same package name, but a higher version number),
      <command>nix-env -u</command> will replace the installed package
      in the user environment with the newer package.  For example,
      
      <screen>
$ nix-env -uf pkgs/system/i686-linux.nix pan</screen>

      looks for a newer version of Pan, and installs it if found.
      Also useful is the ability to upgrade <emphasis>all</emphasis>
      packages:
      
      <screen>
$ nix-env -uf pkgs/system/i686-linux.nix '*'</screen>

      The asterisk matches all installed packages<footnote><para>No,
      we don't support arbitrary regular
      expressions</para></footnote>.  Note that <literal>*</literal>
      must be quoted to prevent shell globbing.
    </para>

  </sect1>


  
  <!--######################################################################-->

  <sect1>
    <title>Writing Nix expressions</title>

    <sect2>
      <title>A simple Nix expression</title>

      <para>
        This section shows how to write simple Nix expressions&mdash;the things
        that describe how to build a package.
      </para>

      <example id='ex-hello-nix'>
        <title>Nix expression for GNU Hello</title>
        <programlisting>
{stdenv, fetchurl, perl}: <co id='ex-hello-nix-co-1' />

derivation { <co id='ex-hello-nix-co-2' />
  name = "hello-2.1.1"; <co id='ex-hello-nix-co-3' />
  system = stdenv.system; <co id='ex-hello-nix-co-4' />
  builder = ./builder.sh; <co id='ex-hello-nix-co-5' />
  src = fetchurl { <co id='ex-hello-nix-co-6' />
    url = ftp://ftp.nluug.nl/pub/gnu/hello/hello-2.1.1.tar.gz;
    md5 = "70c9ccf9fac07f762c24f2df2290784d";
  };
  stdenv = stdenv; <co id='ex-hello-nix-co-7' />
  perl = perl;
}</programlisting>
      </example>

      <para>
        A simple Nix expression is shown in <xref linkend='ex-hello-nix' />. It
        describes how to the build the <ulink
          url='http://www.gnu.org/directory/GNU/hello.html'>GNU Hello
          package</ulink>.  This package has several dependencies.  First, it
        requires a number of other packages, such as a C compiler, standard
        Unix shell tools, and Perl.  Rather than have this Nix expression refer
        to and use specific versions of these packages, it should be generic;
        that is, it should be a <emphasis>function</emphasis> that takes the
        required packages as inputs and yield a build of the GNU Hello package
        as a result.  This Nix expression defines a function with three
        arguments <xref linkend='ex-hello-nix-co-1' />, namely:
        <orderedlist>
          <listitem><para><varname>stdenv</varname>, which should be a
              <emphasis>standard environment package</emphasis>.  The standard
              environment is a set of tools and other components that would be
              expected in a fairly minimal Unix-like environment: a C compiler
              and linker, Unix shell tools, and so on.</para>
          </listitem>
          <listitem><para><varname>fetchurl</varname>, which should be a
              function that given parameters <varname>url</varname> and
              <varname>md5</varname>, will fetch a file from the specified
              location and check that this file has the given MD5 hash code.
              The hash is required because build operations must be
              <emphasis>pure</emphasis>: given the same inputs they should
              always yield the same output.  Since network resources can change
              at any time, we must in some way guarantee what the result will
              be.</para>
          </listitem>
          <listitem><para><varname>perl</varname>, which should be a Perl
              interpreter.</para>
          </listitem>
        </orderedlist>
      </para>

      <para>
        The remainder of the file is the body of the function, which happens to
        be a <emphasis>derivation</emphasis> <xref
        linkend='ex-hello-nix-co-2' />, which is the built-in function
        <varname>derivation</varname> applied to a set of attributes that
        encode all the necessary information for building the GNU Hello
        package. 
      </para>

      <example>
        <title>Build script (<filename>builder.sh</filename>) for GNU
          Hello</title>
        <programlisting>
#! /bin/sh

buildinputs="$perl"
. $stdenv/setup || exit 1

tar xvfz $src || exit 1
cd hello-* || exit 1
./configure --prefix=$out || exit 1
make || exit 1
make install || exit 1</programlisting>
      </example>

    </sect2>

    <sect2>
      <title>A more complex Nix expression</title>

      <example id='ex-svn-nix'>
        <title>Nix expression for Subversion</title>
        <programlisting>
{ localServer ? false <co id='ex-svn-nix-co-1' />
, httpServer ? false
, sslSupport ? false
, swigBindings ? false
, stdenv, fetchurl
, openssl ? null, httpd ? null, db4 ? null, expat, swig ? null
}:

assert !isNull expat; <co id='ex-svn-nix-co-2' />
assert localServer -> !isNull db4;
assert httpServer -> !isNull httpd &amp;&amp; httpd.expat == expat; <co id='ex-svn-nix-co-3' />
assert sslSupport -> !isNull openssl &amp;&amp; (httpServer -> httpd.openssl == openssl);
assert swigBindings -> !isNull swig;

derivation {
  name = "subversion-0.32.1";
  system = stdenv.system;

  builder = ./builder.sh;
  src = fetchurl {
    url = http://svn.collab.net/tarballs/subversion-0.32.1.tar.gz;
    md5 = "b06717a8ef50db4b5c4d380af00bd901";
  };

  localServer = localServer;
  httpServer = httpServer;
  sslSupport = sslSupport;
  swigBindings = swigBindings;

  stdenv = stdenv;
  openssl = if sslSupport then openssl else null; <co id='ex-svn-nix-co-4' />
  httpd = if httpServer then httpd else null;
  expat = expat;
  db4 = if localServer then db4 else null;
  swig = if swigBindings then swig else null;
}</programlisting>
      </example>

      <para>
        This example shows several features.  Default parameters <xref
          linkend='ex-svn-nix-co-1'/> can be used to simplify call sites: if an
        argument that has a default is omitted, its default value is used.
      </para>

      <para>
        You can use <emphasis>assertions</emphasis> to test whether arguments
        satisfy certain constraints.  The simple assertion <xref
          linkend='ex-svn-nix-co-2'/> tests whether the
        <varname>expat</varname> argument is not a null value.  The more
        complex assertion <xref linkend='ex-svn-nix-co-3'/> says that if
        Subversion is built with Apache support, then <varname>httpd</varname>
        (the Apache package) must not be null and it must have been built using
        the same instance of the <varname>expat</varname> library as was passed
        to the Subversion expression.  This is since the Subversion code is
        dynamically linked against the Apache code and they both use Expat,
        they must be linked against the same instance&mdash;otherwise a
        conflict might occur. 
      </para>

    </sect2>

  </sect1>


</chapter>