*********************
Apptainer/Singularity
*********************

Apptainer_ (formerly called Singularity) is a tool to run applications in
isolated images (also often called "containers"), similar to the more widely
known Docker.
We use it to provide you with a well-defined environment that contains the
proper versions of all dependencies, independent of what you have installed
locally on your system.  This ensures that the applications will run in the same
way on different machines, e.g. both when running on the real robots or in
simulation on your own computer.

The image we provide contains all dependencies to get started with the robot.
If you want to use additional libraries in your own code, you can extend the
image accordingly.


Apptainer vs Singularity vs SingularityCE
=========================================

There may be some confusion about different versions of Apptainer/Singularity
that are around.

Originally there was only "Singularity".  Recently Sylabs (a company build
around Singularity) forked from the main project, resulting in two projects,
both called "Singularity" (and both mostly compatible but not exactly the same).
Some time later the main project was renamed to "Apptainer" to resolve this
confusion.  This now leaves us in the situation that there is

- Singularity - The original project
- Singularity (or SingularityCE) - The version maintained by Sylabs
- Apptainer - basically Singularity 3.9 onwards.

At this point in time, they are still all mostly compatible with each other,
assuming you are using a reasonably recent version.  So most likely any of them
will work (simply replace ``apptainer`` with ``singularity`` in the commands
below, when using Singularity).  However, officially we only support Apptainer
1.0 (this is what we use on our side) so you may use any other version on you
own risk.


.. _singularity_install:

Install Apptainer
=================

Note that Apptainer only runs on Linux.  It may work with a virtual machine
but it is recommended to use a computer with a native Linux installation (we are
testing with Ubuntu but other distributions should also be fine).

RPM and DEB packages can be downloaded for each release from the `GitHub
repository <https://github.com/apptainer/apptainer/releases/>`_.

For example on Ubuntu, you can download the DEB package and install it with the
following command::

    $ sudo apt install ./apptainer_X.X.X_amd64.deb


You can also manually build it from source, for this see the `official
installation instructions`_.


.. _singularity_download_image:

Get the Image
=============

You can directly pull the image with Apptainer, using the following command::

    $ apptainer pull oras://ghcr.io/open-dynamic-robot-initiative/trifinger_singularity/trifinger_user:rrc2022

The image uses Ubuntu 20.04 and has all dependencies of the robot software and
the simulation already installed.  For most of the libraries we use the default
version provided through the official Ubuntu repositories.  To explore the
image and to check which libraries and which versions exactly are installed,
you can open a shell inside the container, see :ref:`singularity_shell`.

.. note::

   When you pull the image with the command above, the resulting file will be
   named ``trifinger_user_rrc2022.sif``.  In the following, we will only use
   ``rrc2022.sif`` to keep the commands shorter.  You can either rename the file
   or adjust the commands accordingly.


Using the Apptainer Image
=========================

This is a brief introduction to the basic usage of the Apptainer image.  For
more information, see the `official documentation`_.

.. important::

    By default, Apptainer is not fully isolated.  For example the whole home
    directly is bound into the container at run time.  This can impair
    reproducibility.  See :ref:`singularity_isolation` on how to make it more
    isolated.


.. _singularity_run:

Run a Command using the Container
---------------------------------

You can run commands inside the Apptainer image.  They will have access to
everything installed in the image and will be more or less isolated from your
host system (see :ref:`singularity_isolation`).

::

    $ apptainer run rrc2022.sif python3 path/to/some/script.py


To give a more concrete example, the following runs the simulation with a demo
script, moving the robot to random positions (the ``--nv`` flag is only needed
if you are using Nvidia drivers, otherwise you have to remove it, see
:ref:`singularity_nv`)::

    $ apptainer run --nv rrc2022.sif ros2 run trifinger_simulation demo_trifinger_platform.py


.. note::

    If the image file is marked as executable, you can also drop ``apptainer
    run`` and directly execute the file.  However, this way you cannot pass
    additional arguments like ``--nv`` to Apptainer.


.. _singularity_shell:

Open a Shell in the Container
-----------------------------

With the command above, you can only run a single command inside the container
environment.  If you want to run multiple commands in a row, preserving the
environment, you can open a shell in the container::

    $ apptainer shell path/to/rrc2022.sif

Once inside the container, run the following command::

    Apptainer> source /setup.bash

This is needed set up the environment, so the TriFinger-related software
packages are found.  Now you can again run commands like the demo script (you
may need to pass ``--nv`` to ``apptainer shell``, see :ref:`singularity_nv`)::

    Apptainer> ros2 run trifinger_simulation demo_trifinger_platform.py


.. _singularity_isolation:

Ensure Isolation from the Host System
-------------------------------------

By default Apptainer is not completely isolated from the host system.  If run
without arguments, it automatically binds the full home directory of the user,
the ``/tmp`` directory and a few system directories into the image.

While this is usually very convenient, it can cause some trouble.  For example,
Python may find packages that are installed in your home which may be
conflicting with versions installed in the container.  Further it can impair
reproducability, as your code may unintentionally use some package from your
home directory which are not available when executed on a different machine.  To
avoid these problems, you can run Apptainer with the following command, to be
more isolated::

    $ export APPTAINERENV_DISPLAY=$DISPLAY
    $ apptainer shell --cleanenv --no-home -B path/to/workspace path/to/rrc2022.sif

- ``--cleanenv``: Do not export all environment variables from the host into the
  image.  Since the environment variable ``DISPLAY`` needs to be set in order to
  run graphical applications, it is still exported into the image by setting
  ``APPTAINERENV_DISPLAY`` in the first line.
- ``--no-home``: Do not bind the full home directory.  Other directories like
  ``/tmp`` are still bound.  You may use ``--contain`` to exclude those as well.
- ``-B path/to/workspace``: Explicitly bind your workspace, so it is accessible
  in the container.  You can list multiple directories separated by commas.

For more information, please refer to the `official documentation`_ of
Apptainer.


.. _singularity_nv:

Running GUI-Applications in Apptainer
---------------------------------------

When running applications that use GPU-based rendering (e.g. the PyBullet
visualisation) on a computer with Nvidia drivers, you may need to add the `--nv`
flag when running Apptainer.

::

    $ apptainer run --nv rrc2022.sif <command>

or

::

    $ apptainer shell --nv rrc2022.sif

See the `Apptainer documentation on GPU Support`_ for more information.


.. _singularity_extend_container:
.. _singularity_custom_image:

Add Custom Dependencies to the Container
========================================

The image we provide already includes everything needed to run the robot and the
simulation.  However, you may need additional libraries to use them in our own
code, which are not yet present.  In this case, you can create your own image
which is based on our standard image but extends it with your additional
dependencies.

.. note::

    **You don't need to create a custom container for adding common Python
    dependencies.**  Dependencies that can be installed with pip can simply be
    listed as requirements of your own Python package (e.g. in the ``setup.cfg``
    if you are using our `example package`_).

    You also don't need to add your own package for the challenge to the image
    as this will be provided separately.  So you only need to create a custom
    image if you want to install dependencies that are not yet available in the
    default image and cannot be installed with pip.


Create the Custom Image
-----------------------

To extend the image, create *definition file* like the following:

::

    # Use the rrc2022 image as base
    Bootstrap: oras
    From: ghcr.io/open-dynamic-robot-initiative/trifinger_singularity/trifinger_user:rrc2022

    # alternatively to the above, you can specify the path to a local image:
    # Bootstrap: localimage
    # From: ./rrc2022.sif

    %post
        # Put commands to install additional dependencies here.
        # Make sure everything runs automatically without human input (e.g. add
        # `-y` to automatically say "yes" below).
        apt-get install -y package_name

See the official `Documentation for Definition Files`_ for all options in the
definition file.

Assuming you called your definition file ``user_image.def``, use the following
command to build the image::

    $ apptainer build --fakeroot user_image.sif path/to/user_image.def


.. warning::

   To ensure that your custom image is compatible with our setup for executing
   the code, always use the official image as base and avoid overwriting
   existing libraries or applications with different versions.


Upload the Custom Image
-----------------------

To use your custom image on the real robots, you need to :ref:`upload it to the
submission system and update your configuration
<config_custom_singularity_image>`.



.. _Apptainer: https://apptainer.org
.. _official documentation: https://apptainer.org/docs/user/1.0/
.. _Documentation for Definition Files: https://apptainer.org/docs/user/1.0/definition_files.html
.. _official installation instructions: https://apptainer.org/docs/user/1.0/quick_start.html#quick-installation-steps
.. _Apptainer documentation on GPU Support: https://apptainer.org/docs/user/1.0/gpu.html