Docker Symlink Wrapper: Difference between revisions

From Wiki-DB
Jump to navigationJump to search
Created page with "== Overview == <tt>docker-symlink-wrapper.py</tt> is a wrapper script which creates symlinks for applications. If the symlink is called the script starts the corresoponding a..."
 
Added Debian package information
 
(5 intermediate revisions by the same user not shown)
Line 1: Line 1:
__TOC__
== Overview ==
== Overview ==


<tt>docker-symlink-wrapper.py</tt> is a wrapper script which creates symlinks for applications.
<tt>docker-symlink-wrapper</tt> is a wrapper script which creates symlinks for applications.
If the symlink is called the script starts the corresoponding application in a container and passes all parameters to it.
If the symlink is called the script starts the corresoponding application in a container and passes all parameters to it.
The script can manage multiple symlinks which can point to different container images.
The script can manage multiple symlinks which can point to different container images.
It is also possible to set the tag of the docker image which is used (= version of the application).
It is also possible to set the tag of the docker image which is used (= version of the application).


<tt>docker-symlink-wrapper.py</tt> requires the python modules <tt>requests</tt> and <tt>docker</tt>. The versions supplied by the Debian package repository is, at least in stretch, too old (especially the docker module), so install the recent version via pip:
<tt>docker-symlink-wrapper</tt> is installable over a Debian package.
<tt>pip3 install -r requirements.txt</tt> with using the requirements file supplied at this repository or <tt>pip3 install requests docker</tt> for directly installing the requirements.
To install the package call "<tt>sudo dpkg -i</tt>" with the name of the package file.
If the dependencies are not installed call "<tt>sudo apt -f install</tt>" to install the dependencies.
'''Note''': On Debian Stretch, Docker isn't included in the official package repository, you have to install docker with Docker's own repository.
Here is the link to the installation instructions of Docker: https://docs.docker.com/engine/install/debian/.


The wrapper script uses a configuration file for storing the known symlink and their configration.
The wrapper script uses a configuration file for storing the known symlinks and their configration.
This file is located at <tt>/var/lib/docker-symlink-wrapper/symlinks.json</tt>.
This file is located at <tt>/var/lib/docker-symlink-wrapper/symlinks.json</tt>.
This concludes that you need to superuser privileges if you want to call a command of the script which changes the configuration.
This concludes that you need to superuser privileges if you want to call a command of the script which changes the configuration.


For authentification at docker registries the saved credentials of the Docker CLI are used (usually at <tt>~/.docker/config.json)</tt>.
For authentification at docker registries the saved credentials of the Docker CLI are used (usually at <tt>~/.docker/config.json)</tt>.
So use docker login <tt><registry-address></tt> to login into the registry before using the script.
So use <tt>docker login <registry-address></tt> to login into the registry before using the script.


== Available Commands ==
== Available Commands ==


* <tt>docker-symlink-wrapper.py install <name> <json file>|-</tt> for installing new symlinks to a container and adding them to the configuration file of the script. '<tt>name</tt>' is an unique keyword for the entry in the configuration file (so that the entry can be recognized later), while '<tt>json file</tt>' is a file in the JSON format which has the configuration of the container (Definition of the fields of the JSON file is below).
{| class="wikitable"
| style="width: 440px"| <tt>docker-symlink-wrapper install <name> <json file> <nowiki>|</nowiki> -</tt>  
| Installs new symlinks to a container and adds them to the configuration file of the script. '<tt>name</tt>' is an unique keyword for the entry in the configuration file (so that the entry can be recognized later), while '<tt>json file</tt>' is a file in the JSON format which has the configuration of the container (Definition of the fields of the JSON file is below). Instead of using a JSON file as a parameter you can substitue that with an '<tt>-</tt>', then the script reads the JSON from STDIN.
|-
| <tt>docker-symlink-wrapper uninstall <name></tt>
| Removes the symlinks from a container and from the configuration file of the script. '<tt>name</tt>' is an unique keyword of the entry in the configuration file.
|-
| <tt>docker-symlink-wrapper list</tt>
| Lists all known symlinks, its keyword and the corresponding container.
|-
| <tt>docker-symlink-wrapper list-versions <name></tt>
| Lists all available versions of a container (= tag) for this '<tt>name</tt>'. This command does connect to the corresponding container registry and reads the available tags. This command does NOT work with images which are only local! For using this command with registies which require the addition of certificates (like corporate internal registries) you have to add the certificate (or its root certificate) to the system certificates (usually at /etc/ssl/certs/ca-certificates.crt).
|-
| <tt>docker-symlink-wrapper set-version <name> <tag></tt>
| Sets the used version of the container for the '<tt>name</tt>' to the '<tt>tag</tt>',
|-
| <tt><symlinkname> [...]</tt>
| When the script is called trough one of the symlinks, it looks at the configuration to determine the correct container to start. After that it starts that container and calls inside the container the applictaion which has the same name as the symlink and passes all parameter to it. If something is piped into STDIN of the symlink, the docker-symlink-wrapper script passes this to the STDIN of the program inside the container. The return value of the container is captured and returned.
|}
== JSON-format for adding new symlinks ==
 
When adding new symlinks, JSON must be supplied which contains all neccessary information.
It must consists of one object with the following fields:
 
* '''symlinks''': List of strings which are the application names that should be calles inside the container.
 
* '''registry''': A string that is the address of the container registry. If this field is empty, Docker's own registry Docker Hub is used as a default. This field is needed, because the is no way to definitly distinguish if the first segment of the image name is the address of the registry or that it is the first part of the container repository name.
 
* '''image''': A string that contains the name of the docker image which is used for the applications. The image name usually contains the address of the registry. If the registry is omitted, the image is either local or uses Docker Hub as a registry.


* <tt>docker-symlink-wrapper.py uninstall <name></tt> for removing the symlinks from a container and removing them from the configuration of the script. '<tt>name</tt>' is an unique keyword of the entry in the configuration file.
* '''tag''': A string that is used as default tag of the container image.


* <tt>docker-symlink-wrapper.py list</tt> for listing all known symlinks, its keyword and the corresponding container.
* '''installpath''': A string that defines the path in the system where the symlinks are installed.


* <tt>docker-symlink-wrapper.py list-versions <name></tt> lists all available versions of a container (= tag) for this '<tt>name</tt>'. This command does connect to the corresponding container registry and reads the available tags. This command does NOT work with images which are only local! For using this command with registies which require the addition of certificates (like corporate internal registries) you have to add the certificate (or its root certificate) to the system certificates (usually at /etc/ssl/certs/ca-certificates.crt).
Here an example for a JSON file which can be used to add new symlinks:
<nowiki>{
    "symlinks":[
        "arm-linux-gnueabihf-as",
        "arm-linux-gnueabihf-ld",
        "arm-linux-gnueabihf-gcc",
        "arm-linux-gnueabihf-g++",
        "arm-linux-gnueabihf-ar",
        "arm-linux-gnueabihf-nm",
        "arm-linux-gnueabihf-strip",
        "arm-linux-gnueabihf-objcopy",
        "arm-linux-gnueabihf-objdump",
        "make-cross",
        "cmake-cross"
    ],
    "registry":"",
    "image":"dhelectronics/linaro-cross-build",
    "tag":"buster-arm32v7-8",
    "installpath":"/usr/local/bin"
}</nowiki>
When adding the configuration of the symlink to the configfile of the wrapper script, the JSON object is prepended by the 'name' (the other param of the install command) and added to the main object of the configuration.


* <tt>docker-symlink-wrapper.py set-version <name> <tag></tt> sets the used version of the container for the '<tt>name</tt>' to the '<tt>tag</tt>',
==Limitations==
There are some limitations of the script:


* <tt><symlinkname> [...]</tt> When the script is called trough one of the symlinks, it looks at the configuration to determine the correct container to start. After that it starts that container and calls inside the container the applictaion which has the same name as the symlink and passes all parameter to it.
* Only containers which do not need any special adjustments (like additional capabilities) can be started.
* The container image is only pulled from the registry if the image isn't already in the image storage of the local docker instance. This can result that you work with an older version of a container, even if there is a new version available. (Docker version 20.03 (not released yet as of 10/2020) will introduce a pull-policy flag at docker run, maybe the Python library will get this too, then this can be fixed in future versions of the wrapper)
* If a symlink is called, only the current working directory is mounted into the container. If the program needs to access files outside of the current working directory, you have to properly adjust the current working directory before calling the symlink.
* When you use a script which calls a symlinked program in a huge number of times, the performance is reduced. This is because at each call of the symlinked program, a new container is created and after the execution of the program has finished, the container is destroyed (creation and destruction of containers are a ressource-heavy operation).
* The <tt>list-versions</tt>-command won't work with an image which only exists localy. This is because the command uses the API of the corresponding docker registry to get all available versions.
* If you make symlinks of software which is also installed on the host system, it depends on the <tt>PATH</tt>-varaiable which of them is used (when calling the command without the absolute path). It may happen that you use accidently the wrong version of the software without noticing.

Latest revision as of 15:36, 12 October 2020

Overview

docker-symlink-wrapper is a wrapper script which creates symlinks for applications. If the symlink is called the script starts the corresoponding application in a container and passes all parameters to it. The script can manage multiple symlinks which can point to different container images. It is also possible to set the tag of the docker image which is used (= version of the application).

docker-symlink-wrapper is installable over a Debian package. To install the package call "sudo dpkg -i" with the name of the package file. If the dependencies are not installed call "sudo apt -f install" to install the dependencies. Note: On Debian Stretch, Docker isn't included in the official package repository, you have to install docker with Docker's own repository. Here is the link to the installation instructions of Docker: https://docs.docker.com/engine/install/debian/.

The wrapper script uses a configuration file for storing the known symlinks and their configration. This file is located at /var/lib/docker-symlink-wrapper/symlinks.json. This concludes that you need to superuser privileges if you want to call a command of the script which changes the configuration.

For authentification at docker registries the saved credentials of the Docker CLI are used (usually at ~/.docker/config.json). So use docker login <registry-address> to login into the registry before using the script.

Available Commands

docker-symlink-wrapper install <name> <json file> | - Installs new symlinks to a container and adds them to the configuration file of the script. 'name' is an unique keyword for the entry in the configuration file (so that the entry can be recognized later), while 'json file' is a file in the JSON format which has the configuration of the container (Definition of the fields of the JSON file is below). Instead of using a JSON file as a parameter you can substitue that with an '-', then the script reads the JSON from STDIN.
docker-symlink-wrapper uninstall <name> Removes the symlinks from a container and from the configuration file of the script. 'name' is an unique keyword of the entry in the configuration file.
docker-symlink-wrapper list Lists all known symlinks, its keyword and the corresponding container.
docker-symlink-wrapper list-versions <name> Lists all available versions of a container (= tag) for this 'name'. This command does connect to the corresponding container registry and reads the available tags. This command does NOT work with images which are only local! For using this command with registies which require the addition of certificates (like corporate internal registries) you have to add the certificate (or its root certificate) to the system certificates (usually at /etc/ssl/certs/ca-certificates.crt).
docker-symlink-wrapper set-version <name> <tag> Sets the used version of the container for the 'name' to the 'tag',
<symlinkname> [...] When the script is called trough one of the symlinks, it looks at the configuration to determine the correct container to start. After that it starts that container and calls inside the container the applictaion which has the same name as the symlink and passes all parameter to it. If something is piped into STDIN of the symlink, the docker-symlink-wrapper script passes this to the STDIN of the program inside the container. The return value of the container is captured and returned.

JSON-format for adding new symlinks

When adding new symlinks, JSON must be supplied which contains all neccessary information. It must consists of one object with the following fields:

  • symlinks: List of strings which are the application names that should be calles inside the container.
  • registry: A string that is the address of the container registry. If this field is empty, Docker's own registry Docker Hub is used as a default. This field is needed, because the is no way to definitly distinguish if the first segment of the image name is the address of the registry or that it is the first part of the container repository name.
  • image: A string that contains the name of the docker image which is used for the applications. The image name usually contains the address of the registry. If the registry is omitted, the image is either local or uses Docker Hub as a registry.
  • tag: A string that is used as default tag of the container image.
  • installpath: A string that defines the path in the system where the symlinks are installed.

Here an example for a JSON file which can be used to add new symlinks:

{
    "symlinks":[
        "arm-linux-gnueabihf-as",
        "arm-linux-gnueabihf-ld",
        "arm-linux-gnueabihf-gcc",
        "arm-linux-gnueabihf-g++",
        "arm-linux-gnueabihf-ar",
        "arm-linux-gnueabihf-nm",
        "arm-linux-gnueabihf-strip",
        "arm-linux-gnueabihf-objcopy",
        "arm-linux-gnueabihf-objdump",
        "make-cross",
        "cmake-cross"
    ],
    "registry":"",
    "image":"dhelectronics/linaro-cross-build",
    "tag":"buster-arm32v7-8",
    "installpath":"/usr/local/bin"
}

When adding the configuration of the symlink to the configfile of the wrapper script, the JSON object is prepended by the 'name' (the other param of the install command) and added to the main object of the configuration.

Limitations

There are some limitations of the script:

  • Only containers which do not need any special adjustments (like additional capabilities) can be started.
  • The container image is only pulled from the registry if the image isn't already in the image storage of the local docker instance. This can result that you work with an older version of a container, even if there is a new version available. (Docker version 20.03 (not released yet as of 10/2020) will introduce a pull-policy flag at docker run, maybe the Python library will get this too, then this can be fixed in future versions of the wrapper)
  • If a symlink is called, only the current working directory is mounted into the container. If the program needs to access files outside of the current working directory, you have to properly adjust the current working directory before calling the symlink.
  • When you use a script which calls a symlinked program in a huge number of times, the performance is reduced. This is because at each call of the symlinked program, a new container is created and after the execution of the program has finished, the container is destroyed (creation and destruction of containers are a ressource-heavy operation).
  • The list-versions-command won't work with an image which only exists localy. This is because the command uses the API of the corresponding docker registry to get all available versions.
  • If you make symlinks of software which is also installed on the host system, it depends on the PATH-varaiable which of them is used (when calling the command without the absolute path). It may happen that you use accidently the wrong version of the software without noticing.