ELBE Overview: Difference between revisions
Pzimmermann (talk | contribs) Prepare for ELBE docker container |
Pzimmermann (talk | contribs) Some corrections and inclusion of the docker parts for review |
||
Line 21: | Line 21: | ||
As an addition ELBE is able to create a cdrom image which can be used as a input to reconstruct the exact root-filesystem that was created. | As an addition ELBE is able to create a cdrom image which can be used as a input to reconstruct the exact root-filesystem that was created. | ||
ELBE can also create a Software Development Kit (SDK) for the built root-filesystem. The SDK contains the toolchain with the cross compiler and other tools and the corresponding sysroot with needed headers. These can be used to create software for the specific root-filesystem. The SDK is very similar to Yocto's Standard-SDK and can be used in the same way. An explanation how to use the SDK with some IDEs/buildsystems can be found here: | ELBE can also create a Software Development Kit (SDK) for the built root-filesystem. The SDK contains the toolchain with the cross compiler and other tools and the corresponding sysroot with needed headers. These can be used to create software for the specific root-filesystem. The SDK is very similar to Yocto's Standard-SDK and can be used in the same way. An explanation how to use the SDK with some IDEs/buildsystems can be found here: [[Using Yocto/ELBE SDK for application development]] | ||
==Installation== | ==Installation== | ||
Line 44: | Line 44: | ||
===Cloning the git repository of ELBE=== | ===Cloning the git repository of ELBE=== | ||
There is also the possibility to clone the git repository of ELBE. | There is also the possibility to clone the git repository of ELBE. | ||
'''Up to version 12.5''' the following packages have to be installed on the system: | |||
<tt>apt install python python-debian python-mako python-lxml python-apt python-gpg python-suds python-libvirt qemu-utils qemu-kvm p7zip-full make libvirt-daemon libvirt-daemon-system libvirt-clients</tt> | <tt>apt install python python-debian python-mako python-lxml python-apt python-gpg python-suds python-libvirt qemu-utils qemu-kvm p7zip-full make libvirt-daemon libvirt-daemon-system libvirt-clients</tt> | ||
'''Starting with version 13.0''' ELBE did switch to version 3 of Python (due to the EOL of Python 2), so the following packages have to be installed: | |||
<tt>apt install python3 python3-debian python3-mako python3-lxml python3-apt python3-gpg python3-suds python3-libvirt qemu-utils qemu-kvm p7zip-full make libvirt-daemon libvirt-daemon-system libvirt-clients</tt> | |||
Then you can clone the repo with: | Then you can clone the repo with: | ||
Line 53: | Line 58: | ||
<tt>adduser <user> libvirt && adduser <user> libvirt-qemu</tt> | <tt>adduser <user> libvirt && adduser <user> libvirt-qemu</tt> | ||
===Premade Docker container=== | ===Premade Docker container=== | ||
Line 70: | Line 74: | ||
<tt>docker pull dhelectronics/elbe:git-v12.4</tt> | <tt>docker pull dhelectronics/elbe:git-v12.4</tt> | ||
The container can be started with (the container does need extensive rights because a virtual machine is started inside it): | The container can be started with the following command: (the container does need extensive rights because a virtual machine is started inside it): | ||
<tt>docker run --name=elbe --mount type=bind,src=./,dst=/workdir-w /workdir/ --cap-add SYS_ADMIN --security-opt seccomp:unconfined --security-opt apparmor:unconfined --mount type=bind,src=/sys/fs/cgroup,dst=/sys/fs/cgroup --device /dev/kvm --device /dev/fuse --group-add kvm dhelectronics/elbe:git-v12.4</tt> | <tt>docker run --name=elbe --mount type=bind,src=./,dst=/workdir-w /workdir/ --cap-add SYS_ADMIN --security-opt seccomp:unconfined --security-opt apparmor:unconfined --mount type=bind,src=/sys/fs/cgroup,dst=/sys/fs/cgroup --device /dev/kvm --device /dev/fuse --group-add kvm dhelectronics/elbe:git-v12.4</tt> | ||
Line 79: | Line 83: | ||
Inside the container you can now run your ELBE build, you can exit the shell with CTRL+D and remove the container with <tt>docker container rm -f elbe</tt>. | Inside the container you can now run your ELBE build, you can exit the shell with CTRL+D and remove the container with <tt>docker container rm -f elbe</tt>. | ||
==Usage== | ==Usage== | ||
Line 93: | Line 97: | ||
If an error happens while building the rootfilesystem, no files are returned. It is possible to get the log and other files of the build with an extra command. The complete call of the command (which differs for each build) is printed as exit-message of the ELBE build. | If an error happens while building the rootfilesystem, no files are returned. It is possible to get the log and other files of the build with an extra command. The complete call of the command (which differs for each build) is printed as exit-message of the ELBE build. | ||
===Script for easy use with container images=== | ===Script for easy use with container images=== | ||
We created a script for easing the use of ELBE inside the container. This script is preinstalled in our [[Virtual_Machine_for_Application_Development|Virtual Machine for Application Development]] starting with version | We created a script for easing the use of ELBE inside the container. This script is preinstalled in our [[Virtual_Machine_for_Application_Development|Virtual Machine for Application Development]] starting with version '''Not yet available'''. The script is called '''elbe_container_build_start.sh [OPTIONS] <elbe_build_description.xml>'''. In the standard configuration the CD-images with binary and source packages are created. The SDK is also created. | ||
It is possible to start an additional container with a debian repository, the given path at --repodir has to point at the root of a debian repository. | It is possible to start an additional container with a debian repository, the given path at --repodir has to point at the root of a debian repository. The directory is mounted into the directory <tt><nowiki>/repo/dh_debian_repo</nowiki></tt> of the container. Any container which is configured to publish the mounted directory can be used. Inside the xml the repository can be used by adding the repo to the list of mirrors. Note that after the build process the repository is still inside the <tt>/etc/apt/sources.list</tt> of the resulting rootfilesystem. You have to remove this line at the finetuning section of the xml file or when the device running with the rootfilesystem has started. If you do not remove the repo, <tt>apt-get update</tt> will fail with an error if you want to update the system or install new packages afterwards. | ||
The following options are available for the script: | The following options are available for the script: | ||
Line 114: | Line 118: | ||
| --fast || Speed up the build of the rootfilesystem with only building the rootfilesystem itself, and skipping the creation of the CD-images and of the SDK. | | --fast || Speed up the build of the rootfilesystem with only building the rootfilesystem itself, and skipping the creation of the CD-images and of the SDK. | ||
|- | |- | ||
| --console || Ignore every other option and start the container and enter a console inside of the container. | | --console || Ignore every other option and start the container and enter a console inside of the container. The current working directory is mounted into the container | ||
|- | |- | ||
| --version || Display version info of the script and exit | | --version || Display version info of the script and exit | ||
Line 120: | Line 124: | ||
| --help || Display the command line help and exit | | --help || Display the command line help and exit | ||
|} | |} | ||
It is not possible to append a XML file with an archive via an option of the script. If you need to do this you have to use the "<tt>--console</tt>"-option to open a console inside the container where you can use the "<tt>elbe chg_archive</tt>"-command. | |||
When the build is finished, the container exits and is removed. If an error happenes the container is not exited, and a console inside the container is opened, where you can get the logs and files of the failed build. | When the build is finished, the container exits and is removed. If an error happenes the container is not exited, and a console inside the container is opened, where you can get the logs and files of the failed build. | ||
==XML-Example== | ==XML-Example== | ||
Latest revision as of 09:57, 5 October 2020
This site is being reworked at the moment!
ELBE (Embedded Linux Build Environment) is a buildsystem to generate root-filesystems for embedded devices. This is only a short explanation what ELBE is and a brief tutorial for using ELBE, please go to the official documentation of ELBE for a detailed documentation. This article was made when version 12.4 was the current version of ELBE.
Overview
ELBE uses a virtual machine to create root-filesystems, this VM is called "initvm". At the first time of running ELBE the initvm has to be initialized (a complete automatic installation of Debian is made), subsequent runs of ELBE can reuse that initvm.
Recepies what a built root-filesystem should contain are supplied in form of a xml-file. ELBE doesn't build the software for the root-filesystem, it does instead use pre-built software in form of Debian packages (.deb). Usually the official Debian repositories are used, but it is possible to use custom repositories to get special packages into the root-filesystem. Resulting root-filesystems are basicly customized Debian installations, which can be maintained through Debian's tools like APT. This is the biggest difference of ELBE to other buildsystems like Yocto or Buildroot.
It is possible to define "finetuning"-commands in the xml-file, which are executed when the creation of the root-filesystem is completed. With this it is possible to remove files, add additional users and much more ...
Xml-files can be appended with a bzip2 compressed archive. At the end of the buildprocess, the content of the archive will be extracted into the root-filesystem.
The form of the finished root-filesystem is customizeable. The following variants are possible:
- Tarball: The root-filesystem can be packed as a tarball (usually compressed with gzip). The tarball can be extracted onto an empty partition or can be used for the DHCOM Update Mechanism.
- Disk-/SD-card-image: An image with full partition table can be created. This can be a image of a hard disk or a sd-card. The size, type, label and mountpoint of each partition can be customized via the xml-file.
- UBIFS-Image: ELBE can output images for "Unsorted Block Image"-devices (UBI). These are needed if you want to use the NAND-Flash of our DHSOM modules for the root-filesystem. More information about that type of filesystem can be found under UBI/UBIFS Filesystem.
As an addition ELBE is able to create a cdrom image which can be used as a input to reconstruct the exact root-filesystem that was created.
ELBE can also create a Software Development Kit (SDK) for the built root-filesystem. The SDK contains the toolchain with the cross compiler and other tools and the corresponding sysroot with needed headers. These can be used to create software for the specific root-filesystem. The SDK is very similar to Yocto's Standard-SDK and can be used in the same way. An explanation how to use the SDK with some IDEs/buildsystems can be found here: Using Yocto/ELBE SDK for application development
Installation
There are multiple ways to install ELBE:
Debian Package
The developers of ELBE have their own package repository which can be used to install ELBE.
To add the repository to the /etc/apt/sources.list.d directory and install the public key for the repo:
echo 'deb http://debian.linutronix.de/elbe buster main' >> /etc/apt/sources.list.d/elbe.list wget http://debian.linutronix.de/elbe/elbe-repo.pub && apt-key add elbe-repo.pub
Now you can install ELBE with:
apt update apt install elbe
Important: You have to add your user to the libvirt groups if you want to use ELBE without root:
adduser <user> libvirt && adduser <user> libvirt-qemu
Cloning the git repository of ELBE
There is also the possibility to clone the git repository of ELBE.
Up to version 12.5 the following packages have to be installed on the system:
apt install python python-debian python-mako python-lxml python-apt python-gpg python-suds python-libvirt qemu-utils qemu-kvm p7zip-full make libvirt-daemon libvirt-daemon-system libvirt-clients
Starting with version 13.0 ELBE did switch to version 3 of Python (due to the EOL of Python 2), so the following packages have to be installed:
apt install python3 python3-debian python3-mako python3-lxml python3-apt python3-gpg python3-suds python3-libvirt qemu-utils qemu-kvm p7zip-full make libvirt-daemon libvirt-daemon-system libvirt-clients
Then you can clone the repo with:
git clone https://github.com/Linutronix/elbe
Important: You have to add your user to the libvirt groups if you want to use ELBE without root:
adduser <user> libvirt && adduser <user> libvirt-qemu
Premade Docker container
We created docker container images with ELBE preinstalled, these can be found at Docker Hub under the name dhelectronics/elbe. There are two different versions of container tags available:
Tag | Version the tag is pointing to. |
---|---|
git-latest-tested | Points to the latest version we have used and confirmed that is working with our own recipies. This version will change from time to time and some versions may be skipped. |
git-vX | Always points to the specific version X of ELBE (as an example git-v12.4 points to version 12.4 of ELBE). |
To get an image enter (e.g. version 12.4)
docker pull dhelectronics/elbe:git-v12.4
The container can be started with the following command: (the container does need extensive rights because a virtual machine is started inside it):
docker run --name=elbe --mount type=bind,src=./,dst=/workdir-w /workdir/ --cap-add SYS_ADMIN --security-opt seccomp:unconfined --security-opt apparmor:unconfined --mount type=bind,src=/sys/fs/cgroup,dst=/sys/fs/cgroup --device /dev/kvm --device /dev/fuse --group-add kvm dhelectronics/elbe:git-v12.4
Now you have to start a shell inside the container with:
docker exec elbe /bin/bash
Inside the container you can now run your ELBE build, you can exit the shell with CTRL+D and remove the container with docker container rm -f elbe.
Usage
The ELBE buildsystem consists of the program elbe which is implemented in Python. Similar to git, it can be called with several sub commands. If you installed ELBE through the Debian package, the program is inside your PATH environment and can be called by "elbe", else you have to call it by the complete path. The most important commands are:
elbe initvm create --directory /example example.xml
This command initializes a new initvm in the directory "/example" and builds a new root-filesystem based on "example.xml" . This command can only be used if there isn't any initvm defined yet. If you cloned ELBE from git, you can add the switch "--devel", then ELBE copies the local ELBE version into the initvm (instead of downloading ELBE from their repository).
elbe initvm submit --directory /example example.xml
This command builds a new root-filesystem based on "example.xml" and reuses an existing initvm at the directory "/example".
elbe chg_archive example.xml example-archive.tar.bz2
This command appends the xml-file "example.xml" with the archive "example-archive.tar.bz2". The binary content of the archive is stored in BASE64 encoding inside a xml tag.
If you add --build-sdk at the "initvm create/submit"-command then as an addition to creating the root-filesystem, the SDK (for developing software for the root-filesystem) is created too. Usually CD-Images with all binary packages and all source packages (for the exact recreation of the rootfilesystem) are created. The creation of these can be skipped with the commandline switches --skip-build-bin and --skip-build-sources.
If an error happens while building the rootfilesystem, no files are returned. It is possible to get the log and other files of the build with an extra command. The complete call of the command (which differs for each build) is printed as exit-message of the ELBE build.
Script for easy use with container images
We created a script for easing the use of ELBE inside the container. This script is preinstalled in our Virtual Machine for Application Development starting with version Not yet available. The script is called elbe_container_build_start.sh [OPTIONS] <elbe_build_description.xml>. In the standard configuration the CD-images with binary and source packages are created. The SDK is also created.
It is possible to start an additional container with a debian repository, the given path at --repodir has to point at the root of a debian repository. The directory is mounted into the directory /repo/dh_debian_repo of the container. Any container which is configured to publish the mounted directory can be used. Inside the xml the repository can be used by adding the repo to the list of mirrors. Note that after the build process the repository is still inside the /etc/apt/sources.list of the resulting rootfilesystem. You have to remove this line at the finetuning section of the xml file or when the device running with the rootfilesystem has started. If you do not remove the repo, apt-get update will fail with an error if you want to update the system or install new packages afterwards.
The following options are available for the script:
--elbedockerimage <REPO:TAG> | Sets the docker image which is used for the elbe build. |
--repodir <path> | Starts an additional container with a Debian repo webserver, which uses this repo path. If this option isn't set, then no container with the repo is started. |
--repodockerimage <REPO:TAG> | Sets Debian repo webserver docker image. Has no effect if --repodir is not set. |
--initvmtopdir <path> | Change initvm top directory, default value is the <current-working-directory>/initvm-docker |
--fast | Speed up the build of the rootfilesystem with only building the rootfilesystem itself, and skipping the creation of the CD-images and of the SDK. |
--console | Ignore every other option and start the container and enter a console inside of the container. The current working directory is mounted into the container |
--version | Display version info of the script and exit |
--help | Display the command line help and exit |
It is not possible to append a XML file with an archive via an option of the script. If you need to do this you have to use the "--console"-option to open a console inside the container where you can use the "elbe chg_archive"-command.
When the build is finished, the container exits and is removed. If an error happenes the container is not exited, and a console inside the container is opened, where you can get the logs and files of the failed build.
XML-Example
This is a simple example for a xml file. If you start the build process a root-filesystem for a device of the "ARM EABI with hard float"-architecture (usually ARMv7) based on Debian 10 is built. In extra to the main mirror, the security and buster-updates mirror is also configured, so that the root-filesystem can get security updates. After the package installation is finished, the cached Debian package files are removed to slim down the image. At the end, the finished root-filesystem is packed into gzip-compressed tarball "rootfs.tar.gz". Note that the resulting root-filesystem has no kernel, bootloader and device tree, these have to be supplied externally.
<ns0:RootFileSystem xmlns:ns0="https://www.linutronix.de/projects/Elbe" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" created="2009-05-20T08:50:56" revision="4" xsi:schemaLocation="https://www.linutronix.de/projects/Elbe dbsfed.xsd"> <project> <name>armhf-example</name> <version>1.0</version> <buildtype>armhf</buildtype> <mirror> <primary_host>ftp.de.debian.org</primary_host> <primary_path>/debian</primary_path> <primary_proto>http</primary_proto> <url_list> <url> <source>http://security.debian.org/debian-security buster/updates main</source> <binary>http://security.debian.org/debian-security buster/updates main</binary> </url> <url> <source>http://ftp.de.debian.org/debian buster-updates main</source> <binary>http://ftp.de.debian.org/debian buster-updates main</binary> </url> </url_list> </mirror> <suite>buster</suite> </project> <target> <hostname>armhf-example</hostname> <passwd>foo</passwd> <console>ttyS0,115200</console> <package> <tar> <name>rootfs.tar.gz</name> </tar> </package> <finetuning> <rm>/var/cache/apt/archives/*.deb</rm> </finetuning> <pkg-list> <pkg>bash</pkg> <pkg>less</pkg> <pkg>git</pkg> <pkg>openssh-server</pkg> <pkg>nfs-common</pkg> <pkg>curl</pkg> <pkg>vim</pkg> <pkg>iperf3</pkg> <pkg>i2c-tools</pkg> </pkg-list> </target> </ns0:RootFileSystem>