ELBE Overview: Difference between revisions

From Wiki-DB
Jump to navigationJump to search
Lzenz (talk | contribs)
No edit summary
Some corrections and inclusion of the docker parts for review
 
(9 intermediate revisions by 2 users not shown)
Line 1: Line 1:
<div id="header">
<div id="header">
<big>'''This site is being reworked at the moment!'''</big>


The Embedded Linux Build Environment (elbe) is a Debian based System to generate root-filesystems for embedded devices.
ELBE (Embedded Linux Build Environment) is a buildsystem to generate root-filesystems for embedded devices. This is only a short explanation what ELBE is and a brief tutorial for using ELBE, '''please go to the [https://elbe-rfs.org/docs/sphinx/index.html official documentation of ELBE] for a detailed documentation'''. This article was made when version 12.4 was the current version of ELBE.


__TOC__
==Overview==
ELBE uses a virtual machine to create root-filesystems, this VM is called "initvm". At the first time of running ELBE the initvm has to be initialized (a complete automatic installation of Debian is made), subsequent runs of ELBE can reuse that initvm.
 
[[File:ELBE Build schema.png|frame|none|Basic structure of ELBE building a root-filesystem (as of version 12.4). Orange are inputs for ELBE, green are things that the result of ELBE. "<tt>buildenv.img</tt>" is a special case because the initvm is created at "elbe initvm create" and it is reused by "elbe initvm submit". Loosely based on both diagrams at https://elbe-rfs.org/docs/sphinx/releases_v12.4/article-elbeoverview-en.html#overview]]
 
Recepies what a built root-filesystem should contain are supplied in form of a xml-file. ELBE doesn't build the software for the root-filesystem, it does instead use pre-built software in form of Debian packages (.deb). Usually the official Debian repositories are used, but it is possible to use custom repositories to get special packages into the root-filesystem. Resulting root-filesystems are basicly customized Debian installations, which can be maintained through Debian's tools like APT. This is the biggest difference of ELBE to other buildsystems like Yocto or Buildroot.
 
It is possible to define "finetuning"-commands in the xml-file, which are executed when the creation of the root-filesystem is completed. With this it is possible to remove files, add additional users and much more ...
 
Xml-files can be appended with a bzip2 compressed archive. At the end of the buildprocess, the content of the archive will be extracted into the root-filesystem.
 
The form of the finished root-filesystem is customizeable. The following variants are possible:
* Tarball: The root-filesystem can be packed as a tarball (usually compressed with gzip). The tarball can be extracted onto an empty partition or can be used for the [[DHCOM Update Mechanism]].
* Disk-/SD-card-image: An image with full partition table can be created. This can be a image of a hard disk or a sd-card. The size, type, label and mountpoint of each partition can be customized via the xml-file.
* UBIFS-Image: ELBE can output images for "Unsorted Block Image"-devices (UBI). These are needed if you want to use the NAND-Flash of our DHSOM modules for the root-filesystem. More information about that type of filesystem can be found under [[UBI/UBIFS Filesystem]].
As an addition ELBE is able to create a cdrom image which can be used as a input to reconstruct the exact root-filesystem that was created.
 
ELBE can also create a Software Development Kit (SDK) for the built root-filesystem. The SDK contains the toolchain with the cross compiler and other tools and the corresponding sysroot with needed headers. These can be used to create software for the specific root-filesystem. The SDK is very similar to Yocto's Standard-SDK and can be used in the same way. An explanation how to use the SDK with some IDEs/buildsystems can be found here: [[Using Yocto/ELBE SDK for application development]]
 
==Installation==
 
There are multiple ways to install ELBE:
 
===Debian Package===
 
The developers of ELBE have their own package repository which can be used to install ELBE.
 
To add the repository to the <tt>/etc/apt/sources.list.d</tt> directory and install the public key for the repo:
<tt>echo 'deb http://debian.linutronix.de/elbe buster main' >> /etc/apt/sources.list.d/elbe.list
wget http://debian.linutronix.de/elbe/elbe-repo.pub && apt-key add elbe-repo.pub</tt>
 
Now you can install ELBE with:
<tt>apt update
apt install elbe</tt>
 
'''Important''': You have to add your user to the libvirt groups if you want to use ELBE without root:
<tt>adduser <user> libvirt && adduser <user> libvirt-qemu</tt>
 
===Cloning the git repository of ELBE===
 
There is also the possibility to clone the git repository of ELBE.
 
'''Up to version 12.5''' the following packages have to be installed on the system:
<tt>apt install python python-debian python-mako python-lxml python-apt python-gpg python-suds python-libvirt qemu-utils qemu-kvm p7zip-full make libvirt-daemon libvirt-daemon-system libvirt-clients</tt>
 
'''Starting with version 13.0''' ELBE did switch to version 3 of Python (due to the EOL of Python 2), so the following packages have to be installed:
<tt>apt install python3 python3-debian python3-mako python3-lxml python3-apt python3-gpg python3-suds python3-libvirt qemu-utils qemu-kvm p7zip-full make libvirt-daemon libvirt-daemon-system libvirt-clients</tt>
 
Then you can clone the repo with:
<tt>git clone https://github.com/Linutronix/elbe</tt>
 
'''Important''': You have to add your user to the libvirt groups if you want to use ELBE without root:
<tt>adduser <user> libvirt && adduser <user> libvirt-qemu</tt>
 
===Premade Docker container===
 
We created docker container images with ELBE preinstalled, these can be found at [https://hub.docker.com/r/dhelectronics/elbe Docker Hub] under the name dhelectronics/elbe. There are two different versions of container tags available:
 
{| class="wikitable"
|-
! Tag !! Version the tag is pointing to.
|-
| git-latest-tested || Points to the latest version we have used and confirmed that is working with our own recipies. This version will change from time to time and some versions may be skipped.
|-
| git-vX || Always points to the specific version X of ELBE (as an example git-v12.4 points to version 12.4 of ELBE).
|}
 
To get an image enter (e.g. version 12.4)
<tt>docker pull dhelectronics/elbe:git-v12.4</tt>
 
The container can be started with the following command: (the container does need extensive rights because a virtual machine is started inside it):
 
<tt>docker run --name=elbe --mount type=bind,src=./,dst=/workdir-w /workdir/ --cap-add SYS_ADMIN --security-opt seccomp:unconfined --security-opt apparmor:unconfined --mount type=bind,src=/sys/fs/cgroup,dst=/sys/fs/cgroup --device /dev/kvm --device /dev/fuse --group-add kvm dhelectronics/elbe:git-v12.4</tt>
 
Now you have to start a shell inside the container with:
 
<tt>docker exec elbe /bin/bash</tt>
 
Inside the container you can now run your ELBE build, you can exit the shell with CTRL+D and remove the container with <tt>docker container rm -f elbe</tt>.
 
==Usage==
 
The ELBE buildsystem consists of the program elbe which is implemented in Python. Similar to git, it can be called with several sub commands. If you installed ELBE through the Debian package, the program is inside your PATH environment and can be called by "<tt>elbe</tt>", else you have to call it by the complete path. The most important commands are:
<tt>elbe initvm create --directory /example example.xml</tt>
This command initializes a new initvm in the directory "/example" and builds a new root-filesystem based on "example.xml" . This command can only be used if there isn't any initvm defined yet. If you cloned ELBE from git, you can add the switch "<tt>--devel</tt>", then ELBE copies the local ELBE version into the initvm (instead of downloading ELBE from their repository).
<tt>elbe initvm submit --directory /example example.xml</tt>
This command builds a new root-filesystem based on "example.xml" and reuses an existing initvm at the directory "/example".
<tt>elbe chg_archive example.xml example-archive.tar.bz2</tt>
This command appends the xml-file "example.xml" with the archive "example-archive.tar.bz2". The binary content of the archive is stored in BASE64 encoding inside a xml tag.
 
If you add  <tt>--build-sdk</tt> at the "<tt>initvm create/submit</tt>"-command then as an addition to creating the root-filesystem, the SDK (for developing software for the root-filesystem) is created too. Usually CD-Images with all binary packages and all source packages (for the exact recreation of the rootfilesystem) are created. The creation of these can be skipped with the commandline switches <tt>--skip-build-bin</tt> and <tt>--skip-build-sources</tt>.
 
If an error happens while building the rootfilesystem, no files are returned. It is possible to get the log and other files of the build with an extra command. The complete call of the command (which differs for each build) is printed as exit-message of the ELBE build.
 
===Script for easy use with container images===
 
We created a script for easing the use of ELBE inside the container. This script is preinstalled in our [[Virtual_Machine_for_Application_Development|Virtual Machine for Application Development]] starting with version '''Not yet available'''. The script is called '''elbe_container_build_start.sh [OPTIONS] <elbe_build_description.xml>'''. In the standard configuration the CD-images with binary and source packages are created. The SDK is also created.
 
It is possible to start an additional container with a debian repository, the given path at --repodir has to point at the root of a debian repository. The directory is mounted into the directory <tt><nowiki>/repo/dh_debian_repo</nowiki></tt> of the container. Any container which is configured to publish the mounted directory can be used. Inside the xml the repository can be used by adding the repo to the list of mirrors. Note that after the build process the repository is still inside the <tt>/etc/apt/sources.list</tt> of the resulting rootfilesystem. You have to remove this line at the finetuning section of the xml file or when the device running with the rootfilesystem has started. If you do not remove the repo, <tt>apt-get update</tt> will fail with an error if you want to update the system or install new packages afterwards.
 
The following options are available for the script:
 
{| class="wikitable"
|-
| --elbedockerimage <REPO:TAG> || Sets the docker image which is used for the elbe build.
|-
| --repodir <path> || Starts an additional container with a Debian repo webserver, which uses this repo path. If this option isn't set, then no container with the repo is started.
|-
| --repodockerimage <REPO:TAG> || Sets Debian repo webserver docker image. Has no effect if <tt>--repodir<tt> is not set.
|-
| --initvmtopdir <path> || Change initvm top directory, default value is the <tt><current-working-directory>/initvm-docker</tt>
|-
| --fast || Speed up the build of the rootfilesystem with only building the rootfilesystem itself, and skipping the creation of the CD-images and of the SDK.
|-
| --console || Ignore every other option and start the container and enter a console inside of the container. The current working directory is mounted into the container
|-
| --version || Display version info of the script and exit
|-
| --help || Display the command line help and exit
|}
 
It is not possible to append a XML file with an archive via an option of the script. If you need to do this you have to use the "<tt>--console</tt>"-option to open a console inside the container where you can use the "<tt>elbe chg_archive</tt>"-command.
 
When the build is finished, the container exits and is removed. If an error happenes the container is not exited, and a console inside the container is opened, where you can get the logs and files of the failed build.
 
==XML-Example==
 
This is a simple example for a xml file. If you start the build process a root-filesystem for a device of the "ARM EABI with hard float"-architecture (usually ARMv7) based on Debian 10 is built. In extra to the main mirror, the security and buster-updates mirror is also configured, so that the root-filesystem can get security updates. After the package installation is finished, the cached Debian package files are removed to slim down the image. At the end, the finished root-filesystem is packed into gzip-compressed tarball "<tt>rootfs.tar.gz</tt>". Note that the resulting root-filesystem has no kernel, bootloader and device tree, these have to be supplied externally.
 
<tt>'''<font color="#0000FF"><ns0:RootFileSystem</font>''' <font color="#009900">xmlns:ns0</font><font color="#990000"><nowiki>=</nowiki></font><font color="#FF0000">"https://www.linutronix.de/projects/Elbe"</font>
        <font color="#009900">xmlns:xsi</font><font color="#990000"><nowiki>=</nowiki></font><font color="#FF0000">"http://www.w3.org/2001/XMLSchema-instance"</font>
        <font color="#009900">created</font><font color="#990000"><nowiki>=</nowiki></font><font color="#FF0000">"2009-05-20T08:50:56"</font> <font color="#009900">revision</font><font color="#990000"><nowiki>=</nowiki></font><font color="#FF0000">"4"</font>
        <font color="#009900">xsi:schemaLocation</font><font color="#990000"><nowiki>=</nowiki></font><font color="#FF0000">"https://www.linutronix.de/projects/Elbe dbsfed.xsd"</font>'''<font color="#0000FF">></font>'''
        '''<font color="#0000FF"><project></font>'''
                '''<font color="#0000FF"><name></font>'''armhf-example'''<font color="#0000FF"></name></font>'''
                '''<font color="#0000FF"><version></font>'''1.0'''<font color="#0000FF"></version></font>'''
                '''<font color="#0000FF"><buildtype></font>'''armhf'''<font color="#0000FF"></buildtype></font>'''
                '''<font color="#0000FF"><mirror></font>'''
                        '''<font color="#0000FF"><primary_host></font>'''ftp.de.debian.org'''<font color="#0000FF"></primary_host></font>'''
                        '''<font color="#0000FF"><primary_path></font>'''/debian'''<font color="#0000FF"></primary_path></font>'''
                        '''<font color="#0000FF"><primary_proto></font>'''http'''<font color="#0000FF"></primary_proto></font>'''
                        '''<font color="#0000FF"><url_list></font>'''
                                '''<font color="#0000FF"><url></font>'''
                                        '''<font color="#0000FF"><<nowiki />source></font>'''http://security.debian.org/debian-security buster/updates main'''<font color="#0000FF"></source></font>'''
                                        '''<font color="#0000FF"><binary></font>'''http://security.debian.org/debian-security buster/updates main'''<font color="#0000FF"></binary></font>'''
                                '''<font color="#0000FF"></url></font>'''
                                '''<font color="#0000FF"><url></font>'''
                                        '''<font color="#0000FF"><<nowiki />source></font>'''http://ftp.de.debian.org/debian buster-updates main'''<font color="#0000FF"></source></font>'''
                                        '''<font color="#0000FF"><binary></font>'''http://ftp.de.debian.org/debian buster-updates main'''<font color="#0000FF"></binary></font>'''
                                '''<font color="#0000FF"></url></font>'''
                        '''<font color="#0000FF"></url_list></font>'''
                '''<font color="#0000FF"></mirror></font>'''
                '''<font color="#0000FF"><suite></font>'''buster'''<font color="#0000FF"></suite></font>'''
        '''<font color="#0000FF"></project></font>'''
        '''<font color="#0000FF"><target></font>'''
                '''<font color="#0000FF"><hostname></font>'''armhf-example'''<font color="#0000FF"></hostname></font>'''
                '''<font color="#0000FF"><passwd></font>'''foo'''<font color="#0000FF"></passwd></font>'''
                '''<font color="#0000FF"><console></font>'''ttyS0,115200'''<font color="#0000FF"></console></font>'''
                '''<font color="#0000FF"><package></font>'''
                        '''<font color="#0000FF"><tar></font>'''
                                '''<font color="#0000FF"><name></font>'''rootfs.tar.gz'''<font color="#0000FF"></name></font>'''
                        '''<font color="#0000FF"></tar></font>'''
                '''<font color="#0000FF"></package></font>'''
                '''<font color="#0000FF"><finetuning></font>'''
                        '''<font color="#0000FF"><rm></font>'''/var/cache/apt/archives/*.deb'''<font color="#0000FF"></rm></font>'''
                '''<font color="#0000FF"></finetuning></font>'''
                '''<font color="#0000FF"><pkg-list></font>'''
                        '''<font color="#0000FF"><pkg></font>'''bash'''<font color="#0000FF"></pkg></font>'''
                        '''<font color="#0000FF"><pkg></font>'''less'''<font color="#0000FF"></pkg></font>'''
                        '''<font color="#0000FF"><pkg></font>'''git'''<font color="#0000FF"></pkg></font>'''
                        '''<font color="#0000FF"><pkg></font>'''openssh-server'''<font color="#0000FF"></pkg></font>'''
                        '''<font color="#0000FF"><pkg></font>'''nfs-common'''<font color="#0000FF"></pkg></font>'''
                        '''<font color="#0000FF"><pkg></font>'''curl'''<font color="#0000FF"></pkg></font>'''
                        '''<font color="#0000FF"><pkg></font>'''vim'''<font color="#0000FF"></pkg></font>'''
                        '''<font color="#0000FF"><pkg></font>'''iperf3'''<font color="#0000FF"></pkg></font>'''
                        '''<font color="#0000FF"><pkg></font>'''i2c-tools'''<font color="#0000FF"></pkg></font>'''
                '''<font color="#0000FF"></pkg-list></font>'''
        '''<font color="#0000FF"></target></font>'''
'''<font color="#0000FF"></ns0:RootFileSystem></font>'''</tt>
 
==Further Reading==
* [https://elbe-rfs.org/ ELBE homepage]
* [https://elbe-rfs.org/docs/sphinx/index.html Official documentation of the current ELBE version]
* [https://github.com/Linutronix/elbe ELBE repository at Github]
<!--
The Embedded Linux Build Environment (elbe) is a system to generate root-filesystems for embedded devices. For the latest documentation and version please have a look at [https://github.com/Linutronix/elbe https://github.com/Linutronix/elbe]. The following tutorial was made to get started with elbe 0.3.1 which was used to build debian '''''squeeze'''''


== General Notes ==
== General Notes ==
Line 260: Line 446:
         '''<font color="#0000FF"><rm></font>'''opt/elbe'''<font color="#0000FF"></rm></font>'''                              (6)
         '''<font color="#0000FF"><rm></font>'''opt/elbe'''<font color="#0000FF"></rm></font>'''                              (6)
         '''<font color="#0000FF"><rm></font>'''var/cache/debconf/*'''<font color="#0000FF"></rm></font>'''                    (7)
         '''<font color="#0000FF"><rm></font>'''var/cache/debconf/*'''<font color="#0000FF"></rm></font>'''                    (7)
'''<font color="#0000FF"></finetuning></font>'''</tt>
'''<font color="#0000FF"></finetuning></font>'''</tt>
</div></div><div class="colist arabic">
</div></div><div class="colist arabic">


Line 686: Line 872:
                 '''<font color="#0000FF"></bylabel></font>'''
                 '''<font color="#0000FF"></bylabel></font>'''
                 '''<font color="#0000FF"><bydev></font>'''                                        (6)
                 '''<font color="#0000FF"><bydev></font>'''                                        (6)
                         '''<font color="#0000FF"><source></font>'''proc'''<font color="#0000FF"></source></font>'''
                         '''<font color="#0000FF"><<nowiki />source></font>'''proc'''<font color="#0000FF"></source></font>'''
                         '''<font color="#0000FF"><mountpoint></font>'''/proc'''<font color="#0000FF"></mountpoint></font>'''
                         '''<font color="#0000FF"><mountpoint></font>'''/proc'''<font color="#0000FF"></mountpoint></font>'''
                         '''<font color="#0000FF"><fs></font>'''
                         '''<font color="#0000FF"><fs></font>'''
Line 693: Line 879:
                 '''<font color="#0000FF"></bydev></font>'''
                 '''<font color="#0000FF"></bydev></font>'''
                 '''<font color="#0000FF"><bydev></font>'''
                 '''<font color="#0000FF"><bydev></font>'''
                         '''<font color="#0000FF"><source></font>'''sysfs'''<font color="#0000FF"></source></font>'''
                         '''<font color="#0000FF"><<nowiki />source></font>'''sysfs'''<font color="#0000FF"></source></font>'''
                         '''<font color="#0000FF"><mountpoint></font>'''/sys'''<font color="#0000FF"></mountpoint></font>'''
                         '''<font color="#0000FF"><mountpoint></font>'''/sys'''<font color="#0000FF"></mountpoint></font>'''
                         '''<font color="#0000FF"><fs></font>'''
                         '''<font color="#0000FF"><fs></font>'''
Line 700: Line 886:
                 '''<font color="#0000FF"></bydev></font>'''
                 '''<font color="#0000FF"></bydev></font>'''
                 '''<font color="#0000FF"><bydev></font>'''
                 '''<font color="#0000FF"><bydev></font>'''
                         '''<font color="#0000FF"><source></font>'''tmpfs'''<font color="#0000FF"></source></font>'''
                         '''<font color="#0000FF"><<nowiki />source></font>'''tmpfs'''<font color="#0000FF"></source></font>'''
                         '''<font color="#0000FF"><mountpoint></font>'''/tmp'''<font color="#0000FF"></mountpoint></font>'''
                         '''<font color="#0000FF"><mountpoint></font>'''/tmp'''<font color="#0000FF"></mountpoint></font>'''
                         '''<font color="#0000FF"><fs></font>'''
                         '''<font color="#0000FF"><fs></font>'''
Line 708: Line 894:
                 '''<font color="#0000FF"></bydev></font>'''
                 '''<font color="#0000FF"></bydev></font>'''
                 '''<font color="#0000FF"><bydev></font>'''
                 '''<font color="#0000FF"><bydev></font>'''
                         '''<font color="#0000FF"><source></font>'''tmpfs'''<font color="#0000FF"></source></font>'''
                         '''<font color="#0000FF"><<nowiki />source></font>'''tmpfs'''<font color="#0000FF"></source></font>'''
                         '''<font color="#0000FF"><mountpoint></font>'''/var/log'''<font color="#0000FF"></mountpoint></font>'''
                         '''<font color="#0000FF"><mountpoint></font>'''/var/log'''<font color="#0000FF"></mountpoint></font>'''
                         '''<font color="#0000FF"><fs></font>'''
                         '''<font color="#0000FF"><fs></font>'''
Line 716: Line 902:
                 '''<font color="#0000FF"></bydev></font>'''
                 '''<font color="#0000FF"></bydev></font>'''
                 '''<font color="#0000FF"><bydev></font>'''
                 '''<font color="#0000FF"><bydev></font>'''
                         '''<font color="#0000FF"><source></font>'''tmpfs'''<font color="#0000FF"></source></font>'''
                         '''<font color="#0000FF"><<nowiki />source></font>'''tmpfs'''<font color="#0000FF"></source></font>'''
                         '''<font color="#0000FF"><mountpoint></font>'''/var/run'''<font color="#0000FF"></mountpoint></font>'''
                         '''<font color="#0000FF"><mountpoint></font>'''/var/run'''<font color="#0000FF"></mountpoint></font>'''
                         '''<font color="#0000FF"><fs></font>'''
                         '''<font color="#0000FF"><fs></font>'''
Line 760: Line 946:
|}
|}


</div></div></div></div>
</div></div></div>
-->
</div>

Latest revision as of 09:57, 5 October 2020