Making .deb files

Notes and other various tutorials on how to package .deb files for Debian based distributions

Debian Packaging Tutorial by Gürkan Myczko

Debian Packaging

pretty much the same with Ubuntu

2003-2020 Gürkan Myczko <gurkan@phys.ethz.ch>

Introduction

We will package mguesser for Debian. Knowledge about Makefiles, configure and shell scripting or C programming can be very helpful. Man and the manpages are also very useful. Also have a look at help2man or pod2man. I want to guide you packaging mguesser from the beginning to the end.

What you will need

A development environment, some utilities and debhelper which we will install like this:

  # apt-get install binutils cpp cpio dpkg-dev file gcc make patch \
                    dh-make debhelper devscripts fakeroot lintian \
                    debian-policy developers-reference \
                    man-db manpages reportbug

(or build-essential lintian debhelper dh-make devscripts fakeroot)

Setting up the environment, see www.linuks.mine.nu/conf/. Of course you fill in your email address and your full name and choose your favourite editor, these settings is what I use:

  $ export DEBEMAIL="gurkan@phys.ethz.ch"
  $ export DEBFULLNAME="Gürkan Myczko"
  $ export EDITOR=mcedit

Choosing software you want to package for Debian

For this course I have chosen mguesser, as I packaged it already and it is not too hard (single binary) to get you started. This should give you an idea what it is like. For Debian the package

• must be DFSG compliant
• must have some public download place (usually http/https, or ftp)
• naming should be software-maj.min.tar.gz and unpacks into software-maj.min/

Build systems

There are plenty of build systems. The most classical ones are

  make; make install

  ./configure; make; make install

  mk (Plan 9)

  cmake (ccmake is handy sometimes)

  scons

  qmake (for qt based software)

  gnustep-make

  bazel

Plenty more to check out at https://en.wikipedia.org/wiki/List_of_build_automation_software

Packaging the software

First you need to get the source of the software. Then put it in some directory. There you unpack the software, usually

  $ tar xzvf software-2.1.tar.gz
  $ ln -s software-2.1.tar.gz software_2.1.orig.tar.gz
  $ cd software-2.1/
  $ dh_make

dh_make will make a directory called debian/ with a few files in it. Now your task is to fill them. I usually start with control, then edit copyright. There is also changelog, rules, dirs and more. In rules you will find some dh commands, they are all from debhelper. Each of them has a manpage, please refer to man debhelper for details, as well as the Debian New Maintainers' Guide. Read more about the Debian Policy Manual. Updating the debian/changelog is done with dch or debchange (symlink).

When you think the thing is ready, you can start building the package

The most important files are probably debian/changelog, debian/control, debian/copyright, debian/rules and the following ones are helpful

debian/clean files to be removed that get generated at build time (make clean)

debian/install files to be installed

debian/manpages manpage(s) to be installed

You will find licensecheck -r . | grep -v UNKNOWN useful during the writing of debian/copyright.

  $ debuild

When things are built you should check the package using

  $ lintian -vi mguesser_0.2-1_i386.changes

If that says all is right (no Warnings or Errors). You can install it with

  $ sudo dpkg -i mguesser_0.2-1_i386.deb

You must check if the program works. Also make sure the manpage is working. From what I experienced I will most likely to have go through this several times.

Never forget reportbug, it will be your friend. When you package something from http://www.debian.org/devel/wnpp/requested you can close the bug also with an e-mail to <number>-done@bugs.debian.org. If you don't know what to package, check orphaned packages.

It can be helpful to look at how other packages are packaged

  $ apt-get source package

And being in touch with upstream is important, contact them somehow. My debian/ directory for mguesser. Here is my Makefile changes.

Ensuring completeness of Build-Depends Use `sbuild` to ensure build-depends are complete:

  apt-get install sbuild
  mkdir -p /srv/chroot/sid
  sbuild-createchroot --include=eatmydata,ccache,gnupg unstable /srv/chroot/sid http://deb.debian.org/debian

Now test your packages with:

  sbuild -d sid yourpackage_version-rev.dsc

After Work

Adding a screenshot to the package is recommended: http://screenshots.debian.net/

Tagging the package is also a good idea: https://debtags.debian.org/

Check Lintian output: https://lintian.debian.org/

I did not go into detail with debian/watch.

Make sure it builds on as many architectures as possible: https://buildd.debian.org/

If your software works with data files, that can be detected with the file(5) utility, send patches for its detection.

Links

A good source for free software https://github.com All Debian project participants www.debian.org/devel/people

Other packaging systems (ports, pkg, rpm)

Freebsd: https://www.freebsd.org/doc/en/books/porters-handbook/

Solaris: https://docs.oracle.com/cd/E26505_01/html/E28550/ch2buildpkg-17051.html

RedHat: https://access.redhat.com/sites/default/files/attachments/rpm_building_practice_10082013.pdf

macOS: https://guide.macports.org/#project.contributing

Spack: https://github.com/spack/spack

flatpak/flathub: https://flatpak.org/ / https://flathub.org/

Ubuntu Snap: https://snapcraft.io/

Checkinstall

checkinstall keeps track of all the files created or modified by your installation script, builds a standard binary package (.deb, .rpm, .tgz) and installs it in your system giving you the ability to uninstall it with your distribution's standard package management utilities.

You have an archive with software to build.

• Normal install

  ./configure
  make
  make install
  make clean

• Build and use checkinstall to install and build the .deb file

  ./configure
  make
  checkinstall

• Build and use checkinstall to create the .deb file without installing it

  ./configure
  make
  checkinstall --install=no

Refer to the man page for more information

Simple method #1

From https://www.internalpointers.com/post/build-binary-deb-package-practical-guide A deb file is an archive that contains data. Marked with the .deb extension, it is used to easily distribute and install programs for Linux Debian and derivatives. Deb files are handy when your app needs to take care of additional dependencies, integrate itself with the desktop, run pre and post install scripts and so on.

In this quick tutorial I want to show you how to generate a deb package from scratch that will install a binary executable in the target system. Let's start off with a bit of theoretical background.

Anatomy of a deb package

A deb is a standard Unix ar archive that contains your application and other utility files. The most important one is the control file, which stores the information about the deb package and the program it installs.

Internally, a deb package contains a collection of folders that mimics a typical Linux file system, such as /usr, /usr/bin, /opt and so on. A file put in one of those directories will be copied to the same location in the actual file system during installation. So, for example a binary file put into <.deb>/usr/local/bin/binaryfile will be installed to /usr/local/bin/binaryfile.

On the outside instead, all deb package files follow a specific naming convention:

<name>_<version>-<revision>_<architecture>.deb

That is:

   <name> – the name of your application;
   <version> – the version number of your application;
   <revision> – the version number of the current deb package;
   <architecture> – the hardware architecture your program will be run on.

For example, suppose you want to release your program called hello, version 1.0, built for 64-bit ARM processors. Your deb file name would look something like hello_1.0-1_arm64.deb.

Making the deb package

We are now ready to generate the package. Make sure you have the dpkg-deb program installed in your system: this will be used later on to generate the final archive

Create working directory

Create a temporary working directory to make your package in. Follow the same naming convention we have seen before. For example:

  mkdir hello_1.0-1_arm64

Create internal structure

Put your program files where they should be installed to on the target system. For example, suppose you want your program to be installed to /usr/local/bin:

  mkdir -p hello_1.0-1_arm64/usr/local/bin

The -p flag to the mkdir command will create nested directories. Then copy the executable file in there:

  cp ~/YourProjects/Hello/hello hello_1.0-1_arm64/usr/local/bin

Create control file

The control file lives inside the DEBIAN directory. Mind the uppercase: a similar directory named debian (lowecase) is used to store source code for the so-called source packages. This tutorial is about binary packages, so we don't need it.

Let's create the DEBIAN folder first:

  mkdir helloworld_1.0-1_arm64/DEBIAN

And then create the empty control file:

  touch helloworld_1.0-1_arm64/DEBIAN/control

Fill in control file

Open the file previously created with your text editor of choice. The control file is just a list of data fields. For binary packages there is a minimum set of mandatory ones:

   Package – the name of your program;
   Version – the version of your program;
   Architecture – the target architecture;
   Maintainer – the name and the email address of the person in charge of the package maintenance;
   Description – a brief description of the program.

For example:

  Package: hello
  Version: 1.0
  Architecture: arm64
  Maintainer: Internal Pointers <info@internalpointers.com>
  Description: A program that greets you.
    You can add a longer description here. Mind the space at the beginning of this paragraph.

The control file may contain additional useful fields such as the section it belongs to or the dependency list. The latter is extremely important in case your program relies on external libraries to work correctly. You can fill it manually if you wish, but there are helper tools to ease the burden. I will show you how in the next few paragraphs.

Build .deb package

This is the last step. Invoke dpkg-deb as following:

  dpkg-deb --build --root-owner-group <package-dir>

So in our example:

  dpkg-deb --build --root-owner-group <helloworld_1.0-1_arm64>

The --root-owner-group flag makes all deb package content owned by the root user, which is the standard way to go. Without such flag, all files and folders would be owned by your user, which might not exist in the system the deb package would be installed to.

The command above will generate a nice .deb file alongside the working directory or print an error if something is wrong or missing inside the package.

Test .deb package

It's a good idea to test your deb package once created. You can install it like any other regular deb package:

  sudo dpkg -i <package>

Make sure it can be also uninstalled easily. You can just remove the package:

  sudo dpkg -r <appname>

or remove it along with the configuration files (if any):

  sudo dpkg -P <appname>

Make sure the application has been removed correctly by issuing:

  dpkg -l | grep <appname>

The dpkg -l command lists all the packages installed, while grep searches for <appname>. The output should be blank if the app has been uninstalled correctly.

Sometimes the deb install goes wrong

Especially when you are dealing with pre/post install or removal scripts that fail at some point. This is a typical error message by dpkg:

   Package is in a very bad inconsistent state - you should reinstall it before attempting a removal.

which prevents any progress. The trick is to move all references to your broken package somewhere safe (e.g. the /tmp directory) and then force remove it, like so:

  sudo mv /var/lib/dpkg/info/<packagename>.* /tmp/
  sudo dpkg --remove --force-remove-reinstreq <packagename>

Taking care of external dependencies

You can generate them automatically with dpkg-shlibdeps. It will parse your binary and look for external symbols. At the time of writing, that tool doesn't seem to work out of the box. For some unknown (to me) reasons it wants the debian/control file to be present in the working directory – that's for source packages, remember? The workaround here is to create it, then move into the working folder and run:

  dpkg-shlibdeps -O path/to/binary/file

The -O flag will print dependencies on the standard output. Copy the output and paste it in the Depends section of your DEBIAN/control file. You can get rid of the debian/control file once done. I'm pretty sure there are better ways for this, though: I will update the article once I get additional information on this step.

Run scripts before or after package installation and removal

Four files: postinst, preinst, postrm, and prerm are called maintainer scripts. Such files live inside the DEBIAN directory and, as their names suggest, preinst and postinst are run before and after installation, while prerm and postrm are run before and after removal. They must be marked as executables. Also, remember to set permissions: must be between 0555 and 0775.

Simple method #2

• Configure your paths and filenames

  SOURCEBINPATH=~
  SOURCEBIN=myscript.sh
  DEBFOLDER=~/somescripts
  DEBVERSION=0.1
  
  DEBFOLDERNAME=$DEBFOLDER-$DEBVERSION

• Create your scripts source dir

  mkdir $DEBFOLDERNAME

• Copy your script to the source dir

  cp $SOURCEBINPATH/$SOURCEBIN $DEBFOLDERNAME 
  cd $DEBFOLDERNAME

• Create the packaging skeleton (debian/*)

  dh_make -s --indep --createorig 

• Remove make calls

  grep -v makefile debian/rules > debian/rules.new 
  mv debian/rules.new debian/rules 

• debian/install must contain the list of scripts to install as well as the target directory

  echo $SOURCEBIN usr/bin > debian/install 

• Remove the example files

  rm debian/*.ex

• Build the package. You will get a lot of warnings and ../somescripts_0.1-1_i386.deb

  debuild

GPG sign and verify deb packages and APT repositories

From https://packagecloud.io/blog/how-to-gpg-sign-and-verify-deb-packages-and-apt-repositories/

TL;DR

GPG can be used to create a digital signature for both Debian package files and for APT repository metadata.

Many Debian-based Linux distributions (e.g., Ubuntu) have GPG signature verification of Debian package files (.deb) disabled by default and instead choose to verify GPG signatures of repository metadata and source packages (.dsc). The setting which enables GPG signature checking of the individual .deb packages can be found in /etc/dpkg/dpkg.cfg and is set to no-debsig, but there are important caveats to enabling this option explained below.

Further, most official Debian package files from the publicly accessible repositories do not have GPG signatures. The official repository metadata is GPG signed, as are the source packages, but the .deb packages themselves are not.

If you publish a Debian package and GPG sign the package yourself before distributing it to users, those users’ systems will, in most cases, not verify the signature of the package unless they have done a considerable amount of configuration. However, their system will, in most cases, automatically verify repository metadata.

This post explains how Debian package GPG signatures are implemented, how to enable GPG signature checking of packages on your system, and how to GPG sign packages you create.

This post will also explain how APT repository GPG signatures are implemented and how to sign your APT repository metadata

Debian package GPG signatures

Signing data with a GPG key enables the recipient of the data to verify that no modifications occurred after the data was signed (assuming the recipient has a copy of the sender’s public GPG key).

Debian package files (.deb files), Debian source packages (.dsc files), and Debian changes files (.changes files) can all be signed with GPG.

GPG signing Debian source packages (.dsc files) and changes files

Debian source packages and Debian changes files are text-based files which can be used to rebuild a Debian package from source code. Theses files can be signed with a GPG key by using a program called debsign.

debsign will add a GPG signature to .dsc and .changes files themselves. You can use debsign like this:

  $ debsign -k E732A79A test_1.0-7.dsc

This command specifies the GPG key ID to use when signing the Debian source file test_1.0-7.dsc. The signature is embedded in the file test_1.0-7.dsc and can be verified by running:

  $ gpg --verify test_1.0-7.dsc

The Debian source package file and changes file both contain SHA1, SHA256, and MD5 checksums of all source files that comprise the source package. Since this data is signed, any modifications to the source file checksums, file size, or other data in the dsc or changes files would cause signature verification to fail.

As long as a recipient who wishes to build the deb package from a dsc file verifies the checksums of each file in the source package and verifies the signature of the source package itself, the recipient knows their build uses precisely the same files that the packager used when building.

apt-get source packagename will automatically check the GPG signature of the source package file and the checksums of the source files listed in the source package.

GPG signing Debian package files (.deb files)

A Debian package file itself is actually an AR archive that contains at least three entries:

• debian-binary: a file describing the deb format version
• control: an archive containing package metadata such as pre-install or post-install scripts
• data: an archive containing the actual installable files

The control and data archives are typically tar, tar.gz, tar.bz2, tar.xz, tar.lzma, or another archive format.

When a Debian package is signed with a GPG key, the signature is also included in the archive and is named based on its type.

Debian package signatures are typically associated with a role. Some commonly used roles include:

• origin: the signature of the organization which distributes the .deb file
• maint: the signature of the package maintainer
• archive: the signature of the archive providing the package to reassure users that the package is the actually provided by the archive

In order to sign a Debian package, you can use the program debsigs:

  $ debsigs --sign=origin -k E732A79A test_1.0-7_amd64.deb

In the above example, the Debian file is signed using the GPG key on my system with the ID E732A79A. The signature type is origin and thus the deb package will be modified to include a file called _gpgorigin that contains the GPG signature of the debian-binary file, control archive, and data archive from the deb package concatenated together.

I used the type origin because I am person distributing the deb package test_1.0-7_amd64.deb.

Verifying signatures on Debian .deb package files is a bit more involved than the other verification methods.

Verifying GPG signatures of .deb package files

n order to verify .deb package files, you must have the program debsig-verify installed, import the public GPG keys you will use to verify packages to the debsig-verify keyrings, and you must also create an XML policy document for signature verification.

First, create a keyring directory for the public key and import the public key to the debsig-verify GPG keyring:

  $ mkdir /usr/share/debsig/keyrings/DDDF2F4CE732A79A/

  $ gpg --no-default-keyring \
        --keyring /usr/share/debsig/keyrings/DDDF2F4CE732A79A/debsig.gpg \
        --import my-public-key

In the above example, the keyring subdirectory, DDDF2F4CE732A79A, is the fingerprint of the GPG key which will be imported. Note that the name of the keyring debsig.gpg will be used in the XML policy document created next. You can use any filename you like.

Now that the public key is imported, the XML policy document can be constructed.

First, create a directory for the policy document. It must have the GPG key fingerprint in the path:

  $ mkdir /etc/debsig/policies/DDDF2F4CE732A79A/

Next create the XML document. The debsig-verify man page indicates that the filename is irrelevant so long as the filename ends with .pol and resides in a directory named after the GPG key fingerpint.

You can find more information regarding how the build the policy document in /usr/share/doc/debsig-verify/policy-syntax.txt and some examples in /usr/share/doc/debsig-verify/examples. Here’s an example XML policy document:

<?xml version="1.0"?>
<!DOCTYPE Policy SYSTEM "http://www.debian.org/debsig/1.0/policy.dtd">
<Policy xmlns="http://www.debian.org/debsig/1.0/">

  <Origin Name="test" id="DDDF2F4CE732A79A" Description="Test package"/>

  <Selection>
    <Required Type="origin" File="debsig.gpg" id="DDDF2F4CE732A79A"/>
  </Selection>

   <Verification MinOptional="0">
    <Required Type="origin" File="debsig.gpg" id="DDDF2F4CE732A79A"/>
   </Verification>
</Policy>

Once this document is constructed, you can verify the GPG signature of the Debian .deb package:

  $ debsig-verify test_1.0-7_amd64.deb
  debsig: Verified package from `Test package' (test)

dpkg support for debsig-verify

dpkg has support for verifying GPG signatures of Debian package files, but this verification is disabled by default. This means that when a package is installed on a Debian-based system, the signature checking for individual packages is disabled.

In order to enable it, the file /etc/dpkg/dpkg.cfg will need to be modified to remove the no-debsig option from the config file.

CAUTION: Doing this may cause dpkg to reject installation of unsigned Debian files. If you plan to enable this, it is strongly recommended you try this first in a testing environment. You’ll also need to ensure you have the correct XML policy files created, as explained above.

Another Debian package signing tool: dpkg-sig

here is a another tool available for signing and verifying Debian package files called dpkg-sig. Signatures created by this tool are not compatible with debsigs and the tools cannot understand or verify each other signatures.

dpkg-sig generates a Debian control file with several userful fields:

• Version of dpkg-sig which generated the signature
• Gpg key signer information
• Role
• Files section with checksums, file sizes, and filenames of the control, data, and debian-binary files in the Debian package

This file is named based on the role selected, which is defaulted to builder if none is specified. So, signing a Debian package file with no role would result in a file named _gpgbuilder being created and added to the package file. This file is signed with GPG and the signature is embedded as plain text (a “clearsign” GPG signature).

The result of this scheme is that the GPG signature can be verified using commonly available software. The Debian package can be extracted with ar and the signature of the control file can be checked with just gpg --verify.

Signing a Debian file with dpkg-sig is straight forward:

  $ dpkg-sig -k E732A79A --sign builder test_1.0-7_amd64.deb

The signature can also be checked by using dpkg-sig directly, if desired:

  $ dpkg-sig --verify test_1.0-7_amd64.deb
  Processing test_1.0-7_amd64.deb...
  GOODSIG _gpgbuilder 284E8BE753AE45DFF8D82748DDDF2F4CE732A79A 1414371553

Additionally, no XML policy document is required for signature verification, which makes signature verification much simpler to integrate into a workflow.

GPG signatures for APT repository metadata

It is also possible to generate GPG signatures for APT repositories in addition to generating signatures for Debian packages themselves.

Generating signatures of the repository metadata allows you to verify that the repository metadata has not been tampered with since it was generated by the repository host.

This is critical to preventing a Man-in-the-Middle attack attack where a malicious actor plants repository metadata causing a user’s package manager to install potentially malicious packages.

Luckily, generating GPG sigantures for APT repositories is straightforward with reprepro.

Assuming you have a GPG key you’d like to use to sign the repository metadata, you can simply modify your reprepro repository config to add your GPG key.

Using the same GPG key ID used in the earlier examples, the conf/distributions config file can be modified to add the field:

  SignWith: E732A79A 

This will cause reprepro to generate GPG signatures of the repository metadata. reprepro will generate a signature of the apt Release file and store the signature in the file Release.gpg. This is called a “detached GPG signature.”

Newer version of reprepro will generate an embedded text GPG signature (a “clearsign” GPG signature) in a file named InRelease.

The apt client will download the GPG Release file(s) it understands and will automatically attempt to verify the GPG signature of the repository.

Users of the repository will need to have the GPG key installed on the system. This can be done by end users running the command: apt-key:

  $ sudo bash -c 'wget -O - https://packagecloud.io/gpg.key | apt-key add -'

The above command downloads a GPG key and adds the GPG key to the system using apt-key

Conclusion

GPG signing of Debian package files, source packages, and repository metadata is critical to establishing the authenticity of the package objects and repository metadata.

Knowing that a package being installed on your system is the same package that was generated by a vendor, and that you were able to retrieve that package from the repository generated by the vendor is the only way to ensure that you are safely installing software in your infrastructure.

Unfortunately, signing and checking the signatures of Debian packages is not as streamlined and integrated as it could be, so vendors currently signing their Debian package files may be surprised to learn that the majority of their customers are not automatically checking the GPG signature of their packages. At the very least, vendors should be signing their repository metadata. Also signing their Debian package files and providing instructions for how users can verify those signatures would be a great way to help improve their end user’s security.

packagecloud.io signs repository metadata it generates using a GPG key so that users of our repositories know, without a doubt, that the repository metadata their package manager downloaded was generated by packagecloud.io.