Upstream package sources

Supply chain? 🤔

The Linux packaging supply chain is composed of all elements (e.g. infrastructure, technical processes and individuals) involved in building and delivering packages to users.

Supply chain attack? 🤔

A supply chain attack happens when one (or more) of the supply chain components is compromised.

Do we have protections? 🛡️

Fortunately, mechanisms exist to prevent (or at least detect) such supply chain attacks…

…but monitoring & securing the entire supply chain remains challenging.

Checksums 🔒

• Ensure that sources have not been altered in between multiple downloads or usages.
• Unmatched checksums could indicate a supply chain attack, but may also have different reasons.
• This is technically more of a source locking mechanism than an actual integrity check.

Reproducible builds 🔄

• Allow to independently verify the path from source to binary code, ensuring no supply chain attack occurred in between.
• Provide a technical verification process, avoiding the need to trust an individual.
• While always a desired outcome, Reproducible Builds doesn’t always cover the whole supply chain.

Cryptographical signatures 🔏

• Act as an additional integrity check by guaranteeing that an artifact has not been tampered with (since it was signed).
• Allow for the authentication of an identity tied to the signer…
• …but said identity may or may not relate to a person and may or may not correlate with the creator of the artifact.
• Realistically, signatures are only relevant if accompanied by a trust path / chain of trust.

And yet… 🫣

Despite the infected upstream package sources being locked behind checksums and cryptographically signed:

$ cat PKGBUILD
pkgname=xz
pkgver=5.6.1
pkgrel=1
pkgdesc='Library and command line tools for XZ and LZMA compressed files'
arch=('x86_64')
url='https://xz.tukaani.org/xz-utils/'
license=('GPL' 'LGPL' 'custom')
depends=('sh')
provides=('liblzma.so')
validpgpkeys=('3690C240CE51B4670D30AD1C38EE757D69184620'  # Lasse Collin <lasse.collin@tukaani.org>
              '22D465F2B4C173803B20C6DE59FCF207FEA7F445') # Jia Tan <jiat0218@gmail.com>
source=("https://github.com/tukaani-project/xz/releases/download/v${pkgver}/xz-${pkgver}.tar.gz"{,.sig})
sha256sums=('2398f4a8e53345325f44bdd9f0cc7401bd9025d736c6d43b372f4dea77bf75b8'
            'SKIP')
sha512sums=('8af100eb83288f032e4813be2bf8de7d733c8761f77f078776c1391709241ad8fe3192d107664786e2543677915c5eeb3fe7add5c53b48b50c10a9de7c9f4fda'
            'SKIP')
[...]

And despite the infected package being reproducible:

$ repro -dnf xz-5.6.1-1-x86_64.pkg.tar.zst
[...]
==> Leaving fakeroot environment.
==> Finished making: xz 5.6.1-1 (Mon Jan  5 14:09:54 2026)
==> Cleaning up...
  -> Delete snapshot for xz_294294...
==> Comparing hashes...
==> Package is reproducible!

How did that happen? 🕵️

• The malicious code was introduced directly in the upstream sources by one of the upstream maintainers.
• Such early parts of the supply chain are more difficult to cover and verify.
• (Part of) the malicious code was obfuscated by only being present in the custom-made release tarball but not in the “raw” sources / git repository.

Different types of sources 🗂️

• Plain VCS objects (e.g. git commits, git tags)
• Auto-generated tarballs by git forges
• Custom-made release tarballs

Why custom-made release tarballs? 🤷

• Generally easier / more straightforward to work with (packaging wise):
  • autotools, git submodules, translation files, man pages
• Historically cryptographically signed more often than other sources types.
• But opaque by nature and way more difficult & involved to audit.

Transparent sources 📖

• Publicly readable, easily accessible and therefore generally more audited.
• “What you see is what you get.”
• Despite an eventual cost in terms of complexity (packaging wise) they are therefore generally considered more trustworthy.

Resources 🔗

• XZ Arch Linux ASA
• Arch Linux “Upstream package sources” RFC
• whatsrc.org

Contact ☎️

• antiz@archlinux.org
• https://antiz.fr
• https://fosstodon.org/@Antiz

Slides 📋