Properly handle package licenses

[jkloetzke]

Because this came up lately (see BobBuildTool/basement-gnu-linux#85), I wanted to get the discussion rolling on the license tracking front. Out of the top of my head, I see the following points when tracking the licenses of packages properly:

So far we just defined in the basement documentation that each recipe should have an PKG_LICENSE metaEnv variable with a valid SPDX license expression. So far, so good.

• For licenses not in the list, we probably need to use the LicenseRef- references that allows us to create our own identifiers.
  • How do we define the identifiers properly?
  • Where and how do we put the underlying license? Put the text somewhere next to the recipes? Reference some URL?
  • Is it OK to copy the license text or does this create license issues on its own?
• Packages with multiple licenses where only a subset applies to the result in packageScript.
  • We should probably only name licenses that are applicable to the result of a recipe
  • In case different multiPackages contain different licenses, the PKG_LICENSE should be part of the individual multiPackage I guess.
• Do we want to use checkoutAssert in the future for additional safety? This would also document the license files that for example buildroot is naming explicitly.

I probably forgot many things.

[rhubert]

Because this came up lately (see BobBuildTool/basement-gnu-linux#85), I wanted to get the discussion rolling on the license tracking front. Out of the top of my head, I see the following points when tracking the licenses of packages properly:

Thanks for continuing on this. We're using this PKG_LICENSE_FILE variable to point to a license file in the checkout workspace. When building the OSS-License-List we simply pick this file up. This needs a --checkout-only run with a query testing for this variable first. It's not nice but works...

I looked into Annex D of the SPDX spec. Maybe LicenseCrossReference is what could be used for this as well, e.g. for the Intel Microcode:

PKG_LICENSE: 'LicenseCrossReference-${GITHUB_MIRROR}/intel/Intel-Linux-Processor-Microcode-Data-Files/blob/microcode-20250211/license`

This could be added to a license list as is without any further actions needed?

* For licenses not in the list, we probably need to use the `LicenseRef-` references that allows us to create our own identifiers.
  
  * How do we define the identifiers properly?
  * Where and how do we put the underlying license? Put the text somewhere next to the recipes? Reference some URL?

I think these questions can't be answered individually. If we put the licenses into a common folder we need to use a unique identifier (package name), while the name can be freely chosen if the license is in a folder next to the recipe (like patch files are).

  * Is it OK to copy the license text or does this create license issues on its own?

I guess we need to contact some lawyers about this. Most likely we'll get 6 different answers from 5 different lawyers...

* Packages with multiple licenses where only a subset applies to the result in `packageScript`.
  
  * We should probably only name licenses that are applicable to the result of a recipe
  * In case different multiPackages contain different licenses, the `PKG_LICENSE` should be part of the individual multiPackage I guess.

Fine for me for packages where the lib is licensed different to some tools. But there are packages where each source file has it's own license I it's not clear (at least to me) which of them is used. Might also depend on how configure detects packages, so modifying a dependency might pull in more licenses,.... I wouldn't go to much into detail and play safe here by listing them all.

* Do we want to use `checkoutAssert` in the future for additional safety? This would also document the license files that for example buildroot is naming explicitly.

👍

[rhubert]

We're using this PKG_LICENSE_FILE variable to point to a license file in the checkout workspace.

I think this is not sufficient. For example, how do you handle the linux-firmware package? It contains a metric ton of different licenses. 🤦

Luckily we do not need much from this package. I divided it into "some" multipackages and declared the PKG_LICENSE_FILE for each...

I looked into Annex D of the SPDX spec. Maybe LicenseCrossReference is what could be used for this as well, e.g. for the Intel Microcode:

Could you point me to the specification? I couldn't find LicenseCrossReference.

Sure -> https://spdx.github.io/spdx-spec/v2.3/other-licensing-information-detected/#104-license-cross-reference-field

Since we need to checkout the sources to collect the licenses with the PKG_LICENSE_FILE the build job doing this needs a noticeable amount of time. sometimes it's the longest running job of a pipeline. We also had the idea to use this variable to package the license files within the tgt packages, but no time to rework this....

Anyway, I think we must have a generic way of declaring custom licenses. It's not just about pointing to the license. We also need a way how the license texts can be included in the build. Apparently, there are licenses that require the license next to the binary. See the Yocto documentation.

I'll try to sketch a rough plan.

👍

[sixtyfourktec]

Well, looking at this shows half of the recipe is dealing with licenses. So I guess there is no way around in proper support for this. In a yocto build you get a tar file with spdx files for all installed packages. So first of we need to identify the license files itself. So I guess PKG_LICENSE_FILE or similar is one way. It should of course be possible to specify more than one file. Next I would propose some class method which copies this stuff into the .bob/license directory. As usual this method would be called by the cmake/autotools/whatever package step automatically. Others would have to call it by them self. Than we also write some license class helper which will collect all this stuff from the dependencies and do something with it. Either just copied it together in a first step and later maybe also do some spdx stuff. @rhubert: what are you doing with the found license files?

[rhubert]

We put them into a generator generating some fancy code and compile them into our image...

[jkloetzke]

Ralf, do you need to reproduce the exact copyright text for each package? Or would it be OK to have only one generic license text for common licenses like GPL?

[rhubert]

We have a separate copyright field. For generic licenses we're just using the generic license texts.

[jkloetzke]

The following is what I can imagine at the moment. I'll update it as the discussion evolves...

• PKG_LICENSE must always be present
• PKG_LICENSE must be a valid SPDX license expression
  • We could use https://github.com/aboutcode-org/license-expression to validate license expressions
  • Will probably be an offline/CI step
• The PKG_LICENSE must obviously match the content of the package
  • Some source packages have multiple licenses
  • Put the PKG_LICENSE into the multiPackage if it differs between the packages
• A recipe should have a checkoutAssert to verify all licenses to detect changes on upgrades
• PKG_LICENSE is saved in package as .bob/license
  • by "licenses" class
  • Included by default by all autotools/cmake/meson/...
• Custom license handling
  • PKG_LICENSE must have a unique LicenseRef-$ID for each custom license
  • Rules for $ID
    • Must be unique for each license
    • Must conform to SPDX: 1*(ALPHA / DIGIT / "-" / "." )
    • Should be globally unique.
  • PKG_LICENSE_PATH
    • newline separated list of license paths
    • Example: Firmware-Abilis:LICENSE.Abilis.txt
  • Are copied to .bob/licenses/$ID for downstream consumption

At the moment I would leave out any further processing. In the long run we might want to create SPDX documents that also describe relationships. But this seems like overkill currently.

Open TODOs:

• How to handle proprietary packages?