Version: kirkstone_1-08-00

Add Custom Layer

Setup the repositories

The first step to implement a new custom layer is to create the repositories for the manifest and the actual meta-layer. These repositories are placed into a group with the customer name. Let's consider for instance a new customer called custom-example, the group and the repositories on GitLab should look like:

custom-example
├── manifest
└── meta-custom-example

The second step is to configure the repositories following the guidelines https://git.seco.com/clea-os/infrastructure/gitlab-ci/-/blob/master/docs/add-new-project-to-pipeline.md.

info

Proper credentials are required to perform these steps. Ask to a Clea OS mainteiner if the credentials are not sufficient.

Populate the layer repository

The basic structure of a Seco custom layer is:

meta-seco-custom-example
├── conf
│   ├── bblayers.conf
│   ├── configs/
│   ├── layer.conf
│   ├── machine/
│   ├── SRCREV.conf
│   └── Yconfig/
├── custom_layer
├── .git
│   ├── ...
│   └── ...
├── .gitlab-ci
│   ├── ...
│   └── ...
├── .gitlab-ci.yml
├── .gitmodules
├── README.md
└── recipes-XXX
    ├── recipe-dir-one
    └── recipe-dir-two

Where:

the directory .gitlab-ci is a git submodule containing the CI related processes.
the file .gitlab-ci.yml is the configuration file of the CI submodule.
custom_layer is a file with the name of the layer written into it.

The other directories are described here below with an higher level of details.

Add the `conf/` directory

In a Yocto Project meta-layer, the conf/ directory typically contains configuration files specific to that layer. These files define variables, configuration options, and environment settings used during the build process. Here's an example of what the conf/ directory might look like on Seco's meta-layers:

└── conf
    ├── bblayers.conf
    ├── layer.conf
    ├── SRCREV.conf
    ├── configs
    │   └── seco_<custom-name>_defconfig    
    ├── machine
    │   └── seco-<custom>-<soc>-<board>.conf    
    └── Yconfig
        └── Yconfig_machine

Each of the main files are discussed here.

`bblayers.conf`

This file describe what layers need to be included on top of the layers included by Clea OS. Its purpose is to define the layers that BitBake, the build tool used in Yocto, should scan for recipes and configuration files during the build process. It specifies the paths to the directories containing Yocto layers (meta-layers). BitBake scans these directories to find recipes, configuration files, and other metadata needed for the build. The layer newly create must be added here. The layers to include should be added as follow in the example:

# Custom layer for custom-example
BBLAYERS += "\  
  ${BSPDIR}/layers/meta-qt6 \  
  ${BSPDIR}/layers/meta-openembedded/meta-gnome \
  ${BSPDIR}/layers/meta-openembedded/meta-xfce \
  ${BSPDIR}/layers/meta-browser/meta-chromium \
  ${BSPDIR}/layers/meta-seco-custom-example \
"

`layer.conf`

The layer.conf file is a mandatory configuration file for Yocto layers. Its primary purpose is to provide metadata about the layer, including its name, description, priority, and dependencies on other layers. It plays a crucial role in integrating the layer into the Yocto build system. Some key purposes of the layer.conf file:

Metadata Definition: It defines metadata attributes of the layer, such as its name, version, and description. This information helps users understand the purpose and content of the layer.
Layer Dependencies: It specifies dependencies on other layers using the LAYERSERIES_COMPAT or LAYERDEPENDS variables. This ensures that the required layers are present in the build environment and are compatible with each other.
Layer Priority: It sets the priority of the layer relative to other layers in the build environment. This determines the order in which the layers are processed during the build, which can affect layer inheritance and configuration resolution.
BitBake Configuration: It includes BitBake configuration settings specific to the layer, such as preferred versions of tools and build options.
Integration with Yocto Build System: It integrates the layer into the Yocto build system, allowing BitBake to recognize and process recipes and configuration files within the layer.

These attributes provide essential metadata about the layer and facilitate its integration into the Yocto build system.

An example of the content of this file is shown here below:

# Let bitbake find our .bbclass and .conf files
BBPATH .= ":${LAYERDIR}"

# Let bitbake know where to look for .bb and .bbappend files
BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \
            ${LAYERDIR}/recipes-*/*/*.bbappend "

BBFILES_DYNAMIC += ""


BBFILE_COLLECTIONS += "seco-custom-example"
BBFILE_PATTERN_seco-custom-example := "^${LAYERDIR}/"
BBFILE_PRIORITY_seco-custom-example = "9"

LAYERDEPENDS_seco-custom-example = " \
    seco-imx \
"

# This should only be incremented on significant changes that will
# cause compatibility issues with other layers
LAYERVERSION_seco-custom-example = "1"
LAYERSERIES_COMPAT_seco-custom-example = "kirkstone"

`SRCREV.conf`

Another file needed for a custom layer is SRCREV.conf. This file is very important for custom layers to freeze the BSP version to specific commits. That it's needed to prevent updates on the project without the acceptance of the customer. For example, for NXP iMX projects, the SRCREV.conf may be:

SRCREV:pn-imx-boot = "imx-boot-sha"
SRCREV:pn-imx-mkimage = "imx-mkimage-sha"
SRCREV:pn-u-boot-seco-imx = "u-boot-seco-imx-sha"
SRCREV:pn-linux-seco-imx = "linux-seco-imx-sha"

SRCBRANCH:pn-imx-boot = "imx-boot-branch-name"
SRCBRANCH:pn-imx-mkimage = "imx-mkimage-branch-name"
SRCBRANCH:pn-u-boot-seco-imx = "u-boot-seco-imx-branch-name"
SRCBRANCH:pn-linux-seco-imx = "linux-seco-imx-branch-name"

`configs/`

This directory hosts the defconfig for the custom layer. One defconfig for each custom board is needed. If the customer uses an off-the-shelf board this file is not needed and the defconfig in the public layer should be pointed. When a custom defconfig is added, it can be viewed and selected running seco-setup when setting-up the Yocto environment.

`machine/`

Create a directory in the layer's root directory called machine that will host all the boards machine files. This file contains the Yconfig flag needed to select the custom board. Note that in the local.conf generated by the script seco-setup there will be the MACHINE variable that will look into the machine folder for a file called with the value inserted and into the Yconfig folder for the value inserted in upper case. For example, if in the local.conf there is the variable:

MACHINE ??= 'seco-custom-soc-board'

Yocto will search:

into the Yconfig the string in uppercase MACHINE_SECO_CUSTOM_SOC_BOARD:
info the machine directory the file called seco-custom-soc-board.conf.

`custom_layer`

Finally, the file custom_layer containing the layer's name is required to link the layer with its manifest file, described here. The content of this file is:

seco-custom-example

Add the `gitlab-ci` repo as submodule

The gitlab-ci repository contains GitLab CI scripts for Clea OS infrastructure stasks, e.g. automatic integration of project changes into the manifest. It's meant to be included into all relevant Clea OS repositories as a git submodule.

To add the repository into the new custom meta-custom-example run the command:

git submodules add https://git.seco.com/clea-os/infrastructure/gitlab-ci.git .gitlab-ci

then edit the .gitmodules file to use relative paths:

[submodule ".gitlab-ci"]
    path = .gitlab-ci
    url = ../../../infrastructure/gitlab-ci.git

The CI repository needs a configuration file called .gitlab-ci.yml to be created with:

---
# ---------------------------------------------------------------------------------------
# Include the default CI steps from the gitlab-ci repo
# ---------------------------------------------------------------------------------------

include:
  - project: '${CI_PROJECT_ROOT_NAMESPACE}/infrastructure/gitlab-ci'
    ref: <gitlab-ci-sha>
    file: 'manifest-integration.yml'

variables:
  GITLAB_CI_REVISION: <gitlab-ci-sha>

where <gitlab-ci-sha> is the SHA of gitlab-ci's commit.

For more information on gitlab-ci repository read the README.md.

Populare manifest's repository

Add the `seco-manifest` repository as submodule

The manifest of a custom layer is a sort of an appendix of the main Clea OS manifest called seco-aminfest containing all Seco's off-the-shelf boards. The seco-manifest must be added to the custom manifest repository as a submodule running:

git submodule add https://git.seco.com/clea-os/seco-manifest main

The submodule is stored into a directory called main.

Now create symbolic links to the directories base, distro and soc that are located into the submodule:

ln -s main/base base
ln -s main/distro distro
ln -s main/soc soc

These links are required because the seco-manifest contains some relative paths and this helps not modifying them and maintain it easier.

Add the `gitlab-ci` repository as submodule

To add the submodule follow the same steps described in the previous section.

The configuration of the CI repository needs a configuration file called .gitlab-ci.yml to be created. For a custom project, the manifest's .gitlab-ci.yml requires more variables to be configured compared to the layer's one, such as:

CUSTOM Name of the custom project
PROCESSOR Name of the processor for the custom board, e.g. rk3568
MACHINE Name of the custom machine (used in the SW artifacts naming)
DEFCONFIG Name of the Clea OS Things defconfig file used by seco-setup.sh script, e.g. seco_sbc_c20_clea_os
BUILD_DIR Name of the Clea OS Things build directory created by the seco-setup.sh script, e.g. build_c20
KERNEL_FILE Name of the Kernel file, e.g. Image
UBOOT_FILE Space separated list of U-Boot files, e.g. uboot.img idblock.img
AZURE_PATH Path to save the SW artifacts to in Azure container
CHANGELOG_PROJECTS List of the projects to be considered in the custom changelog.
RETRIGGER_GROUP Path of the custom group in GitLab.
CUSTOM_CONF Lines to be added to the 'conf/local.conf' file just before the Yocto build starts, e.g.

CUSTOM_CONF: |-
  QT_EDITION = "commercial"
  LICENSE_FLAGS_ACCEPTED += " commercial "

As an example, the full .gitlab-ci.yml for the customer custom-example's layer might look like:

    ---
    # ---------------------------------------------------------------------------------------
    # Include the default CI steps from the gitlab-ci repo
    # ---------------------------------------------------------------------------------------
    include:
      - project: '${CI_PROJECT_ROOT_NAMESPACE}/infrastructure/gitlab-ci'
        ref: 6f901acf33b59ce458e6df0cac6c831e60d98bb9
        file: 'manifest-pipeline-yocto.yml'

    # ---------------------------------------------------------------------------------------
    # Define the custom project specific variables
    # ---------------------------------------------------------------------------------------
    variables:
      MANIFEST_PROJECT_ID: 3234
      GITLAB_CI_REVISION: 6f901acf33b59ce458e6df0cac6c831e60d98bb9
      MASTER_BRANCH: kirkstone
      CUSTOM: custom-example
      MACHINES: c57
      PROCESSOR: imx8qxp
      MACHINE: seco-imx8qxp-c57
      DEFCONFIG: seco_custom_example_c57_clea_os_embedded_wayland
      BUILD_DIR: build_custom_example_c57_embedded_wayland
      DISTRO: embedded-base embedded-full
      GRAPHIC_BACKEND: wayland
      KERNEL_FILE: Image
      UBOOT_FILE: imx-boot
      AZURE_PATH: custom/custom-example/kirkstone
      RETRIGGER_GROUP: "${CI_PROJECT_ROOT_NAMESPACE}/custom/custom-example"
      CHANGELOG_PROJECTS: "${RETRIGGER_GROUP}/meta-seco-custom-example"

To uniform projects handling, the file .gitmodules must be edited to use relative paths:

[submodule "main"]
    path = main
    url = ../../../seco-manifest.git
[submodule ".gitlab-ci"]
    path = .gitlab-ci
    url = ../../../infrastructure/gitlab-ci

Add the custom manifest

The project-specific manifest file is called default.xml. This contains the reference (URL and SHA of the commit) to the custom group repositories needed in the Yocto build (typically only the 'meta-seco-custom' repository is present, i.e. the custom layer containing among others the patches needed for the custom BSP to work properly). Moreover, this manifest file imports the default.xml file from the seco-manifest repository. An example of this file, for the customer custom-example is:

<?xml version="1.0" encoding="UTF-8"?>

<manifest>

    <include name="main/default.xml" />
    <remote name="seco-custom" alias="origin" fetch="https://git.seco.com/clea-os/custom/custom-example" />

    <project name="meta-seco-custom-example" revision="bd114dac99082f5b93d76871b2844adbe3c5db4b" remote="seco-custom" path="layers/meta-seco-custom-example">
    	<linkfile src="custom_layer" dest="custom_layer" />
    </project>

</manifest>

Setup the repositories​

Populate the layer repository​

Add the conf/ directory​

bblayers.conf​

layer.conf​

SRCREV.conf​

configs/​

machine/​

custom_layer​

Add the gitlab-ci repo as submodule​

Populare manifest's repository​

Add the seco-manifest repository as submodule​

Add the gitlab-ci repository as submodule​

Add the custom manifest​