Skip to content

build: replace auto-discovered package lists with explicit definitions#5751

Open
rucoder wants to merge 2 commits intolf-edge:masterfrom
rucoder:rucoder/refactor-pkgs-and-platform
Open

build: replace auto-discovered package lists with explicit definitions#5751
rucoder wants to merge 2 commits intolf-edge:masterfrom
rucoder:rucoder/refactor-pkgs-and-platform

Conversation

@rucoder
Copy link
Copy Markdown
Contributor

@rucoder rucoder commented Apr 6, 2026

Description

NOTE: PR depends on alpine 3.22 update and must be rebased and retested when it is merged

We build some packages that are not part of architecture or platform e.g. u-boot, imx-bsp, etc make no sense for x86

Replace the fragile find pkg | grep -Ev ... auto-discovery of packages with explicit package lists split by category:

  • COMMON_PKGS — packages built for every arch/hv/platform combination- ARCH_PKGS_* — architecture-specific (e.g. u-boot for arm64/riscv64)
  • HV_PKGS_* — hypervisor-specific (e.g. kube for HV=k)
  • PLATFORM_PKGS_* — platform-specific (e.g. nvidia, bsp-imx, optee-os)

This is done in two commits:

  1. Fix get-deps tool to accept an explicit package list as positional arguments instead of unconditionally scanning all of pkg/. The Makefile now passes $(PKGS) to get-deps, and the pkg/kernel no-op workaround
    is removed.

  2. Replace auto-discovery with explicit lists defined in the new mk/packages.mk. The final PKGS is assembled as: pkg/alpine $(COMMON_PKGS) $(ARCH_PKGS_$(ZARCH)) $(HV_PKGS_$(HV)) $(PLATFORM_PKGS_$(PLATFORM)). Moves kernel-version.mk into mk/ directory. Eliminates the hardcoded PKGS_riscv64 override that existed only to work around get-deps.

How to test and validate this PR

  1. Run make pkgs for the default build (amd64/kvm/generic) and verify it builds the same set of packages as before.
  2. Run make HV=k pkgs and verify pkg/kube and pkg/external-boot-image are included.
  3. Run make PLATFORM=nvidia-jp5 ZARCH=arm64 pkgs and verify pkg/nvidia is included.
  4. Verify pkg-deps.mk is generated correctly with make pkg-deps.mk — it should only contain rules for packages in PKGS.

Changelog notes

No user-facing changes.

PR Backports

  • 16.0-stable: No, build-only refactoring.
  • 14.5-stable: No, build-only refactoring.
  • 13.4-stable: No, build-only refactoring.

Checklist

  • I've provided a proper description

  • I've added the proper documentation

  • I've tested my PR on amd64 device

  • I've tested my PR on arm64 device

  • I've written the test verification instructions

  • I've set the proper labels to this PR

  • I've checked the boxes above, or I've provided a good reason why I didn't
    check them.

@rucoder rucoder requested a review from eriknordmark as a code owner April 6, 2026 16:34
@github-actions github-actions Bot requested a review from jsfakian April 6, 2026 16:34
@rucoder rucoder marked this pull request as draft April 6, 2026 16:51
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 6, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 28.34%. Comparing base (2281599) to head (ba4408e).
⚠️ Report is 445 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5751      +/-   ##
==========================================
+ Coverage   19.52%   28.34%   +8.81%     
==========================================
  Files          19       18       -1     
  Lines        3021     2417     -604     
==========================================
+ Hits          590      685      +95     
+ Misses       2310     1588     -722     
- Partials      121      144      +23     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Contributor

@andrewd-zededa andrewd-zededa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rucoder From a quick review the hv=k pkg list looks correct

@rucoder
Copy link
Copy Markdown
Contributor Author

rucoder commented Apr 7, 2026

@rucoder From a quick review the hv=k pkg list looks correct

thank you @andrewd-zededa

@rucoder rucoder force-pushed the rucoder/refactor-pkgs-and-platform branch from ba4408e to bfb2fe8 Compare April 22, 2026 14:12
@rucoder rucoder marked this pull request as ready for review April 22, 2026 14:12
Copy link
Copy Markdown
Contributor

@rene rene left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Full build for riscv64 must be supported first.

@rucoder rucoder self-assigned this Apr 24, 2026
rucoder added 2 commits April 24, 2026 08:26
The get-deps tool unconditionally scanned all directories under ./pkg
to generate dependency rules. This caused issues where packages not
relevant to the current build (e.g. bsp-imx on riscv64) would appear
in pkg-deps.mk, requiring workarounds like the pkg/kernel no-op target.

Now get-deps accepts an optional list of packages as positional arguments.
When provided, only those packages are scanned for dependencies. When
omitted, it falls back to the previous behavior for backward compatibility.

The Makefile now passes $(PKGS) to get-deps, and the pkg/kernel no-op
hack is removed.

Signed-off-by: Mikhail Malyshev <mike.malyshev@gmail.com>
Replace the 'find pkg' auto-discovery of packages with explicit package
lists split by category:

  COMMON_PKGS     - packages built for every build combination
  ARCH_PKGS_*     - architecture-specific (e.g. u-boot for arm64/riscv64)
  HV_PKGS_*       - hypervisor-specific (e.g. kube for HV=k)
  PLATFORM_PKGS_* - platform-specific (e.g. nvidia, bsp-imx, optee-os)

Package lists are defined in the new mk/packages.mk file. The final PKGS
variable is assembled in the Makefile from these components.

This eliminates:
- The fragile 'find pkg | grep -Ev ...' auto-discovery
- The hardcoded PKGS_riscv64 override (which included bsp-imx/optee-os
  only to work around the get-deps tool scanning all packages)
- Implicit coupling between adding a pkg/ directory and having it built

Also moves kernel-version.mk into the mk/ directory and updates all
documentation references accordingly.

Signed-off-by: Mikhail Malyshev <mike.malyshev@gmail.com>
@rucoder rucoder force-pushed the rucoder/refactor-pkgs-and-platform branch from bfb2fe8 to a2b5864 Compare April 24, 2026 08:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants