Skip to content

testing.md

Latest commit

 

History

History
313 lines (225 loc) · 10 KB

testing.md

File metadata and controls

313 lines (225 loc) · 10 KB
Raw

Testing

This document explains the test layout and what each group protects.

For the external API behavior these tests help protect, see External API assumptions.

Test types

The project uses:

The Catch2 tests are the fastest place to check backend and client behavior.

Test dependencies

The test build needs:

  • pkgconfig(gio-2.0)
  • pkgconfig(catch2-with-main)

On Fedora, catch-devel provides pkgconfig(catch2-with-main).

Catch2 tests

Key files:

These tests protect:

  • package search and merge behavior
  • Latest only and all-version package query behavior
  • config file parsing and fallback behavior
  • installed snapshot behavior
  • pending transaction action row selection for install, upgrade, downgrade, remove, and reinstall
  • package table CSV export formatting
  • package table column text and sorting behavior
  • dnf5daemon transaction preview parsing, upgrade-target parsing, and failure handling
  • daemon upgrade snapshot ownership, stale-state invalidation, and current-target validation
  • transaction history row formatting, action labels, and page cursor behavior
  • transaction request validation
  • offline and cached metadata behavior

Daemon smoke tests

The app now talks to DNF5 dnf5daemon for privileged transactions. Use Docker and native Fedora testing to verify upgrade-target listing, preview, apply, cancel, failure, and session cleanup behavior.

Common commands

Run the native test suite:

make test

Run the offline cached metadata smoke tests separately:

make dockerofflinetest

Run the dnf5daemon transaction client tests in Docker:

make dockerdnf5daemontest

Run the dnf5daemon transaction client tests on native Fedora:

make dnf5daemontest

The native dnf5daemon tests may install and remove cowsay unless DNFUI_TEST_DNF5DAEMON_INSTALL_SPEC is set to another package. The script restores whether the test package was installed before the run. The optional downgrade preview test runs when DNFUI_TEST_DNF5DAEMON_DOWNGRADE_SPEC names a safe installed package version that can be downgraded in the test environment. Native apply tests are skipped by default because they require dnf5daemon authorization. Run them explicitly with:

make dnf5daemonapplytest

On a native Fedora test machine, installing dnf5daemon-server-polkit lets the apply tests run without a desktop authorization prompt.

Without that package, run make dnf5daemonapplytest from the desktop session where a Polkit authentication agent can show the authorization dialog. Running the apply target over SSH may fail with Not authorized.

Native repository key prompt test

Use a disposable Fedora VM or test machine. Do not use your main system for this test.

Create the local signed test repo:

utils/setup_gpg_key_prompt_test_repo.sh setup

Run DNF UI from the branch:

make run

Search for:

dnfui-gpg-test-package

The key prompt can appear while preparing the preview or while applying the transaction. Both paths must ask before trusting the key.

Reject test:

  • Click Reject in the DNF UI key prompt.
  • The transaction should fail.
  • The package should not be installed.
  • utils/setup_gpg_key_prompt_test_repo.sh status should not list an imported test key.

Close test:

  • Start the same transaction in DNF UI again.
  • Close the DNF UI key prompt without trusting the key.
  • The package should not be installed.
  • utils/setup_gpg_key_prompt_test_repo.sh status should not list an imported test key.

The Apply progress window is not user-cancellable. Cancellation after UI approval but before daemon confirmation is covered by a unit test.

Run utils/setup_gpg_key_prompt_test_repo.sh restore before the accept test.

Accept test:

  • Run utils/setup_gpg_key_prompt_test_repo.sh setup.
  • Start the same transaction in DNF UI again.
  • Click Trust Key in the DNF UI key prompt.
  • The transaction should continue.
  • utils/setup_gpg_key_prompt_test_repo.sh status should show the package and imported test key.

Restore the disposable test machine:

utils/setup_gpg_key_prompt_test_repo.sh restore

Run the Docker app target with networking disabled:

DOCKER_NETWORK_MODE=none make dockerrun

Run the Docker app target with the Swedish translation:

LANG=sv_SE.UTF-8 LANGUAGE=sv make dockerrun
CONTAINER_RUNTIME=podman LANG=sv_SE.UTF-8 LANGUAGE=sv make dockerrun

The Makefile is a task runner. Meson owns build configuration and test definitions.

Memory checks

Run a quick smoke test under Valgrind Memcheck:

make memcheck

Run the desktop app under Valgrind Memcheck:

make memcheck-app

Memcheck logs are written under build/memcheck/.

The default Memcheck setup fails on definite and indirect leaks from this project. Reachable and possible leak noise from GLib, GTK, DNF, and related libraries is suppressed in utils/valgrind-dnfui.supp.

Useful options:

MEMCHECK_SMOKE_FILTER="Transaction request validation rejects an empty request" make memcheck
MEMCHECK_SMOKE_TIMEOUT=5m make memcheck
MEMCHECK_SMOKE_TIMEOUT=10m make memcheck

Package table memory regression check

Use this before releases that change package table storage, package query workers, all-version listing, or List Upgradable row handling:

  1. Start DNF UI and record process memory after the first window is idle:
pid="$(pgrep -n dnfui)"
grep -E 'Rss|Pss|Private' "/proc/$pid/smaps_rollup"
  1. Run List Packages ten times, waiting for each run to finish before recording memory again.
  2. Repeat with List Installed, a broad Search, and List Upgradable.
  3. Repeat Search and List Packages with Latest only disabled.
  4. Alternate Latest only and all-version searches to verify all-version rows do not accumulate outside the visible table.
  5. Alternate List Upgradable and List Packages a few times to verify old daemon-backed rows are released.
  6. A high but stable value is acceptable. Continued substantial growth after every completed run is a release blocker.
  7. Run make memcheck-app once if memory keeps growing or if table row ownership changed.

Docker notes

  • make dockerrun starts a system bus in the container and uses dnf5daemon
  • make dockerdnf5daemontest runs upgrade-target listing, preview, downgrade preview when configured, apply, remove, reinstall, and failure checks against dnf5daemon
  • The dnf5daemon failure checks include applying a prepared session after another session changed package state. The stale prepared session must fail and remain releasable.
  • Use native Fedora to test the real desktop Polkit prompt

Fedora review checks

Fedora package review expects rpmlint output for the source RPM and all binary RPMs produced by the build. It is also normal to check that the source RPM builds in mock.

Build RPMs with the Make targets and run rpmlint:

make srpm
make rpm
rpmlint dnf-ui-latest.src.rpm rpmbuild/RPMS/*/*.rpm

Keep the rpmlint output so it can be included in the Fedora package review.

Build the source RPM in a clean Fedora build environment:

mock -r fedora-rawhide-x86_64 --rebuild dnf-ui-latest.src.rpm

Use the Fedora release and architecture that match the package review you are preparing. Running mock may require the user to be a member of the mock group.

Reference: https://docs.fedoraproject.org/en-US/packaging-guidelines/ReviewGuidelines/

What to test after changes

For documentation-only changes, run git diff --check.

For comments inside C++ source files, also run a target that compiles the changed files.

For backend query changes, run the relevant Catch2 tests.

For transaction changes, test the dnf5daemon Docker path and then verify apply behavior on native Fedora with a real desktop Polkit prompt.

For package apply behavior, use test packages that are safe to install and remove in the test environment.

Multi-version package browsing checks

Use these checks after changes to Latest only, package queries, package table display, or pending action identity:

  1. Start with Latest only enabled.
  2. Run Search and List Packages and verify the compact one-row-per-package view.
  3. Disable Latest only and run the same query again.
  4. Verify that distinct exact NEVRAs can appear as separate rows.
  5. Verify that duplicate repository copies of the same exact NEVRA are hidden.
  6. Verify that older repository versions show Older in repository.
  7. Verify that intermediate newer versions can be selected for Details but cannot be marked for upgrade.
  8. Verify that the newest available version remains the only normal upgrade target.
  9. Verify that downgradeable rows mark a pending downgrade using the selected exact NEVRA.
  10. Verify that removing or reinstalling one exact installed NEVRA does not change pending actions for another installed NEVRA with the same name and architecture.
  11. Verify that List Installed, List Upgradable, Upgrade All, and Mark Listed Upgrades do not depend on the Latest only checkbox.