Skip to content

Fix metadata retries blocking provider instance updates - #16522

Open
hehuang139 wants to merge 2 commits into
apache:3.3from
hehuang139:codex/fix-16149-metadata-events
Open

hehuang139 wants to merge 2 commits into
apache:3.3from
hehuang139:codex/fix-16149-metadata-events

Conversation

@hehuang139

@hehuang139 hehuang139 commented Oct 19, 2026 •

Copy link
Copy Markdown

What is the purpose of the change?

Fixes #16149.

A metadata retry holds the instance listener monitor while fetching metadata from providers. If that request stalls, registry callbacks cannot update the provider list and subsequent retries continue using obsolete instances.

Update instance lists under a short synchronized section and let one refresh processor retrieve metadata from a snapshot. Notifications received during retrieval update the latest snapshot and return; obsolete results are discarded before address publication. Pending changes are also processed after a retrieval failure. Destroyed listeners reject late results and release queued instance events, including when destruction races with the next refresh iteration.

Validation on Linux with JDK 17 and Maven 3.9.9:

  • Reproduced the issue before the fix: a provider-change callback timed out while a retry metadata fetch was paused.
  • Passed 39 tests covering concurrent provider changes, obsolete snapshots with multiple revisions, repeated notification failures, event acceptance, destruction, existing listeners with both empty-protection settings, and service discovery registry behavior.
  • Passed module spotless:check and git diff --check.

The test command was:

mvn -B -ntp -pl dubbo-registry/dubbo-registry-api -am test jacoco:report -Dtest=ServiceInstancesChangedListenerConcurrencyTest,ServiceInstancesChangedListenerTest,ServiceInstancesChangedListenerWithoutEmptyProtectTest,ServiceDiscoveryRegistryTest -Dsurefire.failIfNoSpecifiedTests=false -DfailIfNoTests=false -Pjacoco,skip-spotless -Dcheckstyle.skip=true -Drat.skip=true

Checklist

  • Make sure there is a GitHub_issue field for the change.
  • Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
  • Write necessary unit-test to verify your logic correction.
  • Make sure GitHub Actions can pass.

@codecov-commenter

codecov-commenter commented Oct 19, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.74074% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 60.94%. Comparing base (b73b9d1) to head (1db131f).

Files with missing lines Patch % Lines
...vent/listener/ServiceInstancesChangedListener.java 90.74% 2 Missing and 3 partials ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##                3.3   #16522      +/-   ##
============================================
+ Coverage     60.89%   60.94%   +0.05%     
- Complexity    11753    11770      +17     
============================================
  Files          1953     1953              
  Lines         89278    89324      +46     
  Branches      13475    13485      +10     
============================================
+ Hits          54367    54440      +73     
+ Misses        29329    29305      -24     
+ Partials       5582     5579       -3     
Flag Coverage Δ
integration-tests-java21 32.16% <64.81%> (+0.08%) ⬆️
integration-tests-java8 32.22% <64.81%> (+0.03%) ⬆️
samples-tests-java21 32.14% <55.55%> (+<0.01%) ⬆️
samples-tests-java8 29.75% <55.55%> (+<0.01%) ⬆️
unit-tests-java11 59.18% <90.74%> (+0.05%) ⬆️
unit-tests-java17 ?
unit-tests-java21 58.67% <90.74%> (+0.06%) ⬆️
unit-tests-java25 58.62% <90.74%> (+0.05%) ⬆️
unit-tests-java8 59.20% <90.74%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@hehuang139

Copy link
Copy Markdown
Author

Added follow-up coverage in 1db131f for repeated notification failures, obsolete multi-revision snapshots, ignored events, and destruction during refresh. Reviewing destruction also found a queued event that stayed referenced after the listener was destroyed. The listener now releases that event and safely stops before taking another snapshot.

The focused listener/registry run passes all 39 tests on Linux with JDK 17. Both Spotless and the diff checks pass. The five lines previously reported as missing by Codecov are exercised by the new tests; CI will validate the updated commit.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] a dead-lock case in ServiceInstanceChangeListener will cause consumer can`t use latest provider instances list

2 participants