Skip to content

feat: implement MQTT wildcard subscription matching - #6906

Open
wy471x wants to merge 10 commits into
apache:masterfrom
wy471x:feat_Wildcard-subscription-matching-not-implemented
Open

feat: implement MQTT wildcard subscription matching#6906
wy471x wants to merge 10 commits into
apache:masterfrom
wy471x:feat_Wildcard-subscription-matching-not-implemented

Conversation

@wy471x

@wy471x wy471x commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Make sure that:

  • You have read the contribution guidelines.
  • You submit test cases (unit or integration tests) that back your changes.
  • Your local test passed ./mvnw clean install -Dmaven.javadoc.skip=true.

Summary:

Problem

Subscriptions using + (single-level wildcard) or # (multi-level wildcard) were silently broken. Publish.send() performed an exact-key lookup (ConcurrentHashMap.getOrDefault), so a publish to sensor/room1/temperature would never match a subscription filter like sensor/+/temperature.

Changes

  1. New: TopicMatcher.java — Utility class implementing MQTT topic filter matching per the MQTT-4.7 spec:
    - + matches exactly one topic level
    - # matches any number of levels (must appear at the end of the filter)
    - Wildcards at the first level do not match $-prefixed topics
  2. Modified: SubscribeRepository.java — Added getChannelsByTopic(String topic) method that iterates over all stored subscription filters and returns channels whose filter matches
    the published topic using TopicMatcher.matches().
  3. Modified: Publish.java:116 — Changed send() from get(topic) (exact-key lookup) to getChannelsByTopic(topic) (wildcard-aware matching).
  4. New: TopicMatcherTest.java — 6 unit tests covering: exact match, + single-level, # multi-level, mixed wildcards, $ topic protection, and null inputs.

close #6851

Aias00
Aias00 previously approved these changes Aug 14, 2026

@Aias00 Aias00 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: #6906 — feat: implement MQTT wildcard subscription matching

Verdict: ✅ Approve (with two follow-up suggestions, one a real correctness edge case)

This fills a genuinely broken feature — wildcard subscriptions silently never matched because Publish.send did an exact Map.get. The fix is well-structured.

What's correct

  • TopicMatcher is spec-accurate. I traced the algorithm across the test matrix and beyond:
    • + matches exactly one level (sport/+/player1sport/tennis/stadium/player1, ✓ sport/football/player1);
    • # matches any number of levels incl. the parent (sport/#sport);
    • # only matches when it's its own level (sport# / sport/tennis# correctly rejected);
    • $-prefixed topics are not matched by a leading wildcard (#/+ → false) but are matched by explicit $SYS/# / $SYS/+ — exactly MQTT-4.7.2-1;
    • null inputs return false (no NPE).
  • The get() exact-lookup method is retained and still used by add()/remove() internally, so this isn't introducing dead code — good call keeping it.
  • getChannelsByTopic is a clean O(N) scan that delegates entirely to TopicMatcher; no logic duplicated.
  • TopicMatcherTest is thorough — exact, single-level, multi-level, mixed, $-topic, and null cases all covered.

Suggestions (non-blocking)

  1. Duplicate delivery on overlapping subscriptions (real, please track as a follow-up). getChannelsByTopic does result.addAll(entry.getValue()) over every matching filter. If one client holds two overlapping subscriptions (e.g. sport/# and #), it appears under both keys, so a publish to sport/x adds the same Channel twice → the client receives the message twice. MQTT requires at most one delivery per publish per client. Collect into a Set<Channel> (or LinkedHashSet if you care about order/stability) before returning to avoid this.
  2. Performance fast-path. Every publish now scans all subscriptions. For the very common case where the topic has an exact (non-wildcard) subscriber, you could result.addAll(get(topic)) first (O(1) exact hit) and then only scan filters containing +/#. Not necessary for correctness, just a scale consideration.
  3. Minor: invalid filters (e.g. # not as its own level, or trailing text after #) silently return false here. Optionally reject malformed filters at subscription time in Subscribe.add so bad subscriptions fail fast instead of silently never matching.

Verdict

Approving. The core matching logic is correct and well-tested, and get() is correctly preserved. Suggestion #1 (dedupe) is worth a quick follow-up PR before wildcard support sees production traffic with multi-subscription clients.

@wy471x

wy471x commented Aug 14, 2026

Copy link
Copy Markdown
Contributor Author

Review: #6906 — feat: implement MQTT wildcard subscription matching

Verdict: ✅ Approve (with two follow-up suggestions, one a real correctness edge case)

This fills a genuinely broken feature — wildcard subscriptions silently never matched because Publish.send did an exact Map.get. The fix is well-structured.

What's correct

  • TopicMatcher is spec-accurate. I traced the algorithm across the test matrix and beyond:

    • + matches exactly one level (sport/+/player1sport/tennis/stadium/player1, ✓ sport/football/player1);
    • # matches any number of levels incl. the parent (sport/#sport);
    • # only matches when it's its own level (sport# / sport/tennis# correctly rejected);
    • $-prefixed topics are not matched by a leading wildcard (#/+ → false) but are matched by explicit $SYS/# / $SYS/+ — exactly MQTT-4.7.2-1;
    • null inputs return false (no NPE).
  • The get() exact-lookup method is retained and still used by add()/remove() internally, so this isn't introducing dead code — good call keeping it.

  • getChannelsByTopic is a clean O(N) scan that delegates entirely to TopicMatcher; no logic duplicated.

  • TopicMatcherTest is thorough — exact, single-level, multi-level, mixed, $-topic, and null cases all covered.

Suggestions (non-blocking)

  1. Duplicate delivery on overlapping subscriptions (real, please track as a follow-up). getChannelsByTopic does result.addAll(entry.getValue()) over every matching filter. If one client holds two overlapping subscriptions (e.g. sport/# and #), it appears under both keys, so a publish to sport/x adds the same Channel twice → the client receives the message twice. MQTT requires at most one delivery per publish per client. Collect into a Set<Channel> (or LinkedHashSet if you care about order/stability) before returning to avoid this.
  2. Performance fast-path. Every publish now scans all subscriptions. For the very common case where the topic has an exact (non-wildcard) subscriber, you could result.addAll(get(topic)) first (O(1) exact hit) and then only scan filters containing +/#. Not necessary for correctness, just a scale consideration.
  3. Minor: invalid filters (e.g. # not as its own level, or trailing text after #) silently return false here. Optionally reject malformed filters at subscription time in Subscribe.add so bad subscriptions fail fast instead of silently never matching.

Verdict

Approving. The core matching logic is correct and well-tested, and get() is correctly preserved. Suggestion #1 (dedupe) is worth a quick follow-up PR before wildcard support sees production traffic with multi-subscription clients.

Thank you for the code review on this PR.

Fixes for the three review comments:

  1. Duplicate delivery on overlapping subscriptions — SubscribeRepository.getChannelsByTopic now collects channels into a LinkedHashSet before returning, so a client holding
    overlapping filters (e.g. sport/# and #) receives at most one delivery per publish, per MQTT spec.
  2. Performance fast-path — exact topic subscribers are added via an O(1) map lookup first; the wildcard scan then skips filters without +/#.
  3. Malformed filters fail fast — added TopicMatcher.isValidFilter (MQTT-4.7.1 rules); Subscribe registers only valid filters and sends SUBACK return code 0x80 (FAILURE) for
    invalid ones.

Tests added: TopicMatcherTest validation cases, new SubscribeRepositoryTest (dedup/fast-path), new SubscribeTest (rejection + SUBACK codes).

wy471x added a commit to wy471x/shenyu that referenced this pull request Aug 15, 2026
Publish.publishWill used an exact SubscribeRepository.get() lookup, so
clients subscribed to wildcard filters (e.g. status/#) never received
wills published to concrete topics like status/client-001. Port the
TopicMatcher and SubscribeRepository.getChannelsByTopic from apache#6906 and
route will delivery through it, consistent with normal publish routing.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Wildcard subscription matching not implemented — +/-# subscriptions never receive messages

2 participants