BLE peripheral/controller ownership

[ljodal]

Hey, I'm playing around with a sorta-kinda hub style device where I want to communicate with downstream sensors over BLE while the hub is responsible for the matter layer. If I've not misunderstood things too much this package current takes full control over the BLE peripheral, even though it's only used during setup.

Are you open for a PR that changes this interface so full ownership of the peripheral is not required?

I've not looked too closely yet, but I did point Claude at the issue and it made a fairly large patch that I wasn't particularly happy about. If you're open to a change like this I'll take a deeper look and see if I can find a good way without changing too much.

[ivmarkov]

Hey, I'm playing around with a sorta-kinda hub style device where I want to communicate with downstream sensors over BLE while the hub is responsible for the matter layer. If I've not misunderstood things too much this package current takes full control over the BLE peripheral, even though it's only used during setup.

Kinda, but see (2) below. BTW by "hub" I guess you meant Matter Bridge, right? As in, bridge some BLE sensors or similar devices as Matter sensors? Using the Aggregator cluster handler and friends?

I'm open to making rs-matter-embassy more flexible, but I think we should have some sort of initial PR so that we have something concrete to discuss.

Some background.
rs-matter-embassy is really about two separate things:

1. Orchestration a.k.a. the "stack" (XXXMatterStack). The "stack" is just saving code-lines for the user by taking opinionated decisions on his behalf. For example, the stack polls the BLE and the operational transport via a single giant future. The stack wants to bring up / tear down the BLE & operational networks whenever it sees fit. Whether you would like to re-use the stack for your hub use case is unclear to me, hence the request for some sort of initial PR. Also, you might not even want this, see (2)

2. Drivers for the BTP/BLE Matter transport (nimble and trouble), for the IP-over-Thread transport (openthread), for IP-over-Wifi (embassy-net + a bunch of HAL-specific Wifi controllers) and for IP-over-ethernet (embassy-net). Now, the drivers are or should be usable regardless whether you buy into the whole "stack" notion or not. In other words, you might want to re-use the trouble GATT Peripheral code, but not use the "stack" notion and arrange everything yourself, just like the examples in upstream rs-matter do.

My feeling is that we should start making (2) more flexible, and only then see if we can also make (1) more flexible. So again, having some sort of "skeleton" initial PR so that I gen an idea as to what you are trying to achieve might be a good start, even if the PR is LLM-generated, and contains mostly hand-waving.

Perhaps, you can direct Claude at looking at TroubleBtpGattPeripheral first and see what it can do there.
Currently, this type "owns" the whole trouble host which is instantiated on top of the BLE controller. This might or might not be a problem for you.

• It won't be a problem, if you are OK NOT to poll / deal with your BLE devices until after your Matter accessory is fully commissioned. If that's the case, then the change would be minimal, if any, because you can already now share the BLE Controller between the rs-matter-embassy stack and your own code. It is just that you might have to instantiate your own trouble Host only after you get a signal that your Matter accessory is fully commissioned, as I'm not sure how the BLE controller and trouble would react if there are two hosts instantiated over a single controller (yours and the rs-matter-embassy one). Might work actually, but I have not tried this
• Now, if you really need to deal with your BLE devices while the Matter accessory is not commissioned yet, we need to carve out of TroubleBtpGattPeripheral a subset where it takes a Peripheral (i.e. you construct the trouble Server). And even that assumes you need only the "central" role when you talk to your devices. If you also need the "peripheral" role, it becomes even more complicated. In fact, so complicated that if you are in that scenario (you need your own GATT services sitting next to the Matter one), you might be better off just implementing your own custom TroubleBtpGattPeripheral which is dealing with both the Matter GATT services and your own ones.

[ljodal]

Thanks for the detailed reply!

To quickly detail my idea/use case: I have hydronic underfloor heating with separate zones for each room. The water flow to each zone is controlled with electronically actuated valves in one central place in the house. My plan is to have a RPi Pico 2W based hub/controller here that does the switching of these valves, which exposes itself as a Matter Bridge with a thermostat cluster for each room. The auxiliary BLE communication is with battery powered temperature sensors in each room. Since the central controller is the one actually making the heating decisions this feels like the right architecture to me, as the sensors in each room will not know if the heat is active or not at the moment.

My plan was to pair and set up the room sensors first, but I guess I can look into doing that after pairing with Matter. I might also be that I should just bypass the "stack" part here and use the underlying primitives directly. I'll have a look and see if I find a good way to solve this, as well as point Claude in the direction you suggest and see what it finds!

[ivmarkov]

which exposes itself as a Matter Bridge

I think you only need to deal with the notion of a Matter Bridge (which involves a Bridge device type and the Aggregator cluster) only if you plan to have multiple endpoints (each endpoint is a different device - a room sensor, or a new heating zone/valve actuator) where endpoints can come and go dynamically without restarting your hub.

Given that you have fixed zones, fixed rooms and likely fixed (or very slowly changing) number of BLE sensors, you might get-by with a static set of endpoints then at least initially, and not deal with the complexities of the matter bridge at all. Changing the number of endpoints in your static setup would still be possible, but you'll have to update your firmware and then decommission and recommission your device.

Once you want to commercialize your solution, and need to adapt it to any unknown in advance number of zones and sensors - sure. But if that's just for your home, perhaps take the easiest route and just hard-code the endpoints.

My plan was to pair and set up the room sensors first, but I guess I can look into doing that after pairing with Matter. I might also be that I should just bypass the "stack" part here and use the underlying primitives directly. I'll have a look and see if I find a good way to solve this, as well as point Claude in the direction you suggest and see what it finds!

Are you sure that your temp sensors need any pairing? The simplest of them report their temperature out there in the open by just using BLE ads. So you'll just have to listen to the ads.

[ivmarkov]

BTW, if you can attach Ethernet to your device (say, via a simplistic W5500 controller) you can save yourself a lot of trouble by just implementing your device over Ethernet in which case it won't need BLE (nor Wifi, for that matter). And then, you are free to use the BLE for your own purposes.

[ivmarkov]

One more thing: if you have a large property, you might need to also worry whether you'll be able to "hear" your BLE devices from that single Pico 2W. BLE is not a mesh by default... you might have a better luck by just modeling your temp sensors each with a separate Pico 2W which directly has the temp sensor attached with wires to it and speaks Matter natively over BLE + Wifi or Ethernet...

[ljodal]

I think you only need to deal with the notion of a Matter Bridge (which involves a Bridge device type and the Aggregator cluster) only if you plan to have multiple endpoints (each endpoint is a different device - a room sensor, or a new heating zone/valve actuator) where endpoints can come and go dynamically without restarting your hub.

Given that you have fixed zones, fixed rooms and likely fixed (or very slowly changing) number of BLE sensors, you might get-by with a static set of endpoints then at least initially, and not deal with the complexities of the matter bridge at all. Changing the number of endpoints in your static setup would still be possible, but you'll have to update your firmware and then decommission and recommission your device.

I haven't looked too closely at it, but I had Claude throw together a dummy example and was able to add it to my apple home and see the four thermostats I wanted as a proof of concept: https://github.com/ljodal/rs-matter-embassy/blob/ljodal/rp235x-thermostat/examples/rp/src/bin/thermostat_wifi.rs Guess I can ask it to try without the bridge type and see if I'm still allowed to assign each one to a separate room in Apple home

The hardware as I've designed it right now can handle up to six zones, but I only have four hooked up at the moment. I can do it statically at build-time at the moment, but I want it dynamic eventually. I still need some kind of pairing setup with the sensors, so I figured that might make sense as the step that exposes it to matter.

Once you want to commercialize your solution, and need to adapt it to any unknown in advance number of zones and sensors - sure. But if that's just for your home, perhaps take the easiest route and just hard-code the endpoints.

Not sure if this will be commercialized, but I will at least make both the hardware and software open and it would be nice if it had at least some flexibility.

Are you sure that your temp sensors need any pairing? The simplest of them report their temperature out there in the open by just using BLE ads. So you'll just have to listen to the ads.

I'm planning on showing and adjusting the setpoint on the sensors, so I need two-way sync in case of remote updates. After some experimentation and power usage calculations a slow persistent BLE connections seemed like the best option (that bit I've already tested a bit).

BTW, if you can attach Ethernet to your device (say, via a simplistic W5500 controller) you can save yourself a lot of trouble by just implementing your device over Ethernet in which case it won't need BLE (nor Wifi, for that matter). And then, you are free to use the BLE for your own purposes.

This is my current work-around, I'm just hard-coding my wifi credentials at build-time, so I can commission over "ethernet". I don't have wired network to where the heat pump is unfortunately (it's an old house, so no easy solution either), so was hoping to get by with Wi-Fi.

One more thing: if you have a large property, you might need to also worry whether you'll be able to "hear" your BLE devices from that single Pico 2W. BLE is not a mesh by default... you might have a better luck by just modeling your temp sensors each with a separate Pico 2W which directly has the temp sensor attached with wires to it and speaks Matter natively over BLE + Wifi or Ethernet...

My sensors are battery powered and very compact (where all of this mess started, I couldn't find any wireless thermostats I'm happy with that actually works). I'm using nRF54L15's as the sensors, and they go very low power while maintaining a connection. Still need to test it in-situ, but I don't think I need mesh for my specific use case since all the rooms are fairly close. Will see 🤞

[ivmarkov]

My sensors are battery powered and very compact (where all of this mess started, I couldn't find any wireless thermostats I'm happy with that actually works). I'm using nRF54L15's as the sensors, and they go very low power while maintaining a connection. Still need to test it in-situ, but I don't think I need mesh for my specific use case since all the rooms are fairly close. Will see 🤞

Heh. Since recently, rs-matter-embassy does work on the NRF54L15 with Matter + BLE + Thread, but the ICD (sleepy end devices) - if you want these battery powered - is not thoroughfully tested yet. This topic was recently covered in the rs-matter Matrix room so there could be other users looking into that as well.

So that's another alternative to your custom BLE-based sensors and the need to proxy these. Also IKEA sells the TIMMERFLOTTE sensors, which work over Thread and are battery powered (but haven't tested them myself).

EDIT: Unlike BLE+GATT, Thread is a mesh automatically, and theoretically a better alternative to Wifi for anything but IOT devices that want high bandwidth, like Matter cameras.

[ivmarkov]

Anyway, if you would like to pursue your original setup, ask Claude to assess and eventually implement something along the lines of item (2) from my original/first comment.

The thermostat_wifi.rs thing looks decent, Claude got all the patterns right.
Just keep in mind once #75 lands, you'll have to ask it to re-work the wiring of your cluster handlers.

[ljodal]

The thought of using thread has crossed my mind, but then I'd need to either replace the Pico with something capable of thread (or the communication has to go through my home hub maybe?). I did research it a while back but dropped it because of the lack of sleepy end device support when I started looking at this.

I'm much more familiar with BLE as that's part of my day-to-day work, which is why I'm going down that route to begin with. Thread is an interesting approach though.

I actually have a couple of the timmerflotte sensors, so I guess I could theoretically try to pair one with something I write on a thread capable dev board and see. If the room sensors are just normal thread/matter devices that might actually be a nice and flexible solution!

Nothing in my current sensor hardware binds me to BLE, and for now I'm just gonna use the nRF54L14-tag devkits with a 3D printed case or something like that.

Lots of potential!