I've a number of Qubino Mini Dimmers, support S2 security, in my new home setup.
I've been running this setup since November 2019, and have had intermittent random failure of some of these Qubino S2 devices.
Now I didn't have the perfect reception setup antenna wise yet, so I figured it would improve when I would have a better setup.
Now that 3.06 z-way has stabilised my system, and the antenna reception is also greatly improved by adding a u.fl and external antenna, I still have the random failure of some of the Mini Dimmers.
It's hard for me to pin down when this occurs exactly, it can be fixed by cutting power to the Qubino Module. However I'd like to understand what causes this "Is Failed" state.
Is it the razberry or the qubino device having an issue, and how should I go about debugging this?
Thing worth noting:
I don't have any other S2 devices except the Qubino Mini Dimmers.
It's not the same module failing all the time, It seems random a bit
Although it seems like the modules that tend to be furthest away from the controller suffer this issue the most.
Also, but I don't have enough empirical data, it's just a hunch, this tends to happen when my raspberrypi is busy / having a high CPU load.
By analysing the logs, it looks like the razberry is waiting for a reply on the "S2 Nonce Get" command, but never gets it when the module is on "Is Failed" state.
When I trigger an interview when the module is in failed state, and explicitly ask SecurityS2 interview, you'll see the request command being scheduled again:
[2020-06-12 08:45:08.510] [D] [zway] SETDATA devices.26.instances.0.commandClasses.159.data.grantedKeys.S0 = False
[2020-06-12 08:45:08.511] [D] [zway] SETDATA devices.26.instances.0.commandClasses.159.data.grantedKeys.S2Unauthenticated = False
[2020-06-12 08:45:08.513] [D] [zway] SETDATA devices.26.instances.0.commandClasses.159.data.grantedKeys.S2Authenticated = True
[2020-06-12 08:45:08.515] [D] [zway] SETDATA devices.26.instances.0.commandClasses.159.data.grantedKeys.S2Access = False
[2020-06-12 08:45:08.516] [D] [zway] SETDATA devices.26.instances.0.commandClasses.159.data.security = True
[2020-06-12 08:45:08.518] [D] [zway] SETDATA devices.26.instances.0.commandClasses.159.data.secureNodeInfoFrames.S2Authenticated = byte[0]
[2020-06-12 08:45:08.520] [D] [zway] ( zero-length buffer )
[2020-06-12 08:45:08.521] [zway] Node 26:0 CC SecurityS2: requesting secure NIF for security class S2 Authenticated
[2020-06-12 08:45:08.523] [zway] Using security scheme S2 Authenticated
[2020-06-12 08:45:08.524] [zway] Adding job: Secure S2 NIF Get
[2020-06-12 08:45:08.526] [zway] Job 0x13 (Security S2 Nonce Get): Cancelling job: Security S2 Nonce Get
[2020-06-12 08:45:08.527] [zway] Removing job: Security S2 Nonce Get
[2020-06-12 08:45:08.527] [zway] Adding job: Security S2 Nonce Get
[2020-06-12 08:45:08.527] [D] [zway] Security S2 send timer set to 100.000000
[2020-06-12 08:45:13.758] [D] [zway] Job 0x3b: deleted from queue
followed by
[2020-06-12 08:46:56.057] [D] [zway] Security S2 send timeout event
[2020-06-12 08:46:56.059] [zway] Job 0x13 (Secure S2 NIF Get): Security S2 failed to prepare S2 packet
[2020-06-12 08:46:56.071] [zway] Job 0x13 (Security S2 Nonce Get): Cancelling job: Security S2 Nonce Get
[2020-06-12 08:46:56.072] [zway] Removing job: Security S2 Nonce Get
[2020-06-12 08:46:56.074] [zway] Adding job: Security S2 Nonce Get
[2020-06-12 08:46:56.076] [D] [zway] Security S2 send timer set to 100.000000
Controller info:
Firmware
Library Type: Static Controller
SDK Version: 6.81.01
Serial API Version: 05.37
Software Information
Version number: v3.0.6
Compile-ID: 82a4fd59993dc610c06507351714f95f21475262
Compile-Date: 2020-04-28 13:52:20 +0300
UI
UI version 1.5.1
Built date 03-04-2020 13:45:40
S2 Qubino Mini dimmers randomly go to "Is Failed"
Re: S2 Qubino Mini dimmers randomly go to "Is Failed"
Looks like unreachable device. May be it hangs?
Few things to check:
1. Does the device send something in this failed state when you click in it? Do you see anything from it in the log?
2. Are neighboring devices reachable from the controller? Try to update routes to devices in the vicinity of the failed one and then update routes to the failed one.
This will help to check if it is a device issue or routing issue
Few things to check:
1. Does the device send something in this failed state when you click in it? Do you see anything from it in the log?
2. Are neighboring devices reachable from the controller? Try to update routes to devices in the vicinity of the failed one and then update routes to the failed one.
This will help to check if it is a device issue or routing issue
Re: S2 Qubino Mini dimmers randomly go to "Is Failed"
I had to wait a bit for one of the devices to go into failed state again, currently have 2 in failed state.
I checked the logs for any incoming packets to the controller when hitting the switch button (turning lights on / off or dimming), no incoming packets what so ever, although when you check the "timing info" in expert interface, it shows about 4 to 6 (Depending on the device) of packets that were received, not OK though, and with a ~400 * 10ms packets delay.
Might be an indication as to the routing not being optimal?
I've tried restarting the z-way controller software, and tried a re-organization (the latter should update routes to all devices right? also the ones in vicinity)
I tried to create a manual route to one of the failed devices, however looks like that's not an option when the device is failed?
One of the failed devices should be able to directly communicate with the controller.
Both devices are the same type Mini Dimmers from Qubino with the S2 security specs.
I checked the logs for any incoming packets to the controller when hitting the switch button (turning lights on / off or dimming), no incoming packets what so ever, although when you check the "timing info" in expert interface, it shows about 4 to 6 (Depending on the device) of packets that were received, not OK though, and with a ~400 * 10ms packets delay.
- Screenshot 2020-06-26 at 08.39.34.png (174.29 KiB) Viewed 4970 times
I've tried restarting the z-way controller software, and tried a re-organization (the latter should update routes to all devices right? also the ones in vicinity)
I tried to create a manual route to one of the failed devices, however looks like that's not an option when the device is failed?
One of the failed devices should be able to directly communicate with the controller.
Both devices are the same type Mini Dimmers from Qubino with the S2 security specs.
- Screenshot 2020-06-26 at 08.43.06.png (98.73 KiB) Viewed 4970 times
Re: S2 Qubino Mini dimmers randomly go to "Is Failed"
Timing info shows sent packets. Red means undelivered, so your device never got it.
Looks like a device issue. If it does not answer, you can not do anything. Please contact Qubino support and show them logs and screenshots. They are familiar with Z-Way
Looks like a device issue. If it does not answer, you can not do anything. Please contact Qubino support and show them logs and screenshots. They are familiar with Z-Way
Re: S2 Qubino Mini dimmers randomly go to "Is Failed"
Update on the issue, Qubino has been able to reproduce the issue, and one of their developers is looking into it, full issue description below:
Qubino Support
11:15 AM (10 hours ago)
to me
Hello Cor,
We've managed to replicate the issue by including three Mini Dimmers, in S2unauth, in ZwaveMe and toggling the inputs. After some attempts, the first Mini Dimmer was marked as failed. After playing around with all three for some more time, all three were marked as failed.
Our current understanding is, that this happens due to the routed packets (range) and due to the packets being encrypted (secure inclusion). With secure inclusion, the device and the gateway have to exchange additional packets (Nonce Get/Report) before the actual encrypted application packets is sent. Each of the packet also has to be acknowledged. It appears that is some cases, some of the packets are not acknowledged, so the gateway resend the mentioned packets multiple times - first directly, then to all the other known neighbours of the node (it tries to send these packets three times, via each node). We currently believe that these retries result in the Mini Dimmer being blocked.
A bug ticket was opened for this issue, so one of the developers will have a more detailed look at it, to confirm that this is indeed the issue and to see, what could be done about it. This is a functionality, that is handled by the zwave sdk. Currently, we can only recommend, that you try to include your devices in unsecure mode, to see if that will resolve the issue.
We thank you for your patience and your help.
Qubino Support
11:15 AM (10 hours ago)
to me
Hello Cor,
We've managed to replicate the issue by including three Mini Dimmers, in S2unauth, in ZwaveMe and toggling the inputs. After some attempts, the first Mini Dimmer was marked as failed. After playing around with all three for some more time, all three were marked as failed.
Our current understanding is, that this happens due to the routed packets (range) and due to the packets being encrypted (secure inclusion). With secure inclusion, the device and the gateway have to exchange additional packets (Nonce Get/Report) before the actual encrypted application packets is sent. Each of the packet also has to be acknowledged. It appears that is some cases, some of the packets are not acknowledged, so the gateway resend the mentioned packets multiple times - first directly, then to all the other known neighbours of the node (it tries to send these packets three times, via each node). We currently believe that these retries result in the Mini Dimmer being blocked.
A bug ticket was opened for this issue, so one of the developers will have a more detailed look at it, to confirm that this is indeed the issue and to see, what could be done about it. This is a functionality, that is handled by the zwave sdk. Currently, we can only recommend, that you try to include your devices in unsecure mode, to see if that will resolve the issue.
We thank you for your patience and your help.
Re: S2 Qubino Mini dimmers randomly go to "Is Failed"
Did you get any further updates from Qubino about this?
I have 42 Mini Dimmers in my network, all added with S2-Authenticated. The nodes randomly fail in the same way as for you, both those with firmware v20.2 and v21.6. I'm reluctant to re-including all of them without encryption, since adding them without SmartStart was a pain. Upgrading the firmware on them would be much worse though, considering the fact that OTA isn't supported.
I have 42 Mini Dimmers in my network, all added with S2-Authenticated. The nodes randomly fail in the same way as for you, both those with firmware v20.2 and v21.6. I'm reluctant to re-including all of them without encryption, since adding them without SmartStart was a pain. Upgrading the firmware on them would be much worse though, considering the fact that OTA isn't supported.
Re: S2 Qubino Mini dimmers randomly go to "Is Failed"
Hi Martinen,
I have had no further update from Qubino. I have been checking their support website to see if a newer firmware would be released, but so far there seem to be no updates. Though if I read your post correctly, you have mini dimmers with two different firmware versions? Would be interesting to know what the release notes are of those / what they say they have fixed.
I have to occasionally power cycle my zwave mini dimmers (once every two to three weeks) when they become unresponsive. I have 2 networks with over 7 qubino mini-dimmers and in both networks 2 are acting up. They're also well in range, so I suspect multiple bugs in the Qubino mini dimmers are present. One issue related to not being in full signal range and another related to multiple switch on-off resulting in dimmer becoming unresponsive (this is one connected to a fibaro motion sensor, stops functioning between 2 tot 4 weeks of use).
In my network non of the other devices (Aeotec and Fibaro) behave this way, some of them are even further away in terms of range.
I had hoped zway-700 pro chip upgrade in my RaspberryPi ZWAY gateway would solve some of the issues, however that's not the case. The issues still remain. It might help if you would raise a case with Qubino support. the more people having this issue and complaining to them might raise the priority. Especially since you're having over 42 dimmers.
I have to say, it is always the same ones acting up over here, have you detected a pattern in the failure?
I have had no further update from Qubino. I have been checking their support website to see if a newer firmware would be released, but so far there seem to be no updates. Though if I read your post correctly, you have mini dimmers with two different firmware versions? Would be interesting to know what the release notes are of those / what they say they have fixed.
I have to occasionally power cycle my zwave mini dimmers (once every two to three weeks) when they become unresponsive. I have 2 networks with over 7 qubino mini-dimmers and in both networks 2 are acting up. They're also well in range, so I suspect multiple bugs in the Qubino mini dimmers are present. One issue related to not being in full signal range and another related to multiple switch on-off resulting in dimmer becoming unresponsive (this is one connected to a fibaro motion sensor, stops functioning between 2 tot 4 weeks of use).
In my network non of the other devices (Aeotec and Fibaro) behave this way, some of them are even further away in terms of range.
I had hoped zway-700 pro chip upgrade in my RaspberryPi ZWAY gateway would solve some of the issues, however that's not the case. The issues still remain. It might help if you would raise a case with Qubino support. the more people having this issue and complaining to them might raise the priority. Especially since you're having over 42 dimmers.
I have to say, it is always the same ones acting up over here, have you detected a pattern in the failure?
Re: S2 Qubino Mini dimmers randomly go to "Is Failed"
Does Qubino ever publish firmware files or release notes? I can't find anything even for my 3-Phase Smart Meter that supposedly supports OTA upgrades. I wish I had seen this before buying all those dimmers:
I'm actually not using Z-Way, but that probably doesn't matter since the controller receives absolutely nothing from a dimmer once it has become unresponsive. The software I'm using is Zwavejs2Mqtt and Home Assistant. I started with an Aeotec Z-Stick Gen5 and switched to a Z-Stick 7 with the same result.
I have some good news though. By giving up S2-Authenticated and re-including all the dimmers without encryption, I got rid of the unresponsive states completely. The unprotected communication isn't optimal, but it's good enough. Creating a support case with Qubino is a bit hard at this point since I no longer can recreate the problem.
There's another issue with the dimmers that I have a passable solution for. It's quite common that reports from them don't properly reach the controller, so the states of the corresponding lights in Home Assistant is not always correct. Using the polling feature of Zwavejs2Mqtt floods the network completely, since all 42 dimmers are polled simultaneously. I decided to use the Power Reporting Interval feature of the dimmers and set them to different prime numbers in order to spread the reports as even as possible. After some tweaking, I ended up with 709 - 997 seconds which means incorrect power values are usually corrected withing 16 minutes. I wrote some pyscript code that polls the dimming value of any device that has a discrepancy between that and the power value. It's far from perfect, but the Mini Dimmers are finally useful in Home Assistant.
They have to come up with a revolutionary new firmware if I'm to remove them all from the walls and send them to Slovenia. I haven't seen any difference in functionality between the dimmers with firmware v21.6 and those with v20.2. Both are unfortunately as prone to fail when added with S2-Authenticated. I haven't been able to find a pattern in which of the dimmers fail, but it usually ends up with about six failed before I power cycle them.Mini Dimmer doesn’t support over-the-air (OTA) software updates. Updates can be done through a custom tool and software. We can do the update for free; you only need to cover the shipping costs. In some cases, our official distributors offer the option to do it.
I'm actually not using Z-Way, but that probably doesn't matter since the controller receives absolutely nothing from a dimmer once it has become unresponsive. The software I'm using is Zwavejs2Mqtt and Home Assistant. I started with an Aeotec Z-Stick Gen5 and switched to a Z-Stick 7 with the same result.
I have some good news though. By giving up S2-Authenticated and re-including all the dimmers without encryption, I got rid of the unresponsive states completely. The unprotected communication isn't optimal, but it's good enough. Creating a support case with Qubino is a bit hard at this point since I no longer can recreate the problem.
There's another issue with the dimmers that I have a passable solution for. It's quite common that reports from them don't properly reach the controller, so the states of the corresponding lights in Home Assistant is not always correct. Using the polling feature of Zwavejs2Mqtt floods the network completely, since all 42 dimmers are polled simultaneously. I decided to use the Power Reporting Interval feature of the dimmers and set them to different prime numbers in order to spread the reports as even as possible. After some tweaking, I ended up with 709 - 997 seconds which means incorrect power values are usually corrected withing 16 minutes. I wrote some pyscript code that polls the dimming value of any device that has a discrepancy between that and the power value. It's far from perfect, but the Mini Dimmers are finally useful in Home Assistant.
Re: S2 Qubino Mini dimmers randomly go to "Is Failed"
They do not release firmware updates and do not support OTA. Even if they decided to support OTA, the current firmware doesn't support it. This is one of the big drawbacks to their products.
Re: S2 Qubino Mini dimmers randomly go to "Is Failed"
Hopefully in Z-Wave Plus v2 it is mandatory