[userModule] Network Presence

Tips, Tricks and Scripts to enhance your home automation and workaround known device bugs, limitations and incompatibilities
Eagle
Posts: 69
Joined: 18 Jun 2017 19:04

Re: [userModule] Network Presence

Post by Eagle » 26 Feb 2018 18:29

Hi,

yes, the system is only lagging, when i use ping and i track only two ips (2 phones). i also tried to restart zway and reboot the NAS.
I used the standard values; Interval 1 minute, Timeout 2 seconds, Count 1, no hysteresis.

For Netcat im also using the default values; Interval 1 minute, Timeout 10 seconds, port 80, no hysteresis.

Output for the ssh command is:

root@MyCloudPR4100 root # time netcat -v -z -w 600 192.168.1.6 80
android-d5a76c3aeee3230e [192.168.1.6] 80 (http): Connection refused
Command exited with non-zero status 1
real 0m 0.39s
user 0m 0.00s
sys 0m 0.00s

Me and my phone were absent for the last 9 hours, but the network presence element did not change.

Thank you for your help!

Provo
Posts: 112
Joined: 19 Oct 2016 19:54

Re: [userModule] Network Presence

Post by Provo » 26 Feb 2018 21:11

Eagle wrote:
26 Feb 2018 18:29
yes, the system is only lagging, when i use ping and i track only two ips (2 phones). i also tried to restart zway and reboot the NAS.
That is very strange. There is no difference in the ping command used by this module and the one used by the other Ping Presence module. That seems to point to the problem being in the surrounding code, but the fact that it doesn't happen with netcat points to the command itself.. It's hard to know where to go from here, as I haven't seen this behavior myself. But could you try increasing the ping interval to something like five minutes, and first see if the laggy system then only appears every five minutes instead? Then you could try to manually trigger the ping by clicking the clock or time on the virtual device – this should trigger a polling of that specific IP – and see if the system becomes immediately laggy.

Also, if you don't mind, please check if you have this laggy behavior with only one tracked device as well.

Looking at the code, I did in fact spot a bug that prevents the proper removal of the cron task on reinitialization or removing of the module. I made the creation of it instance specific to prevent different instances triggering each other, but forgot to do the same with its removal. With plenty of reinitializations, this could produce a large amount of pings, and could in principle cause some lag (although it would probably have to be pretty extreme to be noticable). But it should go away with a complete restart of Z-Way. I'll fix the bug as soon as I find the time, but if you experienced a laggy system even after a restart, and not with netcat, this is almost certainly not the cause of your problem.
Eagle wrote:
26 Feb 2018 18:29
Output for the ssh command is:

root@MyCloudPR4100 root # time netcat -v -z -w 600 192.168.1.6 80
android-d5a76c3aeee3230e [192.168.1.6] 80 (http): Connection refused
Command exited with non-zero status 1
real 0m 0.39s
user 0m 0.00s
sys 0m 0.00s
I forgot to say that you should do this when your device is not connected to the network. I'm assuming the device with IP 192.168.1.6 was connected to the network when you ran that command, or something very weird is going on.

Eagle
Posts: 69
Joined: 18 Jun 2017 19:04

Re: [userModule] Network Presence

Post by Eagle » 26 Feb 2018 23:05

I noticed several things while trying the ping method with !default! settings:

Lag seems to happen on every full minute (xx:xx:00), after the 2 to 30 seconds the system is full responsive again.
When both IPs are connected there is NO lag!! (maybe thats the case with the other app too, but i only tested it when i was at home and my phone was connected)
Same behaviour while only one IP is being tracked, connected => no lag, unconnected => lag


I tried the netcat command again, while my phone was not connected:

root@MyCloudPR4100 root # time netcat -v -z -w 600 192.168.1.6 80
192.168.1.6 80 (http): No route to host
Command exited with non-zero status 1
real 0m 3.03s
user 0m 0.00s
sys 0m 0.00s

other try:

root@MyCloudPR4100 root # time netcat -v -z -w 600 192.168.1.6 80
android-d5a76c3aeee3230e [192.168.1.6] 80 (http): No route to host
Command exited with non-zero status 1
real 0m 18.04s
user 0m 0.00s
sys 0m 0.00s

Provo
Posts: 112
Joined: 19 Oct 2016 19:54

Re: [userModule] Network Presence

Post by Provo » 27 Feb 2018 13:32

Eagle wrote:
26 Feb 2018 23:05
I noticed several things while trying the ping method with !default! settings:

Lag seems to happen on every full minute (xx:xx:00), after the 2 to 30 seconds the system is full responsive again.
When both IPs are connected there is NO lag!! (maybe thats the case with the other app too, but i only tested it when i was at home and my phone was connected)
Same behaviour while only one IP is being tracked, connected => no lag, unconnected => lag
It would be great if you could try the other app with your phone off the network as well, to see if that also produces lag. Try increasing the timeout to see if that causes the lag to last longer.

When you're present, the ping finishes very quickly, in just a few milliseconds. When you're not present, the ping command will take as long as you have set your timeout. That could at least be part of the explanation of the difference.

I have a growing suspicion here. If somehow the internal threading of Z-Way, at least on WD, makes it so that the syscommands are run on the same thread as other important parts of the core, you may experience lag while the syscommands are executed. The reason it might take 2 seconds, is because you have 2 seconds timeout. The reason you might see 30, is that the cron jobs are piling up (because of the mentioned bug) so every time you are reconfiguring the module, it will add another trigger. This means that you would see 2 seconds after a fresh Z-Way restart, but more and more as you keep reconfiguring or removing and readding the app instance. If that's not what you're experiencing, or if the other Ping app does not cause lag even when your phone is absent, this hypothesis might well be wrong. Another thing pointing to it being wrong, is that you say you don't have this problem when using netcat.
Eagle wrote:
26 Feb 2018 23:05
I tried the netcat command again, while my phone was not connected:
Ok, it looks like netcat is working like expected, but takes a bit long sometimes. You could try to increase the netcat timeout to 30 seconds or so, and see if it works better.

Eagle
Posts: 69
Joined: 18 Jun 2017 19:04

Re: [userModule] Network Presence

Post by Eagle » 28 Feb 2018 20:03

Youre right, i restarted zway to clear cron jobs, added a new Ping Presence State App with my IP being absent and a timout of 30 seconds.

=> The Lag was exactly 30 seconds long
=> The Ping Presence State App shows the same behavior as your app


For Netcat i tried timouts of 30 and 40 seconds, but the ui element wont change to off

Provo
Posts: 112
Joined: 19 Oct 2016 19:54

Re: [userModule] Network Presence

Post by Provo » 02 Mar 2018 08:52

Eagle wrote:
28 Feb 2018 20:03
Youre right, i restarted zway to clear cron jobs, added a new Ping Presence State App with my IP being absent and a timout of 30 seconds.

=> The Lag was exactly 30 seconds long
=> The Ping Presence State App shows the same behavior as your app
Ok, without knowing for sure, that points towards this being caused by how Z-Way internally does threading. To my knowledge there isn't anything I can do on the app side to prevent this. The polling is initiated as a cron job, and the polling of each device is in turn done asynchronously and in parallel, and I don't think I can "fork" it any more than that. But I haven't seen (or heard about) this behavior in Z-Way before, so maybe it's specific to the WD version, if there is any version difference. I suggest that you create an issue on their github page, as you are able to reproduce this issue easily.
Eagle wrote:
28 Feb 2018 20:03
For Netcat i tried timouts of 30 and 40 seconds, but the ui element wont change to off
This whole situation leads me to think that Netcat for some reason exits immediately. If it didn't, you would probably experience the same lag as with ping. When I get the time, hopefully this weekend, I will add some optional extra logging. I'll tell you when it's done, and you can download it directly from gitlab. Hopefully, the log will show what's wrong, and I can implement a fix and include it in the next version.

Provo
Posts: 112
Joined: 19 Oct 2016 19:54

Re: [userModule] Network Presence

Post by Provo » 04 Mar 2018 15:01

I have just now pushed a fix for the cron job bug and an added feature of optional logging. But I would like to include a fix for your Netcat issues in the same version upgrade if possible, so I haven't uploaded this to the app store yet. Try to uninstall your current version and install the latest version from GitLab instead, either by traditional downloading or cloning with git, and let me know what the log says when the app tries to poll your device(s) using Netcat. You need to activate the logging under Advanced config when instantiating the app.

All relevant log entries will have "[NetworkPresence]" in their prefix, but the log entry with the ping or netcat command output will be several lines, so take that into consideration if grepping.

If you need any help, let me know.

Eagle
Posts: 69
Joined: 18 Jun 2017 19:04

Re: [userModule] Network Presence

Post by Eagle » 12 Mar 2018 13:29

Provo wrote:
04 Mar 2018 15:01
I have just now pushed a fix for the cron job bug and an added feature of optional logging. But I would like to include a fix for your Netcat issues in the same version upgrade if possible, so I haven't uploaded this to the app store yet. Try to uninstall your current version and install the latest version from GitLab instead, either by traditional downloading or cloning with git, and let me know what the log says when the app tries to poll your device(s) using Netcat. You need to activate the logging under Advanced config when instantiating the app.

All relevant log entries will have "[NetworkPresence]" in their prefix, but the log entry with the ping or netcat command output will be several lines, so take that into consideration if grepping.

If you need any help, let me know.
Sorry for my late reply but i was not at home. I tried to install the new version by cloning it with git into UserModules. This created a folder named zway-network-presence. Before i uninstalled NetworkPresence in zway, but there is no new app showing up? Then i installed Network Presence again (AppStore) and tried to exchange the (new) files from zway-network-presence with the old files in NetworkPresence. The app still says version 0.4 and i might have done something wrong?

Thanks for your work (and patience)

Provo
Posts: 112
Joined: 19 Oct 2016 19:54

Re: [userModule] Network Presence

Post by Provo » 12 Mar 2018 14:38

Eagle wrote:
12 Mar 2018 13:29
Sorry for my late reply but i was not at home. I tried to install the new version by cloning it with git into UserModules. This created a folder named zway-network-presence. Before i uninstalled NetworkPresence in zway, but there is no new app showing up? Then i installed Network Presence again (AppStore) and tried to exchange the (new) files from zway-network-presence with the old files in NetworkPresence. The app still says version 0.4 and i might have done something wrong?
The folder needs to be named the same as the class, that's why it didn't work the first time. They way you did it the second time should work, but it might be saying version 0.4 if you got it before i pushed the version update.

I didn't know if you would reply, so I bumped the version to 0.5 and intended to upload the new version, but the upload mechanism failed for some reason so it's not there yet. If you replace your files with the new ones in the gitlab repo, or delete it locally and do another git clone in UserModules, but add the folder name "NetworkPresence" at the end of the command, like below, it should say version 0.5 now. Although you have to reinitialize the module through the API or restart ZWay for it to load the new files.

git clone https://gitlab.com/kvolden/zway-network-presence.git NetworkPresence

Eagle
Posts: 69
Joined: 18 Jun 2017 19:04

Re: [userModule] Network Presence

Post by Eagle » 12 Mar 2018 15:10

Reinitializing did the trick! Version 0.5 is installed now but where do i find the logfile?

Post Reply