Page 1 of 4
V4.1.2: steadily increasing cpu load
Posted: 15 Dec 2023 15:34
by piet66
One week ago, I upgraded from 3.2.3 to 4.1.2.
Unfortunately, I noticed that the CPU load is constantly increasing, just like in 4.1.0.
- cpuK.png (61.22 KiB) Viewed 21052 times
CPU load of z-way-server in 3.2.3: 5.9%, in 4.1.2 at the moment:38.5%.
In my opinion it's caused by zway/core, which takes in 3.2.3 about 45 percent of the z-way-server cpu, and in 4.1.2 about 87 percent.
CPU load in 4.1.2:
Code: Select all
COMMAND %CPU TIME SECONDS PERCENT
z-way-server 38.5 3224:29 193469 100.00
*******
z-way-server 0.0 0:18 18 0.01
OptimizingCompi 0.0 0:16 16 0.01
v8:SweeperThrea 0.0 0:49 49 0.03
v8:SweeperThrea 0.0 0:35 35 0.02
v8:SweeperThrea 0.0 0:21 21 0.01
v8:SweeperThrea 0.0 0:10 10 0.01
zway/core 33.5 2811:38 168698 87.20 !!!!!!!!!!!!!
zway/webserver 0.7 61:13 3673 1.90
zway/timers 0.2 18:07 1087 0.56
zway/dev/ttyAMA 3.8 325:50 19550 10.10
zway/sockets 0.0 0:59 59 0.03
zway/http/async 0.0 0:00 0 0
For comparison in 3.2.3:
Code: Select all
COMMAND %CPU TIME SECONDS PERCENT
z-way-server 5.9 215:09 12909 100.00
*******
z-way-server 0.0 0:08 8 0.06
OptimizingCompi 0.0 0:12 12 0.09
v8:SweeperThrea 0.0 0:20 20 0.15
v8:SweeperThrea 0.0 0:12 12 0.09
v8:SweeperThrea 0.0 0:06 6 0.05
v8:SweeperThrea 0.0 0:03 3 0.02
zway/core 2.6 97:34 5854 45.35 !!!!!!!!!!!!!!
zway/webserver 0.5 21:08 1268 9.82
zway/timers 0.2 7:14 434 3.36
zway/dev/ttyAMA 2.3 86:33 5193 40.23
zway/sockets 0.0 0:26 26 0.20
I will revert to 3.2.3.
Re: V4.1.2: steadily increasing cpu load
Posted: 15 Dec 2023 20:39
by seattleneil
Sigh, increasing CPU load in the zway/core thread has been an ongoing problem. Although my having the same problem as you does nothing to fix the problem, I'm hoping that having multiple users reporting this problem will cause @PoltoS to pay more attention to the problem and provide a fix, as he did with the 4.1.0 job queue problem.
I tend to think the solution will involve running a multi-thread profiling tool such as gperftools. Several users have made educated guesses about the cause such as the http device app, but nothing conclusive.
My fingers are crossed that a solution will be provided. If not, we may need to consider other Z-Wave platforms. This post looks interesting:
Code: Select all
https://www.home-assistant.io/blog/2023/12/12/silicon-labs-official-partnership-nabu-casa/
Re: V4.1.2: steadily increasing cpu load
Posted: 18 Dec 2023 21:33
by michap
piet66 wrote: ↑15 Dec 2023 15:34
Unfortunately, I noticed that the CPU load is constantly increasing, just like in 4.1.0.
Hi,
what tool you have used for this (CPU load table)? (for compare the values)
My htop does not show such high usage - but maybe really depend on the used modules...
Michael
Re: V4.1.2: steadily increasing cpu load
Posted: 18 Dec 2023 23:06
by aLiEnHeAd
Same problem here.
Version 4.1.2 - load of 1.0
z-way-server process uses 100% CPU
Code: Select all
top - 21:02:54 up 2 days, 1:37, 2 users, load average: 1.08, 1.01, 1.00
Tasks: 121 total, 1 running, 120 sleeping, 0 stopped, 0 zombie
%Cpu(s): 26.1 us, 0.0 sy, 0.0 ni, 73.9 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
MiB Mem : 922.1 total, 553.2 free, 155.7 used, 213.2 buff/cache
MiB Swap: 100.0 total, 100.0 free, 0.0 used. 712.0 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
427 root 20 0 246508 117340 21612 S 100.0 12.4 2058:11 z-way-server
I suspect a new HTTP-Device as the cause (Polling the motion status of my reolink camera)
Re: V4.1.2: steadily increasing cpu load
Posted: 19 Dec 2023 12:28
by piet66
michap wrote: ↑18 Dec 2023 21:33
piet66 wrote: ↑15 Dec 2023 15:34
Unfortunately, I noticed that the CPU load is constantly increasing, just like in 4.1.0.
Hi,
what tool you have used for this (CPU load table)? (for compare the values)
My htop does not show such high usage - but maybe really depend on the used modules...
Michael
My own tool, MxBash V2.0.0
. You can download it from ZWay store. I've just uploaded it.
I'm not sure that the zway team is able to verify it speedy. Till that you have to add the token 'MxBash' to your Smarthome Settings/App Store Access to see it.
Re: V4.1.2: steadily increasing cpu load
Posted: 19 Dec 2023 19:01
by seattleneil
You can also use the top command to see similar info, as follows:
Code: Select all
$ top -b -n 1 -H -p `pidof z-way-server`
top - 17:11:34 up 36 days, 5:27, 1 user, load average: 0.13, 0.11, 0.09
Threads: 10 total, 0 running, 10 sleeping, 0 stopped, 0 zombie
%Cpu(s): 3.2 us, 0.0 sy, 0.0 ni, 96.8 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
MiB Mem : 1848.2 total, 649.6 free, 190.6 used, 1008.0 buff/cache
MiB Swap: 100.0 total, 100.0 free, 0.0 used. 1499.1 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
24456 root 20 0 242868 100904 21640 S 6.7 5.3 267:09.56 zway/core
24450 root 20 0 242868 100904 21640 S 0.0 5.3 0:15.83 z-way-server
24451 root 20 0 242868 100904 21640 S 0.0 5.3 0:00.76 OptimizingCompi
24452 root 20 0 242868 100904 21640 S 0.0 5.3 0:00.81 v8:SweeperThrea
24453 root 20 0 242868 100904 21640 S 0.0 5.3 0:00.55 v8:SweeperThrea
24454 root 20 0 242868 100904 21640 S 0.0 5.3 0:00.40 v8:SweeperThrea
24455 root 20 0 242868 100904 21640 S 0.0 5.3 0:00.22 v8:SweeperThrea
24457 root 20 0 242868 100904 21640 S 0.0 5.3 31:39.69 zway/webserver
24458 root 20 0 242868 100904 21640 S 0.0 5.3 11:41.00 zway/timers
24459 root 20 0 242868 100904 21640 S 0.0 5.3 69:12.23 zway/dev/ttyAMA
A useful experiment would be for @aLiEnHeAd to disable the reolink http device and then restart z-way-server. Is the zway/core CPU load much lower after running 24 hours? If so, this information should help the Z-Way developers isolate the problem.
On a tangent, my sense is that doing an http get every minute within Z-Way's javascript engine is asking for trouble since the Z-Way software architecture does not isolate a poorly-behaved javascript app from a well-behaved javascript app. For example, a long-running javascript app will impact the entire core thread. A safer approach would be to create a shell script that does the polling and when motion is detected, the shell script pushes the alert to a way http sensor device (i.e., move the polling logic outside of z-way and use a shell script that calls the curl command in a shell script loop). The benefit of Z-Way's software architecture is that it can run more efficiently and with less RAM than more complicated home automation platforms such as Home Assistant.
Re: V4.1.2: steadily increasing cpu load
Posted: 19 Dec 2023 19:40
by aLiEnHeAd
seattleneil wrote: ↑19 Dec 2023 19:01
A useful experiment would be for @aLiEnHeAd to disable the reolink http device and then restart z-way-server. Is the zway/core CPU load much lower after running 24 hours? If so, this information should help the Z-Way developers isolate the problem.
[...]A safer approach would be to create a shell script [...]
I deactivated the device yesterday and the load is 0.01 with z-way-server using 2-3% CPU.
So at least in my case it solves the problem. I'll try creating a shell script for my task, thanks for your idea.
Re: V4.1.2: steadily increasing cpu load
Posted: 20 Dec 2023 04:10
by seattleneil
Eureka! Based on the information provided by @aLiEnHeAd, I replaced the HTTP devices with instances of the app called "Virtual Device (JavaScript)". As a result, the z-way-server process on my Raspberry Pi has a CPU load of 4% instead of 12%.
Here's what I did:
1. Added the following line to the file /opt/z-way-server/automation: sh /home/pi/cgi-bin/tasmota_switch.sh
2. Added a file called "tasmota_switch.sh" in directory /home/pi/cgi-bin with the following contents:
Code: Select all
#!/usr/bin/bash
ERR_MSG="ERROR - need 2 arguments: LR_Xmas_Tree|Xmas_Outside_Lights|Railing_Xmas_Light on|off|stat"
case $1 in
LR_Xmas_Tree)
IP=tasmota-1329.lan
;;
Xmas_Outside_Lights)
IP=tasmota-1141.lan
;;
Railing_Xmas_Light)
IP=tasmota-7253.lan
;;
*)
echo "$ERR_MSG"
exit
;;
esac
case $2 in
on)
wget -t 2 -T 5 --connect-timeout=5 -w 2 --no-http-keep-alive --max-redirect=0 -q -O /dev/null "http://$IP/cm?cmnd=Power%20On"
echo -n "on"
;;
off)
wget -t 2 -T 5 --connect-timeout=5 -w 2 --no-http-keep-alive --max-redirect=0 -q -O /dev/null "http://$IP/cm?cmnd=Power%20Off"
echo -n "off"
;;
stat)
if `wget -t 2 -T 5 --connect-timeout=5 -w 2 --no-http-keep-alive --max-redirect=0 -q -O - "http://$IP/cm?cmnd=Power" | grep -q OFF`; then echo -n off; else echo -n on; fi
;;
*)
echo "$ERR_MSG"
exit
;;
esac
I used wget instead of curl since wget consumes less resources. I also set various wget command options to limit the amount of time the HTTP GET took and to close the TCP connection.
3. Created 3 devices using the "Virtual Device (JavaScript)" app. Here's a screenshot for 1 of the devices:
- Virtual Device (JavaScript).png (47.07 KiB) Viewed 19318 times
4. Deleted the corresponding HTTP device.
In addition to binary switches, I created a similar shell script for multilevel sensors and multilevel switches. Does anyone sees a downside to this approach?
Re: V4.1.2: steadily increasing cpu load
Posted: 20 Dec 2023 11:51
by piet66
Now I have a question of understanding: do you think, that the http.request() command is buggy?
I'm also using this command in my apps. And that could explain my problem too.
But then the bug would have to be new in ZWay release 4, as I don't have this problem in release 3.
Can anyone confirm that the issue is new?
Re: V4.1.2: steadily increasing cpu load
Posted: 20 Dec 2023 13:43
by rowlockterms
piet66 wrote: ↑20 Dec 2023 11:51
Now I have a question of understanding: do you think, that the http.request() command is buggy?
I'm also using this command in my apps. And that could explain my problem too.
But then the bug would have to be new in ZWay release 4, as I don't have this problem in release 3.
Can anyone confirm that the issue is new?
Experiencing the same CPU load increase with http.request() in V4.1.2. Never encountered this in release 3. Anyone else facing this new bug?