job queue runs full after upgrade to v4.1.0

Official announcements from Z-Wave.Me team. Check this forum to get information about new releases and bug fixes.
piet66
Posts: 268
Joined: 04 Feb 2017 17:00

Re: job queue runs full after upgrade to v4.1.0

Post by piet66 »

If you are looking for a more convenient method to supervise the CPU load, I can promote my Linux script collection 'MxBash'.
If you want to have a try, download it from Z-Way store. For to see it, you have to add the token 'MxBash' to smarthome/settings/App Store Access.
Raspberry Pi 3 Model B Rev 1.2
Raspbian GNU/Linux 10 (buster, 32bit)
RaZberry by Z-Wave.Me ZW0700 7.20.00 07.38/1766938484 1025/257
Z-Way version v4.1.2 from 2023-10-18 03:34:26 +0300
piet66
Posts: 268
Joined: 04 Feb 2017 17:00

Re: job queue runs full after upgrade to v4.1.0

Post by piet66 »

seattleneil wrote:
01 Aug 2023 00:24
Since no one else on this forum has reported the z-way-server process consuming anything close to 65% %CPU,
I think we must distinguish between 2 different kinds of CPU values:
top displays the current CPU consumption, which may be very high for short moments. That's not a problem. ps computes the average consumption over a certain interval. This should be relatively constant. In my system about 7%.
Raspberry Pi 3 Model B Rev 1.2
Raspbian GNU/Linux 10 (buster, 32bit)
RaZberry by Z-Wave.Me ZW0700 7.20.00 07.38/1766938484 1025/257
Z-Way version v4.1.2 from 2023-10-18 03:34:26 +0300
AlesKO
Posts: 87
Joined: 24 Nov 2016 09:58

Re: job queue runs full after upgrade to v4.1.0

Post by AlesKO »

ss.jpg
ss.jpg (3.13 KiB) Viewed 16908 times
Just returned from one week vacations

installed v4.1.0-27-g589c9812 version.
harre
Posts: 95
Joined: 24 Nov 2020 02:22

Re: job queue runs full after upgrade to v4.1.0

Post by harre »

4.1.1 solved my issues with long queues that have been bothering me for a while.
feuersee
Posts: 34
Joined: 07 Jan 2016 01:53

Re: job queue runs full after upgrade to v4.1.0

Post by feuersee »

running the commands: ps -L -p `pidof z-way-server`; ps -F -p `pidof z-way-server`
Screenshot 2023-08-05 084844.gif
Screenshot 2023-08-05 084844.gif (33.52 KiB) Viewed 16877 times
feuersee
Posts: 34
Joined: 07 Jan 2016 01:53

Re: job queue runs full after upgrade to v4.1.0

Post by feuersee »

sorry, second screenshot for ps -F -p `pidof z-way-server`:
Attachments
Screenshot 2023-08-05 114808.gif
Screenshot 2023-08-05 114808.gif (7.52 KiB) Viewed 16870 times
seattleneil
Posts: 182
Joined: 02 Mar 2020 22:41

Re: job queue runs full after upgrade to v4.1.0

Post by seattleneil »

The output from the ps commands you provided confirms the zway/core thread is consuming an excessive amount of CPU time. Specifically, the z-way-server process has been running for 11 days (since July 27) and the zway/core thread has consumed nearly 5 days of CPU time. The output for all of the other threads look normal which suggests the problem is related to your Z-Way configuration. Assuming the expert UI does not show an excessive amount of jobs, then you'll need to do some digging as your issue is different from the job queue issue.

The hard part is determining the culprit. There are several linux commands that provide low-level information such as strace and gdb, but the output is difficult to interpret. Instead of using these low-level linux commands, I suggest starting at the Z-Way application layer since it's much friendlier. For example, one possible explanation is that you've accidentally created an automation rule that's a loop. So start by disabling all of the rules using the web UI (http://[IP ADDRESS}:8083/smarthome/#/rules) by selecting the stop icon. If you still have high CPU usage, another explanation is you may have a mis-behaving app. So continue by pausing all of the non-essential apps (http://[IP ADDRESS}:8083/smarthome/#/apps/instance) by selecting the pause icon for your local apps (do not pause "System Cron", "Inband Notifier" and "Z-Wave Network Access"). If you still have high CPU usage, check the log file (/var/log/z-way-server.log) to see it something unexpected is being logged. If top continues to show high CPU usage of the z-way-server process, a draconian approach is make a full backup of your Z-Way configuration and a Z-Wave network backup, stop the Z-Way process, rename the /opt/z-way-server directory (run: sudo mv /opt/z-way-server /opt/z-way-server-KEEP), download and reinstall the latest Z-Way software (run: PKG=z-way-4.1.1-lws16_armhf.deb && wget -O /tmp/$PKG https://storage.z-wave.me/z-way-server/$PKG && sudo dpkg -i /tmp/$PKG) and then restore the .zbk backup (using the expert web UI - http://[IP ADDRESS}:8083/expert/#/network/control). The idea behind this draconian approach is to see whether a clean Z-Way install fixes your excessive CPU utilization problem. Note that you should be able to control your Z-Wave devices using the expert UI. Also note that all of your automaton settings will be undefined (those settings are contained in the .zak full backup file made from the smarthome UI). With a clean Z-Way installation, I'm assuming/hoping the CPU utilization of the z-way-server process will become normal. If this assumption proves to be correct, then you can either rebuild your automation info and add-on apps or compare the contents of /opt/z-way-server/automation/storage/configjson-*.json with the original file from your renamed version (opt/z-way-server-KEEP). You can also try restoring the .zak file, but it's reasonably likely that using the .zak backup file will restore the excessive CPU usage problem).

If you want to try isolating the culprit using a low-level linux command, to trace what's going on with the z-way-server process zway/core thread, run:
sudo strace -fp `pidof z-way-server` 2>&1 | grep XXXXX | grep -v nanosleep (where XXXXX is the zway/core LWP value as shown by the ps -L -p `pidof z-way-server` command).

Good luck.
feuersee
Posts: 34
Joined: 07 Jan 2016 01:53

Re: job queue runs full after upgrade to v4.1.0

Post by feuersee »

Hello and thank you,

i stopped all apps i do not need and now the cpu-load is down again! There are some apps that not longer work under 4.1.1 (i lost philips hue app too :| )

Greetings Stephan
User avatar
PoltoS
Posts: 7601
Joined: 26 Jan 2011 19:36

Re: job queue runs full after upgrade to v4.1.0

Post by PoltoS »

Which app was causing it? You can stop apps one by one to locate it.
piet66
Posts: 268
Joined: 04 Feb 2017 17:00

Re: job queue runs full after upgrade to v4.1.0

Post by piet66 »

You could also provide the list of your stopped apps. That way I can check if I am using any of them.
Raspberry Pi 3 Model B Rev 1.2
Raspbian GNU/Linux 10 (buster, 32bit)
RaZberry by Z-Wave.Me ZW0700 7.20.00 07.38/1766938484 1025/257
Z-Way version v4.1.2 from 2023-10-18 03:34:26 +0300
Post Reply