Page 12 of 16

Re: job queue runs full after upgrade to v4.1.0

Posted: 01 Aug 2023 11:01
by piet66
If you are looking for a more convenient method to supervise the CPU load, I can promote my Linux script collection 'MxBash'.
If you want to have a try, download it from Z-Way store. For to see it, you have to add the token 'MxBash' to smarthome/settings/App Store Access.

Re: job queue runs full after upgrade to v4.1.0

Posted: 01 Aug 2023 11:20
by piet66
seattleneil wrote:
01 Aug 2023 00:24
Since no one else on this forum has reported the z-way-server process consuming anything close to 65% %CPU,
I think we must distinguish between 2 different kinds of CPU values:
top displays the current CPU consumption, which may be very high for short moments. That's not a problem. ps computes the average consumption over a certain interval. This should be relatively constant. In my system about 7%.

Re: job queue runs full after upgrade to v4.1.0

Posted: 03 Aug 2023 21:08
by AlesKO
ss.jpg
ss.jpg (3.13 KiB) Viewed 21142 times
Just returned from one week vacations

installed v4.1.0-27-g589c9812 version.

Re: job queue runs full after upgrade to v4.1.0

Posted: 03 Aug 2023 23:21
by harre
4.1.1 solved my issues with long queues that have been bothering me for a while.

Re: job queue runs full after upgrade to v4.1.0

Posted: 05 Aug 2023 09:52
by feuersee
running the commands: ps -L -p `pidof z-way-server`; ps -F -p `pidof z-way-server`
Screenshot 2023-08-05 084844.gif
Screenshot 2023-08-05 084844.gif (33.52 KiB) Viewed 21111 times

Re: job queue runs full after upgrade to v4.1.0

Posted: 05 Aug 2023 12:51
by feuersee
sorry, second screenshot for ps -F -p `pidof z-way-server`:

Re: job queue runs full after upgrade to v4.1.0

Posted: 07 Aug 2023 00:49
by seattleneil
The output from the ps commands you provided confirms the zway/core thread is consuming an excessive amount of CPU time. Specifically, the z-way-server process has been running for 11 days (since July 27) and the zway/core thread has consumed nearly 5 days of CPU time. The output for all of the other threads look normal which suggests the problem is related to your Z-Way configuration. Assuming the expert UI does not show an excessive amount of jobs, then you'll need to do some digging as your issue is different from the job queue issue.

The hard part is determining the culprit. There are several linux commands that provide low-level information such as strace and gdb, but the output is difficult to interpret. Instead of using these low-level linux commands, I suggest starting at the Z-Way application layer since it's much friendlier. For example, one possible explanation is that you've accidentally created an automation rule that's a loop. So start by disabling all of the rules using the web UI (http://[IP ADDRESS}:8083/smarthome/#/rules) by selecting the stop icon. If you still have high CPU usage, another explanation is you may have a mis-behaving app. So continue by pausing all of the non-essential apps (http://[IP ADDRESS}:8083/smarthome/#/apps/instance) by selecting the pause icon for your local apps (do not pause "System Cron", "Inband Notifier" and "Z-Wave Network Access"). If you still have high CPU usage, check the log file (/var/log/z-way-server.log) to see it something unexpected is being logged. If top continues to show high CPU usage of the z-way-server process, a draconian approach is make a full backup of your Z-Way configuration and a Z-Wave network backup, stop the Z-Way process, rename the /opt/z-way-server directory (run: sudo mv /opt/z-way-server /opt/z-way-server-KEEP), download and reinstall the latest Z-Way software (run: PKG=z-way-4.1.1-lws16_armhf.deb && wget -O /tmp/$PKG https://storage.z-wave.me/z-way-server/$PKG && sudo dpkg -i /tmp/$PKG) and then restore the .zbk backup (using the expert web UI - http://[IP ADDRESS}:8083/expert/#/network/control). The idea behind this draconian approach is to see whether a clean Z-Way install fixes your excessive CPU utilization problem. Note that you should be able to control your Z-Wave devices using the expert UI. Also note that all of your automaton settings will be undefined (those settings are contained in the .zak full backup file made from the smarthome UI). With a clean Z-Way installation, I'm assuming/hoping the CPU utilization of the z-way-server process will become normal. If this assumption proves to be correct, then you can either rebuild your automation info and add-on apps or compare the contents of /opt/z-way-server/automation/storage/configjson-*.json with the original file from your renamed version (opt/z-way-server-KEEP). You can also try restoring the .zak file, but it's reasonably likely that using the .zak backup file will restore the excessive CPU usage problem).

If you want to try isolating the culprit using a low-level linux command, to trace what's going on with the z-way-server process zway/core thread, run:
sudo strace -fp `pidof z-way-server` 2>&1 | grep XXXXX | grep -v nanosleep (where XXXXX is the zway/core LWP value as shown by the ps -L -p `pidof z-way-server` command).

Good luck.

Re: job queue runs full after upgrade to v4.1.0

Posted: 08 Aug 2023 12:11
by feuersee
Hello and thank you,

i stopped all apps i do not need and now the cpu-load is down again! There are some apps that not longer work under 4.1.1 (i lost philips hue app too :| )

Greetings Stephan

Re: job queue runs full after upgrade to v4.1.0

Posted: 09 Aug 2023 04:28
by PoltoS
Which app was causing it? You can stop apps one by one to locate it.

Re: job queue runs full after upgrade to v4.1.0

Posted: 09 Aug 2023 16:08
by piet66
You could also provide the list of your stopped apps. That way I can check if I am using any of them.