V4.1.2: steadily increasing cpu load

Discussions about Z-Way software and Z-Wave technology in general
User avatar
PoltoS
Posts: 7544
Joined: 26 Jan 2011 19:36

Re: V4.1.2: steadily increasing cpu load

Post by PoltoS »

Yes, it is possible to make the other way and use modhttp.so from 3.2.3 in 4.1.2.
piet66
Posts: 266
Joined: 04 Feb 2017 17:00

Re: V4.1.2: steadily increasing cpu load

Post by piet66 »

Perhaps any of the 4.1.2 users can do that?
Raspberry Pi 3 Model B Rev 1.2
Raspbian GNU/Linux 10 (buster, 32bit)
RaZberry by Z-Wave.Me ZW0700 7.20.00 07.38/1766938484 1025/257
Z-Way version v3.2.3 from 2022-04-06 04:56:23 +0300
User avatar
PoltoS
Posts: 7544
Joined: 26 Jan 2011 19:36

Re: V4.1.2: steadily increasing cpu load

Post by PoltoS »

Do we have more evidence? Can somebody help us investigate the issue in more details by providing us with the remote access?
User avatar
GokMasE
Posts: 59
Joined: 13 Mar 2016 01:04
Location: Sweden

Re: V4.1.2: steadily increasing cpu load

Post by GokMasE »

PoltoS wrote:
19 Jan 2024 19:39
Do we have more evidence? Can somebody help us investigate the issue in more details by providing us with the remote access?
Surely there must be SOMEONE of all who reported increasing CPU load with more recent version that can assist the developers to examine this more thoroughly? :o
User avatar
PoltoS
Posts: 7544
Joined: 26 Jan 2011 19:36

Re: V4.1.2: steadily increasing cpu load

Post by PoltoS »

Hey, someone ! ;)
piet66
Posts: 266
Joined: 04 Feb 2017 17:00

Re: V4.1.2: steadily increasing cpu load

Post by piet66 »

PoltoS wrote:
09 Feb 2024 17:47
Hey, someone ! ;)
I actually wanted to wait a little longer and do some more tests before reporting my findings. But I'm convinced that I'm right.

The comments about garbage collection in other thread led me in the right direction. I was able to find the app that generates the growth and isolate the reason.

The app is doing socket communications. I made some changes and now I don't have CPU growth anymore in 4.1.2.
I have:
- consistently cleared all buffer variables as soon as they are no longer needed (>undefined)
- deleted a recursive function call with passing a buffer variable

Please look here for the result:
changed2.png
changed2.png (72.57 KiB) Viewed 573 times
In the left picture, you see there's no CPU load growing.

Interesting is the right picture.It shows the development of the memory of two different versions of the app:
- The last 5 days (week 06) a version without the recursive function call, but also without the explicit clearing of the buffer variables. The memory is growing, but suddenly is comes back.
- The 5 days before (week 05) the version without the recursive function call and now with the explicit clearing of the buffer variables in the app. The memory is constant.

I conclude from this:
- In 4.1.2 there is a problem with the release of buffers received from external. In my case It's socket communication, but this probably also applies to http communication.
- The problem was worsened by the recursive function call. This led to the growth of the CPU load.

As a workaround you can clear all buffers in the app.
Raspberry Pi 3 Model B Rev 1.2
Raspbian GNU/Linux 10 (buster, 32bit)
RaZberry by Z-Wave.Me ZW0700 7.20.00 07.38/1766938484 1025/257
Z-Way version v3.2.3 from 2022-04-06 04:56:23 +0300
User avatar
PoltoS
Posts: 7544
Joined: 26 Jan 2011 19:36

Re: V4.1.2: steadily increasing cpu load

Post by PoltoS »

That is a great work!

Please let us know what was not released and which buffer you cleared to make it consume less memory and less CPU?

What was producing the high CPU? Was it the GC working? How have you changed this?

As for releasing buffers in the JS code versus letting the GC do it, I see a 20 Mb difference in memory (which is big!), but Z-Way is still handling it by clearing. I'm interested to see which buffer were you playing with to understand why wasn't it cleared on time.
piet66
Posts: 266
Joined: 04 Feb 2017 17:00

Re: V4.1.2: steadily increasing cpu load

Post by piet66 »

Hi PoltoS,
Your questions are difficult to answer.
Perhaps you should take a look at the code? Preferably all 3 versions for comparison. I can put this together in a clearly arranged form.
As for releasing buffers in the JS code versus letting the GC do it, I see a 20 Mb difference in memory (which is big!)
I agree, it looks very strange. And it continues looking strange:
process02z.2week.png
process02z.2week.png (40.71 KiB) Viewed 471 times
In the meantime, I only made a restart.
Raspberry Pi 3 Model B Rev 1.2
Raspbian GNU/Linux 10 (buster, 32bit)
RaZberry by Z-Wave.Me ZW0700 7.20.00 07.38/1766938484 1025/257
Z-Way version v3.2.3 from 2022-04-06 04:56:23 +0300
User avatar
PoltoS
Posts: 7544
Joined: 26 Jan 2011 19:36

Re: V4.1.2: steadily increasing cpu load

Post by PoltoS »

Let's separate memory usage and CPU load... I'm not even sure this is related.

Could you please show your code diff that lead to lower CPU usage?
Post Reply