Event log stops when full

[micky1500]

Since Upgrading to 2.3.5. release I have noticed that after about 3 days
the event logs have a huge chunk missing.
It will give the last 30 minutes of history (about 8 entries), then jump to 2 days ago.

Restarting the z-way server does not resolve the issue !
Deleting the Notifications-xxx.json file and then starting the server again works until
3-5 days later when the same issue occurs.

Looking at the Notifications-xxx.json file it seems to get to 4999 entries and then gets stuck
at 1053854 bytes

v2.3.5 on RPI3 Jesse 4.9.35-v7+



Further details below:-









Snapshot on 29/08/2017 19:25 but at 19:21 It's suddenly looking at 2 days before.
//---------------------------
device-gas
Utility
19:25 | Gas Meter is 2.7 Qubic meter
device-OnOff
Lounge
19:25 | PIR is off
device-OnOff
Kitchen
19:24 | PIR is off
device-OnOff
Kitchen
19:23 | PIR is on
device-OnOff
Kitchen
19:23 | PIR is off
device-OnOff
Lounge
19:23 | PIR is off
device-OnOff
Kitchen
19:22 | PIR is on
device-OnOff
Lounge
19:22 | PIR is on
device-OnOff
Lounge
19:22 | PIR is off
device-OnOff
Lounge
19:21 | PIR is on
device-OnOff
Lounge ------ Missing 2 days of data here
27.8. - 19:05 | PIR is on
device-OnOff
Lounge
27.8. - 19:05 | PIR is off
device-OnOff
Lounge
27.8. - 19:05 | PIR is on
device-OnOff
Garden
27.8. - 19:03 | Conservatory PIR is off


30 minutes later. Still only see the last 2 mins of history. Then jumps back to 48 hours previous.


Lounge
19:55 | Light is 33%
device-OnOff
Passage
19:55 | Light bedroom is on
device-temperature
Garage
19:55 | Temperature is 21.1 °C
device-OnOff
Utility
19:55 | Shut/Open ALL Curtains is on
device-OnOff
Lounge
19:55 | Curtains - French Doors is on
device-OnOff
Bedroom2
19:55 | Curtains is on
device-OnOff
Kitchen
19:55 | Curtains is on
device-OnOff
Lounge
19:55 | Curtains - Patio is on
device-OnOff
Lounge
19:55 | Curtains Click twice TV is on
device-OnOff
Utility
19:55 | Daylight 202 is off
device-OnOff
Lounge
19:54 | PIR is off
device-OnOff
Kitchen
19:54 | PIR is off
device-OnOff
Lounge
19:54 | PIR is on
device-OnOff
Lounge
27.8. - 19:05 | PIR is on
device-OnOff
Lounge
27.8. - 19:05 | PIR is off


----------------------------------
03/09/2017 09:35 PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
16910 root 20 0 168108 75592 16780 S 11.7 8.0 309:17.90 z-way-server

at 13:15
-rw-r--r-- 1 root root 1053854 Sep 3 12:50 notifications-f37bd2f66651e7d46f6d38440f2bc5dd.json

-rw-r--r-- 1 root root 1053854 Sep 3 13:18 notifications-f37bd2f66651e7d46f6d38440f2bc5dd.json

-rw-r--r-- 1 root root 1053854 Sep 3 14:01 notifications-f37bd2f66651e7d46f6d38440f2bc5dd.json

----------------------------------------
device-OnOff
Lounge
12:54 | PIR is on
device-OnOff
Kitchen
12:54 | PIR is on
device-luminiscence
Bathroom
12:54 | Luminiscence is 37 Lux
device-humidity
Bathroom
12:54 | Humidity is 66 %
device-temperature
Bathroom
12:54 | Temperature is 20.2 °C
device-luminiscence
Kitchen
12:52 | Luminiscence is 136 Lux
device-temperature
Kitchen
12:52 | Temperature is 20.9 °C
device-OnOff
Kitchen
12:51 | PIR is off
device-OnOff
Kitchen
08:01 | PIR is off
device-OnOff
Lounge
08:01 | PIR is on
device-OnOff
Lounge
08:01 | PIR is off
device-OnOff
Kitchen
08:01 | PIR is on
device-OnOff
Lounge
08:00 | PIR is on
device-luminiscence
Bathroom
07:59 | Luminiscence is 13 Lux


@13:20

Lounge
13:20 | PIR is off
device-OnOff
Lounge
13:19 | PIR is on
device-OnOff
Lounge
13:19 | PIR is off
device-OnOff
Kitchen
13:19 | PIR is on
device-OnOff
Kitchen
13:19 | PIR is off
device-OnOff
Lounge
13:19 | PIR is on
device-OnOff
Kitchen
13:18 | PIR is on
device-OnOff
Kitchen
13:18 | PIR is off
device-OnOff
Lounge
13:18 | PIR is off
device-OnOff
Kitchen
08:01 | PIR is off
device-OnOff
Lounge
08:01 | PIR is on
device-OnOff
Lounge
08:01 | PIR is off
device-OnOff
Kitchen
08:01 | PIR is on
device-OnOff
Lounge
08:00 | PIR is on
device-luminiscence
Bathroom
07:59 | Luminiscence is 13 Lux

[piet66]

Same issue here.

I'm only displaying very few important notifications. But notifications*.json is crowded with lines I never need.

[piet66]

Here the daily sizes of my notifications*.json since upgrading to 2.3.5. Sometimes size is changing, but mostly not:

Code: Select all

Di 15. Aug 08:13:01 CEST 2017 notifications-f37bd2f66651e7d46f6d38440f2bc5dd.json: 1094570 B
Mi 16. Aug 08:13:01 CEST 2017 notifications-f37bd2f66651e7d46f6d38440f2bc5dd.json: 1106133 B
Do 17. Aug 08:13:01 CEST 2017 notifications-f37bd2f66651e7d46f6d38440f2bc5dd.json: 1106133 B
Fr 18. Aug 08:13:01 CEST 2017 notifications-f37bd2f66651e7d46f6d38440f2bc5dd.json: 1106133 B
Sa 19. Aug 08:13:01 CEST 2017 notifications-f37bd2f66651e7d46f6d38440f2bc5dd.json: 1106133 B
So 20. Aug 08:13:01 CEST 2017 notifications-f37bd2f66651e7d46f6d38440f2bc5dd.json: 1106133 B
Mo 21. Aug 08:13:01 CEST 2017 notifications-f37bd2f66651e7d46f6d38440f2bc5dd.json: 1106133 B
Di 22. Aug 08:13:01 CEST 2017 notifications-f37bd2f66651e7d46f6d38440f2bc5dd.json: 748425 B
Mi 23. Aug 08:13:01 CEST 2017 notifications-f37bd2f66651e7d46f6d38440f2bc5dd.json: 1108860 B
Do 24. Aug 08:13:01 CEST 2017 notifications-f37bd2f66651e7d46f6d38440f2bc5dd.json: 1101117 B
Fr 25. Aug 08:13:01 CEST 2017 notifications-f37bd2f66651e7d46f6d38440f2bc5dd.json: 933145 B
Sa 26. Aug 08:13:01 CEST 2017 notifications-f37bd2f66651e7d46f6d38440f2bc5dd.json: 1106208 B
So 27. Aug 08:13:02 CEST 2017 notifications-f37bd2f66651e7d46f6d38440f2bc5dd.json: 1106208 B
Mo 28. Aug 08:13:01 CEST 2017 notifications-f37bd2f66651e7d46f6d38440f2bc5dd.json: 1106208 B
Di 29. Aug 08:13:01 CEST 2017 notifications-f37bd2f66651e7d46f6d38440f2bc5dd.json: 1106208 B
Mi 30. Aug 08:13:02 CEST 2017 notifications-f37bd2f66651e7d46f6d38440f2bc5dd.json: 1106208 B
Do 31. Aug 08:13:02 CEST 2017 notifications-f37bd2f66651e7d46f6d38440f2bc5dd.json: 1106208 B
Fr 1. Sep 08:13:01 CEST 2017 notifications-f37bd2f66651e7d46f6d38440f2bc5dd.json: 854045 B
Sa 2. Sep 08:13:01 CEST 2017 notifications-f37bd2f66651e7d46f6d38440f2bc5dd.json: 1105804 B
So 3. Sep 08:13:01 CEST 2017 notifications-f37bd2f66651e7d46f6d38440f2bc5dd.json: 1105804 B
Mo 4. Sep 08:13:01 CEST 2017 notifications-f37bd2f66651e7d46f6d38440f2bc5dd.json: 1110998 B
Di 5. Sep 08:13:01 CEST 2017 notifications-f37bd2f66651e7d46f6d38440f2bc5dd.json: 1110998 B

[piet66]

No reaction?

It lasts only half a day after removal till notifications is full again and stops working. After that only the modification time changes but not the contents any more.

In this state event log is not usable. Can we expect a solution, is there a workaround or can I help analysing?

[micky1500]

I reported this 12 days ago.
Is anyone from support actually reading our concerns ?

I have since upgraded to v2.3.6-rc3
It varies from 1 hour of recent logs, to the last day of updates. with missing last 2-3 days.

Since upgrading though my Notifications file is not in the correct place \\ZWAY-PI3\PiShare\opt\z-way-server\automation\storage
There is not a notifications file in this folder since I deleted it a few days ago.

It has not moved from the previous 2.3.5 location which was renamed by the update process. \\ZWAY-PI3\PiShare\opt\z-way-server_2017-09-10-17-03-21\automation\storage

There are 5 files in the pre-update location (z-way-server_2017-09-10-17-03) that are still being used as the current files.
-rw-r--r-- 1 root root 283K Sep 15 20:00 configjson-06b2d3b23dce96e1619d2b53d6c947ec.json
-rw-r--r-- 1 root root 1.1M Sep 15 19:45 notifications-f37bd2f66651e7d46f6d38440f2bc5dd.json
-rw-r--r-- 1 root root 1.5M Sep 15 20:00 history-3cd15f8f2940aff879df34df4e5c2cd1.json
-rw-r--r-- 1 root root 1.9M Sep 15 19:54 zwayparsedPacketsjson-f1aed46af7fae1368e78db7768e4d9e5.json
-rw-r--r-- 1 root root 7.0M Sep 15 19:54 zwayoriginPacketsjson-2ad2998bc2cb543d29ac586cec215398.json

Please respond with an update.

Thankyou.

[PoltoS]

This problem have to be fixed in the latest v2.3.6-rc. Does it for you?

[micky1500]

I loaded v2.3.6-rc8
Yes it does work.
Thanks.

[piet66]

Using v2.3.6-rc8 too. Old events are regularly removed from file. So far it works.

But today morning (7.10. 9:30) I recognized, that all events earlier than this morning 2:29 have gone.
file size: 551,9 KiB, 2500 events.
first event: timestamp":"2017-10-07T02:29:02.162Z"
last event: timestamp":"2017-10-07T07:31:53.621Z"

And also it contains only events of type "device-info".
All other notifications are missing. For example 1 module notification at 04:01 (checked from log file).

That means, I cannot check events from yesterday and from last night. It can't stay like this.

[PoltoS]

The logic is to keep a certain ammount of events AND not older that certain period. All this not to store too old stuff and not to let the size grow too much.

[piet66]

This approach is correct. But a period of 7 hours is much too short. You must hold at least the events of yesterday and of last night. And you may hold all events within this period, not only device-infos but also module notifications and others.
In case of doubt I can do without the device-infos, but the module notification (errors) are very important!