Corrupt SD card...again!

Discussions about Z-Way software and Z-Wave technology in general
micky1500
Posts: 298
Joined: 07 Feb 2016 16:29
Location: England

Re: Corrupt SD card...again!

Post by micky1500 »

I always planned to look into the PI4 4GB Ram and use that instead of SD for frequent changes.
had other things that needed doing first.
As xurg says, it would be nice if zwave would be more SDCard friendly.
Raspi 4 - (Buster - 32 Bit) Zwave Version 4.1.1, Raz 7 Pro, Serial API Version: 07.38
insiorc
Posts: 43
Joined: 01 Apr 2019 21:55

Re: Corrupt SD card...again!

Post by insiorc »

Like log2ram? I wasn't aware of such a thing but it would be interesting if it worked ok with ZWay. I power the Rpi4 off a UPS anyway so it should be ok, providing no crashes but that doesn't seem to be an issue, but other than changes to the setup then I can't think that a PIR etc. not recorded in a given state would be much of a big deal. I'll do some more reading and probably give it a go.
Rpi4 2Gb, Z-Way 3.2.2, UZB 5.39
User avatar
PoltoS
Posts: 7562
Joined: 26 Jan 2011 19:36

Re: Corrupt SD card...again!

Post by PoltoS »

You can disable logs or move them to to filesystem. Is that worth doing?
micky1500
Posts: 298
Joined: 07 Feb 2016 16:29
Location: England

Re: Corrupt SD card...again!

Post by micky1500 »

The logs are a start !
There are other files on the SD card that have high write access.
https://forum.z-wave.me/viewtopic.php?f ... ata#p88181

/opt/z-way-server/config/zddx/e4918c29-DevicesData.xml
/opt/z-way-server/automation/storage/zwayparsedPacketsjson-f1aed46af7fae1368e78db7768e4d9e5.json
/opt/z-way-server/automation/storage/zwayoriginPacketsjson-2ad2998bc2cb543d29ac586cec215398.json
/opt/z-way-server/automation/storage/notifications-f37bd2f66651e7d46f6d38440f2bc5dd.json

Zram or log2ram is probably the answer for a PI4 with 4GB, or even now with 8GB Ram.
Raspi 4 - (Buster - 32 Bit) Zwave Version 4.1.1, Raz 7 Pro, Serial API Version: 07.38
insiorc
Posts: 43
Joined: 01 Apr 2019 21:55

Re: Corrupt SD card...again!

Post by insiorc »

I installed monit after reading of it earlier in the thread (thanks micky1500), yesterday evening I reboot the ZWay Rpi then this morning I see that in 6hrs 38minutes it has a disk write value of 35.8Gb. Then only 18 minutes later there it's up to 37.9Gb, another 1.1Gb. Surely this can't be normal, at that write rate it's no wonder my SD card failed?
3Zway 35.8gb.jpg
3Zway 35.8gb.jpg (129.94 KiB) Viewed 4745 times
3ZWay 37.9gb.png
3ZWay 37.9gb.png (63.31 KiB) Viewed 4745 times
Rpi4 2Gb, Z-Way 3.2.2, UZB 5.39
insiorc
Posts: 43
Joined: 01 Apr 2019 21:55

Re: Corrupt SD card...again!

Post by insiorc »

I setup a fresh install, latest buster & ZWay to rule out any other issue's causing the disk write. So now after only 47 minutes it 8.7Gb written to the disk. Is this normal?
New Zway pic.png
New Zway pic.png (80.96 KiB) Viewed 4740 times
Rpi4 2Gb, Z-Way 3.2.2, UZB 5.39
michap
Posts: 437
Joined: 26 Mar 2013 10:35
Contact:

Re: Corrupt SD card...again!

Post by michap »

try to find out what are the big files/ directories (from root directory):

Code: Select all

du -aBM 2>/dev/null | sort -nr | head -n 50 | more
maybe it help...
micky1500
Posts: 298
Joined: 07 Feb 2016 16:29
Location: England

Re: Corrupt SD card...again!

Post by micky1500 »

monit.JPG
Probably explains why SD cards wear out.
My stats similar to yours, 10GB worth of Writes per day.
Not sure any SD cards relocate new writes, if the same block is rewritten over and over, it is likely to fail.
USB SSD drives might use new locations, but I have not found solid proof of that.

Stuff I posted Saturday/Sunday got deleted, maybe I complain too much.

I'm definitely looking to use the spare Ram to ease this issue.
Log2ram or Zram with built in config might do that, maybe an additional startup script.
Raspi 4 - (Buster - 32 Bit) Zwave Version 4.1.1, Raz 7 Pro, Serial API Version: 07.38
insiorc
Posts: 43
Joined: 01 Apr 2019 21:55

Re: Corrupt SD card...again!

Post by insiorc »

michap, thanks for that, unfortunately I'd already wiped the installation that provided the above pics, I've installed from sratch again on a different SD card, it still wrote 1.3Gb in 14 minutes. I ran what you posted and got;

pi@raspberrypi:~ $ du -aBM 2>/dev/null | sort -nr | head -n 50 | more
1M ./.profile
1M ./.gnupg/private-keys-v1.d
1M ./.gnupg
1M ./.bashrc
1M ./.bash_logout
1M ./.bash_history
1M .

micky1500, Since last night i've got another installation with both zram and log2ram running, monit still says very high disk writes but I'm not sure if that is writing to ram or the USB drive. I set 700Mb for the log2ram, but I don't know what happens when it get's full, does it dump to SD/USB anyway?
Rpi4 2Gb, Z-Way 3.2.2, UZB 5.39
micky1500
Posts: 298
Joined: 07 Feb 2016 16:29
Location: England

Re: Corrupt SD card...again!

Post by micky1500 »

I don't really know.
Zram writes can be ignored. as they are to Ram only.
Depends on the config of your Zram as to how frequent it saves the files to SD.
Default is 1 hour, mine seems to be 24 hours.
You loose logs on a crash.
if you want to reboot, use 'sudo reboot'. and wait 1 minute. but not 'Sudo reboot now'. as it seems to close without updating.
Raspi 4 - (Buster - 32 Bit) Zwave Version 4.1.1, Raz 7 Pro, Serial API Version: 07.38
Post Reply