Page 1 of 1

Corrupt Rasbian

Posted: 25 Jan 2016 20:15
by leonmf
I don't have enough understanding of what's going on yet, but I am repeatedly getting corrupt filesystems on my Z-Way install. Prior to the 2.2 RCs, I had been running the same install of Raspbian on an rPi1 for over a year.

Since 2.1.2-RC17, I've had to rebuild and re-install my rPi about 5 times. After the second time, I switched to an rPi2. I'm in the process of building with a new SD card for the rPi2 but I don't have a lot of hope that this will fix it.

Is anyone else seeing something similar or is there something unique to my setup/method/whatever?

Re: Corrupt Raspian

Posted: 26 Jan 2016 13:58
by vh342
Filesystem corruptions can have a couple of reasons. From my experience, the following are the most striking:

- overclocking of RPI
- inadequate power supply
- not genuine SD-Cards (there are a lot of fake products on the market)
- a lot of write cycles on SD (look for rapidly growing Log-files! Z-way-server.log is written several times per second usually.)

Especially for the last point I recommend to mount /var/log with a tmpfs in the RAM. Together with a well configured logrotate.conf a ramdisk amount of 100MB would be adequate. With this setup my problem with corrupt filesystems are gone.

Re: Corrupt Raspian

Posted: 26 Jan 2016 19:40
by leonmf
@vh342,

Do you have any links to instructions on mounting var/log to a ramdisk and logrotate?

Re: Corrupt Raspian

Posted: 07 Feb 2016 15:31
by vh342
You may have already found a solution in the meantime. Here are my settings for a 24/7 duty, may be it helps anyway.

1) Mount /var/log to tempfs - edit /etc/fstab and add:

Code: Select all

tmpfs /var/log tmpfs defaults,noatime,nosuid,mode=0755,size=128m 0 0
tmpfs /var/run tmpfs defaults,noatime,nosuid,mode=0755,size=8m 0 0
128MB for the log files are absolutely sufficient. If you are low on memory, even half of it would be enough. Important is the correct configuration of log rotate to avoid that ramdisk runs out of space...

2) Configure Logrotate.d and add a separate file for z-way-server in /etc/logrotate.d/:

Code: Select all

/var/log/z-way-server.log {
        daily
        rotate 3
        compress
        delaycompress
        copytruncate    
        missingok
        notifempty
}
With this config you have one revision of each log for the last 3 days available in Ram. If you want to save these files to SD, it's possible to add the option "olddir" above and the logs are rotated to an alternative dir or just use rsync as a cron-job instead.

Re: Corrupt Raspian

Posted: 08 Feb 2016 15:27
by pz1
Is it necessary to replace this existing content of file z-way-server

Code: Select all

/var/log/z-way-server.log {
        daily
        size=10M
        rotate 4
        compress
        nodelaycompress
        missingok
        notifempty
        postrotate
                /usr/bin/killall -HUP z-way-server 2>/dev/null || true
                endscript
}
with the scheme you suggest? (I still am a *nix noob)

Re: Corrupt Raspian

Posted: 10 Feb 2016 00:57
by vh342
Yes, for my setup it was necessary to change the existing file. The given postrotate script was not working properly! After the rotation the new logfile couldn't be written anymore.

With the config I posted above everything works so far now.

Re: Corrupt Raspian

Posted: 10 Feb 2016 12:13
by pz1
The existing rotate configuratiojn seems to work for me. i.e. I do see a *.gz archive. I 'll watch it the coming days.

Re: Corrupt Raspian

Posted: 12 Feb 2016 13:29
by pz1
leonmf wrote:I don't have enough understanding of what's going on yet, but I am repeatedly getting corrupt filesystems on my Z-Way install.
Because I had a corrupted Pi again this morning, I did a search on the Pi forum. I found this rather shocking thread:
https://www.raspberrypi.org/forums/view ... 28&t=36533.

Shocking, not only because the number of people affected, but even more because this thread dates back to March 2013, and still no solution. Very little response from the Pi organisation.
They don't see this as a real problem. Users are blamed for cutting the power without properly shutting down the Pi, even if quite a few solid reporters had taken the necessary precautions.

And yes this happened to me AFTER I added PICO UPS and configured logs and tmp to write to ramdisk :evil:

Re: Corrupt Raspian

Posted: 12 Mar 2016 13:35
by pz1
vh342 wrote:You may have already found a solution in the meantime. Here are my settings for a 24/7 duty, may be it helps anyway.

1) Mount /var/log to tempfs - edit /etc/fstab and add:
I decided to add an USB stick to store the logs, so I can still see them after a crash (hopefully)
Draft Recipe

On my latest fresh installation of v2.2.1 on Rasbian-Jessie I discovered with command df-h that the system already set a few tmpfs folders:
df-h.PNG
df-h.PNG (8.26 KiB) Viewed 11634 times
The last line /dev/sda1 is the USB drive I added in /etc/fstab. None of the tmpfs were defined there!