Corrupt Rasbian

Discussions about RaZberry - Z-Wave board for Raspberry computer
Post Reply
leonmf
Posts: 44
Joined: 23 Oct 2014 02:59

Corrupt Rasbian

Post by leonmf »

I don't have enough understanding of what's going on yet, but I am repeatedly getting corrupt filesystems on my Z-Way install. Prior to the 2.2 RCs, I had been running the same install of Raspbian on an rPi1 for over a year.

Since 2.1.2-RC17, I've had to rebuild and re-install my rPi about 5 times. After the second time, I switched to an rPi2. I'm in the process of building with a new SD card for the rPi2 but I don't have a lot of hope that this will fix it.

Is anyone else seeing something similar or is there something unique to my setup/method/whatever?
vh342
Posts: 14
Joined: 19 Jan 2016 23:20

Re: Corrupt Raspian

Post by vh342 »

Filesystem corruptions can have a couple of reasons. From my experience, the following are the most striking:

- overclocking of RPI
- inadequate power supply
- not genuine SD-Cards (there are a lot of fake products on the market)
- a lot of write cycles on SD (look for rapidly growing Log-files! Z-way-server.log is written several times per second usually.)

Especially for the last point I recommend to mount /var/log with a tmpfs in the RAM. Together with a well configured logrotate.conf a ramdisk amount of 100MB would be adequate. With this setup my problem with corrupt filesystems are gone.
leonmf
Posts: 44
Joined: 23 Oct 2014 02:59

Re: Corrupt Raspian

Post by leonmf »

@vh342,

Do you have any links to instructions on mounting var/log to a ramdisk and logrotate?
vh342
Posts: 14
Joined: 19 Jan 2016 23:20

Re: Corrupt Raspian

Post by vh342 »

You may have already found a solution in the meantime. Here are my settings for a 24/7 duty, may be it helps anyway.

1) Mount /var/log to tempfs - edit /etc/fstab and add:

Code: Select all

tmpfs /var/log tmpfs defaults,noatime,nosuid,mode=0755,size=128m 0 0
tmpfs /var/run tmpfs defaults,noatime,nosuid,mode=0755,size=8m 0 0
128MB for the log files are absolutely sufficient. If you are low on memory, even half of it would be enough. Important is the correct configuration of log rotate to avoid that ramdisk runs out of space...

2) Configure Logrotate.d and add a separate file for z-way-server in /etc/logrotate.d/:

Code: Select all

/var/log/z-way-server.log {
        daily
        rotate 3
        compress
        delaycompress
        copytruncate    
        missingok
        notifempty
}
With this config you have one revision of each log for the last 3 days available in Ram. If you want to save these files to SD, it's possible to add the option "olddir" above and the logs are rotated to an alternative dir or just use rsync as a cron-job instead.
pz1
Posts: 2053
Joined: 08 Apr 2012 13:44

Re: Corrupt Raspian

Post by pz1 »

Is it necessary to replace this existing content of file z-way-server

Code: Select all

/var/log/z-way-server.log {
        daily
        size=10M
        rotate 4
        compress
        nodelaycompress
        missingok
        notifempty
        postrotate
                /usr/bin/killall -HUP z-way-server 2>/dev/null || true
                endscript
}
with the scheme you suggest? (I still am a *nix noob)
vh342
Posts: 14
Joined: 19 Jan 2016 23:20

Re: Corrupt Raspian

Post by vh342 »

Yes, for my setup it was necessary to change the existing file. The given postrotate script was not working properly! After the rotation the new logfile couldn't be written anymore.

With the config I posted above everything works so far now.
pz1
Posts: 2053
Joined: 08 Apr 2012 13:44

Re: Corrupt Raspian

Post by pz1 »

The existing rotate configuratiojn seems to work for me. i.e. I do see a *.gz archive. I 'll watch it the coming days.
pz1
Posts: 2053
Joined: 08 Apr 2012 13:44

Re: Corrupt Raspian

Post by pz1 »

leonmf wrote:I don't have enough understanding of what's going on yet, but I am repeatedly getting corrupt filesystems on my Z-Way install.
Because I had a corrupted Pi again this morning, I did a search on the Pi forum. I found this rather shocking thread:
https://www.raspberrypi.org/forums/view ... 28&t=36533.

Shocking, not only because the number of people affected, but even more because this thread dates back to March 2013, and still no solution. Very little response from the Pi organisation.
They don't see this as a real problem. Users are blamed for cutting the power without properly shutting down the Pi, even if quite a few solid reporters had taken the necessary precautions.

And yes this happened to me AFTER I added PICO UPS and configured logs and tmp to write to ramdisk :evil:
pz1
Posts: 2053
Joined: 08 Apr 2012 13:44

Re: Corrupt Raspian

Post by pz1 »

vh342 wrote:You may have already found a solution in the meantime. Here are my settings for a 24/7 duty, may be it helps anyway.

1) Mount /var/log to tempfs - edit /etc/fstab and add:
I decided to add an USB stick to store the logs, so I can still see them after a crash (hopefully)
Draft Recipe

On my latest fresh installation of v2.2.1 on Rasbian-Jessie I discovered with command df-h that the system already set a few tmpfs folders:
df-h.PNG
df-h.PNG (8.26 KiB) Viewed 10089 times
The last line /dev/sda1 is the USB drive I added in /etc/fstab. None of the tmpfs were defined there!
Post Reply