Upgrade v2.3.8 to v3.2.2

Discussions about Z-Way software and Z-Wave technology in general
zuperman
Posts: 8
Joined: 11 Jan 2022 00:16

Upgrade v2.3.8 to v3.2.2

Post by zuperman »

Hello

I scattered the board for hours trying to find a good explanation for upgrading my z wave installation from v2.3.8 to the latest version.
Unlike others my setup has run for years without any hiccups. I now want to start using the API and thought it would be best to upgrade first to the latest version, also from a security perspective.
If I try to upgrade through the Updater, it doesn't go further than a couple percent and then nothing happens.
I tried a manual update and ended up with a bunch of missing dependencies and had te revert to get things working again. Suppose I need to do some things in the right order.

I'm on Jessie
v2.3.8
SDK Version: 6.51.06
Serial API Version: 05.04

Vendor: RaZberry by Z-Wave.Me
Vendors Product ID: 1024 / 2
Z-Wave Chip: ZW0500

Can anyone provide a step by step upgrade plan?
Thanks in advance!
Erik
seattleneil
Posts: 42
Joined: 02 Mar 2020 22:41

Re: Upgrade v2.3.8 to v3.2.2

Post by seattleneil »

Jessie is not supported by newer versions of Z-Way software. As a result, you'll need to upgrade your Raspberry Pi from Jessie (based on Debian 8) to either Buster (Debian 10) or possibly Bullseye (Debian 11). Upgrading the OS is going to take some effort, especially if you follow the apt dist-upgrade method. Have you considered doing a virgin install of Buster with Z-Way software onto a new SD card as described here: https: //help.z-wave.me/en/knowledge_base/art/55/cat/60/installing-z-way-on-raspberry-pi ?

If you back up your 2.3.8 Z-Way network information (use expert UI->Network->Control->Backup and Restore) and back up your Z-Way configuration (use smarthome UI->Settings->Backup & Restore), with luck, you'll be able to restore these files onto your new SD card.

The proposed steps are as follows:
1. Follow the virgin Buster/Z-Way installation process
2. Back up your 2.3.8 Z-Way configuration
3. Power down your Pi and insert the new SD card
4. Power up your Pi and do a restore of the Z-Way backup files

The biggest risk is that the 2.3.8 backups may not be compatible with newer versions of Z-Way. The other downside is that any changes you've made on your old SD card will need to be repeated on your new SD card (e.g., pi login password, network settings). If the restore of you 2.3.8 backup files fail, chances are good that your efforts haven't been completely wasted - you can manually install version 2.3.8 onto your new SD card, restore the Z-Way backups, upgrade to 3.0.0, upgrade to 3.1.0 and then upgrade to 3.2.2.

If this method doesn't work, no harm has been done - you can simply re-install your old SD card.

Perhaps someone else on the forum has a better method.
zuperman
Posts: 8
Joined: 11 Jan 2022 00:16

Re: Upgrade v2.3.8 to v3.2.2

Post by zuperman »

Hello

Got a new SD card and installed v3.2.2 (https://razberry.z-wave.me/z-way-server ... ry.img.zip). Pi is up and running again. First thing I did was restoring the network config through the Expert interface.
Now I wanted to SSH into the Pi for a simple reboot and can't login. User = pi, Password = raspberry doesn't seem to be correct?

Thanks for helping me out.
Erik
seattleneil
Posts: 42
Joined: 02 Mar 2020 22:41

Re: Upgrade v2.3.8 to v3.2.2

Post by seattleneil »

Hmm, I would have expected the default pi password (raspberry) to work. Here's a link to the generic procedure for resetting the pi password: https: //howtoraspberrypi.com/recover-password-raspberry-pi/

Note that you will need a PC to modify a file on the SD card and you'll need to plug in a USB keyboard and an HDMI monitor to your Pi.

Perhaps Z-Wave.Me has done something to change the default password. Maybe setting it to null (i.e., no passwprd, just hit enter at the prompt), set it to the GUI admin password, or decided to obfuscate the password. Regardless of the cause, the procedure described on the link will reset the pi password.
zuperman
Posts: 8
Joined: 11 Jan 2022 00:16

Re: Upgrade v2.3.8 to v3.2.2

Post by zuperman »

Hi

The steps I need to take don't seem to work.

After booting up with the changed cmdline.txt file, I execute the "mount -o remount, rw /" command. As soon as I enter, first thing I get is "raspberrypi login:". If I enter "pi" I am prompted for a password on which linux tells me "Login incorrect".

For every action I try to execute, I need to enter a password. On another site I found that entering "su" without password would be enough to start and execute commands, but also after command "su", the systems asks for a password.

I have the pi connected with keyboard and hdmi now, so I can experiment a bit more.

My guess is that somehow the latest image is written in a different way so other password is needed. (@PoltoS? - don't know how to tag someone)
Erik
zuperman
Posts: 8
Joined: 11 Jan 2022 00:16

Re: Upgrade v2.3.8 to v3.2.2

Post by zuperman »

Hi @seattleneil

Letting you know I went the "hard" way and installed a fresh Raspberry OS Bullseye. I then was able to set my own "pi" user password and then installed Z-Way by "wget -qO - https://storage.z-wave.me/RaspbianInstall | sudo bash".
Booted up, can access the web UI and can SSH into the pi.
Next step: getting my devices back in the fresh installation (hopefully just by importing the v2.3.8 backup I made in ExpertUI).

Thanks again for your guidance, much appreciated!
Erik
zuperman
Posts: 8
Joined: 11 Jan 2022 00:16

Re: Upgrade v2.3.8 to v3.2.2

Post by zuperman »

If somebody else would bump into this thread and wonders...
Importing the network backup v2.3.8 into v3.2.2 seemed to have worked (all my devices were listed with the names I gave them in the past), but I could not read or set the devices. I am now going through the process of excluding and including them all again :s
User avatar
PoltoS
Posts: 6688
Joined: 26 Jan 2011 19:36

Re: Upgrade v2.3.8 to v3.2.2

Post by PoltoS »

Please describe in more details what mean "could not read". Backup from v2.3.8 should work perfectly on v3.2.2
zuperman
Posts: 8
Joined: 11 Jan 2022 00:16

Re: Upgrade v2.3.8 to v3.2.2

Post by zuperman »

Hello PoltoS

When going into expert the first time, I could see my devices as in Device1, Device 2,... I was able to import my backed up network settings which gave my devices their aliases back, so I thought everything was going well.
First thing I noticed was even though my motion sensors picked up motion, the events did not arrive in zway. I then tried to get or set values to my wall plug and that didn't work.
Next step, I tried to interview the devices and they failed.
So I ended up excluding and including again which worked immediately. (This will be pia for the devices that are in wall sockets...)
Interviews failed mostly with security and groups item, sometimes other items. Can't get to the interviews right now as I'm on mobile.
New mobile app seems to lose its connection very often (as well on Wi-Fi or data)?

Regards
Erik
seattleneil
Posts: 42
Joined: 02 Mar 2020 22:41

Re: Upgrade v2.3.8 to v3.2.2

Post by seattleneil »

Here's what I understand about Z-Way backups. Hopefully, @PoltoS will consider filling in the blanks. Z-Way has 2 different backups. The first backup uses the smarthome UI and downloads a .zab file. The second backup uses the expert UI and downloads a .zbk file. The 2 backup files serve different purposes, with the .zbk file (expert UI) being a gzipped tar file (i.e., the same as a .tgz or .tar.gz file). It contains the config/ directory contents which includes the files Defaults.xml and Profiles.xml, the zddx/ subdirectory and the maps/ subdirectory. The zddx/ subdirectory is very important as it contains a file named something like 12345678-DevicesData.xml that holds the Z-Wave network configuration data (where 12345678 is the Z-Wave Home ID). This xml file contains the node IDs, command class information, and routing information for every Z-Wave device in the Z-Wave network. With this file, I think it's possible to have full functionality of the Z-Wave network, but only through the expert UI (i.e., without support for the upper-layer Z-Way features such as virtual devices and automation). The maps/ subdirectory nominally contains the jpg image for the floorplan that you may have uploaded as the route map image.

There's a partial mystery surrounding the .zab backup (smarthome UI). Its contents are json format with a "data" tag where the value is a base64 encoded json file with tags "config.json", "moduleTokens.json", "zway_expertconfig.json", "userIcons.json", "__userModules" and "__ZWay". The "config.json", "moduleTokens.json", "zway_expertconfig.json", "userIcons.json" data values are the contents of the corresponding files under /opt/z-way-server/automation/storage. The "__userModules" are the contents of the /opt/z-way-server/automation/userModules directory. The mystery is with the "__ZWay" data value. This data looks to be UTF-16 encoded where the decoded data is binary data and may be obfuscated or possibly encrypted. I'm hoping someone can shed light on the __ZWay data value. For example, does the __ZWay data value contain most or all of the DevicesData.xml information? Is anyone else concerned 349 kBytes of unknown data are being stored on Z-Wave.Me servers?

Erik - Thanks for sharing your frustrating experience. Perhaps it will motivate Z-Wave.Me to update their installation documentation. My personal experience was to install the Z-Way software using the same approach you took - install Raspberry Pi OS first and then add the Z-Way software by "wget -qO - https://storage.z-wave.me/RaspbianInstall | sudo bash". It will be interesting to understand the solution to the pi password problem you encountered when following the Z-Wave.Me recommended approach.

I suspect your modification to the boot command line had some sort of error - if the init program was /bin/sh, you would not see a login prompt.

The explanation about the 2 backup files is why the restore was only partially successful - you only restored the lower-layer Z-Wave network information. To restore full Z-Way functionality, both backup files (smarthome .zab and expert .zbk) need to be restored. Unfortunately, since you've excluded some or all of your devices, I doubt your backup files will work since the devices in your network have new IDs. IMHO, the frustration you've experienced speaks to the need for updated documentation. Perhaps something as simple as having the expert UI backup and restore function explain the need for the smarthome UI backup/restore (and vice-versa).
Post Reply