Read value from XML document

[pz1]

I am in the process of moving some elements of my Solar Energy monitor to RaZberry. This involves reading values from XML documents with JavaScript. Almost all tutorials on that topic take it from the webbrowser perspective. For this application I have to do it on the server side. My xml file is simple:
<response>
<power>1817</power>
<temp>50.2</temp>
<response>

Sofar I only found E4X as a potential solution, but that route seems to be deprecated.
Edit: In the meantime I found xmldoc by Nick Farina (http://nfarina.com/post/34302964969/a-l ... javascript). It looks as if it depends on node.js, but that does not become clear to me. It also mentions depency on Sax, but does desribe how. At least not for my novice eyes.


Is there somebody in this community who can give me a hint how to proceed?

Thanks,
Pieter

[pz1]

With help of the guys of Stackoverflow I managed to get the xml element extraction working with the following bash script xmlparse.sh:

#!/bin/sh
# $1 is filename without .xml extension. $2 is tagname without surrounding brackets
grep -E -m 1 -o "<$2>(.*)</$2>" ./private/$1.xml | sed -e "s,.*<$2>\([^<]*\)</$2>.*,\1,g"

This script is stored in the automation subdirectory, the xml files are supposed to be in the ./private directory. Next I defined in the the Java script file my_openremote.js the following function:

GetxmlElement = function(file,tag) {
var test=system("/opt/z-way-server/automation/xmlparse.sh " + file + " " + tag);
return test;
}


Whereas the bash script nicely returns the power value of 1817, the js function if called from chrome:

http://raspberry_IP:8083/JS/Run/GetxmlElement("pvlogger","power")

does return [0,""], where I had expected [0,"1817"]

edit1:
1) I also get [0,""] if I only do the first part before the pipe.
2) If the bash script has only one line saying:
echo "abc"
I do get abc returned in the browser. So the overall mechanism seems to work OK.

edit2:
If I do change the script, to let echo the result:

#!/bin/sh
# $1 is filename without .xml extension. $2 is tagname without surrounding brackets
power=$(grep -E -m 1 -o "<$2>(.*)</$2>" ./private/$1.xml | sed -e "s,.*<$2>\([^<]*\)</$2>.*,\1,g")
echo $power

I do get [0,"
"], but not the value 1139 that the script returns on the commandline.

Apparantly I am overlooking something with the outputs of grep and sed

Any one out here who has a clue?

[pz1]

As this thread has shown sofar, I got stuck on parsing values from an XML file with earlier versions of Z-Way. The System() call obviously won't work

My source of solar production data is only available in a XML format like:

Code: Select all

<response>
 <power>1817</power>
 <temp>50.2</temp>
<response>

I think the OpenWeather module looks like a suitable template to derive a virtual device for my solar logger.
Is there a XPath (or equivalent) function available to do the extraction?

[pofs]

It it rather easy to do with ZXmlDocument (available since v1.5):

Code: Select all

var doc = new ZXmlDocument("<response><power>1817</power><temp>50.2</temp></response>");
var result = [ 
  parseInt(doc.findOne("/response/power/text()")), 
  parseFloat(doc.findOne("/response/temp/text()")) 
];

Here's how your xml document looks like in JS:

Document model

Screen Shot 2014-06-02 at 13.06.06.png (52.74 KiB) Viewed 11645 times

Note that if you request data with http.request(), and response (or enforced) mime type is xml (application/xml, text/xml etc.), then result will be automatically parsed into ZXmlDocument. You'll just need to fetch required values.

[pz1]

Note that if you request data with http.request(), and response (or enforced) mime type is xml (application/xml, text/xml etc.)

Thank you for the quick response. I remember PoltoS mentioned it to me, but I lost it, because I did not understand it at the time. I am doing a crash course on learning this more advanced JS, GiT etc. So please be gentle.

Actually I have to fetch the the xml every 5 minutes or so from the pvlogger with url
http://pvlogger/status.xml. I am not a trained programmer so I have one more question at the moment.
In the example the xml is pasted directly in the code. How does the code change if I read from the url?

PS: the complete XML is shown below. For the moment I'll only use gauge_power and energy_today

Code: Select all

<response>
 <gauge_power>827</gauge_power>
 <gauge_temp>39.4</gauge_temp>
 <gauge_vpv>279.0</gauge_vpv>
 <gauge_iac>3.7</gauge_iac>
 <energy_today>3.196</energy_today>
 <energy_total>5822.8</energy_total>
 <hours_total>7911</hours_total>
 <time_stamp>20140602 11:29</time_stamp>
</response

[pofs]

Something like:

Code: Select all

http.request({
    url: "http://pvlogger/status.xml",
    method: "GET",
    async: true,
    success: function(response) {
        if (response.parseError) {
            console.log("failed to parse data: " + response.parseError);
            return;
        }
        var doc = response.data; // it is already ZXmlDocument
        var gauge_power = parseFloat(doc.findOne("/response/gauge_power/text()"));
        var energy_today = parseFloat(doc.findOne("/response/energy_today/text()"));
        console.log(gauge_power, energy_today);
    },
    error: function(response) {
        console.log("failed to fetch data: " + response.statusText);
    }
});

Cron-related code might be taken from BatteryPolling module.

[pz1]

Thanks, I'll try this later today or tomorrow.
For my understanding: Don't I have to mention the mime type somewhere? The xml that I receive from the device is plainly as I posted it. They even didn't add a minimal XML header:

Code: Select all

<?xml version="1.0" encoding="UTF-8"?>

EDIT 2015-01-12: The latest version of my PVlogger software does add this xml header now

[pofs]

In most cases it will be returned by server, so there's no need to specify it.
For those rare cases when content type is not returned (or returned incorrectly), you may add

Code: Select all

contentType: "text/xml"

to the request.
XML header is not required.

[pz1]

02 Jun 2014 11:56
Something like:

Code: Select all

http.request({
    url: "http://pvlogger/status.xml",
    method: "GET",
    async: true,
    success: function(response) {
        if (response.parseError) {
            console.log("failed to parse data: " + response.parseError);
            return;
        }
        var doc = response.data; // it is already ZXmlDocument
        var gauge_power = parseFloat(doc.findOne("/response/gauge_power/text()"));
        var energy_today = parseFloat(doc.findOne("/response/energy_today/text()"));
        console.log(gauge_power, energy_today);
    },
    error: function(response) {
        console.log("failed to fetch data: " + response.statusText);
    }
});

Sorry, a bit late with my response
I merged this code with the framework of the OpenWeather module, to create virtual devices that I can re-use elsewhere:

Code: Select all

// ----------------------------------------------------------------------------
// --- Module methods
// ----------------------------------------------------------------------------

PVLogger.prototype.fetchSolar = function (instance) {
	var self = instance;

	http.request({
		url : self.config.url,
		method : "GET",
		async : true,
		success : function (response) {
			try {
				var doc = response.data; // it is already ZXmlDocument
				gauge_power = parseFloat(doc.findOne("/response/gauge_power/text()"));
				energy_today = parseFloat(doc.findOne("/response/energy_today/text()"));
				console.log(gauge_power, energy_today);

				self.vDev.set("metrics:level", gauge_power);
				self.vDev.set("metrics:energy", energy_today);
			} catch (e) {
				self.controller.addNotification("error", "Can not parse Solar information", "module");
			}
		},
		error : function () {
// The following line is commented, because the logger is not available at night. Uncomment for initial testing.
//			self.controller.addNotification("error", "Can not fetch Solar information", "module");
		}
	});
};

The values for power and energy are correctly written to the log file. Sofar so good for the XML parsing!

>>>> edit 20150226:1457 Made a mistake. Edited post from here <<<<
Unfortunately, something seems to go wrong with the Virtual Device creation.

Code: Select all

http://rasp_IP:8083/JS/Run/controller.devices.get("PVLogger_17").get("metrics")

produces

Code: Select all

 {"scaleTitle":"W","title":"Solar Power","probeTitle":"Watt","gauge_power":473,"energy_today":0.548,"power":614,"energy":5}

I am surprised by getting the sensors twice, and with different values. I did not expect to get gauge_power and energy_today

Full code of index.js, and module.json are in the attached ZIP (REmoved)

[pz1]

Unfortunately, something seems to go wrong with the Virtual Device creation.

Code: Select all

http://rasp_IP:8083/JS/Run/controller.devices.get("PVLogger_17").get("metrics")

produces

Code: Select all

 {"scaleTitle":"W","title":"Solar Power","probeTitle":"Watt","gauge_power":473,"energy_today":0.548,"power":614,"energy":5}

I am surprised by getting the sensors twice, and with different values. I did not expect to get gauge_power and energy_today

After upgrade to 2.0.1 RC16 I do get as expected:

Code: Select all

{"scaleTitle":"W","title":"Solar Power","probeTitle":"Watt","level":2797,"energy":4.6}