OpenRemote does not authenticate with ZWay

[pz1]

@pofs

Could you have a look at this. If I understand Marcus correctly they have a two step approach. First try a connection without authentication, and second if ZWay has returned http 401, they send a request with basic authentication. According to Marcus that is how the protocol should work.

I did get this answer from Marcus on the OpenRemote forum:

I did some investigation and the situation is as follows:

1) If the username is not given then no authentication will be performed at all (empty username is not supported)

2) Our library first tries the connection without basic authentication. If that fails (HTTP 401 given by server) then the authentication header is added and the request is performed a second time. I confirmed this with tcpdump.

It looks like razberry is not following the HTTP specs. They send a "403 forbidden" even if no authentication information is provided.
They first need to send a "401 unauthorized" and only if the wrong user is given they are allowed to send "403 forbidden".

Thank you for your attention

[pz1]

I did a bit further digging into Basic Authentication on Wikipedia.
It says that the server side should return a HTTP 401 Not Authorized status.

The following log is from a OpenRemoteHelper call. It does return error 403 instead, which means that the OpenRemote client does not take the next step, i.e. sending the authorisation credentials:

Code: Select all

16:28:56.009233 IP (tos 0x0, ttl 64, id 17915, offset 0, flags [DF], proto TCP (6), length 217)
    DS212.58303 > raspberry.fritz.box.8083: Flags [P.], cksum 0x8459 (incorrect -> 0xbbab), seq 28134605:28134770, ack 2268132064, win 183, options [nop,nop,TS val 16517429 ecr 287195428], length 165
E...E.@.@.p....!..........L..0.......Y.....
..      5..A$GET /OpenRemote/SwitchBinaryStatus/9/0 HTTP/1.1
User-Agent: OpenRemoteController
Content-Type: application/json
Host: raspberrypi:8083
Connection: Keep-Alive


16:28:56.031561 IP (tos 0x0, ttl 64, id 46021, offset 0, flags [DF], proto TCP (6), length 271)
    raspberry.fritz.box.8083 > DS212.58303: Flags [P.], cksum 0xa826 (correct), seq 1:220, ack 165, win 470, options [nop,nop,TS val 287195431 ecr 16517429], length 219
E.....@.@..........!.....0....Mr.....&.....
..A'..  5HTTP/1.1 403 Forbidden
Connection: close
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true
Content-Type: text/plain
Content-Length: 17
Transfer-Encoding: chunked

11
Permission denied
0

Can you confirm that this is intended behaviour? (And as a consequence this makes the OpenRemoteHelpers useless for secure applications?)

[pz1]

TCPDUMP of OpenRemote ZWay authentication
With v2.1.2-rc-17 still the same problems, though the TCPDUMP looks slightly different from before.

Code: Select all

DS212> tcpdump -A -v -s 10240 'tcp port 8083 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 10240 bytes
15:38:44.400120 IP (tos 0x0, ttl 64, id 53660, offset 0, flags [DF], proto TCP (6), length 217)
    DS212.56798 > raspberry.fritz.box.8083: Flags [P.], cksum 0x8459 (incorrect -> 0x73aa), seq 785799336:785799501, ack 3299199293, win 183, options [nop,nop,TS val 191007037 ecr 51423], length 165
E.....@.@......!..........X....=.....Y.....
.b.=....GET /OpenRemote/SwitchBinaryStatus/9/0 HTTP/1.1
User-Agent: OpenRemoteController
Content-Type: application/json
Host: raspberrypi:8083
Connection: Keep-Alive


15:38:44.425443 IP (tos 0x0, ttl 64, id 483, offset 0, flags [DF], proto TCP (6), length 269)
    raspberry.fritz.box.8083 > DS212.56798: Flags [P.], cksum 0x75ba (correct), seq 1:218, ack 165, win 470, options [nop,nop,TS val 51425 ecr 191007037], length 217
..@.@..z.......!.......=..YM....u......
.....b.=HTTP/1.1 401 Unauthorized
Connection: close
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true
Content-Type: text/plain
Content-Length: 13
Transfer-Encoding: chunked

D
Not logged in
0

DS212 is the Synology box where the OpenRemote service runs. I don't see any of the handshaking that Marcus refers to. (two posts above this one)

TCPCUMP of OpenRemote KODI authenthication
As a reference I give here the trace for the working Open Remote - Kodi(XBMC) authentication steps

Code: Select all

DS212> tcpdump -A -v -s 10240 'tcp port 8080 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 10240 bytes
14:39:43.990946 IP (tos 0x0, ttl 64, id 45101, offset 0, flags [DF], proto TCP (6), length 206)
    DS212.52627 > Fetuut.fritz.box.webcache: Flags [P.], cksum 0x8455 (incorrect -> 0xf5c5), seq 2906046474:2906046628, ack 1501338075, win 183, options [nop,nop,TS val 17788962 ecr 59536], length 154
E....-@.@..h...!...#.....6.
Y|.......U.....
..p"....POST /jsonrpc HTTP/1.1
User-Agent: OpenRemoteController
Content-Type: application/json
Content-Length: 0
Host: fetuut:8080
Connection: Keep-Alive


14:39:43.991434 IP (tos 0x0, ttl 64, id 29004, offset 0, flags [DF], proto TCP (6), length 194)
    Fetuut.fritz.box.webcache > DS212.52627: Flags [P.], cksum 0x0520 (correct), seq 1:143, ack 154, win 235, options [nop,nop,TS val 59536 ecr 17788962], length 142
E...qL@.@.EU...#...!....Y|...6....... .....
......p"HTTP/1.1 401 Unauthorized
Content-Length: 0
Connection: close
WWW-Authenticate: Basic realm=XBMC
Date: Wed, 25 Nov 2015 13:39:43 GMT



14:39:43.994825 IP (tos 0x0, ttl 64, id 46501, offset 0, flags [DF], proto TCP (6), length 241)
    DS212.52628 > Fetuut.fritz.box.webcache: Flags [P.], cksum 0x8478 (incorrect -> 0xd9fd), seq 2916992386:2916992575, ack 488566679, win 183, options [nop,nop,TS val 17788962 ecr 59537], length 189
E.....@.@......!...#.................x.....
..p"....POST /jsonrpc HTTP/1.1
User-Agent: OpenRemoteController
Content-Type: application/json
Content-Length: 0
Host: fetuut:8080
Connection: Keep-Alive
Authorization: Basic a29kaTpiYWNo


14:39:43.995282 IP (tos 0x0, ttl 64, id 1799, offset 0, flags [DF], proto TCP (6), length 236)
    Fetuut.fritz.box.webcache > DS212.52628: Flags [P.], cksum 0xd4c8 (correct), seq 1:185, ack 189, win 235, options [nop,nop,TS val 59537 ecr 17788962], length 184
E.....@.@..p...#...!...........?...........
......p"HTTP/1.1 200 OK
Content-Length: 76
Content-Type: application/json
Date: Wed, 25 Nov 2015 13:39:43 GMT

{"error":{"code":-32700,"message":"Parse error."},"id":null,"jsonrpc":"2.0"}
^C
4 packets captured
4 packets received by filter
0 packets dropped by kernel

[pofs]

Probably because it lacks WWW-Authenticate header.

But I don't see any good reason for being so paranoid in following ancient standards. Even curl (which looks more trustworthy to me than some half-baked java library they use) doesn't follow it.

When you have explicitly told the client you want to use username and password, why don't just send it away as curl does? If there's no authentication (or it differs from Basic), the header will be simply ignored by server (or it will give you 401). So why not just send it and save one request (and cut down latency by half) in case of valid auth?

I highly doubt OR supports anything besides Basic authentication, so getting supported auth scheme doesn't seem like a valid excuse to me.

[pz1]

Got this answer from Marcus Redeker from OpenRemote:

openremote is using apache http client library which is a de facto standard in the java world. I never had any issues with this and I did a lot of java projects who use this.
OpenRemote is using version 4.0.1 which is a littel outdated since 4.5 is the current version but on their website they mention rfc2617: https://hc.apache.org/httpcomponents-client-ga/

So Marcus and I updated my OpenRemote controller to the current version of the latest Apache library. In addition we had set up a Kodi (formerly called XBMC) server. OpenRemote authenticated and communicated fine with the Kodi server both in the unpatched and in the patched version. Unfortunately in both situations the authentication and communication with ZWay does not work.

Most of the problems with this, that OpenRemote users have reported on their forum, have been solved. In a few cases this has not happened, but there the other side was ill defined, or the users did not answer the questions of the experts.

Imho the Zway implementation does not comply with the International Engineering Task Force (IETF) RFC2617 HTTP Authentication: Basic and Digest Access Authentication standard. Github #299
That is the cause of the problems in the communication with OpenRemote.