Step 1: installing HouseAgent prerequisites

The first step is to install the HouseAgent prerequisites, there are two (Erlang and RabbitMQ) but you can install them using a single command line.
Start the terminal in Ubuntu:

Now enter the following command:

Hit “Y” to continue the installation.

Step 2: Installing HouseAgent

Download “HouseAgent.tar.gz” from the HouseAgent forum pages, and put it in a location of your liking. I chose HouseAgent in my home directory.
Now extract HouseAgent:

After extracting you need to copy the configuration file to /etc, you can do this with the following command line (you need to sudo this):

Step 3: Starting HouseAgent

You are now ready to start HouseAgent. You can do this by running: “./Houseagent” :

After HouseAgent has been started, point your browser to http://localhost:8080:

Congratulations, you have successfully installed HouseAgent! In tomorrow’s blog post we will look into installing the first plug-in in order to make HouseAgent actually useful.

That was the short introduction of something I have been working on for quite some months already. HouseAgent has been running as my own home automation software for some time.
I hear you asking, why not use one of the standard software products available? There are a couple of reasons for that:

• Most software is expensive, for a full set-up €500,- is no exception;
• I wanted to learn from my home automation hobby, by writing my own software I got and still get a lot of insight on new frameworks/protocols/techniques etc;
• Really, I wanted free home automation software to be available to others out there as well!
• Most software out there is not designed to run on a low power/low profile systems. HouseAgent can be run on low power system without hassle.
• Also most software only runs on one platform i.e. Windows or Linux. I believe in a truly open platform where all kinds of developers can contribute.

Let’s look at some core components used in HouseAgent:

AMQP/RabbitMQ
One of the core components used in HouseAgent is AMQP. The Advanced Message Queuing Protocol (AMQP) is an open standard application layer protocol for message-oriented middleware. The defining features of AMQP are message orientation, queuing, routing (including point-to-point and publish-and-subscribe), reliability and security.
AMQP is used within HouseAgent for plugin communication. Here is a picture, which hopefully explains all the bits and pieces a bit better:

As you can see all plug-ins are separate isolated components, in fact the plug-ins don’t even know about the coordinator or each other. The coordinator is the main HouseAgent application which runs important features such as the event handler and the web-based GUI. A plug-in communicates with the broker, and the message protocol used for this is AMQP.
Certain types of messages are “published” on the broker, for example a device_value_update message might be send to the broker by a plug-in. The coordinator is “subscribed” to such messages and handles them appropriately (i.e. run a certain action based upon an event or save history to a history file)
The coordinator on it’s turn can publish messages to the broker as well, commanding the plug-ins to do certain action. One example would be the power_on message. This message commands a certain plug-in to turn on a device with a certain address.

So what’s RabbitMQ? RabbitMQ is a broker which implements AMQP, there are several other AMQP implementations which might work as well with HouseAgent (not tested!). You can read more about AMQP on it’s wikipedia page. You can read more about RabbitMQ on it’s homepage.

Twisted (Matrix)
Twisted is used for serial communication and all networking communication in HouseAgent. Twisted is a networking engine written in Python, supporting numerous protocols. It contains a web server, numerous chat clients, chat servers, mail servers, and more. The main difference between Twisted and other networking engine’s is that Twisted is asynchronous. You can read about the differences between synchronous and asynchronous communication here. Twisted certainly was an eye opener for (thanks Rene) but also required a completely different way of thinking about networking and communication in general.

Among these core components some other frameworks/packages are used:

• Mako Template (for web page templating)
• PyRRD (for historic graphing)
• py-serial (serial communication)
• pywin32 (Windows only)

Although the development of HouseAgent is just getting started, it already includes some nice features. Some core functionality HouseAgent offers right now:

• Storing and logging of device values
• Graphing (based upon RRDtool)
• Event handling (triggers include: time based, device value changes)
• Flexible plug-in development and easy extension

This concludes the short introduction of HouseAgent. In future blog posts I will dive deeper in different aspects of HouseAgent.
Right now a group of beta testers has been selected, I will send out a first beta version a.s.a.p. after that a public version will come available.
As of today the source code is available for the core of HouseAgent! You can review the source code, and watch the development progress on Github. But please don’t expect a complete system out that source code, stand-by for binary releases.

In the Plugwise protocol unleashed series, I will describe the plugwise protocol. As far as I have been able to re-engineer it. I did this re-engineering work because there is no Linux/OSX software available to control your plugwise devices. All information in these series are based on the 2.0 firmware version!

DISCLAIMER: use of the information in these articles is at your own risk.
This information is not supported by Plugwise B.V. in any way.

In this part of the series we will have a look at the device information request and response.

Circle internal clock

This blog post is about the internal clock of the Plugwise circle. This internal clock is used to for example log information about kWh usage per hour.
It’s important to keep the internal clock in sync with your PC, for correct data readings. Imagine this going wrong: logged kWh information about an device appears in a different hour. Not something you want.

So, how to test this? Well I just fiddled around a bit with my PC’s date and time.
Using a serial port sniffer I was able to see the following (unknown) data:

Image 1: serial port sniffing log after the clock change.

Following after that is the following data:

Image 2: more data following the clock change.

So looking at this I realized that code 003E must be some kind of extra check, to see if the clock is out of sync.
The 0016 must be the clock set command as it contains extra data in the command.

The clock information request (003E)

The request is fairly easy:

003E000D6F000055XXXXBF07

003E -> This is the request command code.
000D6F000055XXXX -> This is the mac address of the circle.
BF07 -> This is the CRC16 checksum

The response looks like this:

003F0D52000D6F0000552XXX1325350401457A0EFF

Split up for readability:

003F 0D52 000D6F0000552XXX 13 25 35 04 01 457A 0EFF

003F -> request return command code.
0D52 -> reverse checksum
000D6F0000552XXX -> mac address
13 -> internal clock hour (HEX)
25 -> internal clock minutes (HEX)
35 -> internal clock seconds (HEX)
04 -> internal clock day of the week (HEX)
01 -> ?
457A -> ? looks like some kind of checksum to me.
0EFF -> CRC16 checksum

The clock set command

The clock set request looks like this:

0016000D6F0000551C140A021501FFFFFFFF112536043B11

Split up for readability:

0016 000D6F0000551XXX 0A 02 1501 FFFFFFFF 11 25 36 04 3B11

0016 -> Command code
000D6F0000551XXX -> Mac address
OA -> Year (HEX) note this is the year in the plugwise style: 2010 = 10 (always substract 2000)
02 -> Month (HEX)
1501 -> Number of minutes (HEX)
FFFFFFFF -> Log buffer address (you can use this to reset your log buffers, more on this in later posts), FFFFFFFF won’t reset a thing!
11 -> Hour (HEX)
25 -> Minute (HEX)
36 -> Second (Hex)
04 -> Day of the week (HEX)
3B11 -> CRC16 checksum

So now we know how to read and set the internal clock of the plugwise circle.
See you guys next week!
Let me know if you have any questions, and also if there is anything specific you would like to see in the next plugwise unleashed post!

This is a multipart series, the other parts can be found here:
Plugwise protocol unleashed part 1:  introduction
Plugwise protocol unleashed part 2: device information

In the Plugwise protocol unleashed series, I will describe the plugwise protocol. As far as I have been able to re-engineer it. I did this re-engineering work because there is no Linux/OSX software available to control your plugwise devices. All information in these series are based on the 2.0 firmware version!

DISCLAIMER: use of the information in these articles is at your own risk.
This information is not supported by Plugwise B.V. in any way.

In this part of the series we will have a look at the device information request and response.

Device information request

The device information request looks like this:

0023000D6F000023XXXX231B

0023 – indicates the request code, in this case it’s the device information request code.
00D6F000023XXXX – is the mac address of the device we are requesting information about.
231B - is the CRC checksum.

Device information response

The device information response looks like this:
00240025000D6F000023XXXX0A01866F0004DFE801850000047300074AA663800136EE
0024 0025000D6F000023XXXX 0A 01 866F 0004DFE8 01 85 0000 0473 0007 4AA66380 01 36EE

0024 - indicates the response code, in this case it’s the device information response code.
0025 - unknown, reverse checksum? Not interesting anyway.
000D6F000023XXXX - mac address of the device we requested information about.
0A - Year of the internal Plugwise clock, convert 0A to integer and you get 10.. add 2000 to it and you have 2010.
01 - Month of the internal Plugwise clock, convert 01 to integer and you get 1.
866F - Number of minutes of the internal Plugwise clock, convert 886F to integer and you get 34415. You can use this to calculate the days, hours and minutes, to get a datetime value.
0004DFE8 - Latest log buffer address, each log buffer contains 4 hours of information about the power usage. Convert this to integer, Then use: (logaddress – 278528) / 32 to get the log address. In this example the log address is: (319464 – 278528) / 32 = 1279
01 - Relay status (power on/power off) 01=ON, 00=OFF.
85 - Amount of herz the plugin operates on.
I use the following python code to determine the amount of hz:

    def determinehz(self, herz):
        """
        Determines number of hz of a plugwise device.
        """
        if herz == '85':
            return 50
        elif herz == 'C5':
            return 60
        else:
            return 0

000004730007 - This indicates the hardware version of the circle. No conversion is needed. In this case the hardware version is: 0000-0473-0007.
4AA66380 - This is the firmware version of the circle. The datetime is a POSIX (epoch) timestamp. The following python function converts it to a datetime object:

datetime.fromtimestamp(timestamp[, tz])

If you have any questions regarding the device information please let me know.. till next week! And happy plugwising! :-)

This is a multipart series, the other parts can be found here:
Plugwise protocol unleashed part 1: introduction

Introduction

In the Plugwise protocol unleashed series, I will describe the plugwise protocol. As far as I have been able to re-engineer it. I did this re-engineering work because there is no Linux/OSX software available to control your plugwise devices. All information in these series are based on the 2.0 firmware version!

DISCLAIMER: use of the information in these articles is at your own risk.
This information is not supported by Plugwise B.V. in any way.

So what is plugwise anyway?

Plugwise is a smart way of measuring power usage using a device called the Circle.
It’s basically a plug between your device and the power socket.
The device can be used to keep track of the following:

- Current power usage;
- historic power usage, using buffers in the device itself;
- you can also use it to power on/power off a device;
- you can also use it as standby killer (using a schema).

For more information visit: http://www.plugwise.com

General information

Before we can dive deep into the plugwise protocol, we are first going through some basics of the protocol in part one of this series.
Plugwise uses the zigbee protocol to communicatie over the air.
Once all of your devices have been setup using Plugwise source (the default Plugwise software) you can control your devices using the plugwise stick (this is a little USB stick)

Image 1: the plugwise stick

To send commands to the Plugwise stick, plugwise uses a serial communication protocol.
Actually the Plugwise stick has a FTDI FT232 UART aboard. The following information shows up when plugged into a Linux machine:

Bus 006 Device 003: ID 0403:6001 Future Technology Devices International, Ltd FT232 USB-Serial (UART) IC

The serial communication takes place with the following settings:

• Baud rate: 115200
• Data bits: 8
• Stop bits: 1
• Parity: none

Plugwise data packet

The Plugwise data packet looks like this:

Image 2: the Plugwise protocol frame.

The start frame must be sent before sending any other data, the hexadecimal presentation of the start frame looks like this:

\x05\x05\x03\x03

The data formats varies, an example might look like this:

0017000D6F0000236XXX01

This actually is an example of the power change command (switch a device on or off, in this case a device on command)
0017 is the command, 00D6F0000236XXX the mac address and 01 the power code (power on)

Followed by the data, is the CRC checksum.
The used CRC checksum by Plugwise is a CRC16 checksum (often reffered to as ymodem or zmodem), it has the following properties:

• Polynomial: 0×11021
• Seed value: 0×00000
• Xor mask: 0×00000
• Width: 16

The checksum is generated over the data piece of the packet.
Last part of the packet is the end frame, this is basically a control feed followed by a linefeed. Which looks the following way, in hexadecimal:

\x0d\x0a

This concludes the first part of the Plugwise protocol unleashed series. If you have any specific protocol questions, please leave them in the comments so I can talk about it in the next part of this serie.