Description
There is a desire for 3rd-party devices (controllers) to send sensor data to Kaizen for use in analytics. Doing so is actually very straight-forward. The key is for the 3rd-party device to mimic enough of a Coppercube’s behavior so that Kaizen’s Intake system recognizes the device’s data as sensor data and can determine the building it belongs to so that it can be written into the Kaizen Trend database.
There are several items involved in getting this mimicry to succeed. The device needs:
- to connect to the Internet – to connect to Kaizen.
- an Ethernet MAC address – to identify itself to Kaizen.
- to speak AMQP (advanced message queuing protocol) – to send its data
- to format its data in JSON (as detailed below)
- to organize its data along with BACnet concepts and guidelines.
Then Kaizen needs:
- to be informed of the device’s existence.
- a client and building created to receive the device’s data.
Once Kaizen is configured to receive the data, and the device has data to send, then the device:
- may send data as often as needed. Data should be sent in batches. Typically these would be every 4 hours with a maximum of once per minute, and a minimum of weekly).
- data may be sent out-of-date-order.
- data may be repeated.
- data may be sent for past dates.
- data describing the sensor data (aka meta-data) (see below) should be sent daily.
- [recommended] Status (Heartbeat) information about the device’s operating status should be sent every 15 minutes (via HTTP)
Concepts
JSON – Kaizen uses JSON as its native data format. See: https://en.wikipedia.org/wiki/JSON
JSON (JavaScript Object Notation), is an open standard format that uses human-readable text to transmit data objects consisting of attribute-value pairs. Kaizen uses it to store and transmit data (internally & externally), as an alternative to XML.
BACnet – Kaizen expects to receive BACnet Object data and BACnet Trend log data. See: https://en.wikipedia.org/wiki/BACnet.
BACnet is a standard that was developed from within the HVAC industry to able to provide communication between devices. It views sensor data as a series of time-value pairs – called Trend Logs. It also views the meta-data that describes the trend logs, as objects (sets of key-value pairs). BACnet defines many types of objects and many services for accessing those objects. For our purposes, only the Device (DEV) object, the Trend Log (TL) object, and the trend log contents are of interest. Objects are grouped (and identified by) the controller that contains them. An HVAC controller contains only ONE Device object. It may contain an unlimited number of trend logs, each one defined (and described by) a TL object. The trend logs contain a history of the sensor values; sampled at frequency and times of the controller’s choosing.
* NOTE: there is no requirement that your device uses BACnet, only that the data send to Kaizen looks like BACnet data.
AMQP – Kaizen expects to receive data via AMQP. See: (https://en.wikipedia.org/wiki/Advanced_Message_Queuing_Protocol).AMQP is a protocol for sending messages between queues on different systems across the Internet. It views communications as sessions, occurring on channels, and targeting specific queues. For us, your device is the sending client (producer), and Kaizen is the receiving server (consumer). Kaizen uses AMQP extensively. It exposes several public queues for the receipt of incoming data. It uses private queues to move data around internally.
Trend Log Data
Kaizen expects to see Trend Log messages in the following format on its AMQP input queue (cta.trendlogs) at amqps://intake1.coppertreeanalytics.com:5671Note any number of samples, for any time period within the day, can be sent at any time, any number of times, and in any order; without restriction. Kaizen keeps merging the messages to build complete day_buckets. Typically we send 6 or more messages per TL per day. If you want to correct or replace data, just send another message containing the replacement data.
{
"mac": "000bab39f41e", # Ethernet MAC address of sending !CopperCube
"d" : 100000, # BACnet device# [1- 4194303; 0x1- 0x3fffff]
"dt" : "2014-10-20", # Building local date of the samples
"i" : 2, # BACnet Trendlog Instance# [1- 4194303; 0x1- 0x3fffff]
"l" : "building name", # User assigned name
"rev" : 4, # internal use. ignore
"t" : "TL", # BACnet type mnemonic. TL-trendlog
"ts" : ISODate("2014-10-20T15:46:18.000-08:00"), # local epoch timestamp when data sent
"tt" : "Polled", # Type of trend. Polled || COV (change-of-value)
"uf" : 14400, # TL collection frequency (in secs). i.e. TL is retrieved every 4 hrs. NOT the TL Polling freq (i.e. 15 min.)
"00:00:00" : 22.94293403625488, # sample data. This trend is - recording an Analog Input every 15 min.
"00:15:00" : 22.92698097229004, # Note: time of sample. (Date is redundant & stored only once in 'dt' to save space).
"00:30:00" : 22.90200424194336,
"00:45:00" : 22.87795639038086,
"01:00:00" : 22.86405563354492,
…
"23:00:00" : 22.47274589538574, # Note: all times are originating controller's local time in 24 hour format. UTC is not used.
"23:15:00" : 22.45730209350586, # Too many issues with timezones, time conversion libraries, etc.
"23:30:00" : 22.44637489318848, # Building local time is all that matters.
"23:45:00" : 22.43587493896484,
}
Trend Log (TL) Object
{
"mac" : "003018c3184a", # Ethernet MAC of device sending this report
"l" : "BRWHQ", # Device assigned building identifying string
"d" : 260047, # Device assigned controller identity number owning the trend log
"t" : "TL" # BACnet memonic for trend log
"i" : 1, # TL identifier within the controller
"ts" : ISODate("2014-12-05T05:06:51.045-08:00"),
"LogInterval" : 3000, # controller's sampling frequency (in seconds)
"InputRef" : "260001.AI1.Present_Value", # I/O point in controller whose values are being sampled
"Out_Of_Service" : false,
"Object_Name" : "ANALOG INPUT 1 (Percent) (260001.AI1.Present_Value)",
"Log_DeviceObjectProperty" : "260001.AI1.Present_Value",
"Manual_Override" : false,
}
Device (DEV) Object
{
"mac" : "003018c3184a", # Ethernet MAC of device sending this report
"l" : "BRWHQ", # Device assigned building identifying string
"d" : 260047, # Device assigned controller identity number owning the trend log
"t" : "DEV" # BACnet memonic for trend log
"i" : 1, # TL identifier within the controller
"ts" : ISODate("2014-12-05T05:06:50.977-08:00"),
"System_Status" : "Operational",
"Vendor_Identifier" : 0, # Vendor's assigned BACnet vendor ID
"v" : 0, # Vendor's assigned BACnet vendor ID
"Object_Name" : "BRWHQ TR-1",
"Application_Software_Version" : "1.0",
"Vendor_Name" : "your company's name",
"Model_Name" : "your device's model name",
}
Heartbeats
Heartbeats are optional status reports a device may send to inform Kaizen (and more importantly, CopperTree’s Support Staff) of its operational status. At a minimum, a heartbeat shall contain the device’s make, model, some hardware statistics, and some date transmission statistics. Additional fields may be included in the device’s option.
Note: A CopperCube sends more detailed internal information to aid Customer Support with client configuration issues.
Heartbeats are JSON documents sent (via HTTPS) to https://kaizen.coppertreeanalytics.com/device_monitoring/heartbeat/post_heartbeat/heartbeat.json
{
"mac": "00500807d892", # Device identifier (Ethernet MAC)
"model": "CopperCube-XL-SQL", # Basic device Make, Model, and last reboot time
"version": "1.21.1403",
"uptime": "2015-07-01 01:39:58",
# Data transmission statistics
"samples": {"success": 9650, "failure": 26 }, # #of trend log reports sent (today). #of retries encountered
"objects": {"success": 17091, "failure": 0 }, # #of meta-objects sent (today). #of retries encountered
# basic hardware statistics.
"cpu": 11, # CPU load (in %)
"memory": {"used": 1015, "tot": 1989}, # RAM (in MB)
"disk": {"used": 13, "tot": 54} # Disk space (in GB)
}
Device Constraints
- Data collection algorithm – no constraints.
- Internal database – no constraints.
- Data volume (transmitted): no limit.
- Rule-of-thumb guidelines:
- Trends: Devices sending data for 5000 Trends are normal.
- Objects: 10 Trends per device. 500 devices per building are very common.
- Data frequency (transmitted): Trend updated every 4 hours and BACnet Objects once daily is normal.
- Data frequency (sampling rate): max: 1 minute. min: none. typical rates: 15 min, 1 hour, and daily.
- TL types: Polled or COV (change-of-value).
