Soft RS485 UART using interrupts on ESP8266

My non technical readers can skip this :-)

Background

As you will have noticed, I have made a complete access control and alarm system based on a Raspberry Pi but working over RS485 bus connections to standard Honeywell Galaxy parts like RIO, Max Readers, and Keypads.

As you will have noticed, I have now started making WiFi connected parts for the alarm system based on ESP8266 parts (ESP-01, ESP-12F, ESP-12S). These include a small ESP-01 based board with RS485 driver that fits inside a Galaxy keypad talking RS485 to the keypad.

As you may have noticed, I have NFC readers sorted and working with secure NXP MIFARE DESFire EV1 cards. These are way more secure than poxy proxy cards used by Max Readers.

Challenge

The concern some people have raised is the reliance on WiFi. It will do the job, but it is not that hard to disrupt externally. Now, much like using a screwdriver on a Max Reader, it will set off the alarm to disrupt it, but it is perhaps a tad easier. One has to consider if it is easier than simply forcing the door or breaking a window, which is always the benchmark for any security :-) But it is a concern.

So can I make the door controller, which does the secure cards, look like a Max Reader and do RS485 to a conventional Galaxy system? I think I can...

Approach

Initially, for the keypad system, I used the built in serial drivers on the ESP8266. These are good, interrupt driven, well supported, except...

The code on the ESP8266 using Arduino is the NONOS SDK, and means one thread for all. It can be delayed a bit for all sorts of reasons, even many milliseconds. For serial that is all handled behind the scenes on interrupts this is no problem, until is it RS485!

The problem is that for RS485 you have to enable and disable the line driver around your transmission. Initially I simply set a time to disable after poking the message in the Tx queue. But that could be delayed and mean after transmitting we held the line driven for too long.

The solution was to wait in the main loop task until sent, and then disable the driver. This is not nice, as any other operations of the device are delayed, but for the keypad driver application there is nothing else. So it works well.

This will not work for the card reader as it has to, well, talk to the card reader, and inputs, and outputs for the door control, and so on. Also, if I am a slave device (unlike the keypad where I am master) I am expected to answer polls promptly and not hang around for 10ms randomly.

My plan - interrupts! Make a soft serial RS485 UART on an ESP8266 to do the grunt in the background.

UARTs

A UART (Universal Asynchronous Receive/Transmit) is a device to send and receive old fashioned serial byte / character data. The concept dates back to what? the 50s or 60s. It is simple...

Each byte, or character, to be sent is coded as a series of 1 or 0 on a bus. What 1 and 0 mean in terms of current or voltage depends on the flavour of system (RS232, RS485, RS422, etc). Each bit time (1/Baud rate) the line has a 0 or a 1. The line is idle (1) and then the byte/character starts with a start bit (0), bits in the byte (low bit first), then any parity bit, and then stop bit (or bits) which are 1.

Obviously both ends have to agree the rate (Baud rate) for bits, and byte size, and parity present and if even or odd, and even number of stop bits.

The Galaxy uses 9600 Baud (so a bit every 104⅙µs), 8 bit data, no parity and one stop bit. Simples!

Sending is easy, you need some sort of timer, and you set 1 or 0 on the output to do the start bit, data bits, and stop bit. The Arduino ESP8266 SoftwareSerial sends in real time from the command using careful timers, it seems. I plan to use interrupts.

Receiving is a tad harder. You have to find the start bit for each byte, and then sample each bit, ideally in the middle. But once you have the bits you shift in the data bits and check the stop bit is 1 (else an error). Not that hard?

Edge interrupt on rx?

Interestingly the SoftwareSerial code simply interrupts on each edge and looks at time from last edge to work out the received data (I think). That is one approach but I decided to clock in bits on interrupt - I just need to sample mid bit somehow.

My initial approach was to set an interrupt on the falling edge of the data line to set the recurring timer half a bit later. However, this has all sorts of challenges, no least of which is race conditions on a regular per-bit interrupt and the edge interrupt. I did work, but not 100%. I am sure I could have solved the race conditions, but actually there is a good reason to stick to simple regular interrupts.

So the approach I took was just a regular interrupt, and this was 3 times per bit. This means ⅔ of the interrupts just count a sub bit and return in most cases, so very light weight in terms of time. But what it does mean is that I could in fact have multiple instances of RS485 busses running on this one interrupt if all the same Baud rate. I have not yet coded that though. It is, however, nice and simple and working for one instance well.

Why 3?

The 3 times per bit is simple. For Tx it makes no odds, skip 2 out of 3 interrupts and set the line to 1 or 0 for the next bit to send. For Rx, it is the same for sampling, except when waiting for start bit. If waiting for the start bit you check for the line going 0 every interrupt, and if it does set the sub bit counter to sample one interrupt later and then every 3 interrupts.

This is not as clean as an edge interrupt adjusting the timing to the microsecond, but it does mean that we can have sampling of the centre of the bit, ±⅙ of a bit which is more than adequate. Doing 2 times bit rate is no good as you could be sampling in the middle or right at the end. Hardware UARTs usually do 16x bit rates, but 3x is fine in my opinion.

This is how I have coded soft UARTs for over 20 years dating back to PIC16C84s, realy!

Messages

The way the RS485 bus works on Galaxy systems is that messages are sent as a sequence of bytes with no gaps, and then a gap - no length indicator in the message (but there is a checksum). So I coded the system to handle this, and even, if we are a slave, auto respond with current status response.

This allows the application to work on a message level, and if it is slow, we answer polls promptly anyway with current status. The driver also does the checksum.

Interrupts on ESP8266

Now the challenge is making it work on an ESP8266. They are rather special. They are a weird mix of old and new. The ESP-12S has 4M of flash! As an embedded controller that is huge! But it has around 80k of RAM (yes, k) and I had a BBC micro with more in the 80s (32k main and 64k second processor). The fact it does TCP at all is mental, but it does, and it does WiFi too.

The usual pressures on coding are complex - embedded systems have pressures on code space, RAM, execution time, power, all sorts.

In this case interrupts are special. The processor has a cache for the code, but this cannot be used by an ISR (presumably some interrupt process handles cache fails). This means the ISR has to be in RAM.

The trick is marking the code you use for the ISR with ICACHE_RAM_ATTR attribute. The RAM used is small, so not only does the ISR have to be fast it has to be small in code space too.

Thankfully a UART is simple code in terms of space and time. Small TARDIS code, even?

This worked, well, sort of, it worked mostly if started after about a second after boot. It turns out that using digitalRead, digitalWrite and pinMode is bad as these Arduino functions are not marked ICACHE_RAM_ATTR. Instead you need to find the underlying GPIO_REG_WRITE and GPIO_REG_READ functions. That then works reliably. The GPIO access is also faster, which is good, but very processor specific.

The result is working RS485, on interrupt, on an ESP8266!

I even added a clock track (blue on that trace) for debug - toggling an output for tx and rx clocking per bit.

It works!

This was deployed as new code to talk to the keypad, but means I can also act as a slave and pretend to be a Max Reader and provide a fob ID derived from the NFC card ID when checked using AES handshake.

It is also quite important as I am using the hardware serial at 115200 Baud to talk to the PN532 NFC card reader!

P.S. I have tested now as both master talking to Keypad, and as slave, pretending to be a Max Reader.

Choosing interface for PN532 (I²C, SPI or HSU) to ESP8266

I have been working with ESP8266 controllers and NFC readers. My preferred reader is the PN532 based Elechouse reader. As I have noted before, buy from Elechouse direct as there are a lot of bad copies!

However, when connecting to the processor I have a choice of interface. It is one of the rather nice features of the PN532. There are three interface types I²C, SPI or HSU.

So how to choose...

I²C

I²C stands for Inter-Integrated Circuit, so the acronym is IIC, but someone thought it would be fun to treat letters in an acronym in the same way as algebraic variables and write II as I². Arrg!

It is a simple bus for connecting devices and uses a clock and data line. Wikipedia has details (here). Both clock and data lines have a passive pull-up resistor and are actively driven low by master or slave. The protocol is well defined, and works well. It allows multiple devices on one bus and is very popular for a lot of peripherals.

Looking at the Elechouse web site I saw this: "On-board level shifter, Standard 5V TTL for I2C and UART, 3.3V TTL SPI" which rather concerned me. The ESP8266 is all 3.3V, and I did not want to have to mess about. This immediately put me off using I²C. However, looking at the schematic, the level shifter is completely optional, in that the same two pins are used for SPI, I²C and HSU and connections have these at the 3.3V level direct from the PN532 (via 100Ω) as well as via the level shifter, so it can be used for HSU and I²C if you want.

✓ Only two wires, simplifies wiring and frees up GPIO pins
✘ ESP8266 does not have hardware I²C support, so uses more processor and is slower
✓ ESP8266 has software I²C which means almost any GPIO pins can be used for clock and data, making PCB layout much simpler.
✘ Data sheet suggested level shifted to 5V (which turns out to be optional)
✘ Both clock and data use passive pull up resistors, meaning they can be subject to more interference on long lines

SPI

SPI stands for Serial Peripheral Interface. Wikipedia has details (here). It is more complex than I²C in the way it works - using 4 wires not 2, and allowing different configurations. It is also quite fast, allowing several MHz operation.

The ESP8266 has hardware support for SPI, which is good, but the downside of that is that specific GPIO pins are used for 3 of the 4 signals, so PCB layout is more complex.

✘ Four wires, so more to connect from controller to reader and uses more GPIO pins

✓ ESP8266 has hardware support, so allows faster working

✘ ESP8266 has hardware support, so the pin assignments are fixed making PCB layout more complex

✓ Lines are driven (from master or from slave) rather than using passive pull ups

✓ 3.3V direct connection to ESP8266

HSU

HSU stands for High Speed UART. This is simple old fashioned serial communications, on a Tx and Rx line, and defaults to 115,200 Baud.

The ESP8266 has hardware support for UART, and will work at these speeds. The ESP8266 library has good, interrupt based, serial handling. Unlike SPI there is actually a choice of pins that can be used, but this is still a tad limited. Of course the ESP8266 uses the UART for other things like programming and debug, but there is an additional Tx available for debug (Serial1).

✓ Only two wires, simplifies wiring and frees up GPIO pins

✓ ESP8266 has good hardware and library support for serial UART

✘ Pin usage is a fixed, but a choice of two sets

✘ Data sheet suggested level shifted to 5V (which turns out to be optional)
✘ Serial on ESP8266 is used for programming and debug
✓ Tx and Rx are driven (one each way) rather than passive pull up resistors
✓ The same connector can use used for programming and connection to PN532, reducing number of headers needed on the board

What did I do?

I was initially put off by the level convertor comment, leaving SPI as the only option. The first project I made using an NFC reader was connected to weighing scales, and used the serial on the ESP8266 to talk to the scales, so that rules out HSU. So SPI it is.

There are 4 connections needed (so 6 wires including power and ground). This means you can use 5 pins in a row on the ESP8266 (VCC, D13, D12, D14, D16) and GND to make a 6 way connector. Sadly the pins are not quite the same order as on the Elechouse PN532 board. This just means the connectors need wiring carefully.

This works well. I then used the same design for the door controller.

Hindsight

Firstly, the fact that the level convertors are optional means I²C and HSU are an option. I could have, for example, used I²C on a smaller ESP-01 for the scales rather than using an ESP-12F and SPI. However, the use of passive pull-ups is still a concern if the reader is on any length of wire from the controller as may be the case on a door controller. I could have used HSU, as the ESP8266 libraries have a soft serial mode, which can work at the scales 9,600 Baud rate, so could have used Tx/Rx for PN532 and one of the other pins to talk to the scales serial interface, and used an ESP-01.

I have since tried HSU, which just works, and is only 4 wires to the reader (power, ground, Tx and Rx). I was concerned that it would be slower (115,200 Baud rather than 2MHz) but actually it is slightly quicker (28ms rather than 30ms for InListPassiveTarget on same card), and a lot of the time is the PN532 operation and the NFC card itself. The Baud rate can also be configured!

It also turns out that had I used the more usual GPIO15 as the SPI SS pin not GPIO16 I would have actually wired the PN532 such that it could be used for HSU or SPI, as the PN532 uses MOSI for Tx and SS for Rx and the ESP8266 alternative Tx/Rx for UART0 are on those pins!

In practice, what I have done is make a 4 pin header with GND, 3V3, Tx and Rx. This is the same order as the 4 pins on the Elechouse board, but is also a header that can be used for programming the ESP8266.

I am slightly kicking myself for not trying HSU and I²C to start with, but to be honest I was just pleased to get things working. Freeing up two extra GPIO pins is nice, as is having fewer wires to reader, and pins in the same order both ends of the cable. The libraries do, however, make it very easy to change interface. Just remember to set the switches on the Elechouse board to match what you are using.

I knocked up a library

So I have made a library from scratch today for NXP MIFARE DESFire cards using AES.

Been a long day.

But here it is: https://github.com/revk/DESFireAES

Have fun.

P.S. Updated and tested and, and made an ESP8266 version....

NXP MIFARE DESFire EV1 for access control

Earlier this week I set myself the task of understanding the NXP MIFARE DESFire cards.

This was more of a challenge than you may expect because the manufacturers don't publish the protocol (except under NDA). Why?!?

• It does not stop people getting the data, as is apparent!
• It means there is also misinformation out there which cannot easily be corrected by reference to the official manual.
• It puts people off using the product, which cannot be good for sales.
• It means subtle details in the official manual will not be known, which could lead to failures in edge cases, making the product look bad.
• It does not help security - as any secure system must not rely on secret specifications.

I have written up what I have found out so far: PDF on github

So, why not just use one of the few (unofficial) libraries out there?

• I probably will for the host application on linux anyway.
• Coding it myself helped me learn more about the cards and how they work and what they can do.
• It was fun!
• The code on my ESP8266 devices to read cards does not need what is in a library - half of it is for handling legacy formats, and DES and 2KDES and 3KDES whereas I need a few specific operations using AES only.

The result - I have managed quite a lot, and made the door control system use cards which cannot simply be copied (unlike the 125kHz proxy tags, which can be, very very easily).

I have even managed to convert from DES to AES for the master key, and crucially I have documented detailed examples of this, and the CMAC logic, and so on, in the manual I have written. A lot of what I have learned has come from other sources as listed at the top, but I have gone through testing things directly and coding stuff myself to confirm things as well.

I have a cyclic record file on the card logging usage (door ID, and timestamp). I still need to add some handling for expiry, and time period controls on the card, maybe.

I am a tad concerned over response times. I am selecting the app, authenticating, getting file list, writing a usage log record and updating a counter, and you notice the slight delay using the card, which is a concern. This may be something that can be improved, not sure. I have not worked out how much is the card, and how much is the ESP8266 doing AES!

But it is working! I am even checking the CMAC on responses to confirm no man-in-the-middle upgrade attack (making one card seem like another once it has done AES handshake).

Happy to take comments and update any mistakes in the manual, but I hope it is useful to people.

P.S. speed improvements by tweaking what operations I do and in what order. Now works very slickly.

P.P.S. I made my own linux/C library (here) and light weight ESP8266 functions (here).