Reverse engineering

Reverse engineering can be fun - and I usually end up doing it where a supplier does not provide linux based drivers for something.

This is the case with the card printer we use. We have gone through many card printers over the years, including an Evolis pebble, a Zebra, a Matica printer and now a newer Matica printer. Each one lacked any linux drivers. What is also a shame is they did not publish the interface so someone could write some.

This is where regulation could be an interesting option - publishing specs to allow interworking of systems. There is much debate on this, and something for another blog.

However, with some debugging and reverse engineering, I have working drivers for all of the card printers.

Why Matica

The XID9300 and XID8600 are "retransfer" printers - they print on a transfer film, and then transfer to the card. This works really well on SIM cards. The Zebra was too, but did not like cards with gaps or contacts (e.g. around SIM), but the Matica is really good. Retransfer also means proper edge to edge with over the edge bleed on artwork.

Not your typical printer

Usually a printer is write only - you send a print job, it does it, job done. But the card printing is a little specialised. You may want to load a card to a "contact station" (e.g. a SIM) or "contactless (RFID) station", or mag stripe encode or mag stripe read. The way these contact stations work is a pain, but seems pretty industry standard - the printer itself takes a hands-off approach - you tell it to put the card on the station and then talk to something else (over USB) to "encode" the card.

It is also not quite as simple as just sending the print job, even when not encoding a card - you have colour and black, and you want control of each separately (the black is 1 bit per pixel and crisp solid black on top, but the colour is 8 bits per colour). You also have the fact the card has two side, and can even have a UV fluorescent layer. So you need to be able to send "layers" and "sides".

What I wanted was a way to do all of this but over Ethernet from remote systems. We have many systems (like our control pages) from which we need to send card print jobs and encode cards.

I decided to use a TLS over TCP link and send back to back JSON objects each way. It worked well as a system. I also decided that image data to print should be sent as PNG images coded as base64 data URLs in the JSON. This worked surprisingly well, and I have a handle on how the png linux library works now (an annoyingly complex library).

The interface also allows exchange of hex data strings to send to/from a contact or RFID card allowing any "encoding" I wish. This is the key thing that the client does not have to talk to something separate to "encode" cards. It also allows me to hold a job all the way and not have other jobs sneak in whilst trying to encode cards (which happened before).

Ethernet

The previous printer, in this case a Matica XID9300, has Ethernet or USB. I had working drivers from long ago that just did printing, and separately had scripts to talk over USB to the cards. Not ideal.

The new printer is a later model Matica XID8600. This too has Ethernet or USB, but instead of being a setting in a menu it picks one. If USB is connected on power up it is USB. If you want to use Ethernet you must not have USB connected (has an internal USB hub). But if you want to encode cards you have to have USB connected, and if you want to do what we did on the XID9300 (Ethernet print, USB for encoding) you have to unplug USB during power up every time - WTAF?

This was only really apparent after I had written some nice new Ethernet based drivers - a combined system to print over Ethernet and talk to contact stations over USB, all using this TCP TLS JSON connection to the client.

Basics

The basic steps are "use the windows drivers" and "observe what is sent to printer". Wireshark is your friend, and I was able to see the Ethernet traffic sent to the printer. There was a nice obvious message structure.

I was really lucky with this printer though as there is a diagnostics tool that the suppliers sent, one they use for maintenance/repair and testing. It has buttons and check boxes for each operation - load a card, move a card, load image data, print images, transfer image to card, etc, etc. And control of all of the settings.

This allowed me to test each operation separately and each option separately and dump the messages. This is absolute slog from a coding point of view - it is spending days trying each separate command and option and looking at hex dumps to work out what each option and setting does in the underlying message. It was tedious.

Once you have that you make test code to generate the same messages and confirm they work, and then build a driver with all of the functions you need.

USB

Whilst I know a lot about Ethernet, I know bugger all about USB. But to avoid the whole "unplug USB during power up" issue, I decided to give it a go.

I was pleased to see that Wireshark comes to the rescue again, and allows dumping of USB traffic on windows. This allowed me to do exactly the same slog of testing each operation and seeing what is exchanged with the printer.

What did I expect?

I expected the XID8600 to be similar to the XID9300, and I was pleased that they were almost the same (Ethernet drivers), but different ports.

The Ethernet has a simple packet/message system - a header with type and length and a variety of commands and parameters. I expected USB to be an alternative way to talk to the underlying printer system and so essentially to be the same with maybe different headers more suitable to USB.

How it works!

I was shocked!

Not even close, not at all, really, crazy. It is like two separate printers! Yes, the underlying operations you can do are the same but the commands and parameters are not remotely related.

Some examples of the craziness.

• Ethernet has a "transfer" command with an option of where to move the card to after (eject, reject, back to print station, etc), and a flag to say if to flip the card. The flag and "location" code match the card load and card move commands. But on USB there are three separate transfer commands (return, eject, flip) and a flag for "immediate" (no clue what that actually does).
• Ethernet has a way to get and set some "settings". These are one byte values, and you send a list of tagged values and get back a list of tag/value pairs. To change you send tag/value. So a setting has a "tag" which is a byte, no real logic to which setting is what tag, but I went through each and documented. But USB is not the same, no, not close. You get settings by two separate commands getting blocks of binary data (64 bytes and 14 bytes), and various bytes are various settings, but their position is not related in any way to the tag used in Ethernet. It gets worse, to set these you send two blocks (32 bytes and 10 bytes), where the settings are specific positions (but not related to the ones you fetch). But then if you want to set some settings just for the job, there is a single block of 24 bytes that have most settings, again in different positions.
• The mag stripe coding messages are similarly different.
• The film layers have numbers when loading the image data, on the Ethernet they were C, M, Y, K, P, U, but on USB the are K, Y, M, C, U, P, just to be different. When printing the them there is a mask of which to print - for Ethernet it is bit numbers for each layer but always prints C/M/Y together if any set. For USB there is a bit for K, YMC, U, P, instead (so obvious that YMC always together).

(Yes, they persistently use YMCK 👷🤷‍♂️ rather than the more common CMYK, which has led to lots of typos in my code)

Basically, it is like two completely separate specification written independently to meet a list of commands and options. Totally separate logic to how they are coded and the how commands are grouped, and so on.

In addition, the USB used a command structure that seems to be based on USB mass storage, using SCSI commands, or possible mis-using them as Wireshark was not entirely happy with some of them. No idea why they would use that structure, perhaps allowing some standard library to be used from Windows.

The good news

Having coded USB for this I tested on the older XID9300. It works. I was shocked!

Deleting code

This led to a decision to delete and remove all of the Ethernet driver code I wrote. The concept for this project is a Raspberry Pi next to printer acting as a single contact for TCP TLS JSON printing. I have a library for the code to run the client side of this as well. The connection from Pi to printer can be USB with no issue, and indeed is possibly easier than having Ethernet from Pi to printer (needs more config to say IP, etc). So using USB is all that is every required (and marginally faster than Ethernet, oddly).

Deleting code is not something I like doing, but in this case it made the code simpler and made for simpler ongoing maintenance. With both drivers - any changes had to be coded and tested for each way of working.

TODO

There are aspects not in the drivers (yet), e.g. we don't have the JIS (Japanese Mag stripe coding) hardware to test that (though easy to code anyway), and we don't have a laminator attachment. So a few bits are not there.

New features

This means we can update our card printing service...

• 600dpi instead of 300dpi.
• The UV print no longer has at the MAC address overlay (windows has it but I found the bits to turn it off, yay).
• We can get the "ID" of an RFID card and store when printing so the API we offer allows you to tie a printed ID to the card ID later for coding - that way we don't need your keys. We even allow the ID to be part of what you print.
• We plan to offer basic MIFARE classic coding of cards (but really, that is not very secure).
• We plan to offer MIFARE DESFIRE coding, but means giving us keys.

All in all a good few weeks work on this. I was, however, rather pleased with myself (having spent a day with wireshark) making working USB drivers in a (long) day this weekend.