Internet in a box

I have finished my cruise now, which was mostly holiday, but also some work. I did some training for my mates (mainly in C coding) and we did various coding as well (there were a few sea days). But now I am back I am making up the next version of my "internet in a box" that I take on cruises like this. I'm doing it now whilst I remember the last cruise in detail, even though my next cruise is some way off.

OK, that is not it - we have one of those, and it would be really cool if I could fit the bits in that box, but at present is is a tad larger... More like this...

So, what's in the box?

• FireBrick FB2900
• Aruba 501
• 2x Aruba AP-303H
• 3x PoE injectors
• 1x 4 way power strip
• Magic tape to hold it all in place

This is obviously somewhat overkill, so worth some explanation...

FireBrick FB2900

The FireBrick is a "swiss army knife" of network contraptions. It does a lot.

When you are trying to use internet on a ship you have a challenging, even hostile, environment. There are blocked ports and protocols, 700ms round trip latency (or randomly much more), packet loss at various levels, strange MTU issues, and seriously messing with TCP packets (acceleration). This can all change on the fly as you travel (the Panama trip was especially complicated).

To be clear, this is not stealing internet service - it is expensive and we pay for the premium, unlimited, steaming package for multiple devices. This does allow connection of devices that do not have WiFi or have a browser.

Whenever I take a FireBrick on a cruise we find new ways to improve it. This can be changes to handle high latency, or new features to handle some of the limitations. Even simple higher level protocols can struggle with the very high latency and low level packet loss. A lot of new features are the result of testing in this harsh environment and have benefitted the FireBrick code. Not sure I can expense my cruises as R&D just yet though, shame.

So, this alone, is one of the reasons for the crazy set up. The FireBrick can do various VPNs, UDP over faked TCP, TCP relaying, all sorts.

The main objective is to connect to the ship internet (WiFi) and provide internet to laptop or apple TV. For the apple TV to work in any expected way without regional blocks, it needs a working UK IP address in some way, and the FireBrick can do that.

The FireBrick can also monitor the connection in various ways and fall back, even to simple NAT over the ship's WiFi as last resort, and report status on an LED to make it obvious. If ever I fit this in one of those black boxes, the LED will not just blink red :-)

Aruba 501

This is a rather nice WiFi client. It connects to the WiFi and can do MAC cloning, where it will associate using the same MAC address the FireBrick is using. We found that the WiFi on ship filters other MAC addresses, and even locks down the connection after a little while if it sees more than one MAC. We were changing MACs every day until we managed to lock it down to no see any others.

Aruba AP-303H

Having connected to the Internet, and set up a VPN, we then provide internet over WiFi. It can be done with cables, but WiFi is fine and not as messy or such a trip hazzard. Previously I took a larger ceiling mount AP, but that gets hot, especially if not ceiling mounted. So this time I have smaller, and lower power, AP-303H units. I also have two, one facing each way, so the box can go in the corridor. Ships have big metal walls which make WiFi tricky. Even so, I am taking some 10m ethernet cables to allow me to place the APs to cover the whole cabin if necessary.

We actually had to set a hidden SSID, as we found that in at least one port we were seeing de-auth attacks. Interestingly this was not happening once we changed to hidden SSID. Even with the metal walls, we often see people running personal hotspots when in port, so it may be an attempt to stop that (AFAIK not legal to de-auth people like that, but who knows on a ship).

Update: Having two APs powered by PoE means I have more options - running a cable to place one, or both, APs, in more suitable locations in the cabin if they don't work in the box.

PoE injectors

This is another change from previous cruise - the last couple of times I took a nice 8 port Aruba PoE switch, which is quite big and has a big chunky power supply. This time I have three small PoE injectors which take a lot less space overall. There are some multiple port in-line PoE injectors which may be a good alternative to consider, but even with just one such unit I still need a power strip to power it and the FireBrick.

The AP-303H includes a switch, so if I need more Ethernet ports, they can provide them, so the bigger switch was not needed.

Power strip

The three PoE injectors and FireBrick mean a 4 way power strip - though I am considering making a lead with daisy chained C13 plugs and a C8 all on one lead perhaps. However, the 4 way strip fits fine. One option may be an IEC socket in the side of the Peli case so it can be closed. It looks like the whole lot is not generating enough heat for that to be an issue, but something to test.

Update: One idea is to use a 4 way IEC distribution board instead, which may well take less space.

Spare space

The whole box, even with all those bits taped in to place, has a lot of space. In fact I can pack my laptop, charger, mouse, mat, Apple TV, spare cables, phone charger, and so on, all in the one case. This means all of the tech in one small Peli case which then just sits in the corridor to provide "internet in a box".

Why?

Update: This allows me to bypass much of the hostile environment, and have clean Internet access on my own IP addresses. It even allows me to have a standard VoIP phone on the table in the cabin if I want. It allows devices that could not connect to ship's WiFi on their own (I had some of my IoT stuff on it). It is not trying to be the cheapest, or even the smallest (though I am trying to make it smaller). It mainly allows testing and development of the FireBrick in such an environment, and it is fun (for me), even if it is overkill.

Update

A few more pictures. I decided to go for an IEC distribution panel inside, and fit connectors to the case itself, and add a 3G/4G dongle.

P.P.S. Using V2.0.0.1-Aruba501-B0013 on the Aruba 501 was Crashy McCrashFace, but V1.0.1.3-HP501-B0012 seems to be stable.

RevK 2.0

So, I think I have svn reverted to 1.0, or maybe 0.9 now, as I have a cold, but my mates were talking of RevK 2.0 over the last month.

Largely because I let them talk me in to :-

• Going out in the sunshine - in tropical climates
• Sitting on a sunny breach
• Going to a Jazz bar
• A tour of a warship
• A tour of an aircraft carrier
• Swimming in the sea
• Swimming with dolphins
• Snorkelling (well, I tried, but kept hyperventilating)
• Parasailing (felt sick, life jacket was too tight)
• Boat trip to see crocodiles (they were mostly tiny)
• Going on a submarine (albeit docked)
• Several open top bus tours (which I do not normally go near)
• A walking tour in a hot city (Cartagena, Colombia)
• Taking a picture of an Aruba access point, in Aruba
• Oh, and wearing a Panama hat, in Panama

So over all, a fun trip! I did not do the jet skiing, just took pictures.

Here is a small selection of the pictures...

Escorted off the ship

Being "escorted off the ship" sounds a tad bad, but this was at the right point, at the end of the cruise, honest...

My mates and I are just back from a cruise (one hell of a holiday, thanks guys), and we stayed in the "Garden Villa" on the Norwegian Gem. It is big suite on the ship. It is often used for celebrities, or Sultans, and the like, but they also auction it to suite guests if not sold. Some times you end up with the likes of us in it!

[A slight aside, I do love how the concierge emailed crew to advise that even though it is three blokes in the Garden Villa, it is not a gay cruise, we have wives - even so people kept asking Mike and Simon if they were a couple, which was amusing]

Anyway, even though not actually celebrities, you do get treated as such, which is both nice, and also slightly embarrassing. I imagine real celebrities are more used to it.

I wanted to tell of one of the perks, which was slightly more embarrassing than we expected. It may be educational for anyone else staying in the Garden Villa on an NCL ship.

You get escorted off the ship!

This means that the concierge waits to the right point for when your bags will be ready (among the first bags off as "priority"), and escorts you from your room (Garden Villa) to the gangway where he/she hands off to an escort to take you off the ship. Just getting the to gangway means going via staff elevators, and past waiting queues of people, and at some points people being stopped and waiting for you to pass before they continue.

Once off the ship this continues, more queues of people and you get ushered past. We are taking to the baggage area, and our bags put on a trolly with a porter, and on we go. All very slick.

Just to be clear, we did not know quite what was going to happen, and were just following instructions from the escort.

Finally we get to immigration control, and a queue for the passport checking desks. This is a well established ship process it seems, and very slick. All the security know what they are doing and were expecting the escort and Garden Villa guests so co-ordinated to move us to the front of the queue.

We get to the border control person, and he is "who are these people, why did they jump the queue?". Our escort says "Garden Villa" which means nothing to him, and he ends up saying "nope, not unless my supervisor agrees" and calls him over. The supervisor just says "let them through and send to secondary screening". We can only assume this is somehow to punish us for jumping the queue, but secondary screening were at a loss as we were not on any lists or anything. They asked if we had any dairy products and basically sent us on with maybe a couple of minutes delay. We are then escorted to a waiting car to take us to the hotel.

[OK, last bit went wrong, and no car, and they went in to headless chicken mode, calling the ship, etc, so we just got in a taxi, which is easy enough in New York, but you get the idea. It was too cold to wait for it to be sorted.]

Now, this whole process is slick. As I say, the ship and port staff know what they are doing, and know about people being escorted, etc.

To be fair to the guy, I can see why the border control person was upset - why should someone be able to effectively pay to go to the front of the line? That said, I thought secondary screening for the hell of it was a tad petty, but what the hell. We were all being quite chilled over it - as to be honest we had imagined all sorts with the Coronavirus scares and ships being refused entry and so on. This was a breeze.

Then it struck me that this "everyone treated equally" is all very well, and laudable, but basically not how a lot of things work in practice. The ship definitely have tiers of guests. So do aircraft, and 1st/upper class get off first. EVERYONE in the queue for immigration control was on "priority disembarkation" which is paying to jump the queue of some 2,000 or so people behind them waiting on the ship, and they effectively paid for that by being in suites. Somehow none of that mattered to him. If he really did feel strongly about everyone treated equally he should have made a stand on that front, and said the people in the cheapest cabins should be coming through first, or at the same time. No, his only beef was that he actually saw that were jumped the queue.

I hind sight, it would have been better, and not really much delay, to have queued with the rest of the priority disembarkation passengers at that point, and we'd have done that if we'd have known what was happening. But we were just doing as we were told by the staff.

Anyway, if you are in the Garden Villa on an NCL ship, this is the sort of thing you can expect, and perhaps decide how far you want it to go, yourself.


I am, of course, interested in views. Should people be able to pay to speed up such tedious processes, or (at the other extreme) should royalty and presidents all have to queue with everyone else in such cases? Should 1st/upper class on planes be scrapped? Where do you draw the lines? Curious?

Standard C function to read lines from a file

[update: As I hoped, there is a simple answer, getline(), see comments, thank you Charles Lecklider]

The classic is fgets(), it is simple, and easy to use...

Of course, for some reason, fgets() gives you the line endings, so I usually end up with more like.

The problem, of course, is you have a line length. This is also an advantage in that you constrain the lines and don't have random memory allocation issues, but computers have so much memory and VM these days. How many times have I seen this code, and seen someone have to change 1000 to 10000 one day?

What I would like is a simple function that reads a line and mallocs space as needed. Indeed, it could return the allocated space or NULL for error (EOF, or malloc fail). You'd have to free it, but no big issue. Would also be nice if (a) it stripped the line ending as I literally NEVER want that in the line, and (b) seamlessly handled bloody DOS style carriage returns...

So whilst trying to explain some basic C to my mates, whilst at sea, in the middle of the Atlantic, I tried to explain this whilst making a simple CSV file parsing program for them. We did some googling, and found that I am not alone in trying to find such a function. It seems that fscanf() may be the answer. [update: clearly I did not google well enough!]

To be honest fscanf() is a function I just don't use enough. It is very powerful, but I always find myself parsing things more directly. However, I had not considered it as a means to just get a line.

The magic incantation is something like...

This reads any characters up to a newline, allocates space (that is what m is), and stores in line. Just what we need. A minor variation to handle carriage returns seems to work too...

Bingo, we have our magic line malloc file reader function. Perfect.

And get this, reading the man page it is clear that using the [ function does not consume the leading white space, which is perfect... So all good

Except that is not what happens. We did the CSV stuff, and then went on to TSV (tab separated) and magically leading TABs (i.e. empty first field) were stripped by fscanf()

Why?!?!?!?!?!

Please someone tell me I am being thick and that there is a standard function to do just this. Yes, I could write my own, but this is surely so basic it should be standard C library stuff.

[code mistakes in examples left in for the reader to find]

VPN on a ship

There are many reasons for a VPN on a ship, and I was quite impressed that my other hack for UDP over fake TCP worked at all.

It did work, but was slightly strange. The experience was that normally VPN (via UDP or TCP) worked, but had 700ms latency round trip.

However, some of the time, the ship would have packet loss on the back haul, and then things changed. The UDP VPN would lose enough packets that normal TCP really struggled. Indeed, some times the VPN itself struggled to establish or stay up. It was actually for this reason that we even tried the TCP based solution.

However, in such cases, the TCP based VPN is special. It is very reliable, but has silly latency (up to many seconds).

The reason is what the satellite link it doing behind the scenes with TCP. It is doing an accelerator, which (I assume) is some level of local ACK combined with some internal re-transmission for dropped packets. But this means our VPN over TCP has the problems of doing TCP properly rather than faked, in that dropped packets result in high latency for every following packet.

Given the "cleverness" going on, it is amazing our faked TCP was good enough at all!

However, we like to have proper IP addresses here. There are a lot of reasons for that, but one of them is simple, it puts us, and our apple TV, geo-located at home. This was rather important for watching the episodes of Picard as they came out, even when in the middle of the Pacific.

So the challenge was proper UK IP addresses but still using the ship's TCP acceleration.

The solution was to de-NAT the traffic. The local brick here would NAT and send via ship's internet, which NATs again, but we change the destination IP to go to a brick in the UK. That then de-NATs, mapping back to original source and destination IP addresses and ports, or NAT the source. This works a treat, but you may have spotted the challenge here... How does the far end know where you wanted to send the packets in the first place?

The answer was to tack them on to the TCP SYN and look a them at the remote end!

My first thought was push them in to TCP options, but that is a challenge as there are only 40 bytes, and MSS takes 4 of those, leaving not enough bytes for two IPv6 addresses and ports. We could for IPv4, or we could just send target IP/port and NAT in UK. So we tried that.

The buggers NOP out any TCP option they do not understand, arrrg!

Alternative was put actual data in the initial SYN. To our surprise that worked, and as a SYN does not normally carry data, it works well (it is one of those theoretical TCP things I have never seen used). But it survives, and does not have the small size limit. We do have to move the SEQ back to allow for it, and strip out at the far end before sending on, but end result is public IPv4 and public IPv6 here, working. There is even space to add some authentication. (Note, this is not encrypted as assumes we will encrypt over it with TLS/https, ssh, etc).

Sadly sessions do not persist over glitches, and no incoming sessions, but I could watch Picard, so mission accomplished.

P.S. We have a choice of how to connect now, VPN over UDP, TCP (faked), relayed TCP, or just NAT on to ship's network raw (no good for Apple TV). Today we found UDP the best - the ship's TCP was unbearably slow (relaying, or not), but UDP was way better. To be honest this is much more what we are used to - everyone on the ship saying internet is crap and we're like "high latency, but not bad really" because the UDP based VPN works so well. The main thing is we can switch between different approaches as needed on a trip like this.

VPN over UDP over *fake* TCP

It has become a bit of a tradition for us now that, on a cruise with my mates, we spend maybe a day on some serious FireBrick development that is relevant to the cruise. In the past we have greatly improved the PPP stack. Last time we greatly improved the SIP handling of retries and duplicate packets and dropped packets (I had a working VoIP phone on my desk, and even got a junk call). Basically, the internet access on a cruise ship is, for want of a better word, "special". It can have horrid packet loss, and typically has 700ms latency but can be way more (as I type this I am seeing 2s). It creates problems for a variety of networking.

So this time, I have coded a special new feature on the FireBrick to make UDP in to fake TCP. The reason was that at one port (Guatemala) even UDP was being a problem.

Let's just be clear here first, this is not hacking in any computer misuse sense - we have paid for the most expensive internet. Not just the premium "unlimited", but the premium "unlimited" with steaming. That's right, "unlimited" somehow has a limitation of not allowing streaming! It specifically says it allows VPN but they are unable to say which VPN, and out-of-the-box IPsec on my phone or laptop simply does not work (even when it uses UDP). Oddly, unlike previous cruises, simple L2TP was blocked, meaning it had to be mapped to another random UDP port to work. But when even UDP struggled somehow, we considered TCP. Let's get that VPN, for which we have paid, working.

One idea was simply to open a TCP connection and push UDP packets through it. Not hard, but has issues. For a start, the entire latency / throughput has to be buffered on the firewall for that to work. But also, one dropped packets causes a backlog of all streams until received, as TCP is always in order. At 700ms+ that matters. So we wanted UDP behaviour but something that the ship would think is TCP.

Dumb idea (thanks Mike) was simply change protocol tag from UDP to TCP. After all the ports are in the same place at the start for both. Unfortunately even the dumbest NAT will look at the TCP flags for SYN, FIN and RST at the least.

So, less dumb, change to a TCP header with sensible flags (SYN on first, SYN+ACK on first reply, and ACK on the rest). But pack the otherwise lost 12 bytes (TCP uses 20, UDP 8), in to the SEQ, ACK, Window, and Urgent fields in TCP. That way, NAT can play with ports and look at the TCP flags but pass through the same data with no extra overhead. I am quite sure that would work on some NAT. I think FireBrick NAT would pass that with no problem.

However, we found that the ships system has a rather heavy handed NAT that not only changes ports but also sequence numbers (why?!). It also expects valid sequence numbers to be used in SEQ and in ACKs. Simply resending a SYN with a different SEQ or sending an ACK that is before the SEQ of the SYN caused a RST and dropped session. So we had to make the TCP look the part.

The answer was that the first packet was changed to just a SYN (and later updated to a SYN with window scale, just in case it cared). At the start we set a random sequence and store it, and for SYN send the SEQ to that minus 1. Only once we had a SYN+ACK did we consider the session properly started and we could send the UDP packets as TCP, moving on the stored SEQ for each to look like a legit TCP stream. On the other side, for a SYN, we generate a SYN+ACK response, and consider the session started. Any protocol over the top loses the first packet to the SYN, but as it is UDP it will resend, which is exactly what L2TP does. Obviously the MTU was dropped by 12, but we were working on 1280 to get through whatever shit the ship was using between us and dry land anyway. The ACK was then based on highest SEQ+length received regardless and so there is no buffering or resending - that is all handled by the VPN and ultimately individual TCP sessions over the VPN.

End result, after more hours than I would have hoped, it works. It is in FireBrick release1.53.025 Flint+ Alpha, as experimental. Part of firewalling rules allowing a protocol 6 to be set to 17, or 17 to be set to 6. Have fun. Likely to change some time, perhaps with option to try using the 12 bytes in TCP header to avoid extra overhead.

Far from #Brexit - some civilised drinks

It happens that, at the time, my friends and I were far from Brexit. Indeed, we were exactly 14.48054467N and 94.175880624W (or -449786.479813 -6160429.922092 1584520.022083 in ECEF) which is in the Pacific ocean.

However, as there were a few Brits on the ship, we got the concierge to invite some to our suite for a few drinks. We engineered nearly live BBC on the TV via a convoluted set of kit and VPN back to the UK, and we had my brexit clock.

It was remarkably civilised, with those for and against agreeing that whatever we do, we now have to find a way to live with this and make the best of any opportunities created.

It is the first time we have "hosted" a party from the suit, but butler managed it quite well and the ship even provided a couple of bottles of champagne for the party.

Amusingly at least one couple assumed it was a scummy NCL promotion to try and up-sell the suites, LOL.