Manually keyed IPsec

My previous post explained a bit about IPsec. We have released it in phases, with manual keying first, then IKEv2 key negotiation, and now EAP with certificates and road warrior stuff.

As part of the work on this release we considered actually removing manual keying as a feature. Whilst it was a useful first stage of the release of the initial version, it is not as secure because the key itself never changes, so is more vulnerable to decoding intercepted traffic eventually. With IKEv2 the key regularly changes making the job a lot harder. We released IKEv2 18 months ago, soon after the initial manual keying release.

The latest change is not primarily about security but convenience. The EAP changes allow identity to be agreed using certificates and usernames and passwords, but the key exchange and updates are managed by IKEv2. EAP makes it a lot easier for mobile phones to connect in to a FireBrick, and get an IP allocation and so on.

So, removing the support for manual keying was a consideration. We looked at other devices and concluded that manual keying was not that commonly supported and that key exchange was more common. IKEv2 is being added to devices more recently, with some devices having supported IKEv1 (which we do not). But manual keying was pretty rare. Indeed, we did not find any examples.

We decided it was going a bit far to actually remove the feature just yet, but managed to leave a problem where the config from an old version using manual keying would not work on the newest release. This, in itself, was a mistake. Having decided to keep support for manual keying, at least for now, we should have made the config compatible (moving the old format to the new one automatically). But the fact we assumed nobody was using manual keying kind of put that on the bottom of the list, and it was left out.

Sadly, that was a bad call! Sorry.

It seems that we have a wiki that explains setting up FireBrick to FireBrick IPsec using manual keying, and even A&A staff were using that until recently. The upshot is that code updates broke some IPsec tunnels. The wiki should have been updated 18 months ago!

So, we have stopped the roll out of new code today, and we should have a new release tomorrow morning. It will automatically map the old config to the new. It seems we only had a hand full of people affected, and support staff have been working to ensure that they are configured correctly.

The other good news is that anyone upgraded today, and using IPsec manual keying, and that has not touched their config, will be updated tomorrow with the config change to make it work again.

But having realised that some people are using manual keying, we need to find a way to try and get people to upgrade their config to IKEv2 with a pre-shared secret. This is not just because we'd like to remove manual keying, but because it is not as secure. The problem is how we know who is using IPsec with manual keying?

Now, we have some mailing lists, but we emailed asking if anyone was using manual keying as part of the research we did in this release, and had no replies. The problem is a lot of people actually using IPsec have had someone else help them set it up, and have no idea.

So, we have a plan. We are very careful about the way in which a FireBrick phones home. It is a security product and people are understandably concerned about security. We have no back-doors in the product. The closest to phoning home is the software and capability update check, based on a DNS lookup, that allows us to do upgrades - but people can control that in the config. The only other thing we have is the default fb-support log target which is there primarily to email us if a FireBrick crashes. All of this is under the control of the person configuring the FireBrick and can be turned off. Though we don't recommend it as crash logs are useful to help fix any problems, and software updates are important. The key thing here is that we are not secretive about this and we give people a choice.

So, the plan is that the release tomorrow will not only update any IPsec manual keying config, but also log the fact to the fb-support log, which means we'll get an email. It may mean a dozen or so FireBricks email us, and we can then contact people to help them with a config update to use IKEv2 pre-shared secrets instead. If we can be reasonably sure that nobody is using manually keyed IPsec we can then look to remove it from future releases.

This should give us a good mechanism for managing such things in the future. Ultimately, whilst this was a bad call by me on this, even if the number of people affected is tiny, we learn from such mistakes and aim to improve the product. Ironically, I suspect this blog will make more people aware of this issue than the mailing lists, which is a shame. If you use a FireBrick, please do sign up.

Update, we are releasing 1.36.002 today (29th) which upgrades the config from older format to the new format for manually keyed IPsec, and logging if manual keying is being used.