How it works: https

Most of us make use of https at some point, usually for accessing on-line banking or on-line shopping of some sort, but increasingly for more mundane things like facebook or twitter or gmail. It is meant to provide some extra security, but what does it provide and how does it work?

Snooping on your messages

Before we even consider encryption, consider that the easiest way to snoop on your communications is either before it is encrypted or after it is decrypted.

At the simplest level, consider you are sending someone an email, and using https to access gmail. You may take all precautions your end to be sure you are not snooped on in any way, but the recipient could be a dick and forward the email on to his mates, or post it on facebook. If you are considering security you have to think of the bigger picture and the people involved, not just the technology.

Thankfully most of us are not considering security but simply privacy. We (supposedly) have a right to privacy. We just don't want someone snooping, which is fair enough.

But if you are considering security, you do have to consider other ways an attacker can access your computer. There are hardware key loggers that plug in line with your keyboard, and are undetectable by the computer itself. There are viruses that log keys and grab images of the screen. Most computers have some sort of remote desktop facility, which means that you could be accessed if you have a poor password and poor  firewall settings, even though you have no viruses on the machine. There are many ways, but all of these really only make sense if you are being individually targeted, so really not a concern for most people.

Encryption

Encryption as a principle has been around thousands of years. The concept is simple, you somehow "scramble" your message before sending it in a way that only the intended recipient can "unscramble". This is putting it very simply though.

Encryption uses maths, and there are some very clever people that understand how it works. There is a whole science of cryptography. Myself, even though I have an A at A-level maths and a degree in computing, I struggle to get my head around the detailed maths involved in some of it. The principles, however, are relatively simple to understand and can be explained with some simple analogies, thankfully.

Using mathematics to encrypt things has been around for a long time, but only in the last few decades have computers been powerful enough for serious encryption to be used routinely. There are many different systems, but basically it is impossible to crack an encrypted message without having the "key". I should be careful what I say here - in cryptography there is no "impossible", it is simply a matter of making something that takes too much time and resources to crack in the lifetime that you need something to be secure. But really, for all practical purposes, we are talking "impossible" to crack.

There are, of course, conspiracy theories. The idea that "they" have scientists that have cracked modern encryption systems. Basically, many encryption  systems work on the difficulty of some specific mathematical problem. However, some mathematical problems have some "short cut" discovered that makes them a lot easier to solve. Some people believe such short cuts have been found and that governments can secretly decode all encrypted messages. This really is rather unlikely. Not only would it be unlikely for only one person to have found such a short cut, but it would be very unlikely for it to be successfully kept a secret. If you must have a conspiracy theory, it is far more likely is that governments just want people to think they have a way to cracking encryption.

Another theory is that "they" have huge computing resources to "brute force" the encryption systems. This has some grain of truth in that some older encryption systems can now be cracked in realistic time frames with large numbers of modern computers. Even so, this only makes sense when targeting a specific message. A realistic time frames could mean weeks to crack one message. In practice, modern encryption uses much larger keys which can't be cracked like this.

Of course, this is speculation on my part - but you can be pretty sure that if encryption in use today is found to be easy to crack, it will be changed very quickly to something that is hard to crack. Oh, and don't try and make any sort of encryption system yourself, it will be easy to crack :-)

Public key encryption

One of the key encryption techniques used is "public key encryption". Fortunately this is very easy to explain to someone without using any maths - a simple analogy using padlocks works well. In reality it is more complex, and public keys are used to encrypt random symmetric keys that are used to encrypt the message, but the basic principle is the same.

The idea is simple - imagine you have a very good padlock and a key for that padlock. You give me the padlock and you keep the key. Later, I want to send you a message and want to be sure nobody on the way can read the message, only you. I put the message in an impenetrable strong box and lock it with your padlock, and then send it to you. Nobody can open it. You get it and use your key to open it. Simples!

Trust, and man in the middle attacks

Of course you want all sorts of people to be able to send you messages, so you have loads of identical padlocks made, all of which open with only your key. You have your name engraved on them.

This means that when I want to send you a message, I just get hold of one of the padlocks with your name on it, and use that.

But what if there is an impostor, making padlocks with your name on. I end up getting one of these fake padlocks, and send your message. The impostor intercepts the message, unlocks it with his key (as it is his padlock), reads it, then locks it again with one of your real padlocks, and sends it on to you. Neither you, nor I, are aware of this. Oops.

The answer therefore is that all padlocks come with a certificate which states exactly who's padlock it is and lists the locks serial number (that somehow cannot be forged). This certificate has a seal on it (which also cannot be forged) which is one of the well known "certificate authorities" which we all trust to issue certificates for padlocks.

OK, that sounds a tad woolly doesn't it. The whole "cannot be forged" is achieved using public key encryption to "sign" things. I won't go in to detail, and there are analogies using padlocks and keys for that too, but lets just assume for the moment that it is possible.

This still leaves an issue - how do I know the seals of the trusted certificate authorities, and to be frank, how can I trust these people? After all, I am trusting them not to issue a fake certificate to the imposer?

The first answer is that my web browser comes with a list of certificate authorities (CAs). I can poke around with the settings to see the list. This just leaves the matter of "How do I know I can trust then?"

That is harder - the list of CAs in my browser may have some familiar names, but will have lots I do not know. If I have not personally inspected these companies, checked the security, processes, staff, and ethics I have no way to trust them, yet I do, every day!

The main reason I can trust them is that they trade on their integrity. If they did certify a fake padlock (so to speak) that would be found out eventually, and they would lose credibility, They would be removed from lists in browsers and people would not trust them. They would go out of business, and they know this. So they have to "do the right thing" to stay in business. It is not ideal, but it is a basis of trust, just.

https

When you access a web site, all of these principles are deployed. Your browser gets the "padlock" from the other end, with the certificate signed by one of the certificate authorities in the browser. It checks the certificate. It then uses this to negotiate the keys for the encryption to be used. Then you communicate with encrypted messages that cannot be decoded.

How serious is a man in the middle attack?

A man in the middle (MITM) attack means intercepting the communications - the whole "impostor with fake padlock" thing... This has some problems, thankfully.

Firstly, you have to actually be able to intercept communications. This is hard to do generally. It is a lot harder than simply monitoring unencrypted traffic (which can be done by tapping phone lines, or even bending fibres until light leaks!). It could be done by your ISP, or, in theory, by government mandated "black boxes".

The second issue is these pesky certificate authorities. Your fake padlock has to be certified. One way is to somehow get a new CA (which the impostor controls)  in the CA list in the browser. This is hard, and certainly hard to do without being noticed. In theory a government could make it a law, but that would be very obvious and not very popular. Also, the browsers are not all made but companies - some are made "by the people" i.e. community open source projects where there is no legal entity to legislate against or intimidate. The other way is to get a copy of the "seal" from a CA that is already in the list. This will be bad when it is found out as it ruins the credibility of that CA and they get removed from browsers.

Basically, on a small scale, to target someone specifically, if you can arrange the physical access to intercept traffic, and if you can get a new CA on the users browsers somehow, you can do this. In practice this is hard. Where this is done is in corporate environments where regulation or corporate paranoia mean they install black boxes in the office. In such cases the staff know about it, and so it is no surprise that they have a special CA on their browsers.

You cannot covertly do this on any large scale - the change of key (the fake padlock) is always detectable if you look for it. Imagine that I get a fake padlock and certificate and meet up with you and compare with one of your real padlocks - we can see it does not match and know something is up. We also get to see who certified the fake padlock.

Of course, always remember, the web site you are communicating with can see the data - that is the idea. If they want to, they can use that data in various ways you may not like (legal, or non legal), and they could be compelled by their government to hand over data.

Summary

There is no way in hell that any government can snoop on all https traffic in the middle without the public knowing they are doing it. It only takes one person to check the keys to discover it.

If someone is snooping on your https then they are able to see everything, not just "communications data", if they want to. You no longer have the privacy to which you thought you had a right.

If https becomes snoopable by any means, then the "community" will come up with better systems to make it impossible. There are already changes afoot in the area of https that will thwart snooping by governments, and now there is even more incentive for such changes in the last few weeks.