2016 should be interesting! A&A have a lot going on.
I personally have a lot going on - I won't bore you with details (that is what Facebook is for), but I am pleased to see my daughter planning to get married.
We have the new terabyte services launching - and whilst these are initially very specific services, FTTC, 80/20 capped, with the phone line part (no calls) as a new 12 month service (see here), we really hope that over the coming months some of the restrictions will go away, and more services can start to be measured in terabytes of usage and not gigabytes. I remember ISDN dialup and the dream of megabytes, let alone gigabytes or terabytes. I am actually really pleased we can start to measure usage in such terms, even if only on these specific new services. DON'T HASSLE THE SALES TEAM (they told me to make this big and bold) as we won't have a proper ordering system for about 2 weeks at the soonest. If you want this soon - best to get on the irc channel as we'll be asking for guinea pigs next month.
Our data SIMs are expected to allow 4G real soon. I have seen this working, but there is a lot going on at our suppliers with this and a lot of testing and tweaking, but we know it can work and expect it any day. The only down side is no improvement on pricing - so 4G lets you spend a lot very quickly. However, there are plenty of good business uses of proper fixed IPv4 with 1500 byte MTU and no filters on a mobile device. And for occasional use, like my iPad, it works out really quite sensibly priced! We'd love to do the same on voice SIMs, but that will take some more work.
2015 was actually quite good for finally sorting SIM card printing - it is silly, I know, but getting printers that work properly was not easy. We print cards with artwork of your choice and our logo, and so on. It should be irrelevant as you throw that away but it makes all the difference in the presentation, and we can do proper corporate branding on SIMs now, either as a mobile telephony service provider or as a corporate network. We even do an ID card printing service now which is starting to take off with schools and even ISPs using them for staff ID cards.
We had been making progress on SMS and other related stuff, but that is stalled slightly. We are trying to kick start that and have some improvements to services and prices for SMS soon. It is odd, we sort of expected SMS to go the way of fax machines, and the dodo, but the universal nature of mobile SMS and the closed nature of things like iMessage mean it is still a "thing"!
We have some massive work on the core network, and scarily some 40Gb/s optics involved in a fibre ring. This is happening real soon (January I expect), and we have extra transit and peering and links to carriers going in. This is kind of important to handle expansion of our network, especially with terabyte services. We have a new member of the ops team with a lot of experience, and things are plowing ahead on this. As ever there will also be a load of work on our systems, and we recently launched https://control.aa.net.uk/ as a new https link for our control pages (clueless) which meets some of the latest standards for https properly. Much more to do. We also want to improve the way we handle signed and encrypted email with customers this year. Privacy is important and we should be on the ball on this.
We even have new hardware this coming year, with the next generation of FireBrick expected this year with a true fibre interface and faster throughput and IPSec. We are working on the next generation of the larger scale kit as well, obviously.
Of course, fighting the IP Bill is important. I have done a lot in 2015, including talking to the home office, actually talking to the Joint Committee in parliament, talking to a peer in the Lords, and next month talking again in parliament on the matter. I have been on TV a couple of times on this too. I have a petition at over 10,000 signatures on encryption. It is hard to get over the importance of this, and there is a campaign that ORG are running to try and counter the government spin/PR campaign on this - do consider contributing (here) - we have!
Even little things, like the way we post out routers, are changing. We now have the new Royal Mail Shipping API and so should be able to get their tracked services so we can send a router that does not have to be signed for, and can even have a pre-arranged "safe place" to leave it, but is tracked to confirm delivery. We'll know more on that next week.
We are set to grow and improve in every area of our services for a very exciting 2016 ahead.
There are also moves afoot to improve the A&A web site - we'll see how that goes.
Happy New Year to all of our customers from myself and all of the staff. We may be a small family business, but we really appreciate your continued support.
2015-12-31
Royal Mail Shipping API - finally working
Since Royal Mail are pulling the plug on Smart Stamp, tomorrow, we have been rushing to get an alternative postage system in place. The obvious choice was the Royal Mail Shipping API, which they document on their web site.
This allows us to print postage, which is purchased "on account" rather than pre-paid on card like Smart Stamp.
The API uses XML and SOAP, and seems to be documented in the technical specification on the web site. There is an on-boarding platform to allow testing, and a check list of tests done before you can go live.
Frustratingly, in spite of a couple of months notice it took until 3pm the day before Christmas Eve before we got any credentials to test. All of my testing failed with "Authorisation Failure". Looking at the the documentation and the information they sent over this is clearly a common problem. There is a lot in the technical specification and they have code samples for generating the authentication strings in the XML in php, python, exe, and others.
However, today (yes, very last minute) we finally have this working on their live platform, so here is a list of the issues which I hope they can fix, and others can learn from.
Client certificate
Even though the https uses a self signed certificate their end, they want you to use a specific client certificate they issue. This means we'll have to faff with renewing it every year or so.
Why the certificate is not the only authentication is beyond me, or why, when using https, they do not simply use plain text http authentication or plain text SOAP authentication, is also a mystery.
Authentication
The authentication uses SOAP WSSE Password Digest authentication. This is well document, and involves sending a Created date/time, a Nonce random value to avoid replay attacks, and a Password which is a password digest. The digest is defined for WSSE as SHA1(Created+Nonce+Password) and both nonce and the digest are sent in Base64 in the XML.
What Royal Mail expect is: (a) Nonce is 16 byte binary value (b) "Password" as used in the WSSE hash is actually Base64(SHA1(PlainTextPassword)). This means you send Base64(SHA1(Created+None+Base64(SHA1(PlainTextPassword))))
Their documentation does not exactly make this clear at all, and contradicts itself,. Their example code does not do this, it uses just SHA-1(PlainTextPassword) as the password in the WSSE algorithm, so generates credentials that do not work. Their help desk would not actually make an XML file that works to the actual on-boarding system. They confirmed the examples I made using their own code fragments were "right" when there were not, and only used some SOAPUI test platform rather than using the actual on-boarding system. This wasted a LOT of my time.
I note that at least one example used a timestamp that was local time with a Z suffix (meaning UTC), so I half expect this to all go horribly wrong at the end of March.
ApplicationId
The application ID looks to be sort of your account number, a 10 digit code. But for some reason both the on-boarding and live systems cannot actually work with the right application ID, and helpfully just say authorisation failed. They have provided a different one to use for now. This just compounds the confusion over the authentication hash as trying either way of doing the hash still fails when using the right application ID.
Also, they quoted the application ID wrongly in the first place in their email, just to add to the fun.
Schema Errors
The XML you send is checked against the schema, and a single character out of place just says Schema Validation Failed. No clue where it got to. This is a nightmare to debug.
This is made worse by the specification actually quoting at least one object name incorrectly, so if you follow the spec you will get a schema validation error. For example, object unitofMeasure is actually unitOfMeasure.
XML namespace
Anyone that uses an XML library or understands XML namespaces will know how they work - each object and even attribute can have an associated namespace, which is itself usually a long URI.
These are then typically abbreviated to an arbitrary prefix. So soap:Envelope is saying that the Envelope object is in the soap namespace, but there is an xmlns:soap attribute somewhere at or above that object saying what the namespace actually is.
The tags you use are normally totally arbitrary, however, when I tried using different tags to those they used it failed validation! I don't know if this is only the on boarding system, but it means they are not doing XML right. Thankfully, using the tags they expect is not that hard, but it a requirement they should make clear.
It gets worse though, and this may only be on-boarding again, but the location of the namespace matters! Not for the schema validation, no, that would be too easy, but for authentication. My XML library places the namespace declaration as close to the outer most usage of that namespace. But doing that means you get the ever unhelpful Authorisation Failed error. This again compounds the confusion of the hash used. Thankfully my XML library has a mode that puts all namespace declarations on the top level, and that was good enough for the on boarding system (albeit not quite matching their examples).
Rather annoyingly they mix some objects using V1 of their API with some using V2. They also have subordinate objects that are in the default XML namespace (so untagged), which means you cannot simply make the whole message have an xmlns at the top level and avoid using tags. It is just messy.
Service Matrix
They have a service matrix - the services they offer are a combination of a service type, service offering, service format and enhancements, and then separate options for signature. It is actually really messy, e.g. for recorded delivery you use a service enhancement (a number) on a normal service list 1st class post, and do not set the "signature" field! Basically, we had to make a table of what things to set for each service we wanted to use. But we don't have a clear list of options from Royal Mail, i.e. other than just the title of each so not clear initially that CRL is Royal Mail 24 and Royal Mail 48 which are account (cheaper) versions of STL 1st and 2nd class! We hope to find out more next week, and something about the tracked but not signed for options which we want to use for routers.
I hope this is useful to anyone else doing this... P.S. I have C code for this - ask me on irc.
This allows us to print postage, which is purchased "on account" rather than pre-paid on card like Smart Stamp.
The API uses XML and SOAP, and seems to be documented in the technical specification on the web site. There is an on-boarding platform to allow testing, and a check list of tests done before you can go live.
Frustratingly, in spite of a couple of months notice it took until 3pm the day before Christmas Eve before we got any credentials to test. All of my testing failed with "Authorisation Failure". Looking at the the documentation and the information they sent over this is clearly a common problem. There is a lot in the technical specification and they have code samples for generating the authentication strings in the XML in php, python, exe, and others.
However, today (yes, very last minute) we finally have this working on their live platform, so here is a list of the issues which I hope they can fix, and others can learn from.
Client certificate
Even though the https uses a self signed certificate their end, they want you to use a specific client certificate they issue. This means we'll have to faff with renewing it every year or so.
Why the certificate is not the only authentication is beyond me, or why, when using https, they do not simply use plain text http authentication or plain text SOAP authentication, is also a mystery.
Authentication
The authentication uses SOAP WSSE Password Digest authentication. This is well document, and involves sending a Created date/time, a Nonce random value to avoid replay attacks, and a Password which is a password digest. The digest is defined for WSSE as SHA1(Created+Nonce+Password) and both nonce and the digest are sent in Base64 in the XML.
What Royal Mail expect is: (a) Nonce is 16 byte binary value (b) "Password" as used in the WSSE hash is actually Base64(SHA1(PlainTextPassword)). This means you send Base64(SHA1(Created+None+Base64(SHA1(PlainTextPassword))))
Their documentation does not exactly make this clear at all, and contradicts itself,. Their example code does not do this, it uses just SHA-1(PlainTextPassword) as the password in the WSSE algorithm, so generates credentials that do not work. Their help desk would not actually make an XML file that works to the actual on-boarding system. They confirmed the examples I made using their own code fragments were "right" when there were not, and only used some SOAPUI test platform rather than using the actual on-boarding system. This wasted a LOT of my time.
I note that at least one example used a timestamp that was local time with a Z suffix (meaning UTC), so I half expect this to all go horribly wrong at the end of March.
ApplicationId
The application ID looks to be sort of your account number, a 10 digit code. But for some reason both the on-boarding and live systems cannot actually work with the right application ID, and helpfully just say authorisation failed. They have provided a different one to use for now. This just compounds the confusion over the authentication hash as trying either way of doing the hash still fails when using the right application ID.
Also, they quoted the application ID wrongly in the first place in their email, just to add to the fun.
Schema Errors
The XML you send is checked against the schema, and a single character out of place just says Schema Validation Failed. No clue where it got to. This is a nightmare to debug.
This is made worse by the specification actually quoting at least one object name incorrectly, so if you follow the spec you will get a schema validation error. For example, object unitofMeasure is actually unitOfMeasure.
XML namespace
Anyone that uses an XML library or understands XML namespaces will know how they work - each object and even attribute can have an associated namespace, which is itself usually a long URI.
These are then typically abbreviated to an arbitrary prefix. So soap:Envelope is saying that the Envelope object is in the soap namespace, but there is an xmlns:soap attribute somewhere at or above that object saying what the namespace actually is.
The tags you use are normally totally arbitrary, however, when I tried using different tags to those they used it failed validation! I don't know if this is only the on boarding system, but it means they are not doing XML right. Thankfully, using the tags they expect is not that hard, but it a requirement they should make clear.
It gets worse though, and this may only be on-boarding again, but the location of the namespace matters! Not for the schema validation, no, that would be too easy, but for authentication. My XML library places the namespace declaration as close to the outer most usage of that namespace. But doing that means you get the ever unhelpful Authorisation Failed error. This again compounds the confusion of the hash used. Thankfully my XML library has a mode that puts all namespace declarations on the top level, and that was good enough for the on boarding system (albeit not quite matching their examples).
Rather annoyingly they mix some objects using V1 of their API with some using V2. They also have subordinate objects that are in the default XML namespace (so untagged), which means you cannot simply make the whole message have an xmlns at the top level and avoid using tags. It is just messy.
Service Matrix
They have a service matrix - the services they offer are a combination of a service type, service offering, service format and enhancements, and then separate options for signature. It is actually really messy, e.g. for recorded delivery you use a service enhancement (a number) on a normal service list 1st class post, and do not set the "signature" field! Basically, we had to make a table of what things to set for each service we wanted to use. But we don't have a clear list of options from Royal Mail, i.e. other than just the title of each so not clear initially that CRL is Royal Mail 24 and Royal Mail 48 which are account (cheaper) versions of STL 1st and 2nd class! We hope to find out more next week, and something about the tracked but not signed for options which we want to use for routers.
I hope this is useful to anyone else doing this... P.S. I have C code for this - ask me on irc.
2015-12-30
Honoured, thank you.
My petition has exceeded 10,000 signatures and so should get a response from government.
Thank you all - let us see if it is a sane response.
To sign, see https://petition.parliament.uk/petitions/106369
Thank you all - let us see if it is a sane response.
To sign, see https://petition.parliament.uk/petitions/106369
2015-12-27
#IPBill implications
So, it is Christmas, and my son is having some fun knocking up an on-line game. Occasionally he needs my help but to be honest he is not doing badly with some javascript, mysql back-end, and a few of my tools I knocked up many years ago, and he is learning something about designing the software and making it work.
We knocked up an on-line poker game some years ago whilst on holiday, it can indeed be fun.
One of the first, and essential, features of the game was a simple in-game chat. This is a feature of most games.
There are a couple of other features which are likely to be part of this game or any game in future - one is running the game on a cheap VM (Virtual Machine) from some VM provider which are typically not in the UK. The other is the availability of free https certificates from someone like https://letsencrypt.org. Indeed some VM providers are working with Let's Encrypt to make it simple.
But hang on! The second you make a game like this, with an on-line chat like this, you have just created a secure communications platform with encryption that ends outside the UK.
James is not making permanent logs of the in-game chat. He has no intercept capability or any resources to make one. He has not published any contact details (he does not have to - not taking money, not a company, not trading) so nowhere to send a RIPA request.
The on-line secure chat is simply a side effect of a simple free on-line game a kid has knocked up (OK not so much a kid any more, but this could be done by a kid).
Where does the Draft Investigatory Powers Bill fit in with this?
Who tracks the creation of such things - he is not advertising it - it is for him and some mates to play a game, but it could be used by terrorists. It could be created by terrorists in the first place.
Who ensures that such platforms have intercept capabilities, and data retention of communications data? Who pays for it all?
It is just one more example of how the IP Bill is just broken, and not fit for purpose, even though it seeks to reach way beyond normal privacy and human rights.
We knocked up an on-line poker game some years ago whilst on holiday, it can indeed be fun.
One of the first, and essential, features of the game was a simple in-game chat. This is a feature of most games.
There are a couple of other features which are likely to be part of this game or any game in future - one is running the game on a cheap VM (Virtual Machine) from some VM provider which are typically not in the UK. The other is the availability of free https certificates from someone like https://letsencrypt.org. Indeed some VM providers are working with Let's Encrypt to make it simple.
But hang on! The second you make a game like this, with an on-line chat like this, you have just created a secure communications platform with encryption that ends outside the UK.
James is not making permanent logs of the in-game chat. He has no intercept capability or any resources to make one. He has not published any contact details (he does not have to - not taking money, not a company, not trading) so nowhere to send a RIPA request.
The on-line secure chat is simply a side effect of a simple free on-line game a kid has knocked up (OK not so much a kid any more, but this could be done by a kid).
Where does the Draft Investigatory Powers Bill fit in with this?
Who tracks the creation of such things - he is not advertising it - it is for him and some mates to play a game, but it could be used by terrorists. It could be created by terrorists in the first place.
Who ensures that such platforms have intercept capabilities, and data retention of communications data? Who pays for it all?
It is just one more example of how the IP Bill is just broken, and not fit for purpose, even though it seeks to reach way beyond normal privacy and human rights.
2015-12-24
Live on TV via Skype!
I am not a Skype fan. We have used "proper VoIP" for a long time, but it works.
But I am also not a fan of spending 90 minutes or so on train and taxi in to London to Millbank studios, being on-air for 4 minutes, and then spending as long getting home. So when Russia Today (RT, Sky channel 512) wanted me on TV and said maybe Skype, I tried it.
It works OK. But it is not that good quality and the sound even broke up.
So I am wondering if I should set up a studio. I have been on RT twice, Sky News several times, BBC a few times, so it may be worth tinkering. I should be able to set up a decent camera and some lights and a decent mic. The idea is that I could provide a web page with streaming live audio/video in HD and use a telephone call in earpiece as talkback.
I wonder if TV studios can handle that - I would think so if they can manage Skype.
I may have to put one of those big "ON-AIR" red lights outside the man-cave though :-)
But I am also not a fan of spending 90 minutes or so on train and taxi in to London to Millbank studios, being on-air for 4 minutes, and then spending as long getting home. So when Russia Today (RT, Sky channel 512) wanted me on TV and said maybe Skype, I tried it.
It works OK. But it is not that good quality and the sound even broke up.
So I am wondering if I should set up a studio. I have been on RT twice, Sky News several times, BBC a few times, so it may be worth tinkering. I should be able to set up a decent camera and some lights and a decent mic. The idea is that I could provide a web page with streaming live audio/video in HD and use a telephone call in earpiece as talkback.
I wonder if TV studios can handle that - I would think so if they can manage Skype.
I may have to put one of those big "ON-AIR" red lights outside the man-cave though :-)
Royal Mail Shipping API
Update: See latest blog post on this.
Having worked out what the WSSE PasswordDigest uses, I am struggling to get past "Authorization Failure.", and sadly it sounds like they have all buggered off home.
The example they sent me matched what I worked out, it includes a Nonce, and a Created date time, and using the password supplied, I was able to create the same Password digest.
To do this I had to use a Password in the algorithm that was not the actual password.
Password_Digest = Base64 ( SHA-1 ( nonce + created + password ) )
I had to use a base64(sha1(password)) in its place. That worked to match the example. But it is not letting me log in. I used this because the specification says "For Shipping API V2 the password used in the below formula is the base 64 encoding of the SHA-1 hash of the plain text password." even though it goes on to say "Password_Digest = Base64(SHA-1(Nonce + Created + SHA-1(Password)))"
However, I have found the spec has errors, little things like it defines an object unitofMeasure which if used causes the whole fail to report "schema validation failure". The example had unitOfMeasure which works. So I cannot actually trust the spec. Not a good start.
However, they helpfully provide a zip of various versions of the authentication done in different languages. This is useful, but also shows that something that should be simple is in fact clearly very complicated.
In perl they provide :-
my $conct = $nonce . $creationdate . (sha1($password));
my $passworddigest = encode_base64(sha1($conct));
Which seems to use simply the raw SHA-1 of the password in the algorithm, not a base64 of it.
In PHP they provide :-
$nonce_date_pwd = pack("A*",$nonce) . pack("A*",$CREATIONDATE) . pack("H*",sha1($password));
$PASSWORDDIGEST = base64_encode(pack('H*', sha1($nonce_date_pwd)));
Now, my PHP is not good, but reading up, the H* means hex encoded. So that looks like it uses a hex coding of the SHA-1 of the password in the algorithm, not a base64 coded version.
In python :-
hashedpassword = sha.new(password).digest()
digest = sha.new(nonce + CREATIONDATE + hashedpassword).digest()
PASSWORDDIGEST = base64.b64encode(digest)
Which looks like using the raw SHA-1 in the algorithm, not a base64 coded.
So I am thinking the spec, and the example they gave me, is wrong. I even tried using their python code to generate Created, Nonce, and Password fields.
Sadly, it is not working to get past "Authorization Failure.", and it looks like the have buggered off home.
This is NOT THE WAY to specify an API. You need clear documentation that is actually correct, and you need helpful error messages on your on-ramp system. Not impressed at all.
Update: Nope, they are there until 6pm - yay, well done Royal Mail - trying to find the problem for me now.
Update: We are back to the base64(SHA-1(plaintextpassword)) now, and using a different applicationId, and working. Yes, that disagrees with all of their code samples! So hoping to get on live system soon.
Having worked out what the WSSE PasswordDigest uses, I am struggling to get past "Authorization Failure.", and sadly it sounds like they have all buggered off home.
The example they sent me matched what I worked out, it includes a Nonce, and a Created date time, and using the password supplied, I was able to create the same Password digest.
To do this I had to use a Password in the algorithm that was not the actual password.
Password_Digest = Base64 ( SHA-1 ( nonce + created + password ) )
I had to use a base64(sha1(password)) in its place. That worked to match the example. But it is not letting me log in. I used this because the specification says "For Shipping API V2 the password used in the below formula is the base 64 encoding of the SHA-1 hash of the plain text password." even though it goes on to say "Password_Digest = Base64(SHA-1(Nonce + Created + SHA-1(Password)))"
However, I have found the spec has errors, little things like it defines an object unitofMeasure which if used causes the whole fail to report "schema validation failure". The example had unitOfMeasure which works. So I cannot actually trust the spec. Not a good start.
However, they helpfully provide a zip of various versions of the authentication done in different languages. This is useful, but also shows that something that should be simple is in fact clearly very complicated.
In perl they provide :-
my $conct = $nonce . $creationdate . (sha1($password));
my $passworddigest = encode_base64(sha1($conct));
Which seems to use simply the raw SHA-1 of the password in the algorithm, not a base64 of it.
In PHP they provide :-
$nonce_date_pwd = pack("A*",$nonce) . pack("A*",$CREATIONDATE) . pack("H*",sha1($password));
$PASSWORDDIGEST = base64_encode(pack('H*', sha1($nonce_date_pwd)));
Now, my PHP is not good, but reading up, the H* means hex encoded. So that looks like it uses a hex coding of the SHA-1 of the password in the algorithm, not a base64 coded version.
In python :-
hashedpassword = sha.new(password).digest()
digest = sha.new(nonce + CREATIONDATE + hashedpassword).digest()
PASSWORDDIGEST = base64.b64encode(digest)
Which looks like using the raw SHA-1 in the algorithm, not a base64 coded.
So I am thinking the spec, and the example they gave me, is wrong. I even tried using their python code to generate Created, Nonce, and Password fields.
Sadly, it is not working to get past "Authorization Failure.", and it looks like the have buggered off home.
This is NOT THE WAY to specify an API. You need clear documentation that is actually correct, and you need helpful error messages on your on-ramp system. Not impressed at all.
Update: Nope, they are there until 6pm - yay, well done Royal Mail - trying to find the problem for me now.
Update: We are back to the base64(SHA-1(plaintextpassword)) now, and using a different applicationId, and working. Yes, that disagrees with all of their code samples! So hoping to get on live system soon.
2015-12-23
We must all be better than average
We have seen UK politicians afflicted with this stupidity too, but today it seems that Clinton stated, categorically "I wouldn't keep any school open that wasn't doing a better than average job".
http://www.weeklystandard.com/clinton-i-wouldnt-keep-any-school-open-that-wasnt-doing-a-better-than-average-job/article/2000327#.VnoXnsQZTmq.facebook
Now, I should not have to explain this, but just in case I do...
"average" is the "middle". There are lots of technical ways one can work out "average" and I won't go in to any detail, but basically, in generally terms, if you have a set of something, like "schools", always, half will be "below average" and half will be "above average".
And what is worse, if you closed the half of all schools that were "below average", then the average would change, and would now mean that half the of the schools you have left are "above average" and half would be "below average".
You keep doing this until the best school in the country is the only one left open, and it is "only average".
I say this with some dismay - why are people so thick?, but I have to remember that OFCOM defined in the code of practice that 10% of all broadband lines are "faulty" as they are in the "bottom 10th percentile", and again, if removed or fixed, then the average and 10th percentile moves up. It is the same stupidity right here in the UK in official OFCOM code of practice.
http://www.weeklystandard.com/clinton-i-wouldnt-keep-any-school-open-that-wasnt-doing-a-better-than-average-job/article/2000327#.VnoXnsQZTmq.facebook
Now, I should not have to explain this, but just in case I do...
"average" is the "middle". There are lots of technical ways one can work out "average" and I won't go in to any detail, but basically, in generally terms, if you have a set of something, like "schools", always, half will be "below average" and half will be "above average".
And what is worse, if you closed the half of all schools that were "below average", then the average would change, and would now mean that half the of the schools you have left are "above average" and half would be "below average".
You keep doing this until the best school in the country is the only one left open, and it is "only average".
I say this with some dismay - why are people so thick?, but I have to remember that OFCOM defined in the code of practice that 10% of all broadband lines are "faulty" as they are in the "bottom 10th percentile", and again, if removed or fixed, then the average and 10th percentile moves up. It is the same stupidity right here in the UK in official OFCOM code of practice.
WSSE PasswordDigest
Update: See latest blog post on this.
Posting for posterity - as it was driving me nuts. Next person stuck on this may find this post!
Trying to sort the API to Royal Mail right now. They use SOAP headers for XML messages posted via https. This is very common and very easy to put together if you have an XML library.
Now, apart from the fact it is all over https, and they want us to use a client certificate even, the XML that is sent has a system for authentication in the SOAP header which uses a UsernameToken
This contains a Username and could just contain a Password. We are using a secure channel so why not. But no, it has more. You can send, instead, a PasswordDigest which is a hash.
Sensibly, to avoid replay attacks and make the password itself hidden, the hash used is not just a simple hash of the password (that would be too easy, though it is an option). What they do is send a Nonce and a Created timestamp, and then use these with the password to make a PasswordDigest.
The timestamp allows for Nonce retention for a short tim for replay controls. The random data of the Nonce ensures the hash is not the same and cannot be rainbow table attacked for the password. So sensible enough if you were using a less secure channel.
The specification explains the digest used. It even has an example. But like every other example I have seen so far it fails to state the password that applies so that someone can actually work through the example and confirm they have the code right.
What is worse is the Royal Mail complicate the password they use which is not the password they tell you but a hash of it, sort of.
So, this is specific to Royal Mail ApiAPI, but I expect if general to WSSE Password Digests.
The spec says:-
Password_Digest = Base64 ( SHA-1 ( nonce + created + password ) )
Base64 of SHA-1 is simple, so what is the data you pass. Password seems simple, it is the password. Created, well, that is the actual text of the Created field, so an XML style date/time. The fact that it is that exact text seems not 100% clear, but it is. They go to the bother of saying it is the UTF-8 text of the date/time which seems odd as a date time surely can never contain any non simple printable ASCII characters. The Password is also UTF-8 which is worth specifying.
But what of the Nonce? Well, that is, for Royal Mail, a 16 byte random number, encoded when sent using Base64. It was very unclear if the SHA-1 used the text Nonce as sent (i.e. the Base64 text) or the raw 16 byte value. I have seen several code fragments. I think it depends on how the Nonce is encoded, and if it specifically states it is Base64 encoded in the data structure (e.g. EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary") then you use the underlying 16 byte binary value in the SHA-1. That is what is needed for the Royal Mail API.
That sounds fine, but the Royal Mail do not do quote that. They state "For Shipping API V2 the password used in the below formula is the base 64 encoding of the SHA-1 hash of the plain text password". OK, so instead of "password" you use the base64 of the SHA-1. They then state the digest is:-
Password_Digest = Base64(SHA-1(Nonce + Created + SHA-1(Password)))
Well, that is simple enough. Take password, SHA-1 it, Base64 it, then use that in above, so SHA-1 that, then append on to Nonce and Created and SHA-1 that and base64 it. Well no!!!
What them mean actually is, where Password is plain text, use :-
Password_Digest = Base64(SHA-1(Nonce + Created + Base64(SHA-1(Password))))
And that matches their examples!
Update: It matches the example, but is apparently wrong. They have sent a new example where the password is just the SHA-1(plaintextpassword) not the Base64(SHA-1(plaintextpassword)), see later post...
Update: Finally working on a test application ID, and using this crazy base64(SHA-1(Password)) logic as I originally documented here. Not working on the actual on boarding system though.
In this example the password is "password".
Posting for posterity - as it was driving me nuts. Next person stuck on this may find this post!
Trying to sort the API to Royal Mail right now. They use SOAP headers for XML messages posted via https. This is very common and very easy to put together if you have an XML library.
Now, apart from the fact it is all over https, and they want us to use a client certificate even, the XML that is sent has a system for authentication in the SOAP header which uses a UsernameToken
This contains a Username and could just contain a Password. We are using a secure channel so why not. But no, it has more. You can send, instead, a PasswordDigest which is a hash.
Sensibly, to avoid replay attacks and make the password itself hidden, the hash used is not just a simple hash of the password (that would be too easy, though it is an option). What they do is send a Nonce and a Created timestamp, and then use these with the password to make a PasswordDigest.
The timestamp allows for Nonce retention for a short tim for replay controls. The random data of the Nonce ensures the hash is not the same and cannot be rainbow table attacked for the password. So sensible enough if you were using a less secure channel.
The specification explains the digest used. It even has an example. But like every other example I have seen so far it fails to state the password that applies so that someone can actually work through the example and confirm they have the code right.
What is worse is the Royal Mail complicate the password they use which is not the password they tell you but a hash of it, sort of.
So, this is specific to Royal Mail ApiAPI, but I expect if general to WSSE Password Digests.
The spec says:-
Password_Digest = Base64 ( SHA-1 ( nonce + created + password ) )
Base64 of SHA-1 is simple, so what is the data you pass. Password seems simple, it is the password. Created, well, that is the actual text of the Created field, so an XML style date/time. The fact that it is that exact text seems not 100% clear, but it is. They go to the bother of saying it is the UTF-8 text of the date/time which seems odd as a date time surely can never contain any non simple printable ASCII characters. The Password is also UTF-8 which is worth specifying.
But what of the Nonce? Well, that is, for Royal Mail, a 16 byte random number, encoded when sent using Base64. It was very unclear if the SHA-1 used the text Nonce as sent (i.e. the Base64 text) or the raw 16 byte value. I have seen several code fragments. I think it depends on how the Nonce is encoded, and if it specifically states it is Base64 encoded in the data structure (e.g. EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary") then you use the underlying 16 byte binary value in the SHA-1. That is what is needed for the Royal Mail API.
That sounds fine, but the Royal Mail do not do quote that. They state "For Shipping API V2 the password used in the below formula is the base 64 encoding of the SHA-1 hash of the plain text password". OK, so instead of "password" you use the base64 of the SHA-1. They then state the digest is:-
Password_Digest = Base64(SHA-1(Nonce + Created + SHA-1(Password)))
Well, that is simple enough. Take password, SHA-1 it, Base64 it, then use that in above, so SHA-1 that, then append on to Nonce and Created and SHA-1 that and base64 it. Well no!!!
What them mean actually is, where Password is plain text, use :-
Password_Digest = Base64(SHA-1(Nonce + Created + Base64(SHA-1(Password))))
And that matches their examples!
Update: It matches the example, but is apparently wrong. They have sent a new example where the password is just the SHA-1(plaintextpassword) not the Base64(SHA-1(plaintextpassword)), see later post...
Update: Finally working on a test application ID, and using this crazy base64(SHA-1(Password)) logic as I originally documented here. Not working on the actual on boarding system though.
In this example the password is "password".
<UsernameToken xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" wsu:Id="55555555555555555555555555555555">
<Username>FredBloggs</Username>
<Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordDigest">Z+VcU4ICFQReHwG/Kt70jhJgtaM=</Password>
<Nonce EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary">VVVVVVVVVVVVVVVVVVVVVQ==</Nonce>
<Created>2015-12-23T18:50:29Z</Created>
</UsernameToken>
</UsernameToken>
- SHA-1 of "password" is 5baa61e4c9b93f3f0682250b6cf8331b7ee68fd8 hex
- Base64 of that is W6ph5Mm5Pz8GgiULbPgzG37mj9g=
- Nonce is VVVVVVVVVVVVVVVVVVVVVQ== in base 64
- Which is 55555555555555555555555555555555 hex
- Which is UUUUUUUUUUUUUUUU in text
- Created is 2015-12-23T18:50:29Z
- SHA-1 of UUUUUUUUUUUUUUUU2015-12-23T18:50:29ZW6ph5Mm5Pz8GgiULbPgzG37mj9g= is 67e55c53820215045e1f01bf2adef48e1260b5a3 hex
- Which, in base64 is Z+VcU4ICFQReHwG/Kt70jhJgtaM=
My experience with @iMakrStore
I do not believe I had dealt with iMakr before, but having met them in a show early last year and seen the Wasp 2040 Turbo I was keen to get one.
May 26: I emailed asking if they could let me know when they have them in stock.
Nov 13: Good start - emailed to say "we have the Turbo printer!"
Nov 13: I could not order as the on-line order form would not get past the "pick courier" stage
Nov 13: They apologised, fixed that, but then the card payment page was not a secure page*
Nov 13: I paid by bank transfer and placed an order
Nov 17: Informed "should have that machine by the end of next week" [i.e. 27th] - not happy as was told they had the machine on the 13th - seems I was misled
Nov 27: No printer, chasing, not even getting replies to emails now
Nov 30: Told expected stock on 2nd Dec so deliver 4th.
Nov 30: Told DHL accident in Italy, so delay
Dec 01: Various messages over several days told different dates
Dec 04: Told parcel finally on the way.
Dec 07: Finally arrived - does not work - keeps beeping.
Dec 08: Told likely firmware, so waiting for firmware from Wasp
Dec 09: Told very likely firmware, still waiting for firmware - how is this not just an email?
Dec 10: Told still waiting for firmware?!?
Dec 10: Now told they cannot have the firmware and maybe it is not the firmware?
Dec 10: Offered on-site visit and/or loan unit if cannot fix - this is good of them
Dec 11: Engineer on-site, cannot fix, lent us the older model**
Dec 14: iMakr did get me the necessary config for CURA to work the printer they loaned. It works.
Dec 16: Told a replacement part has been shipped by Wasp and expected on 17th
Dec 17: We chased several times and told "We just received your part from Wasp and our engineering team will now proceed with the replacement." yay!
Dec 18: Told "Your machine is being worked on now."
Dec 21: Chased again, eventually (6pm) told they sent the wrong part - why could they not say this on the 17th when they got it and started working on it, or 18th when they were working on it?
Dec 21: We pointed out that had we ordered a new one on the day the faulty one arrived, according to the "two week delivery" on their web site, we would have had it by now. Told that the "two week delivery" is not actually "two week delivery"!
Dec 22: We give up and ask for a refund
Dec 23: Refund arrived, good.
* I later found it was using an iframe, but this is clearly very bad practice, see http://www.revk.uk/2015/11/https-and-shopping-for-dummies.html
** Whilst I do think it was good that they sent an engineer and loan unit, it is a far cry from simply having sent a replacement printer the day it arrived broken. I was told today that they did this, not because it was the "right thing to do" but because we said we have a non working printer sat in our lobby and iMakr considered that bad publicity for them. In fact we do not have customers coming to our office, as it happens. I was somewhat surprised that the chap from iMakr admitted that was the reason they sent an engineer!
I really do not know if I will buy from them again. Maybe they were just very unlucky.
May 26: I emailed asking if they could let me know when they have them in stock.
Nov 13: Good start - emailed to say "we have the Turbo printer!"
Nov 13: I could not order as the on-line order form would not get past the "pick courier" stage
Nov 13: They apologised, fixed that, but then the card payment page was not a secure page*
Nov 13: I paid by bank transfer and placed an order
Nov 17: Informed "should have that machine by the end of next week" [i.e. 27th] - not happy as was told they had the machine on the 13th - seems I was misled
Nov 27: No printer, chasing, not even getting replies to emails now
Nov 30: Told expected stock on 2nd Dec so deliver 4th.
Nov 30: Told DHL accident in Italy, so delay
Dec 01: Various messages over several days told different dates
Dec 04: Told parcel finally on the way.
Dec 07: Finally arrived - does not work - keeps beeping.
Dec 08: Told likely firmware, so waiting for firmware from Wasp
Dec 09: Told very likely firmware, still waiting for firmware - how is this not just an email?
Dec 10: Told still waiting for firmware?!?
Dec 10: Now told they cannot have the firmware and maybe it is not the firmware?
Dec 10: Offered on-site visit and/or loan unit if cannot fix - this is good of them
Dec 11: Engineer on-site, cannot fix, lent us the older model**
Dec 14: iMakr did get me the necessary config for CURA to work the printer they loaned. It works.
Dec 16: Told a replacement part has been shipped by Wasp and expected on 17th
Dec 17: We chased several times and told "We just received your part from Wasp and our engineering team will now proceed with the replacement." yay!
Dec 18: Told "Your machine is being worked on now."
Dec 21: Chased again, eventually (6pm) told they sent the wrong part - why could they not say this on the 17th when they got it and started working on it, or 18th when they were working on it?
Dec 21: We pointed out that had we ordered a new one on the day the faulty one arrived, according to the "two week delivery" on their web site, we would have had it by now. Told that the "two week delivery" is not actually "two week delivery"!
Dec 22: We give up and ask for a refund
Dec 23: Refund arrived, good.
* I later found it was using an iframe, but this is clearly very bad practice, see http://www.revk.uk/2015/11/https-and-shopping-for-dummies.html
** Whilst I do think it was good that they sent an engineer and loan unit, it is a far cry from simply having sent a replacement printer the day it arrived broken. I was told today that they did this, not because it was the "right thing to do" but because we said we have a non working printer sat in our lobby and iMakr considered that bad publicity for them. In fact we do not have customers coming to our office, as it happens. I was somewhat surprised that the chap from iMakr admitted that was the reason they sent an engineer!
I really do not know if I will buy from them again. Maybe they were just very unlucky.
2015-12-21
Bah, humbug!
'tis the season, and all that.
Always seemed a bit odd. There are many people in one's life, friends, relatives, and it is nice to get them gifts, presents, to help out.
Always seemed odd that one delayed doing so until a specific date such as birthday or Christmas, or worse, one gets credit to get something for a date and paying for it and interest later rather than waiting until you can afford to give gifts.
I try to give gifts when I can, and when I think of something, and when people need it and I can afford to be generous. Though, to be honest, I get to give the gift of a car mechanic when someone's car breaks down, or to buy something that has broken or been stolen, or to pay for someone's petrol when they come to visit, or any number of things I am randomly called to pay for almost every bloody day. And to say "no" is to be made to feel bad and mean. I bet this blog post is seen as "mean" which proves my point.
So maybe the whole Christmas thing makes sense, when someone demands money I can say "see what Santa brings" [in 8 months time] or some such, rather than reaching for my wallet yet again. I may have to try that next time.
Happy winter solstice for tomorrow, everyone.
Update: I nearly forgot, and thanks for the comment on my post, the other side of this can be worse.
I was thinking of getting a new roomba for my "man cave", but when I mentioned it several people said "wait and see what Santa brings". I foresee three scenarios...
Always seemed a bit odd. There are many people in one's life, friends, relatives, and it is nice to get them gifts, presents, to help out.
Always seemed odd that one delayed doing so until a specific date such as birthday or Christmas, or worse, one gets credit to get something for a date and paying for it and interest later rather than waiting until you can afford to give gifts.
I try to give gifts when I can, and when I think of something, and when people need it and I can afford to be generous. Though, to be honest, I get to give the gift of a car mechanic when someone's car breaks down, or to buy something that has broken or been stolen, or to pay for someone's petrol when they come to visit, or any number of things I am randomly called to pay for almost every bloody day. And to say "no" is to be made to feel bad and mean. I bet this blog post is seen as "mean" which proves my point.
So maybe the whole Christmas thing makes sense, when someone demands money I can say "see what Santa brings" [in 8 months time] or some such, rather than reaching for my wallet yet again. I may have to try that next time.
Happy winter solstice for tomorrow, everyone.
Update: I nearly forgot, and thanks for the comment on my post, the other side of this can be worse.
I was thinking of getting a new roomba for my "man cave", but when I mentioned it several people said "wait and see what Santa brings". I foresee three scenarios...
- Nobody gets a a roomba, so I delayed buying one for no good reason, inconveniencing myself for the sake of "Christmas".
- Someone gets me one, but way cheaper / inferior model than I would have gotten, but I feel obliged to accept it, use it, and never actually get the model I would have bought, forever annoyed by some lack of feature until one day it breaks in many years time
- Someone gets me one, the expensive one that I would have got myself, or possibly one more expensive than I would have got. This would be great, but would be an expensive gift that I need to some how have reciprocated. What is worse, is if it is way more than I could afford, and I would feel compelled to spend more than I can afford getting that person a gift at the next opportunity.
All of these mean delaying getting something I want to get, one way or another! Alternatively, I get one anyway, and end up embarrassing me or someone else if they do get me one too. Worse, more than one person gets me one!
I am with Sheldon on this. The only gifts that really work in such a scenario are the unexpected gift that you would never have thought of or realised was possible, where the cost is not known or not relevant somehow. I was impressed that Sandra was got a signed copy of a script for her birthday and only wish I had thought of that. I once got someone a radiation counter! But still, why not get such gifts when you can, rather then delay them to some specific date?
2015-12-17
More on #IPBill
I have submitted more written evidence to the Joint Committee on the Draft Investigatory Powers Bill today, after giving oral evidence on the 9th.
There are still a lot of issues, but I thought I would try and explain some of the technical challenges as well as possible for them.
PDF.
Written evidence is open to all to submit, and anything that helps explain the issues: moral, ethical, technical, practical, etc, will help challenge the bill and hopefully make changes.
There are still a lot of issues, but I thought I would try and explain some of the technical challenges as well as possible for them.
PDF.
Written evidence is open to all to submit, and anything that helps explain the issues: moral, ethical, technical, practical, etc, will help challenge the bill and hopefully make changes.
You have less than a week to make your voice heard on the draft #IPBill: https://t.co/i013YQ09Cp pic.twitter.com/bOIfkgux8O
— Big Brother Watch (@bbw1984) December 16, 2015
Splitting up BT?
One of the questions being considered at the moment is whether BT should be somehow split up, e.g. Openreach, BT Wholesale, BT Retail, all separate actual entities somehow. People have asked my views on this many times.
Sadly I do not think I can provide a concrete answer. For a start, BT "operate" as if they were separate companies - a sort of corporate schizophrenia - so would splitting them for real change the way they operate - possibly not.
The issues come down to the money - at the moment, as a BT shareholder, it makes no odds how much BT Retail pay BT Wholesale or how much BT Wholesale pay Openreach. Such prices can be set as high as they can get away with as moving money within one entity does not really change the company as a whole but gets more money from the other telcos also paying the same price. However, I suspect that is a very simplistic view.
Personally I am not convinced the logical split we have now is in the right place. I can see merit in a near monopoly infrastructure provider if they work for the general good of the telecommunications industry - perhaps as a non-profit organisation even. But should that be at a level that charges for duct space, or for copper pairs, or what? I really do not know.
However, we do have some fun with the current situation. A case that crops up rather a lot where someone in BT Wholesale needs to have some work done to get a fault fixed, and that work is done by "their suppliers", Openreach. But even though BT Wholesale has a service level agreement with us, they have a "contract" with their "suppliers" that does not match that and allows their suppliers to take way longer to fix the fault than has been agreed with us.
They hide behind this as if saying "the contract with our suppliers does not allow us to get the work done any quicker" was actually a valid excuse, and they will even outright say "Openreach is a separate entity"!
Now, if their suppliers were a separate company, you have a situation where they have agreed a service level agreement with us, but then agreed with subcontractors a service level agreement which means they cannot meet the agreement with us. Selling us a service with an unattainable service level agreement would surely be some sort of fraud? Or at the very least a shady business practice.
For example, we sell the "7 hour guaranteed fix" service to our customers for a fee, and even pass on the relatively small pay out from BT for failing to meet that time scale. We buy the "7 hour guaranteed fix" service from BT. But if you do the maths you find that actually we would be better not buying the "7 hour fix" from BT. This is because the pay out is so low that even if BT always miss the 7 hour target we have with our customer, we still make more profit. On top of which BT will sometimes either fix within a short time anyway (if not needing an engineer) so we don't have to pay out, or miss the 20 to 40 hour SLA we have with them normally and so pay us anyway. I would not do that as I think it would be a dodgy and dishonest practice. But it seems like BT think it is quite acceptable as long as they pay the small amount of compensation agreed for missing the target.
However, at present, they are contracting with themselves not a real subcontractor or supplier. I do wonder if legally one can in fact contract with oneself at all anyway?
This means that the engineers and people working for "their suppliers" (Openreach) have agreed a service level with us directly - they are contractually bound to try and fix within the SLA agreed with us, not some fictitious SLA agreed with themselves trading as BT Wholesale.
Trying to get that through to them is really quite hard - the staff have been brainwashed in to thinking they are actually separate entities when they are not. They are simply operating as if they are separate entities. British Telecommunications plc (company 1800000) and Andrews & Arnold Ltd (company 3342760) have a contract and they should adhere to it, whichever part of company 1800000 they represent!
Sadly I do not think I can provide a concrete answer. For a start, BT "operate" as if they were separate companies - a sort of corporate schizophrenia - so would splitting them for real change the way they operate - possibly not.
The issues come down to the money - at the moment, as a BT shareholder, it makes no odds how much BT Retail pay BT Wholesale or how much BT Wholesale pay Openreach. Such prices can be set as high as they can get away with as moving money within one entity does not really change the company as a whole but gets more money from the other telcos also paying the same price. However, I suspect that is a very simplistic view.
Personally I am not convinced the logical split we have now is in the right place. I can see merit in a near monopoly infrastructure provider if they work for the general good of the telecommunications industry - perhaps as a non-profit organisation even. But should that be at a level that charges for duct space, or for copper pairs, or what? I really do not know.
However, we do have some fun with the current situation. A case that crops up rather a lot where someone in BT Wholesale needs to have some work done to get a fault fixed, and that work is done by "their suppliers", Openreach. But even though BT Wholesale has a service level agreement with us, they have a "contract" with their "suppliers" that does not match that and allows their suppliers to take way longer to fix the fault than has been agreed with us.
They hide behind this as if saying "the contract with our suppliers does not allow us to get the work done any quicker" was actually a valid excuse, and they will even outright say "Openreach is a separate entity"!
Now, if their suppliers were a separate company, you have a situation where they have agreed a service level agreement with us, but then agreed with subcontractors a service level agreement which means they cannot meet the agreement with us. Selling us a service with an unattainable service level agreement would surely be some sort of fraud? Or at the very least a shady business practice.
For example, we sell the "7 hour guaranteed fix" service to our customers for a fee, and even pass on the relatively small pay out from BT for failing to meet that time scale. We buy the "7 hour guaranteed fix" service from BT. But if you do the maths you find that actually we would be better not buying the "7 hour fix" from BT. This is because the pay out is so low that even if BT always miss the 7 hour target we have with our customer, we still make more profit. On top of which BT will sometimes either fix within a short time anyway (if not needing an engineer) so we don't have to pay out, or miss the 20 to 40 hour SLA we have with them normally and so pay us anyway. I would not do that as I think it would be a dodgy and dishonest practice. But it seems like BT think it is quite acceptable as long as they pay the small amount of compensation agreed for missing the target.
However, at present, they are contracting with themselves not a real subcontractor or supplier. I do wonder if legally one can in fact contract with oneself at all anyway?
This means that the engineers and people working for "their suppliers" (Openreach) have agreed a service level with us directly - they are contractually bound to try and fix within the SLA agreed with us, not some fictitious SLA agreed with themselves trading as BT Wholesale.
Trying to get that through to them is really quite hard - the staff have been brainwashed in to thinking they are actually separate entities when they are not. They are simply operating as if they are separate entities. British Telecommunications plc (company 1800000) and Andrews & Arnold Ltd (company 3342760) have a contract and they should adhere to it, whichever part of company 1800000 they represent!
2015-12-16
The future of door keys?
I was pondering the future of conventional keys.
We already see cars not needing the conventional bit of cut/bent metal but using keyless access.
And I do not just mean my concerns over the ease of 3D printing an ABS "security" key. Any mechanical key can be copied.
Sadly many electronic keys can also be copied, it is just that the technology is a tad different. So where next?
One thought is proper cryptographically secure challenge/response keys using RFID and the like. Some places use these. They are ultimately secure and un-copiable.
But really, why do we even need "keys" any more, in any meaningful way - we all have smartphones. Surely unlocking a door is a simple matter of an app on your smart phone?
With any reader, the issue is, of course, the lock. You need electrically operated locks. There are several out there, even ones designed to bolt in to a euro-profile cylinder if you want.
But such a system has way more security options.
For a start, this can be local network only on the home wifi, so not reliant on the "Internet" in any way. An app can talk over the house wifi to a system that operates the lock. You need some UPS or backup power for wifi and the lock, but to be honest that is not difficult.
The data can be encrypted, and you can even use the GPS to confirm the device is really local and use finger print reader to authenticate.
From a user point of view it could be a matter of finger click on app. No need for readers or key fobs or shit at all.
Maybe "readers" and "keys" will truly become a thing of the past?
Of course it may eventually get to the point of good facial recognition and the doors you are allowed to open always are just open for you.
P.S. I love the idea that an estate agent will be totally flummoxed when selling my house that there are no actual "keys" to hand over :-)
We already see cars not needing the conventional bit of cut/bent metal but using keyless access.
And I do not just mean my concerns over the ease of 3D printing an ABS "security" key. Any mechanical key can be copied.
Sadly many electronic keys can also be copied, it is just that the technology is a tad different. So where next?
One thought is proper cryptographically secure challenge/response keys using RFID and the like. Some places use these. They are ultimately secure and un-copiable.
But really, why do we even need "keys" any more, in any meaningful way - we all have smartphones. Surely unlocking a door is a simple matter of an app on your smart phone?
With any reader, the issue is, of course, the lock. You need electrically operated locks. There are several out there, even ones designed to bolt in to a euro-profile cylinder if you want.
But such a system has way more security options.
For a start, this can be local network only on the home wifi, so not reliant on the "Internet" in any way. An app can talk over the house wifi to a system that operates the lock. You need some UPS or backup power for wifi and the lock, but to be honest that is not difficult.
The data can be encrypted, and you can even use the GPS to confirm the device is really local and use finger print reader to authenticate.
From a user point of view it could be a matter of finger click on app. No need for readers or key fobs or shit at all.
Maybe "readers" and "keys" will truly become a thing of the past?
Of course it may eventually get to the point of good facial recognition and the doors you are allowed to open always are just open for you.
P.S. I love the idea that an estate agent will be totally flummoxed when selling my house that there are no actual "keys" to hand over :-)
2015-12-13
Bed levelling
Most 3D printers, use a system called fused filament fabrication which means extruding plastic (or whatever) a layer at a time fusing with the previous layer.
The most important layer is the first layer, layer 0, the one that sticks the filament, not to previous layers, but to the print bed itself.
The print bed can be various materials, for example the Makerbot has an acrylic sheet, and the Wasp has a glass sheet. In some cases the bed will have something on top of it, such as tape, or glue, and there is even a flexible mat that someone has made to allow the based to be removed, and flexed to get the print off the bed.
It can be, err, fun, removing items from the bed cleanly - if stuck too well it can be hard to remove without breaking the printed item. However, if not stuck well enough the item comes detached during printing. There is also an issue that large areas of plastic can bend as they cool, so a heated bed is often useful too, keeping the material a little flexible until printing is finished. The Wasp Delta 20x40 has a nice heated bed with glass plate.
However, with any 3D printer like this, one crucial step is ensuring the bed is level. I don't mean gravitationally level, though that no doubt helps, I means so that every point on the bed is seen by the printing mechanism as Z axis zero.
When printing that first layer the extruder nozzle has to be just above the bed. Too close and you block plastic coming out. To high and the plastic just sit on the bed rather than being forced on to it and sticking to it. The tolerance here is very small. Given that you may be printing 100μm layers, the levelling has to be well under that in terms of accuracy.
I have to say this is where the Wasp lets me down somewhat.
Firstly, one wonders why you cannot simply make the machine with a fixed bed that is level. I guess, with such fine tolerances that turns out to be hard, fair enough.
But what Wasp do is they have an adjustable bed - three knobs. And a process of move the head to key points and confirm it is just above the bed. Ideally you level just so you can fit a bit of paper between the bed and the print head.
To be fair, this is what Makerbot do too. However, with the Makerbot the mechanism has a fixed carriage for X and Y, so the bed has to be level so that you can print without moving the Z screw drive. Moving in Z axis is slow on that type of printer.
But Wasp is a delta printer - to move in X, Y, or Z you have to move all three motors. The levelness of the Z axis is a matter of mathematics. So you do not actually need a bed that is mechanically exactly level, you just need to measure where it is, and include that in the maths. This is one of the beauties of a delta printer.
So, what they need is a sensor to check the position of the bed. A simple microswitch will do - this could be on the head itself if slightly spring loaded, or perhaps a probe that you flip in to place to measure the bed. The measurement could be totally automatic, checking the key points (3, to get a plane), and confirming other points fit, and the bed is indeed actually flat. I can't help feeling that the mechanical elements to make one contact sensor on the head would be less work and cost that making the bed adjustable and 3 knobs to adjust it. If the sensor does not need manually putting in to place, the process could be done automatically at the start of every print even.
Please, Wasp, consider this in a future design.
The most important layer is the first layer, layer 0, the one that sticks the filament, not to previous layers, but to the print bed itself.
The print bed can be various materials, for example the Makerbot has an acrylic sheet, and the Wasp has a glass sheet. In some cases the bed will have something on top of it, such as tape, or glue, and there is even a flexible mat that someone has made to allow the based to be removed, and flexed to get the print off the bed.
It can be, err, fun, removing items from the bed cleanly - if stuck too well it can be hard to remove without breaking the printed item. However, if not stuck well enough the item comes detached during printing. There is also an issue that large areas of plastic can bend as they cool, so a heated bed is often useful too, keeping the material a little flexible until printing is finished. The Wasp Delta 20x40 has a nice heated bed with glass plate.
However, with any 3D printer like this, one crucial step is ensuring the bed is level. I don't mean gravitationally level, though that no doubt helps, I means so that every point on the bed is seen by the printing mechanism as Z axis zero.
When printing that first layer the extruder nozzle has to be just above the bed. Too close and you block plastic coming out. To high and the plastic just sit on the bed rather than being forced on to it and sticking to it. The tolerance here is very small. Given that you may be printing 100μm layers, the levelling has to be well under that in terms of accuracy.
I have to say this is where the Wasp lets me down somewhat.
Firstly, one wonders why you cannot simply make the machine with a fixed bed that is level. I guess, with such fine tolerances that turns out to be hard, fair enough.
But what Wasp do is they have an adjustable bed - three knobs. And a process of move the head to key points and confirm it is just above the bed. Ideally you level just so you can fit a bit of paper between the bed and the print head.
To be fair, this is what Makerbot do too. However, with the Makerbot the mechanism has a fixed carriage for X and Y, so the bed has to be level so that you can print without moving the Z screw drive. Moving in Z axis is slow on that type of printer.
But Wasp is a delta printer - to move in X, Y, or Z you have to move all three motors. The levelness of the Z axis is a matter of mathematics. So you do not actually need a bed that is mechanically exactly level, you just need to measure where it is, and include that in the maths. This is one of the beauties of a delta printer.
So, what they need is a sensor to check the position of the bed. A simple microswitch will do - this could be on the head itself if slightly spring loaded, or perhaps a probe that you flip in to place to measure the bed. The measurement could be totally automatic, checking the key points (3, to get a plane), and confirming other points fit, and the bed is indeed actually flat. I can't help feeling that the mechanical elements to make one contact sensor on the head would be less work and cost that making the bed adjustable and 3 knobs to adjust it. If the sensor does not need manually putting in to place, the process could be done automatically at the start of every print even.
Please, Wasp, consider this in a future design.
2015-12-12
Wasp Delta 20x40
I saw a new 3D printer back in March, the Wasp Delta 2040 Turbo which boasts a speed of 1m/s which outpaces pretty much all of its rivals.
In November I got an email confirming that iMakr had one, and ordered. To be honest iMakr have been a nightmare - from dodgy card payment page on their site, to delays and more delays shipping (one excuse was DHL van in Italy had crashed). Finally it arrived on Monday and did not work - dead on arrive (tone and blank display). It has taken until Friday to get them to sort anything, and all they have done is confirmed it really is dead, and lent me the slow Wasp Delta 2040.
However, it has given me a chance to play - the only thing this model lacks as I understand it is the extra speed.
I have commented on the Makerbot Replicator 2 before, which I still see as the clear "plug-in and print" model of 3D printers, but the Wasp is pretty close to that. The first thing I printed "just worked". However, there is a lot more to tinker with in the settings on the Wasp.
But let's look first at the most obvious difference - the Wasp is a "delta" printer. This means that it does not have a conventional X/Y axis, which has a motor to move a gantry one way, and a motor on that to move the head the other way, and a Z axis (in the case of Makerbot, the print bed moves). Instead, it has three vertical drive threads which move three arms, allowing the head to position anywhere. It is defined as having a cylindrical print area 200mm across by 400mm high. This is smaller in width than the Replicator 2 which is 285mm x 153mm and 155m high. However, it seems the Wasp will print a larger area - the scope of the delta is more of a triangular shape, and I have printed items over 220mm wide. It also allows the head to go up to 445mm high in the centre (not as high at the edges). Of course the height means it is quite a big machine.
It also has a heated bed and glass print plate which works really well - surprisingly so. On the Replicator 2 I used blue tape on the acrylic print bed to avoid things being welded to it, and then always had to print a raft. Large items would bend on the bed. On the Wasp I can print things with no raft and they just work - well, with exception of things with a very small contact like dice on their corner - but for that you can print a raft. You use a glue stick on the glass to help ensure it sticks well.
Like the Replicator 2 it uses an SD card for the print files.
Unlike the Replicator 2 it does not come with any software - but they recommend using Cura or other free slicer applications which generate generic gcode. Cura seems quite nice (using on Mac), very fast, understands when you insert an SD and is one click save and then offers an eject button direct from the app.
The downside is that the slicer has loads of options. Unlike the Makerbot where its software is all set up for PLA printing, and offers basically fast or fine, Cura offers loads, and I struggled to find any sensible defaults. I am printing PLA at 200℃ and a heated bed at 40℃. I have been playing with speeds around 200mm/s. Interestingly, printing at 0.1mm layers seems to just work - on the Makerbot you normally print at 0.2mm layers and use 0.1mm for "fine" detail, but that causes stringing. On this printer you can go to 0.05mm if you want (not tried it yet).
I'll post more when I get the Turbo version, and may do some side by side printing with the Makerbot.
However, I thought I would try a good test. This is with s/w and printer I only got last night. I don't actually have a front door key for the house at the moment (I use my own front door to my office here with electronic lock), but I ask my son to send me a photo of his key from his iPhone. He did. Working just from that one (slightly blury) photo, 15 minutes and 10 seconds later, I was opening the door with a 3D printed key. The actual print took 12 minutes, after a few minutes rendering the OpenSCAD 3D model and then slicing and transferring on SD card. I was able to "read" the depths from the photo first time. This is with an ABS high security lock ( happen to have a few of the tiny magnets lying around). I am pondering what locks to get for the house next :-)
In November I got an email confirming that iMakr had one, and ordered. To be honest iMakr have been a nightmare - from dodgy card payment page on their site, to delays and more delays shipping (one excuse was DHL van in Italy had crashed). Finally it arrived on Monday and did not work - dead on arrive (tone and blank display). It has taken until Friday to get them to sort anything, and all they have done is confirmed it really is dead, and lent me the slow Wasp Delta 2040.
However, it has given me a chance to play - the only thing this model lacks as I understand it is the extra speed.
I have commented on the Makerbot Replicator 2 before, which I still see as the clear "plug-in and print" model of 3D printers, but the Wasp is pretty close to that. The first thing I printed "just worked". However, there is a lot more to tinker with in the settings on the Wasp.
But let's look first at the most obvious difference - the Wasp is a "delta" printer. This means that it does not have a conventional X/Y axis, which has a motor to move a gantry one way, and a motor on that to move the head the other way, and a Z axis (in the case of Makerbot, the print bed moves). Instead, it has three vertical drive threads which move three arms, allowing the head to position anywhere. It is defined as having a cylindrical print area 200mm across by 400mm high. This is smaller in width than the Replicator 2 which is 285mm x 153mm and 155m high. However, it seems the Wasp will print a larger area - the scope of the delta is more of a triangular shape, and I have printed items over 220mm wide. It also allows the head to go up to 445mm high in the centre (not as high at the edges). Of course the height means it is quite a big machine.
It also has a heated bed and glass print plate which works really well - surprisingly so. On the Replicator 2 I used blue tape on the acrylic print bed to avoid things being welded to it, and then always had to print a raft. Large items would bend on the bed. On the Wasp I can print things with no raft and they just work - well, with exception of things with a very small contact like dice on their corner - but for that you can print a raft. You use a glue stick on the glass to help ensure it sticks well.
Like the Replicator 2 it uses an SD card for the print files.
Unlike the Replicator 2 it does not come with any software - but they recommend using Cura or other free slicer applications which generate generic gcode. Cura seems quite nice (using on Mac), very fast, understands when you insert an SD and is one click save and then offers an eject button direct from the app.
The downside is that the slicer has loads of options. Unlike the Makerbot where its software is all set up for PLA printing, and offers basically fast or fine, Cura offers loads, and I struggled to find any sensible defaults. I am printing PLA at 200℃ and a heated bed at 40℃. I have been playing with speeds around 200mm/s. Interestingly, printing at 0.1mm layers seems to just work - on the Makerbot you normally print at 0.2mm layers and use 0.1mm for "fine" detail, but that causes stringing. On this printer you can go to 0.05mm if you want (not tried it yet).
I'll post more when I get the Turbo version, and may do some side by side printing with the Makerbot.
However, I thought I would try a good test. This is with s/w and printer I only got last night. I don't actually have a front door key for the house at the moment (I use my own front door to my office here with electronic lock), but I ask my son to send me a photo of his key from his iPhone. He did. Working just from that one (slightly blury) photo, 15 minutes and 10 seconds later, I was opening the door with a 3D printed key. The actual print took 12 minutes, after a few minutes rendering the OpenSCAD 3D model and then slicing and transferring on SD card. I was able to "read" the depths from the photo first time. This is with an ABS high security lock ( happen to have a few of the tiny magnets lying around). I am pondering what locks to get for the house next :-)
2015-12-10
GCHQ and snooping
I need to make this clear to the non-techies, and this needs to be explained.
The Draft Investigatory Powers Bill covers a lot of stuff, but there are two main things...
1. What the hell the spooks do in GCHQ and making that legal now!
2. Snooping on us all for police investigations
A lot of what I have posted is on the second point. The stuff GCHQ do is harder to tackle. There is a shitload that is clearly morally wrong, and way too "big brother", but let's put that to one side for a second.
The bill covers "Data Retention" and "Intercept capabilities" that can impact us all.
Data Retention means your ISP, hopefully only the large ISPs and not A&A, keeping track of your life - everything you do on-line, every web site, the lot, and hence making it available to your local plod if they want. It is a lot of data about you, and simultaneously is both "sensitive personal data" that says a lot about you and also very "not useful to actually investigating crime".
"Intercept capabilities" mean designing systems that are not actually secure so that, just in case, the authorities can snoop on you if they want. It means any attempt for any UK provider to sell "secure communications" becomes a lie as they may have to include a way for them to "remove protection" from communications. Whilst you may not be a suspect, if your provider has that means then what they are selling is not secure, and not only can police access what you are doing, but so can determined criminals.
So the objections I have been raising here are not anything about "protecting us from terrorists" at all, they are about your life on display- for any hacker to extract from your ISP, and for any police officer that does not understand these things to assume you are up to no good.
It is about "big brother" in so many ways. Ultimately you have to accept that if the police had cameras in every room of everyone's house it would help with investigations. This is that sort of thing by the back door. It is not acceptable. It is SNOOPING!
Let's be clear here - it is SNOOPING. One person giving evidence did not recognise "snooping" as a warrant was needed. Well, even if a warrant is needed it is still snooping, but more importantly data retention and maintenance of intercept capability do not need warrant. Data retention is SNOOPING on all that we do on the Internet, simple as that.
So please, tell people to talk to their MP, to read my numerous blog posts, to take a stand.
This is not about headline grabbing and statistically insignificant issues like terrorists or pedophiles, this is about your life on record and held by your ISP for police to look at.
[Quite separately GCHQ need reining in, and even in the US the NSA has had many powers restricted]
The Draft Investigatory Powers Bill covers a lot of stuff, but there are two main things...
1. What the hell the spooks do in GCHQ and making that legal now!
2. Snooping on us all for police investigations
A lot of what I have posted is on the second point. The stuff GCHQ do is harder to tackle. There is a shitload that is clearly morally wrong, and way too "big brother", but let's put that to one side for a second.
The bill covers "Data Retention" and "Intercept capabilities" that can impact us all.
Data Retention means your ISP, hopefully only the large ISPs and not A&A, keeping track of your life - everything you do on-line, every web site, the lot, and hence making it available to your local plod if they want. It is a lot of data about you, and simultaneously is both "sensitive personal data" that says a lot about you and also very "not useful to actually investigating crime".
"Intercept capabilities" mean designing systems that are not actually secure so that, just in case, the authorities can snoop on you if they want. It means any attempt for any UK provider to sell "secure communications" becomes a lie as they may have to include a way for them to "remove protection" from communications. Whilst you may not be a suspect, if your provider has that means then what they are selling is not secure, and not only can police access what you are doing, but so can determined criminals.
So the objections I have been raising here are not anything about "protecting us from terrorists" at all, they are about your life on display- for any hacker to extract from your ISP, and for any police officer that does not understand these things to assume you are up to no good.
It is about "big brother" in so many ways. Ultimately you have to accept that if the police had cameras in every room of everyone's house it would help with investigations. This is that sort of thing by the back door. It is not acceptable. It is SNOOPING!
Let's be clear here - it is SNOOPING. One person giving evidence did not recognise "snooping" as a warrant was needed. Well, even if a warrant is needed it is still snooping, but more importantly data retention and maintenance of intercept capability do not need warrant. Data retention is SNOOPING on all that we do on the Internet, simple as that.
So please, tell people to talk to their MP, to read my numerous blog posts, to take a stand.
This is not about headline grabbing and statistically insignificant issues like terrorists or pedophiles, this is about your life on record and held by your ISP for police to look at.
[Quite separately GCHQ need reining in, and even in the US the NSA has had many powers restricted]
#IPBill Oral evidence to Joint Committee on Draft Investigatory Powers Bill
I was honoured to have the chance to give evidence to the Joint Committee on the IP Bill yesterday.
It was interesting, and I was not at all nervous. I think the number of live TV interviews I have done, where I have random questions fired at me, have helped a lot. Being politics I actually had a list of likely questions in advance, which helped me rehearse some answers.
http://parliamentlive.tv/event/index/54f42d6d-2377-4e98-9f6b-f21149c2b21f?in=17:12:14
In fact, most of the answers were much more on-the-spot than my prepared notes.
I was lucky to be on with only one person, James Blessing, who is very clued up. I am not sure which of us is more guilty of talking for too long when answering a question. I think we both did very well and I hope we helped the committee understand the issues.
[Somewhat edited from this point]
"Internet Connection Record" is not a defined thing - in the bill or in industry!
One thing that was a concern, and predicted by one of the ISPs at our Home Office meeting, was that one of the members (David Hanson MP) was adamant that an "Internet Connection Record" was "defined in the bill". He referred to page 25 and asked us to work out costs based on that definition. Page 25 is in the "explanatory notes" and not the bill, and itself is massively unclear. It basically says "It is a record of the services that they have connected to".
To try and explain this a little more, as I plan to tweet to David Hanson MP and ask him to take a look here if he can. I fully understand that to someone not technical, saying "It is a record of the services that they have connected to" seems reasonably clear. Sadly it really is not, and if you look at the actual wording of the bill, and not just the explanatory notes, it is less clear still.
Unlike a telephone call, or even just sending an email, even the definition of the term "connected" is complicated, as is defining the term "service". Actually what happens is packets of data are sent between devices, and as an ISP we send those packets on towards their destination. We don't "see" any sort of "connection" or "service", all was see is "packets".
One possible meaning could be that we log the destination IP address of each packet. Sadly this is not either easy or cheap as there are literally billions of such packets whizzing through our network every minute, and we are a small ISP.
There is a protocol for a type of "connection" used in the Internet, called TCP. This is only one of many types of connection that can be made but is the most common. So the meaning could be to log each such logical connection. This would mean making something of a jigsaw puzzle of the meta data (the destination and source addresses) in each of those billions of packets as they pass and tracking millions of simultaneous logical "connections" that are happening at any one time, then logging these. Again, this is neither easy nor cheap, and even more work than above. There are also many types of "connection" - an "Internet phone call" using a protocol calls SIP does not normally even use TCP but a "connectionless" protocol called UDP, so somehow that would need to be tracked and logged too.
Of course, it could be that what is meant is we must log is more a matter of logging each "web page" accessed with the name of the web site, and similarly for other "services". Indeed, some comments made by the Secretary of State suggested this may be what was meant. This means not only the jigsaw puzzle to construct those TCP connections, but actually looking in to the data that passes on those connections, connecting the data from many packets together, and looking for a part of the information sent called a Host: header. This is yet more complexity and work and cost. Again, web pages are just one type of communication that uses a "connection". There are many other types of "connection" that could be made, and new types will come along every day or even every few hours as new applications are developed and new innovations made. Each of these is not published - we know how "web pages" work because they follow a published standard, but mobile phone apps do not have to follow any such standard, they do not even have to use TCP to communicate. So we'd have to constantly research each and every new application and protocol that people invent anywhere in the world, work out what part of that data counts as "Relevant Communications Data" and record it in some format that the police know to ask for and understand. We would not have the help of the developers in this. Indeed, we'd have to buy and test every app ever published and reverse engineer it to work out what to log. That would be a huge on-going undertaking at huge cost, made massively worse by the fact that each ISP is on their own not allowed to tell anyone else what they are doing with data retention.
So the meaning of recording "what services you connect to" is really very very unclear, and the cost involved in making such logs is not something one can sensibly estimate without actual details.
I am considering another written submission to basically explain how this all works and the issues.
It was interesting, and I was not at all nervous. I think the number of live TV interviews I have done, where I have random questions fired at me, have helped a lot. Being politics I actually had a list of likely questions in advance, which helped me rehearse some answers.
http://parliamentlive.tv/event/index/54f42d6d-2377-4e98-9f6b-f21149c2b21f?in=17:12:14
In fact, most of the answers were much more on-the-spot than my prepared notes.
I was lucky to be on with only one person, James Blessing, who is very clued up. I am not sure which of us is more guilty of talking for too long when answering a question. I think we both did very well and I hope we helped the committee understand the issues.
[Somewhat edited from this point]
"Internet Connection Record" is not a defined thing - in the bill or in industry!
One thing that was a concern, and predicted by one of the ISPs at our Home Office meeting, was that one of the members (David Hanson MP) was adamant that an "Internet Connection Record" was "defined in the bill". He referred to page 25 and asked us to work out costs based on that definition. Page 25 is in the "explanatory notes" and not the bill, and itself is massively unclear. It basically says "It is a record of the services that they have connected to".
To try and explain this a little more, as I plan to tweet to David Hanson MP and ask him to take a look here if he can. I fully understand that to someone not technical, saying "It is a record of the services that they have connected to" seems reasonably clear. Sadly it really is not, and if you look at the actual wording of the bill, and not just the explanatory notes, it is less clear still.
Unlike a telephone call, or even just sending an email, even the definition of the term "connected" is complicated, as is defining the term "service". Actually what happens is packets of data are sent between devices, and as an ISP we send those packets on towards their destination. We don't "see" any sort of "connection" or "service", all was see is "packets".
One possible meaning could be that we log the destination IP address of each packet. Sadly this is not either easy or cheap as there are literally billions of such packets whizzing through our network every minute, and we are a small ISP.
There is a protocol for a type of "connection" used in the Internet, called TCP. This is only one of many types of connection that can be made but is the most common. So the meaning could be to log each such logical connection. This would mean making something of a jigsaw puzzle of the meta data (the destination and source addresses) in each of those billions of packets as they pass and tracking millions of simultaneous logical "connections" that are happening at any one time, then logging these. Again, this is neither easy nor cheap, and even more work than above. There are also many types of "connection" - an "Internet phone call" using a protocol calls SIP does not normally even use TCP but a "connectionless" protocol called UDP, so somehow that would need to be tracked and logged too.
Of course, it could be that what is meant is we must log is more a matter of logging each "web page" accessed with the name of the web site, and similarly for other "services". Indeed, some comments made by the Secretary of State suggested this may be what was meant. This means not only the jigsaw puzzle to construct those TCP connections, but actually looking in to the data that passes on those connections, connecting the data from many packets together, and looking for a part of the information sent called a Host: header. This is yet more complexity and work and cost. Again, web pages are just one type of communication that uses a "connection". There are many other types of "connection" that could be made, and new types will come along every day or even every few hours as new applications are developed and new innovations made. Each of these is not published - we know how "web pages" work because they follow a published standard, but mobile phone apps do not have to follow any such standard, they do not even have to use TCP to communicate. So we'd have to constantly research each and every new application and protocol that people invent anywhere in the world, work out what part of that data counts as "Relevant Communications Data" and record it in some format that the police know to ask for and understand. We would not have the help of the developers in this. Indeed, we'd have to buy and test every app ever published and reverse engineer it to work out what to log. That would be a huge on-going undertaking at huge cost, made massively worse by the fact that each ISP is on their own not allowed to tell anyone else what they are doing with data retention.
So the meaning of recording "what services you connect to" is really very very unclear, and the cost involved in making such logs is not something one can sensibly estimate without actual details.
I am considering another written submission to basically explain how this all works and the issues.
2015-12-09
Can you justify your Internet Connection Records?
The proposed "Internet Connection Records" in the Draft Investigatory Powers Bill will potentially log every web site you have visited and retain that for a year.
If you are reading this blog post, you would now have an "Internet Connection Record" showing that you visited pornhub.com, or more specifically pornhub's content delivery network cdn1b.static.pornhub.phncdn.com, because there is a 1 pixel image included in this blog post which your browser will have fetched automatically.
Now, that is not illegal, thankfully, but it is probably in the filtering for many large ISPs, so even before this bill comes in it is probably logged.
Don't worry - the image is just a graphic of the word "hub", nothing rude, but this leads to another slight irony. In order to make this slightly less of an invasion of privacy the "Internet Connection Records" are not the whole URL to log which image it was, just the web site. What is worse is if logging by DNS then it does not matter if one innocent image like this or an hour watching the site, it would make one DNS lookup for the web site name.
Could you explain that to the police if it was some illegal site?
It is probably just as well it was not Inspire, a magazine produced by an al-Qaeda-affiliated group, because the police said downloaders of the mag would face prosecution "irrespective of their motive". See this article for more on that (yes, the article is quite safe).
http://blogs.telegraph.co.uk/news/brendanoneill2/100215723/in-a-truly-free-society-i-should-have-the-right-to-read-al-qaedas-magazine/ [link seems no longer valid, oddly]
If you are reading this blog post, you would now have an "Internet Connection Record" showing that you visited pornhub.com, or more specifically pornhub's content delivery network cdn1b.static.pornhub.phncdn.com, because there is a 1 pixel image included in this blog post which your browser will have fetched automatically.
Now, that is not illegal, thankfully, but it is probably in the filtering for many large ISPs, so even before this bill comes in it is probably logged.
Don't worry - the image is just a graphic of the word "hub", nothing rude, but this leads to another slight irony. In order to make this slightly less of an invasion of privacy the "Internet Connection Records" are not the whole URL to log which image it was, just the web site. What is worse is if logging by DNS then it does not matter if one innocent image like this or an hour watching the site, it would make one DNS lookup for the web site name.
Could you explain that to the police if it was some illegal site?
It is probably just as well it was not Inspire, a magazine produced by an al-Qaeda-affiliated group, because the police said downloaders of the mag would face prosecution "irrespective of their motive". See this article for more on that (yes, the article is quite safe).
http://blogs.telegraph.co.uk/news/brendanoneill2/100215723/in-a-truly-free-society-i-should-have-the-right-to-read-al-qaedas-magazine/ [link seems no longer valid, oddly]
Why #IPBill is bad for business
In simple terms, why is the Draft Investigatory Powers Bill bad for business if it was to pass as it is now?
Two simple points...
1. Certainty
Companies cannot be certain if they will be required to help with data retention or equipment interference or intercept. Even normal companies, not communications providers can come in scope, as can schools, coffee shops with WiFi, almost anyone. We need certainty in business - so we can plan and run a business.
2. Trust
Companies need trust of customers and users - the bill can make your ISP in to "big brother" secretly, and make them be the people spying on you even when you have done nothing wrong.
This all makes the UK a country to avoid when setting up a company.
2015-12-08
Apparently, for a fee, BT can control the weather?
One of our customers has been inconvenienced by an appointment being missed for an fault repair. Sadly this does happen occasionally. We are arranging another appointment.
However, we just got an email that explains "Our suppliers are unable to influence the appointment date due to bad weather conditions it is really hard to bring the appointment date/ slot".
By the way when this person working for British Telecommunications plc (company 1800000) trading as BT Wholesale says "suppliers" he means British Telecommunications plc (company 1800000) trading as Openreach, so he is talking about his employer and the company he represents. I know, it is confusing.
Well, bad weather is indeed a problem some times, so understandable, except they go on to say "Alternate suggested by suppliers is to agree for extra charges so that appointment date can be forwarded."
I can only assume BT have means to influence the weather now, for a fee, of course.
However, we just got an email that explains "Our suppliers are unable to influence the appointment date due to bad weather conditions it is really hard to bring the appointment date/ slot".
By the way when this person working for British Telecommunications plc (company 1800000) trading as BT Wholesale says "suppliers" he means British Telecommunications plc (company 1800000) trading as Openreach, so he is talking about his employer and the company he represents. I know, it is confusing.
Well, bad weather is indeed a problem some times, so understandable, except they go on to say "Alternate suggested by suppliers is to agree for extra charges so that appointment date can be forwarded."
I can only assume BT have means to influence the weather now, for a fee, of course.
Police misleading the public to pass the Investigatory Powers Bill?
This is an excellent article that is well worth reading. http://www.thinksip.uk/2015/12/police-misleading-the-public-to-pass-the-communications-data-bill/
When I saw the tweet by NCA, I too thought it was misleading, but clearly the author of that article, Aled Treherne, has thought about it a lot more and has some experience in this area. He makes a lot of sense.
They also have a slightly different info-graphic (the one in blue) which is very nearly the same and full of exactly the same holes.
Finding which on-line services have been accessed is somewhat pointless - for a start, if an Apple phone, then it will be Apple, and almost certainly FaceBook, Twitter, and WhatsApp. There are only a few major on-line services that a kid might be accessing, and asking the parents and friends will get a lot more information a lot more quickly that waiting to convince a judge to order a foreign social media or messaging service to hand over information and hoping they comply.
They talk of linking actions to devices? Well, what action? having a FaceBook account? Basically, any social media or messaging service will have permanent connections and regular traffic regardless of "usage" or "actions", so Internet Connection Records tell you very little - you can probably link half the smart phones in they country to FaceBook - this helps you how exactly?
The very nature of social media is that people share things with their friends, so ask their friends what is going on - there may not even be an actual crime!
When I saw the tweet by NCA, I too thought it was misleading, but clearly the author of that article, Aled Treherne, has thought about it a lot more and has some experience in this area. He makes a lot of sense.
This is basically the same crap story that was trotted out when we met with the home office, and I find it shocking that it is still being pushed as if it made sense.
They also have a slightly different info-graphic (the one in blue) which is very nearly the same and full of exactly the same holes.
Finding which on-line services have been accessed is somewhat pointless - for a start, if an Apple phone, then it will be Apple, and almost certainly FaceBook, Twitter, and WhatsApp. There are only a few major on-line services that a kid might be accessing, and asking the parents and friends will get a lot more information a lot more quickly that waiting to convince a judge to order a foreign social media or messaging service to hand over information and hoping they comply.
They talk of linking actions to devices? Well, what action? having a FaceBook account? Basically, any social media or messaging service will have permanent connections and regular traffic regardless of "usage" or "actions", so Internet Connection Records tell you very little - you can probably link half the smart phones in they country to FaceBook - this helps you how exactly?
The very nature of social media is that people share things with their friends, so ask their friends what is going on - there may not even be an actual crime!
2015-12-06
Relativity
I am going to stick my neck out here and try and discuss something that is on the edge of my knowledge. There are a couple of areas I know are just beyond my grasp...
One is the maths of public key encryption and sadly I had this explained and I remember getting it properly for a couple of days and losing it - that must be what Alzheimer's feels like. To be clear I do understand all of the principles and implications, it is just the maths that is that far beyond my everyday usage that I cannot retain it.
The other is relativity - mostly I get this but it is the time dilation that has me stuck, so I'll explain the bits I do grasp and where I fail. I am not really expecting a lot of answers - I will get some, and they will either be maths over my head or examples I don't find convincing. There was a recent case of some kid getting a prize for explaining relativity but the explanation had holes in it as I saw it, big holes.
So far I have grasped that gravity and acceleration are the same. This seems actually quite intuitive to me - in that they feel the same. Free fall is the same as simply travelling through space and zero gravity. Gravity at 1G on the surface of the Earth feels the same as 1G of acceleration in a rocket in space. It all fits and make perfect sense to me that they are the same.
Also, the fact that in gravity you experience time dilation - time goes more slowly compared to this not in gravity. The effect of a gravitational force is seen the same for all observers. If I am accelerating at 1G, people observing can see that, and so can I, and we all agree. The same as if I am sat on a planet with 1G force, every observer agrees. The idea that there is a time dilation due to gravity (or acceleration) is fine by me. At 1G time is slower than at 0G, and faster than at 2G. All observers see the same. Everything is fine. Let's accept that.
The other effect is speed - which is always relative. If two people are travelling at a speed away from each other, each see the other as going more slowly. This is an effect that I would see as logically an "illusion", in the same way the seeing someone at a distance they look smaller, but you look smaller to them - a paradox. If ever they change to coming towards you, you see them going faster than you, and they see you going faster to them. Ultimately this effect cancels out when you meet and have no time dilation that has accumulatively between you as a result. Whether the effect is "real" or an "illusion" is academic.
So far so good.
The problem I have is that there seems to be some effects that are speed related and do not fall in to either of the above.
The idea that someone whizzing off in a space ship and coming back later will appear to have some measurable cumulative time dilation effect. Now, this makes sense if they have some acceleration involved either way as that would slow their time compared to the person that had no acceleration and was left behind. But is that the only effect?
Apparently the ISS in orbit had two effects, one is due to lower gravity so faster time, and one is a speed effect and slower time, and they do not cancel out quite. It is the speed effect I cannot get my head around as surely to an observer on Earth they spend half their time travelling away (and hence seem slower) and half their time travelling towards (and hence seem faster), but the net effect of speed is always zero, only gravity ultimately has a cumulative effect.
So that is where I am lost - I hope one day to grasp it, and blog it.
One is the maths of public key encryption and sadly I had this explained and I remember getting it properly for a couple of days and losing it - that must be what Alzheimer's feels like. To be clear I do understand all of the principles and implications, it is just the maths that is that far beyond my everyday usage that I cannot retain it.
The other is relativity - mostly I get this but it is the time dilation that has me stuck, so I'll explain the bits I do grasp and where I fail. I am not really expecting a lot of answers - I will get some, and they will either be maths over my head or examples I don't find convincing. There was a recent case of some kid getting a prize for explaining relativity but the explanation had holes in it as I saw it, big holes.
So far I have grasped that gravity and acceleration are the same. This seems actually quite intuitive to me - in that they feel the same. Free fall is the same as simply travelling through space and zero gravity. Gravity at 1G on the surface of the Earth feels the same as 1G of acceleration in a rocket in space. It all fits and make perfect sense to me that they are the same.
Also, the fact that in gravity you experience time dilation - time goes more slowly compared to this not in gravity. The effect of a gravitational force is seen the same for all observers. If I am accelerating at 1G, people observing can see that, and so can I, and we all agree. The same as if I am sat on a planet with 1G force, every observer agrees. The idea that there is a time dilation due to gravity (or acceleration) is fine by me. At 1G time is slower than at 0G, and faster than at 2G. All observers see the same. Everything is fine. Let's accept that.
The other effect is speed - which is always relative. If two people are travelling at a speed away from each other, each see the other as going more slowly. This is an effect that I would see as logically an "illusion", in the same way the seeing someone at a distance they look smaller, but you look smaller to them - a paradox. If ever they change to coming towards you, you see them going faster than you, and they see you going faster to them. Ultimately this effect cancels out when you meet and have no time dilation that has accumulatively between you as a result. Whether the effect is "real" or an "illusion" is academic.
So far so good.
The problem I have is that there seems to be some effects that are speed related and do not fall in to either of the above.
The idea that someone whizzing off in a space ship and coming back later will appear to have some measurable cumulative time dilation effect. Now, this makes sense if they have some acceleration involved either way as that would slow their time compared to the person that had no acceleration and was left behind. But is that the only effect?
Apparently the ISS in orbit had two effects, one is due to lower gravity so faster time, and one is a speed effect and slower time, and they do not cancel out quite. It is the speed effect I cannot get my head around as surely to an observer on Earth they spend half their time travelling away (and hence seem slower) and half their time travelling towards (and hence seem faster), but the net effect of speed is always zero, only gravity ultimately has a cumulative effect.
So that is where I am lost - I hope one day to grasp it, and blog it.
All about trust
At the moment, at A&A, we provide Firebricks to customers in various circumstances, including as part of our high availability multiple line Office::1 product.
Usually, and with the customer's knowledge and consent, we will include a login (suitably locked down and secure) for us to access the FireBrick. This means that we can address tech support issues, and make config changes if requested and so on.
This works because our customers can, and do, trust us.
Of course, even where such access is via IPSec tunnels, it still involves trusting us!
The problem is not that we are untrustworthy, but the law, if the Draft Investigatory Powers Bill passes as it is now, would legally compel us to be untrustworthy as it forces us to co-operate with "equipment interference" and "removal of protection on communication" in some circumstances.
If it passes it is simply the case that our customers cannot trust us any more, as that trust will have been officially removed by the law of the land.
So, we are considering what we should do, before any such law comes in to force.
One obvious measure is to not have a login in to the FireBrick. However, the FireBrick does have a trick up its sleeve here - in that we can put our login on a profile and have that controlled by another login that the customer knows. That way they can decide to turn on or off our access as and when they need.
But we may have to go further and ensure that the signing of FireBrick code for automatic updates is managed outside A&A. The issue is that whilst A&A do most of the FireBrick R&D, A&A is also a communications provider. FireBrick is not, and hopefully is outside the scope of this legislation. So A&A might be compelled to add a back door to FireBrick code. If we make the signing authority for code updates exist outside A&A, in FireBrick Ltd only, then we cannot be compelled to issue code with holes in it just to allow equipment interference or removal of protection. What I don't know yet is whether that can be done simply by use of separate hats - i.e. can I be director of A&A and director of FireBrick, but only have authority to sign code in that second role and not the first. May have to take legal advice on that. If not, we will have to have a FireBrick member of staff and one that is not part of any telco that signs the code after reviewing changes - which is a nuisance.
Why do we care - surely this is all about targeted surveillance? Well maybe it is, but the wording of the bill allows bulk equipment interference, and a general intercept capability, neither of which is targeted.
But there is also the principle here - can our customers trust us? I would like them to be able to trust us. Should trusting your ISP be like trusting your lawyer or your MP? I really hate the idea that any law can officially undermine trust customers have in us - so yes, of course - we'll take steps to ensure that trust is valid and continues.
This is massively about perception. Even if we have no such orders - which we really doubt we would ever have - if people believe we may have, and lose trust, they may disable code updates. That would be bad for the product and ultimately their security as we would not be able to fix any weaknesses in their FireBrick that we discover. Ultimately, this really is all about trust!
Usually, and with the customer's knowledge and consent, we will include a login (suitably locked down and secure) for us to access the FireBrick. This means that we can address tech support issues, and make config changes if requested and so on.
This works because our customers can, and do, trust us.
Of course, even where such access is via IPSec tunnels, it still involves trusting us!
The problem is not that we are untrustworthy, but the law, if the Draft Investigatory Powers Bill passes as it is now, would legally compel us to be untrustworthy as it forces us to co-operate with "equipment interference" and "removal of protection on communication" in some circumstances.
If it passes it is simply the case that our customers cannot trust us any more, as that trust will have been officially removed by the law of the land.
So, we are considering what we should do, before any such law comes in to force.
One obvious measure is to not have a login in to the FireBrick. However, the FireBrick does have a trick up its sleeve here - in that we can put our login on a profile and have that controlled by another login that the customer knows. That way they can decide to turn on or off our access as and when they need.
But we may have to go further and ensure that the signing of FireBrick code for automatic updates is managed outside A&A. The issue is that whilst A&A do most of the FireBrick R&D, A&A is also a communications provider. FireBrick is not, and hopefully is outside the scope of this legislation. So A&A might be compelled to add a back door to FireBrick code. If we make the signing authority for code updates exist outside A&A, in FireBrick Ltd only, then we cannot be compelled to issue code with holes in it just to allow equipment interference or removal of protection. What I don't know yet is whether that can be done simply by use of separate hats - i.e. can I be director of A&A and director of FireBrick, but only have authority to sign code in that second role and not the first. May have to take legal advice on that. If not, we will have to have a FireBrick member of staff and one that is not part of any telco that signs the code after reviewing changes - which is a nuisance.
Why do we care - surely this is all about targeted surveillance? Well maybe it is, but the wording of the bill allows bulk equipment interference, and a general intercept capability, neither of which is targeted.
But there is also the principle here - can our customers trust us? I would like them to be able to trust us. Should trusting your ISP be like trusting your lawyer or your MP? I really hate the idea that any law can officially undermine trust customers have in us - so yes, of course - we'll take steps to ensure that trust is valid and continues.
This is massively about perception. Even if we have no such orders - which we really doubt we would ever have - if people believe we may have, and lose trust, they may disable code updates. That would be bad for the product and ultimately their security as we would not be able to fix any weaknesses in their FireBrick that we discover. Ultimately, this really is all about trust!
Lies, damn lies, and Royal Mail
I am not happy.
I have a parcel coming - nothing that important, but I was hoping to have it Friday, and indeed it should have arrived Friday. However, it was not guaranteed, so I'll let them off with that.
On Saturday I got a card. This was odd because I was at home most of the day and James was at home all day. The postman clearly did not knock.
The card states that the item is "being held safe at the local office, ready to collect the next working day" and lists times, including today (Sunday) as a working day 10:00-14:00.
I went to collect - it took half an hour even though only two people ahead of me, one of which was told they could not find his parcel!
I was told the item is not back at the depot. They say the card was left at 14:45, and the office closes at 15:00 so probably on the van still or something, and tough!
This is odd - as the card was left long before 14:45, and the card clearly shows the office was open until 17:30 on Saturday, not 15:00, so clearly I am being told lies. Indeed, so far I count 4 lies from the Royal Mail, and a waste of over half an hour of my time.
What is the deal with ID?
Normally the only credentials you need to accept a delivery, even if signed for, is to be in the right place at the right time - i.e. at the door when the postman arrives. There is no check of ID or even name, just that the item is handed to someone at the address.
But when collecting you need ID. Well, if you did not have the card I could understanding wanting some proof that you are someone living at that address, such as a driving licence matching photo and address.
But when you have a card, the mere possession of the card proves you were in the right place just not at the right time. Had the postman arrived at a different time they would have been happy to hand the item to you. So why even check ID if you have the card.
What is also weird is they want to check your name, not the address. Normally your name is irrelevant, when delivered in person. No check is made on name. So you would expect that an ID showing a matching address is all that would be needed. But no, they want to check name, and do not care if address matches.
Also a bit odd what ID they accept. The card says the full list of ID is on royalmail.com/redelivery which is only true if you are not accessing from a mobile, as the mobile site does not list ID types! Passport and driving licence, being official documents, I can understand, but bank cards - they are just issued by a company, so why not any old company ID card. I did not get to try my company ID (which was accepted by the Home Office even) because they did not actually have the parcel. I'll try tomorrow.
I have a parcel coming - nothing that important, but I was hoping to have it Friday, and indeed it should have arrived Friday. However, it was not guaranteed, so I'll let them off with that.
On Saturday I got a card. This was odd because I was at home most of the day and James was at home all day. The postman clearly did not knock.
The card states that the item is "being held safe at the local office, ready to collect the next working day" and lists times, including today (Sunday) as a working day 10:00-14:00.
I went to collect - it took half an hour even though only two people ahead of me, one of which was told they could not find his parcel!
I was told the item is not back at the depot. They say the card was left at 14:45, and the office closes at 15:00 so probably on the van still or something, and tough!
This is odd - as the card was left long before 14:45, and the card clearly shows the office was open until 17:30 on Saturday, not 15:00, so clearly I am being told lies. Indeed, so far I count 4 lies from the Royal Mail, and a waste of over half an hour of my time.
What is the deal with ID?
Normally the only credentials you need to accept a delivery, even if signed for, is to be in the right place at the right time - i.e. at the door when the postman arrives. There is no check of ID or even name, just that the item is handed to someone at the address.
But when collecting you need ID. Well, if you did not have the card I could understanding wanting some proof that you are someone living at that address, such as a driving licence matching photo and address.
But when you have a card, the mere possession of the card proves you were in the right place just not at the right time. Had the postman arrived at a different time they would have been happy to hand the item to you. So why even check ID if you have the card.
What is also weird is they want to check your name, not the address. Normally your name is irrelevant, when delivered in person. No check is made on name. So you would expect that an ID showing a matching address is all that would be needed. But no, they want to check name, and do not care if address matches.
Also a bit odd what ID they accept. The card says the full list of ID is on royalmail.com/redelivery which is only true if you are not accessing from a mobile, as the mobile site does not list ID types! Passport and driving licence, being official documents, I can understand, but bank cards - they are just issued by a company, so why not any old company ID card. I did not get to try my company ID (which was accepted by the Home Office even) because they did not actually have the parcel. I'll try tomorrow.
2015-12-05
The demise of SmartStamp
We make a lot of use of SmartStamp, thousands of pounds a year, as we use it for sending letters and parcels (routers, firebricks, etc) to customers every day.
It was quite a good system. When it started we reverse engineered the back end and integrated in to our systems. In a lot of ways this was superior to using any sort of franking machine:-
It was quite a good system. When it started we reverse engineered the back end and integrated in to our systems. In a lot of ways this was superior to using any sort of franking machine:-
- Address label or envelope printed in one go with postage included, instead of separately franking every item
- Postage selected from simple pull down of types of postage instead of typing the amount in to a franking machine which is easy to get wrong
- Some of our systems know the postage and link straight to that, so address and stamp is one button without thinking
- All postage properly logged to the user requesting it, and to the customer account to which it relates - makes misuse near impossible
- We can update the franking marks as we wish, e.g. Christmas, etc.
Basically the only disadvantage is that franking is usually a few pence cheaper, but the ongoing costs reduce that, and mostly we charge customers at cost for postage, so making no real difference.
However, sadly, for no apparent reason, they are stopping SmartStamp at the end of the year. So what next? Well, the good news is that Royal Mail have a postage API for parcels. This actually looks mostly sensible, but I am only just starting to look at this now - if I am crying in a corner later this week, you'll know why.
It does not look like this covers simple letters, so we may have to come up with a solution to that - I suspect just pre-paid C5 envelopes for now.
Anyway, for parcels, we'll need to do the labels, and they recommend this 4" wide thermal label printer. I was not that keen on yet another printer, but actually I think it may be a lot easier than putting labels through the big printer anyway, and Royal Mail supply the media for free (well, because you are spending on postage)! It is also nice and quick - just over a second to print a label.
I am going to integrate this API in to our linux based systems. I will need to print to the label printer from that system, which means ideally I want a printer that is Ethernet/IP networked and supports postscript. The label printer they supply is USB (or parallel or serial!), not Ethernet... Hmmm
Simple - try it on linux and see. Well, I am impressed. CUPS found it, recognised it, and only one option needed a couple of trial and error to get right. It just works from linux. So I set up a Raspberry Pi, installed CUPS, set up the printer and shared it, and then set up promote printing from the system that needs to print labels, and bloody hell it just works! As far as the system is concerned it has a networked postscript printer.
So, next step is testing the API and integrating with out systems - what could go wrong? I'll keep you "posted".
2015-12-03
Threat from terrorists
Whilst the threat to all of us in the UK from terrorist attacks may have massively increased from today, please remember that it still remains a fraction of risk we all face from every day things like driving.
Maybe I do need protection from some terrorists as defined by the law - I can see one person who seems to be threatening serious violence and serious damage to property by means of airstrikes to advance political cause and influence a government. Have I misread the definition?
Section 1 of Terrorism Act 2000
1 Terrorism: interpretation.
(1) In this Act “terrorism” means the use or threat of action where—
(a) the action falls within subsection (2),
(b) the use or threat is designed to influence the government or an international governmental organisation or to intimidate the public or a section of the public, and
(c) the use or threat is made for the purpose of advancing a political, religious, racial or ideological cause.
(2) Action falls within this subsection if it—
(a) involves serious violence against a person,
(b) involves serious damage to property,
(c) endangers a person’s life, other than that of the person committing the action,
(d) creates a serious risk to the health or safety of the public or a section of the public, or
(e) is designed seriously to interfere with or seriously to disrupt an electronic system.
Maybe I do need protection from some terrorists as defined by the law - I can see one person who seems to be threatening serious violence and serious damage to property by means of airstrikes to advance political cause and influence a government. Have I misread the definition?
Section 1 of Terrorism Act 2000
1 Terrorism: interpretation.
(1) In this Act “terrorism” means the use or threat of action where—
(a) the action falls within subsection (2),
(b) the use or threat is designed to influence the government or an international governmental organisation or to intimidate the public or a section of the public, and
(c) the use or threat is made for the purpose of advancing a political, religious, racial or ideological cause.
(2) Action falls within this subsection if it—
(a) involves serious violence against a person,
(b) involves serious damage to property,
(c) endangers a person’s life, other than that of the person committing the action,
(d) creates a serious risk to the health or safety of the public or a section of the public, or
(e) is designed seriously to interfere with or seriously to disrupt an electronic system.
2015-12-01
Back-dooring encryption, via the back door?
Yesterday's Draft Investigatory Powers Bill Select Committee included a question about iMessage. The question was asked several times, and did not get a straight answer. The replies even started to quote the Cameron/May rhetoric about "no safe place for terrorists to communicate".
This is am important issue - the government have made some clear contradictory statements that they feel there should be no way for people (including terrorists) to communicate such that the authorities cannot read what they are saying (with court order, oversight, etc). They also say encryption is important to the economy and safety and so on. These two things are contradictory.
The point about encryption is that you make it mathematically impossible for anyone else to read the message.
["impossible" meaning that with the resources of any adversary the message could not be revealed in the time frame during which its secrecy is important]
You cannot make a system where nobody else can read the message "unless they have a court order". Mathematics does not know about court orders - so anything you do to allow access with a court order weakens the system so that it is not longer safe. Whatever weakness or backdoor exists, it means criminals could exploit it. It is not proper encryption any more.
So far the government have not tried to ban encryption - to do so would make the UK a laughing stock, drive lots of business away from the UK, and make the UK a target for criminals making us all very much less safe.
You also have to remember that no matter what you do, what law you pass, how much you restrict or cripple software, people can always communicate completely secretly with nothing more than a pen, paper and dice. This means that the "safe place for terrorists to communicate" will always exist anyway.
iMessage
iMessage came up because it is a problem for the authorities - they cannot read it! Apple have chosen to make it secure. You can be sure, if you trust Apple, that messages you send cannot be "snooped" on by the authorities or Apple. There is some reason to trust Apple in that if they were lying, it would eventually come out if ever such communications were used in any court case or hacked, and that would massively damage their reputation - they have no reason to lie about this.
The problem is that the government don't like this - but can they ban a company offering end-to-end encrypted communications? Well, I think they are trying (and failing) :-
Thanks to Neil Brown for doing the leg work here :-
In RIPA, it is an obligation which can be imposed under a maintenance of capability order, under paragraph 10 of the Schedule to Regulation of Investigatory Powers (Maintenance of Interception Capability) Order 2002.
In the draft bill, it is part of the broad power to require a provider to maintain technical capability, in s189. It is called out by way of example in s189(4)(c), but this is just an example, and not a limitation.
The two positions can be contrasted: under the 2002 Order under RIPA, the list in the schedule sets out everything that the SoS can impose: it is exhaustive unless the Order is amended. Under the draft IP bill, there is no limited list of obligations, but rather anything the SoS considers to be reasonable, with some examples which are not limiting.
So, my understanding is that Apple could be served with an order to "maintain technical capability" and that includes the capability to "to remove any encryption applied". If so ordered, Apple would have to either change their system not to do encryption end-to-end, or at least have the means to do so when ordered to intercept communications. Also, they would not be able to tell us that they have done it.
The fact that someone could get such an order may have a chilling effect on the willingness to offer end-to-end encrypted services based in the UK.
This is really not acceptable, and needs addressing in the bill and RIPA, and removing. The reasons being that (a) anyone can communicate secretly anyway (as above) and (b) there are a couple of huge holes in the idea that you could force someone like Apple to change their systems :-
1. Apple is not in the UK. In fact the company that provides iMessage is, as far as I know, in Luxembourg, and so they can just tell the UK to sod off when they get an order. Also note that iMessage is free, so crippling the ability for UK "customers" to pay for the service does not help (Apple would have to separate itself from the company providing iMessage, but that is not hard).
2. Apple could simply separate iMessage in two - one is a service, perhaps a simple xmpp platform, that provides communications but does not in any way provide encryption. Then a separate company, which offers an app (software) that uses that service to provide end-to-end encryption. From the users point of view it would be a minor change to those Terms & Conditionings that you never read, and everything works the same - but legally it is different. The s/w company making the app is not a communications provider and so not subject to such orders. The communications company does not have means to decrypt the communications (mathematically impossible, remember).
Trying to make anyone that produces software include back-doors is back to the "batshit insane" category we started with, and fails to address open source software, or that the software can simply be provided by companies not based in the UK.
Ultimately, all you do is drive communications and software business out of the UK - you don't get encrypted communications decoded. Remove this requirement from the bill and RIPA!
This is am important issue - the government have made some clear contradictory statements that they feel there should be no way for people (including terrorists) to communicate such that the authorities cannot read what they are saying (with court order, oversight, etc). They also say encryption is important to the economy and safety and so on. These two things are contradictory.
The point about encryption is that you make it mathematically impossible for anyone else to read the message.
["impossible" meaning that with the resources of any adversary the message could not be revealed in the time frame during which its secrecy is important]
You cannot make a system where nobody else can read the message "unless they have a court order". Mathematics does not know about court orders - so anything you do to allow access with a court order weakens the system so that it is not longer safe. Whatever weakness or backdoor exists, it means criminals could exploit it. It is not proper encryption any more.
So far the government have not tried to ban encryption - to do so would make the UK a laughing stock, drive lots of business away from the UK, and make the UK a target for criminals making us all very much less safe.
You also have to remember that no matter what you do, what law you pass, how much you restrict or cripple software, people can always communicate completely secretly with nothing more than a pen, paper and dice. This means that the "safe place for terrorists to communicate" will always exist anyway.
iMessage
iMessage came up because it is a problem for the authorities - they cannot read it! Apple have chosen to make it secure. You can be sure, if you trust Apple, that messages you send cannot be "snooped" on by the authorities or Apple. There is some reason to trust Apple in that if they were lying, it would eventually come out if ever such communications were used in any court case or hacked, and that would massively damage their reputation - they have no reason to lie about this.
The problem is that the government don't like this - but can they ban a company offering end-to-end encrypted communications? Well, I think they are trying (and failing) :-
Thanks to Neil Brown for doing the leg work here :-
In RIPA, it is an obligation which can be imposed under a maintenance of capability order, under paragraph 10 of the Schedule to Regulation of Investigatory Powers (Maintenance of Interception Capability) Order 2002.
In the draft bill, it is part of the broad power to require a provider to maintain technical capability, in s189. It is called out by way of example in s189(4)(c), but this is just an example, and not a limitation.
The two positions can be contrasted: under the 2002 Order under RIPA, the list in the schedule sets out everything that the SoS can impose: it is exhaustive unless the Order is amended. Under the draft IP bill, there is no limited list of obligations, but rather anything the SoS considers to be reasonable, with some examples which are not limiting.
So, my understanding is that Apple could be served with an order to "maintain technical capability" and that includes the capability to "to remove any encryption applied". If so ordered, Apple would have to either change their system not to do encryption end-to-end, or at least have the means to do so when ordered to intercept communications. Also, they would not be able to tell us that they have done it.
The fact that someone could get such an order may have a chilling effect on the willingness to offer end-to-end encrypted services based in the UK.
This is really not acceptable, and needs addressing in the bill and RIPA, and removing. The reasons being that (a) anyone can communicate secretly anyway (as above) and (b) there are a couple of huge holes in the idea that you could force someone like Apple to change their systems :-
1. Apple is not in the UK. In fact the company that provides iMessage is, as far as I know, in Luxembourg, and so they can just tell the UK to sod off when they get an order. Also note that iMessage is free, so crippling the ability for UK "customers" to pay for the service does not help (Apple would have to separate itself from the company providing iMessage, but that is not hard).
2. Apple could simply separate iMessage in two - one is a service, perhaps a simple xmpp platform, that provides communications but does not in any way provide encryption. Then a separate company, which offers an app (software) that uses that service to provide end-to-end encryption. From the users point of view it would be a minor change to those Terms & Conditionings that you never read, and everything works the same - but legally it is different. The s/w company making the app is not a communications provider and so not subject to such orders. The communications company does not have means to decrypt the communications (mathematically impossible, remember).
Trying to make anyone that produces software include back-doors is back to the "batshit insane" category we started with, and fails to address open source software, or that the software can simply be provided by companies not based in the UK.
Ultimately, all you do is drive communications and software business out of the UK - you don't get encrypted communications decoded. Remove this requirement from the bill and RIPA!
Subscribe to:
Posts (Atom)
Fencing
Bit of fun... We usually put up some Christmas lights on the house - some fairy lights on the metal fencing at the front, but a pain as mean...
-
Broadband services are a wonderful innovation of our time, using multiple frequency bands (hence the name) to carry signals over wires (us...
-
For many years I used a small stand-alone air-conditioning unit in my study (the box room in the house) and I even had a hole in the wall fo...
-
It seems there is something of a standard test string for anti virus ( wikipedia has more on this). The idea is that systems that look fo...