I'd like to thank the patience of customers over this - as I said before, the main delay was the new fibre links between the London data centres we use, and we had planned to be at this stage late last year, before there were any capacity issues.
The good news is that the new switches are deployed (last Thursday). This took a bit longer than expected but was all done within the planned work window. Yesterday evening we were able to start using the new fibre links and relieve the congestion on our Talk Talk back-haul.
The next step is not quite what we originally expected. We were expecting to need more BT back-haul links, but the recent moves of many lines from BT to TT had meant we actually have enough capacity to both BT and TT for now.
Surprising the next bottleneck is the LNSs themselves. Only a few months ago we moved from 3 active LNSs to 4, and we have several more waiting to be plugged in. Sadly the original design linked the LNSs to BT back-haul, so we can't simply add more just yet - not without change.
The next step is therefore a change to the way we link to BT and TT back-haul, and that is planned for Sunday morning starting at 2am. This will mean that the number of BT links, and the number of TT links, and the number of LNSs will all become independent, and we can deploy new LNSs next week.
I hope that is one of the last major disruptive bits of work we have to do. Once we have that change, it will be possible to go back to our normal rolling over-night upgrades on LNSs when needed. It will also be possible to deploy new LNSs without major disruption.
After all of that we have to look carefully at our transit and peering - again, we have enough capacity for now, but it may need a bit of juggling to ensure no links are getting hot.
We can then take a bit of a step back and work out where we need more capacity next, and get it in place before needed.
The good news is that now the TT back-haul is not the issue, I am launching the Home::1 ADSL Terabyte service later today. Thank you for your patience.
2016-03-30
2016-03-26
Internet Connection Records, a small taste of the problems with #IPBill
However, there have been a small number of consequences which we have been working on. Obviously not show stoppers otherwise the planned work would have been reversed, but oddities.
One of them was that we were having difficulty getting SNMP from some of our LNSs, which meant some of our monitoring was unavailable. This had left us scratching our heads somewhat as the LNSs were not rebooted or reloaded or anything.
Then, another snag was that today one of our servers that does syslog started to run out of disk. Again, a puzzle. But this was easier to understand just by looking at the logs.
It turns out these are related. We have some debug logs from the LNSs related to setting up PPP sessions and allocation of IP addresses. These are kept for a couple of days to help resolve any connection problems.
One of the things logged is the IPv6 allocation, and this is logged by logging the DHCPv6 request/reply exchange from the customer router. Usually these either happen once after connection or maybe once an hour.
The problem, it seems, is rather odd. Some customers still use the Technicolor ADSL broadband routers that we used to sell from years ago. It seems many of these got upset in a rather odd way after the work on Thursday. We can see no logical reason for this, but they are now in a state where they are on-line and working, but generating approximately 1GB of uplink traffic a day, each, sending DHPCv6 requests! We were logging all of these. It seems the logging may actually have been so much load that it was impacting the SNMP responses.
The fix is rebooting the Technicolor routers, which, thankfully, we can do remotely.
But this gives me a slight insight in to the difficulty of collecting Internet Connection Records. Each of these DHCPv6 exchanges would be something that might well be logged as an ICR.
In practice, just trying to log this one type of packet we could not keep up - the log file was only 16GB (158 million entries) since 4am today. Looking at the traffic levels, that is a tiny fraction of the number of requests being sent by these routers. Our LNS logging system has built in limiting to try and avoid overloading things, and it was being pushed to the limits.
If we had to log every session (TCP/UDP/SCTP/IPSEC/ICMP, etc) there is just no way any of our existing kit could keep up. Of course it wasn't designed to! It was designed to shift packets quickly and provide Internet access to our customers, not snoop on anybody.
This also highlights the issue with any deliberate generation of ICRs by s/w on customer networks. It is easy with relatively low levels of traffic to cause a lot of ICRs to be created, if the #IPBill passes.
2016-03-25
CISCO and ARP?
The FireBrick has quite a good ARP handling subsystem, including exponential back-off, configurable ARP timeouts and so on. It has served us well, but we have recently encountered a slight problem talking to a CISCO Nexus switch.
So I did some tests - and would love to know if this is typical. Any CISCO experts reading this may be able to comment.
Testing using arping from linux, I could see that the CISCO would respond to only some of my ARP requests. Maybe one in five, but not very consistent. This is a tad odd, and may be down to some general ARP rate limiting perhaps.
On top of that, when it did respond, it did so after 2.99 seconds. This was very consistent - I had to use arping one ARP request at a time to confirm this.
I have to wonder what the hell it is doing! From a coding point of view, holding on to the ARP request or reply for that length of time is more work than just answering the ARP right away. I am at a loss as to what is going on.
For comparison, a FireBrick is timed by linux at 180us response and answered every ARP.
Anyway, it means I have had to tweak the way the ARP system renews ARPs to try a bit longer, otherwise every now and then the CISCO vanishes for a few seconds.
Oh, and yes, they still look like this with some arbitrary padding to min packet size for Ethernet.
P.S. It was CoPP, but we don't understand why it would delay ARPs 3s in that process.
So I did some tests - and would love to know if this is typical. Any CISCO experts reading this may be able to comment.
Testing using arping from linux, I could see that the CISCO would respond to only some of my ARP requests. Maybe one in five, but not very consistent. This is a tad odd, and may be down to some general ARP rate limiting perhaps.
On top of that, when it did respond, it did so after 2.99 seconds. This was very consistent - I had to use arping one ARP request at a time to confirm this.
I have to wonder what the hell it is doing! From a coding point of view, holding on to the ARP request or reply for that length of time is more work than just answering the ARP right away. I am at a loss as to what is going on.
For comparison, a FireBrick is timed by linux at 180us response and answered every ARP.
Anyway, it means I have had to tweak the way the ARP system renews ARPs to try a bit longer, otherwise every now and then the CISCO vanishes for a few seconds.
Oh, and yes, they still look like this with some arbitrary padding to min packet size for Ethernet.
09:40:20.688429 ARP, Request who-has 91.240.176.1 tell 91.240.176.254, length 46
0x0000: 0001 0800 0604 0001 0003 971d c009 5bf0 ..............[.
0x0010: b0fe 0000 0000 0000 5bf0 b001 474e 5520 ........[...GNU.
0x0020: 5465 7272 7950 7261 7463 6865 7474 TerryPratchett
P.S. It was CoPP, but we don't understand why it would delay ARPs 3s in that process.
2016-03-22
Public Bill Committee Written Evidence #IPBill
I am sorry that it has taken so long to get together a proper report on my findings when considering this Bill. It has been a lot of hard work, and I am very grateful to the assistance of many colleagues on this, including working through much of the Bill with me page by page on a Sunday!
There are, as before, a lot of issues.
My submission (PDF).
If you are thinking of making a submission, please do so ASAP. The first oral hearing is Thursday 24th and they have asked for evidence by Wednesday to allow time to consider it.
Please do feel free to quote me or copy to your MP. Ultimately they are the ones that vote on this.
I am happy to meet with MPs and Lords on this matter.
Oh, damn, once again, under this Bill your computer would now be logged as accessing a porn site, just because you read my blog. It would not log that you only accessed a benign favicon image, as that would be content, just that you accessed something on the site. Oops.
There are, as before, a lot of issues.
My submission (PDF).
If you are thinking of making a submission, please do so ASAP. The first oral hearing is Thursday 24th and they have asked for evidence by Wednesday to allow time to consider it.
Please do feel free to quote me or copy to your MP. Ultimately they are the ones that vote on this.
I am happy to meet with MPs and Lords on this matter.
Oh, damn, once again, under this Bill your computer would now be logged as accessing a porn site, just because you read my blog. It would not log that you only accessed a benign favicon image, as that would be content, just that you accessed something on the site. Oops.
2016-03-21
Signal #IPBill
Why? well, for one simple reason it allows both iPhones and Android to message each other using data and not SMS or MMS. It also allows calls via data.
But the real reason is privacy - what you send and receive or say using signal is private.
It is free and literally took a matter of seconds to install and start using. It ties in to your phone number and contacts and just works.
But wait a second! Encryption is difficult because of validating keys. People have "key signing parties" for things like PGP email. How can you tell the person you are talking to is the person you think they are?
Well, signal actually makes that easy too - you can easily, when you meet someone, point your phone at their screen and it reads a fingerprint of their key and checks it matches. If ever it changes later the phone will tell you that there is a problem.
They make privacy simple, and the Investigatory Powers Bill has nothing in it to allow snooping on your texts and calls in the network, when using Signal. It does not outlaw using Signal either (would be hard to without outlawing https for access to banks too). As worded now it could try to order this non UK company to put in a back door, and they are pretty guaranteed to tell them to sod off. The source code is available and inspectable so even if they were compelled to comply it would be obvious.
You can even secure the message archive in the phone independently to any encryption the phone offers.
So, download, install and use Signal - why not?
As used by members of the House of Lords to protect their privacy...
Replacing switches
The first step in upgrading our network is replacing some of the core switches with new, much faster and more powerful, switches.
Replacing switches is always fun!
For a start, they are in pairs to try and ensure continued operation of at least some of the network if one was to fail. Where possible devices are connected to both switches, and where we have pools of devices they are spread between the two. We actually have some new changes in the pipeline that will allow more of our equipment to actually use link aggregation over two switches for better redundancy even.
So, to move to a new switch, what do you do?
Well, first off, and surprisingly, you have to make space - you need the new switches basically next to the old ones in the rack. This may not be obvious, but if you are moving cables from one switch to another you need to make the move as short as possible. If not, then you have to re-route the cables or even get longer cables. So you have to shuffle stuff up/down to make space. Thankful that worked well. You also have to check cables are going to be able to move, and none are too short or snagged on anything.
Then, you make sure the new switch is the same config as the old. This is not simple as switch configuration is far from standard. There are VLANs and jumbo frames and all sorts to check very carefully. A lot of double checking is needed.
You also configure the old and new switch so that all of the VLANs can link between them. This means you can plug the new switches in to the old ones.
Then, on the day, you move one cable at a time. Ideally, shutting down operations of what you are moving cleanly to fall back to other devices, and then move the cable, check it, re-enable the functions, and check that. One by one very carefully. Done right you can move a lot of things with no impact on service at all - pairs of BGP servers can cleanly switch over, move, and switch back. Some things have disruption like LNSs which cause traffic to reconnect to other LNSs when shut down.
There can be (and were) problems! Basically the old switches had a head fit after moving many of the cables! This makes no sense, and meant power cycling the damn things. And, of course, moving cables back. It was not pretty.
We have tried this twice, and the second time we have Talk Talk suffer a major issue as well which complicated matters so even reverting the changes left us with all TT lines off line for a couple of hours.
So, this time, on Thursday, new approach, called "big bang". The same careful config, and checking, but not linking the old switch, just carefully but quickly moving every cable to the new switch and then spending time checking each one. It will cause more issues than the more usual step by step approach (when it works), but it is pretty predictable that it should actually work this time. However, there will be a clear time limit and move all the cables back if we cannot get everything working within that time, in the middle of the night.
Good luck to the ops team doing this work...
Replacing switches is always fun!
For a start, they are in pairs to try and ensure continued operation of at least some of the network if one was to fail. Where possible devices are connected to both switches, and where we have pools of devices they are spread between the two. We actually have some new changes in the pipeline that will allow more of our equipment to actually use link aggregation over two switches for better redundancy even.
So, to move to a new switch, what do you do?
Well, first off, and surprisingly, you have to make space - you need the new switches basically next to the old ones in the rack. This may not be obvious, but if you are moving cables from one switch to another you need to make the move as short as possible. If not, then you have to re-route the cables or even get longer cables. So you have to shuffle stuff up/down to make space. Thankful that worked well. You also have to check cables are going to be able to move, and none are too short or snagged on anything.
Then, you make sure the new switch is the same config as the old. This is not simple as switch configuration is far from standard. There are VLANs and jumbo frames and all sorts to check very carefully. A lot of double checking is needed.
You also configure the old and new switch so that all of the VLANs can link between them. This means you can plug the new switches in to the old ones.
Then, on the day, you move one cable at a time. Ideally, shutting down operations of what you are moving cleanly to fall back to other devices, and then move the cable, check it, re-enable the functions, and check that. One by one very carefully. Done right you can move a lot of things with no impact on service at all - pairs of BGP servers can cleanly switch over, move, and switch back. Some things have disruption like LNSs which cause traffic to reconnect to other LNSs when shut down.
There can be (and were) problems! Basically the old switches had a head fit after moving many of the cables! This makes no sense, and meant power cycling the damn things. And, of course, moving cables back. It was not pretty.
We have tried this twice, and the second time we have Talk Talk suffer a major issue as well which complicated matters so even reverting the changes left us with all TT lines off line for a couple of hours.
So, this time, on Thursday, new approach, called "big bang". The same careful config, and checking, but not linking the old switch, just carefully but quickly moving every cable to the new switch and then spending time checking each one. It will cause more issues than the more usual step by step approach (when it works), but it is pretty predictable that it should actually work this time. However, there will be a clear time limit and move all the cables back if we cannot get everything working within that time, in the middle of the night.
Good luck to the ops team doing this work...
2016-03-17
Call apparently from VERSO GROUP (UK) LIMITED - junk calls
Getting pissed off with junk calls today.
Two so far.
One was apparently from IDENTITY PROTECT LIMITED. Listen here.
The second, and much more amusing, was apparently from VERSO GROUP (UK) LIMITED. Listen here (posted with permission granted in the call recording itself). It seems that they were actually trying to sell me broadband. Wow!
I am not sure they were not the same person calling even, but that may be my being a tad racist... Listen and try and work it out yourself.
Update: Corrected first company name which was mistyped/linked.
Update: Clarified that second recording published with permission given within the call itself.
Update: For more details on why only "apparently" from Verso Group (UK) Ltd, see later blog post.
Two so far.
One was apparently from IDENTITY PROTECT LIMITED. Listen here.
The second, and much more amusing, was apparently from VERSO GROUP (UK) LIMITED. Listen here (posted with permission granted in the call recording itself). It seems that they were actually trying to sell me broadband. Wow!
I am not sure they were not the same person calling even, but that may be my being a tad racist... Listen and try and work it out yourself.
Update: Corrected first company name which was mistyped/linked.
Update: Clarified that second recording published with permission given within the call itself.
Update: For more details on why only "apparently" from Verso Group (UK) Ltd, see later blog post.
2016-03-16
#IPBill - Next Step the public bill committee
Well, I had my say on RT yesterday.
The next stage is for written (and maybe oral) submissions to the public bill committee. I think I need to do some work on this over the weekend and next week and get it in early.
And if you are writing to them or your MP, feel free to quote me when I say that we used to be proud of saying Made in UK on our products and now that could become a badge of concern.
The next stage is for written (and maybe oral) submissions to the public bill committee. I think I need to do some work on this over the weekend and next week and get it in early.
And if you are writing to them or your MP, feel free to quote me when I say that we used to be proud of saying Made in UK on our products and now that could become a badge of concern.
2016-03-13
Brexit?
This is a big topic and one we all get to decide on soon.For a start I am not publishing a personal stance on this. I do not feel qualified to do so, sorry. It does not mean I don't have one, but I don't have the evidence to advise anyone, and I am not sure anyone does.
Being in the EU carries a lot of shit. I mean both obligations and benefits. Some are very "big picture" things, like right to free trade and free movement of labour. Notably UK has one of the highest emigration rates to the rest of Europe of any country, apparently. And why not - we can. Even my son worked in Sweden for a year, because he could.
Being in EU carries problems, both big things and a shit load of small niggles, like the fucking cookie law on web sites which did not actually stop any tracking at all but made everyone have to click through annoying as hell "you agree to cookies" boxes on all sorts of web sites. It is one of the worse example of knee-jerk and bad legislation ever. There are many other small annoyances, and many other good points too.
Anyway, the problem is that there is no one single definitive test. There is no one metric one can look at and decide this is better than that. Obviously there are some big economic metrics one could look at, but even then it is basically impossible to predict what is better in or out on those. If we leave we will make some new political and trade relationships with many countries and even with "Europe" itself. These will have good and bad points.
But at the end of the day we cannot now say what will be better or worse, and even in ten years time when the dust has settled, either way, we cannot say if it is better or worse. It will always be a guess at what could have been or could be with no certainty and then a toss up of each metric impacts the "total" for good or bad. All we can say for sure is that leaving is "change".
Either way, some individuals, and companies, may be better or worse off. In some cases, some things can be predicted each way with some certainly, but mostly not. Even so, what is best for the country is not always best for me, or you.
The only thing you can say with any certainty is that leaving means change. Probably a decade of change.
Is change good?
This is, perhaps, the one question we can ask ourselves when deciding which way to vote. Do we want change or not?
We know that, sometimes, in hindsight, change can be good. There have been many changes to look back on. Wars can be very good examples of change, but would people have voted for war? But we also know that change can be bad.
Some people and some companies manage change way better than others, and that is where it matters to individuals. Can you manage change well? Can your employer manage change well?
Personally, I think I can. I think my company can. That does not mean I will vote not to leave the EU, as change is always "hard work". And this is one key point on the referendum, the age spectrum. As you get older you aim for an "easier life", and the means less change.
Personally I find myself at an odd time of life - I know how much I, and A&A, could exploit change by being agile and jumping on new opportunities. I also appreciate the reliability of no change and coasting on to the end of my life.
Who should be in charge?
I worry that some of the crap comes from Europe (back to cookie law), but I am increasingly worried about the crap that comes from our own Parliament. It is scary what they are planning in the move to Big Brother surveillance and police state with the IPBill - moves the EU can tackle and should tackle. So, again, I offer no answer as to who should be in charge! Well, maybe I should be, but sounds like a lot of hard work.
How to decide?
All I would say is that I would like everyone reading this to please be rational. Please research any concrete evidence and credible opinions you can find, as I will be doing. Please make an informed choice.
And do, please, think of yourself in this - what is best for you. Sometimes that may be what is best for your employer as that may be best for you, but do consider the overall impact of your choice.
And remember, being in or out of EU does not change whether we adhere to the European Convention of Human Rights or not, and if we adhere to similar UN conventions.
Micro controller stuff
When it comes to small stand alone micro controller projects I did quite a lot on the PIC16C84 which were really nice. Things like a device actually powered by a 9 pin RS232 port and on a board fitted between the pins of the plug so in the socket on the end of a lead, that does shit with bit-bashed serial. I made my own assembler for that too. They really were fun devices, and I recall when my kids were young making a pedestrian crossing for a Scalextric set with red/amber/green lights, button, red/green "cross" man, beeps, the lot. Indeed, for this project one of those would seem fine, but times have changed a bit...
The project
The project is simple - a device with a temperature sense and an infra red LED. The plan is to send the air-con unit commands to control the temperature properly. The damn air con is stupid. This time of year is especially a problem as my office is starting the day below 21℃ and later in the afternoon is above 21℃.
I set the air-con to heat to 21℃ and it does well, maybe as high as 21.7℃ but if the room starts to get hotter it stops.
I set the air-con to cool to 21℃ and it does well, maybe as low as 21.1℃ but if the room starts to get colder it stops.
OK, so I set "auto" mode for 21℃ and I even turn off the "eco" mode that makes for a wider margin. I can find it sat in heat mode but not heating or in cool mode and not cooling as low as 18℃ or as high as 24℃, or so it seems. The margins it uses are more than my personal comfort margins, and it is for my personal comfort that I had air-con installed!
I can achieve what I want only by changing manually between cool and heat modes when, during the day, the temperature of the room with no air-con starts to get above 21℃. I find it annoying that it takes me some time to realise it needs changing, suffering extra heat or cold for like half an hour and then looking at the wall thermometer and realising the problem.
But it having a tighter margin does not cause it to go mad - heating and cooling and heating and cooling, the point in the day it is hotter is pretty clear. If I allowed even a ±0.5℃ margin, e.g. 20℃ to 21℃ it would not spend time flapping between heat and cool.
So, I have found a nice temperature sensor, the ADT7420, which has a typical accuracy of 0.0017℃ (max 0.2℃). An IR LED is simple enough. So I could code something to control the air-con via what it thinks is the IR remote. All I then need to do is blu-tac up the sounder that beeps when you use the remote and I can have a stable temperature all year round.
The platform
This is what is tricky these days as so many choices.
At one extreme I could go old-school and make up a PIC16C84 or something. I doubt wire wrapped 65C102 is sensible these days :-) To be honest this is tempting - they were still "fun", like Z80 and 6502, but you could do shit with one leaded chip and a couple of other components on a bit of veroboard.
At the other end I could fit something to a PC. I have a linux PC in this room certainly but other rooms could do with this too.
In between, well, at the low end, things like Arduino or ARM Cortex and some assembler.
Or, at the higher end, maybe a Raspberry Pi, linux, and USB attached temperature sensors and IR emitter.
Choices, choices.
P.S. Worth mentioning that is it nice that things like the stupid remote control for the air-con is still itself a small micro-controller based device. You can tell as it does not take 5 minutes to boot up when you change batteries, and does not need s/w updates every week. That is good old fashioned "real" software... Old school.
P.P.S. I am almost disappointed - but someone makes such a gadget, so I am going to try it. If any good (or if not) I'll blog about it...
There can be only one!
Not sure if I am honoured or not.
Bracknell Forest Borough Council are now addressing me as The Adrian Kennard :-)
Bracknell Forest Borough Council are now addressing me as The Adrian Kennard :-)
2016-03-11
Phone lines and broadband, and OFCOM
Generally a broadband service requires the use of a phone line - this is down to the way the technology works, in that it is applied to a "normal copper pair" that is otherwise used for telephone service.It is possible for the broadband to be on a copper pair with no telephone service and even with no connection back to the exchange, but that is not really how it is done. This is partly down to the pricing model used - where someone pays for the copper pair as a telephone service and then someone pays to "share" that pair to provide broadband.
OK, yes, sounds technical, and that is part of the problem.
One of the other issues is that people are not using a phone line as a phone line any more... This is a change since broadband first started. For personal use a mobile is often both cheaper and more convenient. For anything business related VoIP is usually far cheaper and way more flexible. So who uses a phone line to make/receive phone calls any more?!
As an ISP we recognise this and over 10% of our customers take a "copper pair for broadband use" along with their broadband as a single package. Other ISPs offer similar. Some provide a phone line and broadband together as a package. Some insist you have their phone line for their broadband and juggle the prices of the two parts to make one look very cheap (a practice that OFCOM and ASA are keen to stop).
There are moves afoot in BT to start offering single broadband services. They already do some limited FTTP (Fibre to the premises), but they should start offering something they call "Single Order GEA", which is a copper pair with VDSL to the cabinet as a single BT service we can buy. But that is all in the future and will have its own problems :-)
So for now, it is straight forward. A combined package has lots of advantages such as a single bill and a single point of contact for faults. One of the problems we have with customers from time to time is that they do not understand the way the two separate services work. If they have a phone line fault, they have to get the phone line provider to fix it even if it is what is breaking their broadband. This is not us being difficult, it is the way it works and we have no real choice in the matter, sadly. This is partly why we offer the package of both, so we can handle either type of fault in one place.
Until recently, if someone wanted to move ISPs, the losing ISP could talk to the customer. The customer had to ask for a migration code, and many ISPs would try to suggest other packages, discounts, incentives, etc. ISPs had whole "retentions departments". It is the way it is in many industries. It also meant that it was easy to ensure the customer understood the whole phone line and broadband being separate services, and would be migrating both.
However, OFCOM changed things, and made the process gaining provider led. You contact the new ISP, and they claim the line/service (with 14+ days notice). The losing ISP has very limited option to contact their customer, simply advising the migration is happening and the date and details of any related services that are impacted.
This sounds sensible, but we are starting to see a problem.
As customers do not always appreciate the link between phone line and broadband, and may even see it as a "single service" (after all, they don't use a phone line any more), they may simply migrate the broadband to a new ISP. The losing ISP does have to tell them if a related service, like the phone line, is affected (e.g. ceased), but that is it.
We have seen this both ways - someone moves to us, and we provide broadband, and then a few days later the phone line part is ceased and that means the broadband is ceased. This can happen with basically no notice.
This all ends up costing us cease charges (yes, we have to pay when BT cease a service on us - that is another gripe) and minimum term charges, and costing us to reinstall the service and provide a phone line, and so on. It is not nice.
We have also seen it the other way, where someone is leaving us and we make very clear in the notice we send them that the phone line part will cease and that will kill their broadband! But we are only really allowed to do what OFCOM say, which is a notice of migrate and related services that are affected.
Sadly, some times, customers ignore this, and then get upset that their new broadband has been ceased some days after they moved away from us. Very frustrating consequence of the new OFCOM process. Having had this happen recently we are making the wording in our notices even clearer but not a lot more we can do really.
Once again, well done OFCOM - helping the consumer with an ill thought out process.
2016-03-08
This is special
I know the inventor of email died recently, shame.
But still, that is no excuse for BT being special. I mean really special.
We have a fault open on a line with them, and escalated. So they are sending us an update on the escalation. Simple enough.
Now, how to send that - well why not email?
Let's make up an email shall we :- (some xxx's added by me)
From: xxxxx,A,Abhishek,K3N73P C
Sent: 07 March 2016 09:12
To: `suxxort@aa.net.uk`
Cc: xxxxx,R,Rajesh,K3N74H C; xxxxx,HK,Harish,K3N73E C; S,J,xxxxx,K3N73F C
Subject: RE: [5816XC] ESC ID-1014504//BBEUxxxxxx//DN-xxxxxx
Hi All,
Please confirm the status of the broadband connection.
Thanks and Regards
Abhishek xxxxx
Broadband Customer Service Team Manager
OK, a bit odd. The from address is not an email address and neither are Cc addresses, and our support address in back quotes. Strange.
But still - an email - to be sent by email, yes?
No! They find an old order for the line, dating back to last year, one that is closed off and completed long ago. They add an Ad hoc note to CP to the order, and send us an update via the B2B signed SOAP XML over https process.
I mean, look :-
Yes, that is some sort of email embedded in a long closed order as an update on the Completed status of that order. One wonders why they did not put a note on the fault report - oh, I know - the fault report was auto-closed because their system does that when you escalate a fault most of the time.
Confused our system - it thought the order had just closed, again!
Please BT, do not be quite that special - we can handle email, honest.
But still, that is no excuse for BT being special. I mean really special.
We have a fault open on a line with them, and escalated. So they are sending us an update on the escalation. Simple enough.
Now, how to send that - well why not email?
Let's make up an email shall we :- (some xxx's added by me)
From: xxxxx,A,Abhishek,K3N73P C
Sent: 07 March 2016 09:12
To: `suxxort@aa.net.uk`
Cc: xxxxx,R,Rajesh,K3N74H C; xxxxx,HK,Harish,K3N73E C; S,J,xxxxx,K3N73F C
Subject: RE: [5816XC] ESC ID-1014504//BBEUxxxxxx//DN-xxxxxx
Hi All,
Please confirm the status of the broadband connection.
Thanks and Regards
Abhishek xxxxx
Broadband Customer Service Team Manager
OK, a bit odd. The from address is not an email address and neither are Cc addresses, and our support address in back quotes. Strange.
But still - an email - to be sent by email, yes?
No! They find an old order for the line, dating back to last year, one that is closed off and completed long ago. They add an Ad hoc note to CP to the order, and send us an update via the B2B signed SOAP XML over https process.
I mean, look :-
<utcc:OrderLineNote>
<utcc:Note>From: xxxxx,A,Abhishek,K3N73P C
Sent: 07 March 2016 09:12
To: `suxxort@aa.net.uk`
Cc: xxxxx,R,Rajesh,K3N74H C; xxxxx,HK,Harish,K3N73E C; S,J,xxxxx,K3N73F C
Subject: RE: [5816XC] ESC ID-1014504//BBEUxxxxx//DN-xxxxx
Hi All,
Please confirm the status of the broadband connection.
Thanks and Regards
Abhishek xxxxx
Broadband Customer Service Team Manager</utcc:Note>
<utcc:NoteType>Customer Update</utcc:NoteType>
</utcc:OrderLineNote>
Confused our system - it thought the order had just closed, again!
Please BT, do not be quite that special - we can handle email, honest.
Calling from OFGEN? Really?
Got an anonymous call claiming to be from The Energy & Climate Department, or is it OFGEN...
I suspect not. I suspect a scam call.
How do these people sleep at night? I am sure ICO will not bother to do anything.
Have a listen: MP3
P.S. In case you are worried, we are getting the insulation all changed, but probably to some nice Celotex and boarding on top, like I have in the man-cave.
I suspect not. I suspect a scam call.
How do these people sleep at night? I am sure ICO will not bother to do anything.
Have a listen: MP3
P.S. In case you are worried, we are getting the insulation all changed, but probably to some nice Celotex and boarding on top, like I have in the man-cave.
BT refusing to fix faults
This happens every few years - BT keep juggling their definition of fault repair in various ways, trying to somehow make it a chargeable service even though it is clearly part of the broadband services we buy that they investigate and fix faults.
The latest fiasco, as I have blogged before, is BT plc t/a BT Wholesale have stated that SFI is an optional service provided by BT plc t/a Openreach which they make available to us. They have gone on to say that an SFI engineers sole job (though he may choose to do more) is to test the line to SIN349, and charge us if the line meets SIN349. He does have the job to fix the line if it does not meet SIN 349 though.
SIN 349 is the technical spec for the copper pair for telephone use - it is not even a spec for broadband, so such a service makes no sense as something we would every want to buy. Also, we can test a line to SIN 349 for free from the exchange end tests and it would be unusual for these to disagree with a test on site by an engineer.
The problem is that BT have no process for actually fixing broadband faults - it is either a phone line fault and fixed by the line provider part of BT, or it is a broadband fault at which point they offer the option of a pointless SFI service that will only test/fix a phone line fault rather than any means to fix a broadband fault.
So we have the argument over and over again.
What would be especially amusing this time, if it was not causing delay and inconvenience for us and our customer, is that BT are insisting we book an SFI engineer for a PPP fault on an FTTP service.
FTTP is fibre to the premises. I don't mean like "Virgin fibre optic cable", I mean actual real fibre optic cable that actually goes to the house. Not a copper pair. So insisting we send someone to test the "copper pair" to SIN349 is just farcical.
I do despair at BT some times.
P.S. 24 hours of nothing happening, and finally some progress.
P.P.S. They finally got an engineer out, changed the ONT and upgraded its software and then cleared the fault as no BT fault - I bet we get charged! Good news is customer is on line.
GCHQ boss: Tech firms should co-operate over encryption
Unfortunately there is a conflict of interest here - what the tech firms wish to do is keep user's data safe - they should do this - it is even in the Data Protection Act that personal data is important and should be kept safe.
So the objective of the tech firm is at odds with the objective of GCHQ which is to access user's data when they want to.
The gold standard for the tech firm is to make the data so safe that even they cannot access it. Even someone that knows exactly how it all works, that wrote the code that is used, cannot, by any means, access the data. Apple are pretty close and I am sure are working on ensuring this is the case.
If a tech firm is successful in this goal then there is not really a lot to discuss with GCHQ, is there? They cannot have the data, end of story. If there was something to discuss, some way that the data could be accessed by any means, then that is a loophole the tech company should be working on plugging!
One statement "The solution is not, of course, that encryption should be weakened, let alone banned. But neither is it true that nothing can be done without weakening encryption," shows the problem.
Let's be clear - this is not about the mathematics - this is a very simple high level thing. Anything that allows a third party (such as GCHQ) access to data is weakening the encryption. It does not matter if that is some procedural change, some storage of keys in a "safe place", some trick in the mathematics to allow a third key - none of that matters - the very possibility of access is a "weakening of encryption" by definition.
I am shocked that they seem not to understand this. Well, I am sure they do, but want to gloss over it.
Of course, the real "back door" to any system is the software update. It is essential to have this, not just for new features in a product, but also to fix vulnerabilities. Software is never 100% perfect, and even if it was the world changes and what is necessary to defend against attacks changes. So s/w updates are needed and should be encouraged. They should be digitally signed to ensure the s/w is genuine, of course. The issue is that new software can help access data - whether by allowing lots of attempts very quickly (what the FBI want) or by capturing keys next time the user legitimately unlocks the data.
There are steps a tech firm can take, and I expect Apple are working on this, such as ensuring there is no way to update the software on a locked phone. Even make the security hardware not allow an update without correct use of the PIN or password (and not allow many attempts). This addresses the issue of access to a device after it has been seized, but not the possibility of a systemic vulnerability being introduced on devices in advance - that needs trust in the suppler.
Of course if you do not trust your supplier or the government, you can do encryption yourself, and none of this will then apply. I should not have to keep saying this but criminals can always use encryption, and even do so covertly. Such laws or discussions only impact the non criminals!
Sadly the UK wants to remove all trust in any UK firm by allowing secret orders that could do exactly that - compromise security on all devices in advance. It will be a sad state of affairs very soon when we have to trust a foreign supplier as we cannot trust anyone in our own country.
"Made in UK" will become the hallmark of distrust by the end of the year!
P.S. The original talk was actually more balanced, but still misses the key points in many ways and thinks there can be a way for law and encryption not to clash, and somehow that criminals would obey any such laws anyway.
His comment "On encryption, it simply repeats the position of earlier legislation: where access to data is legally warranted, companies should provide data in clear where it is practicable or technically feasible to do so. No-one in the UK Government is advocating the banning or weakening of encryption." clearly lacks an understanding of the power of the bill going through parliament, that can secretly demand much much more.
2016-03-07
Child Safety Online
The government has launched a consultation, and anyone can reply, so read it and express your view even if you do not agree with my view.
So where do I stand?
I see porn like any other fiction entertainment, and like any other fiction entertainment there are themes that are clearly unsuitable for young children. We already avoid exposing very young children to extreme violence or themes they are ill equipped to understand.
I fully support helping parents be parents and managing what their children do and access. As an ISP we have many ways to help with that.
As people get older they can handle such fiction and recognise it as the fiction it is and an escape from reality that we all enjoy. Watching porn is not really any different from watching any other fantasy fiction entertainment.
The problem with society is that unlike most other things - like violence or science fiction - we cannot easily see what is the normal case for things like sex and relationships because of the massive social taboo that surrounds the topic. This is the problem.
People can see that it is not socially acceptable to go all Die Hard and shoot everyone, or even to beam up to a space ship. They cannot easily see it is not right to abuse a woman in private because the private relationships are hidden away. We need more education to explain what is good and bad in such relationships that teenagers can understand. Once we do that, they can understand porn as fiction as much as Die Hard for Star trek. [I am waiting for someone to tell me "Die Hard" is actually a very dodgy porn movie].
To be honest, we already expose children to some seriously screwed up influences from religion with no age verification at all - judgemental sky fairies, talking snakes, rules on keeping slaves, boats that can carry every species after a genocide, stoning, and revering a roman torture and execution device as a jewellery! Some of the shit kids are exposed to is just not right and really should be reserved for when they are 18 and able make their own choice. If your religion only works if you get hold of them young you are pretty insecure, in my opinion.
Thankfully the report seems to cut short of forcing ISPs to filter things - that would be bad for lots of reasons. ISPs are specifically not liable for what they carry for the very reason that the Internet would not exists if they were. We enjoy the benefits of the Internet (and the downsides) because of that mere conduit protection. Take it away and it all falls apart. ISPs could not actually filter any content 100%, and even if 1% then 100% of people can search for the way to use that 1% loophole. It is futile. If ISPs were liable the insurance costs for that passed on to customers would make the Internet unviable.
So let's not try and bottle porn up and censor it - let's make education work and ensure children can cope with what is out there, like the rest of life. The porn industry should be, and is, regulated in most countries to ensure people are paid and not exploited. People may enjoy the fiction entertainment and still be normal in real life, whether watching porn or the X-files.
So, that is my view... comments?
This is basically my reply, which I will be submitting.
Question 1: In your opinion, should age verification controls be placed on all forms of legal pornography (‘sex works’) online that would receive a British Board of Film Classification rating of 18 or R18?
My issue here is that a lot of porn sites are well outside UK jurisdiction and so placing such controls is not going to be effective in any way. I suspect most sites charging for porn will be happy with this as the fact they charge means they have an effective age verification by the fact they want a credit card. So the sites you can make comply already do, and the sites that do not will ignore UK law, so why the hell are we discussing this?
Question 2: Do you think age verification controls should be placed on sites containing still as well as moving images of pornography?
I don't see much difference - porn comes in all sorts - stills and videos.
Question 3: To what extent do you agree with the introduction of a new law to require age verification for online pornographic content available in the UK?
Again, this is not about the UK - most sites are not UK - I have no problem with UK hosted sites having age verification, apart from the commercial disadvantage they will face, but that cannot have realistic impact on non UK sites.
Question 4: If age verification controls are to be required on pornographic websites, how do you think they should work (select all that apply, and please suggest other ideas that you may have).
I do not think there is actually any way to do this - whatever you do a teenager can mimic what an adult did or does, even borrowing their credit card. Nothing will work against an adolescent boy that wants to access porn, sorry. And if they VPN or Tor to an non UK IP, the verification will vanish as UK specific.
Question 5: Do you agree that a regulator should have the power to direct payment and other ancillary services to remove their services from non- compliant websites? Please give reasons.
You could, but that simple means kids will access the thousands of free (paid by adverts) sites instead and not actually help matters at all.
Question 6: Do you have any suggestions for other actions that could be taken to ensure that commercial providers of online pornography comply with the new law? Please give details.
No - everyone outside the UK is not subject to UK law, sorry.
Question 7: Do you think that the regulator should have the power to direct parent and umbrella companies of pornographic websites to comply
No - as such company structures can be re-engineered at a whim and any law that worked would immediately be worked around. That is assuming any of the parties are subject to UK law.
Question 8: Do you agree with the introduction of a civil regime to regulate pornography websites? Please explain your answer.
No - would only work on UK providers - so actually putting UK at a commercial disadvantage and not actually addressing the perceived problem at all.
Question 9: Would the introduction of a new criminal offence be a better form of regulation?
No - would only work on UK providers - so actually putting UK at a commercial disadvantage and not actually addressing the perceived problem at all.
Question 10: To what extent do you agree with the introduction of a new regulatory framework?
Disagree - see top of this blog post. Not the way to solve the problem, if there is one.
Question 11: Should a new framework give powers to a regulator/ regulators to (select all that apply):
Powers only work in UK, so no.
Question 12: Do you think that a co-regulatory approach involving more than one regulator would be appropriate in this context?
Can't see how that helps.
Question 13: Do you agree that the regulator’s approach should focus on having the greatest proportional impact, for instance by looking at the most popular sites, or those most visited by children in the UK?
Again, such sites will be outside UK - so outside jurisdiction.
Question 14: Wherever new regulation is proposed, the Government must consider impacts on smaller and micro-sized businesses (those with fewer than 50 employees) based in the UK, and whether these impacts are proportionate. Should smaller and micro-sized businesses (such as some payments and ancillary services) be exempt from the scope of the policy?
Puzzled by this - why would size of operation change anything? If harm is done, the need exists, if not, then it does not. How is size of company even a consideration?
Question 15: Overall, are you broadly in favour of the proposals set out in the consultation?
No - see top of blog post.
Question 16: How effective do you think the Government’s preferred approach would be in preventing children from accessing online pornography?
Zero - actually negative - there is possible impact on payment providers and advertisers and UK porn industry that would have to comply when competing overseas providers would not have to. The end result being no help to kids in the UK but harm to some UK industries.
So where do I stand?
I see porn like any other fiction entertainment, and like any other fiction entertainment there are themes that are clearly unsuitable for young children. We already avoid exposing very young children to extreme violence or themes they are ill equipped to understand.
I fully support helping parents be parents and managing what their children do and access. As an ISP we have many ways to help with that.
As people get older they can handle such fiction and recognise it as the fiction it is and an escape from reality that we all enjoy. Watching porn is not really any different from watching any other fantasy fiction entertainment.
The problem with society is that unlike most other things - like violence or science fiction - we cannot easily see what is the normal case for things like sex and relationships because of the massive social taboo that surrounds the topic. This is the problem.
People can see that it is not socially acceptable to go all Die Hard and shoot everyone, or even to beam up to a space ship. They cannot easily see it is not right to abuse a woman in private because the private relationships are hidden away. We need more education to explain what is good and bad in such relationships that teenagers can understand. Once we do that, they can understand porn as fiction as much as Die Hard for Star trek. [I am waiting for someone to tell me "Die Hard" is actually a very dodgy porn movie].
To be honest, we already expose children to some seriously screwed up influences from religion with no age verification at all - judgemental sky fairies, talking snakes, rules on keeping slaves, boats that can carry every species after a genocide, stoning, and revering a roman torture and execution device as a jewellery! Some of the shit kids are exposed to is just not right and really should be reserved for when they are 18 and able make their own choice. If your religion only works if you get hold of them young you are pretty insecure, in my opinion.
Thankfully the report seems to cut short of forcing ISPs to filter things - that would be bad for lots of reasons. ISPs are specifically not liable for what they carry for the very reason that the Internet would not exists if they were. We enjoy the benefits of the Internet (and the downsides) because of that mere conduit protection. Take it away and it all falls apart. ISPs could not actually filter any content 100%, and even if 1% then 100% of people can search for the way to use that 1% loophole. It is futile. If ISPs were liable the insurance costs for that passed on to customers would make the Internet unviable.
So let's not try and bottle porn up and censor it - let's make education work and ensure children can cope with what is out there, like the rest of life. The porn industry should be, and is, regulated in most countries to ensure people are paid and not exploited. People may enjoy the fiction entertainment and still be normal in real life, whether watching porn or the X-files.
So, that is my view... comments?
This is basically my reply, which I will be submitting.
Question 1: In your opinion, should age verification controls be placed on all forms of legal pornography (‘sex works’) online that would receive a British Board of Film Classification rating of 18 or R18?
My issue here is that a lot of porn sites are well outside UK jurisdiction and so placing such controls is not going to be effective in any way. I suspect most sites charging for porn will be happy with this as the fact they charge means they have an effective age verification by the fact they want a credit card. So the sites you can make comply already do, and the sites that do not will ignore UK law, so why the hell are we discussing this?
Question 2: Do you think age verification controls should be placed on sites containing still as well as moving images of pornography?
I don't see much difference - porn comes in all sorts - stills and videos.
Question 3: To what extent do you agree with the introduction of a new law to require age verification for online pornographic content available in the UK?
Again, this is not about the UK - most sites are not UK - I have no problem with UK hosted sites having age verification, apart from the commercial disadvantage they will face, but that cannot have realistic impact on non UK sites.
Question 4: If age verification controls are to be required on pornographic websites, how do you think they should work (select all that apply, and please suggest other ideas that you may have).
I do not think there is actually any way to do this - whatever you do a teenager can mimic what an adult did or does, even borrowing their credit card. Nothing will work against an adolescent boy that wants to access porn, sorry. And if they VPN or Tor to an non UK IP, the verification will vanish as UK specific.
Question 5: Do you agree that a regulator should have the power to direct payment and other ancillary services to remove their services from non- compliant websites? Please give reasons.
You could, but that simple means kids will access the thousands of free (paid by adverts) sites instead and not actually help matters at all.
Question 6: Do you have any suggestions for other actions that could be taken to ensure that commercial providers of online pornography comply with the new law? Please give details.
Question 7: Do you think that the regulator should have the power to direct parent and umbrella companies of pornographic websites to comply
No - as such company structures can be re-engineered at a whim and any law that worked would immediately be worked around. That is assuming any of the parties are subject to UK law.
Question 8: Do you agree with the introduction of a civil regime to regulate pornography websites? Please explain your answer.
No - would only work on UK providers - so actually putting UK at a commercial disadvantage and not actually addressing the perceived problem at all.
Question 9: Would the introduction of a new criminal offence be a better form of regulation?
No - would only work on UK providers - so actually putting UK at a commercial disadvantage and not actually addressing the perceived problem at all.
Question 10: To what extent do you agree with the introduction of a new regulatory framework?
Disagree - see top of this blog post. Not the way to solve the problem, if there is one.
Question 11: Should a new framework give powers to a regulator/ regulators to (select all that apply):
Powers only work in UK, so no.
Question 12: Do you think that a co-regulatory approach involving more than one regulator would be appropriate in this context?
Can't see how that helps.
Question 13: Do you agree that the regulator’s approach should focus on having the greatest proportional impact, for instance by looking at the most popular sites, or those most visited by children in the UK?
Again, such sites will be outside UK - so outside jurisdiction.
Question 14: Wherever new regulation is proposed, the Government must consider impacts on smaller and micro-sized businesses (those with fewer than 50 employees) based in the UK, and whether these impacts are proportionate. Should smaller and micro-sized businesses (such as some payments and ancillary services) be exempt from the scope of the policy?
Puzzled by this - why would size of operation change anything? If harm is done, the need exists, if not, then it does not. How is size of company even a consideration?
Question 15: Overall, are you broadly in favour of the proposals set out in the consultation?
No - see top of blog post.
Question 16: How effective do you think the Government’s preferred approach would be in preventing children from accessing online pornography?
Zero - actually negative - there is possible impact on payment providers and advertisers and UK porn industry that would have to comply when competing overseas providers would not have to. The end result being no help to kids in the UK but harm to some UK industries.
Growing pains
When we first started we had a few months when we actually stopped taking on new customers - it was all down to a silly long lead time and some other issues with a new BT pipe.
Back in those days there were 2Mb/s, 8Mb/s, and 34Mb/s pipes, and we were going to a 34Mb/s pipe at the time. We then moved to 155Mb/s and then two of them. That took a whole rack of BT kit!
Finally we moved to Gb/s host links, and we currently have many Gb/s fibres to both BT and TalkTalk. Surprisingly these are getting full quicker than we expected.
Well, not quite. We expected that by start of 2016 we would need more capacity in our core network to link all of this together and we started planning early last year. We have even taken on a highly experienced network architect to help us manage this all.
We have been taking things slowly, one step at a time, and cautious not to break things, but then things did not quite go to plan. A fibre install has taken a lot longer than expected, and then turned out to have some stupid jointing issues which have taken some time to fix. We have had to invest in some rather expensive but very fast switches with lots of 10Gb/s ports, to make our new network in two London data centres with 80Gb/s cross connect.
The problem is that we are not quite there yet - we are about a week away before we can start to actually use the new network and links, and right now we have higher than expected traffic causing some congestion in the evening for some of our TalkTalk customers and even some of the Ethernet customers.
This is embarrassing. I know lots of ISPs have congestion, and for some it is simply part of the business model, but we pride ourselves in not slowing down in the evenings and not being a bottleneck, and we have failed in that for about a week now in the evenings on some lines.
So it is a trade off to get stuff moving quickly, and to make 100% sure we are doing it right. I think we have the right focus now, and assuming some optics arrive this week we should be able to have the new cross connect links between the data centres under test this weekend. There is some planned work which will cause some disruption early one morning at the weekend, but once that is done we can start fixing these congestion issues once and for all.
In the mean time I have delayed the launch of the Home::1 ADSL Terabyte service - a package which I hope it will mean lots more customers. Right now I would rather our existing customers have the best possible service than just piling more people on. We also have done some juggling of links to try and alleviate the problem for tonight, and expect to be able to move more traffic to another router in the next couple of days as well. If all goes well, we will be OK this week and can get started on the long term solutions next week.
This latest expansion should last a few years, but even so we expect that we will have to add more LNSs and more routers as time goes by.
Fortunately we don't have Theresa May poking her nose in as it would cause a lot of problems trying to manage this network expansion and manage snooping for the spooks at the same time. We can only hope that we never have that problem!
Thank you all for your patience.
P.S. New switches and links going in on Sunday morning (13th)
Back in those days there were 2Mb/s, 8Mb/s, and 34Mb/s pipes, and we were going to a 34Mb/s pipe at the time. We then moved to 155Mb/s and then two of them. That took a whole rack of BT kit!
Finally we moved to Gb/s host links, and we currently have many Gb/s fibres to both BT and TalkTalk. Surprisingly these are getting full quicker than we expected.
Well, not quite. We expected that by start of 2016 we would need more capacity in our core network to link all of this together and we started planning early last year. We have even taken on a highly experienced network architect to help us manage this all.
We have been taking things slowly, one step at a time, and cautious not to break things, but then things did not quite go to plan. A fibre install has taken a lot longer than expected, and then turned out to have some stupid jointing issues which have taken some time to fix. We have had to invest in some rather expensive but very fast switches with lots of 10Gb/s ports, to make our new network in two London data centres with 80Gb/s cross connect.
The problem is that we are not quite there yet - we are about a week away before we can start to actually use the new network and links, and right now we have higher than expected traffic causing some congestion in the evening for some of our TalkTalk customers and even some of the Ethernet customers.
This is embarrassing. I know lots of ISPs have congestion, and for some it is simply part of the business model, but we pride ourselves in not slowing down in the evenings and not being a bottleneck, and we have failed in that for about a week now in the evenings on some lines.
So it is a trade off to get stuff moving quickly, and to make 100% sure we are doing it right. I think we have the right focus now, and assuming some optics arrive this week we should be able to have the new cross connect links between the data centres under test this weekend. There is some planned work which will cause some disruption early one morning at the weekend, but once that is done we can start fixing these congestion issues once and for all.
In the mean time I have delayed the launch of the Home::1 ADSL Terabyte service - a package which I hope it will mean lots more customers. Right now I would rather our existing customers have the best possible service than just piling more people on. We also have done some juggling of links to try and alleviate the problem for tonight, and expect to be able to move more traffic to another router in the next couple of days as well. If all goes well, we will be OK this week and can get started on the long term solutions next week.
This latest expansion should last a few years, but even so we expect that we will have to add more LNSs and more routers as time goes by.
Fortunately we don't have Theresa May poking her nose in as it would cause a lot of problems trying to manage this network expansion and manage snooping for the spooks at the same time. We can only hope that we never have that problem!
Thank you all for your patience.
P.S. New switches and links going in on Sunday morning (13th)
2016-03-05
INSERT INTO table SET field=value,...
Doing a few searches I guess this is a mysql specific variation. I thought for a moment I was being stupid.
One of the pitfalls of learning any new programming language or system is that your knowledge can be incomplete and you do not realise it. I have had this on occasion with various systems, usually because of ongoing improvements to the system itself (like linux libraries) and simply not knowing some new and simpler way to do things now exists. This is one of the reasons code gets old - looking at code I wrote a year ago I cringe as I have learned new ways to do things over time.
Just occasionally my original learning of the system meant I missed some detail, because it was not needed in the first few cases of using the system, and then over time I never go back and re-read the text books (web pages) to realise I have missed something.
This syntax for SQL INSERT is an example I have only just realised exists and I was beginning to think I was going mad, and cursing why I did not know about it. Another example was when I discovered the "ON DUPLICATE" feature of INSERT statements.
I did a few searches, and one rather odd comment was "What a confusing syntax! If I didn't know how to use web search to lookup the documentation, I'd have thought someone mixed up UPDATE and INSERT!", so I thought I would explain good reasons why this syntax is so much better for some uses.
The main example is when using this in a program. (For a start, I have formatting functions that know how to escape values for SQL to avoid injection attacks, obviously).
1. The fact that INSERT and UPDATE were different syntax meant you have to have two separate bits of code for creating and entry and for updating the entry. This was annoying as often you want almost all of the same fields. Now you can make code that creates the query for either in one go :-
(a) INSERT INTO or UPDATE depending if existing or new
(b) table name
(c) SET
(d) lots of field=value pairs (and comma)
(e) either a comma or WHERE depending if new or existing
(f) keyfield=value;
Nice and simple.
2. The other issue with the INSERT INTO table (fields...) VALUES (values...) syntax is maintenance of code, and even worse if some fields are not always needed... You end up with something like
(a) INSERT INTO table (
(b) List of field names
(c) ) VALUES (
(d) List of format controls like %s, %d, etc
(e) );
(f) List of variables to be printed in the formatting where the %s, %d, etc are located
Points (b), (d), and (f) have to all match up exactly, so adding a new field you have to count the position in the list and add field name, and in list of format controls, and in list of variables. If one of the field does not always want to be included you have to have a conditional entry in all three lists. It is tedious in code and messy and easy to get wrong.
Using the SET syntax you have
(a) INSERT INTO table SET
(b) Add each entry separately using field=, format control, variable
This means that if any field is optional, you have one condition test around that field, not three. It means if a new field is needed, you add one formatted print command. If a field is to be removed, you remove one formatted print command.
If you don't have libraries for formatting SQL as I do (uses %#s as a quoted escaped string, for example), you can have an add_string(field,value) function that adds ,field='escapedvalue' to the query in one go.
It just makes for easier to understand queries and neater code.
Now I have to resist the urge to grep all my code and change the syntax of existing usage - as that will definitely introduce new bugs. I just have to use this on new code and when maintaining old code, one step at a time...
2016-03-01
Investigatory Powers Bill published
http://www.publications.parliament.uk/pa/bills/cbill/2015-2016/0143/cbill_2015-20160143_en_1.htm
Summary
Way too much still in terms of serious powers for the state to spy on its own people. Way too much in the way of bulk powers. Way too much in data retention - snooping on pretty much any data.
Comments to follow...
Retention
78(9) is still as wide scoped as before, but says "and this expression therefore includes, in particular, internet connection records." which does not help matters at all.
84(2) I see retention notices are still secret but can be disclosed with permission from secretary of state, which is some progress I guess.
225(1) slight improvement: “data” includes data which is not electronic data and any information (whether or not electronic),
Encryption
217(4) Still has removal of protection (encryption) though now "applied by or on behalf of that operator to any communications or data" which is an improvement, and 217(3) and 218(4) relate to orders being "practicable" which is again an improvement. However, reading this, it could require an operator like Apple to change iMessage so that it is possible for them to remove encryption, or any hardware or software supplier in the UK or overseas. This leaves criminals able to encrypt but normal people that do not want that hassle unable to trust operators offering encrypted services.
218(7) looks a bit powerful
218(8) notices are secret but allow for permission to be given to disclose them, slight improvement.
More...
Some people are working on something of a public diff from draft to proposed bill, and that will help comment further on this.
See a diff here https://github.com/StuBez/UK-Investigatory-Powers-Bill/compare/master...StuBez:final?diff=split&name=final&w=1
P.S. I appreciate that I have not said a lot here - there are many more issues, and I'll try and post something much more concrete on this so people have something on which to base letters to their MPs later.
P.P.S. Open Rights Group (as have many others) have done a good page: here. Please contact your MP.
Summary
Way too much still in terms of serious powers for the state to spy on its own people. Way too much in the way of bulk powers. Way too much in data retention - snooping on pretty much any data.
Comments to follow...
Retention
78(9) is still as wide scoped as before, but says "and this expression therefore includes, in particular, internet connection records." which does not help matters at all.
84(2) I see retention notices are still secret but can be disclosed with permission from secretary of state, which is some progress I guess.
225(1) slight improvement: “data” includes data which is not electronic data and any information (whether or not electronic),
Encryption
217(4) Still has removal of protection (encryption) though now "applied by or on behalf of that operator to any communications or data" which is an improvement, and 217(3) and 218(4) relate to orders being "practicable" which is again an improvement. However, reading this, it could require an operator like Apple to change iMessage so that it is possible for them to remove encryption, or any hardware or software supplier in the UK or overseas. This leaves criminals able to encrypt but normal people that do not want that hassle unable to trust operators offering encrypted services.
218(7) looks a bit powerful
218(8) notices are secret but allow for permission to be given to disclose them, slight improvement.
More...
Some people are working on something of a public diff from draft to proposed bill, and that will help comment further on this.
See a diff here https://github.com/StuBez/UK-Investigatory-Powers-Bill/compare/master...StuBez:final?diff=split&name=final&w=1
P.S. I appreciate that I have not said a lot here - there are many more issues, and I'll try and post something much more concrete on this so people have something on which to base letters to their MPs later.
P.P.S. Open Rights Group (as have many others) have done a good page: here. Please contact your MP.
Subscribe to:
Posts (Atom)
FB9000
I know techies follow this, so I thought it was worth posting and explaining... The FB9000 is the latest FireBrick. It is the "ISP...
-
Broadband services are a wonderful innovation of our time, using multiple frequency bands (hence the name) to carry signals over wires (us... -
For many years I used a small stand-alone air-conditioning unit in my study (the box room in the house) and I even had a hole in the wall fo... -
It seems there is something of a standard test string for anti virus ( wikipedia has more on this). The idea is that systems that look fo...