"Up to" speeds

I have just had a very interesting conversation with a Canadian, moved to UK and looking to get our services.

Step 1 was to say, please, please, please, read the terms carefully - we don't know what is "normal" for you and how that compares to what people in UK expect, so read them and ensure you are happy with the terms... I think that is fair.

But he went on our order system and saw we quoted VDSL at 64Mb/s. The fact I had to explain "Mb/s" is same as "Mbps" is a concern.

What was interesting is we quoted the 64Mb/s and he asked how that compared to Plusnet's "up to 76Mb/s".

This is the crux of the matter. The "up to 76Mb/s" is a general UK wide "up to" for VDSL, I assume. We quoted 64Mb/s for a specific address.

The point I had to make was that for VDSL, the speed will be the same for anyone using the same copper pair and the same modem in the cabinet. The forecasts and estimates are just that and ALL WILL BE THE SAME regardless of the ISP.

Now, we know the backhaul and ISP network matter. AFAIK Plusnet are no that bad. But for forecasts on the last mile VDSL sync speed it does not matter who he goes to.

So finally, having explained that, he asked why buy from us?!

19:44 <RevK> We are better in many other ways, best to ask customers on #A&A for

             unbiased opinion.
19:44 <RevK> We usually have better backhaul - you have looked at VDSL sync speed

19:44 <RevK> We have better technology (working IPv6)

19:44 <RevK> We have better politics (I sat in parliament talking to a committee to 

             help protect the freedoms of our customers)

Where are we with Unifi and iPhone roaming?

As you will know I have spent a long time trying to understand the issues we see with the Unifi access points and roaming between them using an iPhone.

A&A sell these, and some of their PoE switches as well. We may start selling more stuff in due course. Overall the Ubiquiti stuff is pretty impressive and there is an increasingly large range of devices. The WiFi is technically very good at the hardware level, and we sell in boxes of three even for businesses.

So it is important to us that they work. I also use them at home, and my family treat me as tech support (obviously) so it is important to me if I want a quite life. They were all round this Sunday - we had sort of cancelled Mothering Sunday for obvious reasons, but everyone came round and we had pizza and chatted. They all told me in no uncertain terms that the WiFi here is crap and they even turn off on their phones and use 3G/4G when round the house. They all use iPhones. That really is a bad sign.

I myself spend a lot of my time in my office at home, but whenever I leave for the rest of the house I find I have to turn wifi off and back on. Though, technically, it is far from every time and can even be the odd day with no apparent problem, whilst other days I see many times. The problem is, as always, you remember the times it breaks.

This also makes testing hard - something changes and you watch it, and see you spend all day with no issues and think it fixed, when actually it is just intermittent, still, just as before.

I have an AC Pro and two AC LR in the house, and they are now on latest firmware. I thought that may have helped, but no. We also tried changing switches, and thought that had helped, but no.

The current state is that I have managed to mess with wiring enough in the house to actually have all three APs off a single Ubiquiti EdgeSwitch8 - one of their switches - so as to eliminate the switches as the cause of the issue.

Tip: Some of the Ubiquiti kit is still passive 24V PoE, and their switches are great as they support that, but you have to configure on the switch! It is not automatic as PoE normally is.

We also did tests with just IPv4 on the LAN, only for a few days, but that seemed to just work. This means the current thinking is that it is the IPv6 being present that is causing the issues. It could be some combination of bugs in iPhone, Ubiquiti, and even FireBrick code, for all we know. Reports from others that use this kit say no problems. We did a lot on FireBrick to try and eliminate that as the cause. However, with IPv6 on the LAN, even with IPv4 being static on the iPhone and no DHCP, it can still fail. Setting up DHCPv6 on the LAN does not seem to change things, we normally use just RA/SLAAC.

The symptoms are a sudden lack of connectivity when it roams. For a few seconds the phone may show the old IP addresses, but quickly switches to showing no IPs and then to showing the 169 auto addresses. Wait as long as you like, it is broken. You need to turn WiFi off and on (on the phone) to fix it.

Part of the reason for writing this up again is for the engineers at Ubiquiti - they are trying to fix this. Good news (though I seem to have to poke on twitter to get things progress, sorry guys). They sent me some switches and a router and gateway. Big thank you - nice to eval the kit as some of it we may start selling. We sent them a fully loaded FireBrick FB2700.

At this point the next stage is for me to try and create a setup using their kit as the gateway on the LAN and so doing IPv4 DHCP and IPv6 RA/SLAAC, and see if that breaks still. It is a pain as I cannot exactly replace my router as it is the office router. So I have set up a new IPv4 and IPv6 subnet for WiFi use. Not ideal, but will do for testing.

They, for their part, need to try and set up with a FireBrick to do the same. Can they make it break? Obviously I am on hand to help them set that up.

So setting up the Edge Router. It is a simple set up. No NAT. Fixed IP /24 IPv4 and /64 IPv6 on LAN with DHCP serving IPv4,and RA for SLAAC doing IPv6. On WAN is a simple IPv4 which can be DHCP client or static, and a simple IPv6 which can be SLAAC or static. Obviously need to set IPv6 DNS servers for RA on LAN.

So far I have managed to set up:-

• Firewall off
• NAT off
• Static IPv4 on WAN (a /24 for testing)
• Gateway 0.0.0.0/0 route on WAN, can ping out to internet
• Static IPv6 on WAN (a /64, obviously, from my PI block)
• Gateway for IPv6 on WAN
• Static IPv4 on LAN
• DHCP IPv4 on LAN
• Static IPv6 on LAN
• RA on LAN configured by ubiquiti for me

And I am stuck. So waiting on Ubiquiti at this stage. Suffice to say I don't think they are a threat to FireBrick as this is all pretty simple on a FireBrick.

No word on where they are with FireBricks. Obviously keen to help them test the other way around. To be fair, if this is either a bug in FireBrick in some way, or more likely, something we can work around by changing FireBrick in some way, I am more than happy to do the work to make that happen. We have implemented a number of "pragmatic" aspects to the way the FireBrick works (sometimes on a config setting so as to be "standard" by default) and I'd really like this WiFi kit to work...

I think best if I update this post as we make progress for a bit rather than new posts. Let's get to the bottom of this, shall we?

Updates:

• From comments, it is not just FireBrick, but is some rare combination of things clearly, and seems to be Ubiquiti APs and iPhones and "something" else.
• IPv4 gateway not working was user error, I mistyped as 0.0.0.0/24 for some reason
• Someone from Ubiquiti, in Austin, Texas, in the middle of the night, is working with me on this now.
• IPv6 gateway was not working as I was using the zero address in the /64 which the ER had assumed it can have making it a router on the WAN side, which is unexpected. I changed to the ::1 in the /64.
• Now wifi all on ER not using FireBrick, thanks to guys from Ubiquiti working in middle of the night. Roaming appears to be working, more testing to do. I am being sent a cap of working roaming as seen by ER, and will get same from FireBrick.
• We now have two interchangeable set-ups. Both on same sets of IPs as a separate subnet for my WiFi run as a LAN side of a router. I have the ubiquiti EdgeRouter set up, and the same set up on an FB2700. At present both seem to "just work" but as I say, this can take a whole to see the fault. I have lots of logging. One clue is that I am sure I have seen the iPhone re-do DHCP on roam, and the current testing (on both set-ups) does not do that - it just flips over to new AP basically seamlessly. So, just more testing for now. If both these "just work" we have to go back and see what else on the main LAN could be upsetting things in any way.
• This morning (Saturday), still no apparent roaming issues! This is using a FireBrick but on a separate LAN the same as the ER set-up. Again, if the roaming happens without involving the gateway router, no way the FireBrick can be to blame. If it is OK for a few days I look to swap back to main LAN and see if that shows the problem again.
• Sunday, still using a separate FireBrick as gateway, and have set up the second VLAN that was being used before on it. Still not failing. This makes no sense at all.

The new £1 coin

The new £1 coin comes out today!

It has a lot of nice security features, making it hard to copy. Obviously not impossible to copy, but that is not what is needed. What is needed is something that it is not economical to copy.

All but one of these "features" are explained. They are quite clever, from micro writing to a lenticular feature that appears too change an image depending on the angle (described as "like a hologram").

But there has always been listed this one "special" security feature to make forgeries detectable.

The problem is that it appears to be "secret".

This is bad!

Firstly, it is a concern that they have to keep it secret. That suggests that if it was not secret then a forger could reproduce it. That makes it "security by obscurity" which never works. If a forger could not reproduce it even if they knew the secret, then why keep it a secret?

Secondly, how is anyone supposed to check the coin is not forged if they do not know what this secret security feature is. It may as well not be there if people cannot actually test it.

Thirdly, assuming some people get to know the secret so they can check it, that means the secret has to be distributed to a lot of people, even if it is "only banks", and so that means the secret will definitely get out.

Who thought any sort of "secret" security feature could possibly be a good idea?!?

And as if to prove a point: The secret is out!

http://www.telegraph.co.uk/news/2017/03/28/revealed-new-1-coins-hidden-security-feature-works/

Misleading customers?

Speaking as director of A&A I can say for sure that we absolutely do not want to mislead or trick customers in any way whatsoever, at all, ever.

We actually want to make the products and services we sell "intuitively" reasonable and fair if we can. The sad state of things is that some things have some caveats. The main one being the minimum term on some of our broadband services and so some early termination charges.

In some cases we have taken a pragmatic and statistical view, so FTTC on BT backhaul we sell on 6 month terms even though we are stung with 12 months terms from BT. This is because what we pay BT for the circuit is only part of the cost - it does not cover the bandwidth usage, for example. We feel that overall, even for those ceasing within 12 months, we do not lose out, much, on average, by having a 6 month term and a low monthly early termination charge for remaining term.

Sadly, for the TT FTTC backhaul, not only are we stuck paying a to a 12 month term, we even get stung with a nonsensical "early termination fee" on top of that - even though TTB get the full cost for the term without having to provide service. It is actually more expensive to have service for 355 days than 356 days. It is a crazy situation which we have tried to re-negotiate without success several times. As a result we sell these services with a 12 month term and a higher monthly early termination charge just to make sure we usually cover our costs.

If only Openreach did not do FTTC/FTTP on 12 month min term and if only BTW and TTB did not insist on 12 month min term too, we would not insist on it, simple as that.

However, on the bases that we are stuck with some shit like this, and hence some services we sell that are not as "intuitively fair" as we like, we are really keen to ensure transparency and making sure no customer is fooled or tricked by what we sell or misled, or even misunderstands by mistake.

The web site explains. The order form lists "key contract terms" and requires ticking a box to agree the minimum term and early termination costs. These are the very first of these terms. Not hidden away with all the bureaucratic crap we have to do anyway - but up front, at the top. We then email those terms as an order confirmation just to re-enforce what was agreed.

Recently a customer purchased one of these 12 month term services at a premises where they would be living there for only a few weeks. And as such was shocked at the cost to terminate or move to new address within 12 months.

I was horrified! How did this happen? We try to be so clear. We even have services (with higher install costs) that have only 1 month term that would have been way better. How did this happen?

The customer did speak to sales, and so far I have confirmed that the sales staff have reviewed the calls, but I want to as well. It is all very well saying "you ticked this box", but the whole customer experience matters.

What is the way A&A handle this? To be honest this is rare so comes down to me to decide. What I have said is that if we, in any way, on the phone call (for example), misled the customer, then the customer should not lose out and not pay the early termination. If we said something on the call, we stand by it. If that is wrong, then I have to make sure that staff have the training they need to not make that mistake in future.

But, as it seems to be, assuming we are right, and we were not misleading in a call or website or order form, this is still a big concern. How did this happen, and importantly what can we do to stop any sort of misunderstanding like this in future? Bigger text, bolder font, what?

I take this seriously - we are not here to trick customers. When there are caveats, like minimum terms, we need to be really clear. I really wish we did not have them, but we are not trying top "trap" customers. Customers should stay with us because they want to - because the service is worth every penny. Definitely not because we tricked them.

So my job this week is going through the call recordings, the web site, the order form, and doing all I can to ensure no customer ever has this misunderstanding again. I have offered this customer the option to spread payments (though she declined). If my investigation finds any hint that we misled her in any way, I will refund the early termination charges.

If A&A do not trade on integrity, we should not be trading at all.

Update:

So far we have added this extra box above the "Key terms", tweaked wording a bit, reviewed call recordings, and discussed further ways we can improve. Thanks for all the suggestions.

Was I wrong on the oven?

I am puzzled. I checked the manual for the oven (here).

The instruction is

"Operating the Ovens

Push in and turn the knob controlling the oven to gas mark 9 (Fig.2-22). The oven will light automatically.
Turn the oven knob to the desired gas mark (Fig.2-23)."

OK, no suggestion of delay, or that going to 9 will be a way to get to temperature faster - just to ensure it lights by the look of it. It does not say why.

Fair enough, but I really have to wonder why. The manufacturer of the oven should have a reason for this, surely?

I could understand if there was some aspect of the lighting mechanism that was somehow more reliable if you do this.

But that cannot be the case. The oven has to be able to self light at any gas mark because it can be set for a timer. If you set for a timer, it will not have the "turn to 9 first" aspect. That is why the spark is automatic and not a button like the top burners. So it must have a reliable mechanism to light anyway without turning to 9 first.

Yet, they have put it in the manual.

Perhaps I'll write to them and ask them.

Critical thinking

There are a lot of memes around - ideas and rituals that get passed on to people and spread by word of mouth (or worse, the Internet), even distorting and evolving and they spread.

Some of these are plain silly.

My kids all seem to have got the idea that you should never, ever, set the volume control on your car stereo or TV to an odd number. Why? because it is bad luck! Personally I think it is bad luck to be superstitious.

Some may have had a good reason once.

Our industry is not immune to this - BT would always set Ethernet ports on leased lines to no auto negotiation, fixed full duplex and fixed speed. Why? nobody really knows. Many "CISCO people" say the same. The best explanation I ever got is that when auto negotiation first came in it had a lot of problems and this was the work around. These days that is simply not the case, but the work around persists causing its own problems. A better work around of fixed settings announced by auto-negotiation never seems to get used. Historical problems have led to a meme, one we are stuck with mostly and will be for years to come. To be honest, we have much the same feeling in the office when it comes to spanning tree having been badly bitten by that years ago and with no real reason to think it is actually a problem now, but we don't really want to risk it.

Some may be a basic misunderstanding.

Setting a thermostat to max to get to desired temperature more quickly. This drives me nuts and I posted on how it was even applied to an oven. In the car, if it is cold the temperature gets set to max (like 30 ℃), then it ends up too hot, so it is set to min (16 ℃) and this is repeated! Setting to 21 ℃ does that for you, and gets to 21 ℃ just as fast as if you set to 30 ℃. Somehow people don't understand how things actually work, not realising the binary nature of the output of most thermostat based systems. This "solution" works for them, well, sort of, so they don't question it. Indeed, people refuse to understand why they may be wrong! We had much the same when air-con installers said never use "auto" mode, but could not in any way explain why that was a problem, or, if it was a problem, why manufacturers include the feature. At work the auto mode allows the two control temperatures to be set and to allow a narrow range. At home there is one setting an around ±2 ℃ making an annoying 4 ℃ window. Apart from an "economy" mode making the window wider I have no other control - so I end up using "heat" in winter and "cool" in summer so as to maintain a more controlled temperature. But this was not an explanation the installers had, and would not apply to the more controllable office system they were installing.

Some come with their own explanation.

The first thing I do is ask "why?". This is the critical thinking, and I think I have managed to get my kids to do the same. The odd numbers on volume does rather surprise me, but I think that falls in to "playing a game" rather than a lack of critical thinking. If there is no explanation then I don't do it or pass it on. If we were all like that then stupid memes would die out. However, some memes come packaged with their own plausible explanation. This is clever as it is basically targeting the critical thinkers. Ideally it is an explanation that is plausible but hard to verify. Sadly the explanation may be totally made up, as part of the way memes are spread - someone once asked why and someone made a guess or invented an answer and it stuck. There are examples like "why never boil the kettle twice" which often comes with "because every time you boil it you reduce oxygen, and so boiled twice does not taste as good". See this (with some nice graphs) for a clue why that is almost certainly bullshit. One clue is when the same meme comes with a variety of unrelated "explanations".

Sometimes the explanation is the meme.

Sometimes the whole meme boils down to the made up explanation - and this spreads because people want to be smart and tell someone something they know and the other person did not. Of course, when the explanation is made up, there can be a counter meme to explain why actually that is a common mistake. I suspect there are cases where these exchanges can go on several times.

Even so, memes, these stories and "facts" and rituals that float around, are an interesting topic. I am sure that once upon a time, before books and the Internet, they were essential in ensuring collective knowledge allowed people to learn from other's mistakes.

Thermostats (again)

(Update: Sorry to my wife. According to all of my five kids I am being tactless, again, which I know I can be. But some interesting comments have been made on this post. Perhaps the best is don't argue with the cook!)

Blazing row with my wife, again. This time over gas oven.

She is adamant that you start by setting the oven to max (gas mark 9) first and then down to what you want, (e.g. gas mark 6) to get it to temperature quicker, and apparently the man that came to repair the oven today said the same.

I am sorry, but the gas oven, like most devices using a thermostat, has a binary output (on or off). It is either a low level pilot or full blast. That is it. I am sure some systems work differently, but most thermostats are binary output, as that works, and is simple.

They have a target temperature, a current temperature, some level of internal hysteresis, and they have a binary output. Pretty much end of story. Fact.

Of course, being a bit scientific, I went and checked. I would not want to be stating fact when I am wrong. I turned oven on to gas mark 1, door open. There is an initial low temperature state where output is in "low" pilot light, but after a few seconds it goes to the expected full blast. Changing dial to gas mark 9 does not change level of blast. There clearly are only the two options, pilot/low and full blast. That is it, as expected.

This means even if you only want gas mark 1, selecting gas mark 1 gets to temperature just as fast, at full blast, as selecting gas mark 9 to start with. There simply is no point in this ritual. Indeed, it can lead to the oven being over temperature and having to cool down. I think, for a cook, that is also bad. I am not a cook.

This simple fact about the way thermostats work in so many systems, whether the car air-con, the house air-con, the cooker, whatever, is apparently beyond my wife's comprehension. I don't know why, she is not stupid.

She is apparently sick of me lecturing her! Well, duh, if she finally got this really simple fact of life I would stop doing so. That is sort of how lecturing people when they are wrong on something works! You keep doing it until it sinks in. Well, I think that is the rule. It is so frustrating. As I tried to explain, black is black and white is white and if she was insisting that was not the case I would lecture her until she accepted it. This is not different. It is a simple fact of life, not opinion, not something that is unknown, is is so simple.

Why is this hard?

Update:

First off, yes, absolutely, I may be wrong in some (or all) aspects of my understanding of how this thermostat works in this case. Having played around with the oven a bit (not done proper tests with thermocouples and the smart meter measuring gas flow yet) I think I am not, but I may be. Even so, that does not mean what she is saying is right. However, if I am wrong, I am prepared to listen to reasoned arguments and references, and change my view and learn something. I'll even apologise.

What makes it so frustrating is that I have something to back my point of view - I can explain how a thermostat works, and why there is a binary output, and so on. The other side of the argument comes down to "stop lecturing me" rather than any explanation of why I might be wrong. I might indeed be wrong, but that is not the way to have any rational discussion, and that is what winds me up, sorry.

Why would I be wrong? Well, firstly, I don't like to say how something works without knowing how it works. I am sure that happens some times, when I have deduced how something works but got it wrong, but obviously I don't like to be wrong - who does? I am pretty sure I dismantled something with a thermostat in it when I was a kid - so I saw, first hand, how it works. In most cases it is the same today, I am sure, where the output is binary - on or off. This means in the case of an oven you do run at "full blast" until you get to temperature and then stop, and the cycle on/off in some way. Interestingly someone has suggested that this may not be the case, so I am hoping some time to test that. It was suggested there is a linear control near temperature. I can believe that, but it seems unnecessarily complicated and expensive. It is also suggested that the oven will not be fully at the right temperature when the thermostat in one point thinks it is. Even so, getting to temperature by setting a wrong and higher temperature and letting it then cool is likely to take longer over all, so my overall conclusion would be right. The idea of being hotter before opening the door or putting in a cold container is an interesting one, maybe that has some merit.

The additional heater on the thermostat is interesting. I have heard of this before, and I am pretty sure that it effectively reduces or cancels out the temperature hysteresis inherent in a sprung bi-metalic strip based thermostat, possibly even introducing a more controlled time based hysteresis. This would mean the switching between on and off at the target temperature is more rapid even if the temperature is stable thus creating an (albeit slow) pulse width modulation effect to maintain a proper temperature without the temperature swings you would expect with the simple temperature based hysteresis. This is clever, but won't change what happens in the time before you get to the selected temperature - which will be full blast all the way.

In light of the comments so far, I feel I am quite correct in saying that putting it up to 9 to get to temperature fasters is just silly, a meme, and old wives tale, and not needed. Even so, I may have something to learn about the details of how it does work, and will see if I can find the time to test and learn something.

P.P.S...

I am not alone :-

OFCOMs plan for automatic compensation for broadband faults

OFCOM have published their proposals for automatic compensation for broadband and phone line faults (here). It is a consultation and we are replying. Even if it goes ahead it will be some time before it comes in to force.

The proposals have some good ideas, and as a consumer it really helps to understand where you stand.

The key aspects are pretty simple...

1. Delay fixing a total loss fault, £10 per day (after end of second working day after fault report)
2. Delay providing service (if promised a date), £6 per day
3. Missed appointment, or cancelled/changed within 24 hours before, £30

This is simple, easy to understand, and not that daft...

The good news for us, as an ISP, is that all three of these are basically someone else's problem. Appointments to visit a customer for any reason are not by us as such, we don't go to customers, we arrange through carriers, which arrange with Openreach. So missed appointments will be 100% down to Openreach. The same is true for delays in provision, and delays in total loss fault repair in most cases. We're careful not to promise an installation date, so the delayed provision pretty much bypasses us, but we may as well pay out if we get compensation from carriers.

The fault stuff is good for us as it is "total loss of service". This is important. There are a lot of grey areas, and a lot of cases of vague faults. We spent a lot of time arguing with carriers about cases of low level packet loss, unexpected throughput issues, extra latency, intermittent service and frequent drops. None of those are "total loss". All of these are, thankfully, out of scope. A total loss of service is generally pretty clear cut. So OFCOM have got their head screwed on there.

In theory a total loss of service could be us, but unlikely for such a thing to ever last for two working days. So basically, this is all down to carriers and Openreach. Yay!

To be honest I like the simplicity of it all. Well done OFCOM. But...

Carriers not forced to pay ISPs!

The biggest issue is that whilst it is clear that every one of these cases is someone else's problem it is us that have to pay out. It is right that we pay out, we have the contract with the end user. But OFCOM think we can negotiate with carriers to get the same compensation from them so not be out of pocket if not our fault, which it won't be. OFCOM clearly have no fucking clue. We may be lucky - we may find the big players manage to beat up BT Wholesale, Talk Talk Business, etc, and they do in fact offer us the same compensation. Given that we struggle to negotiate for BT Wholesale to actually have a process to even fix faults in the first place, let alone define what a fault is, I do not hold out a lot of hope. It it works, and we get the same compensation for the same things, we are fine. We will have an admin cost of recognising and automating the payouts, and checking that we get the compensation from carriers, and disputing with carriers when they have not paid us (there will be disputes, guaranteed). That is a cost, but overall this will be OK, we are good at this stuff. Fingers crossed.

Broadband depends on a phone line!

As you probably know most broadband services needs a working phone line. If the phone line breaks then the broadband cannot work. This is pretty basic stuff, but OFCOM think most services have same provider for both. That may be true on aggregate but for small ISPs it is not, and most of our lines have someone else providing the phone line part. Oddly the phone lines we provide are out of scope as they don't allow calls, so if we did a line for someone else and it failed (killing their broadband), we would have no obligation to pay compensation for the phone line being dead. Crazy or what?!

The way it is worded now, someone can complain of no broadband to us. We look, see PSTN fault, and say "report to phone line provider". That is all we can do. We have no contract with the part of BT doing the phone line. We cannot do anything to fix it. Forcing us to pay money won't change that! The end user has to contact their phone line provider. But according to the proposed rules, if the broadband is not working in two working days the we have to pay out £10 a day until it is, even though not our fault and we have no means to fix it. There is not even an obligation on the end user to report the fault to the phone line provider. They could sit on it, raking in £10 a day, from us, forever!

There has to be an end to payouts at some point?

OFCOM decided on no cap on daily payouts. I understand, but ultimately we could find there are services we cannot provide. The checker may be wrong, the line may be just too long. In rare cases even a fault may not be possible to fix. In these cases there is the option for us, or the customer, to cancel the order or cease the service. But the wording as proposed does not clarify that this stops the daily payments. If we accepted an order and gave a date (based on carriers telling us), and then cannot provide, could we be paying £6 per day forever? Scary!

Using ADR if not compensated - bad idea!

OFCOM propose that ADR should cover cases where a customer thinks automatic compensation is due and not paid. This is a huge problem.

OFCOM have sensible limited faults, for example, to "total loss", which is pretty clear cut. But a customer with an intermittent fault or other speed issue, that takes time to fix, may want compensation. It will be a very easy mistake to make, or a tricky argument over "total loss of service". It could even be someone with a router problem, and needing a new router, feels we should pay when no compensation under this scheme is due.

The problem is that even with many days delay the compensation is way less than the circa £350 cost of an ADR case (even if we win). So the customer simply has to say they feel compensation is due and threaten ADR. Any ISP will have to pay up else pay more for ADR even if they win the case.

So the limit to "total loss" is moot once you factor in the threat of ADR. However, assuming we get compensation from carriers they won't pay us for such cases and we cannot take them to ADR.

Ultimately this extends the compensation to cover every fault, not just total loss, but intermittent and customer router issues, and the ISP pays out with no way to "improve the service" which is what this whole scheme is trying to do.

Post takes time!

The normal process for most faults is to start by eliminating end user equipment. The carriers make us do this, and it is not totally daft. So usual process is send a router out, which takes a working day, at least.

OK, so we do that and it turns out that it really is the broadband at fault. So we report to carrier. But we have already wasted one working day on sending a router.

Even if we negotiate with carriers to pay compensation to us, they will not allow for the wasted working day in such cases, even though they insist we replace equipment first.

This means we have to change our processes and report a fault to carriers at the start, then send router while they are saying "no fault, book an SFI", and then we say, yes, a fault as we have now tried a new router. All a bit iffy, but process needs working out. I can see the carriers being pissed over this as it gives them at least one fewer working days to fix things.

It costs money!

At the end of the day, if we have to pay out more because of all of these gaps we have more costs, and have to consider higher prices. If the carriers have to pay us more to pay out, they may charge us more, and we have to consider higher prices. At the end of the day the consumer pays. Well done with that OFCOM!

Keeping Customer Informed

We are starting on a fun project at A&A. Well, to be fair there are a few major projects going on, but this is likely to be one of mine rather than the ops team, at least to start with...

There is this horrid term BT use, "KCI", which is Keeping Customer Informed. They have these stages KCI-1, KCI-2, and so on.

But, in spite of the annoying term, the principle is reasonably good, and we are working on a system for A&A.

The basic concept is that there are a lot of cases where automated systems (usually) need to update a customer on some progress of something - whether an order or a fault or something else. At the moment we have a lot of systems, some of which get KCIs from BT or TT, and trigger events, some from our own systems, some as a result of an action by staff or a customer, etc. There are some consistent systems for some subsets of what we do, but nothing as a whole.

So the plan is to make a new system, a general purpose system, that can easily be bolted in to all of the systems in place of what we do now, and be consistent and helpful to customers.

The first issue we identified is there are two main grades of notice to customers. The simple "short message" type thing such as "order accepted by BT", "appointment booked for the 3rd, 8am-1pm", etc. The second type is more detailed long messages we currently send by email, such as the detailed order confirmations, or notices about open DNS servers, etc.

The short messages can be sent in many ways, and we currently, for some parts of our network, have messages with a choice of SMS, Twitter or email. These are all ideal for the simple short message type notifications.

So the plan is to allow customers to define, at various levels, e.g. a control page login, where they want notifications sent, and maybe even more than one place at once. Also, especially with text messages, time windows such as (8am-8pm Mon-Fri), etc, so not woken by unimportant messages. We may in future be able to extend to Signal, or WhatsApp, or whatever, where there are APIs.

We have to allow for message to be time sensitive, e.g. no point sending a message about an appointment after the appointment has happened. We may have to delay some messages, e.g. if a line is flapping, the line up/down messages get delayed (for both cost and annoyance reasons). We may also have to pair and cancel messages, e.g. if you have texts 8am-8pm but at 3am your line drops and reconnects a minute later then those two messages can cancel out and not be sent at 8am. We have to also consider load and rate limits on things like texts.

Now, when we get to emails it is also a bit fun. These can be used for the simple texts and for longer notices we send. We already try to sign most emails with an automation signature, but we are considering encrypting emails. We have been asked about this by a few customers, and we need a central system to handle this (makes no sense for everyone to have their own keyring). Our ticketing system could do it for us even.

So how would customers register a public key. Well, the plan is they email it to us, and we send them an encrypted email with a link to confirm the email address. Once done, we have a database of customer public keys and email addresses to use for sending email.

That is pretty simple, and the wonders of GPGME library have been impressing me for the last two days.

The huge problem is turning it off. Technically simple, and we can have have a staff interface for that, but the issue is policy. If someone wants encryption and has any risk of emails being read in transit, they do not want someone to just be able to phone up and turn off encrypted emails from us. Indeed, we cannot even sent a link to confirm which is not in fact encrypted to be safe. Sending an encrypted link will work for someone simply wanting to turn it off, but what of when someone loses a key??

Indeed, even accepting a replacement public key is tricky as it could be sent by someone that has means to intercept email, and they can then extract the confirmation link from our reply as they made the key.

Obviously the traditional face to face key signing is not practical on scale.

We could use customer login, 2FA, and so on, but how do we know the email they are using is really them. They could use their account but with someone else's email address which they have means to intercept, even if temporarily.

I am slightly at a loss on best practice on this at the moment. Comments welcome.

I suspect the best we will do is create policy and a good practice which minimises risk, but can never be bullet proof.

No guarantees on timescales yet, and it will be a gradual deployment, but watch this space...

P.S. PGP/MIME is a pile of shit, IMHO, so far.

P.S. One issue is that we are not expecting to register public keys "per account" or "per login" but "per target email address". We would send an encrypted reply with confirmation link. This makes validating changes or removal more complex.

P.S. After a lot of work on the library, I am the PGP/MIME king - all working as expected.

Proud dad

I have done a few posts like this, and I understand it is a bit much for some people. I appreciate all of the comments, thank you. But it will carry on for a bit I am sure, at least until the coroner's report and funeral. Sorry if it is not your cup of tea. This is new to me, and new to the whole family.

We spoke to the funeral directors and they said neither they, nor the crematorium, charge for an infant funeral. I was happy to pay whatever it costs but it is free, which is not something you expect.

But with all of that I want to say how proud I am of my daughter, and how she is coping. In some ways I think she is coping better than I am. Today she has been home and been in to the nursery that they had set up, with all the presents from the baby shower and all the knitting from my wife. They are both getting there and coping really well. They are so strong.

What really made me realise quite how grown up my daughter is, and I did not realise this until now, is that she has the hard choice over Thomas. After over a day not sleeping, in pain from an emergency C Section, and in the early hours of the morning when his heart stopped for the third time, she had to decide if to try and restart it yet again. She said she realised he was gone. She made the right choice, I am sure, but I struggle to understand how I could have faced such a choice myself. There is nothing more grown up I can imagine for my baby girl, or anyone.

And typing this I have set myself off again... Blurred vision once again.

I hope she does not mind me posting this. I really am proud of her.

Think of the Children (again)

Once again, the nanny state is on about censoring the internet because "Think of the Children".

See ISP review article (here).

I have said it all before, but it it may be worth explaining a bit about what AAISP do here.

Firstly, all of our customers are adults, we do not sell to minors at all. And all broadband customers have actively selected that they want no filtering. So I suspect we comply with even these latest suggestions.

But there is a lot that our adult customers can do to take responsibility for their children using the Internet. Remember, if you have teenage kids that want to access porn, they will always be able to - it is not clear that there is evidence that this does any harm to be honest, and a solution to any harm (such as skewed ideas on relationships) is better education - talk to your kids, explain that porn is fiction just like thrillers on TV, and sci-fi. Talk to them about relationships.

However, for younger children, every computer system these days provides a range of "parental controls". Indeed, sometimes it is hard to set up a machine with these turned off! Use these tools.

Also, you can set up additional free and even paid for tools if you need - just search for them!

We can also help - we can set the default DNS servers on your broadband to be OpenDNS, which allows you to set up DNS level filtering that will help stop young children stumbling across the more dodgy parts of the Internet. If you are unsure, call us and ask for help (or chat on the web page, or irc, or email, or SMS). The only thing we don't do is filtering in the connection we provide to you.

Ultimately, as our customer, you are in control of your computers and have access to a range of tools to help.

Back to work

So, the plan is, back to work tomorrow. I have been off all week to some extent, but have managed to spend most of this weekend working on some updates to FireBrick code to better handle IPv6 / IPv4 fallback logic (pain in the neck as IP addresses embedded in messages in SIP). I also improved the config options for L2TP client mode as well. All good fun (!)

I need to go see an undertaker still, but we'll get around to that soon enough.

My daughter is not even ready to go back home, and has to decide what to do about the nursery - i.e. do we go and clear it out and re-decorate before she goes home? She'll get there in her own time.

Only a couple of weeks until the A&A customer event on 2nd at our offices. I'm worried at how many people will be awkward around me. We're getting plenty of drink in, so that should be fine.

One size does not fit all

The standard model of grief is Kübler-Ross model. Grief follows Denial, Anger, Bargaining, Depression, and Acceptance.

Clearly it does not always follow that model.

When Thomas died, acceptance of the situation was basically immediate. I know there were a few brief cries of "No!" in the house, but very brief. We all accepted the truth very very quickly. There was no denying it.

Depression has lasted some time - a great sadness - nobody to blame - it happened, It was very very sad. Lots of things have triggered tears for us, in different ways. Sadness abounds. Time marches on.

We accepted it early on, we need to "move on" and "get on with life". That does not fit the model well.

Today Thomas had an autopsy. It was a challenging day for us, we know what it means, and none of us felt we could go and see him yesterday before it happens (it was an option). We have some good pictures of him. This one was edited to remove the tube in his nose, and well done by Glenn for doing that for us. Something to remember him by.

We all accept what happened, but there is the very real chance of a major anger stage depending on the coroner's report. If, in any way, someone was to blame, we will all be very very angry. But that is all on-hold, maybe for weeks. If someone else was to blame, I can see anger for many of us for many years, and that is not healthy. We wait and see.

Brexit Broadband

More questions than answers here, and I would not be surprised if nobody really knows the answers (yet).

UK leaving EU should not, in itself, have an impact on the broadband services A&A sell. We only sell broadband in the UK. There is quite a bit of regulation of communications providers, and some of that may change once we leave, but should not be a massive impact, we hope.

But what of Northern Ireland? We provide services there. If NI leaves UK and becomes part of a united Ireland (and presumably part of EU)?

And what of Scotland? If Scotland separates from England, do both parts stay a part of the EU as they were before, and then England leaves EU, leaving Scotland still in EU? Or does Scotland somehow leave EU as part of breaking from UK? No idea. But either way, what if Scotland is a separate country?

At present, BT plc operate the local loop and exchanges, and (trading as BT Wholesale) sell us circuits that link premises in England (and Wales), Scotland, and Northern Ireland to host links in London. We sell to people in all three areas.

We do not know if a split would be a BT Scotland as a separate company? And we do not know if circuits in Scotland can continue to connect to a host link in London? Would we need a POP in Edinburgh to keep selling to Scottish customers?

As a business we do very little international trade. We have been known to sell the odd piece of equipment, paid on a card, in advance. We got stung on at least one occasion. It is quite rare. But would we want to provide broadband to someone in another country?

Issues, off the top of my head...

• Currency - how to price things in Euros, or Scottish pounds? Bitcoin anyone?
• Direct Debit and payment - how to get the money to us?
• Disputes and legal issues - bad enough Scottish courts now, but what of Scottish ADR?
• Regulations - meeting all EU communication provider regulations even though not in EU?!
• VAT, how much, who paid to? EU stuff is bad enough now, and we managed to avoid it.
• Export tariffs even? Not just on broadband as a service but routers we sell, etc?

It may be that we have to stop selling to people not in the county we are in even? Or maybe it will be easy somehow? Or maybe we have to create a separate Scottish company?

Right now - I have no idea!

Tactless

I am not known for my tact.

No, really, believe me, I am tactless at best. Yes, those that know me (and some that do not know me well) will know this.

So I have had an interesting couple of days, tact wise. The whole issue is all around taking offence, and, well, giving offence. I strongly believe in free speech, and that offence is something people take. But I also fully understand that there are perfectly good ways to avoid offending people and many circumstances where one really does not want someone to take offence, even if none was intended.

So the two sides of the coin here.

Firstly, I found I was pre-thinking everything I have said around family, and especially my daughter and her partner. This is hard work for me (tactless, remember), and there any many things I would normally feel I should make a joke over (albeit tactless). On this occasion I think I managed not to put my foot in it, but it was hard work.

But the other side is that I actually found myself feeling slightly offended by couple of things, and then realising how silly it was to feel offended.

One was a person that clearly does not know there is more than just a "like" button on Facebook. With all others clicking "heart" or "sad" one person clicked "like" on a post about my grandson dying. I know he was just acknowledging it, and no way he actually meant he "liked" the news, but I did feel slightly cross.

Another was a slightly surreal email exchange. I got an email about something, and it had "Hope things are going fine." on the end. He had not heard. I replied, quoting his questions and answering and I quoted "Hope things are going fine." and followed with "Err, no.". In the next reply, he read my quote, ignored the "Err, no" and said "Things are going fine thanks". I just felt that was tactless and annoying. Again, I am sure it was not deliberate, but being tactless is not deliberate.

It is one of those rare times I have felt on the receiving end of tactlessness. I new experience really.

Anyway, update. I think we are starting to come to terms with it. Difficult times, and once again thanks for the the well wishes.

Why did I make a blog?

If you actually go back to the first post, you will see the initial reason for making a blog.

I was angry.

Simple as that - I was fuming at the stupidity of BT, or as I put it a lot "our favourite telco". There were so many things that wound me up. I was posting comments about this on the company status pages at the time, and that seemed rather unprofessional for the company to be doing. It is a sign that the company was growing up that I felt that way, and we are now 20 years old next month - definitely old enough to buy alcohol for the party. I called it RevK's Rants for that very reason.

What I have found is the the blog is a perfect medium for parking emotional issues. In general, I am not that emotional - well - I say that, but what I mean is that I am not the sort of person to burst in to tears over almost anything. I do get emotional - cross - angry - over anything I see as wrong or injustice - but crying my eyes out is not me... Or so I would have said.

What I found is that if I put my issues in a blog and told that small portion of the world that bothered to read my blog, I would feel I had "dealt" with the issue enough to calm down. Until I had a blog I would lose sleep over things, and now I do not (as much). If anyone needs therapy - I would suggest a blog. I suspect in times gone by it would have been a diary people would suggest, which is just a blog that nobody else reads!

Over time I found it also gave me a chance to show off some of the things I was doing. The 3D printing stuff is a lot of showing off. That was also therapeutic!

But I have also had the chance to be educational. That I find really rewarding, when I can explain something on the blog and people appreciate that I have done so.

Occasionally I can be controversial, and the comments can be very useful to help me understand what other people think. It works both ways.

Over all, the blog has been beneficial to my mood in many ways, and benefited some others too. I hope to continue to blog for years to come.

But today I find myself in a new state of mind, one I have not had before I do not think, one of extreme sadness. My eyes are sore from crying all day. And I wonder if posting this blog post will make even a small dent for me. It may not, but it is worth a try, and I hope it does. I hope it does not make my readers too sad. I am really sad for my daughter and her partner. This is where I am usually the last to have any empathy, but today I can think of little else.

Everyone we know, and the whole family have been very supportive of her.

It is all over this little man...

Thomas Arthur, born yesterday morning. He was doing well - a healthy baby boy until last night when he and his mum were rushed to another hospital in London. Doctors worked all night, and this morning he died.

A stillbirth would have been devastating, but this seems so much worse for all of us, and especially my daughter. There is nothing I can say and nothing I can do to make anything better for her. Only time can help, and then only so much. I am so proud of her.

He will be remembered, even though I never got to meet him myself.

Apologies for typos - but proof reading when you cannot see through tears is not easy.

P.S. Thank you all for messages via various media, I have not replied to them all as I don't know what to say, but thank you. It is the next day, and it was not just a bad dream, and I am at a loss as to what to do now.

Gearing up!

I really must stop looking at wikipedia - I read up on involute gears.

This time I did look on thingiverse first and there are loads of people that have published tools to make gears, but still, I really do feel the best way to learn is to do it yourself. It is not about re-inventing wheels (or gears) which I do a lot anyway, it is more about the process of understanding it by doing it. Published on thingiverse...

I ended up making a simple library to make the shape for involute gears. It is pretty simple, the shape of the gear is like the shape that would be made if you had a bit if string attached to the circle and unwound it. There it a great animation on that wikipedia site.

I had to take it further, of course. A general purpose gear module with loads of options...

A simple gear...

A chamfered edge - makes it easier to print...

Helical....

Double helix - herringbone/chevron (keeps gears aligned)...

With a hole...

With a recess for a hex bolt head (and a hole)...

With spokes...

Hollowed out with a base...

With a spindle in the middle...

Or pretty much any combination of the above.

But I did not stop there - I made a nice gear example module. Allowing some interlocking gear examples...

Why stop at 2? (I went up to 4 as it happens)...

And they work!

So more toys for my grandson to play with...

Getting screwed

For some reason I looked up the wikipedia page on screw threads, well, technically bolt threads, (here). As one may expect this led me to have some fun with 3D models Monday night. It was very educational.

I have made screw threads before on OpenSCAD - it is very powerful. The simple way to make a screw thread is to use the linear_extrude function to extrude a slightly offset circle with a twist setting.

The end result works well enough as a screw thread. For the internal thread in a nut you simply make the circle slightly bigger to give some clearance. However there are a few issues with this.

One is that the cross section of the thread is a sine wave. This is not really a problem, but it is far from conventional. Do look at the wiki page and see the difference. As I say, it works, and is very simple.

But I thought I would have a go at making a standard thread instead. My first idea was that instead of rotating a circle I would create a 2D shape that matches the cross section of a standard thread, and linear_extrude/twist that.

That, in itself, was not easy, I ended up using a lookup function to linear interpolate the radius for each angle: an arc at the higher radius; then an arc linearly reducing to the lower radius; then an arc at the lower radius; then a final arc linearly increasing back to the higher radius. Using lookup and a loop to create a set of 2D points for the polygon function I was able to create the correct shape, and use it with linear_extrude/twist, and it looked fine and worked.

However, I realised another problem with the whole idea of using linear_extrude. It comes down to how it works. It makes a number of slices (you can say how many) and each slice takes the 2D shape, and then a small increment above it, and twisted accordingly, the same shape, and it creates the sides to join them together, like this.

As you see, the triangles are used to make the edge between the two slices.

The issue is when the twist between two slices is large, like this, and each of the side triangles ends up stretched across a chord which cuts off the edge of the circle (or whatever) creating this sort of squeezed middle between the layers.

If you slice exactly at the layer height you are printing and it is all aligned, then this may not matter, but it does not make for a very general purpose model - even slightly off and you will print very different shapes and dimensions. So the only answer is to have a lot of slices so that the twist between each slice is small. When you realise that for a 1mm pitch screw thread that is 360 degrees per 1mm, you need a hell of a lot of slices. You also need a lot of points on the circle simply to ensure the rotation works. Millions of faces for a simple screw thread.

So I started again. I created a set of 3D points of the helix that is the bottom of the outer edge of the thread, then the next edge above it, and then the bottom of the inner edge, and then the top of the inner edge (I did not round the inside of the thread, as that is optional). I added a point for the centre top and bottom, and then used loops to create the triangles for the polyhedron function.

First the bottom, joining the centre to each point at the bottom of the thread.

But as you can see it is not joined to a solid face, so 4 more triangles needed.

Then the side, I need two triangles for each square(ish) section, so they are done in separate loops.

Then the second part of the squares.

Two more loops for the next part of the thread.

The next.

And then finally.

Now the top.

But we still have a hole, so 4 more triangles to seal it up and we have a fully contained solid shape.

And if you want a longer thread... Just keep going.

As you can see, this uses far fewer faces to make the shape, and can be sliced at any spacing you like. It is much quicker to create and makes for a smaller file.

Of course you need the internal thread too, within a nut, for example. In this case I take a clearance value to add extra spacing so that the model will work when 3D printed.

And there you have it, some complicated OpenSCAD, published on thingiverse.

P.S. As someone has commented, there are thread libraries on thingiverse, including some very odd ones, but this was more about education - learning how exactly to use OpenSCAD and the polygon and polyhedron functions like this.

Today's work

My achievement for the day is this small plastic clip...

It involved many iterations, probably at least 10. But finally it is a small plastic clip that attaches to the spool reels for ColorFabb plastic and allows me to "park" the filament when stored.

I find it amazing how much detail one can include in something so simple and small.

But at the end of the day it does the job and I have printed dozens now to sort the various reels of plastic I have. Such a small thing but it is today's achievement.

Meet your ISP

We are doing a thing, again, on 2nd April at our offices in Bracknell. We will be 20 years old on 1st April this year.

There will be food - there will be a BBQ.
There will be drink.
It will be fun.

Eventbrite tickets.

Resistance is futile

I mentioned previously that I designed some stacking number blocks for my grandson (on thingiverse here). He loves them, and knows the numbers and the order now. Technically you can sort of stack the 10 on top of the 6 instead of the 9, but not the right way up or even remotely "square", but otherwise it works very well - you can only stack in the right order, which is the idea!

Now I have some new plastic, I have been able to print them in the proper colours: Black, Brown, Red, Orange, Yellow, Green, Blue, Violet, Grey, and White.

Let the learning begin :-)