Being fair to A&A customers

Some people may be concerned over some of my fights with BT and what it all means.

• A&A offer, generally, an "Internet service" to our customers.
• We buy a "Broadband service" linking us to our customers from BT plc t/a BT Wholesale
• BT plc t/a BT Wholesale "buy"* a metallic path from BT plc t/a Openreach

At each stage there are gaps, and each entity is responsible for those gaps.

E.g. if the broadband link from us to you is fine, but the "Internet connection" is broken, it is up to us to fix that. There are some things beyond our control, sadly, but we are selling an Internet service and it is up to us to try and make that work.

When the problem is with something we buy, like BT plc t/a BT Wholesale's broadband link and backhaul, they are responsible for fixing that, and we are responsible for making sure they do that.

When BT plc t/a Openreach are failing to provide a working metallic path, then they are responsible for fixing that, and BT plc t/a BT Wholesale are responsible for chasing BT plc t/a Openreach, and we are responsible for chasing BT plc t/a BT Wholesale.

At each stage there is added value. BT plc t/a BT Wholesale do not just buy a metallic path, they also have modems and BRASs and backhaul links and all sorts to make it all work. They expect the metallic path to provide working broadband which is more than BT plc t/a Openreach sell them, so BT plc t/a BT Wholesale have to find a way of making that work, just as they have to make a BRAS work.

Similarly, we add value, and if our routers are not working, or our LNS's are not working or our transit providers are not providing the service we contract them to provide, we have the job of getting that working and chasing people that need to do that.

It is all layers, with each layer agreeing to provide a specific layer of service.

All we are doing is making sure each responsibility fits in the right place and has the right party paying - we would never expect our customer to pay for repairing a metallic path or a broadband service or an Internet service.

We might expect someone to pay if sent on a wild goose chase and incurring costs when it was in fact their own modem or PC or wifi that was actually at fault, which is why we work with end users to check these things first. But if everything checked an eliminated, there is no risk of costs, and the same should apply at each layer in this. That is all we ask.

At the moment, one of these "layers", the BT plc t/a BT Wholesale and the Talk Talk part are trying to absolve some responsibility for the "broadband" aspect of what they sell, and pass on BT plc t/a Openreach charges even when they are for fixing broadband issues and so not our responsibility. That is all that needs fixing here.

So, just like Virgin(!) we will not charge for fixing a fault in what we provide.

We just need our suppliers to not charge us for a fault in what they provide to us.

* I say "buy" as the idea of one legal entity buying from itself is strange, at best.

What comes after SFI2?

This is what we have asked BT plc t/a BT Wholesale (and similar to Talk Talk). Will be interesting to see what they say. Other ISPs may like to ask them the same question.

I have been going over this SFI2 issue somewhat and I think I see the flaw in the process. I hope this helps you, and I would be interested in your comments.

When we report a broadband issue, having eliminated end user wiring and equipment and checked dial tone, BT do various tests, including checking the metallic path is OK to SIN349 from the exchange end. I understand you have some pretty comprehensive testing systems for this.

If the metallic path is not OK, we can arrange an engineer and there is generally no issue with charges. This is not a problem.

However, if you do not find any issue with the metallic path, the next step is that you offer us an SFI2 visit. You do not let us take any other action at that point in the fault process.

This is the step that is broken!

An SFI2 visit is simply to test/ensure a line meets SIN349 - something the exchange tests have already done pretty reliably. It is totally pointless, and severely uneconomical, for us to repeat the tests already done for this from the exchange.

So, from now on, we'll be happy to accept BT's diagnostics and agree that the metallic path meets SIN349. What we need to know is what happens next? An SFI2 is clearly pointless as we both agree the SIN349 test would pass, so what is the next step in resolving a broadband fault within the SLG when it is not caused by a failure of the metallic path to meet SIN349?

And, specifically, how do we get your fault reporting past the insistence on booking a pointless SFI2 visit at that point?

We have a policy now of never disagreeing your SIN349 exchange test and so not booking any SFI2 engineers. So I need to know the next process step please, as a matter of urgency. After all, BT have put a time limit on fixing faults (the SLG) and we want to ensure we do our part to ensuring we follow the process to get the broadband service fixed.

So, what comes after/instead of an SFI2 visit when a line meets SIN349 please?

I look forward to your prompt reply.

I have a cunning plan

As usual, when BT make a change, we have to adapt.

At one point SFI was charged for "work done on end user equipment beyond the NTE", so we made sure no work was done.

Then they decided "a visual inspection of end user equipment" counted as work, making visits chargeable. That was some genius weaseling on their part! So we had end users actually hide their routers. They still had the cheek to charge!

SFI was an "optional extra service" which we refused to order, so they changed it to be "part of the fault repair process" hence us arguing that we should therefore only pay if we actually breached some term such as checking the equipment before reporting.

Now we have the latest that SFI2 is a product or service they offer and which checks the line to SIN349, and fixes (free) to that if it was not to SIN349, or maybe does other work if it does but is chargeable. So once again SFI2 is an optional service.

So the cunning plan...

• We are have got a JDSU, which is the test kit BT engineers use to test a line to SIN349. We hope we will not have to use it that often to prove the point here.
• When we get a fault near us, we will send someone (our staff) to test the line.
• If the line fails SIN349, we report a PSTN fault and get fixed
• If/when the line passes SIN349, we report the broadband fault

Now, at this point BT will, as always, offer an SFI2 engineer. However, we will have the justifiable reason to not take that offer as we will have tested the line to SIN349 already so do not need to pay for a service described as "The SFI2 visit simply checks whether a line is working within the specification of SIN 349". Job done, move on to next step in fixing fault!

However, we will have evidence in our line testing of a broadband fault and be well within our rights to insist BT fix the broadband fault, within the 40 hour SLG, and without using an SFI2 engineer.

If they refuse to fix the fault, county court claim.

We'll see how it goes, and what next crazy step BT come up with to stitch up ISPs after this one.

P.S. Same will apply to TalkTalk.

Update: The JDSU has a lot of useful tests which should help where we can use it, including ADSL and VDSL tests. Looking in detail at SIN349, which is literally a "Metallic path facility interface description" the tests are purely electrical characteristics, and most are tested by BT from their end for us. Indeed, we are not really able to test these without isolation and/or loop at the exchange end. We'll ask BT and TT how we can arrange open/loop at exchange end for independent testing and see what they say. Even so, the specification of SFI2 makes it very clear that it is simply not the appropriate "product" to consider as a means to address broadband faults so we will be back to "Not buying SFI2, are you refusing to fix the fault?" as before. Oh well.

Update: It seems that a SIN349 test needs an earth. If the end user does not allow use of power socket or other earth in premises a full SIN349 test cannot be done. If doing a SIN349 test is what we are paying for and the engineer cannot do that, well, then we won't have to pay, will we?! That could be another angle (assuming engineer does, as they usually do, go on to fix the problem).

I wonder if we need to get OFCOM involved

There is a serious issue with SFI charges, as I keep ranting on. This is because BT have pushed and pushed and pushed to stitch up ISPs, even when there is a genuine fault in the service they sell us.

The issue is not unique to BT "Wholesale" but also carriers like Talk Talk.

Both buy copper pairs from BT "Openreach", who have a monopoly on doing any work on that pair. They cannot send their own engineers to fix the copper pair to resolve broadband related issues, they have to ask BT "Openreach".

The problem is that BT "Openreach" offer "SFI2" as a service, and that is clearly defined as an engineer who will test a line to SIN349 (and fix if it does not meet it). The engineer might do more work to try and fix broadband issues, but might not, it seems. All he has to do is prove to SIN349.

This means that neither BT "Wholesale" nor Talk Talk have any way to get an "Openreach" engineer to fix broadband specific issues on the line. But they cannot send their own engineers either.

It seems that, from what I can see, there is no official way for a carrier offering broadband to get broadband issues fixed. All they offer us is BT "Openreach's" SFI2 service, for which they charge.

Surely Openreach need to offer this type of engineer, and surely the carriers need to absorb any cost for such engineering work on the basis that they sell a broadband service, and not a phone line, to ISPs like us.

The current situation means that nobody can properly offer a working broadband service using BT copper pairs - a situation that surely needs OFCOM intervention.

Or have I missed the bleeding obvious?

Some times BT just don't understand

We have a customer getting extra latency in the evenings, and noticeable slow down in performance (e.g. 20M on a 70M line).

It is clearly evening/residential congestion somewhere within BT's network. This is something we handle quite often. It could be cabinet level, exchange level, BRAS level or somewhere else. We only have one customer affected, so seems quite local to him. It also came on suddenly after an outage of a few seconds one day, suggesting something has been re-routed within BT to a congested link.

The excuses and replies from BT on this relatively simple issue are getting beyond a joke.

Bear in mind, they end emails with things like: Our vision for BT is to be dedicated to helping customers thrive in a changing world.  and BT Wholesale is a world-class enabler of converged network solutions.

• All BT tests completed ok, diagnostics show no fault condition.
• Right when tested (several times)
• Suggest sending an SFI (even though SFI will not test latency, will not go at the time there is latency, and will only test for phone line to SIN349 so will charge).
• Service cannot be provided due to line loss limitations;Broadband service cannot be maintained due to Line Loss limitations (even though this is a 70Mb/s line).
• Congestion is due to Internet (even though we check LCPs between use and modem, so only "internet" involved is BT's back haul network).
• Lift and shift to new port at cabinet cannot be done if no access to premises (even though they will happily install FTTC with no access to premises).
• Latency issues at peak times are beyond our control (even though it is their network).
• Several unexpected engineer visits to site with no idea why there were there.

As ever we will continue to battle this one, but it is a tad depressing that these issues cannot quickly be escalated to someone in BT that has the same basic understanding as our own front line technical staff.

Man cave going nowhere

This is getting silly.

We did have someone that was very good in sorting the kitchen last year, as a prime contractor, organising all of the necessary workmen quickly and reliably.

Sadly he has badly let us down converting the garage and we are making no progress at all. He did eventually get the builder in, to look and go away and come back with a quote ... except he has not come back with a quote! WTF?

I need someone to co-ordinate builder, electrician, gas engineer, plasterer, decorator, etc, and get this all sorted for me, as well as sorting building regs and inspection as needed.

Schrödinger's meme

I saw a cartoon today - it was very funny. A vet's waiting room, and a nurse talking to a waiting patient: "Mr Schrödinger, about your cat, I have good news, and bad news". It was really funny!

But it got me thinking. Marketing people still don't quite have the formula for what makes something go viral on the Internet, and it occurred to me that this is not a new phenomenon. It pre-dates the Internet rather a lot.

Schrödinger's cat is well know, but it dates back to 1935 in correspondence with Einstein. It was just a thought experiment. It was not some great breakthrough in physics in itself, but somehow that one meme has caught on, and gone viral, as it were.

Anyone even vaguely intellectual, and many not, will get that joke. Yet none of us, with possible exception of Sheldon, have any need for knowledge of disagreements on the basics of quantum physics in the 1930's. Yet we do know it!

So I wonder, is it true? that one of the key ingredients in something going "viral" on the Internet, or otherwise, is "cats", and especially "cats doing something silly"?

Did Schrödinger accidentally stumble on this marketing genius by mistake?

Well, I tried - GNU Terry Pratchett

Hi Adrian:

Again this year we had more April 1 submissions than we can publish;
we have selected a few for publication.

Having considered all the submissions carefully, we are unable to
publish your "Recommended explicit padding bytes used in Internet related protocols" submission this year.

However, thank-you for submitting it. 

http://www.me.uk/draft-kennard-padding.txt

SIN349

I had not realised this was quite the official stance before, but it was confirmed today.

If a BT SFI (Special Faults Investigation) engineer goes out for a broadband fault, and tests the line to SIN349 (which is a standard for the quality of the copper pair for voice), then even if he goes on and fixes a reported broadband issue, the visit will be charged by BT (Wholesale).

So that means the engineer found and fixed a broadband issue in the broadband service we buy and yet they charge for fixing it, and that is, it seems, official BT policy to do so.

They have pushed SFI step by step over the years getting more and more outrageous, but this is really pushing too far. They offer no other choice to fix a fault in the service we buy apart from booking an SFI engineer visit. It used to be that if they found and fixed the issue they did not charge, but now they will actually charge for that engineer to fix a broadband fault, even though we have paid for a working broadband service already.

Just to be crystal clear here - if there is an issue with the broadband, as opposed to the phone line itself (i.e. line meets SIN349), BT have no way for us to get that fixed other than an SFI engineer, for which they will charge in that case. They have no option to actually fix the broadband fault otherwise.

I don't think that would get past a judge somehow!

Hopefully we can work with BT to resolve this broken policy - we are working on that, and it may be a chance for our Wholesale Broadband Buyers Forum to get started.

What are other ISPs doing about being charged to fix a fault?

Ironic

I have had a damn song running around in my head.

OK that happens, but this has been for fucking months now, almost as long as this damn cold virus. Maybe it is related somehow!

Whilst at Mike's in Gloucester this weekend it even came on the radio. Arrg.

It is the "ironic" song, the one that is ironic because it lists a load of things that are supposedly ironic when they are not in fact ironic, making the song itself ironic.

"like rain, on your wedding day"

That is not ironic in any way at all. At best it is bad luck, but that depends on when and where you arranged your wedding. In some places and climates it would be 100% expected even.

The song annoys the hell out of me at the best of times.

Why the hell is it stuck in my head?

I hope my blog is a form of room 101 - somewhere to put this and be done with it forever... Please, no more of the song (or the virus).

Industry wide process (SFI)

So, once again, the same old issues all over again but this time with Talk Talk.

We had managed to previously get them to agree that they would not charge for an engineer unless it is proved that the cause of a fault is the end user equipment. We are generally happy with such a system because where there is proof given to us we can use that to justifiably charge our customer who will have claimed to checked/replaced his equipment as part of the process. (Obviously if we screwed up in the process, that is our cost).

It seems, once again, that things are not working well, possibly because we have a long list of outstanding disputes over SFI charges, and oddly TRC (Time Related Charges) too.

Once again it is "won't send an engineer unless you agree possible charges", and once again it is "charges is engineer fails to find fault in BT network". This last point is subtly (but importantly) different from "proving it is end user equipment at fault" as they are often reasons to "not find a fault in BT network" even when one exists (incompetence or bad luck or intermittent fault).

Just like we had with BT all those years ago, with Talk Talk, "SFI" (Special Faults Investigation) is a product they sell us to investigate a fault. As such we can choose not to buy such a product - why would we - we have investigated the fault ourselves already and found it to be in the network which we pay Talk Talk for already. All we want is the fault fixed, not some private investigator (SFI) to investigate it for us.

Of course this means essentially Talk Talk refuse to fix the fault, and we get deadlock again.

Their latest is that this is the industry wide processes. Of course I don't give a shit about that! For a start we have a contract with Talk Talk for a broadband service and we expect it to be fixed if there is a fault. It is no concern of ours if TT have engaged the services of some company that seems hell bent on ripping off the whole industry (just my opinion). But importantly the industry wide process is that every ISP disputes these crazy charges so much that BT have to create special processes on top of normal billing systems to manage the number of disputes (another industry wide process).

This really has to be sorted somehow.

Just to be clear (and this is where the lawyers reading may like to comment, unofficially). We consider it that the supplier has to provide a working service and repair faults if not working. We do accept that we have some role to play, checking the equipment, and eliminating it as the cause of the apparent fault. We recognise that if we fail to do what is agreed, and do not check the equipment, we should be liable for the wasted engineer visit that may ensue, as a pre-agreed penalty for a breach of contract. What we do not wish to do is purchase a new investigation service or pay for fault repair, or more importantly - pay if an engineer is unable to find a fault that exists.

It is quite unacceptable for the whole SFI system to encourage poor performance of engineers; to encourage the failure to find a fault.

So, once again, we are at a deadlock, until someone blinks. Only we have done this before and won the argument many times before. Let's see how it goes.

The real challenge is picking the fights that will not impact end users - they should not suffer from this, but if we do not fight it simply means us paying more and charging end users more. It is better to have a system that actually fixes faults and to work with BT and TT to do so.

P.S. We have offered to help Talk Talk break up this industry wide extortion racket process and try and fix things.

When Sky give you our number!

https://soundcloud.com/thrall-horde/2015031717453500005a

Someone called us and Alex answered and was remarkably helpful to the poor chap that called.

Why Sky would give out our number, I have no idea.

But well done Alex - only very minor comments on that, well done. And actually, the caller was really very good humoured on the whole thing. We wish him well with his issues.

Free repairs and servicing!

We got in to something of an interesting twitter debate with Virgin Media.

It all started when Alex asked Virgin what they meant by their latest advert that says "repairs and service included free". The reason to ask is that, well, of course repairs and service are included free, duh!

What was odd is the response, which is "we're the only service provider to provide completely free service & repairs so it is a bonus".

Now, that is a bold claim, and an odd one. Basically, anyone providing a service (such as broadband) has a legal obligation to ensure the service they provide is as described and working, so has to provide servicing and repairs of the service free (including in the price). Similarly if someone is renting equipment, either separately, or as part of a service, they would have to provide servicing and repairs in the price of the rental. Basically, Virgin's claims seems to be no more than "We do what we are legally required to do, the same as every other ISP" (my wording).

Even so, they continue to debate, and make another bold statement. Alex asked "Pls qualify what u mean by 'completely free repairs and service'?" and they replied "If, for example, any equipment in your house breaks then we'll fix or replace free of charge, no call-..."

A bold statement indeed, after all any equipment is quiet a lot. I asked if that means they offer a free laptop repair, and they confirmed that they only meant "our equipment".

So really, this bold claim in an advert really is "We do what we are legally required to do, the same as every other ISP", which is, in my opinion, a totally crap advertising claim.

Claims of being the only ISP offering this are, of course, totally crap... Well done Virgin marketing dept. Next you'll be claiming coax copper is actually glass fibre - oh! wait?

But well done engaging with us.

No one is actually dead until the ripples they cause in the world die away

There is a lot of code out there "on the wire".

If you follow Going Postal you will know the story of John Dearheart, a clacksman who was murdered. Like many who have died running the "clacks" (the discworld's version of a telegraph system) his name lives on. The message "GNU John Dearheart" continues. G means send on the message, N means not-logged, and U means return the message at the end of the line. This makes a datagram that continually bounces around the network.

IP as a protocol has a time-to-live (or hop-count) that stops a message going on forever, but for the names of those who should continue, we do not want a "time to live".

It seems that many systems and software around he world are finding GNU Terry Pratchett "on the wire".

Indeed, even the latest FireBrick alpha has a header, X-Clacks-Overhead: GNU Terry Pratchett in the software. His name will live on for a long time, buried in the code of the very Internet itself.

Update:

Rather than do this is a header in HTTP, which does add overhead, we have a different plan. It seems we had a slight issue with the code to handle small Ethernet packets (under 64 bytes) which ensures they are padded consistently (with NULL bytes). The internal operation of the FB2500/FB2700 was such that the last 4 bytes were not NULL. In practice this was not a leak of other buffers but an Ethernet checksum of an internal packet with a VLAN tag. However, the fix is to ensure the packets are sent with padding to 68 bytes before the VLAN tag is removed. OK, yes, this is technical.

However, padding can be anything, it does not have to be NULLs, so this is perhaps a far better place for this particular Easter Egg. On a zero payload IPv4 ICMP ping, we have 18 bytes to play with, which is one short for "GNU Terry Pratchett", but with one space removed, it aligns nicely on a packet dump as follows. In fact, ARP replies have exactly the same amount of space...

Now, yes, as one person has said, this is silly, but the actual content of the padding really does not matter at all, and can be anything. There is no harm, or waste in it being this string - so why not?

I'd like to get this in to an RFC http://www.me.uk/draft-kennard-padding.txt and I have emailed to the auditors of reality (I mean RFC editors) today (17th). P.S. I was considered but rejected, sorry.

THE ONE TIME WHEN TYPING IN ALL CAPS IS NOT SHOUTING

Oook

iTunes down

All day, today, iTunes has been down, and even though Apple say "Customers may be unable to make purchases from the App Store, iTunes Store, iBooks Store, or Mac App Store." in practice I cannot even view films I have already purchased.

I have never been in to the "tunes" side of iTunes, but I have purchased some TV series which I have been watching back to back on my computer and TV (Apple TV connected). It is funny having a full resolution SD or even HD window on a 5k monitor - it looks so small! It is nice to have in background. I have been watching old Dr Who, Stargate, even some Red Dwarf. iTunes just works (well until today).

But this is DRM stricking back - the very fact I have paid for these means I am hampered by today's outage. If I had simply pirated them I would have none of this. Indeed, you loose any warnings and adverts on videos if you pirate (though, to be fair, you don't get those on iTunes).

This did raise an interesting point though. Why don't I just pirate download the next episode and watch it? Would that be illegal or what?

Well, I am not a lawyer but...

1. If I could find somewhere to stream it then I would simply not be breaching any copyright. A court case determined that accessing web pages (and by implication steaming stuff) is not a violation of copyright. The analogy is someone reading a book. Reading a dodgy illegal photocopy of a book is not a breach of copyright by the reader (the person making the copy or distributing is breaching copyright).

2. If I download a pirate film though, one I have paid for, my understanding is that is at most a civil wrong and means I will possibly have a liability to be sued for damages for doing so.

But what damages? FFS I have paid to have a copy of that episode already so there are no damages. In fact the hassle, time, and cost of finding and downloading a pirate copy when I should just be able to just watch what I own already might make for a counter claim...

Of course there is a huge problem for the copyright troll industry! If my IP is found downloading something, they have no way to tell if it is something I have paid for. So they might try all the means they have (including some things in legislation) to go after me, and all they do is incur me costs which I can ultimately (hopefully) claim when I win the case against me.

What fun... TBH, I have only just realised it is not Thursday!

Update: Yay - pirated!

Naked DSL

There is some debate on the exact meaning of "Naked DSL". Some simply consider it is DSL without a phone service or dial tone. Others consider it has to be using a copper pair that only connects to the DSLAM and not to any voice equipment of any sort. What it means if connected only to an MSAN that can do voice but has the voice turned off, is not clear. Some consider it simply a billing matter - not buying a phone service.

The basic issue comes about because broadband service is an add-on to a phone line, and so to get broadband you need to also have a phone line (from same or different supplier).

Many people do not want to have to buy a phone line - they may use mobiles, VoIP, or even have ISDN lines still but simply do not need a phone line and are somewhat aggrieved at having to pay for something they do not actually want (phone service) in order to get something that they do want (DSL).

So the idea is that naked DSL is simply buying ADSL/VDSL with no phone service involved.

However, DSL does still need a copper pair.

Now, one way this may be achievable is a new service BT are considering which appears to be a connection to the FTTC cabinet and not a phone service. I think it may still have a pair back to the exchange for testing, it is not clear. In any case, if and when BT do offer this, we'll be trying it out and selling it, especially if any cheaper.

At the other end of the spectrum is something A&A have done for a while which is offer a no calls phone line with the broadband service. But this is buying a phone line from us, and has a dial tone. We even allow free calls to be made and incoming calls. Something like a quarter of our broadband services are provided with a phone line in this way.

The problem is, for us, is that selling a crippled phone line service is not nice - people often want calls on it, now that they have it. We are not really in to selling phone lines at this stage, and so what we really want is something closer to selling naked DSL.

So, to start with, we are stopping calling it a "phone line" - we are now calling it a "dedicated broadband only copper pair". We are also removing the dial tone and not allowing calls in or out at all. Don't worry, existing lines won't be changed, just new services.

Now, we had something like this before, when we first started - i.e. a line with no dial tone. It caused a slight snag in that a silent copper pair is liable to get nicked by a lazy engineer installing someone else's line or fixing a fault somewhere. It is not supposed to happen, but we have seen it many times. This is why we now ensure the line has a dial tone. Even so, we have had cases of pairs nicked because the engineer tried calling 123 (speaking clock, and chargeable call) and when that failed he assumed it was not a live service, and nicked the pair!

To solve this we think we have finally found a way to include an audio message on the line which says it is a dedicated broadband only copper pair, and even goes on to do a quiet line test (a useful diagnostic when there is a fault). No dial tone. No calls in or out. But it does have audio to tell the engineer not to nick the pair!

The end result is something pretty close to "naked DSL" at last, a way of having DSL without a phone service. On Home::1 we only charge £10/month more for it, which is not bad for the copper pair, and no messing with a minimum term, etc.

We're testing this new system this week and hope to have it for new lines very soon. At the same time we have reduced the install price for new copper pairs to closer to the cost price. We see the copper pair simply as a way of getting the broadband business.

Of course it totally breaks OFCOM's new broadband / line migration rules, but that is another matter. I'm sure we'll sort that out in due course.

Update: Given some of the comments, it is worth explaining that, as ever, we can be a bit flexible on this.

4G dongles

Dongles are a pain in the arse!

The problem is that they do not really follow much of a standard.

The 3G dongles we support in the FireBrick all appear as a "file system" initially so that one can install windows drivers, and then magically change to some sort of serial interface.

The way the change varies - some "eject CD" on the file system, and some do something else. At least one does a USB profile setting (the right way to do it, arguably) and one is just strange.

For almost all you then end up with what looks at a USB level like a serial port. What you do then is talk old fashioned Hayes modem dialling and set up commands, and then serial PPP coding. It really is that daft.

The dongle is faking all of this, it actually talks a "PDP context" on the air, having pretended to talk LCP and PAP/CHAP auth and even IPCP on the serial interface. Eventually it connects and passes IP packets over PPP over serial (with escaping characters).

The new 4G dongles appear to be different - they seem to appear as an Ethernet NIC.

But we have yet to get to the bottom of it - do they talk faked PPPoE or what. That is the next challenge. We'll see.

Sales dropping - increase prices!

We (A&A) are putting up the install price of the Office::1 package from £300 to £500 from 1st April. If you want the lower price, order now!

This is mainly down to the cost of setting up two phone lines, with DSL or FTTC, modems, and a FireBrick, but to be honest I think it is just way too low.

The Office::1 package really is a very good alternative to a "leased line" for small/medium (or even large) offices. People should be comparing to "lines" that cost at least £3k to install!

So I am not sure we are really capturing the market as we are a bit too cheap, to be honest. We put a lot of work in to Office::1 as a premium multiple line package with bonding and fallback.

I may be able to lower some ongoing prices, but make 200G the entry level too. This won't affect existing 100GB customers, obviously, but it is an idea. I'll contemplate that.

It is an odd marketing thing - put prices up to get the market that you want. But I think it really applies to this sort of package.

Anyway, it is available at the lower cost for the rest of the month.

I have also tweaked new Home::1 installs - partly in preparation for the wires-only FTTC install and partly because we have been throwing away money on simultaneous FTTC and phone line. It used to be cheaper (same cost as ADSL and phone line) but BT quietly dropped that discounted price from the price list about a year ago and we have been handing out discounts for a while. Not good.

I hope to also be able to lower the included phone line install price when getting "naked DSL" where we do the phone line only for broadband. That may be later this month.

Juggling as ever, and BT pricing is not an easy thing to juggle.

P.S. Sales of Office::1 are not actually dropping, just thought the title would be fun.

A&A and PGP

We use PGP a lot already but I think we can do more - and I am interested in views on what we need to do.

For a start, I think all automated and even staff emails (e.g. ticket replies) from the company should be signed. We already have a system of staff keys signed by the company (which I control personally as owner/director).

At present accounting emails are signed, and some staff emails are but most are not.

But we need to go further.

My thought is that we need a way for customers to register that they want encrypted emails as standard and register key and email address.

Then, all automated and ticket manages emails should not only sign emails but encrypt, even including things like call recordings from our VoIP systems.

I suspect we need to work out the top level registration system first, then we can work system by system to ensure emails signed and, where requested, encrypted.

But ideas welcome.

The future of 999

We have excellent emergency services in the UK, but technology is moving on and the way we contact them really has to move with it.

Interestingly the ability to SMS 999 now exists, if you are registered.

For a long time the way to contact emergency services is by calling 999 (or 112) from a landline phone. The location of the installation is automatically made available by the telco (mainly BT). Even with the next step - mobile - there is some basic cell location data passed. But ultimately this is still just a normal phone call.

But times change - and I have had some rather interesting discussions with the people that work on this sort of thing (NICC) and agree with national and international agreement on the way things should work.

The discussions (to which I was invited, thanks), started off rather badly. I have to find out where they have got to now and see if sanity has prevailed. I'll post more if I find out more. But when I left them, the discussions where around handling location data on VoIP calls.

This rather loses the plot a tad - the plans all centred around having ways to have emergency services get end point IP details and have a way to translate that to a location via APIs in to the ISPs. I.e. they wanted some proper and quick way to turn an IP address in to a real geographic address. They were really dead against user device supplied location data?!

This has all sorts of problems:-

1. Technical - it is not always possible to turn an IP in to a location, and even where it is, there is no way to know that it is the end of the line as anyone can operate tunnels and relays and phone systems that mean the "IP you can see" is not the end of the link to the person talking.
2. Even where it may be possible, it is not always possible in one step (which is what they want, and even want very low time scales of a few seconds). I.e. one may turn an IP in to a BT DSL circuit ID, which means asking BT where that is. Or to another IP because it is a tunnel service. 
3. Even that only makes sense where the services for such are within ISPs and not individuals or companies who will not be providing some API to allow tracing IPs through corporate or domestic networks.
4. CGNAT makes that even harder - exact timing, IP and ports, both ends, makes it hard to trace the original link.
5. If an ISP has to get the location from someone else, another ISP for a tunnel endpoint - why would other ISP provide an API for that (as opposed to an API for emergency services use)?
6. What if the user is not even in the UK, or *any* step is via an ISP not in the UK even though the user is?
7. Why would an ISP not providing VoIP/telephony, and so not subject to the legislation on 999 location data, want to comply?
8. If such an API existed, even at a basic level, for common major ISPs even those working through CGNAT, you create all sorts of illegal and legal issues. Hackers get location from IP. Advertisers want location from IP. Copyright holders want location from IP. Before you know it, the existence of an API means people will demand access to it for god knows what.
9. There are a lot of cases where the IP is mobile and tracing it then gets complex - wifi or mobile data.

Now, I think all of this is missing the point massively.

We should not even be talking of VoIP calls to 999. It is almost never that a VoIP platform using some data link does not have either a "normal mobile" or a "physical landline" behind it which can be used for the 999 anyway. It does happen - at home I have glass and no phone lines at all - but we would call from a mobile if needed. So what we need is for any attempt to use 999 to revert to such old fashioned means. Some VoIP/DECT phones do this - having a landline connection as well, and calling 999 does not use VoIP but uses landline. Any VoIP hosted on a mobile phone device can (and should) revert 999 to the underlying GSM network.

But with so many people using mobile and wifi and data rather than conventional landline, we need something new. This is especially new when you have newer means to connect and communicate.

What do we need?

Well, obviously traditional landline and mobile calling 999 need to work as you expect. The last resort. That is in place and works fine.

But what we need is a proper published IP (i.e. data not phone call) based API for emergency services, and it needs to allow end user reporting of location, voice over IP, video, images, etc. It may even be sensible for personal health monitoring wrist bands and the like to be passed to the emergency services in real time.

But this needs not only an API but the back end to get all of that data to the services that need it. It would be massively helpful to have real time accurate GPS reporting to police. It would be massively helpful for Ambulance service call handling people to have access to video and health monitoring data. It would help anyone responding to an RTA to have video and image data in advance.

Ironically, such ancillary information can currently be gleaned from twitter and Facebook faster than any official API!

Once the API is defined (and please, not defined by committee) we need standard iPhone and Android apps that just use it as well as PC/Mac apps.

Then the whole issue of VoIP calling 999 will be moot.

I can FaceTime my mum - why can I not FaceTime an a hospital?

OFCOM above the law?

Well, making progress...

My argument is simple - OFCOM and the TPS failed to meet the requirements or 26(2A) of the PECR. These state a simple requirement...

“(2A) Where a number allocated to a corporate subscriber is listed in the register maintained under paragraph (1), OFCOM shall, within the period of 28 days following each anniversary of the date of that number being first listed in the register, send to the subscriber a written reminder that the number is listed in the register.”

This is pretty clear, and an OFCOM letter confirms :-

"As explained in our letter of 3rd June 2014, your TPS registration was automatically renewed

under a particular arrangement which TPS set up in order to address your specific concerns
with the standard process for annual reminders to corporate subscribers. It follows that you
would not receive such a reminder, which was actually sent to an internal TPS to effect
automatic renewal."

So OFCOM admit that no reminder was sent, which makes OFCOM in breach of 26(2A) of the PECR.

But oddly the latest is that OFCOM state :-

"For the reasons set out in previous correspondence (and particularly the letter to you dated June 3rd 2014 from Sukh Walia-Chahil) Ofcom does not accept that there has been a breach of the Privacy and Electronic Communications Regulations 2003."

So they are in breach, their own letters admit it, but they deny it. Even so, they are liable to pay my costs for the breach. The PECR says so.

Next step? I sue OFCOM or, for more fun, I take them to the Parliamentary and Healthcare Ombudsman first. I may do that first and see what happens.

The facts are crystal clear.

OFCOM ACTED UNLAWFULLY and confirmed this and I AM ENTITLED TO COMPENSATION for my costs/losses as a result.

How simple can it be?

Latest steps are I have asked OFCOM two simple questions :-

1. Does the PECR *require* OFCOM to send a reminder of our numbers held on the register each year?

2. Did OFCOM (or even TPS) send such a reminder last year?

Awaiting reply.

Helping BT

Just to counter the idea that we are always shouting at BT it is worth explaining that we do help them as well!

Today we have been helping them diagnose a link fault. We have seen a BRAS that is not working for a load of lines, but only to one of our LNSs. We have found another BRAS with the same issue from the same time but to a different LNS. We change the IP of the LNS and the lines start working (albeit on the same box).

Some time ago we set up alternative IP addresses on each LNS because of an occasional fault condition that can happen where a Link Aggregation Group develops some sort of fault on one part of the group. The typical way a LAG works is that the specific link used for any traffic is based on a hash of the traffic (IP addresses each end. maybe port numbers, and even MAC addresses). The idea is that a "flow" of traffic ends up on the same link avoiding any reordering issues, but load gets reasonably well shared out.

The impact of a link failure on a LAG is that specific combinations of IP addresses at each end will go via the faulty link, meaning that from a specific BRAS to a specific LNS does not work.

We have improved our system of managing alternative IP addresses now so that we are able to quickly switch addresses more easily to allow customers to get on line when we have this sort of issue.

But today I, and Shaun at the office, have been on calls with senior network engineers in BT (they called us!) to help them out, and understand where the problems are on what IPs and what BRAS so they can try and locate the underlying problem.

We could just say "It's broken, fix it! fix it! fix it!", but no - we are working with BT to help resolve issues and provide as much information as possible. We know these things can be a pig to find and the more accurate data you can get, the better chance of finding it.

And what do we charge for this valuable and special fault investigation service - bugger all!

Banging head against wall (BT)

We pretty much have to employ someone full time to handle disputes now, it is crazy.

Today Alex, myself and Stuart were reviewing the disputes on one of the latest bills. There are half a dozen still left after we have disputed charges twice. But I really do despair over this.

These are disputes that have supposedly been looked at TWICE by a real person who has responded saying that we are wrong and the dispute is still valid.

One of them has some really detailed engineering notes:-

Engineer notes: On arrival I was met by Adam who represented the service provider as this issue has been on going. I was told we have dial tone. I couldnt perform a Pq as there was no earth. I also had sync on my tester, but Adam could nt see the sync on his laptop. I had called wholesale and spoke to mark, and they couldnt see my tester in sync either. He confirmed that the mapping is incorrect. The llux and lluy on out records show its on ports 24 when infact it should be on 26. Mark said he will update the routing as I couldnt. I then went to the exchange and had a c0-op to get it changed to pair 26 which we did. Went back to ur and demoed it to Adam who was delighted as he was able to access the router. They test head states fault on ex nudge when infact its ok as its junpered to the correct ports now.

Now, this really is detailed (ignoring typos, which I am happy to do, and even though Adam did not represent us), and I have to congratulate the engineer on such good notes and for getting to the bottom of the problem and fixing it. It explains that BT Wholesale had the records wrong and that was the problem - the line was on the wrong ports at the exchange. There is no doubt whatsoever that this was a fault in BT and that the customer equipment was not in any way at fault. Obviously we disputed the charge for this...

The first and second response from BT to this dispute is :-

"Engineer notes clearly states as issue with the customer equipment. Hence, charges are valid"

So we are basically banging our head against a brick wall here - how on earth do these people get dressed in the morning?

Also, several of the disputes are TRC (Time Related Charges) for an engineer visit (about £90 rather than the usual £144 for an SFI). Now, to try and explain this you have to understand the usual logic :-

1. Customer reports an issue to us, and we can see it as well on our monitoring.
2. We get customer to go through checks, including changing all his equipment.
3. We run BT diagnostic tests which come back saying that they cannot see a fault, but if we would like to book an SFI (Special Faults Investigation) we can.
4. SFI engineer goes out, typically does not find or fix the fault.
5. BT charge us.

OK some times engineer finds and fixes the fault, but the principle here is that we say there is a fault, and BT say there is not one, so they offer an investigation service to resolve this disagreement. If that finds it was a BT fault we do not pay, if not then we have to pay. That is the logic.

For a start this logic is very flawed :-

1. BT's tests are not testing the service as provided at the delivery point (master socket) so they are inherently unable to determine accurately whether there is a fault or not.
2. It is in BT's interests to find no fault and offer an SFI as an SFI makes them money.
3. It is in BT's interests for the engineer to fail to find or fix a fault as then they can then charge (even if a later engineer does find and fix the fault!)
4. BT have a monopoly on working their side of the NTE so there can be no competition to address the huge commercial interest bias in steps 2 and 3.

Anyway, putting that aside for a second, just occasionally, and in the case of these specific disputes, we have a slightly different sequence of events:-

1. Customer reports an issue to us, and we can see it as well on our monitoring.
2. We get customer to go through checks, including changing all his equipment.
3. We run BT diagnostic tests which come back saying "report fault to BTW to investigate" (i.e. they are agreeing that something is hinky).
4. We report the fault, and BT come back and say "Please book an engineer appointment with your End User" (no mention of SFI)
5. We arrange an appointment
6. Engineer arrives and fails to find or fix fault
7. BT charge us "Time Related Charges" for his time.

This is odd on two fronts. For a start, Time Related Charges (TRCs) always have to be pre-approved in all work with BT, and we don't approve such. But also, in this case, BT agreed there was a problem, and BT asked us to book the engineer so there is no way in hell we are going to pay BT for that!

Sadly, BT have returned the disputes twice now saying TRC charges apply, and the next step is solicitors and court!

This is, of course, one reason we have started wbbf.uk.

Update: Whilst we are chasing the rest of the issues - the really stupid one at the start of this post has now been "waived" (not a good word, as it implies they are voluntarily giving up some right rather than correcting a mistake). They have not really explained why we had to dispute it so many times, but suggest it was because they cannot see the notes after 28 days. Odd really, as the notes still show on eco+ even a year afterward a fault, and they are included in the dispute we sent, but what is really odd with this excuse is that their reply to the dispute references the engineer notes - or are they admitting that they just make this shit up I wonder? We've asked for more details of exactly what they person who sent the response was referencing. Should be fun.

The ICO do not record when someone makes a request under section 32 of the PECR

That's right.

There is legislation that allows someone to make a request for them to exercise their enforcement functions, and they do not even count when someone makes such a request.

It is bad enough that they have no form on their site to allow one to make such a request, but this is just crazy.

How much more inept can you get?

See FOI request.

I have made 3113 separate reports and requests under section 32 now, and am still ignored, so next step is the Parliamentary and Health Services Ombudsman.

Net Neutrality

A small clip from The Oatmeal comic
(which I hope counts as fair use)

This is a complex one.

Firstly, to explain to those that are not technical, this is about settling a battle between ISPs and content providers that has been waged for some years. The Oatmeal try and explain it in very easy to understand terms (here).

As an ISP I am very keen on the principles that we "just shift packets", we don't care what people do with the service, and have no reason to - we are neutral and impartial. We peer with anyone, and we buy transit that connects to anyone neutrally.

However, you could have crazy situations where a major content provider pays an ISP for preferential access to its network, either to be the service the ISPs customers prefer (as it works better) or because the ISP is threatening to stifle the traffic if they are not paid. In some places this has happened.

So the US has just made some slightly controversial rulings on this making Internet have to work as a neutral carrier. The same stuff is happening in the EU.

"The EU Council of Minister has finalised its position on net neutrality and roaming. The Council will now begin trilogue meetings with the European Parliament and the European Commission (The Council and Parliament need to agree a text before it can become EU law)."

Unfortunately the idea of net neutrality has some issues.

So let's look at some of the problems I can see so far :-

"Providers should not limit the rights of end-users to: access and distribute information and content, use and provide applications and services and use terminal equipment of their choice. This is irrespective of the end-user’s or provider’s location or the location, origin or destination of the service, information or content (agreements between providers and end-users in relation to commercial or technical conditions, e.g. price, volume and speed are allowed)."

This is good - it will be interesting to see how BT's wires only FTTC will work with this as BT are currently limiting the VDSL modems that can be used on that service. Not all ISPs give out the login details for lines if you want to use your own terminal equipment.

But this also has implications where users might want to run email servers or use email servers or DNS not provided by the ISP. At A&A we have no issues, and allow anything, but some ISPs have some basic filtering as standard.

"Providers can offer non-internet access services that require a specific level of quality, as long as there is sufficient network capacity available so that the availability and quality of internet access services for other end-users are not impaired in a material manner."

Sounds sensible. Can't see immediate holes in that.

"When providing internet access services, providers shall equally treat equivalent types of traffic"

Again, sounds pretty sensible so far.

"Traffic management measures may be implemented to: comply with legal obligations (laid down in EU or national legislation); preserve the integrity and security of the network, services provided via the network or end-users’ terminal equipment; prevent pending network congestion and mitigate the effects of exceptional or temporary network congestion (provided that equivalent types of traffic are treated equally); or comply with an explicit request from the end-user, in order to prevent transmission of unsolicited communication or to implement parental control measures. When implementing traffic management measures, providers shall not block, slow down, alter, degrade or discriminate against specific content, applications or services, except as necessary and only for as long as necessary, to achieve one of the stated purposes"

This does get fun...

1. Obviously this makes IPv6 mandatory, else discriminating against specific content (e.g. IPv6 only web sites). That is good news but not what they meant I am sure.
2. This also outlaws IWF (child abuse image) filtering, as there is no legal requirement to filter such. My main objections to that have always been that it is the thin end of the wedge and ineffective at stopping child abuse. Though this could be provided as part of parental controls.
3. This outlaws blocking of extremist material or anything else the government wants to block by the back door. They would have to actually pass laws, which is much more controversial. Again, this can be part of optional parental controls.
4. This also outlaws all of the default filtering that some ISPs are doing as there is a requirement for explicit request from the end user.
5. "End user" may not be same as "account holder" - it will depend on definitions. This could create problems where the account holder has asked for parental controls but the end user has not!

So, mostly this is good...

Now, I appreciate some of my comments may be a bit silly, I know, but if this ends up in actual law, it could be quite complex to define some of these things and difficult to be compliant. We have seen how badly worded laws end up with stupid convictions.

But can it go wrong?

You even have daft things like, obviously, our customers get good access to our email and VoIP servers as they are on the same network, but access to other networks email and VoIP services may not always be as good - simply due to the practical fact that they are connected by shared transit and peering links. Will that be seen as us as an ISP giving preferential treatment of traffic for us as a provider of email and VoIP? Will we have to move our servers off network to ensure they are treated equally?

I think we will need to watch the wording of this at EU and UK law level very carefully.

Net neutrality is great in principle but the details have to be right.

Mornings

I was pondering the meaning of life, as I do, or rather more specifically the strange way that some things bother me and some do not with no rational reason why.

Like most people, I am sure, in the morning I brush my teeth, shave, get dressed, and in my case have to take a few tablets and an insulin injection. But this daily routine is not without some engineering considerations. For example, I have a glass of water with a fizzy vitamin tablet with which I wash down the tablets I take. So I ensure I fill the glass with water and put in the tablet after brushing my teeth - that is because, by then, the cold water is running nicely cold, and that is nicer to drink (I have mains drinking water in the bathroom). But I don't do it later as the tablet takes a couple of minutes to dissolve and I would be waiting for it when taking my tablets. So a bit of critical path analysis involved in ensuring I am not waiting an extra 10 seconds watching something go fizz in a glass.

I ponder why I am annoyed by a few seconds waiting for a tablet to dissolve. It does not matter. The whole process is not critical - I do not have a set time to get to work, and I get there hours before anyone else most days. What seems most inconsistent is that this whole highly efficient daily routine often follows an indeterminate time spent relaxing in a hot bath. That is also time doing nothing but which does not annoy me.

I can only conclude that people are strange, or that at least I am!

These go to 11

Bit of a general drunken rant - "my personal opinion" even more so...

I have had a few concerns over advertising over the last months - I am annoyed by most adverts, and even more so when they are about something I understand (broadband service), but of late there has been a spate of "race the the bottom" stuff.

We are seeing people selling "Free broadband for 6 months", which is disconcerting. For a start, it is far from "Free", this is always with a "with phone line at £XXX", and some times that is quite a lot for a landline! Bear in mind we do a landline for broadband only use for £10/month.

But the big issue I have been trying to work out is how we can be different, or more to the point explain how we are different. The problem is that we try to be very honest with what we say, and that sometimes puts us at a disadvantage. We'd love to make claims that we are "better" than many other ISPs, but if we cannot say so with a definite measurable metric then we won't say it. Honesty is pretty important to us.

We gather that we are the highest average per-line peak usage of BT wholesale broadband, at least at some point when that was measured, but we have no proof of that (BT don't say that officially). But what that does mean is that "trying not to be the bottleneck" is indeed what we are managing - we are investing a lot in backhaul bandwidth to ensure people get what they pay for.

One of the big issues is that any "shared" service cannot guarantee the peak usage to everyone all the time. We cannot. What we can do is try very hard to ensure everyone can use what they need or want to use when they want. We can try not to be the bottleneck.

So I am unsure how we market ourselves. How we make truthful statements and good marketing statements, at the same time, in this industry.

We have has some lovely reviews! Support: As Nigel Tufnel (Spinal Tap) said: "These go to 11"

A lot of what we can say relates to when things go wrong. We know we are tenacious at beating up BT and TalkTalk over backhaul congestion and line faults and ensuring there are no charges when an engineer has to fix a fault in the network. But selling the "we do well when things break" angle is hard as it is saying "things break"! Sadly they do.

We monitor loss and latency on every line every seconds, and to this day BT do not accept a packet loss or latency issue as a fault category (in spite of a lot of pressure from us).

Whenever I personally get involved in any fault handling on any line (at weekends!) I almost always get very annoyed. I dispare despair at what my staff have to face every day. Many times we have managed to get carriers to change systems and improve things. Sadly there is an ingrained policy of not fixing the underlying problem in the likes of BT, but instead making extra layers to add on top. The latest, with so many ISPs disputing SFI charges does not result in any fix to the issue but a whole new process for the management of the disputes ISPs raise. It is layer on layer of sticking plaster and no real progress. In the past we automated re-tests of a fault, so they changed the system to require an SFI appointment on any failed re-test! They fight us at each step instead of working with us to actually make things better.

Some how we need to find a way to explain to people that we are actually quite a good ISP and worth every penny, but to do so in honest statements that we can back with facts and tests and proof.

I wonder if I need a marketing strategy that is more than "If you build it, they will come" :-)

We'll have to have a meeting to figure it out (thanks xkcd)