The fastest BT FTTC services used to be advertised as "up to 80M" (I think), and now it is advertised as "up to 76M". Why?
The answer is new rules that say that ISPs have to show that the speeds they quote can be achieved by at least 10% of customers.
Why is this? and has this new rule helped?
I have trouble understanding why this was an issue. I, personally, never had any issue with use of "up to". It is clear to me that it means a maximum. It means, in essence, that I would not get faster than that speed but could get anything that is slower, possibly much slower. After all, when we had 56k modems, did anyone ever see one sync at 56k? I never did, but did ASA get involved, no!
It seems that people would buy services "up to 24M" or some such, and then complain when they got 23M, or got 2M or whatever. The fact that some, indeed many, got the 24M, did not matter, as they "did not get what was advertised". I have actually heard people say that about their broadband (not with us), and get very puzzled when I try to explain what "up to" means, and that as long as what they got "was not faster" then they are getting the speed that was advertised. But then I never understand why I have to explain thermostats to people, either!
Surely the solution to this is education. To explain what "up to" means so that people don't misunderstand. There have been a load of sensible moves to ensure ISPs provide availability checks that give realistic ideas of speeds likely to be achieved. They even went as far as insisting a range of speeds is quoted. That is all sensible, and managing expectations. It does not help with BT charging ISPs for this data, but it is good for educating customers.
So, this move has meant, for example, the BT adverts I see on my blog saying "up to 76M" instead of saying "up to 80M". Clearly, assuming they picked this based on these new rules, that means 90% of people will get less than the advertised speed. So instead of being a nice round number and a reflection of the technology and the service behind it, it is now a less easy number that is slightly lower. Does that help? Also, all ISPs are affected, so it does not really make a comparison between ISPs any better. It is also true that the "feel" of an internet access technology will depend on a lot more than the raw speed of the "tail" link. It will depend on a lot of factors including stuff "in the Internet" and not down to the ISP at all. There are differences between technologies, such as the way DSL works, the way satellite links work, and the way cable services work, which have a major impact on the way the service feels, but are nothing to do with headline speed. These are what you really need when comparing ISPs using different technologies. These new rules do not help a jot with that.
So all this does, in theory, is reduce complaints to ASA by 10%. Does that actually help consumers? I think not. I think it makes matters worse.
What do A&A do? We don't say what speed the line can go up to. For a long time we used "not faster than", but we don't even say that now. In a technical description we do explain that FTTC technology (VDSL) can, in theory, do over 100Mb/s, but that *no* customers can get that as the service is available capped to either 80M or 40M. These caps are not variable or a grey area - e.g. everyone (who can get FTTC) can get a service with a 80M cap. Their service might be 30M or 20M (see the checker!), but the cap we are applying is 80M, not "up to 80M". We're not saying "up to" at all, we're saying "look at the checker for your number". So this means we have an 80/20M capped FTTC service which looks better than BTs "up to 76M" service, even though the last mile technology is exactly the same and a specific customer may only get 30M on either service. Madness. I am not going to say "up to 76M" on a service where any customer could get more than 76M as it would be a lie, and I don't lie in adverts.
Given how irrational the original complaint of "not getting the speed advertised" was, I wonder if we can start a campaign of new complaints. When someone buying an "up to 76M" service gets 80M, or anything over 76M, they should complain to ASA. The basis of the complaint? Well, that the advert is clearly wrong and clearly a lie as they get more than the "up to" rate stated. So if the advert is a lie, what speed are others getting. Is a customer that is getting 80M losing out because others are getting 90M or 100M. After all, the "up to" was a lie, so what else is a lie in the advert? It is totally irrational, but no more so than the original complaints, IMHO.
Anyway, just my opinion.
2012-11-30
2012-11-29
Correct Horse Battery Staple
As some of you may know (I blogged, a year ago), we generate easy to remember, but long, passwords for various systems. The entropy in these is high (see xkcd/936 for details). The beauty of them is that they are easy to remember as they are constructed from real words.
We have some nice long word lists and have even organised them as adjectives followed by nouns to make even easier to remember. The word lists are longer than those suggested by Randall. But even if you had the exact word lists we use you still have very high entropy in the passwords. We use a true random number generator even (because we can).
There is, however, a problem, a side effect Randall did not explain. It is not just rude words (which are not too hard to eliminate from the word list) but word combinations that can be offensive.
One we noticed today, and have re-generated, is "starvingchildstillebony". None of those words are a problem on their own, but together they may be offensive.
There are some real gems that come up. But, of course, there are even more complex cases where we do not punctuate the words. I saw on facebook a hash tag of #susanalbumparty which is a bit "sus" to say the least. I suspect the word "therapist" is in our list as well...
Anyway, I think our customers are suitably understanding that a random number generator cannot really be out to get them, and they can just click to regenerate a new password.
We have some nice long word lists and have even organised them as adjectives followed by nouns to make even easier to remember. The word lists are longer than those suggested by Randall. But even if you had the exact word lists we use you still have very high entropy in the passwords. We use a true random number generator even (because we can).
There is, however, a problem, a side effect Randall did not explain. It is not just rude words (which are not too hard to eliminate from the word list) but word combinations that can be offensive.
One we noticed today, and have re-generated, is "starvingchildstillebony". None of those words are a problem on their own, but together they may be offensive.
There are some real gems that come up. But, of course, there are even more complex cases where we do not punctuate the words. I saw on facebook a hash tag of #susanalbumparty which is a bit "sus" to say the least. I suspect the word "therapist" is in our list as well...
Anyway, I think our customers are suitably understanding that a random number generator cannot really be out to get them, and they can just click to regenerate a new password.
2012-11-28
If you build it, they will come
That "field of dreams" type logic sums up our marketing strategy most of the time.
I have the new Home::1 tariff - you can buy it now.
Whilst we have not done the press releases yet - that is for Friday, I was hoping that linking in on the main web site would have seen someone order. Even if I have said "officially" it launches on 1st Dec.
We have over 30 existing customers switched over to it (thank you, all) - I just want some new random customer to buy it to prove my order page all works as expected and give me some feedback. Is that too much to ask?
Closest I have so far is an IPv6 visitor, who is, as you may expect, and existing customer that went no further than the order page.
Someone buy this, if not for themselves, then get their mum or dad on the tariff. You know you want to. We can even do "manager" access on friends and family lines on the control pages to allow you to see the line and order tracking amd faults and stuff if you like (ask me on irc). We have a very cool service - it works way better than many "main stream" services and nobody else does the sort of graphs we do, do they?
/me ponders that he lacks enough friends to which to suggest this tariff - but in fact, what friends and relatives I have are either ISPs or already on our service or even this new tariff. I have a party to go to Friday - maybe I need some cards to hand out :-)
I have the new Home::1 tariff - you can buy it now.
Whilst we have not done the press releases yet - that is for Friday, I was hoping that linking in on the main web site would have seen someone order. Even if I have said "officially" it launches on 1st Dec.
We have over 30 existing customers switched over to it (thank you, all) - I just want some new random customer to buy it to prove my order page all works as expected and give me some feedback. Is that too much to ask?
Closest I have so far is an IPv6 visitor, who is, as you may expect, and existing customer that went no further than the order page.
Someone buy this, if not for themselves, then get their mum or dad on the tariff. You know you want to. We can even do "manager" access on friends and family lines on the control pages to allow you to see the line and order tracking amd faults and stuff if you like (ask me on irc). We have a very cool service - it works way better than many "main stream" services and nobody else does the sort of graphs we do, do they?
/me ponders that he lacks enough friends to which to suggest this tariff - but in fact, what friends and relatives I have are either ISPs or already on our service or even this new tariff. I have a party to go to Friday - maybe I need some cards to hand out :-)
Reading fiction
I am pretty confident that my readers are not swayed by them unduly, but are amused by them. Indeed, this is one source of amusement in comments I have received privately from several people.
It is funny when I complain about BT's ineptitude and a BT advert comes up, or DHL when I moaned about them. Ironic that they are paying me, albeit a few pence, to slag them off in public!
But back to the fiction... This is a real advert as it appeared on this very blog for UK2NET. It claims "Speeds up to 100TB"
WTF? Seriously they need to think about that. Well, it is not a speed, but then neither is 1 Gigabit, so "per second" is assumed here. 100Tb/s is not really practical. Even LINX are not at that level, are they? The big transatlantic links are nearer 10Tb/s.
So, you need to talk the adds with a pinch of (Schwartz) salt...
A picture speaks a 1000 words
I ordered some stuff on-line, and it arrived promptly. But one of the items was cracked in the post. Well, shit happens.
So, I called up the email order confirmation on my iPhone, clicked reply, took a picture, copied and pasted, and sent.
I did not say anything else in my email, so the whole process was a matter of seconds using a device already in my pocket and not having to walk to the computer or anything. Quicker than faffing with calling them or anything. Technology is great some times, isn't it.
I got a quick and very polite reply, apologising and saying they will send a replacement immediately. Well done!
Makes a change to say something nice about a company :-)
Update: A new tub arrived by post.
Update: A new tub arrived by post.
Stealing back Christmas
Last Christmas my parents broadband broke and BT were a pain... I blogged :-)
Believe it or not, there is a maintenance level available on broadband lines that we get from BT which has a target fix time of 7 hours, 24/7. This is a lot better than their usual target fix of 40 hours, and not working weekends.
So, what does that mean exactly? To be honest, we don't know. We do know that the compensation for not meeting it is peanuts (just like the other SLAs). But in theory, at least, BT should jump when they get a fault report on this service level.
So, I am going to put it on my parents line over Christmas this year, just in case. Last year we had a lot of argument over terms, and whether 25th Dec was a Bank Holiday or not, and so on. With the 7 hour fix, that specifically says they should be working on this, even on weekends, bank holidays, and Christmas day!
Of course, their line will be fine, such is the nature of any sort of insurance.
I wonder whether more businesses should put lines on this over Christmas, given that we can put it on a line for just a month, even. The catch is, from our point of view, that we don't have staff on call 24/7 (well, apart from major outages). Even so, we can usually be contacted out of hours (e.g. on irc) and if someone was on 7 hour fix we can try and get BT on the case. So it may make sense.
It will be interesting to ever see one of these lines with a fault and see if BT do in fact jump!
Update: We are looking to try this on a line where the line comes in with a fault late in the day, so we order 7 hour maintenance, an order which goes on over night, and then the next morning we report the fault (on 7 hour fix). Overall this should be less than the normal 40 hour target fix time, even waiting for the 7 hour fix order to be applied to the line. Sadly this only works if the next day is a normal working day as it takes a working day to apply the change of maintenance level. We'll see if that works...
Believe it or not, there is a maintenance level available on broadband lines that we get from BT which has a target fix time of 7 hours, 24/7. This is a lot better than their usual target fix of 40 hours, and not working weekends.
So, what does that mean exactly? To be honest, we don't know. We do know that the compensation for not meeting it is peanuts (just like the other SLAs). But in theory, at least, BT should jump when they get a fault report on this service level.
So, I am going to put it on my parents line over Christmas this year, just in case. Last year we had a lot of argument over terms, and whether 25th Dec was a Bank Holiday or not, and so on. With the 7 hour fix, that specifically says they should be working on this, even on weekends, bank holidays, and Christmas day!
Of course, their line will be fine, such is the nature of any sort of insurance.
I wonder whether more businesses should put lines on this over Christmas, given that we can put it on a line for just a month, even. The catch is, from our point of view, that we don't have staff on call 24/7 (well, apart from major outages). Even so, we can usually be contacted out of hours (e.g. on irc) and if someone was on 7 hour fix we can try and get BT on the case. So it may make sense.
It will be interesting to ever see one of these lines with a fault and see if BT do in fact jump!
Update: We are looking to try this on a line where the line comes in with a fault late in the day, so we order 7 hour maintenance, an order which goes on over night, and then the next morning we report the fault (on 7 hour fix). Overall this should be less than the normal 40 hour target fix time, even waiting for the 7 hour fix order to be applied to the line. Sadly this only works if the next day is a normal working day as it takes a working day to apply the change of maintenance level. We'll see if that works...
2012-11-27
BT Wholesale is a world-class enabler of converged network solutions?
What can I say - my sister-in-law Pauline has broadband FTTC from us. Simple.
But when we changed the phone line over to us, because it is cheaper (we do broadband only phone line on our new Home::1 tariff for only £10/month including VAT). This should be a simple records change, not anything technical.
That was over 8 days ago now. The order completed and a couple of hours later the FTTC went off line.
Now, lets be clear here, the line is in sync. All the bits of wire and fibre needed for this to be working are in place. There is no fault. It is entirely down to config on equipment. It should take seconds to fix.
They have not yet escalated to high level complaints even!
BT admit they got it wrong, and they are expediting getting this fixed !!!
If 8 days with no internet is an expedite then BT have no clue whatsoever.
We are over 40 emails on this that I have been copied on, and numerous calls, emails an echats that Shaun has been chasing as well.
Here is a selection of the crap we have received from this world-class enabler of converged solutions. Yes, that is what the sig on their email states. Like the hell!!??
Apparently BT say "Our vision for BT is to be dedicated to helping customers thrive in a changing world" which sounds like the Dilbert mission statement generator.
20th Nov, BT saying the service is ceased, but no explanation as to why
This needs to be addressed with PSTN team, wholesale will close the fault as no further action can be taken by us.
I have raised concerns with openreach who have finally agreed to get a free expedite due to their system error. We are currently in the process of clearing the tags i.e. a open modify order should be cancelled and then a cease light order to update adsl checker.
So, that seems to mean they have accepted they has system errors in the first place.
23rd Nov: The cease order is delayed due to the system issue and has been highlighted to the back end team for investigation and quicker progression.
We shall revert back with an update by 10:00hrs in the morning today.
24th Nov: The ASG team are from BT Operate hence they do not work over the weekends. If this is not moving further on Monday we will raise it to HLE.
26th Nov: We are chasing the ASG team to know the further actions on progression of the orders and to get the line cleared.
We will push this further with them to get the order moved and closed at the earliest today. We will update you on the status by 12:00hrs.
P.S. BT's total incompetence when they have something like this that is not even an actual fault is one reason we recommend businesses have two lines.
Updates:
29th Nov: Finally, after 10 days, we have finally got to the stage that BT can START the expedite of the fix for this. What the hell?
1st Dec: Finally, with me, the MD, visiting Pauline's and staying here from 8am until an engineer arrived, we have internet back. Sandra and Pauline should get back from shopping soon. Shame we don't see BT, who made the mistake in the first place, going to such lengths.
But when we changed the phone line over to us, because it is cheaper (we do broadband only phone line on our new Home::1 tariff for only £10/month including VAT). This should be a simple records change, not anything technical.
That was over 8 days ago now. The order completed and a couple of hours later the FTTC went off line.
Now, lets be clear here, the line is in sync. All the bits of wire and fibre needed for this to be working are in place. There is no fault. It is entirely down to config on equipment. It should take seconds to fix.
They have not yet escalated to high level complaints even!
BT admit they got it wrong, and they are expediting getting this fixed !!!
If 8 days with no internet is an expedite then BT have no clue whatsoever.
We are over 40 emails on this that I have been copied on, and numerous calls, emails an echats that Shaun has been chasing as well.
Here is a selection of the crap we have received from this world-class enabler of converged solutions. Yes, that is what the sig on their email states. Like the hell!!??
Apparently BT say "Our vision for BT is to be dedicated to helping customers thrive in a changing world" which sounds like the Dilbert mission statement generator.
20th Nov, BT saying the service is ceased, but no explanation as to why
This needs to be addressed with PSTN team, wholesale will close the fault as no further action can be taken by us.
I have raised concerns with openreach who have finally agreed to get a free expedite due to their system error. We are currently in the process of clearing the tags i.e. a open modify order should be cancelled and then a cease light order to update adsl checker.
So, that seems to mean they have accepted they has system errors in the first place.
21st Nov: The modify order is now cancelled and we are placing a new cease light order which should be closed before EOD today.
22nd Nov: Unfortunately, the cease light order has taken 5 days lead time, we are now progressing with cancel and reissue activity. We are hopeful that the cease order would be completed by today EOD.
This is a cease light order to update BTW Records and we need to review this tomorrow for further updates.
We are still waiting for the cease order to be completed, We will keep you posted on the progress tomorrow morning by 11:00 GMT. Thanks.
23rd Nov: The cease order is delayed due to the system issue and has been highlighted to the back end team for investigation and quicker progression.
We shall revert back with an update by 10:00hrs in the morning today.
Further to update you, a bridge case has been raised to progress the cease light order and it is currently under investigation.
We will review this and will keep you posted with the updates by 15:00hrs.
We will review this and will keep you posted with the updates by 15:00hrs.
24th Nov: The ASG team are from BT Operate hence they do not work over the weekends. If this is not moving further on Monday we will raise it to HLE.
26th Nov: We are chasing the ASG team to know the further actions on progression of the orders and to get the line cleared.
We will push this further with them to get the order moved and closed at the earliest today. We will update you on the status by 12:00hrs.
The PSTN order has been closed in our suppliers applications and is currently open on the back end applications. The order will be closed today and this will let us take further actions.
We are not denying the escalation to be passed to HLE team as the actions are already in place and we are waiting for the PSTN order to be completed in the back end applications.
The order should be closed by EOD today. We will review and update the status tomorrow morning by 11:00hrs.
The order should be closed by EOD today. We will review and update the status tomorrow morning by 11:00hrs.
P.S. BT's total incompetence when they have something like this that is not even an actual fault is one reason we recommend businesses have two lines.
Updates:
29th Nov: Finally, after 10 days, we have finally got to the stage that BT can START the expedite of the fix for this. What the hell?
1st Dec: Finally, with me, the MD, visiting Pauline's and staying here from 8am until an engineer arrived, we have internet back. Sandra and Pauline should get back from shopping soon. Shame we don't see BT, who made the mistake in the first place, going to such lengths.
2012-11-26
New Home::1 order form
Well, OSPF is still on hold waiting for Paul to put together a test system, but he is a tad rushed at present. So I am progressing the new tariff Home::1.
We have the initial pricing and usage model all pinned down. It does not mean it cannot change in the future, but we have the starting point, and even a bonus on Christmas day (extra 3GB allowance added).
So, we need a new ordering page for it - a simple one, as it is targeted at single line home customers.
But we are sort of trying to do a completely new ordering system. Catch is, we can't really wait for that. So I spent this afternoon coding an order page for Home::1, in C.
We have some pretty good libraries and wrappers for sql and xml, and using curl lib and a few other bits I was able to put together some reasonably maintainable C code to handle the whole order page.
The idea is a "single form". This is not a new idea - we started with this 16 years ago with one form for mobile phone orders, and then later one form for broadband orders. I think people like it.
With the range of services we normally offer, that has not been the way to go, but this Home::1 tariff can be just one form. So it asks about the line, your name and address, what tariff you want, sortcode and account number, and bingo - ordered.
Of course I expect loads of people get errors, so the form comes back. It sanitises some fields (postcode, phone numbers) and consistently highlights fields with errors, clearly stating the problem and putting the cursor in the right place. Making sure all fields are maintained after posting the form is critical, obviously, but so many web sites get that wrong. The most common issue seems to be the "untick this to avoid junk mail" always comes back ticked after an error. We don't even have one of those boxes :-)
I think I can even do it with no cookies!
I will call it a night now I think - I have all the validation and just need the final checks on some of the response codes from the BT availability check before I can make it actually place an order. That last stage is not too complicated. I already have functions from the existing ordering system for allocation of login names, and IPv6 addresses, and so on.
With any luck, by end of tomorrow, I can have a new Home::1 order form up and running.
Then we just need it on the web site and TBB and ispreview and so on. But lets wait for some test customers first :-)
Update: I think I have it ready - just need a new customer wanting Home::1 now!
Update: On the main web site - and awaiting first order!
We have the initial pricing and usage model all pinned down. It does not mean it cannot change in the future, but we have the starting point, and even a bonus on Christmas day (extra 3GB allowance added).
So, we need a new ordering page for it - a simple one, as it is targeted at single line home customers.
But we are sort of trying to do a completely new ordering system. Catch is, we can't really wait for that. So I spent this afternoon coding an order page for Home::1, in C.
We have some pretty good libraries and wrappers for sql and xml, and using curl lib and a few other bits I was able to put together some reasonably maintainable C code to handle the whole order page.
The idea is a "single form". This is not a new idea - we started with this 16 years ago with one form for mobile phone orders, and then later one form for broadband orders. I think people like it.
With the range of services we normally offer, that has not been the way to go, but this Home::1 tariff can be just one form. So it asks about the line, your name and address, what tariff you want, sortcode and account number, and bingo - ordered.
Of course I expect loads of people get errors, so the form comes back. It sanitises some fields (postcode, phone numbers) and consistently highlights fields with errors, clearly stating the problem and putting the cursor in the right place. Making sure all fields are maintained after posting the form is critical, obviously, but so many web sites get that wrong. The most common issue seems to be the "untick this to avoid junk mail" always comes back ticked after an error. We don't even have one of those boxes :-)
I think I can even do it with no cookies!
I will call it a night now I think - I have all the validation and just need the final checks on some of the response codes from the BT availability check before I can make it actually place an order. That last stage is not too complicated. I already have functions from the existing ordering system for allocation of login names, and IPv6 addresses, and so on.
With any luck, by end of tomorrow, I can have a new Home::1 order form up and running.
Then we just need it on the web site and TBB and ispreview and so on. But lets wait for some test customers first :-)
Update: I think I have it ready - just need a new customer wanting Home::1 now!
Update: On the main web site - and awaiting first order!
2012-11-25
Shibboleet
Shibboleet is a shibboleth that was created by Randall for xkcd. It is a nice idea. It is a keyword to say to people on support to confirm you are a member of the "leet" and know more that the average customer.
I did a blog post on this - I have told all the staff that "Shibboleet" is a magic word, and means to stop arguing and transfer the call to someone that knows what they are talking about. Of course, this is tricky as most of my staff know what they are talking about anyway, and indeed many meet the "someone who knows a minimum of two programming languages" criteria defined in the cartoon.
I think today was possible the first use of "Shibboleet" by a prospective customer. Someone that apparently not only knows programming languages themselves but makes them. Someone interested in being a customer.
So, on the irc channels, the customers on there are well aware of Shibboleet, and flagged the query to me. They were gobsmacked at finding an ISP that has an irc channel in the first place! Hopefully I gave a good impression and we have a new customer as a result. We'll see.
This may lead to the first "Shibboleet" sale we have made. Thanks Randall.
I did a blog post on this - I have told all the staff that "Shibboleet" is a magic word, and means to stop arguing and transfer the call to someone that knows what they are talking about. Of course, this is tricky as most of my staff know what they are talking about anyway, and indeed many meet the "someone who knows a minimum of two programming languages" criteria defined in the cartoon.
I think today was possible the first use of "Shibboleet" by a prospective customer. Someone that apparently not only knows programming languages themselves but makes them. Someone interested in being a customer.
So, on the irc channels, the customers on there are well aware of Shibboleet, and flagged the query to me. They were gobsmacked at finding an ISP that has an irc channel in the first place! Hopefully I gave a good impression and we have a new customer as a result. We'll see.
This may lead to the first "Shibboleet" sale we have made. Thanks Randall.
2012-11-24
Rank
I noticed that rank is indicated but "pips", and the more pips, the higher the rank.
But surely in any hierarchy of control there is a limit to how far "up" one can go, but no limit to how far "down" you can go.
Surely any numbering system should have 1 (or 0) as the top, and more "pips" for those lower down....
After all, one can always have subordinates.
2012-11-23
Should I sue BT, or what?
Pauline, my sister in law (hi! Pauline) has broadband from us (doh!).
And we took over the phone line as we can provide a phone line for broadband use cheaper for her. Like many people she uses the mobile to actually talk to people, but also uses facebook, and iMessage and FaceTime, and so on.
So BT screw up - they even admit they screwed up - they cease the FTTC (like BT Retail's "Infinity" type service). They even did that in an odd way that meant the bit of BT we deal with (BT plc t/a BT Wholesale) did not know they have ceased it, so two screw ups.
Now they are dragging on the fix to this, and it now in to the fifth day.
The wires are connected to the kit - all is physically there - just some config on some equipment is not right because someone thinks they cease something. This is not (technically) a hard problem for them to solve.
They have repeatedly promised progress on this and failed to meet their promises. We even put the line on a 7 hour fix expensive maintenance category, an order they accepted, and they still don't do anything.
It shows BT's promises are a sham, they are worthless. They are, in my opinion, total fucking idiots.
So what do I do - sue them?
It is so tempting to just issue a county court claim against BT plc for this, you would not believe.
Sorry, Pauline, we are trying, honest.
And we took over the phone line as we can provide a phone line for broadband use cheaper for her. Like many people she uses the mobile to actually talk to people, but also uses facebook, and iMessage and FaceTime, and so on.
So BT screw up - they even admit they screwed up - they cease the FTTC (like BT Retail's "Infinity" type service). They even did that in an odd way that meant the bit of BT we deal with (BT plc t/a BT Wholesale) did not know they have ceased it, so two screw ups.
Now they are dragging on the fix to this, and it now in to the fifth day.
The wires are connected to the kit - all is physically there - just some config on some equipment is not right because someone thinks they cease something. This is not (technically) a hard problem for them to solve.
They have repeatedly promised progress on this and failed to meet their promises. We even put the line on a 7 hour fix expensive maintenance category, an order they accepted, and they still don't do anything.
It shows BT's promises are a sham, they are worthless. They are, in my opinion, total fucking idiots.
So what do I do - sue them?
It is so tempting to just issue a county court claim against BT plc for this, you would not believe.
Sorry, Pauline, we are trying, honest.
Just doing my job
I had a rather interesting comment on an old blog post where I somewhat persecute a cold caller. The comment being "What a prick you are !! this young lad is only doing his job and your trying to prosecute him" [sic].
Obviously I am not trying to prosecute him, that would be for the CPA or the Information Commissioner. I was trying to verbally persecute him though.
But it does leave me wondering why anyone thinks that "only doing his job" is a valid comment? After all, if criminal gang employ someone to break people's legs, a criminal activity, would the same comment be said "he was just doing his job"? After all, whoever employs this cold caller is getting him to do criminal acts. I am not saying they are as severe as breaking someone's legs, but they are just as much a criminal act - illegal - against the law - wrong - offensive - etc.
Anyway, just to make life simpler for such people I am formally making it "part of my job" that I persecute cold callers and ridicule them publicly whenever I can. Now, you can take reassurance that, in such cases, I am "just doing my job", and not anything to get upset about...
This should make all better with the world :-)
Obviously I am not trying to prosecute him, that would be for the CPA or the Information Commissioner. I was trying to verbally persecute him though.
But it does leave me wondering why anyone thinks that "only doing his job" is a valid comment? After all, if criminal gang employ someone to break people's legs, a criminal activity, would the same comment be said "he was just doing his job"? After all, whoever employs this cold caller is getting him to do criminal acts. I am not saying they are as severe as breaking someone's legs, but they are just as much a criminal act - illegal - against the law - wrong - offensive - etc.
Anyway, just to make life simpler for such people I am formally making it "part of my job" that I persecute cold callers and ridicule them publicly whenever I can. Now, you can take reassurance that, in such cases, I am "just doing my job", and not anything to get upset about...
This should make all better with the world :-)
Fun with captive portals
This is the first time we have ever tried to run any sort of captive portal - which is where normal web access to the Internet is redirected to a page telling you that you can't access the Internet!
The two main ways to do this are DNS poisoning (returning wrong entries for DNS queries directing people to the portal) and IP level mapping. We have chosen not to poison DNS, partly because it is "nasty", and things cache it, and partly because it will not work with DNSSEC.
So we are redirecting IP traffic (IPv4 and IPv6, obviously). The good news is that the LNSs allow putting a line in a separate routing table (sort of VRF) which allows us to set a gateway to a FireBrick that can do the re-writes at an IP level. By using a separate VLAN on a linux box we can even present the source IP to the portal web server cleanly.
We are directing web (TCP port 80), DNS (TCP/UDP port 53) and NTP (TCP/UDP port 123) to the portal, but limiting DNS to avoid tunneling, and also locking down the speed of lines that are redirected to avoid any abuse.
The web server itself is redirecting any requests to a portal URL which it then serves. By preserving the source IP we can work out what line it is and so why they are restricted. We are polling the routers and LNSs (FireBricks) with the IP to find where it is going, which should make this 100% reliable for identification, especially where customers have blocks of IP addresses and IPv6 address space and so on.
Then we are offering customers who are over their quota on the new Home::1 tariff a top-up which invoices and issues a DD collection. We can do auto-topup if people want, and just allow slow access to the Internet if they want.
The LNS is then kicked to do an interim accounting update, and the RADIUS accounting spots that the line is now topped up and removes the restrictions and routing table. Within a fraction of a second the service is back in operation.
The whole process, changing routing tables, clamping speed down, and returning to normal, all happens with no change to IP addressing and no drop of the PPP link, which I think is pretty slick.
This all means we are pretty much ready now for existing customers switching over to the new Home::1 tariff, at last. We have to make sure sales and accounts teams are fully up to speed on this, obviously. We have had a few triallists move over already which has allowed testing of the various tools and scripts involved - thank you to all those that have helped out. Hopefully we can launch for new customers next month in plenty of time for Christmas, but anyone that wants it can contact the trials team in the mean time.
For existing customers moving over we have taken the view that we will be as helpful as possible and allow a retrospective change of tariff - crediting the current month's charges and re-issuing as Home::1. However, usage on Home::1 counts from the start of the month also, and any excess usage at the start of the month is charged (at pre-pay, not top-up, rates) from the old tariff to start the month with a clean slate. I think this is about as fair as I can make it, but it does mean people can switch to the new tariff before Christmas. We are, however, expecting to make Christmas week the "special" Christmas period on the old tariff still anyway.
All good fun making this stuff work - it will be interesting to see how customers find it.
The two main ways to do this are DNS poisoning (returning wrong entries for DNS queries directing people to the portal) and IP level mapping. We have chosen not to poison DNS, partly because it is "nasty", and things cache it, and partly because it will not work with DNSSEC.
So we are redirecting IP traffic (IPv4 and IPv6, obviously). The good news is that the LNSs allow putting a line in a separate routing table (sort of VRF) which allows us to set a gateway to a FireBrick that can do the re-writes at an IP level. By using a separate VLAN on a linux box we can even present the source IP to the portal web server cleanly.
We are directing web (TCP port 80), DNS (TCP/UDP port 53) and NTP (TCP/UDP port 123) to the portal, but limiting DNS to avoid tunneling, and also locking down the speed of lines that are redirected to avoid any abuse.
The web server itself is redirecting any requests to a portal URL which it then serves. By preserving the source IP we can work out what line it is and so why they are restricted. We are polling the routers and LNSs (FireBricks) with the IP to find where it is going, which should make this 100% reliable for identification, especially where customers have blocks of IP addresses and IPv6 address space and so on.
Then we are offering customers who are over their quota on the new Home::1 tariff a top-up which invoices and issues a DD collection. We can do auto-topup if people want, and just allow slow access to the Internet if they want.
The LNS is then kicked to do an interim accounting update, and the RADIUS accounting spots that the line is now topped up and removes the restrictions and routing table. Within a fraction of a second the service is back in operation.
The whole process, changing routing tables, clamping speed down, and returning to normal, all happens with no change to IP addressing and no drop of the PPP link, which I think is pretty slick.
This all means we are pretty much ready now for existing customers switching over to the new Home::1 tariff, at last. We have to make sure sales and accounts teams are fully up to speed on this, obviously. We have had a few triallists move over already which has allowed testing of the various tools and scripts involved - thank you to all those that have helped out. Hopefully we can launch for new customers next month in plenty of time for Christmas, but anyone that wants it can contact the trials team in the mean time.
For existing customers moving over we have taken the view that we will be as helpful as possible and allow a retrospective change of tariff - crediting the current month's charges and re-issuing as Home::1. However, usage on Home::1 counts from the start of the month also, and any excess usage at the start of the month is charged (at pre-pay, not top-up, rates) from the old tariff to start the month with a clean slate. I think this is about as fair as I can make it, but it does mean people can switch to the new tariff before Christmas. We are, however, expecting to make Christmas week the "special" Christmas period on the old tariff still anyway.
All good fun making this stuff work - it will be interesting to see how customers find it.
2012-11-20
Failed
FireBrick training course tomorrow and Thursday and OSPF not finished enough to demo, sorry.
Unless I go mad tomorrow morning and Thursday morning, no chance.
Annoyed with self.
I blame new tariffs.
At least the training room is ready :-
(well done Lee on the chairs and Andrew for the tech, and everyone else)
Unless I go mad tomorrow morning and Thursday morning, no chance.
Annoyed with self.
I blame new tariffs.
At least the training room is ready :-
(well done Lee on the chairs and Andrew for the tech, and everyone else)
Porting numbers or not...
As I am sure every single one of our VoIP customers knows, we can't (yet) port numbers out to another provider.
What we can do is direct the calls to a SIP endpoint anywhere on the planet for no charge (for the calls), so pretty flexible.
To be honest I suspect it will be a while before we can. We use SIP based carriers and have no SS7 ourselves, yet. You never know, one day we may have it sorted. But we have no actual porting agreements in place to port out numbers at present.
OFCOM make it clear that have to explain this to customers, including during the sales process, and we do. And we keep a clear record of it. People have moaned that there are rather a lot of check boxes on the number orders, and this is one of them. OFCOM make us point of a number of things to customers!
So what the hell do you do when someone is making formal complaints that we are obstructing his number port? He keeps hassling and now is making a formal complaint to us and OFCOM!
It has got to the stage that he now agrees that he saw and agreed the terms but he still expects something more to be done on his complaint. As far as I am concerned, having agreed we told him we can't port numbers, the fact we don't port numbers is not actually a dispute - it is what was agreed!
Hopefully this is now clear enough evidence of a non-complaint that when he tries to go to ADR we can get it thrown out without a case being started. It would be an interesting test of the new ADR provider.
There is no dispute to resolve - we 100% agree with him that we don't port numbers out. No dispute at all to resolve.
We charge £1/month for a number, and nothing for incoming calls. We make nothing either. We can't be having a service like that which could cost over £300 for ADR and a lot of time. The whole thing is mental. If this went to ADR I *would* put up the prices of our numbers and you can thank OFCOM for that.
Oh well, we'll see how it goes. What we can be sure of is that ADR will make this customer uneconomical to supply service to, which would be a shame.
Update: Is this fair?
What we can do is direct the calls to a SIP endpoint anywhere on the planet for no charge (for the calls), so pretty flexible.
To be honest I suspect it will be a while before we can. We use SIP based carriers and have no SS7 ourselves, yet. You never know, one day we may have it sorted. But we have no actual porting agreements in place to port out numbers at present.
OFCOM make it clear that have to explain this to customers, including during the sales process, and we do. And we keep a clear record of it. People have moaned that there are rather a lot of check boxes on the number orders, and this is one of them. OFCOM make us point of a number of things to customers!
So what the hell do you do when someone is making formal complaints that we are obstructing his number port? He keeps hassling and now is making a formal complaint to us and OFCOM!
It has got to the stage that he now agrees that he saw and agreed the terms but he still expects something more to be done on his complaint. As far as I am concerned, having agreed we told him we can't port numbers, the fact we don't port numbers is not actually a dispute - it is what was agreed!
Hopefully this is now clear enough evidence of a non-complaint that when he tries to go to ADR we can get it thrown out without a case being started. It would be an interesting test of the new ADR provider.
There is no dispute to resolve - we 100% agree with him that we don't port numbers out. No dispute at all to resolve.
We charge £1/month for a number, and nothing for incoming calls. We make nothing either. We can't be having a service like that which could cost over £300 for ADR and a lot of time. The whole thing is mental. If this went to ADR I *would* put up the prices of our numbers and you can thank OFCOM for that.
Oh well, we'll see how it goes. What we can be sure of is that ADR will make this customer uneconomical to supply service to, which would be a shame.
Update: Is this fair?
Your emails, and in particular the order confirmation you agreed and your recent statement that you agreed those terms, will be held as evidence should you try and take this matter further. If you do take this matter further I feel it will become uneconomical to continue providing services to you. As director I am legally required by the Companies Act to act in the best interests of the shareholders, so will have no choice but to terminate your services in accordance with agreed contract terms should they become uneconomical to provide. However, at this stage we have no plans to terminate your services.
More than "complete" broadband?
Had an interesting complaint today - well more of a query - apparently we provided a customer with more than 100% uptime in October!
Well, you can guess what it was, I hope. It was 28th October. It has 25 hours in the day. Customers were seeing 104% uptime for October 28th.
It struck me as odd - as I am not usually that foolish, and I coded this.
This is where it gets interesting though. I checked my code. We log uptime in seconds, and we work out percentages based on local time. That means we were working on a 90,000 seconds day for 28th October. So we allowed for the extra hour in the percentage calculation... I thought I was not that foolish. So how do we see 104% uptime?
Looking in to it more, we are seeing the uptime for the day recorded at 93,600. That is 26 hours in a 25 hour day...
Arrrg!
I believe I have eventually tracked it down though.
Each hour we get RADIUS accounting. We record the change since the last hour. The "last hour" record, well, "last update", is stored in an sql database. It includes the tx and rx, byte and packet, counts and the time of the last update.
When we process the new update we can see the time and usage has moved on, and we update the stats. The uptime normally adding 3600.
The catch is that the time is recorded in local time.
At 00:00:00BST we record stats and save the time as 00:00:00.
At 01:00:00BST we see a last time as 00:00:00 and assume BST, so record 3600 seconds, and record time as 01:00:00
At 01:00:00UTC we see a last time as 01:00:00 and assume BST, so record 3600 seconds, and record time as 01:00:00
At 02:00:00UTC we see a last time as 01:00:00 and assume BST, so record 7200 seconds.
At 03:00:00UTC we see a last time as 02:00:00 and assume UTC, so record 3600 seconds.
If a customer logs out and back in between 1am and 1am then we amplify this, recording yet another hour. We had one customer with 107% uptime for that day!
The fix is simple - those fields are now assumed to be UTC. Lets hope sql has no issue with March where a time of 02:00:00 is not valid.
Just goes to show, you can give "more than 100%" :-)
P.S. Before you look, I am fixing the erroneous stats.
Well, you can guess what it was, I hope. It was 28th October. It has 25 hours in the day. Customers were seeing 104% uptime for October 28th.
It struck me as odd - as I am not usually that foolish, and I coded this.
This is where it gets interesting though. I checked my code. We log uptime in seconds, and we work out percentages based on local time. That means we were working on a 90,000 seconds day for 28th October. So we allowed for the extra hour in the percentage calculation... I thought I was not that foolish. So how do we see 104% uptime?
Looking in to it more, we are seeing the uptime for the day recorded at 93,600. That is 26 hours in a 25 hour day...
Arrrg!
I believe I have eventually tracked it down though.
Each hour we get RADIUS accounting. We record the change since the last hour. The "last hour" record, well, "last update", is stored in an sql database. It includes the tx and rx, byte and packet, counts and the time of the last update.
When we process the new update we can see the time and usage has moved on, and we update the stats. The uptime normally adding 3600.
The catch is that the time is recorded in local time.
At 00:00:00BST we record stats and save the time as 00:00:00.
At 01:00:00BST we see a last time as 00:00:00 and assume BST, so record 3600 seconds, and record time as 01:00:00
At 01:00:00UTC we see a last time as 01:00:00 and assume BST, so record 3600 seconds, and record time as 01:00:00
At 02:00:00UTC we see a last time as 01:00:00 and assume BST, so record 7200 seconds.
At 03:00:00UTC we see a last time as 02:00:00 and assume UTC, so record 3600 seconds.
If a customer logs out and back in between 1am and 1am then we amplify this, recording yet another hour. We had one customer with 107% uptime for that day!
The fix is simple - those fields are now assumed to be UTC. Lets hope sql has no issue with March where a time of 02:00:00 is not valid.
Just goes to show, you can give "more than 100%" :-)
P.S. Before you look, I am fixing the erroneous stats.
2012-11-19
If I knew any marketing, I'd be dangerous!
We are gradually sorting the triallists for the new Home::1 tariff today. We have some technical aspects to roll out (LNS upgrades) as well as manually checking each for billing accuracy and so on. It is quite interesting doing this sort of stuff, honest.
I have yet to sort just how the top-up is is to be handled - we really want to make it a secure page anyway (even though completely internal to us and BT). We'll probably just ask for the accounts password to top up. I did consider simple "click through" to agree a top-up, but you can't leave your kids torrenting all day if all they need to do is click "Yes" on a page when it pops up!
It is only tricky because it is integration with billing, which is a carefully controlled system with very carefully locked down interfaces, for obvious reasons. The last thing you want is a script running amok with direct debits!
What we have decided, from various feedback, is to allow three options on reaching quota.
We still have a lot to do, for example the form/page to change to the new tariff (which is surprisingly complex), and for new customers to sign up. We would like to sort the new ordering system this year as well, but maybe we'll do a simple sign-up form for Home::1 first.
What is interesting, and hence the title of this blog post, is that most of the people moving to the new tariff today will be paying us slightly more per month than before. It is interesting that people are prepared to pay a few pounds a month for peace of mind ("piece of mind" being brain surgery) and the convenience of not having to watch the clock. Very reassuring.
Update: People leaving us have changed their mind for Home::1 and people that left us are coming back for it. Wow!
I have yet to sort just how the top-up is is to be handled - we really want to make it a secure page anyway (even though completely internal to us and BT). We'll probably just ask for the accounts password to top up. I did consider simple "click through" to agree a top-up, but you can't leave your kids torrenting all day if all they need to do is click "Yes" on a page when it pops up!
It is only tricky because it is integration with billing, which is a carefully controlled system with very carefully locked down interfaces, for obvious reasons. The last thing you want is a script running amok with direct debits!
What we have decided, from various feedback, is to allow three options on reaching quota.
- The default, which is block access, lock down to 128k and provide top-up portal web page.
- Slow down, which is allow access to Internet still but lock down to 128k and take usage from next month's allowance. That may have to have some limits as 128k constantly is more than 25GB a month.
- Auto top-up, which is charging for a 50GB top-up automatically, emailing details. Again, we may need to limit how often that can be done.
We still have a lot to do, for example the form/page to change to the new tariff (which is surprisingly complex), and for new customers to sign up. We would like to sort the new ordering system this year as well, but maybe we'll do a simple sign-up form for Home::1 first.
What is interesting, and hence the title of this blog post, is that most of the people moving to the new tariff today will be paying us slightly more per month than before. It is interesting that people are prepared to pay a few pounds a month for peace of mind ("piece of mind" being brain surgery) and the convenience of not having to watch the clock. Very reassuring.
Update: People leaving us have changed their mind for Home::1 and people that left us are coming back for it. Wow!
If I was running BT, how would I have solved it?
Someone asked me, if I ran BT, what would I have done differently to stop these problems?
The basic problem is the service BT provide, which is one-sided. They have a modem at the exchange and the end user or ISP provides a modem. If they work, then good. If not, then there is no way to really be sure who's at fault. The service is provided "at the master socket", but to test at that point is expensive for BT as they have to send an engineer.
The result is ISPs ordered engineers, because they were free, even when it was end user problems. BT retaliated by charging for engineers when proved to be end users fault. Except they went too far and charged when they could not find a fault in BT, so assuming end user is cause (not proving). They also refused to meet at the handover point - i.e. to test where the service is delivered - insisting one orders an optional engineering service without proving it was not a BT fault first. There have been many shots fired both ways in this war, and it is still an issue. The latest is over co-op calls from engineers, but the battles will keep going forever I am sure.
So really, what should be done. What is the right way to do this? What would I have done?
The answer is very simple, and probably not actually stupidly expensive. See the picture. It is of a faceplate splitter and of a small DSL modem. The ADSL chipsets are getting smaller and more integrated all of the time. You can see how a DSL modem could fit in the space of one of these splitters with very little effort.
What we need is a faceplate DSL modem with Ethernet PPPoE jumbo frame (either talking PPPoA on the wire or jumbo frame bridging). Preset to BT's settings of 0/38. No config pages, nothing. If acting as a PPPoE endpoint then it could do with some PPPoE negotiation options (lets write an RFC) to allow an attached router to set alternative options for the line, and to get sync status, but the default should be a proper PPPoE endpoint supporting full 1508 PPPoE (1500 payload). If bridging, it does not need these options even.
It probably needs a power jack on the side as I doubt it can, yet, be line powered. For bonus points make it work on PoE as well.
Importantly it also needs back-end OAM type tests allowing BT to see line status, carry out low level echo tests on the ATM side, and even do tests out on to the Ethernet port including basic TDM (which is standard in many Ethernet chipsets). It must be able to do good quality line level testing and test beyond the handover point on the Ethernet (so confirming cable presence, length, number of pairs, far end connected, etc).
These need to be mass produced and made a standard Openreach level service that plugs in to GEA in the exchange. BT need to provide tests to check the line and Ethernet port. The faceplate itself could be self install of the BT modem, or engineer visit, but self install makes a lot more sense.
We see DSL routers as low as £15. In the quantities BT would need I bet the could get these for £10 or less.
So why would this solve the SFI battle? The answer is simple, and we see this with FTTC now. It is because the service from BT would no longer be one-sided. It would be delivered to the Ethernet port, and include testing up to and including that port. BT could actually prove the service to the handover point. Engineers would only ever be needed if there really is a fault, so no chance of spurious engineers being ordered. BT could go back to the same system where engineers do not get charged for. ISPs could eliminate BT from the problem by simple BT provided tests.
It would still allow all of the features of both sophisticated and cheap routers, allowing end user and ISP choice of kit. The routers would be PPPoE, but would work as well for ADSL, FTTC or FTTP. If the modem took PoE the routers would have PoE to power them, avoiding extra power bricks. Routers already exist!
By making this GEA, it means that GEA in the exchange (as used by BTW and other ISPs) could handle FTTP, FTTC and ADSL all on the same links with the same ordering and management interfaces and fault reporting. Obviously it would be neat to get VDSL chipsets down to the same size and having same levels of testing. FTTP is already done as an "active NTE", and this would just complete the set offering 250k to 330M in the same orthogonal Openreach service.
It could run along side existing DSL services as an option that costs slightly more to install. BT could do self install via ISPs who manage the shipping of the modem to the end user. BT would be able to tell their modem is on the line, and the design would mean it has to be at the master socket, eliminating splitter issues and extension wiring.
To quote from last nights excellent stand-up at Woking, in the words of Alan Davies: That's my view, anyway.
2012-11-18
New A&A Home::1 tariff
We've come up with a new tariff for home users, called Home::1.
It is always tricky coming up with any tariff for Internet usage, and a lot of customers do not realise the issues. For some things we have a straight forward cost to providing the thing we are selling, so we simply have a profit margin for selling that service. Some things create extra support work, so need a higher margin. Some things need equipment to handle lots of customers, making the cost per customer hard to work out and vary depending on how many customers we have. This sort of issue applies to many businesses.
However, the biggest issue every ISP has to consider in creating a tariff is usually the actual Internet usage element. I have to say that I am hopeful that this issue will go away in time. It has to, and the technology to supply faster and cheaper backhaul links is always advancing, so it should. But right now, the bandwidth/usage of Internet is an issue.
As a really simple example, customers will see an 80Mb/s FTTC line as something that should cost very little (we start at £30/month). But if they, as one extra customer, use 80Mb/s at our peak time of day (just for the peak 15 minutes, one day in the month), we have to have 80Mb/s extra capacity to BT (we pay BT on 100th percentile usage). That extra 80Mb/s costs us (and any other ISP) just for the BT backhaul on 21CN (which is cheaper than 20CN), around £4,700 a month. There are many other costs involved, that is just the main one. Thankfully, at that exact moment, hundreds of other customers are using much less. With enough customers it averages out sensibly. But you can see how usage is an important factor.
It is not inherently a special issue to Internet usage - the same issue happens with water supply. It has costs for supply and limited size pipes (literally!). It has contention and congestion - if everyone turns their taps on in the same street at the same time then they will go at a trickle. It is still sold at a fixed price in many places (depending on size of house), even though usage per house can massively vary.
Internet is a bit like that - the average usage at peak times for Internet, given unlimited usage and speed, is not a lot more than 100Kb/s. In our case it is nearer 120Kb/s at present, but it is a lot less than people realise. On that basis one can, in theory, dimension and price a service on that basis.
One issue is that the actual usage varies a lot. Whilst the average is low, unlike water supply, the peak users are massively higher, just there are usually only a few of them. It is possible now to get 330Mb/s FTTP links, and if someone has one of these on an uncongested network it is possible for someone to run that link flat out at that speed, something like 3000 times the average. Scary!
A big issue is competition. If one service provider has a really good deal, and no usage caps, and no fair usage policy, and no usage charging, then the people drawn to that service are the ones with way higher average usage! If there was only one provider, the pricing could indeed be based on the actual average of usage. But we all want competition, so we have this problem.
So, what of the new A&A tariff?
The current tariff system has a cut off at 6pm, before which usage is more expensive. Customers don't like that. It also has automatic charging if you use too much (beyond a degree of give and take within the tariff), and customers don't like that. Well, residential customers don't like it. The business customers pay for usage during the day, and use it, and generally are very happy.
The new tariff is just usage per month (download only) that is any time of day or night. This is simple, and easy to understand, and avoids the issues with time of day. It also stops usage at the cap, allowing top up if you want, but you don't have to. We are starting at £25/month for 25GB, but for an extra £10/month that can be 100GB. It makes the service easier to compare with other ISPs.
Whilst this is simple, it is a risk for us. It could be that we start to attract heavy users that download a lot during the day. As I explain above, as long as the averages work out, that is not a problem, but if we start attracting a lot of such users then the tariff won't work.
To try and manage this gamble we are making it residential only, and setting what we think are sensible usage levels which would discourage people that torrent 24/7. Even so, the main risk management is making this a separate and new tariff. We can try it out and see how it goes. If, say, in three months time, we find that the usage is rising too much and starting to be uneconomical, we can simply stop new supply of this tariff, and have a re-think. This also means we can make sure that the new service does not have any adverse impact on our business customers.
This is not the first time we have done this sort of tariff. We used to have unlimited usage tariffs, and a handful of people are still on those tariffs from a decade ago. We hope we have picked sensible levels and prices to appeal to sensible customers who want the lower risk and simpler tariff.
So, right now, we are trialling with some existing customers (see status pages). Hopefully we can fully launch this before Christmas.
It does require some coding, which we have been busy on over the last few days. We have to make the LNS handle a quota system so that we know when someone hits their limit without any delay. We have to have a captive portal to advise customers that their line is shut down and offer the top-up option. We have to make new billing options. It is all good fun :-)
What we have been very keen to do is ensure the service is the same high quality we always provide, with the same level of monitoring and technical support and control. And, of course, IPv6 addresses, which is one reason it is called Home::1.
P.S. Go Pauline! first on the new tariff :-)
It is always tricky coming up with any tariff for Internet usage, and a lot of customers do not realise the issues. For some things we have a straight forward cost to providing the thing we are selling, so we simply have a profit margin for selling that service. Some things create extra support work, so need a higher margin. Some things need equipment to handle lots of customers, making the cost per customer hard to work out and vary depending on how many customers we have. This sort of issue applies to many businesses.
However, the biggest issue every ISP has to consider in creating a tariff is usually the actual Internet usage element. I have to say that I am hopeful that this issue will go away in time. It has to, and the technology to supply faster and cheaper backhaul links is always advancing, so it should. But right now, the bandwidth/usage of Internet is an issue.
As a really simple example, customers will see an 80Mb/s FTTC line as something that should cost very little (we start at £30/month). But if they, as one extra customer, use 80Mb/s at our peak time of day (just for the peak 15 minutes, one day in the month), we have to have 80Mb/s extra capacity to BT (we pay BT on 100th percentile usage). That extra 80Mb/s costs us (and any other ISP) just for the BT backhaul on 21CN (which is cheaper than 20CN), around £4,700 a month. There are many other costs involved, that is just the main one. Thankfully, at that exact moment, hundreds of other customers are using much less. With enough customers it averages out sensibly. But you can see how usage is an important factor.
It is not inherently a special issue to Internet usage - the same issue happens with water supply. It has costs for supply and limited size pipes (literally!). It has contention and congestion - if everyone turns their taps on in the same street at the same time then they will go at a trickle. It is still sold at a fixed price in many places (depending on size of house), even though usage per house can massively vary.
Internet is a bit like that - the average usage at peak times for Internet, given unlimited usage and speed, is not a lot more than 100Kb/s. In our case it is nearer 120Kb/s at present, but it is a lot less than people realise. On that basis one can, in theory, dimension and price a service on that basis.
One issue is that the actual usage varies a lot. Whilst the average is low, unlike water supply, the peak users are massively higher, just there are usually only a few of them. It is possible now to get 330Mb/s FTTP links, and if someone has one of these on an uncongested network it is possible for someone to run that link flat out at that speed, something like 3000 times the average. Scary!
A big issue is competition. If one service provider has a really good deal, and no usage caps, and no fair usage policy, and no usage charging, then the people drawn to that service are the ones with way higher average usage! If there was only one provider, the pricing could indeed be based on the actual average of usage. But we all want competition, so we have this problem.
So, what of the new A&A tariff?
The current tariff system has a cut off at 6pm, before which usage is more expensive. Customers don't like that. It also has automatic charging if you use too much (beyond a degree of give and take within the tariff), and customers don't like that. Well, residential customers don't like it. The business customers pay for usage during the day, and use it, and generally are very happy.
The new tariff is just usage per month (download only) that is any time of day or night. This is simple, and easy to understand, and avoids the issues with time of day. It also stops usage at the cap, allowing top up if you want, but you don't have to. We are starting at £25/month for 25GB, but for an extra £10/month that can be 100GB. It makes the service easier to compare with other ISPs.
Whilst this is simple, it is a risk for us. It could be that we start to attract heavy users that download a lot during the day. As I explain above, as long as the averages work out, that is not a problem, but if we start attracting a lot of such users then the tariff won't work.
To try and manage this gamble we are making it residential only, and setting what we think are sensible usage levels which would discourage people that torrent 24/7. Even so, the main risk management is making this a separate and new tariff. We can try it out and see how it goes. If, say, in three months time, we find that the usage is rising too much and starting to be uneconomical, we can simply stop new supply of this tariff, and have a re-think. This also means we can make sure that the new service does not have any adverse impact on our business customers.
This is not the first time we have done this sort of tariff. We used to have unlimited usage tariffs, and a handful of people are still on those tariffs from a decade ago. We hope we have picked sensible levels and prices to appeal to sensible customers who want the lower risk and simpler tariff.
So, right now, we are trialling with some existing customers (see status pages). Hopefully we can fully launch this before Christmas.
It does require some coding, which we have been busy on over the last few days. We have to make the LNS handle a quota system so that we know when someone hits their limit without any delay. We have to have a captive portal to advise customers that their line is shut down and offer the top-up option. We have to make new billing options. It is all good fun :-)
What we have been very keen to do is ensure the service is the same high quality we always provide, with the same level of monitoring and technical support and control. And, of course, IPv6 addresses, which is one reason it is called Home::1.
P.S. Go Pauline! first on the new tariff :-)
2012-11-17
Think of the fucking children!
Err, OK, deliberate blog post title there...
But really, WTF!
http://www.telegraph.co.uk/technology/internet/9684797/David-Cameron-orders-new-curbs-on-internet-porn.html
This is as bad as every other proposal.
ISPs shift packets.
Nobody ever said that telcos have to filter what is said on a phone call!
Nobody ever said that royal mail have to filter what is written in a letter!
So, lets put this in perspective...
I am a parent, with 5 kids (most now grown up, and some are step-kids). Deity help me! Some work for me now.
Porn is the least of the worries a parent should have in terms of "what their kids can see on the internets"... Really. Porn is (mostly) "normal" stuff, and kids will (teenage boys, I am assuming) get an idea of what is normal and what is somewhat pervy... No amount of "filtering" will stop a teenage boy getting access to porn, and the Internet is not special in that respect.
However, the concept of an ISP level control on this? That is mad!
There is some logic on PC/operating-system control, maybe, but ISP level is mad! It is impractical. It is easily bypassed. It is a false sense of security. It avoids the real issues.
As I have said before, it is like making Spec-Savers responsible for what is seen through the glasses they sell. It is just plain silly.
What is needed is parents taking an actual interest in their kids, and actually being involved in what they do. Talking to them, not just about "nice" stuff, but about what they could encounter as soon as they leave the cotton wool ball. Make sure the kids are well balanced and able to decide for themselves what is sensible and what is not. What they like and what they do not. Some basic self respect for themselves and other people. Armed with that, Internet porn is not an issue.
This is a "parenting thing" not an "Internet thing" and certainly not a "technology thing".
Please, someone, tell the government to get a grip!
But really, WTF!
http://www.telegraph.co.uk/technology/internet/9684797/David-Cameron-orders-new-curbs-on-internet-porn.html
This is as bad as every other proposal.
ISPs shift packets.
Nobody ever said that telcos have to filter what is said on a phone call!
Nobody ever said that royal mail have to filter what is written in a letter!
So, lets put this in perspective...
I am a parent, with 5 kids (most now grown up, and some are step-kids). Deity help me! Some work for me now.
Porn is the least of the worries a parent should have in terms of "what their kids can see on the internets"... Really. Porn is (mostly) "normal" stuff, and kids will (teenage boys, I am assuming) get an idea of what is normal and what is somewhat pervy... No amount of "filtering" will stop a teenage boy getting access to porn, and the Internet is not special in that respect.
However, the concept of an ISP level control on this? That is mad!
There is some logic on PC/operating-system control, maybe, but ISP level is mad! It is impractical. It is easily bypassed. It is a false sense of security. It avoids the real issues.
As I have said before, it is like making Spec-Savers responsible for what is seen through the glasses they sell. It is just plain silly.
What is needed is parents taking an actual interest in their kids, and actually being involved in what they do. Talking to them, not just about "nice" stuff, but about what they could encounter as soon as they leave the cotton wool ball. Make sure the kids are well balanced and able to decide for themselves what is sensible and what is not. What they like and what they do not. Some basic self respect for themselves and other people. Armed with that, Internet porn is not an issue.
This is a "parenting thing" not an "Internet thing" and certainly not a "technology thing".
Please, someone, tell the government to get a grip!
2012-11-15
Usage quotas
In an attempt to distract me from OSPF, I have been playing with ways to handle usage quotas.
The idea is that on some services, some customers would like a way to pre-set a cap on usage, whether it is a data SIM or a broadband line.
The catch with such ideas is that the usage metering is by RADIUS accounting which we run every hour. Now, we could do this more often, but even short intervals can be vey large amounts of usage on some services (especially with things like 330Mb/s FTTP lines in the pipeline).
So the trick is to tell the LNS a quota for a connection and have it spot the usage has exceeded that in something like real time.
Now, RADIUS already has something like that, but it was designed for dialup and is a Session-Timeout or Idle-Timeout which is based on time not data usage. We already support Session-Timeout (though not Idle-Timeout as that makes little sense on broadband). I can't find a RADIUS AVP for data quota limit, so I am using the Filter settings (like most of the other special settings we use).
First snag is what you meter. For data SIMs the usage is Tx and Rx (total) but for ADSL it is only Tx that matters. So we have to support a choice of quota metering type.
Then we have the question of what to do when we reach a limit. Well, RADIUS has Terminate-Action AVP which is used with Session-Timeout and allows either ending the session or resending an Access-Request. Terminating the session is messy. Many routers reconnect within a few seconds but some take minutes. It would be neater not to drop the PPP session.
Sadly the idea of re-authentication is somewhat flawed. For a start, the way we work, we throw away all the PPP negotiation and authentication data once the connection is completed so we can't re-auth. We could try a new CHAP challenges, but many routers and systems barf at that (including mobile data links) even though the spec says that should work, and anyway this does not work with PAP. Even if we solved these self imposed issues, the RADIUS server will not have the up to date data to decide what to do as the accounting could be up to an hour before. The solution is not to re-authenticate, but to send an intermediate RADIUS accounting packet instead.
This fits well as the RADIUS accounting server then has the up to date information to decide if over quota, but it can also check the customer database to confirm if there are changes (billing errors, start of new month, top-up, etc), and decide if the line needs to be locked down or not, and what quota it now needs. Sending an accounting update as soon as we hit the quota will allow the accounting server to know we are over limit immediately. It can then use a RADIUS CoA (Change of Authorisation) to change the line in some way if needed.
The CoA can be used to disconnect the line, clamp it to a low speed, or force on to a special routing table to hit a captive portal prompting people to top-up. It can also un-do these effects, and all without ever dropping the PPP link. If can update the quota as we roll over to a new month. All handled in one place. As the saying goes, "simples!".
So, having changed the LNS to support Terminate-Action choice of hang-up or accounting update, and to handle filters for Tx limit or Tx+Rx limit, we can now make some new service options around that. Perhaps pre-pay data SIMs? Or usage capped broadband services... Watch this space.
The idea is that on some services, some customers would like a way to pre-set a cap on usage, whether it is a data SIM or a broadband line.
The catch with such ideas is that the usage metering is by RADIUS accounting which we run every hour. Now, we could do this more often, but even short intervals can be vey large amounts of usage on some services (especially with things like 330Mb/s FTTP lines in the pipeline).
So the trick is to tell the LNS a quota for a connection and have it spot the usage has exceeded that in something like real time.
Now, RADIUS already has something like that, but it was designed for dialup and is a Session-Timeout or Idle-Timeout which is based on time not data usage. We already support Session-Timeout (though not Idle-Timeout as that makes little sense on broadband). I can't find a RADIUS AVP for data quota limit, so I am using the Filter settings (like most of the other special settings we use).
First snag is what you meter. For data SIMs the usage is Tx and Rx (total) but for ADSL it is only Tx that matters. So we have to support a choice of quota metering type.
Then we have the question of what to do when we reach a limit. Well, RADIUS has Terminate-Action AVP which is used with Session-Timeout and allows either ending the session or resending an Access-Request. Terminating the session is messy. Many routers reconnect within a few seconds but some take minutes. It would be neater not to drop the PPP session.
Sadly the idea of re-authentication is somewhat flawed. For a start, the way we work, we throw away all the PPP negotiation and authentication data once the connection is completed so we can't re-auth. We could try a new CHAP challenges, but many routers and systems barf at that (including mobile data links) even though the spec says that should work, and anyway this does not work with PAP. Even if we solved these self imposed issues, the RADIUS server will not have the up to date data to decide what to do as the accounting could be up to an hour before. The solution is not to re-authenticate, but to send an intermediate RADIUS accounting packet instead.
This fits well as the RADIUS accounting server then has the up to date information to decide if over quota, but it can also check the customer database to confirm if there are changes (billing errors, start of new month, top-up, etc), and decide if the line needs to be locked down or not, and what quota it now needs. Sending an accounting update as soon as we hit the quota will allow the accounting server to know we are over limit immediately. It can then use a RADIUS CoA (Change of Authorisation) to change the line in some way if needed.
The CoA can be used to disconnect the line, clamp it to a low speed, or force on to a special routing table to hit a captive portal prompting people to top-up. It can also un-do these effects, and all without ever dropping the PPP link. If can update the quota as we roll over to a new month. All handled in one place. As the saying goes, "simples!".
So, having changed the LNS to support Terminate-Action choice of hang-up or accounting update, and to handle filters for Tx limit or Tx+Rx limit, we can now make some new service options around that. Perhaps pre-pay data SIMs? Or usage capped broadband services... Watch this space.
2012-11-13
Odd day
Not sure if suffering the after effects from yesterday's migraine, or just one of those days.
I read the OSPF spec on the Dijkstra shortest path algorithm to use, cringed, read the wikipedia page, smiled, and coded some of it. It is, as I remembered, simple enough to do.
I now need to make a test network to test it. But in theory I have the shortest path cost and should tomorrow have the next hop stuff sorted too, and so be injecting routes (yay!).
We are really at the stage of trying this all for real now, and that means testing, fixing, and even making design changes as we realise we can't actually test something as we can't create that configuration. I suspect I need more places where I define an OSPF "cost" in the config, and have to add some point to point LSAs for VPN tunnels.
I expect a few more days work.
Having done all that, my day sort of ran out of steam, and it was like 11:00 or something.
I eventually gave up, went home at lunch time, and watched old episodes of House on Sky. Not as productive as yesterday which was working 12 hours solid and getting a migraine!
I have started to perk up a bit this evening, but feeling a tad depressed most of today, which is not like me at all.
I will be happier once OSPF is in the wild and being tested. There will be many bugs and more work but I will be over the hard part. I want an alpha out there and in testing before next weeks FireBrick training course.
Sadly, I have a number of other major projects to work on during the coming months. The next challenge is "new ordering system" which is where A&A get a proper XML interface for dealers and customers...
I read the OSPF spec on the Dijkstra shortest path algorithm to use, cringed, read the wikipedia page, smiled, and coded some of it. It is, as I remembered, simple enough to do.
I now need to make a test network to test it. But in theory I have the shortest path cost and should tomorrow have the next hop stuff sorted too, and so be injecting routes (yay!).
We are really at the stage of trying this all for real now, and that means testing, fixing, and even making design changes as we realise we can't actually test something as we can't create that configuration. I suspect I need more places where I define an OSPF "cost" in the config, and have to add some point to point LSAs for VPN tunnels.
I expect a few more days work.
Having done all that, my day sort of ran out of steam, and it was like 11:00 or something.
I eventually gave up, went home at lunch time, and watched old episodes of House on Sky. Not as productive as yesterday which was working 12 hours solid and getting a migraine!
I have started to perk up a bit this evening, but feeling a tad depressed most of today, which is not like me at all.
I will be happier once OSPF is in the wild and being tested. There will be many bugs and more work but I will be over the hard part. I want an alpha out there and in testing before next weeks FireBrick training course.
Sadly, I have a number of other major projects to work on during the coming months. The next challenge is "new ordering system" which is where A&A get a proper XML interface for dealers and customers...
2012-11-12
Migraine
It is all very well having a clear MRI, as I did, but today I have my third ever migraine. See what OSPF has driven me to!
Sadly, whilst at this stage I simply have visual distortion growing slowly, and making it impossible to read what I have typed, I know that soon I will feel like crap.
I want to file a bug report with my DNA please.
Update:
This was a visual effect starting small in centre of vision. Initially I realised I could not see the letter in the middle of the word I was looking at.
It gradually grew moved in to a sort of C shape with top of C in centre, so some above centre, but almost all of the effect left vision. This made reading anything hard. I could see letters off centre, but that is very difficult to read.
It gradually grew, like looking through broken glass and flickering at the edge of the shape.
Eventually it started to grow more and so the centre of my vision started to clear, the C shape spreading left and out, with top left and bottom left of vision affected.
It reached the edges of my vision and then vanished suddenly.
To be clear, this was not in my eye or eyes as such, but in my vision. i.e. the effect was not only visible via one eye. There are migraine effects people can get that are related to the eye, but this was different.
Apparently it is called a Scintillating Scotoma
My left eye seemed odd for a moment - sort of tunnel vision but no clear edges, and then was fine.
Now I am feeling "spaced out". The effect is clearly moving on from just my vision.
My expectation is feeling like crap for a few hours now.
I went to bed - fine next day.
Sadly, whilst at this stage I simply have visual distortion growing slowly, and making it impossible to read what I have typed, I know that soon I will feel like crap.
I want to file a bug report with my DNA please.
Update:
This was a visual effect starting small in centre of vision. Initially I realised I could not see the letter in the middle of the word I was looking at.
It gradually grew moved in to a sort of C shape with top of C in centre, so some above centre, but almost all of the effect left vision. This made reading anything hard. I could see letters off centre, but that is very difficult to read.
It gradually grew, like looking through broken glass and flickering at the edge of the shape.
Eventually it started to grow more and so the centre of my vision started to clear, the C shape spreading left and out, with top left and bottom left of vision affected.
It reached the edges of my vision and then vanished suddenly.
To be clear, this was not in my eye or eyes as such, but in my vision. i.e. the effect was not only visible via one eye. There are migraine effects people can get that are related to the eye, but this was different.
Apparently it is called a Scintillating Scotoma
My left eye seemed odd for a moment - sort of tunnel vision but no clear edges, and then was fine.
Now I am feeling "spaced out". The effect is clearly moving on from just my vision.
My expectation is feeling like crap for a few hours now.
I went to bed - fine next day.
OSPF final stages
Well, I am working on the last part of OSPF now, and possibly the more interesting part. I now have my routes injected in to OSPF as LSAs, and flooded out. What I need to do as the final step is process the Dijkstra shortest path algorithm and inject routes from OSPF LSAs in to my routing table.
But I have to say this protocol is really getting to be a tad special, and my testing has been tricky in some areas. Even after this last stage there is a lot of testing.
A good example is the LSA sequence number. It is 32 bit, and not allowed to change more than once per 5 seconds, so has a time frame of 680 years. If you restart OSPF it starts again, so this means, with no special processing OSPF can run for 680 years.
However, the spec is quite detailed on this. To be fair, this is good - one should specify the edge case, even if unlikely. Ignoring the "I'll be dead by then" cases is what got us in to Y2K in the first place. Though we are talking a single OSPF network running with no breaks for that long, not just some long distant date. That seems a tad less likely.
So, firstly, how would I do it?
A 32 bit counter is fine, maybe start from 0, as that is sane, and maybe make all comparison relative. This means a "circular" comparison. Basically subtract A from B and test top bit (easy to do in C and in assembler). This means that any sequence is seen as "newer" than another if less that 2^31 later in sequence (340 years minimum). This type of logic is common and very easy. When you start a new OSPF instance you start at sequence zero, but in theory you can encounter an "old" entry from a peer, if "older" you ignore (and so do peers), and if "newer" you update your sequence (as the originator of the LSA) to what you see, plus 1. Simples. This start-up logic is in the OSPF spec anyway already.
How does OSPF do it?
OK, well, I have coded this, but really, I was so tempted to say WTF, I will be long dead by then. Does nobody actually think when designing this stuff? Alternative design rules eliminate the complexity and make less code, less testing, less special cases, and more reliability.
Oh, and to add to the fun, OSPF metrics are 24 bit but BGP are 32 bit, so mapping these will be, err, fun!
So, really, really, I plan to have FB2500 and FB2700 alpha releases with OSPF for testing this week, maybe tomorrow or Wednesday. This has taken way longer that BGP even writing the whole routing core as well. I started at 6am today and still working!
But I have to say this protocol is really getting to be a tad special, and my testing has been tricky in some areas. Even after this last stage there is a lot of testing.
A good example is the LSA sequence number. It is 32 bit, and not allowed to change more than once per 5 seconds, so has a time frame of 680 years. If you restart OSPF it starts again, so this means, with no special processing OSPF can run for 680 years.
However, the spec is quite detailed on this. To be fair, this is good - one should specify the edge case, even if unlikely. Ignoring the "I'll be dead by then" cases is what got us in to Y2K in the first place. Though we are talking a single OSPF network running with no breaks for that long, not just some long distant date. That seems a tad less likely.
So, firstly, how would I do it?
A 32 bit counter is fine, maybe start from 0, as that is sane, and maybe make all comparison relative. This means a "circular" comparison. Basically subtract A from B and test top bit (easy to do in C and in assembler). This means that any sequence is seen as "newer" than another if less that 2^31 later in sequence (340 years minimum). This type of logic is common and very easy. When you start a new OSPF instance you start at sequence zero, but in theory you can encounter an "old" entry from a peer, if "older" you ignore (and so do peers), and if "newer" you update your sequence (as the originator of the LSA) to what you see, plus 1. Simples. This start-up logic is in the OSPF spec anyway already.
How does OSPF do it?
- For a start they define the 32 bit counter as signed.
- They start at 0x80000001 (-ve max, plus 1).
- They outlaw 0x80000000.
- Comparison is linear, not circular.
- If you wrap to 0x80000000 then you have to prematurely age the LSA at 0x7FFFFFFF, flood, and when all acknowledged you replace with a new LSA with 0x80000001 and flood that - thus creating a brief gap without the LSA.
OK, well, I have coded this, but really, I was so tempted to say WTF, I will be long dead by then. Does nobody actually think when designing this stuff? Alternative design rules eliminate the complexity and make less code, less testing, less special cases, and more reliability.
Oh, and to add to the fun, OSPF metrics are 24 bit but BGP are 32 bit, so mapping these will be, err, fun!
So, really, really, I plan to have FB2500 and FB2700 alpha releases with OSPF for testing this week, maybe tomorrow or Wednesday. This has taken way longer that BGP even writing the whole routing core as well. I started at 6am today and still working!
2012-11-09
Cold be gone
OK, finally, really, nearly lost this cold.
Has to be slowest to go ever, and even today, whilst not on decongestants or paracetamol, I have got through a pack of tissues.
I have to say it really has almost totally gone...
Phew.
Must be getting old, that is all I can say!
Has to be slowest to go ever, and even today, whilst not on decongestants or paracetamol, I have got through a pack of tissues.
I have to say it really has almost totally gone...
Phew.
Must be getting old, that is all I can say!
हम इन वैकल्पिक मॉड्यूल आदेश नहीं था, इसलिए उनके लिए शुल्क मान्य नहीं हैं
Being politically correct is one of those odd things. It should be something that just comes naturally from the way we are brought up.
I am not inherently racist, for example. I will, obviously, make some judgement about a person I am talking to (face to face, on phone, on email, on irc, etc), as we all do. I don't base that on their race, and indeed, in many cases I do not know their race (though names are a subtle clue). If I find someone is dishonest, or thick, or some other trait, I am not going to automatically assume all people from the same place, or who look or speak the same, are the same as they are. So, not a racist, honest.
However, I found myself today suggesting that perhaps there is a language barrier somehow when dealing with a particular intransigent billing department. Somehow, suggesting a language barrier, even with the racist connotations, seemed more polite than suggesting that the person was a total fucking idiot, which is how I was starting to feel. I am not saying they are! I don't know if they just don't understand (because of language or cultural issues) or if they are totally thick or if it is some sort of corporate policy to argue disputes forever. The latter would be odd, as the contract allows us to withhold payment of disputes, so they can't win by prolonging the dispute like this.
There really are only so many times you can explain that the items being billed for were not something we ordered or requested, and as such they are not valid!
We have explained "provide evidence that we ordered it, or agree the dispute and issue a credit" so many times now. They still say "charges are valid". This is even after a meeting with a new account manager - he is trying to sort the issues and is keen, but still we have this crap...
But really, I am not sure what we do next. Arrrrg!
Update: Someone else there is being actually quite sane with his replies - we'll see what happens.
I am not inherently racist, for example. I will, obviously, make some judgement about a person I am talking to (face to face, on phone, on email, on irc, etc), as we all do. I don't base that on their race, and indeed, in many cases I do not know their race (though names are a subtle clue). If I find someone is dishonest, or thick, or some other trait, I am not going to automatically assume all people from the same place, or who look or speak the same, are the same as they are. So, not a racist, honest.
However, I found myself today suggesting that perhaps there is a language barrier somehow when dealing with a particular intransigent billing department. Somehow, suggesting a language barrier, even with the racist connotations, seemed more polite than suggesting that the person was a total fucking idiot, which is how I was starting to feel. I am not saying they are! I don't know if they just don't understand (because of language or cultural issues) or if they are totally thick or if it is some sort of corporate policy to argue disputes forever. The latter would be odd, as the contract allows us to withhold payment of disputes, so they can't win by prolonging the dispute like this.
There really are only so many times you can explain that the items being billed for were not something we ordered or requested, and as such they are not valid!
We have explained "provide evidence that we ordered it, or agree the dispute and issue a credit" so many times now. They still say "charges are valid". This is even after a meeting with a new account manager - he is trying to sort the issues and is keen, but still we have this crap...
But really, I am not sure what we do next. Arrrrg!
Update: Someone else there is being actually quite sane with his replies - we'll see what happens.
2012-11-08
Alan Turing Monopoly
My Alan Turing edition of Monopoly arrived today, from Bletchley Park.
Nice idea, and based on the fact that apparently he played monopoly on a made up board (and lost), and the board drawing was recently found in a loft. A copy of the hand drawn board is included, including a line of property diagonally as an optional route.
The community chest and chance cards are all customised too, which is rather nice.
Only thing I found very disappointing is that the currency is not £... Why?!?!?!
Anyway, I look forward to playing it. We have a tradition of playing this with some friends in the office at Christmas, but I am sure we'll have a chance to play before then.
2012-11-07
OSPF blues
Making progress. I have (subject to lots of testing) reached a major milestone at last, which is that we now manage the LSA database.
This means discovering neighbours (Hello packets), electing the designated and backup router, starting associations with other routers correctly, exchanging LSA headers, loading the database, accepting updates and sending acks, sending flooding updates, originating router and network LSAs when needed, refreshing LSAs we originate, and generally being a part of an OSPF community.
There will be bugs! I have not tried area-border stuff at all. But this is, none the less, a major milestone. Only two key steps left: Injecting routes in to OSPF, and processing routes from OSPF (i.e. doing the SPF algorithm, etc).
What I am working on this morning, and have a lot of progress on for IPv4 at least, is injecting "external" LSAs. i.e. where the FireBrick has some external routes and wants to tell the OSPF world about them. This is carefully controlled - you don't want to leak the full table of BGP in to OSPF it seems. This will mostly be used for L2TP connected routes.
When coding, you have to cover every possible outcome in the code, so you find edge cases that should never happen and you have to cater for them. Sadly I have run in to a snag already.
In OSPFv2 the LSA-ID for an external route is the IP address of the route, but it is allowed to use the "host bits" to differentiate LSAs that overlap. e.g. if you have 10.0.0.0/16 and 10.0.0.0/24 they cannot both use 10.0.0.0 as the LSA-ID. There is an algorithm to try and sort clashes, which is itself messy as it means changing previously announced routes when adding a new one that overlaps.
The snag is that this does not work for all cases. e.g. (and we'll test this on bird later), what if I have 10.0.0.0/32 and 10.0.0.1/32 and 10.0.0.0/31. There are only two LSA-IDs that could be used, 10.0.0.0 and 10.0.0.1 but I have three routes to announce.
Now, one can consider that the 10.0.0.0/31 is not needed as covered completely by the two /32s. That is fine, until one of the /32s is withdrawn. Now you have to reinstate the /31 covering route. This means every time you withdraw a route you have to check for all containing routes that may have not been announced just in case. Arrrrg!
So, once again, OSPF is piling more special cases in my code, which is more to go wrong, and more to test.
Some protocols and algorithms are "elegant" and "simple". I really do start to feel that OSPF is not either of these. Sorry.
P.S. I wonder what bird does with 0.0.0.0/32 as the LSA-ID 0.0.0.0 is reserved for 0.0.0.0/
Update: OK, coding for IPv4 (OSPFv2) working, picking an LSA-ID and handling clashes by removing the covering route where all subordinate space is occupied, and re-instating it when there is a gap. We are treating 0.0.0.0/x the same, so that means we will not use 0.0.0.0 for default route if there is a 0.0.0.0/x route as well, but that is not exactly likely. Now to originate IPv6 routes and that will be next milestone!
This means discovering neighbours (Hello packets), electing the designated and backup router, starting associations with other routers correctly, exchanging LSA headers, loading the database, accepting updates and sending acks, sending flooding updates, originating router and network LSAs when needed, refreshing LSAs we originate, and generally being a part of an OSPF community.
There will be bugs! I have not tried area-border stuff at all. But this is, none the less, a major milestone. Only two key steps left: Injecting routes in to OSPF, and processing routes from OSPF (i.e. doing the SPF algorithm, etc).
What I am working on this morning, and have a lot of progress on for IPv4 at least, is injecting "external" LSAs. i.e. where the FireBrick has some external routes and wants to tell the OSPF world about them. This is carefully controlled - you don't want to leak the full table of BGP in to OSPF it seems. This will mostly be used for L2TP connected routes.
When coding, you have to cover every possible outcome in the code, so you find edge cases that should never happen and you have to cater for them. Sadly I have run in to a snag already.
In OSPFv2 the LSA-ID for an external route is the IP address of the route, but it is allowed to use the "host bits" to differentiate LSAs that overlap. e.g. if you have 10.0.0.0/16 and 10.0.0.0/24 they cannot both use 10.0.0.0 as the LSA-ID. There is an algorithm to try and sort clashes, which is itself messy as it means changing previously announced routes when adding a new one that overlaps.
The snag is that this does not work for all cases. e.g. (and we'll test this on bird later), what if I have 10.0.0.0/32 and 10.0.0.1/32 and 10.0.0.0/31. There are only two LSA-IDs that could be used, 10.0.0.0 and 10.0.0.1 but I have three routes to announce.
Now, one can consider that the 10.0.0.0/31 is not needed as covered completely by the two /32s. That is fine, until one of the /32s is withdrawn. Now you have to reinstate the /31 covering route. This means every time you withdraw a route you have to check for all containing routes that may have not been announced just in case. Arrrrg!
So, once again, OSPF is piling more special cases in my code, which is more to go wrong, and more to test.
Some protocols and algorithms are "elegant" and "simple". I really do start to feel that OSPF is not either of these. Sorry.
P.S. I wonder what bird does with 0.0.0.0/32 as the LSA-ID 0.0.0.0 is reserved for 0.0.0.0/
Update: OK, coding for IPv4 (OSPFv2) working, picking an LSA-ID and handling clashes by removing the covering route where all subordinate space is occupied, and re-instating it when there is a gap. We are treating 0.0.0.0/x the same, so that means we will not use 0.0.0.0 for default route if there is a 0.0.0.0/x route as well, but that is not exactly likely. Now to originate IPv6 routes and that will be next milestone!
2012-11-06
Height of rudeness?
Have you ever had a situation where, in a meeting, you hand someone a formal notice / letter to them (their company), and politely ask them if they could sign the copy to confirm receipt of the letter?
Not unusual practice, surely. I have had that from people before now. It is a formality to sign a receipt, surely...
And they say no!
I mean, why? Does that mean they want to be able to deny having been given the letter later? What? Rude, or what?
Is that the height of (commercial) rudeness?
Or is my then saying "no problem", and picking up the 1Dx and taking a picture of them and the letter, is that the height of rudeness?
Just wondering... :-)
Not unusual practice, surely. I have had that from people before now. It is a formality to sign a receipt, surely...
And they say no!
I mean, why? Does that mean they want to be able to deny having been given the letter later? What? Rude, or what?
Is that the height of (commercial) rudeness?
Or is my then saying "no problem", and picking up the 1Dx and taking a picture of them and the letter, is that the height of rudeness?
Just wondering... :-)
2012-11-05
Yes, BT are serious
Just waiting for the briefing now, but after a long call on the matter BT are quite serious that:-
This means that they charge us for them to ask us for our help regarding a fault.
I really find it hard to believe they have the balls to pull this one. They must realise that it is fundamentally wrong. Surely they realise that there are limits to how far they can push this crap.
I am giving them formal notice of our rates for consultancy for such calls tomorrow as an open contractual offer to them. We will talk to BT engineers to help them fix faults for a charge that we make to them.
Update: BT plc t/a BT Wholesale are disputing the policy change made by BT plc t/a Openreach, and as such will be crediting charges for co-op calls that were not request (on the following bill). Obviously we'll be disputing and withholding the charges anyway so they know why the bill was not paid in full (i.e. we are not lending them money for a money).
- Whilst investigating a fault, a BT engineer can call the customer (us) to discuss it if they deem it necessary. We no longer have an opt-out for that.
- Regardless of how the fault pans out, and even where it is undisputedly a BT fault that we reported, and was fixed by BT, the customer (us) will be charged £35+VAT for the "co-op call" the engineer made to us.
- We are required to provide a contact number for this purpose when booking the engineer.
This means that they charge us for them to ask us for our help regarding a fault.
I really find it hard to believe they have the balls to pull this one. They must realise that it is fundamentally wrong. Surely they realise that there are limits to how far they can push this crap.
I am giving them formal notice of our rates for consultancy for such calls tomorrow as an open contractual offer to them. We will talk to BT engineers to help them fix faults for a charge that we make to them.
Update: BT plc t/a BT Wholesale are disputing the policy change made by BT plc t/a Openreach, and as such will be crediting charges for co-op calls that were not request (on the following bill). Obviously we'll be disputing and withholding the charges anyway so they know why the bill was not paid in full (i.e. we are not lending them money for a money).
Multicast
Moaning about "most complete" broadband, someone has, quite sensibly, asked if we can do multicast. We are a very techie ISP, so why not?
So, here is my reasoning why not, at the moment. It is not totally ruled out, but I do think it will disappear to be honest.
First off, I'll try and explain what multicast is. In a packet network you send packets of data, and this is usually a packet going to one end destination. The Internet as a whole is all about routing a packet to its destination.
However, there are two other modes of operation. One is broadcast which is sending a packet everywhere, and one us multicast which is sending to a set of devices. In practice broadcast is "everywhere on this LAN" or some suitable "broadcast network", and not everywhere on the Internet all at once.
On a LAN multicast and broadcast just work. The switch or hub ensure that every device sees the packet, and the devices filter to get those in which they are interested. Multicast and broadcast are effectively the same for most switches as there is no "layer 2 multicast subscription" process to tell the switches which devices want the multicast traffic. It is done by device drivers configuring the individual hardware to filter for what traffic is wanted. Switches can be smarter and snoop on higher layer protocols in some cases, but its not necessarily a problem to solve as switches and LANs are fast enough anyway.
Where it gets interesting is wide area multicast. The logic sounds simple enough, and sounds like a useful thing. For example, the BBC could stream a TV station as multicast, sending it just the once to a peering ISPs. The ISP duplicates the packets to its links, and so on to its customers. Customer routers send the traffic as mulicast packets on the final LAN and multiple devices can see the stream in real time.
The problem, as an ISP, is simple. We would have to copy the packets to each customer. That is technically a pain, and does not save us any money as most of the cost is the link to customers. It does reduce some traffic on peering to the likes of BBC but that is not as costly.
There is, technically, a way to do multicast on L2TP, which would address traffic. I don't think BT support it yet (or anyone else), and I can imagine the fun they would have billing for it. But it would mean a packet to each BRAS, and there are hundreds of them. This would start to have some saving as long as we commonly get two or more people on a BRAS streaming the same traffic at the same time. At present that is simply not going to happen.
But could there be enough demand if ISPS did multicast? I am not convinced. The problem is that there are already very good technical means to distribute the same content to many - using radio and satellite. It works, and is cheap, and already in use.
But could we make multicast better? If the way broadband worked is proper routers and switches at an exchange level, multicast could just be one packet in to BT and that duplicates and spreads at each aggregation point and efficiently gets to every device that wants it. Yay!
But hang on - the big things now are catch-up TV and on-demand TV. The content delivery networks are providing this at an application level. Colocating kit at ISPs and closer to the end users. Eventually you can see how CDNs may have boxes at an exchange level to serve the unicast streamed content. There are already systems to include content at the BRAS level within BT.
If they do that, they can "multicast" efficiently using unicast traffic at local nodes. Once you have that you don't need the layer 2/3 multicast systems.
So, at the moment, it is simply not worth the effort. I think if we get to the stage where it is worth the effort the CDNs will be able to do it way better than we could by some means.
Multicast on the LAN will stay - but basically, I think the idea of wide area multicast being done at layer 2/3 is probably the wrong way to do it, and that is why it is not taking off. The right way to do it is application level live content distribution and recorded content delivery platforms. IMHO
I'll keep an eye on it though, you never know.
So, here is my reasoning why not, at the moment. It is not totally ruled out, but I do think it will disappear to be honest.
First off, I'll try and explain what multicast is. In a packet network you send packets of data, and this is usually a packet going to one end destination. The Internet as a whole is all about routing a packet to its destination.
However, there are two other modes of operation. One is broadcast which is sending a packet everywhere, and one us multicast which is sending to a set of devices. In practice broadcast is "everywhere on this LAN" or some suitable "broadcast network", and not everywhere on the Internet all at once.
On a LAN multicast and broadcast just work. The switch or hub ensure that every device sees the packet, and the devices filter to get those in which they are interested. Multicast and broadcast are effectively the same for most switches as there is no "layer 2 multicast subscription" process to tell the switches which devices want the multicast traffic. It is done by device drivers configuring the individual hardware to filter for what traffic is wanted. Switches can be smarter and snoop on higher layer protocols in some cases, but its not necessarily a problem to solve as switches and LANs are fast enough anyway.
Where it gets interesting is wide area multicast. The logic sounds simple enough, and sounds like a useful thing. For example, the BBC could stream a TV station as multicast, sending it just the once to a peering ISPs. The ISP duplicates the packets to its links, and so on to its customers. Customer routers send the traffic as mulicast packets on the final LAN and multiple devices can see the stream in real time.
The problem, as an ISP, is simple. We would have to copy the packets to each customer. That is technically a pain, and does not save us any money as most of the cost is the link to customers. It does reduce some traffic on peering to the likes of BBC but that is not as costly.
There is, technically, a way to do multicast on L2TP, which would address traffic. I don't think BT support it yet (or anyone else), and I can imagine the fun they would have billing for it. But it would mean a packet to each BRAS, and there are hundreds of them. This would start to have some saving as long as we commonly get two or more people on a BRAS streaming the same traffic at the same time. At present that is simply not going to happen.
But could there be enough demand if ISPS did multicast? I am not convinced. The problem is that there are already very good technical means to distribute the same content to many - using radio and satellite. It works, and is cheap, and already in use.
But could we make multicast better? If the way broadband worked is proper routers and switches at an exchange level, multicast could just be one packet in to BT and that duplicates and spreads at each aggregation point and efficiently gets to every device that wants it. Yay!
But hang on - the big things now are catch-up TV and on-demand TV. The content delivery networks are providing this at an application level. Colocating kit at ISPs and closer to the end users. Eventually you can see how CDNs may have boxes at an exchange level to serve the unicast streamed content. There are already systems to include content at the BRAS level within BT.
If they do that, they can "multicast" efficiently using unicast traffic at local nodes. Once you have that you don't need the layer 2/3 multicast systems.
So, at the moment, it is simply not worth the effort. I think if we get to the stage where it is worth the effort the CDNs will be able to do it way better than we could by some means.
Multicast on the LAN will stay - but basically, I think the idea of wide area multicast being done at layer 2/3 is probably the wrong way to do it, and that is why it is not taking off. The right way to do it is application level live content distribution and recorded content delivery platforms. IMHO
I'll keep an eye on it though, you never know.
2012-11-03
Unlimited but slow?
This is not really a dig at BT this time, honest. More about the whole perception of "internet speed" that the public, and clearly advertisers, have...
The advert is a halloween party where they are trying to download music (presumably legally) but the internet is "unlimited, but slow". They are unable to play *any* music for the party, having apparently spent ages preparing the play list.
Apart from the fact you would have downloading said "play list" when preparing it, and so not be relying on the Internet, what puzzles me is that even a high data rate audio file is around 300Kbps, which would be incredibly slow for an internet connection. Under half a percent of the BT based broadband lines we operate run at 250Kbps or less, which is enough for a more normal MP3 file, which can be played as soon as you start downloading it. Indeed, the BRAS rate below that, 135K, which is still just about fast enough, is considered to be a fault.
So what sort of "unlimited, but slow" internet connection is it that they are referring to?
Is there really any UK commercial Internet service too slow to download an MP3 faster than you can play it? Apart from dialup?
What am I missing here?
The advert is a halloween party where they are trying to download music (presumably legally) but the internet is "unlimited, but slow". They are unable to play *any* music for the party, having apparently spent ages preparing the play list.
Apart from the fact you would have downloading said "play list" when preparing it, and so not be relying on the Internet, what puzzles me is that even a high data rate audio file is around 300Kbps, which would be incredibly slow for an internet connection. Under half a percent of the BT based broadband lines we operate run at 250Kbps or less, which is enough for a more normal MP3 file, which can be played as soon as you start downloading it. Indeed, the BRAS rate below that, 135K, which is still just about fast enough, is considered to be a fault.
So what sort of "unlimited, but slow" internet connection is it that they are referring to?
Is there really any UK commercial Internet service too slow to download an MP3 faster than you can play it? Apart from dialup?
What am I missing here?
2012-11-01
Adverts
Basically, I think I'll leave them. Those that don't like them can ignore them or adblock them, and some people do. No problem, and they are not in the rss feed either. The ads make a few pence some days and much more others. Only tens of pounds a month though. I think the issue is that few people click on the adverts (understandably).
There is, however, something vaguely gratifying that when I rant about BT, that BT are actually paying me to put adverts on my blog next to the rant about them.
[BT BT BT BT BT BT BT BTW Openreach BT Retail Fibre Telephone Broadband British Telecom plc BT plc BT BT BT BT]
That alone makes it all worth while.
Rumous of my birthday are greatly exaggerated!
Facebook thinks it is Thrall's birthday. Basically, when we set up the account we tried to work out when WoW was first released and so when Thrall was "born". We picked a date based on some googling.
The catch is that loads of people think it is my birthday today - and they should know I am not 18, far from it.
It is also the OS/2 drinking club this evening, and I want to apologise to anyone hoping I would make it. This damn cold is lingering, and whilst mostly over it, I would rather stay at home tonight. However, I really really plan to make next month and have put in the diary even. I am trying to convince Alex and Kev to go as well.
Anyway, still plowing through OSPF. Right now I am at the stage that I hope I never meet the author of RFC2328, at least not on a dark night in an alley, as he might regret it. Chapter 13 is loaded with exceptions. I really think not written by a coder, sorry. But I do have lots in place already and should have flooding the LSAs tomorrow, with any luck, which is a major step and my next major milestone.
The catch is that loads of people think it is my birthday today - and they should know I am not 18, far from it.
It is also the OS/2 drinking club this evening, and I want to apologise to anyone hoping I would make it. This damn cold is lingering, and whilst mostly over it, I would rather stay at home tonight. However, I really really plan to make next month and have put in the diary even. I am trying to convince Alex and Kev to go as well.
Anyway, still plowing through OSPF. Right now I am at the stage that I hope I never meet the author of RFC2328, at least not on a dark night in an alley, as he might regret it. Chapter 13 is loaded with exceptions. I really think not written by a coder, sorry. But I do have lots in place already and should have flooding the LSAs tomorrow, with any luck, which is a major step and my next major milestone.
BT shareholder asking why pay someone to wind up customers?
Here is a specific example, which is just typical.
Last November (yes, that is a year ago), we reported a fault. BT sent two engineers. The engineer did an "uplift of network for ADSL", which means they changed the wiring in BTs network (moved to another pair).
BT billed us!
As normal, where BT have done work to fix the fault on their side of the network we dispute the charges. We sent the dispute and withheld the charges.
In total, they charged £245, which is two lots of £50 "End user wiring module", one £50 "End user equipment module", and £95 "base" module.
This is odd for a start. If they did the "End user wiring module", then how could they charge to do it a second time a few days later unless the first guy did not do it right? In fact, the fact BT did not charge the "base module" for the second engineer which means they agree it was a BT fault already, as it is only free if there is a BT fault and they find it. That means they should not have gone on to do wiring and equipment modules anyway. The charges make no sense in the first place!
Anyway, it did not bother us. We had disputed and withheld payment (as per contact terms) so we did not need to do anything. As we have explained to BT many times, we are happy for a charge to just stay in dispute until written off under the 6 year Limitations Act.
Two months ago they started chasing asking why the bill from November 2011 was underpaid. We explained, again, and sent details of the disputes, again.
Last month (11 months after the fault) they credited us £209. Odd, given we disputed £245. There was no explanation of this discrepancy. There were several disputes and many were under credited by some random small amount.
They threatened that they would send a "formal breach of contract notice" if we did not pay the balances of these various disputes (including the £36 difference in this case). We pointed out they had incorrectly failed to credit the full amount of these agreed disputes.
They then agreed to credit a further £1. Actually they say they are "waiving" the £1 as if it is some good will gesture on their part rather than a cock up. This leave £35 which they say is "valid" and is for a co-op call.
We queried that, pointing out we did not ask for a co-op call (we have no record of a call, as it happens). We are told that in August 2012 there was a change in policy in BT allowing engineers to decide to make co-op calls and for which we have to pay.
Hang on? That dubious change of policy, for which we have no record of any notice or agreement on our part, is 7 months after the fault. So how is that relevant.
And hang on? We were not charged for a co-op call. We were charged for two equipment modules, a wiring module and a base module. They are clearly itemised on the bill, and different prices to a co-op call, so no ambiguity. So, what co-op call?
So we go back, explaining that the charges for this are not valid, again, and we are just told "Please check the previous attachment where we have already explained the Engineer actions and the reason for the charges. We have fully investigated for each and every circuits, mentioned in the attached excel sheet and have added our findings." [sic], and no attachment either but previous one just says "Charge of £35 is valid" and "Engineer made co-op call".
That's it! They have told us they are valid. End of story!
Well sorry, but no! The charges are still in dispute as far as we are concerned. We have advised BT that the charges are not valid. We have offered to provide a Special Billing Investigation where we would extract the XML orders and confirmations to prove the case. We have offered to consider any actual evidence BT have to support their position. All we have so far is an assertion from them that the charge is valid. We assert it is not. So, in the mean time, still in dispute, and still withheld.
How is it that they think this is sensible?
As a BT shareholder myself I have to ask why it is sensible to pay someone to wind up customers like this. It must cost time and effort and gets nobody anywhere. I may have to go to the shareholder's meeting and ask. Yes, I am a shareholder in BT Group plc (sole owner of British Telecommunications plc) as I have one 5p share.
P.S. I have emailed Ian Livingstone asking about this now.
Last November (yes, that is a year ago), we reported a fault. BT sent two engineers. The engineer did an "uplift of network for ADSL", which means they changed the wiring in BTs network (moved to another pair).
BT billed us!
As normal, where BT have done work to fix the fault on their side of the network we dispute the charges. We sent the dispute and withheld the charges.
In total, they charged £245, which is two lots of £50 "End user wiring module", one £50 "End user equipment module", and £95 "base" module.
This is odd for a start. If they did the "End user wiring module", then how could they charge to do it a second time a few days later unless the first guy did not do it right? In fact, the fact BT did not charge the "base module" for the second engineer which means they agree it was a BT fault already, as it is only free if there is a BT fault and they find it. That means they should not have gone on to do wiring and equipment modules anyway. The charges make no sense in the first place!
Anyway, it did not bother us. We had disputed and withheld payment (as per contact terms) so we did not need to do anything. As we have explained to BT many times, we are happy for a charge to just stay in dispute until written off under the 6 year Limitations Act.
Two months ago they started chasing asking why the bill from November 2011 was underpaid. We explained, again, and sent details of the disputes, again.
Last month (11 months after the fault) they credited us £209. Odd, given we disputed £245. There was no explanation of this discrepancy. There were several disputes and many were under credited by some random small amount.
They threatened that they would send a "formal breach of contract notice" if we did not pay the balances of these various disputes (including the £36 difference in this case). We pointed out they had incorrectly failed to credit the full amount of these agreed disputes.
They then agreed to credit a further £1. Actually they say they are "waiving" the £1 as if it is some good will gesture on their part rather than a cock up. This leave £35 which they say is "valid" and is for a co-op call.
We queried that, pointing out we did not ask for a co-op call (we have no record of a call, as it happens). We are told that in August 2012 there was a change in policy in BT allowing engineers to decide to make co-op calls and for which we have to pay.
Hang on? That dubious change of policy, for which we have no record of any notice or agreement on our part, is 7 months after the fault. So how is that relevant.
And hang on? We were not charged for a co-op call. We were charged for two equipment modules, a wiring module and a base module. They are clearly itemised on the bill, and different prices to a co-op call, so no ambiguity. So, what co-op call?
So we go back, explaining that the charges for this are not valid, again, and we are just told "Please check the previous attachment where we have already explained the Engineer actions and the reason for the charges. We have fully investigated for each and every circuits, mentioned in the attached excel sheet and have added our findings." [sic], and no attachment either but previous one just says "Charge of £35 is valid" and "Engineer made co-op call".
That's it! They have told us they are valid. End of story!
Well sorry, but no! The charges are still in dispute as far as we are concerned. We have advised BT that the charges are not valid. We have offered to provide a Special Billing Investigation where we would extract the XML orders and confirmations to prove the case. We have offered to consider any actual evidence BT have to support their position. All we have so far is an assertion from them that the charge is valid. We assert it is not. So, in the mean time, still in dispute, and still withheld.
How is it that they think this is sensible?
As a BT shareholder myself I have to ask why it is sensible to pay someone to wind up customers like this. It must cost time and effort and gets nobody anywhere. I may have to go to the shareholder's meeting and ask. Yes, I am a shareholder in BT Group plc (sole owner of British Telecommunications plc) as I have one 5p share.
P.S. I have emailed Ian Livingstone asking about this now.
Subscribe to:
Posts (Atom)
FB9000
I know techies follow this, so I thought it was worth posting and explaining... The FB9000 is the latest FireBrick. It is the "ISP...
-
Broadband services are a wonderful innovation of our time, using multiple frequency bands (hence the name) to carry signals over wires (us... -
For many years I used a small stand-alone air-conditioning unit in my study (the box room in the house) and I even had a hole in the wall fo... -
It seems there is something of a standard test string for anti virus ( wikipedia has more on this). The idea is that systems that look fo...