We do unfiltered Internet access - simple.
But is it?
We do BCP38 on all end user lines, but reasonably sensibly allowing any source IPs from any of a customer's lines including 2002::/16 equivalent IPv4 in IPv6 addresses, and so on. This is part of the definition of "Internet Access" in our terms. We think it is sane and is obviously "Best Current Practice". You get Internet Access for (and from) your IP address(es).
But what of traffic from the Internet to end users. Until now this has been totally unfiltered. This includes spoofed source IP traffic such as from RFC1918 addresses, and unrouted blocks, and traffic used for amplification attacks, TCP SYN flood attacks and so on.
We are, today, putting in place a simple additional source IP check on ingress from transit. It is simply that the source is routable. This covers a lot of duff traffic, e.g. RFC1918 source addresses.
We think this is sensible and a simple step to avoid some of the spoofed attacks. But we offer "unfiltered Internet". Is that right that we filter this. Do customers have a right to spoofed source traffic from the Internet to their line?
I am genuinely interested in feedback on this...
There is then a further, operational aspect. We have seen significant attacks since Saturday with TCP port 80 SYN floods with a variety of valid and invalid TCP options which appear to be designed to crash TCP stacks. They are working on lots of our customer to crash very old ZyXEL P660 routers where we have external access to administer the router. This is not a password attack, but an attack on the routers TCP stack and takes customers off line totally, sometimes for hours.
The source is a fixed single IP - possibly the real source or possibly the target or some reflection attack. So, with our new filtering we could black hole that /32 address which will now cause blocks to incoming packets from that address. We would use this specifically to mitigate attacks on customers during the attacks or the overall period that attacks are happening (i.e. may be several days).
But that is not unfiltered Internet is it?
It is a specific administrative and short term filter, but it is a filter?
So is that valid? Should we do it? We are talking about only specific cases for attack conditions. It is, by no means, a general means to filter stuff, and very very much not a URL filter.
Comments please?
P.S. Whatever we do - we aim to be completely transparent - if we do block an IP for an attack we will say why and for how long and so on...
Update: In light of some feedback I have disabled this source check. At least we have the feature now if we need to do something in an emergency to protect the network as a whole, i.e. during an attack.
Subscribe to:
Post Comments (Atom)
Don't use UPS to ship to UK
I posted about shipping and importing and tax and duty - general info. But this is specific. DON'T USE UPS! I had assumed the UPS issue ...
-
Broadband services are a wonderful innovation of our time, using multiple frequency bands (hence the name) to carry signals over wires (us... -
For many years I used a small stand-alone air-conditioning unit in my study (the box room in the house) and I even had a hole in the wall fo... -
It seems there is something of a standard test string for anti virus ( wikipedia has more on this). The idea is that systems that look fo...
I think the routable thing is just good overall, you should definitely do that.
ReplyDeleteThe blocking attacks is reasonable if deployed responsibly.
I wouldn't like to see eg "we block port X on all our connections as it's abuse" as some providers do.
But reasonably applied blocks for specific attacks I'm all in favour of.
This is much cruder than blocking ports.
DeleteI suggested this on the ideas site but ill reiterate myself.
ReplyDeleteWhy can't we have filtering options available on the control pages. That way we can opt in or out of your filters and add our own for a localised attack.
By default customers could be opted into the "sensible" filters but could still disable if they really wished.
What he said ^
DeleteDitto. Legitimate research such as Honeypots need access to unfiltered feeds. Provide some "sane" (define how you like) filtering for the uninitiated and allow that to be turned off through the control pages if the customer really understands what they're doing.
DeleteGeneral purpose filtering, i.e. a configurable firewall, is a different thing. What I am talking of here is a simpler routing check. Running a hosted firewalling service is something we have considered, but not something we are really looking to offer at present.
DeleteAny particular reason why you don't want to offer this? Seems it would be a superb USP.
DeleteA general firewall is a much more complicated arrangement at technical level, and in terms of control pages and support. We currently have routers which route. No session tracking and no firewall rules. We may consider a separate user controlled firewall for selected users one day perhaps, but as this would be extra cost, and potentially single points of failure, etc, it is likely to cost, and, apart from DoS type scenarios, it is no better than a user controlled firewall at their premises. Offering DoS protection would be an even bigger job and not simply a user controlled firewall.
DeleteI think for attacks that you know beyond a shadow of a doubt that are malicious that you should be blocking them. Many customers of ours wouldn't have a clue how to sort issues like this out. Maybe an active searchable database of addresses you are blocking would help...
ReplyDeleteI guess one question would be how much extra latency the additional filtering has on the packets and whether it would be notisable to joe blogs.
As I've asked you in the past to filter an attack/misconfiguration, I have to say that it could be a good thing. Transparency is the key.
ReplyDeleteI think you are doing exactly right with the routeable check and I think block the single IP is a good thing as well. Putting the details on your webpage is right too. If you really want a gold star have an opt-out ability
ReplyDeleteI'm in favour of both the routable check and the IP block along with publicity of doing so.
ReplyDeleteFor a gold star have an opt-out mechanism
I'll second the support for filtering non-routable source IPs on ingress - it's not as though you can hold much of a conversation with someone who can't route packets back to you. I filter the RFC1918 addresses on my firewall anyway so it doesn't make much difference to me for those addresses.
ReplyDeleteI also fully support filtering out any packets sent by a customer which aren't from "their" IP addresses - the only thing I'd ask here is whether there's a mechanism for letting a customer know that this was needed - partly because it might highlight a network set-up problem (I think I once let a few RFC1918 addresses out myself for a few days before I spotted the problem) or worse some bit of malware spitting out packets.
Blocking attacks - definitely a greyer area. I'm sure most of your customers understand that unfiltered means "warts and all" but it makes sense to turn a sustained attack away at your border as that saves bandwidth for everyone not just the affected customer(s). In the case you mentioned I'm sure that it also saves your sysadmin effort in sorting routers out, contacting customers etc and again that benefits everyone because you aren't totally tied up by one thing.
So - yes, if it's transparent, customers informed, put up on the status pages I would give it a thumbs up.
But I'd probably leave it at that.
Otherwise I
> Do customers have a right to spoofed source traffic from the Internet to their line?
ReplyDeleteI suppose if they actively ask for it they should get it (research project perhaps?)
Blocking attack sources is sensible network administration and good customer service. Just do it.
ReplyDeleteHowever, I use Claranet specifically because they DO block certain content, I subscribe to their Cleanband based ChildSafe service.
I just do not want a load of porn to be accessible by my family (or myself for that matter) and so I outsource this service because I don't have the time to do it all myself.
Whilst I love that AA do not mess around with things, I do like to have the option to turn content filtering on.
Content filtering is a very different thing and not really something that makes sense at the ISP, IMHO.
DeleteI don't disagree completely, but is it a service that people seem to want. The alternative is a router based content filter (because installing things on all your devices is just too much of a pain and not really workable).
DeleteI've yet to see a decent home router that has any usable content filtering except for DNS based stuff, which is a bit naff really.
So right now, it looks like the only available place for content filtering that covers an entire connection (i.e. a single service to a premises) is network based. Maybe not ideal, but it does work.
Getting your ISP to censor your internet connection sounds to me like "messing around with things" lol
DeleteAn optional and configurable ISP side firewall is a great feature and possible differentiator that I'd really value. However I don't welcome any mandatory filtering and I worry that there's potential for any automated blocking responses to be used to create a DoS. Please make filtering optional, if for no other reason than to allow me to temporarily disable it when I'm investigating problems.
ReplyDeleteI've also found it interesting to plot attacks on my network and perform statistical analysis as a security researcher. You've just made that impossible for me and you did so without warning.
Regarding upstream source filtering, I've always assumed that I could extend the range permitted if I could justify this, for example if I had an additional IP address range that I was entitled to use from another ISP. However I'd really prefer that you terminated users who abuse the service rather than filtering those of us who don't. There are times when I've wanted to spoof another IP that I "control", perhaps to use with sendip for testing a firewall and I've been frustrated by the filtering A&A have implemented.
How would you feel if your transit providers started to implement filtering? I essentially regard A&A as my transit provider. Feel free to enable filtering by default; but PLEASE let me opt out. Thanks.
It is off now. It could be used to protect the network in the event of an attack, obviously.
DeleteAs for source filtering from customers, that is BCP38, but yes, it is possible to turn that off on a per line basis if someone has some justification for needing that.
Basically the feature is now switched off. I may be able to do something with a crude feature like this as an egress filter on a per line basis in the future, but it is very crude anyway and probably not that useful.
ReplyDeleteIt is, however, another tool available to us if we ever need to take any emergency measures to protect the network as a whole - i.e. when there is some sort of attack.
I think of the internet as a community of interconnected networks that agree to route traffic between each other. I think it's a vital feature of that concept that any network operator can decide that the behaviour of another network is unacceptable and stop routing their traffic. That applies to both immediate BGP neighbours, by ending relationships, and to remote network by filtering by AS (for example).
ReplyDeleteAn example of what I mean is where a network is originating vast quantities of spam, because they're providing hosting to spammers. In that situation the internet community can respond by deciding to no longer accept traffic from that network, voting them off the internet. That's a sensible consensus way of dealing with antisocial behaviour.
What you're suggesting is not accepting traffic from a particular source, as it is behaving in an abusive fashion and being disruptive to the internet community. I'd say that's perfectly acceptable. I'd say you should go further and notify the source network's abuse contacts (I'd guess you've done this anyway), and if they don't deal with the problem, drop their routes entirely, as they're not playing fair.
More pragmatically, the situation is that a source is sending traffic which knocks some of your customers off the internet for a significant interval. You have a choice between filtering, which results in a mildly filtered internet for all customers, or not filtering, which results in unfiltered internet for some percentage of your customers, and no internet for the remainder. If the quantity of disruption from the latter would be larger than for the former then you should filter. That filtering should be considered a short term response to an immediate threat, reviewed regularly, and followed up by abuse reports etc.
Short version: I don't view sensible responses to clear and present threats as "filtering", they're a normal part of network operations.
You of course already filter internet data. You only deliver to me packets that are addressed to an IP address that I "own" and not all packets that you have even if I might be interested in receiving packets address to other IP addresses. Given that you filter based on destination address for very valid and worthwhile reasons already the discussion is not about filtering in principle, it's about *which packets should you attempt to deliver to me*.
ReplyDeleteI see no reason for you to ever forward packets to or from me that are -
Outgoing packets with someone else's source IP address.
Incoming or outgoing packets that are malformed in any way.
Incoming packets that you can identify as having spoofed IP source addresses.
Incoming packets that have no purpose other than to act as a DOS attack.
I don't regard this as filtering in the censorship sense, or any restriction on what I can do. I consider this to be the service I want from you. "To deliver any valid data packet send to or from me"
I agree with several people who say that filtering in self-defence is not only fine, but essential! Allowing aome scumbag to disrupt your customers' Internet links is not right, so I feel you should filter where it is obvious, unless the customer has opted out.
ReplyDeleteSo I think an option on the control pages to allow "Completely unfiltered" and possibly a middle-ground "Filter suspected attack-sources" would be a sensible course of action.
Cheers, Howard
Non-routable IPs aren't part of the internet. Job done there.
ReplyDelete+1 for an opt-out safety filter (no clever options, just on/off)
I'd vote for full bogon filtering: https://www.team-cymru.org/Services/Bogons/
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteThis comment has been removed by the author.
ReplyDeleteA question - could you use usernames to opt-out of some of the filtering?
ReplyDeleteE.g. "user@a.1" users get full filtering - routability checks, "bad" traffic, everything you do.
"filter+none-user@a.1" gets no filtering, even if you add more in future.
"filter+routable-user@a.1" gets routability checks only.
"filter+attack-user@a.1" gets your filtering of "bad" traffic.
"filter+routable+attack-user@a.1" gets filtering of both unrouteable source IPs and "bad" traffic.
I'd be happy for anything that makes what's suuposed to work work, and what's not supposed to work not work. ISP-side firewalling would be good - maybe it could even include an "opt in" to some kind of master list of problem IPs.
ReplyDeletecgn-user@a.1 ;) ?
ReplyDelete/me laughs out loud.
what is the state of play now? Iress filtering from transit? BCP 38?
Delete