Between December 1, 2009 and January 15, 2010 *our* (MLD/CMI/BeagleHost/RAW) servers have been under a massive load (load = Denial of Service (DoS), Syncronized DoS, Flood (scan, slam, spam, ping), Abuse (script/form injection) )
At the 'peak' of this 'load/attack/whatever' there were over 1,500,000 internet 'connections' to our four 'front end devices' - 6,000,000 total 'connections' per day from approximately Dec 3 until Dec 15. As of Jan 17th, after a massive effort, the total connections at the 'front end devices' are now under 300,000 per day. These totals do not include 'legitimate user connections' (checking mail etc).
The logs created and parsed to generate the now 'highly effective blacklist' during the 6 weeks total 101GB.
'The Blacklist' is a built-up list of internet IPs (individual) and complete ranges (blocks x.x.x.x/mask) that are completely 'black-holed' from accessing 'the servers.' Some of the 'bans' on this list are permanent, some are only for a few hours or days.
'The Blacklist' on January 17, 2010 aggregates to approximately 55,000,000 individual IP addresses. The 'system' that has been created over the last 6 weeks to identify and add to the list is adding approximately 2,000 to 10,000 IPs per day to the list.
'The System' Summary: 2 Router/Firewall Devices; 2 64-Bit CentOS servers; 1 32-Bit Gentoo server; 1 64-Bit Suse server; 1 32-Bit Windows XP Pro 'server' (syslog host, programming/scripting 'platform'); 5 'internet connections.' [Not included since 1.5: RAW: 2 Windows 64-Bit servers; 1 more 'internet connection' and some firewall/router(s). RAW moved 'off-site' to 'split the load.' One RAW WindowsSBS server remains in-place - very low use and not contributing to the 'blacklist/stop-flood' system.]
Man-Hours: DLW/MLD/CMI/Beagle - 800+; RAW 200+; 6 weeks. Hours spent: building/adding servers; installing/customizing software; writing customized cross-platform scripts/software; telephone/email time w/customers; documentation; server monitoring (and rescue); verification of validity of ban(s)/blocks.
No data-loss or user-data-leaks have occurred. Total 'down-time' during 6 weeks is less than 1 hour (99.99% uptime standard).
How It Works (simplified, as if 1 server/connection; too many color diagrams for 'real world'):
What the 'blacklist' really is: It is a list of IPs and Ranges of IPs that are used at the Firewall level to completly ignore (blackhole) all communications from any computer (IP) on the list. For basic understanding: it is as if they called a phone number and it just rang forever. They get no response (indication) that there is even a server or computer at our IPs. The 'blacklist' is currently stored in a way that all of our computers and devices share the list - all 'bans' and 'unbans' are utilized and contributed to by all servers/devices.
Very Imperfect - This is an 'overly fair' way to build a list - allowing multiple-by-multiple 'abuses' before long term blacklisting. This 'method' allows for a complete DoS attack (possibly overloading, or 'taking down' a server) to occur before locking them out. This is an 'inherent' problem with 'publicly available servers' and there is no way to know in advance the 'intentions' of one computer in the whole world. The severity of the abuse is weighed and the reaction (service block, short ban, permanent ban) is equally weighted.
How would *one* know they were on your list? The *one* would know because they could not email (their ISP would say connection refused or host not found); they could not read this web-page; they would get *nothing* from/to here.
What if *someone* got a virus, or it was someone else at their ISP that caused you to blacklist them? Will you 'un-blacklist/un-ban' them manually? Part A: They'll have to get rid of their virus, or contact their ISP to figure out what is going on. The 2nd Level of the blacklist here is a 10-day ban, so they'll have to wait out the 10 days for the auto-un-ban. The 'Big Boys' (AOL, yahoo, google, msn/hotmail) have published and effective systems of their own, and have not been responsible for any of the direct abuse to our servers yet. Part B: Probably not. Maybe with USPostal mail signed requests or some of the craziness and 'proof' that other server companies or ISP companies have. The only way for any computer/network to get on *our* list is to have repeatedly and destructively 'abused' or 'allowed abuse to happen' to our servers and customers. Dealing with this most recent 'tsunami of internet abuse' has cost us thousands of dollars, hundreds of hours and endless grief. Maybe for signed apologies, admissions of negligence and/or guilt, and financial remuneration we would 'manually lift a ban' or 'remove an IP/network' from 'the list.'
Is this list available for others (isps [small or large], corporate networks, spam/virus utility makers, free?)? As of January 17, 2010 - NO. This is a private list assembled as 'reaction' to abuse against our network(s) and customers. We: have not researched the *legal* ramifications of distributing this list (if any); have not documented a way for other(s) to *use* the list (contribute to or honor expirations on bans); have not calculated a fee to license 'the work' (we like all companies like to get paid for work we do); are still in the 'ongoing' battle to complete the system against the abuse that caused it 'to be' in the first place (W.I.P.). This 'system' is relatively 'small-in-scale' and would be (probably) badly misplaced in a *super-giant* ISP/network/multi-domain(>100) environment. After January 31, 2010: we will post more information about 'efficacy, distribution, etc' of 'the system.'