Editor’s note: Ofer Gayer is a security researcher at Incapsula.
It was late December when we were contacted by a financial service provider who began noticing a spike in online registration requests. Rather than resulting from end-of-the-year sales efforts, this spike was caused by a targeted spam attack that flooded the organization with fake registration forms, most of which looked reputable enough not be discarded on sight.
The reason for concern was that this company, for internal reasons, needed to perform a manual background check of each new registration form before it was passed over to the sales department. So when the small background check team began to collapse under the deluge of seemingly legitimate online forms, all online sales operations were brought to a halt. After a quick examination of the company’s website traffic, it quickly became clear that what the customer was actually experiencing was malicious bot activity.
In this case, the bot operator knew enough about the organization to identify the manual background check process as a “soft spot,” which could be used to monkey-wrench the company’s entire sales funnel.
To exploit that weak spot, the perpetrator hand-crafted a spam bot whose sole purpose was to attack that one specific registration form with details that wouldn’t fit any easily identifiable pattern. Most concerning, these bots were disguising themselves as regular human users, with browser-like HTTP fingerprints and several capabilities that enabled them to circumvent the website’s challenge-based, access-control mechanism.
This is not the first time we’ve come across such sophisticated bots that are purpose-built to mimic human behavior and operate under the cover of a browser-like identity. Collectively, we like to refer to them as “impersonator bots.” Created for stealth and preferred for their ability to bypass commonplace security measures, such automated tools are used by hackers not only for spam attacks, but also to steal data, hijack servers and execute DDoS attacks, among other nefarious activities.
Who are these Impersonator bots?
Many types of bad bots roam the Internet, from scrapers and spammers to the more sophisticated vulnerability scanners and DDoS bots. If other bad bots can be compared to well-trained soldiers carrying out the orders of their commander, impersonator bots are the Special Ops unit. These “commandos” carry out the same malicious activities, but they do so covertly and typically use much more advanced attack techniques.
Often, they’re based on existing malware tools, modified to create a browser-like HTTP fingerprint. This lets them bypass security challenges that would stop a lesser/generic version. Impersonator bots cause significant damage to companies’ websites and web applications, resulting in downtime, financial losses and reputation damage.
Between 2013 and 2014, we saw overall bot traffic volumes decrease from 61.5 to 56 percent of all web visits — a reversal of the upward trend observed the prior two years. Still, despite the dip in total bot traffic, the number of impersonator bots continues to grow.
In fact, over the past three years, impersonators are the only bad bot type to display consistent growth, which does not bode well for most website owners.
Cybersecurity is often described as an arms race, and for a good reason. Hackers and white hats are continually trying to stay one step ahead of each other. When one side finds a better method of defense, the other side usually develops a smarter type of offense. Impersonator bots are a byproduct of this escalation – they are the hacker’s response to an increased use of anti-bot solutions by website owners.
Impersonator bots are the go-to tool for hackers
One way impersonator bots are used is for vulnerability scanning and automated hacking attempts. Such “hacker bots” are proprietary tools and scripts are used to systematically scan sites for vulnerabilities and exploit them at will and in bulk. As soon as a vulnerability is published, the scavenger hunt for an un-patched system is on.
We saw an excellent example of this dynamic following the discovery of the Shellshock mega-vulnerability in September 2014.
Soon after Shellshock’s discovery and the release of a patch, we saw an explosion in scanner traffic. Some of these were legitimate scanning attempts by concerned Internet citizens. However, more than 90 percent of the bots were malicious scanners and other malicious automated tools (e.g. DDoS malware) probing for the Shellshock vulnerability.
For hackers, launching such vulnerability-scanning campaigns is just “another day in the office.” We’ve seen this same dynamic following other major vulnerabilities in 2014, such as Heartbleed, and vulnerabilities in popular WordPress plug-ins like Slider Revolution and FancyBox.
This genre of impersonator bots also covers DDoS bots coming from anonymous proxies, which are simply another way for attackers to mask their true identities. This is the same MO used by impersonator bots. In fact, over the past few months, we’ve seen a significant increase in bots using TOR and other publicly available anonymous proxies to perform application-layer DDoS attacks (e.g. HTTP floods). These proxies were created to enable anonymous web browsing — substituting a users’ IP address with that of an untraceable proxy.
The use of anonymous proxies (most of which are free) holds many benefits for DDoS attackers. This enables them to mask their bot IPs, letting them bypass security solutions based on blacklisting. Rather than using a single address for each bot request, anonymous proxies spread requests among multiple IPs, permitting them to fly under the radar of rate-limiting mechanisms.
In addition to hiding IPs, anonymous proxies also obfuscate header information, enabling them to evade security measures based solely on HTTP fingerprinting. Utilizing these inherent benefits, perpetrators are able to create a large botnet-style impact with minimal effort.
As malicious bots evolve and become more stealthy, it’s no longer enough to know who the visitor is (i.e., block by signature). Security solutions also need to assess why any bot is there in the first place. The use of reputation and behavioral analysis can help examine the context of bot visits, which is an important factor in identifying Impersonators, anonymous proxies and other new types of bot threats.
Looking ahead
Bots are an essential part of the Internet ecosystem. However, they are now more than tools; whether they are used for good or malicious purposes depends entirely on the owner’s intentions and motivation.
When it comes to web threats, bad bots are the preferred tool of today’s cyber criminals; more than 90 percent of all cyber attacks (e.g. DDoS attacks, web application threats) that our researchers identified are executed by them and impersonator bots are the elite commando unit of bad bots.
In terms of lost revenues and remediation efforts, the cost of such assaults can easily reach hundreds of thousands — even millions — of dollars. As we have seen in major data breaches such as the Sony hack, the worst case scenario really depends on an attacker’s intentions and the magnitude of the target.