Tag Archives: sinkhole

Collateral Damage: Microsoft Hits Security Researchers along with Citadel

Before I start this blog post, I would like to express that it is not my intention to offend any person or organisation. Please understand that I’m currently very disappointed and in some ways angry because of the story described below.

Operation b54

As most of you have already read in the press, the Microsoft Digital Crimes Unit (DCU) recently carried out an operation called Operation b54 to shut down Citadel botnet C&C servers. According to Microsoft, they were able to disturb over 1’400 Citadel botnets around the world by seizing more than 4’000 domain names and pointing them to a server operated by Microsoft. Technically, this is what we call “sinkholing”.

Sinkholing domain names isn’t something new, in fact this technique has been around for ages and was already used by the Conficker Working Group and their operation to hijack the Conflicker botnet. The purpose of a sinkhole is the same for the most part: collecting information about infected computers that are connecting to the sinkhole and report these back to the associated network owner so that the responsible end user can be notified about the infection and remove the malware from their computer.

These days there are many sinkholes around, most of them are operated by security researchers. But there are also many commercial sinkholes out there, that are being used by security service providers to gather information about infected computers and sell that information to their customers. As a security researcher I spend a lot of time in researching botnets in my spare time, and abuse.ch is running such a sinkhole as well (in fact for years). The goal is simple: sinkhole malicious botnet domains (not only limited to any specific Trojan / malware family) and report them to Shadowserver. Shadowserver, a non-profit organisation like abuse.ch, then informs the associated network owners about the infections reported by my sinkhole, in addition to infections reported by their own sinkholes and sinkholes run by other operators. In fact, every Computer Emergency Response Team (CERT), Internet Service Provider (ISP) and network owner can get a feed from Shadowserver for their country / network for free. For the time being, Shadowserver notifies more than 1’500 organisations and 60 national CERTs about infected computers within their responsibility.

Today, I’ve suddenly noticed that several domain names disappeared from my sinkhole. I started to investigate and noticed these are now all pointing to a server in Microsoft’s network range ( It was quite obvious to me what had happened. Microsoft seized not only malicious domain names operated by cybercriminals to control computers infected with Citadel, but also Citadel botnet domain names that had already been sinkholed by abuse.ch awhile ago (I want to outline here that my sinkhole is appropriately tagged and clearly shows that it is actually a sinkhole of abuse.ch). I pulled down the list of Citadel domains that Microsoft seized and checked it against my sinkhole’s domain list. I was quite surprised about the result: Microsoft seized more than 300 domain names that where sinkholed by abuse.ch. I was not only surprised but also quite disappointed: Microsoft already showed similar behaviour in their operation against ZeuS last year were they seized thousands of ZeuS botnet domains, including several hundred domain names that were already sinkholed by abuse.ch. Due to this, I’ve set up a (non-public) Sinkhole Registry for LEA and security organisations to avoid similar situations in the future. I had hoped that Microsoft had learned their lesson, but apparently nothing has changed and my efforts didn’t change anything.

Since Citadel domain names previously sinkholed by abuse.ch have been grabbed by Microsoft, Shadowserver will not be able to report the IP addresses of infected clients calling home to these domains to the network owners any more.

Today, I’ve talked to several other sinkhole operators asking them about their experience with Microsoft. All of them confirmed to me that several dozens and for some operators even hundreds of Citadel domain names they had sinkholed have been seized by Microsoft as well. Calculating the numbers together, I can say that nearly 1’000 domain names out of the ~4’000 domain names seized by Microsoft had already been sinkholed by security researchers. In fact these ~1k domain names did no longer present a threat to internet users, but were actually used to help to make the internet a better place.

Unfortunately, this is just the tip of the iceberg. When checking out Microsoft’s sinkhole, I noticed that they are actively sending out valid Citadel configuration files to the connecting bots. A sample configuration file served by Microsoft’s sinkholes looks like this:

url_loader (binary download)

entry “AdvancedConfigs” (backup config files)

This configuration file causes that the blockage of websites of AV vendor gets removed (so the on the infected computer installed AV product can load the most recent definition), but also that the main C&C configuration including the fall-back (backup) C&C domains to get overwritten by servers operated by Microsoft (microsoftinternetsafety.net):

Domain Administrator
Microsoft Corporation
One Microsoft Way
Redmond WA 98052
domains@microsoft.com +1.4258828080 Fax: +1.4259367329

Domain Name: microsoftinternetsafety.net

Registrar Name: Markmonitor.com
Registrar Whois: whois.markmonitor.com
Registrar Homepage: http://www.markmonitor.com

Due to this, Microsoft ensures that once a bot connects to their sinkhole it stays there and won’t try to reach out to a different C&C. In theory, this is a very good idea and I have to say that many sinkhole operators had the same thought years ago. But unlike Microsoft, most of the sinkhole operators came to a different conclusion: Sending out valid configuration files de facto changes settings of a computer without the consent or knowledge of the user (computer owner). In most countries, this is violating local law.

A very important point which Microsoft either didn’t take into account or chose to ignore, are potential countermeasures by the criminals. Let me try to make two examples:

From ZeuS-Licat (aka Murofet) to P2P ZeuS (aka ZeuSv3 aka Gameover ZeuS)

In 2011 ZeuS-Licat (also known as Murofet), a derivative of ZeuS, was using a Domain Generation Algorithm (DGA) which an infected computer (bot) used to “calculate” the current C&C domain. There were a lot of effort by security researchers and information security organisations to shutdown the daily C&C domains immediately after they became active. Every domain name was listed on ZeuS Tracker as soon it went active. Obviously, this effort popped up on the criminals radar quite fast. It didn’t take long for the criminals to roll out a new version of ZeuS-Licat – P2P ZeuS was born. P2P ZeuS is no longer using a central C&C infrastructure that could be sinkholed or take down, rather than using a sophisticated P2P technique to receive commands from the botnet herder and send stolen data to the criminals. Since the source and destination ports used by P2P ZeuS are randomized, its hard to detect such traffic in an ISPs network environment, and even harder to actually block it. A network owner needs an IPS/IDS or a deep packet inspection (DPI) system to detect or block such P2P ZeuS activities.

Torpig – RSA signed C&C communication

Torpig is a highly sophisticated e-banking Trojan that has been around for years. There are various organisations out there doing sinkhole operations on the Torpig botnet. A while ago, one of these operators started to send out valid configuration files to the connecting bots which caused that the bots stay on that specific sinkhole. This is in fact exactly what Microsoft is doing at the moment for Citadel. Of course, it was quite obvious that the criminals would implement countermeasures soon. It didn’t take long for the Torpig gang to roll out a new version of their Trojan that implemented an RSA signed C&C communication. While sinkholing is still possible, it is no longer possible to send a valid response back to the infected computer (bot) unless you know the private RSA key (which is only known to the Torpig gang).


It is common that such takedowns trigger countermeasures by the cybercriminals. Imagine: If someone breaks into houses with a baseball bat and you seizing his bat, what will that person likely do next? If he wants to continue, he will just buy a new baseball bat – or maybe he will even buy a gun next time, to make it harder to take him down. When you apply this to the Citadel case it’s obvious that the criminals using Citadel won’t stop doing cybercrime. It may even have the bad effect of criminals updating their software to prevent that such takedowns are possible in the future again (eg. by implementing P2P techniques like ZeuS-Licat or RSA signed C&C communication for Torpig). The problem with cybercrime is that it can’t be solved with doing takedowns. It’s only possible to solve this issue by implementing legislation related to cybercrime, enforce them by getting bad actors arrested and implementing security by design on different layers (operating system, network layer etc.).

As outlined before, Shadowserver will no longer be able to inform network owners about several thousand Citadel infected computers because the Citadel domain names sinkholed by abuse.ch has been seized by Microsoft.

According to Microsoft, their goal was to disturb Citadel botnet operations. In my opinion their operation didn’t have any big noteworthy impact on Citadel, rather than disturbing research projects of several security researchers and non-profit organisations, including abuse.ch. In my opinion, operation b54 was nothing more than a PR campaign by Microsoft.

Follow me on Twitter: https://twitter.com/abuse_ch

How Big is Big? Some Botnet Statistics

There is a lot of malware out there, and sometimes it’s very difficult for security researchers or AV-vendors to estimate the extent of such a threat (eg. a trojan). One technique to do is called sinkholing: The goal is to register malicious botnet domains proactively or reactively to prevent the criminals exerting command and control over hijacked/infected computers, and at the same time warn ISPs of infected computers.

Some of you might already know that I am running a sinkhole. Therefore I thought it might be interesting to reveal some botnet Statistic based on the drone data I have collected on my sinkhole.

The following data has been collected over a period of 2 months. During this time I’ve sinkholed several botnets. To generate the statistics shown below I have picked out the highest peak of each malware family and printed it to the bar chart. In short this means that the chart shows the highest peak of each malware family during the past two months (within a 24 hour period).

First of all, let’s have a look at each malware family I’ve sinkholed during this time.

Trojan Aliases Reference
Artro Renos, CodecPack Kaspersky Lab
Carberp Symantec
Gbot Sonicwall
Gozi SecureWorks
Ponmocup Swisyn, Changeup Microsoft
Ramnit abuse.ch
SpyEye EyeStye Symantec
TDSS Alureon, Tidsserv, TDL4 ESET
ZeuS Zbot, WSNPoem, ntos Symantec

As shown in the table above we have some banking trojans (Carberp, Gozi, SpyEye and ZeuS), some trojan droppers (Gbot, Ponmocup), a worm (Ramnit) and some Click fraud trojans (Artro, TDSS).

Note: The numbers of infected IPs for each trojan mentioned below does not necessarily reflect the exact botnet size. It does however work fairly well as a relative indication. Some trojans are malware kits being used to run several different botnets (Like ZeuS or SpyEye), where all are not being sinkholed.

Let’s take a look at the sinkhole statistics:

The chart above shows the total number of new and total IPs seen within 24hrs for each malware family. What really sticks out is the fact that the trojans that are being used to attack financial institutions (banking trojans) has a relatively small amount of infected computer (drones) compared to Gbot (that is used to drop/install additional malware on the victims computer) and the well-known click fraud rootkit called TDSS. The size of the TDSS botnet is 6 times the size of the Carberp botnet.

Why is this the case? It’s not very difficult to infect computers today. The trick is to find a good way to monetize the botnet. For banking trojans, the problem becomes getting money mules that the criminal can use for transferring/laundering the stolen money. A cybercriminal won’t benefit from a big botnet if he’s not able to cash out the money from the bank accounts of the victims. Also, banking trojans rather quickly gets attention from both Law Enforcement and individuals in the infosec community.

Doing click fraud is much easier: Who cares about click fraud? Nobody, except the companies that are actually offering/selling online advertisement. If you call someone and tell him “Hey, your computer is infected with a click fraud trojan” you will most probably get a answer like “WTF is click fraud?!?” and even if you explain the situation to him I’m pretty sure you will get an answer like “Well I don’t care, I hate online advertisements anyway. They only distract me when I’m surfing on porn sites… *erm* when I’m doing online shopping”.

Still, I’m not surprised that there are botnets out there that are even bigger than TDSS/TDL:

The chart above shows a botnet that is called Artro. It is also known as “The advertisement botnet” (Kaspersky) or Renos/CodecPack. It is 1,5 times bigger than TDSS. However, Artro is also doing some click fraud stuff. I sinkholed the Artro botnet a year ago. Back then, the botnet had a size of 330’000 infected computers (of course within 24hrs)!

So I’m asking myself: Does this answer our question “How Big is Big”? If we are serious we can say that 330’000 infected computers is quite enough and really big. That’s nearly the same amount of computers as there are inhabitants in the largest Swiss city (Zurich).

What would you say if I told you that there is a botnet out there that is much bigger than the Artro botnet?

Some weeks ago I came across a huge botnet that was pretty unknown to me and that I never had heard of before. Doing some research I came to the conclusion that this trojan was known as Ponmocup. When I’ve started to sinkhole this botnet I was shocked as I saw that more than 1,2 million (yes, 1’200’000) unique IPs connected to my sinkhole just within 24 hours..

Probably most of you don’t even know Ponmocup, so you may ask yourself how this botnet became that big. Well you already answered this question: The criminal obviously managed to stay under the radar for months (maybe even years). I’m sure there are even more botnets out there (like Artro and Ponmocup) that are quite big and still under the radar of the AV-industry / infosec community.

*** Conclusion ***
We have learned that the botnet sizes doesn’t really matter. The criminals don’t need to have a big botnet to make a lot of money: It always depends on the business model the criminals wants to adopt (doing ebanking fraud, clickfraud or whatever).

But what do we have to do to mitigate these threats? My approach is to try to identify such botnets and sinkhole them. Doing so I’m able to collecting data from the connecting bots, which are being fed into the Shadowserver Drone database. If you are an ISP, a company or running your own network/AS you can obtain free-of-charge Drone feed from Shadowserver for your AS. This allows you to get informed about infected computers within your network on a daily basis.

If you are an ISP/network owner I highly recommend you to subscribe to Shadowservers Drone feed (if you are not already subscribed).

You can subscribe and/or obtain more information about Shadowserver’s Reporting Service here:

Follow me on Twitter:

*** Further links ***