We talked about malware analysis on a single infected box, but suppose you got a huge number of infectecd ones, or even many vlans in your network .... I know what you are thinking, the previous tutorials were focusing on a small problem that will get bigger soon if you don't follow the basics of malware containment.
As a system admin, you should depend on many tools and techniques to fight malwares. But as malwares getting more sophisticated, and hard to detect using normal antivirus softwares, we have to build better virus resistant networks.
It's not a rocket-science if you want to understand malwares behaviour, it'll take a little of your time to discover any infection in the network by using handy monitoring tools that keep an eye on every odd event that may occure anytime.
To make it simple i'll use keywords that will be covered here: (Network Sniffer, SNMP, MRTG, Vlan ACLs, Firewall ACLs, Honeybots, Nepenthes, Snort IDS, Gateway Antivirus, Windows Domain Policy, Windows Firewall (ICS), SRP, Netwrok TAP, Host Antivirus, Antispam virus engine, Antispam attachment policy, Switch Broadcast Rules, Switch ACLs, Switch IDS Modules)
I'll mention some famous worms that got their names remembered everytime you reach a discussion about worms and thier ways to move fast and infect huge networks in hours.
- Nimda: uses files infection, mass mailer, and lan propagation techniques.
- Blaster: uses direct vulnerable hosts exploitation
- Netsky: uses mass mailer
- Storm: uses mass mailer (Social Engineering)
So to clear from where we should start monitoring to inspect infections and contain them, we have to list the points of connections:
- Internet-to-Lans: the main gate of our network and the first point to monitor.
- Lan-to-Internet: every host which is able to access the outside world, could be infected and trying to infect other machines, or it could be a "Bot" and trying to "call-home"; a.k.a calling the "Mother Ship".
- Lan-to-Lan: here where we should be careful, any high rise of broadcast traffic means we got infected machines, and to contain them we can use alot of ways that I'll talk about.
- Wireless-to-Internet: employees or guests connected to this zone are higly suspected to be infected and trying to reach other machines in the internet.
- Wireless-to-LAN: infected wireless machines can reach the internal LAN if there are firewall/switch ACLs to deny them.
- VPN-to-LAN: VPN users are far away to controll thier home environment, but alteast we can inspect thier machines at the gateway.
- Branch-to-Lan: Gateway-to-Gateway VPNs are encrypted so, it's a hard task to monitor these links, and it's a considered a secure channel for viruses to move through. But we can inspect the clear traffic after the decryption process.
- Dialup-to-Hosts: If you allow modems to be used inside your networks, you have to stop it. Because this will open a hole in your wall bybassing all your multi-thousands dollars firewalls and lets viruses in.
Ok amigo if you want to flex your muscles in front of your management claiming that your network is secure and malwares-proof ... you have to reconsider it. The latest blogs posts are showing how the current security systems are not able to defend against the latest generations of malwares and bots. So we have to use additional techniques to detect the hidden threats in our networks.
I'll list the risks that you may face in case of internal malware propagation:
- Computer infections: this is the first thing to think about, once any computer got infected, it'll start scanning and propagating. Due to infections, computers became unavailable, untrustful, and less productive if not productive at all.
- Rise of traffic utilization: this will definitely slow down network-based applications, and services. And makes networking devices busy and not responsive due to the high level of packets processing. Some old devices will start crashing and disconnecting links.
- Internal Denial of Service: once any server became unreachable, this is a DoS. because routers & switches will drop new packets if the memory buffer is totally full. Or even the servers themselves will get busy for example DNS, SMTP,Proxy.
- MX record blacklisting: If you don't control which authorised hosts are allowed to send e-mails outside, consider this point as a "knock-knock on your door". Always use ACLs to allow mail servers ONLY to generate SMTP to the internet. Leaving the door open for any compromised hosts to send Spams, will make your days of sending-and-receiving e-mails numbered. And this is against your business aims to provide 24/7 service to your users and customers.
- User Productivity: we know that a healthy computer will make your user happy, and productive. It's always wise to have spare machines / virtual machines to replace the infected ones. Or even in case of a complete system crash or failure, a fast recovery solution is highly recommended. For example, Symantec Backup Exec for Windows Servers / Desktops.
After listing these terrible risks, we should draw a vision on how to start putting the defenses and monitoring points.
Monitoring Points as in Military
You have to treat your network as a battlefield, install radars at the gatway, inside the internal lans, at the critical servers zone. If you want to contain a malware propagation, you have to see it, and to see it, you must sniff and monitor. Sniffers are your eyes on the wire, and it's advisable to start learning about them if you don't know. From my experience, out of ten system admins, one of them uses a sniffer!
What is the main benefit of sniffing? Is giving you the ability to spot zero-day attacks behavior in details at the network layer. But if you want to get deeper you should get a sample, run in a computer and watch. You can create filters to sniff a specific patteren on the wire, this will save time and processing. This is possible in two ways: either you get the packets patteren from the web, or you create it while the malware is in action.
For example, W32.Blackmal worm once it infected a machine, it'll contact an online statistics counter, to capture this either create a DNS request pattern filter or HTTP web request filter to this domain: webstats.web.rcn.net. Here is the capture filter used in the WildPacket OmniPeek sniffer:
Beside Sniffing, we should monitor traffic utilization 24/7 using hand tools like MRTG (Free) or PRTG ($$$) that will use SNMP to contact networking/security devices and get the interfaces current in/out bandwidth. This is very important task, because propagation means generating thousands of packets to reach all of the hosts inside and outside your network, and due to this massive number of traffic you can spot easily on the screen. Faster and more readable than sniffing, because this is the high level monitoring, next step will be the sniffing to see the details of the traffic.
See ths MRTG screen shot of the witty worm activity:
To be continued ....