Passive Network Monitoring of Strong Authentication

There’s been a fair amount of consternation and FUD concerning the effectiveness of “strong authentication” in defending against APT. For example, in their M-trends 2011 report, Mandiant has demonstrated how smart cards are being subverted. If that isn’t bad enough, RSA has recently revealed that they’ve been victim of attacks that they believe are attributed to APT and which resulted in attackers getting access to information that may weaken the effectiveness of SecureID.

Unfortunately, like most people blogging about these issues, I can’t provide any more authoritative information on the topic other than to say that based on my personal experience, targeting and subverting strong authentication mechanisms is a common practice for some targeted, persistent attackers. It’s hard to predict the impact of any of these weaknesses. Additionally, people who have found out the hard way usually aren’t particularly open about sharing their hard knocks.

Nevertheless, I’d like to advance the suitability of passive network monitoring as a method for helping to audit authentication, especially strong authentication mechanisms. While auditing is more properly conducted using logs provided by the devices that actually perform authentication (and authorization, access control, etc if you want to be pedantic), there are real operational and organization issues that may well make passive network monitoring one of the most effective means of gathering the information necessary to perform auditing of strong authentication.

The vast majority of password based authentication mechanisms bundle the username with the password and provide both to the server either in the clear or encrypted. It is possible to provide the username in the clear and the password encrypted which would improve monitoring capabilities at the possible expense of privacy. In general, this bundling of credentials is done because confidentiality is provided through mechanisms that operate at a different layer of the stack: ex. username and password sent through SSL tunnel.

On the other hand, many authentication mechanisms provide the username/user identifier in the clear. For these protocols, passive network monitoring provides the ability to collect information necessary to provide some amount of auditing of user activity. In this post I advance two quick and dirty examples of how this information could be collected. For and simplicity’s and brevity’s sake, I’ll focus solely on collecting usernames. I’ve chosen two protocols that are very frequently used in conjunction with the strong authentication mechanisms: RADIUS and SSL/TLS client certificate authentication.

RADIUS

RADIUS isn’t exactly as the most secure authentication protocol in the world. Since it has some serious weaknesses, it’s normally not used over hostile networks (like the internet). However, it is frequently used internally to organizations. In fact, it is very frequently used in conjunction with strong credentials such as RSA SecureID. One nice thing about RADIUS is that the username is passed in the clear in authentication requests. As such it’s pretty simple to build a monitoring tool to expose this data to auditing.

In my example of monitoring RADIUS, I’ll use this packet capture taken from the testing data sets for libtrace.

In my experience tcpdump is very useful for monitoring and parsing older and simpler protocols, especially ones that usually don’t span multiple packets, like DNS or RADIUS. The following is shows how tcpdump parses one RADIUS authentication request:



/usr/sbin/tcpdump -nn -r radius.pcap -s 0 -v "dst port 1812" -c 1
reading from file radius.pcap, link-type EN10MB (Ethernet)
18:42:58.228064 IP (tos 0x0, ttl 64, id 47223, offset 0, flags [DF], proto: UDP (17), length: 179) 10.1.12.20.1034 > 192.107.171.165.1812: RADIUS, length: 151
Access Request (1), id: 0x2e, Authenticator: 36ea5ffd15130961caafc039b5909d34
Username Attribute (1), length: 6, Value: test
NAS IP Address Attribute (4), length: 6, Value: 10.1.12.20
NAS Port Attribute (5), length: 6, Value: 0
Called Station Attribute (30), length: 31, Value: 00-02-6F-21-EC-52:CRCnet-test
Calling Station Attribute (31), length: 19, Value: 00-02-6F-21-EC-5F
Framed MTU Attribute (12), length: 6, Value: 1400
NAS Port Type Attribute (61), length: 6, Value: Wireless - IEEE 802.11
Connect Info Attribute (77), length: 22, Value: CONNECT 0Mbps 802.11
EAP Message Attribute (79), length: 11, Value: .
Message Authentication Attribute (80), length: 18, Value: ...eE.*.B.._..).


Note that we intentionally haven’t turned the verbosity up all the way. While there’s a lot of other good info in there, let say we only want to extract the UDP quad and the username and then send them to our SIMS so we can audit them. Assuming a configuration of syslog that sends logs somewhere to be audited appropriately, the following demonstrates how to do so:



tcpdump -nn -r radius.pcap -s 0 -v "dst port 1812" | awk '{ if ( $1 ~ "^[0-9][0-9]:" ) { print SRC" "DST" "USER; SRC=$18; DST=$20; USER="" }; if ( $0 ~ " Username Attribute" ) { USER=$NF } }' | logger -t radius_request


This example generates syslogs that appears as follows:



Mar 26 14:45:15 monitor radius_request: 10.1.12.20.1034 192.107.171.165.1812: test
Mar 26 14:45:15 monitor radius_request: 10.1.12.20.1034 192.107.171.165.1812: test
Mar 26 14:45:15 monitor radius_request: 10.1.12.20.1034 192.107.171.165.1812: test


I’ve done no significant validation to ensure that it’s complete, but this very well could be used on a large corporate network as is. Obviously, you’d need to replace the -r pcapfile with the appropriate -i interface.

SSL/TLS Client Certificate

Another opportunity for simple passive monitoring is SSL/TLS when a client certificate is used. It is very common for this mechanism to be used to authenticate users with either soft or hard (ie. smart card) certificates to web sites. This mechanism relies on PKI which involves the use of a public and private key. While the private key should never be transferred over the network, and in many cases they never leave smart cards, the public keys are openly shared. In the case of SSL/TLS client certificate based authentication the public key, along with other information such as the client user identification, is passed in the clear during authentication as the client certificate.

To have data for this example, I generated my own. I took the following steps based on the wireshark SSL wiki:



openssl req -new -x509 -out server.pem -nodes -keyout privkey.pem -subj /CN=localhost/O=pwned/C=US
openssl req -new -x509 -nodes -out client.pem -keyout client.key -subj /CN=Foobar/O=pwned/C=US

openssl s_server -ssl3 -cipher AES256-SHA -accept 4443 -www -CAfile client.pem -verify 1 -key privkey.pem

#start another shell
tcpdump -i lo -s 0 -w ssl_client.pcap "tcp port 4443"

#start another shell
(echo GET / HTTP/1.0; echo ; sleep 1) | openssl s_client -connect localhost:4443 -ssl3 -cert client.pem -key client.key

#kill tcpdump and server

#fix pcap by converting back to 443 and fixing checksums (offload problem)
tcprewrite --fixcsum --portmap=4443:443 --infile=ssl_client.pcap --outfile=ssl_client_443.pcap


You can download the resulting pcap here.

The client certificate appears as follows:



$ openssl x509 -in client.pem -noout -text
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
b0:cc:6b:94:b4:83:0f:78
Signature Algorithm: sha1WithRSAEncryption
Issuer: CN=Foobar, O=pwned, C=US
Validity
Not Before: Mar 26 13:13:12 2011 GMT
Not After : Apr 25 13:13:12 2011 GMT
Subject: CN=Foobar, O=pwned, C=US
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
RSA Public Key: (1024 bit)
Modulus (1024 bit):
00:e5:d6:78:cd:95:4e:89:0c:88:bd:78:98:26:86:
0b:f1:be:df:85:98:a2:93:c1:66:65:44:d2:aa:08:
69:2d:4c:a9:9d:50:08:79:1d:58:6e:6d:b4:2b:24:
ca:37:90:d6:91:9f:6d:73:5f:51:5a:10:af:f0:ce:
85:85:d6:e4:42:7b:ca:b0:af:0c:52:8b:60:1c:5b:
3f:54:10:cc:c4:35:18:a8:a6:a7:c8:ae:df:b7:ab:
a9:d9:20:cf:f7:5c:43:01:2e:12:cf:96:45:87:e7:
7e:87:f7:5e:8f:25:23:1b:ee:bd:0a:79:48:07:99:
ba:cc:68:16:53:43:56:e9:a1
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Subject Key Identifier:
BD:C2:84:BF:76:17:B7:15:BC:2F:8C:7E:A6:E6:18:B1:47:60:A3:B6
X509v3 Authority Key Identifier:
keyid:BD:C2:84:BF:76:17:B7:15:BC:2F:8C:7E:A6:E6:18:B1:47:60:A3:B6
DirName:/CN=Foobar/O=pwned/C=US
serial:B0:CC:6B:94:B4:83:0F:78

X509v3 Basic Constraints:
CA:TRUE
Signature Algorithm: sha1WithRSAEncryption
4c:28:ea:47:20:38:d5:17:dd:cf:aa:f8:13:3e:d0:5f:cf:05:
7d:c7:a1:c3:f4:3e:d7:db:56:f7:d4:d6:d6:c6:f4:5c:47:5b:
99:f6:9c:23:2d:dc:75:ab:51:8b:96:df:26:3b:9e:59:8f:2c:
08:d1:84:bf:4f:98:65:b4:0f:b7:32:9d:2f:eb:d9:a5:a6:69:
b6:75:ce:03:f4:ad:3b:f2:e6:3a:a1:ff:44:ea:8a:98:40:34:
cc:dd:e0:d8:35:0e:8b:97:20:30:e4:7b:07:52:98:63:11:32:
5e:6e:cb:c7:f1:10:67:1c:cd:e2:03:3a:99:98:8b:2f:f8:94:
03:6f


For auditing, we are interested in extracting the CN, which in this case is “Foobar”. As the client certificate is transferred over the network, the CN appears as follows:



000002e0 00 3f 0d 00 00 37 02 01 02 00 32 00 30 30 2e 31 |.?...7....2.00.1|
000002f0 0f 30 0d 06 03 55 04 03 13 06 46 6f 6f 62 61 72 |.0...U....Foobar|
00000300 31 0e 30 0c 06 03 55 04 0a 13 05 70 77 6e 65 64 |1.0...U....pwned|
00000310 31 0b 30 09 06 03 55 04 06 13 02 55 53 0e 00 00 |1.0...U....US...|


Immediately preceding the string “Foobar” is following sequence (in hex):



06 03 55 04 03 13 06


I’m not 100% sure what the "06 03" is for, but I believe this to be invariant in client certificates (if not, this example needs fixing). The "55 04 03" is indicative of the following data being a CN. This is an x509/ASN.1 thing where this sequence maps to the OID 2.5.4.3. The "13" can vary among a few common values (it specifies the data type) and the "06" indicates the length of the data (6 ASCII characters). Using this knowledge of SSL certificates we can create a tool to extract and log all CNs as follows:



$ mkdir /dev/shm/ssl_client_streams
$ cd /dev/shm/ssl_client_streams/
$ vortex -r ssl_client_443.pcap -S 0 -C 10240 -g "svr port 443" | xargs -t -I+ pcregrep -o -H "\x06\x03\x55\x04\x03..[A-Za-z0-9]{1,100}" + | sed -r "s/\x06\x03\x55\x04\x03../ /" | sed 's/c/ /' | logger -t client_cert



This generates logs as follows:



Mar 26 15:26:05 sr2s4 client_cert: 127.0.0.1:41143 127.0.0.1:443: localhost1
Mar 26 15:26:05 sr2s4 client_cert: 127.0.0.1:41143 127.0.0.1:443: localhost1
Mar 26 15:26:05 sr2s4 client_cert: 127.0.0.1:41143 127.0.0.1:443: Foobar1


If you are new to vortex, check out my vortex howto series. Basically we’re snarfing the first 10k of SSL streams transferred from the client to the server as files then analyzing them. Note that since we’re pulling all CNs out of all the certificates in the certificate chain provided by the client, we’re getting not only “Foobar” but “localhost” who is the CA in this case. Also note the trailing garbage we were too lazy to remove.

While this works, this is a little too dirty even for me. The biggest problem is that the streams which are snarfed by vortex are never purged. Second, we’re doing a lot of work in an inefficient manner on each SSL stream, even those that don’t include client certs.

Let’s refactor this slightly. First, we’re going to immediately weed out all stream we don’t want look at. In this example I’m looking for client certs in general, but you could easily change signature to be the CA for the certificates which you are interested in monitoring. Ex. “Pwned Org CA”:



$ vortex -e -r ssl_client_443.pcap -S 0 -C 10240 -g "svr port 443" | xargs pcregrep -L "\x06\x03\x55\x04\x03" | xargs rm


That will leave all the streams which we want to inspect in the current dir. If we do something like the following in an infinite loop or very frequent cron job, then we’ll do the logging and purging we need:



find -cmin +1 -type f | while read file
do
pcregrep -o -H "\x06\x03\x55\x04\x03..[A-Za-z0-9]{1,100}" $file | sed -r "s/\x06\x03\x55\x04\x03../ /" | sed 's/c/ /' | logger -t client_cert
rm $file
done


This implementation is also probably suitable for use on a large network or pretty close to it.

For these examples, it’s assumed that the logs are streamed to a log storage, aggregation, or correlation tool for real time auditing or for historical forensics. I would not be surprised if there were flaws in the examples as presented, so use at your own risk or perform the validation and tweaking necessary for your environment. These examples are intended to be merely that—to show the feasibility. While I’ve discussed two specific protocols/mechanisms there are others that lend themselves to passive network monitoring as well as many that don’t.

In this post I’ve shown how passive network monitoring could be used to help audit the use or misuse of strong authentication mechanisms. I’ve given quick and dirty examples which are probably suitable or are close to something that would be suitable for use on enterprise networks. Notwithstanding the weaknesses in my examples, I hope they provide ideas for what can be done to “trust, but verify” strong authentication mechanisms through data collection done on passive network sensors.

Keeping Targeted Attacks Secret Kills R&D

I’m really impressed with Google’s response to what has been coined Operation Aurora by others. I’m impressed for lots reasons. I’m impressed because they recognize the value of their intellectual property and when they realized that it was threatened, they took decisive actions to protect their interests. I think it’s sad that so many companies in a similar situation would be blinded by short sighted lust for the “emerging market” that they fail to protect themselves and fail to recognize that the same market is far from a fair or open. I’m impressed that when they apparently felt that the espionage was backed or at least condoned by the Chinese government, they called them out. Most of all, I’m happy they made this public.

That being said, I’m not too impressed that google, and the majority of the computer security industry for that matter, were taken off guard by these attacks. The level of sophistication and determination is not new nor is the type of data targeted. For the purpose of this article, when I refer to a targeted and sophisticated attacks I’m referring to attacks where one or more attacker groups repeatedly seeks to (re-)penetrate an organization’s computer systems for ends specific to the victim organization, typically exfiltration of sensitive information. These attacks are characterized by a high degree of knowledge of the victims, often a high degree of social engineering, adequate technical sophistication, and high degree of organization/coordination on the part of the attackers. I refrain from using the term advanced persistant threat (APT), because while it has had a fairly precise meaning among the people using the term for some time, the meaning has been blurred quite a bit of late. For the purposes of this article, the specific identities of the attackers, including affiliation or backing by nation-states, is not important. A few public reports of these sorts of attacks go back to at least the 2003-2005 timeframe, probably earlier, but that’s when I started paying attention. Maybe the one thing that is new is the type of industry targeted. I think google should have known it was coming. I’ll bet they had some warnings they chose to ignore, but I guess I can't fault them too much.

The response by the security industry to these attacks is pitiful. Many people recognize that the state of the art, including mainstream enterprise security tools, can’t stop, let alone detect, this sort of activity. While there are a few valiant incident responders who have been dealing with sophisticated targeted attacks for some time, many with a good deal of success, the security vendors have basically ignored their pleas and ideas for improved security tools. I’ve heard vendors say “You don’t want to do that” and “the market for that isn’t big enough for us to implement it”.

What has to happen for the security industry to realize they need to deal with sophisticated targeted attacks? First, organizations need to realize the value of their intellectual property. Second they need to realize that it’s at risk. I think most organizations are at this point. Third, they need to realize that conventional security wisdom, practices, and tools, won’t protect them against this, for some people new, class of attacker. Unfortunately, all too often, this epiphany only comes after personal and painful experience. Fourth, enough people need to start demanding effective solutions that vendors feel compelled to deliver them and academia recognizes the problems that need researching. Lastly, the solutions--a capable workforce, processes and practices, technology, etc need to be developed.

While there are many hindrances, one of the biggest obstacles to effectively dealing with targeted attacks is silence. While this class of attack is far from new, basically no one talks about it. While there are plenty examples of good public documentation of sophisticated attacks, ex. Businessweek E-espionage threat, NG’s report on Chinese Espionage, and Mandiant M-trends, basically no one credible steps up and confirms the validity of the data, leaving many to dismiss these reports as sensational journalism, conspiracy theories, and marketing hype. Based on solid public data, I guess I don’t blame people for questioning the reality of this threat until they experience it personally.

This code of silence related to compromises is very detrimental to solving the problem through the various available avenues: political/diplomatic, legal, and security systems including technology and people. There are a lot of legitimate reasons for not broadcasting your status as victim of a sophisticated attack and/or the type details required to help prevent future occurrences. Most of them I wouldn’t agree with, especially if everyone in the same industry/sector is in the same boat and you all know it. One of the few legitimate reasons to keep details of these attacks secret is that defending against persistent attackers is best achieved through an attacker focused or security intelligence driven approach. But how long is your threat intelligence still useful? Surely keeping specific attack data secret past a year or two doesn’t buy you much in terms of security intelligence as the most aggressive attackers change tactics and techniques more frequently than this. Hopefully it doesn't reveal too much about your capabilities either, as they need to be evolving that quickly also. Does acknowledging you’ve been attacked after your incident response is finished, or at least well under way, buy you anything in terms of threat intelligence? I don’t think so. I admire google for going public and doing something about it. I’m happy to see some public details, but more details and official acknowledgement from google would be nice. Sadly, google is right when they say they’ve already been more open that most others in the industry.

The organizations that keep targeted attacks and the details of them secret are part of the problem, or at the very least, aren’t doing everything necessary to help solve the problem. I think it’s a little hypocritical for organizations to complain about the security industry and academia not addressing this class of threat when no one will talk about the problem publicly with the requisite level of certainty and specificity.

Focusing on security R&D, there are a few things I think need to happen before the security tools industry and academia can start to address targeted attacks. The people doing R&D need to know what type of attacks are actually occurring, they need to understand the importance of a threat focused response model, and they need some decent data.

Understanding the Targeted Attack Scenario

One of the major problems with current academic and applied research is that most researchers don’t understand the basics of a highly targeted attack scenario. They don’t know how serious the problem is. If you tell an academic that the sky is falling because of targeted attacks and give them a high level overview, they’ll either yawn or laugh at you. Case in point, the following hypothetical conversation:

Boots on Ground Responder: We’ve got to do something about these highly socially engineered spear-phishing attacks!

Heads in Clouds Researcher: If you graph the social network, how many nodes away is the sender from the recipient?

Boots on Ground Responder: Uh, 1. Sometimes 2. Sometimes more, it depends.

Heads in Clouds Researcher: Ok, what about the malware? Rootkit? Polymorphism? Any Red pill/Blue pill?

Boots on Ground Responder: In this case nothing like that. Just simple malware that provides minimal backdoor. Malware isn’t even packed.

Heads in Clouds Researcher: Ok, this stuff isn’t being detected by your AV, IDS, etc but it’s still making it through firewalls, proxies, etc. Any interesting data hiding techniques?

Boots on Ground Responder: No, not really. Malware evades AV because it’s never been seen before. In cases where they need to evade our IDS, they use trivial obfuscation like ceasar ciphers. Usually though, they just hide in plain sight.

Heads in Clouds Researcher: Doesn’t sound too interesting to me. Just patch your systems and tell your users not to click on unsolicited email.

Boots on Ground Responder: Yeah, right. Still, we see repeated patterns in all of these attacks. I can’t give you details, but there’s got to be a way to catch these guys.

Heads in Clouds Researcher: Ok, well I’m going to go back to musing on the trusting trust problem…

The sad part is there are some really interesting problems, true academic problems, but for the most part, academia isn’t seeing them. I don’t think it’s because academia isn’t trying to find good problems to solve, I think it’s because the interesting details aren’t being shared.

Researchers need to learn how different targeted attacks are from opportunistic attacks. They need to understand how the goals and methods differ. They need to understand how different the targeting mechanisms are. They need to understand how valuable an intelligence driven response model is. However, they won’t learn it until someone shows them.

Supporting threat focused response

So much conventional security wisdom and basically all academic research takes a vulnerability focused approach. The focus is on detecting and mitigating individual attacks, not persistent campaigns comprising series of attacks. That’s the best approach for many classes of attacks, but isn’t the best if determined attackers continue attacking the same target over and over again. So many other people have spoken on this topic, that I’ll defer to them and steer my ramblings toward application of these principles to security tool development. For the reader’s reference, I recommend this podcast by some of the thought leaders in this realm. If what they are saying is news to you, check out their blogs, etc.

People doing security R&D have to learn about intelligence driven incident response. While some products support this approach, almost none fully embrace it. Even worse, academia is basically mute on the topic.

One aspect of a threat focused response model that is very important for security R&D is the importance of prioritization of response. While I have seen some products and research that recognizes the importance of prioritization based on the vulnerability/exploit, basically no security R&D addresses prioritization based on intelligence or attacker identity. Given the following choice, which would you rather detect/block: A stealthy rootkit installed by a botnet for the purpose of identity theft/fraud or an email containing a link to an exploit which when visited gives a sophisticated attacker user level access to the compromised computer? Most academics and many in the security industry would take the former because of impact on the system but a small group of security professional will lean hard towards the latter because of impact to the organization’s overall mission.

Another important aspect of threat focused response is relative importance of prevention and detection. For an intelligence driven response model, detection is king, and prevention is a distant second. In fact in some cases, it might actually be beneficial to not mitigate attacker activity if the attack is or will be mitigated further in the attack sequence (or kill chain) and if blocking the attack prevents collection of further threat intelligence (ex. firewall block). On the flip side, being able to detect an attack, even if it wasn’t or couldn’t be blocked, is imperative. If you look at the bigger picture, being able to block an attack is always the best, but if you can’t or didn’t detect it in real time, detecting it in near real time often almost as good. While many don’t appreciate it, being able to do historical detections, or understanding how intrusions started, including attacker activity preceding the actual attack, is also important to an intelligence driven response.

Lastly, post unsuccessful attack analysis is almost ignored by conventional tools and research. However, successful incident responders know the importance of analyzing unsuccessful attacks and developing mitigations across all facets of the attack sequence.

People doing security R&D have to learn to build features supporting threat intelligence into their tools and research.

Irrelevant Data Supports Irrelevant Research

One of the biggest hurdles to overcome for basically any sort of research is obtaining good data. The relative dearth of data related to target attacks kills research. If you were a researcher, would you choose a problem for which there is no public data? How could you? Even if you are doing more applied R&D, getting good data isn’t so easy.

There are a couple approaches to getting data for research: you can either gather the data for yourself, or you can use someone else’s data, usually a public data set. The problem with gathering the data yourself is that most researchers will never be able to gather data on targeted attacks. By their very nature, traditional computer security collection mechanisms such as honeypots, honey monkeys, etc will not normally ever see a targeted attack, definitely not a persistent campaign of target attacks. Even the researchers and vendors that do end up seeing samples representing one phase of targeted attacks, say malware, don’t see the full attack lifecycle. How can you address all phases of the attack if you only see one?

So there are good public data sets and there are some that aren’t so great, however, it seems that once a reasonably valid data set is used, it gets used over and over again. I admire folk who put together quality data sets for the community. One infamous example in the realm of incident detection is the DARPA 99 Intrustion Detection Evaluation dataset. While probably a decent data set at the time, and while memories of winnuke, etc may well be indelibly seared into the minds of some cyber war horses, these sort attacks are about as far from targeted attacks as you can get. DARPA 99 has been used and abused for a long time, but people still use it! Why? There aren’t many other options for public data sets. Other decent options for some types of research include packet captures from events like the Defcon CTF and NSA/West Point Competition, but these events are by their very nature very poor sources for persistent and highly target attacks.

While it will be necessary to develop good data sets involving targeted attacks, it’s going to be a hard effort. First, to demonstrate a persistent attacker, you need months, even years of data. As attacks have moved up the protocol stack and have become incredibly personalized, sanitizing data is going to be a lot more difficult than scrubbing IP addresses and hostnames. To truly address targeted attacks, tools will have to be configured with information about the data and people using the computer systems (not just the computer systems themselves). What that means for researchers is that to understand the significance of a target attack, you have to understand the targeted organization and targeted individuals. Lastly, as incident responders know, to be effective, data needs to be integrated from all phases of the attack and come in all sorts of formats: logs, netflow or packet captures, malware, etc. It’s clear that a perfect public data set for target attacks will never exist, but organizations can make steps by releasing older data.

While I doubt that any quality public data sets will be coming soon, organizations need to learn the value of collecting an internal data set. By nature, Incident Responders aren’t always the most disciplined at things like collecting and labeling data for historical purposes, especially considering the conditions in which they operate. Regardless, a little bit of effort to compile historical attack data for future reference, including labeling of data, pays huge dividends both in responding to future attacks and providing good training/test data for new tools.

Keeping quiet about sophisticated targeted attacks kills, among other things, intelligence driven tool R&D. For the technology to catch up with the threat, the problem needs to be discussed publicly and more details need to be shared. Publicly sharing attack information is critical to the research and development required to catch up technologically with sophisticated attacks. If the code of silence isn't broken, incident responders will continue to flounder with mainstream security tools while security tool vendors will continue to have watershed moments.

Developing Relevant Information Security Systems

In January, I presented at DC3 on Agile Development for Incident Response. I firmly believe that rapid engineering of information security systems is necessary to effectively combat sophisticated threats. I’ve also been struck lately by the lack of relevance of so much information security research and development.

One thing that I am adamant about, but has largely been ignored by the mainstream security community, is the need to face sophisticated and determined attackers with a threat focused response. A few others have already written extensively on this topic. The one reference I will make is to Mike Cloppert’s explanation of security intelligence, specifically his article on attacking the kill chain which takes a conventional military construct (kill chain) and applies it to information security.

Threat focused analysis is necessary, but is not sufficient. Unfortunately, current off-the-shelf security systems do not adequately support this approach. To effectively perform security intelligence, new security tools must be developed. Sadly, sophisticated attackers are not static targets. They change and evolve. What’s more, the enemies themselves change over time.

Working in the defense sector, I often try to contrast the cyber security world to the physical security world. I do this predominately for the purpose of finding ways to apply lessons of the past to present problems. The world has a long history of fighting wars and developing weapons systems. There must be some lessons to be learned from conventional weapons systems that can be applied to the realm of cyber security. As such, I’m going to use 4 conventional weapons systems to express allegorically some of my recent musing on effectively developing threat focused information security systems.

Too Much, Too Late

It wasn’t too long ago I visited the final manufacturing plant for the F22 Raptor. I have to admit, seeing the F22 in person makes the technological marvel it is that much sexier. However, while the F22 largely meets the expectations that the engineers set out to accomplish so many years ago and truly is far superior to any other fighter out there, the US decided we didn’t need it any more, especially at the ~$150 million per plane cost.

What went wrong? Latency. The threat landscape has changed significantly in the last 3 decades. If we had active enemies with technology that could only be adequately matched by the F22, then the F22 would be a bargain. However, since F22s aren’t particularly useful in wars like Iraq an Afghanistan, the cost is unjustifiable. To add insult to injury, it is conceivable that in a decade or two we could have a real need for the F22 that justifies the high price tag, but since the production lines and engineering will have long ceased, simply building more of them won’t be an easy option.

The information security equivalents of the F22 exist. They are technologically magnificent. They operate well for the missions they were designed for. Unfortunately, the cheese has moved. It’s hard to say if the technologies will be relevant in the future, but if they’re not relevant enough to justify further investment today, it will likely mean starting again from scratch.

Smart Bomb

Starkly contrasted to the F22, is a humble artillery round called the M982 Excalibur. This thing is everything the F22 isn’t--mundane, relatively cheap, and fabulously effective against today’s threats. It’s been very popular in Iraq and Afghanistan because its precision allows its use against insurgency close to non-targets or in complex terrain.

What makes the Excalibur great? Was it lack of technical challenges and problems during development? No. Radical new technologies? No.

The Excalibur is great because it is an ingenious marriage of technologies from other high tech devices (insanely expensive guided missiles) with a widely deployed, reliable, and economical infrastructure (howitzer artillery). While the Excalibur is relatively economical, the XM1156 promises to make similar capabilities really cheap.

We need more Excaliburs in the field of information assurance. We need to take our existing IT infrastructure and security tools and make the relatively minor tweaks necessary to keep pace with the changing threat landscape. Just like the howitzer munitions have changed over time to keep pace with enemies, often, we just need minor adjustments to our core IT infrastructure to allow us to respond to today’s attackers. However, if we can’t get the requisite features in a timely manner, we are often forced to make do without or employ a whole new tool just to fill a relatively small role. One general example I can think of is audit logs. All too often, the inclusion of one small piece of information is all that is required to turn a vanilla IT system into a widely deployed IDS.

Waiting for Godot

The Expeditionary Fighting Vehicle (EFV) is an amphibious landing craft being developed for the Marines. The EFV is recognized as one of the top acquisition priorities for the Marines but the program is floundering. I guess it doesn’t take much imagination to figure out how fundamental landing craft are to the mission of the Marines. The EFV was supposed to be in service over a decade ago, but reliability issues have kept that from happening. The current projected deployment date is far enough out that it might slip again or that the project might get canceled or changed drastically.

There are too many EFVs in the realm of information security. There are lots of reasons why this occurs so often, which I don’t want to discuss at the moment. Risking being called an existentialist, I declare that a system that isn’t deployable yet doesn’t exist. We’ve got to stop building and waiting on vaporware. I’ve been burnt too many times by waiting for systems that are perpetually just around the corner. I wish it weren't true, but I have my own fair share of culpability in this regard. I do believe that applying agile instead of waterfall development methods will help curtail perpetually late projects. Clearly professional integrity is also required.

Freedom as in Speech

Probably the least well known weapon system I will use as an example is Acoustic Rapid COTS Insertion (ARCI). In short, ARCI delivers rapid improvements to the sonar systems of the US submarine fleet through frequent deployments of both new software and hardware, building largely from off-the-self hardware, such as Intel and AMD processors, and commercial or open source software such as the Linux operating system. ARCI has demonstrated the value of shifting from completely custom and proprietary solutions to leveraging off-the-shelf platforms in order to focus R&D resources on the features unique to the mission of the system. ARCI delivers new capabilities to the fleet at a previously unknown rate and has become a shining example of the Navy’s quest to acquire open systems. While lacking in historical track record, the Littoral Combat Ship promises to take this open systems approach to a meta level, making a ship a platform for modular mission systems that can be developed and deployed rapidly to fulfill current missions. I see great promise in this open systems approach to weapons systems.

There are already many good examples of openness in the realm of information security systems, but we need more. To remain relevant in the face of changing threats, information security systems must provide flexibility at the architectural, platform, and component level. Re-inventing the wheel is a waste of time that we can’t afford. We need to build upon established technologies and focus new development on the capabilities specific to the threats we face. We have to build openness and flexibility into our information security systems. My personal experience with ARCI has changed the way I think about developing highly specialized systems.

Security Development call to Keyboards

I’ve intentionally masked my complaints about information security systems development with analogies to military weapons systems, so as to not have to name any specific information security tools. Whether you agree with my hasty analysis of these weapons systems or not, I hope that the characterizations I’ve tried to establish allow you to identify the allegorical class of information security systems. The information security community must do better at defending against sophisticated attacks. A portion of the need for improvement rests on the security tool development sector and the people who direct them.

As security system developers, we need to create open systems that are relevant to today’s threats. We need to build flexibility into our systems at the architectural, platform, and component level. We need to build tools that ease customization, extension, and integration with other tools. We need to rapidly respond to our users’ request for changes to functionality. We have to shed the blinders of entrenched methods and truly innovate. We have to stop peddling vaporware.

As people who buy or direct development of security tools, we require open systems that both meet our needs today, and provide us the freedom to react to changes in the future. We must be judicious in asking for highly specialized tools that aren’t possible to develop in a short time frame and which might be irrelevant before they are completed. We must find ways to motivate our vendors to provide what we need and not more. When our vendors can’t or won’t provide the capabilities we need, we have to roll up our sleeves and do it ourselves.