Friday, December 17, 2010

Announcing Ruminate IDS

I’m pleased to announce that Ruminate IDS, a system I’m building in order to conduct my PhD research, has been released as open source.

The goal of Ruminate is demonstrate the feasibility and value of flexible and scalable analysis of objects transferred through the network. Ex. PDFs, SWFs, ZIPs, DOCs, XLSs, GIFs, etc. To the best of my knowledge, there is no other IDS out there that focuses heavily on or provides comprehensive facilities to do this today. Ruminate doesn’t do the stuff that contemporary NIDS do well, such as signature matching, individual packet analysis, port scan detection, etc. If you’re interested in learning about Ruminate, reading the technical report is the best place to start.

The current implementation that is available for download is built largely to gather statistics useful for academic research. I’m hoping a release a version early in 2011 that will be more appropriate for people seeking to use it in operational environments. Regardless, I was somewhat surprised by the ability of Ruminate IDS as presently constituted to detect live attacks by highly targeted and sophisticated actors when used on a production campus network.

Ruminate is a great example of the type of IDS that could be built on top of the utility provided by vortex. It would probably be fair to consider a Ruminate a fabulous example (and facilitator) of Taco Bell Programming with both the good and bad connotations.

Despite the many imperfections and limitations, I hope Ruminate IDS may be of value to both academia and network defenders alike.

2 comments:

  1. I haven't read the technical report yet, but how would you say this differs from Sourcefire's Razorback both in concept and/or technical execution? Would you be willing to give some generic usage models in everyday operation? Does it play well with other detection techniques, and if so, what part does it fill in the process? Is this only for advanced CIRT teams or do you see it realistically being used in a more wide spread manner among general security operations?

    -Thanks.

    ReplyDelete
  2. Anonymous, thanks for the good questions (and not hastily launching a DDoS against me as Anonymous is wont to do:) ).

    >> haven't read the technical report yet, but how would you say this differs from Sourcefire's Razorback both in concept and/or technical execution?

    I actually mention Razorback in the “Related Work” section of the Technical Report:

    In addition, VRT Razorback [6] advances an alternative implementation of a client application object analysis platform allowing the use of arbitrary engines, but does not present a mechanism for comprehensive and scalable analysis of network protocols in order to extract all embedded objects from network traffic.

    I’ll try to expand on this necessarily terse excerpt from the technical report in a subsequent blog post soon. One thing I’ll saw now is that Ruminate focuses more heavily on the problem of how to extract payload objects from network traffic and Razorback focuses more heavily on what to do with them once you have them extracted.

    >> Would you be willing to give some generic usage models in everyday operation? Does it play well with other detection techniques, and if so, what part does it fill in the process?

    The next release of Ruminate, which I hope to complete by Jan 2011, will be more focused at operational environments. I’ll have more concrete to discuss then. Right now I’ll say wouldn’t it be cool if in addition to layer 3 through layer 7 metadata, you had the name, MD5, and size of every file transferred over your network? What if the file is embedded in another zip file? What about deploying off the shelf detection mechanisms, such as xorsearch and jsunpack to all PDFs traversing your network? I don’t advance any new detection mechanisms at this point: my focus is building the framework necessary to be able to apply existing detection techniques to objects transferred through the network.

    >> Is this only for advanced CIRT teams or do you see it realistically being used in a more wide spread manner among general security operations?

    I see network payload analysis as the next frontier of NIDS. It may be that this remains the domain of “advanced CIRT teams” in response to “sophisticated threats” for a while. However, just as layer 3, layer 4, and layer 7 analysis has “trickled down” from over time, I believe the same will be true of network payload analysis.

    ReplyDelete