Let's consider the average web application. It may or may not be secure. It may have obvious vulnerabilities just waiting to be exploited or it could be relatively secure minus a more obscure vulnerability buried in the app. If an attacker were targeting this app, they would perform a series of steps to try and identify a vulnerability. Perhaps they would insert xss type string into various input fields? Maybe integers within the URL arguments would be incremented or replaced with SQL injection attempts. Another great target is the cookie values. Maybe these are modified or removed.
The problem I have is that the application just sits there and waits; waits to be compromised. This is akin to someone taking a stick and hitting you repeatedly while you do nothing. If the application is clearly being attacked, then I believe it should respond.
There are challenges in responding. First, it is tough to respond to an unauthenticated user. You don't know who they are and it would be nearly impossible to accurately track them from request to request. So lets consider only authenticated users. In fact, that is a good move because many of the more complex features are available to the authenticated users anyway. Second, we need to determine what is an attack versus what is user error.
Now, if we can detect a user that is performing basic malicious activity against our application, we can take action BEFORE they locate a vulnerability. Perhaps our action is to logout the user and lock the account? That would make it very difficult for our attacker to discover a vulnerability if their account was locked due to a security error and they had to contact a phone number to unlock it.
The response actions would have to be tuned to prevent taking action against innocent users. However, there are many cases where we can take responsive action with little fear of targeting the innocent. For instance, how many innocent users would modify a POST request and include a cross site scripting attack? Odds are this is bad.
If we identified a collection of detection points within our application and then collected any firings of these intrusion events we would be able to monitor the malicious activity of the users. Once defined thresholds had been reached, the application can perform a defensive response action such as warning the user or locking the account.
These issues are all being explored as part of the OWASP Summer of Code AppSensor project.
-Michael Coates