I would agree with this overall conclusion. The elements that determine if an application is secure is far more complex than simply the language that has been selected. However, I would argue that some languages are more conducive to organized and easy to manage code that follows an MVC approach. Similarly some frameworks provide security controls that are enabled by default to aid in the security of an application. However, framework and language selection are meaningless if security is not baked in to the entire application life cycle.
Here are the three primary findings from the report.
- Empirically, programming languages / frameworks do not have similar security postures when deployed in the field. They are shown to have moderately different vulnerabilities, with different frequency of occurrence, which are fixed in different amounts of time.
- The size of a Web application’s attack surface alone does not necessarily correlate to the volume and type of issues identified. For example Microsoft’s .NET (ASPX) and Struts (DO), with near-average attack surfaces, turned in the two lowest historical vulnerability averages. ASPX and DO websites have had an average of 18.7 and 19.9 serious vulnerabilities respectively.
- Perl (PL) had the highest average number of vulnerabilities found historically by a wide margin, at 44.8 per website and also the largest number currently at 11.8.ties have taken over 50 days to fix.
The full report requires registration and can be viewed here
Here is a rebuttal to some of the claims made by the report: http://pinvoke.wordpress.com/
As mentioned in the above link I encourage you to take these results with a grain of salt. The data is gathered from existing WhiteHat sentinel customers and is obtained from an automated tool (technically it is automated discovery with manual verification). The sample population is not necessarily a reflection of the critical web applications at large, nor is an automated tool able to fully analyze the security of a site. In fact, automated tools are often strong in the areas of input injection based attacks and weaker in areas of authentication, access control, session management and custom logic flaws.
If you are going to take away one item from this study then I hope it is this: there is no magic bullet or super secure programming language. If you want something good then you have to work hard for it. Security is a continuing effort that requires the support of everyone involed (dev, QA, managers, etc) and must be an essential component of each step of the SDLC.
-Michael Coates
Hi Michael,
ReplyDeleteFirst, thanks much for taking an interest in the research. We value the feedback as it helps us improve our statistics reports going forward.
To clarify a couple things: please note that while Sentinel does heavily leverage automation to find technical vulnerabilities, a key differentiator is that business logic flaws are tested for as well -- manually. See here. As such, Sentinel gets about as much coverage as any high-end professional vulnerability assessment / pen-test, I'd even argue more so.
Secondly, the aforementioned "rebuttal" was a little silly for a variety of reasons. Of course our sample set is biased, but not for the reasons stated, we even said so in the report. "Over 300 organizations, generally considered security early adopters, and serious about website security." Among those are seriously major websites that most people have most reading have accounts on, put in their CCs, store personal information, and so on. If that make the data irrelevant and narrow, well... Im not sure what criteria that person would even appreciate.
Thanks for the clarification regarding business logic flaws.
ReplyDeleteI noticed that this type of scanning is included in the Premium Edition of Sentinel. Are these results (e.g. business logic flaws) captured in the report? And if so, how was that data handled in relation to the other data points that did not include that type of scanning?
Jeremiah,
ReplyDeleteYou are welcome to refer to my rebuttal as a little silly. I think your "research" is little silly. If you would like to know more about what criteria I would appreciate, you could always ask me. Or, I suppose you could write something on someone else's blog and hope I read it. Either one works.
-Andrew
@michael, yes, business logic flaw are included in the report when identified -- almost exclusively when PE is deployed. Very little vulnerability is excluded. The testing methodology and coverage is not perfectly consistent across each website, that would be impossible for several reasons beyond our control. On page 4. we account for this and other things under "Factors influencing the results." To your question, there is a very heavy PE representation, a distance second is SE, and some BE.
ReplyDeleteAs we've said number times, it best to view our statistics reprot as a best-case scenario, we must assume there is always more vulnerabilities to be found beyond what we've identified.