Wednesday, July 10, 2013

Study Confirms - Bug Bounties Provide Cost Effective Value

Bug bounties are all the rage today. Mozilla started the first major bounty program in 2004 for Firefox and later added critical websites in 2010, Chrome joined in 2010, Facebook in 2011 and even Microsoft has come around recently in June, 2013.

In addition to bounties offered directly through a specific company there are other programs like HP's ZDI and also a new on-demand approach to bug bounties for any company offered from BugCrowd

But, are bug bounties worth the time to manage, foster the research community, and the cost of the rewards? As someone who has been deeply involved in Mozilla's bounty program my answer has always been a resounding yes.

My opinion aside, I'm happy to now also draw attention to a Berkeley Study from Matthew Finifter, Devdatta Akhawe, and David Wagner titled An Empirical Study of Vulnerability Rewards Programs.

A few select quotes from the study:

On cost & value:

Both programs appear economically efficient, comparing favorably to thecost of hiring full-time security researchers. (pg 1)
 We find that VRPs appear to provide an economically efficient mechanism for finding vulnerabilities, with a reasonable cost/benefit trade-off (Sections 4.1.1 and 4.1.6). (pg 6)
 Benefits of a bug bounty program:
VRPs offer a number of potential attractions to software vendors. Offering adequate incentives entices security researchers to look for vulnerabilities, and this increased attention improves the likelihood of finding latent vulnerabilities.

Second, coordinating with security researchers allows vendors to more effectively manage vulnerability disclosures, reducing the likelihood of unexpected and costly zero-day disclosures. Monetary rewards provide an incentive for security researchers not to sell their research results to malicious actors in the underground economy or the gray world of vulnerability markets.

Third, VRPs may make it more difficult for black hats to find vulnerabilities to exploit. Patching vulnerabilities found through a VRP increases the difficulty and therefore cost for malicious actors to find zero-days because the pool of latent vulnerabilities has been diminished. Additionally, experience gained from VRPs (and exploit bounties [23,28]) can yield improvements to mitigation techniques and help identify other related vulnerabilities and sources of bugs.
Finally, VRPs often engender goodwill amongst the community of security researchers. Taken together, VRPs provide an attractive tool for increasing product security and protecting customer.  (pg 1)

Lastly, I presented on bug bounty programs for websites a few years back at OWASP AppSecUSA. My slides from that talk can be found on slideshare.

-Michael Coates - @_mwc