A few Billion Lines of code Later using Static Analysis to ...

A few Billion Lines of code Later using Static Analysis to find Bugs in theReal World
BY AlBessey, Ken Block, BenChelf, Andy Chou,Bryan Fulton, SethHallem, Charles Henri-gros,AsyaKamsky, ScottMcPeakand Dawson Engle
Introduction
Coverityis a brand of software development products, consisting primarily of static code analysis and dynamic code analysis tools.How static code analysis was commercialized ?What had to be changed for real world use ?How to adapt to the constant change in requirements ?What was the lesson learnt ?
Lab vs Industry
The software was able to find bugs in a large code base easily with few simple analyses and tricks in a lab environment.In the real world, hundreds of programmers use the tool to check hundreds of different code bases.The types of errors, number of false positives, type of build are all different from what is predicted by lab results.The programmers do not know how the tool works, unlike testers in the lab who have a knowledge of the tools’ internal process.
Customer Interaction
Two scenarios of customer interactionInitial trialLong term useThe trial is a pre-sale demonstration that attempts to show that the tool works well on a potential customer’s code.Sales Engineers educate the customers about the tool.Usually happens over a period of 1 day or 2-3 days if code base is very large
Laws of Bug Finding
You can’t check code you don’t seeAbility to extract code from buildNo access to modify buildTest MachineYou can’t check code you can’t parseNo standard compilers
BUGS
Do bugs matter?No, your tool is broken, that is not a bugMisunderstood errors/bugs are considered as false positivesHow to handle cluelessness ?Do not change the results after an upgradeMyth: More analysis is always good
QUESTIONS
What are the lessons learnt ?How would you commercialize a bug finding tool ?Any experience using a bug finding tool ?
Thank You