A few thoughts after the intelligent comments, additional info, sound and fury:
- Microsoft is in the very rare top tier of companies spending time and money on security. In gross $ and time probably number 1 and very high on a percentage of security to software development time. They are also among the most attacked. So the information they provide on fuzz testing effectiveness and other parts of the SDL is an important data point.
- Microsoft’s approach with white box testing and their SAGE tool is useful info for the right people in a vendor’s development team. Again, Daniel will blog on this next week.
- In case I didn’t emphasize it enough, one of Matt’s important points was, “One of my conclusions (which I was pleased to hear echoed in the Microsoft talk) is that no single tool is best, no single approach is adequate–and that there are different types of fuzzing users that will require different feature sets.” Read his full blog entry on this that includes a challenge to a young researcher to do a fuzzer bake-off project rather than develop another fuzzer.
- I would really like to see a bake-off of fuzz testing solutions.
- I buried the lead and put the most important point last, vendor’s need to be fuzz testing their products. So whether it is Mu, Wurldtech, a collage of open source, or home grown tool is still not the most important issue in the control system community, unfortunately. Many more vendors have added fuzz testing to the SDL than five years ago so the trend is positive, and the fuzz testing solution vendors have helped this happen. Hopefully ISCI will help even more. Asset owner’s should be asking for the SDL of their vendors. If it is not readily available, yellow or perhaps red flag. If it can not be explained consistently, red flag. If it does not include fuzz testing, red flag. As has been pointed out in almost every presentation, even those that were not fans of my post, dumb fuzz testing finds exploitable vulns in many products that have not been fuzzed by the vendor.
Some vendors have reached out to provide more information on their approach, and I’ll have our offensive security guys follow up on this.R