Security Testing should be process driven and not tool driven. Running a high-end tool and relying on its scan results for identifying security flaws is not a comprehensive way of testing an application. I am not denying that tools are helpful, undoubtedly they are; tools help in finding a wide variety of vulnerabilities present in all the places of the application. But, these tools check only for some defined set of vulnerabilities, and in reality there are many issues that must be checked over and above what is done by the tools, which requires human efforts and analysis. So, if your application testing methodology is solely dependent on any tool, then it is of course questionable. With regard to cut short the project schedules and to make testing process simple and feasible for everyone, most of the security companies follow this tool based approach. Use of high end scanners/fuzzer’s/tools also increases their credibility factor to a great extent in the market.
The point I want to make here is that just running tools is not enough; security testing must be done differently for different applications. Scanners and tools have a common way of testing the applications; mostly dependent on sending exorbitant number of attack vectors in various input fields and analysing their response. Now, whether that accounts for all the security flaws in the application is still ambiguous and the analysis cannot be considered even exhaustive if not fool proof. At times during our testing even the standard attack vectors have to be tailored and used, depending on the inferences drawn about any preexisting security control in the application, which the tools cannot do. And the application specific cases are never accounted by the tools. Thus, tool based testing approach does not work in all scenarios. What most of the security companies do to make the approach a little more elaborate is to go step further and try to identify and remove the false positive reported by the tools so that they can present a much cleaner report for the application.
So, now how do we know who is testing better?
There are many well known application security scanners/tools which are doing fantastic job in the market. However, as said before if the tools had done all work of application security, the security testing companies would have never been required. Over and above the name of the high-end tools used by the companies we need to go ahead and check how they use them in their testing processes.
It is very important that the security companies must evaluate a tool and build a process around it. If they are sound about security testing, they should have built a stable process, which understands:
1. How much portion of testing is done by the tool that they use?
2. How are tool results used and verified?
3. How to identify areas those are not covered by the tool and test them effectively?
The security companies must use have subtle process to ensure that they are able to accurately look for maximum security test cases in the application in a less time. The tool usage must be a part of the process and the companies must very well blend it with their manual testing to get a quick and application specific test report. If any testing company is comprehensively able to answer the above questions I guess, then it is doing a good job and is keen on identifying security vulnerabilities in the application, correctly.
Posts
