Some of this is taken from ideas I've picked up from various books, talking to knowledgeable people, and experimentation when doing audits in the past. One notable online resource I've drawn on is an interview with John Viega discussing the use of his ITS4 tool which used to be found at securityportal.com. Unfortunately, it is now a 404.
A lot of this took a while to get straight in my head when I attempted it first, so hopefully it may be of benefit to others. Of course, what I find best may not be best for others, and there may be better methods of approaching this. Any feedback or suggested improvements are very welcome.
A software audit is an inspection of software design and implementation. This involves analysing the documentation and source code of the software manually and with the use of tools.
Software is audited to ensure that it meets best practices for secure programing, to reduce the risks which may be introduced by use of the software. A critical review of software by parties not directly involved in its development is often the most effective method of discovering security flaws.
As the reviewers provide proper feedback and assistance to developers, it is expected to improve the future quality of software produced and consequently reduce the cost introduced to the development cycle by the review process.
A software audit begins by looking at the design of the software and determining if this design is sufficient to meet the particular security requirements of the application without introducing any inherent vulnerabilities. If this design is deemed satisfactory then the source code of the software is analysed to ensure that it accurately implements the functionality of the design. No more, and no less. It is also analysed to ensure that no vulnerabilities are introduced due to the manner in which it is implemented.
The vendor of the software should give reasonable notice to help ensure that the security team are involved early enough with the project to assist in identifying security requirements.
Any documentation should be presented for review before the coding begins. A short description of the software being audited is a requirement. This is a simple description of the software, its functionality and its implementation.
The auditor should have at least a reading knowledge of the languages and technologies used in the software and a good grasp of secure programming principles. The auditor is required to thoroughly review the code and present the results in an inspection results report which shall be in an agreed format.
For trying to understand a large amount of software from a security perspective it can often be helpful to look at its inputs and outputs and the external programs it calls or communicates with. Any computer program can be viewed as a function of its inputs that works like so: input -> processing -> output. The processing is influenced by its inputs, which in turn produces the outputs. We are primarily interested in the inputs to the program to see where a malicious user can influence the code we are reviewing or use the code as a launchpad to influence external programs or libraries which trust it.
Tools can help with this process quite significantly. Using rats for C/Perl/Python/php, or flawfinder/its4/lclint for C can spot the inputs to a program very quickly and identify areas of possible misuse. Non security specific static analysers tend to be of higher quality however, and their findings are not to be ignored. Interestingly, a recent paper on statically detecting buffer overflow problems has had its results implemented in a development release of lclint.
In the absense of analysis tools for the software being reviewed, then a simple script using the tool grep on unix or cygwin can be very helpful in finding functions associated with inputs and or known pitfalls. For example, a grep for use of the Request object, the Response object and the creation of new instances of COM objects in Active Server Pages will generally find most inputs and outputs in that code.
Using the list of possible faults gained from the above techniques, inspect the inputs which are not typical of the language or platform being reviewed but arise as a result of the program's function. For example, a GUI application may need to validate the input from a text box before sending it externally, or a general purpose scripting language being used as a CGI script will take inputs from the user via libraries designed to parse the querystring, POSTed data and the pathinfo given to the program. The webserver may also make user supplied data available through environment variables. Most tools allow the option of adding new functions to their databases.
Once you have exhausted your options for quickly building a list of possible faults, work through them to verify if there is a fault or a vulnerability. It can often be useful to either work forward from the inputs or work back from possible misuse of improper input. A mixture of both approaches can also be most beneficial in code where input points far outweighs output points, and vice-versa.
Remember, part of being secure is knowing what your state is. Make sure that the result of using a particular function or operator is defined and safe for all possible values. If you cannot define the result using the language documentation, then you cannot be assured that the code is secure. Require some reasonable sanitising or validation of the operands and/or parameters in that event.
Challenge your assumptions! Do not to assume that things external to your program will be as the programmer expected them to be. This is especially relevant with respect to filesystems and the environment. Do not assume that a piece of code will not be reached. Think about how you might force the program into unlikely situations, and require that these unlikely situations be accounted for and dealt with in the code.
This is tedious, but is far easier once the above steps have been taken, as you will have developed a familiarity with the code from the point of view of its security weaknesses and properties. Familiarity with often used functions and various idioms used by the programmer(s) will result in the reviewer getting bogged down with false positives less and spot problems a lot faster. Stepping through code at this stage should quickly reveal any issues that have not come to light already and serve to double check the obvious issues that arose initially. Any new issues that arise should be checked as above.
The Inspection report must be completed and the entire report presented to the vendor. Where possible, a suggested fix should be included with the requirement, but this may not always be possible if it is unclear as to the intentions of the programmer. In the case where it is known that the fix is notably error prone (I.e, requiring use of strncpy() over strcpy() ) then a suggested code fix is preferred.
Jerry Connolly,
jerry@nologin.net