What is Code Coverage
Code coverage is a measure used in software testing. It describes the degree to which the source code of a program has been tested. It is a form of testing that looks at the code directly and as such comes under the heading of white box testing.Currently, the use of code coverage is extended to the field of digital hardware, the contemporary design methododology of which relies on Hardware description languages (HDL’s). Code coverage techniques were amongst the first techniques invented for systematic software testing. The first published reference was by Miller and Maloney in Communications of the ACM in 1963.
In simple words “A code coverage tool simply keeps track of which parts of your code get executed and which parts do not“. Usually, the results are granular down to the level of each line of code. So in a typical situation, you launch your application with a code coverage tool configured to monitor it. When you exit the application, the tool will produce a code coverage report which shows which lines of code were executed and which ones were not. If you count the total number of lines which were executed and divide by the total number of lines which could have been executed, you get a percentage. If you believe in code coverage, the higher the percentage, the better. In practice, reaching 100% is extremely rare.
The person who is very new to code coverage may came across with so many doubts like
“Why should we have to use code coverage”
“What are the benefits we are getting out of it”
So in the following sections, these questions will be answered.
Value of Code Coverage
provides a quantitative measurement of testing efforts.
Can assist in directing the tester’s future efforts.
Can demonstrate redundancy in test cases.
Can be used as entry/exit criteria between test phases.
Code coverage in practice
Usually the source code is instrumented and run through a series of tests. The resulting output is then analysed to see what areas of code have not been exercised, and the tests are updated to include these areas as necessary. Combined with other code coverage methods the aim is to develop a rigorous yet manageable set of regression tests. Code coverage is ultimately expressed as a percentage, as in “We have tested 63% of the code.” The meaning of this depends on what form(s) of code coverage have been used, as 63% path coverage is more comprehensive than 63% statement coverage. The value of code coverage as a measure of test quality is debated.
What is meant by instrumenting the code
Instrumentation means inserting some type of code in the source code or in the byte code, so that it can ensure that all lines of code in the design have been executed
There are many approaches to code coverage measurement. Broadly there are three approaches, which may be used in combination:
| Source Code Instrumentation | This approach adds instrumentation statements to the source code and compiles the code with the normal compile tool chain to produce an instrumented assembly. |
| Intermediate code Instrumentation | Here the compiled class files are instrumented by adding new bytecodes and a new instrumented class generated. |
| Runtime Information collection | This approach collects information from the runtime environment as the code executes to determine coverage information |
To know further information about how instrumentation works in Visual Studio, you can refer to this link
http://blogs.msdn.com/phuene/archive/2007/05/03/code-coverage-instrumentation.aspx
Coverage criteria
To measure how well the program is exercised by a test suite, one or more coverage criteria are used. There are a number of coverage criteria, the main ones being:
* Function coverage – Has each function in the program been executed?
* Statement coverage – Has each line of the source code been executed?
* Condition coverage – Has each evaluation point (such as a true/false decision) been executed?
* Path coverage – Has every possible route through a given part of the code been executed?
* Entry/exit coverage – Has every possible call and return of the function been executed?
For more information regarding coverage criteria refer to the following links:
http://www.bullseye.com/coverage.html
http://www.javaranch.com/journal/2004/01/IntroToCodeCoverage.html
Fooling yourself
Every now and then, I meet somebody who thinks that a body of code is perfect if its unit tests all pass with 100% code coverage. This obviously isn’t true. Code coverage can only tell you how much of your code is being tested. It cannot tell you how much code you still need to write.
And in turn, some folks think that because 100% code coverage cannot be understood to mean 100% correctness, then code coverage isn’t worth anything at all. To me, that’s like saying we should never talk about the temperature outside because by itself it is not a reliable way of determining how nice the weather is.
Unit testing and code coverage are tools. They provide us a way of increasing the quality of our code, but 100% code coverage certainly does not mean 100% code quality. If you want a complete QA effort, one which offers you high confidence that your code is reliably doing whatever you want it to do, then unit testing and code coverage are just a small part of the story. There are many other tools and techniques you should consider.
External links
TOOLS
IBM Rational Test RealTime can profile embedded software for Code Coverage analysis and more
LDRA Testbed measures statement coverage, branch/decision coverage, LCSAJ Coverage, procedure/function call coverage, branch condition coverage, branch condition combination coverage and modified condition decision coverage (MC/DC) for D0-178B Level A.
Netbeans Code Coverage Plugin: emma-based coverage plugin for Netbeans the visualizes unit tests coverage by coloring sources
CodeCover: Extensible code coverage tool supporting Java and COBOL Supports statement coverage, branch coverage, loop coverage and strict condition coverage, fully integrated into Eclipse and licensed under the EPL.
Dynamic Code Coverage: Code coverage tool for Unix C/C++
No comments:
Post a Comment