Just because every line of your code is run at some point in your tests, it doesn't mean you have tested every possible scenario that the code could be run under. To truly understand what code coverage is, it is very important to understand what it is not.Ī couple of answers/comments here and on related questions have alluded to this:īTW, while code coverage is a good metric of how much testing you are doing, it is not necessarily a good metric of how well you are testing your product. Even if static code analysis reports that 100% code is covered, it does not give reports about your testing set if all possible code coverage is tested. code not covered by any other function call. Note: Static code analysis will find if there are any unreachable code or hanging code, i.e. Inheritance Coverage: In case of object oriented source, when returning a derived object referred by base class, coverage to evaluate, if sibling object is returned, should be tested.Testing only one of these could result in 100% code coverage as each line is covered, but as only one of seven options are tested, means, only 14.2% coverage of parameter value. Failure to test each possible parameter value may leave a bug. For example, a string could be any of these commonly: a) null, b) empty, c) whitespace (space, tabs, new line), d) valid string, e) invalid string, f) single-byte string, g) double-byte string. To check if all possible values for a parameter are tested. Entry and Exit Coverage: Test for all possible call and its return value.Also, if we have assumption on max limit, then, if feasible, test maximum limit times and, one more than maximum limit times. Loop Coverage: means, has every possible loop been executed one time, more than once and zero time.Decision coverage: Not just boolean expressions to be evaluated for true and false once, but to cover all subsequent if-elseif-else body.Condition coverage: All boolean expressions to be evaluated for true and false.But additional criteria to be covered are to what extent is the source code covered by the set of test cases.Īs mentioned in above answers, there are various coverage criteria, like paths, conditions, functions, statements, etc. There are other metrics you should use along with code coverage to ensure the quality.Ĭomplementing a few points to many of the previous answers:Ĭode coverage means, how well your test set is covering your source code. We have actually three code coverage metrics - coverage from unit tests (from the development team), scenario tests (from the test team) and combined coverage.īTW, while code coverage is a good metric of how much testing you are doing, it is not necessarily a good metric of how well you are testing your product. I've also heard there's the gcov tool for GCC, but I don't know anything about it and can't give you a link.Īs to how we use it - code coverage is one of our exit criteria for each milestone. If you are a C++ shop, Intel has some tools that run for Windows and Linux, though I haven't used them. You can also roll some custom tools, like this article describes. NET shop, Visual Studio has integrated tools to collect code coverage. Our team uses Magellan - an in-house set of code coverage tools. A good tool will give you not only the percentage of the code that is executed, but also will allow you to drill into the data and see exactly which lines of code were executed during a particular test. Code coverage is a measurement of how many lines/blocks/arcs of your code are executed while the automated tests are running.Ĭode coverage is collected by using a specialized tool to instrument the binaries to add tracing calls and run a full set of automated tests against the instrumented product.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |