Just finished a paper Evaluating the effectiveness of independent verification andvalidation by Arthur et.al. What the authors state is not that surprising: independent verification and validation (V&V) efforts result in significantly better results than V&V done by the same team. This paper is interesting for a number of reasons:
- The result stem from a controlled study of two identical projects, one realized with "normal" V&V (done by the team), the other using independent V&V
- The independent V&V group detected significantly more critical faults (97 vs. 58). Why? The authors conclude that a new class of faults was identified by the IV&V group: "The IV&V team from Group 1 identified an additional class of critical faults not found by Group 2. More specifically, a substantial number of the critical faults reported by Group 1 stem from the detection and recording of ambiguous or unclear statements in the requirement specifications and design documents. (…) Group 1 reported 40 such faults. (…) Group 2 reported only one such fault."
- Even though both projects were completed and considered a success, things during acceptance testing looked different. An independent acceptance test suite consisting of 36 tests was provided. The non-independent V&V group’s software passed only 11 tests, while the independent V&V group’s software passed 33.