Independent Verification and Validation of Software pays of (big time!)

Just finished a paper Evaluating the effectiveness of independent verification andvalidation by Arthur et.al.  What the authors state is not that surprising: independent verification and validation (V&V) efforts result in significantly better results than V&V done by the same team.  This paper is interesting for a number of reasons: The result stem from a controlled […]

Full Article

Just finished a paper Evaluating the effectiveness of independent verification andvalidation by Arthur et.al.  What the authors state is not that surprising: independent verification and validation (V&V) efforts result in significantly better results than V&V done by the same team.  This paper is interesting for a number of reasons:

  • The result stem from a controlled study of two identical projects, one realized with "normal" V&V (done by the team), the other using independent V&V
  • The independent V&V group detected significantly more critical faults (97 vs. 58).  Why? The authors conclude that a new class of faults was identified by the IV&V group: "The IV&V team from Group 1 identified an additional class of critical faults not found by Group 2. More specifically, a substantial number of the critical faults reported by Group 1 stem from the detection and recording of ambiguous or unclear statements in the requirement specifications and design documents. (…) Group 1 reported 40 such faults. (…) Group 2 reported only one such fault."
  • Even though both projects were completed and considered a success, things during acceptance testing looked different.  An independent acceptance test suite consisting of 36 tests was provided.  The non-independent V&V group’s software passed only 11 tests, while the independent V&V group’s software passed 33.