Testing Based on What Could Go Wrong

Qualify Assurance (QA) plays a critical role in software development, ensuring that every component functions as expected. However, many QA professionals fall into the trap of focusing solely on validating expected behaviors without dedicating sufficient effort to uncovering potential failures. To perform effective QA, we must consider both dimensions of “as expected”: ensuring the system does what it should and ensuring it does not do what it shouldn’t. These two perspectives require distinct approaches and mental models.

The Two Modes of QA Thinking

When reviewing a software system, it is essential to separate the processes of confirming normal operations and identifying potential failures. These two modes of thinking involve different cognitive processes, and blending them may lead to incomplete analysis. Here’s how to approach them systematically:

1. Verification & Validation (V&V): Understanding Expected Behavior

• The first pass through requirements, design, and implementation should focus on understanding how the system is supposed to work.
• This includes verifying that all defined functionalities are covered and that they align with business objectives and user expectations.
• Techniques such as test case derivation from requirements, reviewing design consistency, and executing functional tests can be applied in this phase.

2. Risk Identification: Uncovering What Could Go Wrong

• The second pass should focus exclusively on identifying risks and vulnerabilities.
• This requires a shift in mindset from validating correctness to exploring possible failure points.
• Techniques such as exploratory testing, boundary value analysis, failure mode identification, and security threat modeling can be particularly useful.

Risk Identification: The First Step in Risk Management

Risk Identification is the foundation of an effective risk management strategy. By systematically analyzing potential points of failure, QA professionals can proactively address issues before they become costly defects. Here’s how to integrate risk identification into your QA process:

• Review Artifacts with a Failure-Oriented Mindset: Examine requirements, design, and code with the intention of finding ambiguities, inconsistencies, and areas susceptible to failure.
• Leverage Past Defects: Use historical defect data to predict and prevent similar issues in new implementations.
• Apply Exploratory Testing: Go beyond scripted test cases to uncover edge cases and unexpected failures.
• Engage in Cross-Functional Collaboration: Work closely with developers, designers, and product managers to anticipate risks from multiple perspectives.

A well-rounded QA approach requires balancing validation with risk identification. By consciously separating these two phases, QA teams can ensure thorough coverage of both expected and unexpected behaviors, ultimately leading to more reliable and resilient software systems. Implementing this dual approach is a key step toward mastering the art of risk-based QA and delivering higher-quality products.