Testing and evaluation strategies for QCE Digital Solutions Unit 4
“Design and apply testing strategies, including test data and test cases, to verify a prototype against its requirements, and evaluate the solution using justified criteria”
Testing verifies that a prototype meets its requirements by running planned test cases with normal, boundary and abnormal test data and comparing actual results to expected results; evaluation then makes a justified judgement about how well the solution meets its criteria (reliability, security, usability, fitness for purpose) based on the test evidence and the requirements.
Jump to a section
What this dot point is asking
Building a prototype is not enough; QCAA wants you to prove it works and judge how well. Testing verifies that the solution meets its requirements, using planned test cases and test data. Evaluation then makes a justified judgement about the solution's quality against criteria such as reliability, security, usability and fitness for purpose. Testing and evaluation are distinct, both are assessed in IA3, and confusing them is a common way to lose marks.
Test cases and test data
A test case is a planned check: a description, the input, the expected result and the actual result. Test data should be chosen deliberately to cover three categories:
- Normal (valid) data: typical values that should be accepted (a mark of 72).
- Boundary data: values at the edge of the valid range, where off-by-one errors hide (0 and 100 for a 0-to-100 mark, plus 1 and 101 just outside).
- Abnormal (invalid) data: values that must be rejected (a mark of 250, a letter, an empty field).
Testing only with normal data is the classic weakness; boundary and abnormal data are where defects appear.
| Test | Input | Expected | Actual | Pass? |
|---|---|---|---|---|
| Normal | 72 | Accepted | Accepted | Yes |
| Lower boundary | 0 | Accepted | Accepted | Yes |
| Upper boundary | 100 | Accepted | Accepted | Yes |
| Just over | 101 | Rejected | Rejected | Yes |
| Abnormal type | "abc" | Rejected | Rejected | Yes |
| Empty | "" | Rejected | Rejected | Yes |
The expected column is written before running the test, so a mismatch in the actual column reveals a defect. Covering normal, boundary and abnormal data in one table is exactly the evidence IA3 markers reward.
Types of testing
QCAA expects awareness of several testing levels and kinds:
- Unit testing: checking an individual function or component in isolation.
- Integration testing: checking that components work together, critical for a data exchange where two systems interact.
- System testing: checking the whole solution against requirements.
- Acceptance testing: checking the solution meets the user's needs, often with the user.
For a data exchange prototype, also test the failure paths: a malformed payload, a dropped connection, an unauthorised request. These prove the exchange handles errors reliably, which is a key requirement.
Testing a data exchange specifically
Beyond ordinary input testing, a data exchange prototype needs tests that exercise the exchange itself: send a valid payload and confirm it is received and stored, send an invalid payload and confirm it is rejected with the right status, tamper with data and confirm the integrity check catches it, and simulate a failed transmission and confirm the retry or error handling fires. Each maps to a security or reliability requirement.
Evaluation against criteria
Evaluation is a judgement, not a list of test results. You assess the solution against criteria drawn from the requirements:
- Functionality: does it do what was specified?
- Reliability: does the exchange succeed consistently and handle failure gracefully?
- Security: are confidentiality, authentication and integrity actually achieved?
- Usability: can the intended users operate it?
- Fitness for purpose: does it solve the original problem?
A strong evaluation cites test evidence, acknowledges limitations, and recommends improvements. It explains why the solution does or does not meet each criterion, rather than just asserting it.
Testing versus evaluation
The distinction is examinable. Testing is the process of checking behaviour against expected results, producing evidence (pass or fail). Evaluation is the reasoned judgement built on that evidence about overall quality and fitness for purpose. Testing answers does it work; evaluation answers how well does it solve the problem and what would improve it.
How this appears in assessment
IA3 requires both a test plan with results and an evaluation against criteria, supported by that evidence. The external exam can ask you to design test data for a scenario, identify a missing test category, or evaluate a described solution against a criterion. Practise writing test tables that include boundary and abnormal data and evaluations that quote evidence and recommend specific improvements.
- Testing only normal data. Boundary and abnormal data find the defects; omitting them makes a test plan superficial.
- Confusing testing with evaluation. Testing produces pass/fail evidence; evaluation is the judgement built on it.
- Writing expected results after running. Decide the expected result first, or the test proves nothing.
- Ignoring failure-path testing. A data exchange must be tested for malformed, tampered and failed transmissions, not just success.
- Evaluation with no evidence. Judgements must cite test results and requirements, not opinion.
Exam-style questions
Questions in the style of QCAA exam questions on this dot point, each with a worked answer. They are written by ExamExplained unless tagged "Past paper"; the year shows the paper a question is modelled on.
Practice8 marksA solution accepts a numeric quantity that must be between 1 and 99. Design a set of test cases using normal, boundary and abnormal test data, and justify how this set verifies the input handling against the requirement.Show worked answer →
An 8 mark design-and-justify answer rewards correctly chosen test data across the three categories and a clear rationale.
Design a test table:
| Type | Test data | Expected result |
|---|---|---|
| Normal | 50 | accepted |
| Boundary (low) | 1 | accepted |
| Boundary (high) | 99 | accepted |
| Boundary (just outside) | 0 and 100 | rejected |
| Abnormal | "abc", empty | rejected |
Justify: normal data confirms the common case; boundary data tests the edges and just outside them, where off-by-one errors hide; abnormal data confirms invalid input is rejected. Together they verify the input handling meets the 1 to 99 requirement. Markers reward correct classification and testing both sides of each boundary.
Practice4 marksExplain the difference between testing and evaluation, and why a solution can pass all its tests yet still be judged a weak solution in evaluation.Show worked answer →
A 4 mark explain answer needs the distinction and the reason for the gap.
Testing checks that the solution behaves correctly against specific test cases (does it produce the right output). Evaluation makes an overall judgement of the solution against its requirements and criteria, including usability, efficiency and fitness for purpose.
A solution can pass every test yet be judged weak because the tests only cover what was built; if the solution misreads the requirements, is hard to use, or is inefficient, evaluation against the criteria exposes that even though the code runs correctly. Markers reward the correctness-versus-judgement distinction and a valid reason for the gap.