Verifying, validating, evaluating and maintaining an enterprise system: HSC Enterprise Computing Enterprise Project
“Verify and validate an enterprise computing system, including evaluating test data, trialling the operation and maintenance documentation, reviewing the impact of system implementation within relevant environments, modifying designs to improve functionality, and testing, evaluating and maintaining the developed enterprise computing system”
Verification checks the system was built right; validation checks it is the right system for users. Evaluate thorough test data (normal, boundary, invalid), trial operation and maintenance documentation with real people, review the implementation's impact against success criteria, modify designs and retest, and maintain the system through corrective, adaptive, perfective and preventive maintenance.
Jump to a section
What this dot point is asking
This is the final stage of the Enterprise Project. You need to verify and validate your system, and NESA lists the activities involved: evaluating test data, trialling documentation, reviewing the impact of implementation, modifying designs, and ongoing testing, evaluation and maintenance.
The answer
Verification and validation
- Verification: "Did we build the system right?" Does it meet the specification and work correctly (tests pass, calculations correct)?
- Validation: "Did we build the right system?" Does it meet the client's and users' real needs in their environment?
A system can pass every test (verified) and still fail users (not validated), for example if it is too slow to use at a busy counter.
Evaluating test data
- Design tests using normal, boundary and invalid data, with expected results.
- Record actual results in a test table; investigate and fix every mismatch, then retest.
- Use realistic volumes of data to test performance.
- Evaluate whether the test data was thorough enough: did it cover every requirement and likely error?
Trialling the operation and maintenance documentation
- Operation documentation (user guides, help screens, quick reference cards) is trialled by real users completing tasks.
- Maintenance documentation (data dictionary, system diagrams, configuration, backup and recovery steps) is trialled by someone other than the developer.
- Gaps and confusion lead to revised documentation or interface changes.
Reviewing the impact of implementation within relevant environments
Consider the effect on:
- Users and the organisation: productivity, workload, satisfaction, training needs.
- Customers or the public: service quality, privacy, accessibility.
- Other systems: integration with existing software and hardware.
- The environment and society: energy use, e-waste, social and ethical effects.
Measure against the success criteria set in the problem definition.
Modifying designs to improve functionality
Use evaluation findings to improve the system: fix faults, simplify workflows, improve the interface or add missing features. Retest after every change (regression testing) and update documentation.
Testing, evaluating and maintaining
- Ongoing testing and monitoring of performance, errors and security.
- Evaluation against success criteria at set review points.
- Maintenance: corrective (fix faults), adaptive (respond to changed requirements or environments), perfective (improve), preventive (reduce future problems, such as updates and backups).
Evaluating a student's canteen pre-order system.
- Verification: 24 tests with normal, boundary and invalid data; 22 passed. The two failures (orders at exactly the 10 am cut-off were rejected) were fixed and retested.
- Validation: in a two-week trial, canteen staff said orders arrived as expected, but students wanted order history. Success criterion "average queue time reduced by 30%" was met (reduced by 35%).
- Documentation trial: a new canteen volunteer followed the user guide and could not find how to mark an order collected. A screenshot and step were added.
- Modification: order history added in version 1.1, then regression tested.
- Maintenance plan: weekly backup check, term review of menu and prices (adaptive), and a feedback form for bug reports (corrective).
- Using verification and validation interchangeably
- They answer different questions.
- Only testing valid data
- Boundary and invalid data reveal the most errors.
- Stopping at "it works"
- Evaluate impact, users' needs and maintenance.
Practice questions
Original practice questions graded from foundation to exam level, each with a full worked solution. Try them before revealing the solution.
foundation3 marksA booking form accepts a party size from 1 to 12. Create a test table with one normal, two boundary and one invalid test.Show worked solution →
| Test data | Type | Expected result |
|---|---|---|
| 4 | Normal | Accepted |
| 1 | Boundary | Accepted |
| 13 | Boundary (just outside) | Rejected with an error message |
| "ten" | Invalid | Rejected with an error message |
Marking guide: 1 mark for normal, 1 mark for correct boundary values, 1 mark for invalid data with expected results.
core4 marksExplain why trialling operation and maintenance documentation is part of validating an enterprise system.Show worked solution →
A system only meets users' needs if people can actually operate and support it. Trialling the operation documentation (asking real users to complete tasks using only the user guide) shows whether instructions are clear and complete; if users get stuck, the guide or the interface needs changing.
Trialling the maintenance documentation (asking another person to back up, restore or update the system using only the technical documentation) shows whether the system can be kept running when the original developer is unavailable. Both confirm the system is usable and sustainable in its real environment, which is validation rather than just verification of features.
Marking guide: 2 marks for operation documentation, 2 marks for maintenance documentation, with the link to validation.
exam6 marksSix weeks after implementing a new online stock system, a hardware store evaluates it. Describe how the store should review the impact of implementation, modify designs and plan maintenance.Show worked solution →
- Review the impact
- Compare results with the success criteria and pre-implementation data: stockout rates, time spent on stocktake, ordering errors. Survey and interview staff about workload and usability, and check effects on customers (fewer "out of stock" complaints) and on other systems (accounts, supplier ordering).
- Modify designs to improve functionality
- Suppose staff report that the reorder screen takes too many clicks and suppliers' codes are confusing. The designer adds a one-click reorder for low-stock items and displays product names alongside codes, then retests with staff (regression testing to make sure existing features still work).
- Plan maintenance
- Schedule backups and restore tests, apply software updates, set up a process to log and fix bugs (corrective maintenance), adapt the system when suppliers or tax rules change (adaptive), and review performance quarterly for improvements (perfective). Update operation and maintenance documentation with every change.
Marking guide: 2 marks for reviewing impact with evidence, 2 marks for modifying designs with retesting, 2 marks for a maintenance plan.