Media & Entertainment
Testing Online Games for Reliable Player Experiences
A seven-person QA team executed a large test suite, reported defects and supported regression testing for a Canadian online-game software supplier.
- Duration
- 1 year
- Team
- 7 people
- Test suite
- 40,000 cases
- Scope
- Execution, reporting and regression
Client & project background
Dedicated testing capacity for multiple game systems
The client was a Canadian online-game software developer and supplier serving internationally recognised brands. It wanted to focus development resources on its core business while lowering staffing costs and product risk and improving testing progress and quality.
The client divided testing responsibilities between teams. Shinetech's role was test-case execution, results delivery, bug reporting, progress control and regression testing in cooperation with developers.
The year-long engagement involved one project manager, one senior tester, four intermediate testers and one junior tester.
Knowledge transfer & test scope
Understanding the system before executing 40,000 test cases
Several systems and configurations had to be understood, including T-Server, Author-Server and Game Server. The material included approximately 40,000 test cases, more than 100 Excel documents and 20 requirements or project-change documents.
Client training, requirements review, hands-on use of the games and sample test-case review helped the team understand the application. This initial system familiarisation took two weeks.
The test environments covered Windows 2000, XP, ME and Vista editions with Flash Player 8 or 9. VSTS supported requirements, bugs and task management, and DevTrack was also used for bug management.
Testing & communication
Traceable results and clear defect reports
Each executed test received a Pass, Fail, N/A or Skip result. Failed cases included bug IDs and explanatory comments. Excel reports calculated pass percentages for functional modules.
The team continued discussing requirements with the client, including details missing from test cases and cases that had not caught up with change requests.
Results reports included analysis and periodic statistics for each tester: executed cases, pass percentages, and the quantity and quality of reported bugs. VSTS gave the client visibility into progress.
A standard defect-report format and supporting logs helped developers understand issues. Testers worked with developers through resolution and regression testing; the teams agreed that reported bugs should receive feedback within two days.
Results
Supporting the client's core development work
The engagement helped the client focus resources on its core business, lower staffing costs and product risk, and improve product quality.
Shinetech focused on executing the supplied test cases, documenting results and working with developers through defect resolution and regression testing.