Client’s situation
The client’s project objective was to roll out a new frontend implementation for an important internal system, which is used for client management, policy management, claims, etc. in all client’s branch offices and in back offices in the Group companies. During implementation, the delivery was planned with 6 releases in 5 years in 5 different companies/countries. Each release contained new functionality, which was migrated from the older system. The database layer wasn’t changed during implementation (except some DB query optimization) and the old system had been in operation until the whole system was migrated. New frontend implementation was centrally done by a software company. Deployment of releases and their functional testing were performed by Group company teams locally in their specific environments. The newly developed system performance was important for the client and therefore the client wanted to conduct external and independent Performance testing.
Client’s expectations
- Prepare, conduct, and report performance test of new releases in 5 EU countries
- Find major performance issues
Team
- 1× Test Lead / Senior Consultant (25%)
- 1× Performance Test Engineer / Senior Consultant (100%)
Our solution
The client didn’t have clear performance requirements for the delivery acceptance. It was important to guide each involved Group company through the performance test process from the scratch. After understanding client’s specific requirements (varied from company to company), we prepared and communicated a Performance test strategy and plan. The performance test landscape was prepared in cooperation with local IT support. Load generators and monitoring were prepared and tested. Performance test types, test cases/scripts and performance scenarios were designed including KPI and SLA based on expected usage patterns. The test scripts were subsequently implemented using Apache JMeter (HTTPS protocol) and tested to simulate the expected load.
The performance test runs were divided into two main phases:
- The first one took place in a development environment after development, but before the package was released to the client’s testing. The goal was to uncover some issues as early as possible before the release package was installed on local test environments.
- The second phase was performed after the client’s functional testing in their infrastructure with the objective to perform tests on company-specific HW setup, deployment, configuration, data set, etc. with their specific scenarios (different loads and script combination for each company).
During one release, performance tests were carried out several times with different configurations and intentions (e.g., baseline, load, stress tests). Each test run was reported and proper test results were shared with the interested parties. The results were consulted with the project team and findings were analyzed in cooperation with the local company monitoring team and development team. When the performance test was finished, the final test report was written and shared. Performance testing became part of the testing process within the delivery of almost each release. In some cases, the existing performance test scripts/scenarios were reused in more releases in order to simulate a closer production-like usage. If a performance issue was identified, then the fix was retested in the following release. Long-term cooperation of all involved parties brought efficiency and higher value for the client.
Technologies and Tools
Apache JMeter (HTTPS protocol), Client’s HW monitoring tools.
Client’s testimonial
Managing Director of Austrian Consulting Company: “MOXYMIND specialists have supported us in uncovering crucial performance issues during an international customer project. They are very reliable partners and the cooperation was highly constructive. We are looking forward to our further collaboration!”
