Iterative Optimization

The primary objective of test optimization is to maximize defect yield per cost/time spent on testing. Testing projects often face challenges like bloated regression suite, inadequate test coverage insights and test prioritization driven by SME inputs. Test Optimization Engine is a tool which is built using java as the primary technology and open source similarity analytics algorithms. This tool runs similarity analytics to identify duplicate test cases in a test suite or test cases exercising the same execution paths. This helps optimizing the test suite, identification of common reusable test components like test step and test cases that can be 'called' from other test cases to improve maintainability and re-use. Techniques like clustering and sequencing are used for prioritization of test cases and right resource assignment.

This Solution consists of

  • UI which is built using Angular JS, is used to upload files (test cases) to be analyzed
  • Core where actual computation takes place, includes core algorithm and libraries like Apache Lucene java-based indexing and search technology
  • Rest API calls are used to interface between the Core & UI

features

Open Source Solution

Uses open source Similarity Analytics algorithms

Similarity Analysis

Enables identification of duplicate test cases

Dynamic Sequencing

Order of test case execution is dynamically determined

Clustering

Enables right assignment of test cases to improve productivity

Test Coverage Insights

Provides insight into the coverage/risk, based on the extent of unique test cases executed

Easy Installation

Easy deployment without complex installation process

ARCHITECTURE

Case Studies

Our clients have achieved exciting results in test optimization through implementation of Test Optimization Engine

20% REDUCTION IN TEST SUITE SIZE LEADING TO REDUCTION IN TEST CYCLE TIME FOR LEADING GLOBAL TELECOM CLIENT

Challenges

  • Regression test suite maintenance- the regression suite is large and there is a need to optimize
  • Manual optimization effort is high
  • Regression test execution takes longer time

Our solution

  • Identify duplicate test cases and test cases executing the same path
  • Duplicate test cases were removed, and subset test cases were merged
  • Test cases modularized based on clusters identified

19% REDUCTION IN TEST SUITE SIZE LEADING TO REDUCTION IN TEST CYCLE TIME AND 20% FASTER AUTOMATION THROUGH MODULARIZATION OF COMMON COMPONENTS FOR LEADING GLOBAL BANKING CLIENT

Challenges

  • Bloated regression test pack
  • Insufficient test coverage; thus defect leakage into production
  • Ineffective test automation - multiple dependencies and user interventions
  • Too many test cases in scope for each release, with no prioritization
  • Manual Test case / configuration base of 45,404 Test cases divided into 8 Modules were shared as input

Our solution

  • Applied similarity analytics on test suites using Cognitive engine
  • Functional SMEs verified the accuracy of the recommendations
  • Machine Learning applied to improve the accuracy of recommendations
  • Similarity Analytics and Optimal risk coverage sequencing implemented as weekly exercise during releases

21% REDUCTION IN TEST SUITE SIZE FOR NA BASED LEADING ELECTRIC POWER HOLDING COMPANY

Challenges

  • Regression test suite maintenance- the regression suite is large (~3300 test cases) and there is a need to optimize
  • Manual optimization effort is high
  • Regression test execution takes longer time

Our solution

  • Identify duplicate test cases and test cases executing the same path
  • Duplicate test cases were removed, and subset test cases were merged
  • Functional SMEs verified the accuracy of the recommendations
  • Machine Learning applied to improve the accuracy of recommendations

Contacts

Rohit Patwardhan