Insightful Pattern Mining

The pattern mining framework is a data analytics framework which helps to analyze the data extracted from various kind of logs – transaction logs, webserver logs, syslog, database logs etc. to derive valuable insights that help business stakeholders to identify inefficiencies, streamline work flows and to spot outstanding challenges. The data analytics engine extracts usage and failure patterns from logs which can be used for determining- How the system is being used by live users? What user interface pathways are exercised the most? These insights can be used by testing team to optimize their test, perform risk-based prioritization, identify test coverage gaps and detect failures early.

Failure prediction and modeling is applied to identify system components that are likely to fail and workflows that activate internal fault chains and trigger failures. Anomalies like unusual data patterns, rare event items that are identified which helps to take necessary preventive measures to reduce impact of failures. Root cause of failures are derived from correlation of identified failures with application components.

features

Open Source Solution

Built using Java, Python, Elasticsearch, Logstash and D3.js charts

Data format Agnostic

Supports analysis of unstructured data from varied data sources

Multi Technology /Tools Coverage

Aggregates data from different data sources – application logs to output from application monitoring tools

Intuitive Visualization

Provides visual representation and reports of the analyzed data using D3.js charts

Minimal Optimal Testing

Provides insights on production usage to have the right test coverage

Learning Engine

Builds machine learning model with the analyzed data sets that are stored

ARCHITECTURE

Case Studies

Our clients have achieved significant process efficiencies through implementation of pattern mining

25% reduction in testing cycle time through optimization for a leading American technology conglomerate

Challenges

  • Under coverage of key business scenarios in testing leading to defect leakages
  • Significant amount of time spent on testing scenarios that were never used in production leading to extended testing cycles
  • High defect fix turnaround time

Our solution

  • Leveraged pattern mining framework to identify usage and failures in production
  • Right scenarios were identified for testing based on usage patterns by removing scenarios not used in production resulting in an optimized test suite
  • The framework helped to identify production scenarios that were never tested
  • Through key value pair extraction, it was possible to quickly identify root cause for defects

20% improvement in trading volumes for the world’s leading independent commodity trading and logistics houses

Challenges

  • Application performance issues in specific user machines in different geographic locations leading to low CSAT scores
  • Given the complexity and number of systems in the enterprise landscape it was difficult to identify root cause of defects leading to high defect fix turnaround time

Our solution

  • The framework was used to analyze the logs to determine anomalies and performance bottlenecks that impacted customers
  • Through fault correlation across multiple dependent components in the system, it was possible to diagnose the root cause of failures

8% improvement in defect resolution through common root cause analysis for a large Dutch multinational technology company

Challenges

  • Significant amount of time spent in common root cause analysis leading to high defect fix turnaround time
  • Poor usability due to performance issues in applications

Our solution

  • Leverage pattern mining framework to identify the usage and common root cause analysis across workstreams
  • Identification of non-functional anomalies across technology stack and correlation of affected workstreams enabling reduced defect turnaround time
  • Usage patterns of users across locations were analyzed to provide technical recommendations and user advisory guide resulting in improved user experience

Reduced test cycle time by improving tester effectiveness by 40% for one of the largest consumer division of a multinational financial group

Challenges

  • Delayed and extended test execution as significant amount of time is spent on identifying valid test data
  • Defect leakage due to silent failures in integration layers

Our solution

  • Leveraged pattern mining to identify valid test data combinations and optimized the test data bed
  • Optimization of test data bed in turn reduced test execution cycle time
  • Reduced defect leakage through business event tracking across dependent systems, the silent failures in integration systems were identified which helped to avoid nonfunctional catastrophes in production

Contacts

Jothi Gouthaman