Rahul Sanjay, AI Infrastructure And Quality Engineering Architect, On How Process Mining Reveals Which Enterprise Tests Still Matter

The problem starts when a test suite grows faster than teams can reassess it. Large enterprises can accumulate thousands of tests through software upgrades, ERP migrations and configuration changes. Different teams add scripts at different times, usually in response to the requirements they had at the time.

Several years later, the company may have a large regression suite without a clear picture of how closely it matches current business activity. Some tests remain essential, while others cover processes that employees no longer use. New workflows may also appear without being added to the suite. The problem is therefore broader than test execution. Before asking how quickly a test can run, the team needs to know whether it is still the right test.

This is where Opkey's Test Discovery changes the usual testing workflow. The product connects to an ERP environment and examines historical tests alongside application activity and configuration logs. It then compares the processes represented in the test suite with the workflows found in the system.
Opkey calls the approach "Test Mining." It combines process-mining methods with test modelling and automated test creation. The aim is to identify processes that may be over-tested, under-tested or absent from the regression suite.

Opkey had been developing this idea before the formal launch. In October 2021, the company described using process intelligence to analyze ERP logs and generate automated tests based on the processes it discovered. Test Discovery packages more of that work into a separate discovery layer for QA teams.

Process-mining tools are generally used to reconstruct how work moves through an organization and find delays or deviations. A company might use them to study an order process, for example, and see where employees follow different paths through an ERP system. Test Discovery uses a method similar to quality assurance. Once the software identifies the processes people follow, it compares them with the organization's existing test coverage.

The growth of process mining has attracted substantial investment. Celonis secured another $1 billion in August 2022 at a $13 billion post-money valuation. Microsoft also acquired Minit earlier that year to add process-mining capabilities to Power Automate. Opkey is applying the same basic source of evidence, system activity, to decisions about software testing.

During the development period, I worked with CresTech and Opkey and contributed to the re-architecture of parts of the system to support domain-specific language understanding, conversational search, and semantic retrieval. Related systems work covered natural-language capabilities used in test discovery.

An ERP process may be understood by a business analyst, while the automation is maintained by a separate engineering team. The system therefore needs a way to interpret the relationship between business requirements, existing tests and observed application usage. Natural-language interfaces can also make this information more accessible to people who understand the business process but do not write automation scripts themselves.

Yes. A successful result only tells you that a particular script completed according to its assertions. It does not prove that the script covers the workflow employees now use. Consider a procure-to-pay process that has been modified repeatedly. The regression suite may contain scripts for purchase orders, approvals and payments. If employees now follow a different approval route, every old script could pass while leaving the current workflow poorly covered.

Application logs provide another reference point. If the logs repeatedly show a workflow with little automated coverage, the team can investigate that gap. The same comparison may reveal large numbers of tests for processes that are rarely used.

No. Usage frequency is evidence, but it is not a complete measure of business risk. A workflow may be used only a few times per year and still be important because it handles financial reporting or regulatory obligations. Such a process may require extensive validation even if it rarely appears in activity logs.

New features pose another limitation because they lack historical usage data. Process discovery cannot tell a company how much risk it should accept or whether a process is important enough to require additional controls. Those decisions still belong to the testing and business teams.

The idea of test discovery is a shift from the way test automation has traditionally been approached. Most testing tools have focused on the execution and maintenance of scripts. Selenium made browser automation common and continues to be updated, with version 4.5.0 released during the week of Opkey's product launch. Vendors have also introduced no-code interfaces and self-healing systems to reduce the workload of maintaining scripts.

Test discovery moves the starting point. The system begins with evidence of application usage and asks which tests to include. Execution comes later. That can prevent teams from spending time automating a test suite before checking whether its coverage matches current business processes.

The approach is especially relevant when organizations are already deciding which processes and tests should survive a major system change. ERP migrations and large cloud transformation projects are strong use cases. During a migration, a company must decide which historical processes to carry over into the new system. Simply transferring every existing test can reproduce outdated assumptions. Removing old tests without studying process data can leave important gaps. A discovery layer gives the team a clearer basis for deciding what should be retained or rewritten.

It can also help organizations with several teams maintaining tests across the same ERP environment. The comparison between usage data and test coverage gives those teams a shared view of what is being tested.

Opkey says Test Discovery can create and maintain test documentation and generate test data based on an organization's ERP configuration. The company also says its library contains more than 30,000 prebuilt test components to address identified gaps.

Those figures come from Opkey. Their practical usefulness will depend on the customer's ERP configuration and the amount of customization in its environment. The same qualification applies to adoption figures. Opkey says it has more than 250 enterprise customers. Its $8 million financing round, led by Vertica Capital Partners, closed in the first quarter of 2022 and was disclosed publicly in August.

I do not think process data can replace testing strategy. Someone still has to decide which failures present the greatest risk and which edge cases require attention. Historical data also describes how an application has been used. It does not always show how to use the application. A frequently observed process may reflect a workaround or an inefficient practice that the company does not want to preserve.

The value of test discovery is narrower and more practical. For organizations managing thousands of tests across complex enterprise systems, it can reduce the search space before engineers begin writing or running scripts. The final decisions still require people who understand the software and the business process behind it.