Common Types of Application Testing

Have you ever been in a situation where an IT application was changed in some way and as a result, an unanticipated issue occurred? Chances are somewhere in the conversations that followed, at least one individual asked the question "did we test that?" ….

Software testing is something that everyone loves to hate. It can take an enormous amount of time and doesn’t always result in finding anything awry. In fact, its a great day for a QA (quality assurance) tester if everything works perfectly and there are no issues/bugs identified.

It also happens to be a great day if a significant issue is found ….. wait ….. what?

The purpose of application testing is to find any defects that may be present with the computer code. It is also to uncover any unanticipated issues and to resolve them before implementing the change for users. The goal is to reduce the risk of having an IT-related issue occur in the production environment. In the field of healthcare - this is huge. (It’s also important in other industries but my focus is on this one.)

Testing often gets a pretty bad rap. There can be pushback both in the desire to test and on the timelines that are being requested to complete testing activities. I can’t begin to count the number of times I’ve heard someone say "we do this all the time and nothing ever breaks." ... or ... "Testing it a waste of time that will delay things …." A common reason behind wanting to skip or cut testing short is a desire to have a change that benefits end-users in production as soon as possible. There is no doubt in my mind that in healthcare, every individual who participates in an application change has only the best intentions in mind. In no way do any of them want there to be a negative or unanticipated impact.

As more and more technology finds its way into the daily work of clinicians the testing of a change becomes ever more important. There are several different categories or types of testing that an organization can perform. Each of these has a different intent or purpose. (Note: some categories may have a different name in your organization, but the general purpose for each should align.)

Five common types of testing for healthcare applications:

1)  Unit Testing

Unit testing consists of an isolated evaluation of whether the code change made produces the anticipated result. To follow best practice, this type of testing should be done by someone other than the developer or analyst that made the change. This is to ensure there is a clear understanding of both the action and the goal outcome(s). Several rounds of unit testing may occur. (The extent of the test scripts should correlate to the size of the change and/or the complexity of the application.) An example of unit testing a new lab item is to confirm the correct form displays when the order opens. And, that option fields on the order form present the correct choice options (if applicable) for the user.

2)  Functional / Integrated Testing

Functional/integrated testing confirms the functionality and technical workflow perform as expected. It includes testing both within the application and between any connected systems. For example, functional testing of a lab order item may include such things as ensuring that:

• it s in the correct location in the order search/browse,
• only the appropriate user role can enter the item,
• it appears as expected on the orders tab,
• it transmits appropriately to the lab system/module,
• the status of the order updates as required,
• results file and display correctly, and
• the order item and/or result components appear as expected on any reports.

Note: an integrated test cycle confirming behavior between two systems does not need a code change to have occurred in both. For example, testing between a biomedical device application and the electronic health record (EHR ) if either has a change. This can ensure that information still flows from the device to the connected EHR. (e.g. heart rate monitors or IV pumps).

3)  User Acceptance Testing

User acceptance testing (UAT) confirms that the changes made produce the outcome anticipated by the end-user(s). It is often also used to validate that the design and build meet the requirements that were communicated earlier in a project, or in a change request. With UAT, users interact with the application in a way that mimics the production/live environment as closely as possible. Patient and/or clinician scenario-based test scripts are often used to mimic workflows.

4)  Regression Testing

Regression testing is used to confirm that there are no impacts to existing unchanged functions in an application. Essentially, that a change happening in another area is not causing undesirable impacts. For example, steps to implement hotfix to correct a specific issue/bug, should include validation testing on other areas of the application to confirm there are no negative impacts. (Meaning areas that were not targeted by the hotfix code change.) Regression testing is typically much shorter in duration than the other test cycles but is just as important.

5)  Load / Performance Testing

Load/performance testing evaluates how the application(s) responds with a high volume of users and transactions. (This volume is often set to be at or above production levels.) It is quite common for a separate test environment to be set up with a reduced scale when compared to the production/live environment. (This is true about both computing power and data contents.) With this difference in scale, it is important to confirm there are no negative impacts with an increase in data processing and/or user volume. An example of when load testing could be used  is when assessing a change to ePrescribing functionality. - It may be important to confirm that increasing the concurrent users does not result in the transaction time increasing. (If a notable difference in transaction times occurs this would have a dramatic impact on workflow for providers. Especially when multiplied by their patient volume in a single day.) To test interactions of this type, it may be possible to set up an automated load test that increases the concurrent user count.

Ongoing management of test scripts is essential.

In every organization there are going to be times when an unanticipated event occurs. Even with the most rigorous quality assurance and testing processes in place. As with any other aspect of life and work, we must learn from mistakes/errors and take steps to ensure they are not repeated.
Testing and quality assurance should be ongoing activities. Regular test script updates need to be made to incorporate new or changed functions. Team members should also review documents for accuracy and be sure to update as part of lessons learned activities. Test scripts and testing scenarios are living documents that will grow more robust over time. With the proper structure they can be leveraged to minimize risk with each and every change in the technology environment.

"Testing leads to failure, and failure leads to understanding." - Burt Rutan