How to Set up a Pilot Project

There is no doubt that technology has become an integral part of the healthcare industry. From EHRs, to biomedical equipment and wearables, to billing and administrative software .... applications are everywhere .... and we are starting to love them - right?

Well, you know what else is everywhere?  Ideas for new and better technology.  Ideas to change technology that is already in place. Ideas to add more functionality or to use existing functionality in a different way. Ideas that we can't wait to try out as a pilot project ...

How could we not be on board with all the great ideas that aim to solve key workflow issues? Ideas that will fill the gaps in the electronic data sets we so desperately want to fully leverage for artificial intelligence ....

All we need to do is implement them and voila - magic happens ….Unfortunately, that’s easier said than done :(

For most healthcare organizations, there isn’t the same risk culture as there is in the tech industry. Rolling out new applications or making changes to existing workflows isn’t quite as simple as we'd  like. Before we try anything, we need to know it’s going to work. The “fail fast, fail often” mantra is not something that we feel comfortable embracing in healthcare - for obvious reasons. There also isn’t a strong desire to just go for it and risk failing on a large scale .... again for obvious reasons. So what can we do? One of the most popular ideas is to start by introducing the changes with a pilot.

What is a pilot project?

So what exactly is a pilot project? Or what should it be? It is a short-term, smaller-scale version of an initiative. One that is undertaken to determine feasibility. A pilot should help to gather more information on cost and time requirements. It should also assist with identifying and understanding any impact points. Along with areas that need improvement before a large scale implementation takes place.

To set up a pilot project, an organization needs to have a defined problem to address. Along with a potentially viable solution. A pilot is not an exercise in making random changes to see if they work out well.

What are some benefits of completing a pilot?

A lot of the IT projects that are being rolled out have some pretty significant magnitude. Especially when it comes to their impact on workflow. It’s not surprising to have end-users express some hesitation (aka resistance) with new initiatives. By completing a pilot, it may be possible to gain further insight. To identify impacts or opportunities to address before, or alongside, a larger rollout. A pilot project can help with understanding the training and adoption needs. If successful, it may also assist with garnering support for the solution long term.

Another benefit of starting with a pilot group is that there are opportunities to make adjustments. Completing things on a smaller scale often allows a project to be more nimble. There is also a decreased risk associated with unanticipated issues or challenges. (In relative comparison to an enterprise-wide rollout.) Finally, pilots should help determine if the solution achieves the anticipated outcomes. They should allow you to confirm the established key performance indicators (KPIs).

Once you’ve determined that completing a pilot would be helpful -  how exactly do you go about setting one up?

How to define a pilot project using 5Ws and an H.

There are several methods for defining the parameters of a pilot study. The one I find the easiest to work through (and the simplest to remember) is to use the 5Ws and an H.

WHY are you implementing the solution?

Before starting any project, be it a pilot or a full version, you must understand the 'why' behind it. What purpose is driving the need for implementing the solution? What problem or opportunity is the change supposed to address and how is this going to address it? Is the solution meant to resolve an issue completely? Is it meant to improve an outcome or measurement in some way? Before you can move on to the other components of defining a pilot, you will need to know the why. If you don’t know why you are implementing the change - everything else will be extremely difficult. If not impossible to figure out.

WHO will take part?

To assess the long-term feasibility pilot solution it’s important to identify the right participants. Those that will allow for a meaningful interpretation of the results. If the full-scale project is impacts all roles and is enterprise-wide, a pilot with a few highly specialized providers will not be helpful.  It would be better to identify a representative clinical and/or administrative population. One that will allow you to test the pilot outcome and gather the information needed to continue the movement towards the full rollout. Keeping the scale as manageable as possible of course. Along these same lines, it is important to identify participants from the IT department. (I.e. Who will support the pilot from a technical and workflow perspective?)

WHAT is the scope?

Pilot projects are not free …. well at least none of the ones I’ve been involved with are. If the goal is to obtain more information on feasibility, cost, impacts, and timeline - you will need to determine the scope necessary to get that. (It is also important to consider the cost of the pilot itself.) In some cases, it may be possible to implement full functionality with an appropriate pilot group. In others, it might be necessary to scale back to a reasonable number of core functions.

When determining the scope of a pilot, it is also helpful to consider an unfavorable outcome.  Can functionality be rolled back if the pilot is unsuccessful (meaning feasibility on a large scale is not possible)? - It’s great to be optimistic, but it’s not great to be unprepared in the event you need to back something out.

WHERE will the pilot take place?

Determining where the pilot will take place is similar  to defining who will take part. You don’t want to isolate the pilot for an enterprise solution to only a few roles. You also do not want to isolate the pilot to specific areas that are unable to represent the whole. Let's look at a new communication tool as an example. One that is being assessed for comprehensive use by the Surgical group. Piloting use only in the operating room is unlikely to provide all the information needed. For an accurate assessment of  feasibility, a pilot should assess use in all areas the tool will be used in. (Limiting the geography could even introduce new challenges unique to the pilot.) It would be better to select a smaller number of pilot participants across a larger area.

WHEN will it take place (start and end)?

Every single pilot that takes place should have a beginning and an end. If you are attempting to think of exceptions for this …. there should not be any. The purpose of the pilot is to complete a smaller scale implementation. To determine feasibility for a larger or longer term rollout. To gather information to assist with clarifying cost, timeline, and impact. To achieve this purpose - the pilot needs to reach its end date …. So the team can make a decision.

Having a pilot go on for an extended period of time can be very costly. It may also put the organization into a hybrid situation - which is very rarely conducive to improved outcomes. The pilot length should allow for adequate time to assess the desired components. It should not go on longer than that. There needs to be an end date. (Think of it as a decision date if it makes things easier :)

HOW will you know if the pilot is a success?

I’ve mentioned a few times that implementing a change with a pilot group serves a purpose. It is to gather the information that leads to evaluation. If this is the case, then before you begin the actual pilot - you need to determine the evaluation metrics. What indicators will you look at to determine whether your pilot will be successful? What measurements will allow you to calculate the cost and timeline for the full solution? How will you measure feasibility for the organization? In situations where there are no quantifiable metrics, there is an increased risk in pilot failure. Or in pilots running rogue... Having measurable indicators also allows you to make adjustments during the pilot if necessary.

Things to be mindful of when executing a pilot study.

During a pilot project there are a few things to keep an eye on that can help make things more successful.

• Make sure communication paths have been set up for all participants. As well as for any individuals that may be experiencing unanticipated impacts. Let people know that a pilot is taking place. Encourage feedback.
• Keep an eye out for workarounds, or inconsistencies with pilot workflow. Assess whether the training and change management activities are effective with the pilot.
• Pay attention to any technical issues. Make sure they can be promptly addressed and that resolution is effective.

What happens once the pilot project is over?

After a pilot rollout/project, stakeholders should get together to evaluate the initiative and to review the metrics. Once that is complete, the appropriate individuals will need to decide on the next steps for the organization. Will the solution roll out further? Will the pilot group continue to use the solution in its current form? Are changes needed before a larger-scale rollout can take place?  Maybe the pilot was unsuccessful and a rollback is necessary.

Regardless of the decision and the next steps - the information gathered from pilot studies is incredibly valuable. It should be well documented and referenced for future initiatives. Change will continue to happen in our industry and chances are  the option for another pilot will present itself …..