Nobody Knows the Costs They’ve Seen

The biopharmaceutical industry spends many millions every year on process improvement, and accompanying reorganizations and software implementations. Rarely are these efforts ever satisfactorily justified. The reason is that nobody can put an accurate price on the problem they are fixing. There is a common phrase among process improvement gurus that “quality is free”. What is meant by that phrase, generally, is that a) it costs no more – and perhaps even less – to do things well as to do things poorly; and b) quality improvement will pay for itself – through elimination of waste and inefficiency, or the acceleration of profit.

This slogan works well when top executives are championing process improvement, or their company is financially well off. It is similar to the pronouncement that “this initiative is strategically important” – the implication being that we can’t justify it in short-term dollars but we can’t afford not to do it. But even in the best of circumstances, there always comes a time when somebody says, “Was all that quality improvement (or consulting, or new software) worth it?” Sadly, virtually no company in clinical research, be it a biopharma sponsor, CRO, investigative site or software vendor, can answer this question correctly.

Why is this true? We cannot tell if something is “worth it” unless there is some measure we can all agree on that is applied to a situation and then evaluated. Usually, people want to calculate this measure quantitatively and not subjectively. This measure needs to be both valid (a legitimate evaluation of the issue at hand) and feasible (something that data can be collected about in a practical, cost-effective manner). And more often than not, the measures most important to business are denominated in dollars. Which brings us to cost. If we are evaluating dollars, we are usually trying to prove or discover if the effort or time or dollars spent were worth more than that effort or time or expenditure cost us. This is where we fail: nobody knows the costs they’ve seen.

Let’s use an example which is at the heart of a multi-billion dollar industry (CROs). Are they worth it? How do we know? Generally speaking, we know they are worth it if we cannot get the work done any other way, either through a lack of expertise, manpower, geographic reach or other factors. But let’s look at those situations that are not crisis-driven, where a biopharma is considering outsourcing all of their monitoring or all of their data management as an ongoing strategy. This is a decision that can and should be made analytically. On what basis is this analysis proceeding? Do we actually know what it would cost us to do these activities internally?

You would think that a biopharma that has been conducting clinical research for a generation would have such data at its fingertips, but no one does. They may have data, but it is unusual to find the right data. What is the true, entire cost of a monitoring visit? What is the true, entire, fully loaded cost of cleaning a data field on a CRF? What is the cost of making educated guesses about distributing clinical drug supply among trial investigative sites?

Even more importantly, let’s consider the subtler business decisions we make every day. What is the true, fully loaded cost of designing a protocol that meets the biostatisticians’ satisfaction and uses decade-old FDA-acceptable endpoints, but is not likely to make the new compound stand out competitively? What is the true cost of conducting concurrent worldwide clinical trials for the sake of near simultaneous global submissions? What are the relative fully loaded costs of centralized monitoring, regionalized monitoring, contractor monitoring or outsourced monitoring? The questions are endless, and yet we do not confront them, because of what has so far been a daunting irony: it costs too much to find out how much things cost.

We have always contended that CROs in particular, but now even the fattest biopharma, cannot afford not to understand their costs properly. In the CROs’ case, they are nothing if not cost-driven businesses. How can a CRO properly propose a competitive, fair and profitable bid without knowing exactly how much it will cost them to deliver the service? How can a software vendor properly and profitably price their offering (which after all, in itself costs near to nothing to manufacture) without both vendor and sponsor understanding the true cost-savings using the software will entail? How can a sponsor choose services or software or process improvement implementations without knowing what costs are being affected, and what costs will be incurred, in total?

Why is something so important as this so difficult to calculate? For most companies, “fully loaded” cost is a concept buried inside Finance and, even if discoverable, is not calculated in operational terms (data is gathered and analyzed by Finance in financial line items, not project/study/personnel terms). Most companies have never mapped their actual clinical development processes (not their SOPs) for such tasks as protocol development, site selection or data cleaning, and thus do not know how to assemble the proper elements for which to calculate cost. Instead, most companies use incorrect or misleading measures (if they use any at all).

Let’s look at analyzing the use of software in clinical research as an example. Sponsors think they are doing well when they recognize that software can affect the LPLV-DBL interval. But this is a highly misleading measure. Most companies can find examples of very fast database lock times, using traditional paper methods, through the “heroic” efforts by clinical operations staff. This is confusing when trying to compare paper and electronic data management methods fairly. The confusion comes from failing to understand the difference between duration and effort. It is effort that more accurately captures cost. For instance, the more data points there are on a CRF page, there will obviously be more effort in monitoring it. The right measure to calculate cost, therefore, is not DCFs per page (a common metric) but rather DCFs per “x” number of data points. This comes much closer to calculating effort.

How can clinical research companies do a better job of uncovering their true costs? The first step of course is to realize the issue is an important one, and that it will cost money to save money. The second step is to sit down with Finance, Project Management, Clinical Operations and other important sources of puzzle pieces, and put together the pieces to make the picture become clear. Simultaneously, organize Process Mapping efforts at a sufficient level of detail (not too much, not too little) so you can uncover all the steps in completing the multi-dimensional tasks we do in clinical research. Fourth, determine the correct measures which can be denominated in dollars that accurately reflect true costs. Finally, enlist the aid of all those staff who need to make this effort successful; it cannot be done “to them”, but must be done with them, and the results must be demonstrably used for work improvement.

In the end, you may find that some actions or decisions cannot in fact be justified in dollar terms. What we call the “Value of Necessity” (we can’t get the work done without it) may indeed be the most compelling, and appropriate, reason to choose a service or a product. But in all cases, some day or another, you’re going to have to know your costs. If you work hard at discovering them, you will have the concrete basis on which to measure the benefit of the improvements you work so hard to achieve. And instead of singing “Nobody knows the troubles I’ve seen” to anyone who will listen, you can hum “blue skies, nothing but blue skies, do I see.”