I think the word ‘DIY’-Do It Yourself is one of the most popular words that I hear lately and although I can’t put my finger on it, DIY stars in almost every field imaginable. It starts with planning our vacation on-line without going out to meet a travel agent, knitting warm scarves for the coming winter instead of buying one, decorating an event hall for a party by arranging thousands of flowers in amazing combinations that can be found on-line and furnishing our apartment with furniture that we carried from IKEA and of course built ourselves using their guide and all of it without professional help!

Don’t get me wrong, I have nothing against DIY; I admire people that can do things by themselves and I hope one day I will DIY stuff instead of buying ready-made things from the mall or calling for a decorator to arrange flowers for my special event.

But my question is, can anything be DIY? Or is DIY and online self-service just one more trend that will die out and we will go back to old traditions of using professionals?

I am a Duchenne researcher. As scientists, we believe in the principle of doing thing ourselves and constantly challenging ourselves with new experiments. We also believe in reaching out to the right people for advice and assistance. When we buy ready-made reagents, antibodies and solutions, it’s for only one reason: allow us keep focused on what we need to do, our research! This is my motto and I believe in it. Unfortunately, we can’t do everything by ourselves, but if we find the right people with the best solutions for our needs, we can reach our goals easier, better and faster.

Medical device clinical trials are experiments, designed to test and prove or disprove a scientific hypothesis and determine the clinical endpoint.

According to the NCI definition – “the clinical endpoint is main result that is measured at the end of a study to see if a given treatment worked (e.g., the number of deaths or the difference in survival between the treatment group and the control group). The clinical endpoint is decided before the study begins.”

We can readily see that in order to calculate the clinical endpoint, we need to collect data accurately and in such a way that the data can be easily reduced and analyzed.

Translated into more specialized terms from the world of computer science, we say that we need to start with a good data model, collect data, validate it and populate the data model with the data we collected. Then – we can calculate the clinical endpoints.

And this is why I am skeptical about giving sponsors of medical device clinical trials self-service tools for building an EDC (Electronic Data Capture) application. As a matter of fact, it can be a very bad idea as witnessed by the hair-raising story of the medical device vendor who performed a self-service study without collecting the primary endpoint.

Brilliant in biotech and clueless in computer science?
For some time now, I’ve been working with several brilliant Israeli R&D companies that have taken amazing ideas from concept to real products. They had an idea, they found investors, recruited money, forged partnerships, and hired the right people for R&D, manufacturing and QA/QC. They enlisted medical doctors, biologists, bio-statisticians and many more specialists and invested days on end, writing the precise study protocol and the best CRF.

All that hard work was done for one reason: to find better ways for diagnosis, cure and, treatment for people like you and me.

But, if there is one thing that I keep seeing over and over again (and to be honest, it’s a little scary) is that great scientific innovators are not so great when it comes to building an EDC for their medical device clinical trial. They can be brilliant innovators of non-invasive monitoring technology and forget to collect the primary endpoint. They can design an innovative slow-release drug and design an EDC that misses contraindications and under/over-dosing.
Which would be a disaster.

Confusing IT ease-of-use with clinical trial success?
Google the search term “self-service EDC clinical trials” and you will get 113,000 hits on companies that supply DIY EDC solutions for medical device clinical trials. These companies will offer you a Web-based DIY package, ready-made templates and allow you fit them to your paper CRF. You write some general validation logic and basically you’re good to go on with your clinical trial. Sounds great, easy and cost effective but let me tell you a little secret, the EDC game is not rosy and cheerful.

Without a doubt, self-service cloud EDC offerings for medical device clinical trials are hot and seductive. In the famous last words of an esthetics device sponsor we know – “what’s the big deal? It’s just a few tables in a Business Objects database”. Last time, I checked, she was still stuck in that sentence…

Let’s consider some examples of clinical trial failure due to inappropriate design and edit check implementation.

Edit checks – a curse instead of a blessing?
Poorly-designed ECRFs and edit checks can drive your sites crazy during the trial. Edit checks can be as simple as validating an age range or as complex as correlating age, gender, informed consent, medical history and baseline screening.

Medical device trials can have anywhere from 200 to over 800 edit checks relating to dozens of ECRFS on a timeline of 25 patient visits and more.

These edit checks are the equivalent of the London underground or the New York City subway system. The objective of these complex, highly interrelated systems is to get you from point A to point B but if you make the wrong connections, you miss your train, your station and find yourself in a crummy neighborhood you never wanted to see.

Correct design and execution of edit checks will allow early detection of data quality issues and reduce the amount of time your study monitor spends on source data review.

And what about changes?
You may not have anticipated that the FDA wants you to collect additional data or change inclusion criteria in the middle of the study. After you write a letter to file, you will have to change the EDC – but if your self-service Web app doesn’t support versioning (and some don’t) you are stuck with a stack of old data over here and new data over there. Go figure out how to analyze here and there. You may have to trash your old data, create a new version of the EDC and re-enter all the data.

And what about the statistical analysis?
At the end of the study, you will work hard to clean the data, only to discover that your data model is intractable making it difficult to extract the data in a useful structure for statistical analysis and only then you discover that your results are statistically insignificant and that all that extremely hard work went down the drain.
A robust data model will allow you to perform the statistical analysis without expensive rework by the study statistician.

Show me the money!
For example, let’s take a multi-center medical device clinical trial that expected to last for 24 months with 500 subjects. Assume cost per subject of USD 10K for recruiting, lab tests, PI costs etc. and add another 20% for onsite monitoring and you get USD 6M for the study cost. A cloud EDC service will cost around USD 1000/month. The trade-off is DIY study build versus hiring a qualified professional to build the study. Internal costs of employees for DIY build and QA would be a minimum of 3 man-months, say USD 30K assuming an FTE costs USD10K/month. A professional study build might cost USD 60K; taking the DIY alternative saves you USD 30K on a study that costs USD 6M; or a savings of 0.5 percent at the risk of endangering the success of your entire trial.

The risk of failure is not always justified by cost savings.
DIY EDC is an IT productivity tool that may be a good fit for small investigator-initiated studies, but for multi-center trials, the risk is not justified by the cost savings.

So please tell me why risk the success of your trial and reputation for the pleasure of DIY?
DIY your vacation, but have a specialist design and implement your study data collection, validation and monitoring.

Jenya Konikov-Rozenman is a PhD candidate in biology and a project manager at FlaskData.io.   She can be reached at the FlaskData.io contact page.