Problem
Missing data is a very common problem in health and social studies. Data can be missing because people don’t want to answer some questions, or forget to give some information, or drop out of studies completely. Missing data presents a risk that results of a study will be wrong (“biased”), unless the analysis approach is chosen carefully. One statistical approach that can correct this bias is called multiple imputation (MI). The problem is that MI can be challenging and complex to use in practice, and there is little guidance available.
Developing midoc
With Kate Tilling, Jon Heron, and Rosie Cornish (Medical Research Council Integrative Epidemiology Unit at the University of
Bristol), and James Carpenter (London School of Hygiene and Tropical Medicine), I developed software to bridge the gap between the theory and practice of MI. The software is called the “Multiple Imputation DOCtor”, or midoc (https://elliecurnow.github.io/midoc/).
Workshops
I used the Knowledge Mobilisation Catalyst Award to run three workshops this summer to demonstrate midoc. These involved groups working in clinical trials, as well as statisticians working in the NHS with large health registries, and at the Office for National Statistics. The workshops had two objectives. The first objective was that participants would understand how midoc could help them choose when and how to use MI. The second objective was for me to find out which features of midoc were useful, which needed improving, and whether any extra functions should be added.
Anyone who has ever used R software will know that perfunctory output and obscure error messages are the norm! One of the priorities when developing midoc was including clear interpretation of results as part of the output. I wanted to make midoc as useful and accessible as possible. So I was also keen to find out whether participants in the workshops found midoc user-friendly and with the right level of detail.
Feedback
Participants gave some really positive feedback on midoc. They said they liked walking through a real example and the structured guidance. One participant said: “Thank you for the demo. I found it very useful to understand more about Directed acyclic graphs (DAGs) and how to check the validity of imputation. I will consider more about missing data in my future studies!” However, it was sometimes surprising what participants found confusing. I have already updated some of midoc’s functionality as a result. Lack of time was often mentioned as a barrier to the in-depth approach suggested by midoc. It was also clear there was a wide range of priorities and levels of experience in missing data methods across the participants. I’m now working on ways to streamline and simplify the analysis process used in midoc. I’ve also reflected on how to incorporate time-saving tips in the workshop format.
Next steps
I’m now applying for further development funding for midoc. It’s been incredibly useful to have workshop feedback to incorporate into my funding applications. The workshops have also helped me develop future collaborations and identify suitable studies to apply midoc to. I plan to hold follow-up events with participants to showcase the improvements I’ve made to midoc as a result of their feedback.
Final reflection
Software tools are often developed alongside new statistical methods. However, obtaining user feedback on these tools is frequently over-looked. My Knowledge Mobilisation Catalyst Award has really helped me with this. As a result, I have gained valuable insight into the midoc user-experience. This will ultimately encourage wider take-up of midoc and ensure that as many users as possible are following best practice in missing data methods.
Project links:
Multiple Imputation DOCtor (midoc)
A Decision-Making System for Multiple Imputation • midoc
Elinor is a lecturer in Medical Statistics and a Senior Research Associate in Biostatistics / Epidemiology in Bristol Medical School