MoDN-Flip: Optimizing Modular Clinical Decision Support Networks with Decision Flippability Score

BACKGROUND. Clinical Decision Support Systems (CDSS) are point-of-care questionnaires that guide consultations to evidence-based predictions with the aim of standardizing care and optimizing resource allocation. The data collected in these tools contain potentially useful insights to further train its predictive algorithms, however, the questionnaire structure biases data with systematic missingness, making it unusable in traditional models. Further, privacy and interoperability issues make it challenging to synergize with data collected in related CDSS tools. The Modular Decision Support Network (MoDN) was previously proposed by our group to address these issues. It can predict diagnoses at each step in a consultation using sequentially encoded questions. However, it lacks the ability to provide information on unencoded questions, such as predicting their importance, which could potentially optimize questionnaire structure. AIM. In this work, we aim to optimize MoDN by deriving a flippability score (FS) which computes the importance of a question in the form of how likely its answer would flip (change) the predicted diagnoses. METHODS & FINDINGS. The FS is computed with the help of MoDN’s feature decoders that predict unknown feature values in the data. The optimal questionnaire order is then derived by feeding MoDN the question with the highest FS. This approach—-MoDN-flip—- is validated on two independent real world CDSS-derived data sets of pediatric outpatients. We show that FS-optimized question order can achieve the baseline predictive performance 14.71% faster (or with 3.53 fewer questions) compared to MoDN with the original question order. FS-optimized questionnaires are also able to improve the cumulative F1 score by 5.24%. Furthermore, we test the utility of FS-optimization when porting MoDN to an imperfectly interoperable data set (i.e. with slightly different questionnaire structures and partially-overlapping feature sets). The results show that the score is valid in a new dataset and can provide better predictive performance at an earlier stages, albeit a slight cost to final performance. CONCLUSION. With the aim of providing data-driven questionnaire optimization for CDSS tools, we propose the flippability score, which indicates the importance of each potential question in a consultation. The experimental results show that FS can express predictive importance of questions and has the potential to significantly optimize questionnaire structure. It is now necessary to evaluate this ordering against human consultation logic.