how to establish and evaluate clinical prediction models – statswork

An Academic presentation by

Dr. Nancy Agnes, Head, Technical Operations, Statswork Group www.statswork.comEmail: info@statswork.com

HOW TO ESTABLISH AND EVALUATE CLINICAL PREDICTION MODELS

Outline

TODAY'S DISCUSSION

Introduction

Clinical Prediction Model

Steps to establishing a clinical prediction model

Clinical prediction models CODE

Future Scope:

The use of a parametric/semi-parametric/non-parametric mathematical model to estimate the probability that a subject currently has a certain condition or the possibility of a certain outcome in the future is referred to as a clinical predictive model.

Various regression analysis approaches are used to model clinical prediction models, and the statistical nature of regression analysis is to find "quantitative causality."

INTRODUCTION

To put it another way, regression analysis is a quantitative assessment of how much X impacts Y.

Multiple linear regression models, logistic regression models, and Coxregression models are all widely used approaches.

The secret to s tatistical analysis, data modelling, and project design is assessing and verifying prediction models' efficacy.

It is also the most difficult aspect of data analysis technology.

A clinical prediction model is a tool used in healthcare to measure estimates of the likelihood of the future course of a specific patient outcome using multiple clinical or non- clinical predictors.

A realistic checklist for developing a valid prediction model is presented in a clinical prediction model.

A clinical prediction model can be used in various clinical contexts, including screening for asymptomatic illness, forecasting future events such as disease, and assisting doctors in their decision-making and health education.

CLINICAL PREDICTION MODEL

Contd...

Despite the positive effects of c linical prediction models on practice, prediction modelling is a difficult process that necessitates meticulous statistical analysis and sound clinical judgments.

Contd...

STEPS TO ESTABLISHING A CLINICAL PREDICTION MODEL

There exist several types of research detailingthe methods to construct clinical prediction models.

However, there is no proper method to constructthe prediction model in medicine.

The construction and evaluation of prediction models are classified into five steps.

Contd...

STEP 1: GATHERING THE IDEATIONS AND QUESTIONS FOR ENHANCING THE MODEL.

It incorporates structuring the research questions, such as finding the target variable for predicting which age group of the targeted people you want to predict. etc.

For instance, gathering one patient details and use it as a trained data set to test the other data set of another patient’s details. [1].

Contd...

STEP 2: SELECTION OF DATA

Data collection is a vital part of statistical or clinical research.

Nevertheless, the perfect data and a perfect model can't exist. It would be nice to look for the most appropriate.

The primary dataset with the endpoint of the study and all key predictors may not be available at all the time.

Secondary or administrative data sources are mandatory.

Contd...

Based on the various data types of datasets, prediction models can be utilized.

[2] For instance, the epidemiology study is based on the Data Miningsystematic approach.

STEP 3: WAYS TO HANDLE VARIABLES

Most of the time, researchers may face challenging situations where the variables are highly correlated to each other, excluded in the study.

Variables don't show statistical significance or the petite effect size.

But it will contribute to the predictive model. Researchers will handle the missing data problems, categorical data, etc., before getting the interference.

Contd...

CLINICAL PREDICTION MODELS CODE

Contd...

The Bayesian network was implemented to manipulate the independent variables of some diseases in the crucial stage of treatment.

This model predicts and offers a way to handle the disease along with preventive measures [3].

Contd...

STEP 4: GENERATING MODEL

There are no proper rules to select a particular model for the statistical analysis.

There are some standard methods to build a model using L inearregression a nalysis, logistic regression analysis, and Cox models.

Sometimes the clinical data encounters over-fitting of the model and its results in as estimates.

This over-fitting issue can be detected using Akaike Information Criteria or Bayesian Information Criteria.

Contd...

The smaller AIC and BIC values result in a good fit for the model.

[4] Using Multivariate prediction models for analyzing the different characteristics of various patients.

Contd...

STEP 5: EVALUATION AND VALIDATION OF THE MODEL

After building the model, it is necessary to evaluate and validate thepredictive power of the model.

The key components that evaluate the model are calibration which plotsthe proportion, and discrimination classifies the events like success or

failure.

There are two types of data validation, namely internal and external validation of the model.

Internal validation evaluates the model within the data, whereas external validation can be done using the re-sampling technique, usually through bootstrapping.

Contd...

It means you are creating or generating new data sets with similar characteristics to the original data and validating the study's method through the newly created or bootstrapped data.

Further, there are several statistical measures to evaluate the model.

Some of them are ROC curve, AUC curve, sensitivity and specificity, likelihood ratio, R square value, calibration plot, c-index, Hosmer-Lemeshow test, AIC, BIC, etc.

Contd...

Figure 1: Slope of Calibration plot – Source: Stevens and Poppe (2020)Contd...

Besides, Stevens and Poppe (2020) suggested the Cox- calibration slope using al ogistic regression model instead of using the predictive model's calibration slope.

This suggestion has been made after the scrutiny of around 33 research articles and found that most of the validation is external validation and identified the validity using the calibration slope.

Contd...

Contd...

Figure 2: This flow diagram illustrates the progress through the various phases of the CARDAMON phase II clinical trial, including the impact of COVID‐19 on the 70 patients on maintenance K across the two treatment arms at the start of the lockdown period.

The 15 patients who stopped K maintenance joined the 170 patients who were already on long‐term follow‐up on 24 March 2020, bringing the number up to a total of 185. SCT, stem cell transplantation; K, carfilzomib; C, cyclophosphamide; d, dexamethasone [6].

Contd...

FUTURESCOPE Based on the patient details, we can predict the further

severe causation of disease in the future.

By gathering the data from a single patient may help to predict other similar patients for better treatment.

Big data support for manipulating vast amounts of clinicaltrials, without complexity simultaneously with high accuracy.

Contd...

TABLE 1 Concepts and Techniques of Clinical prediction models:

Contact Us UNITED KINGDOM

+44-1143520021

INDIA+91-4448137070

EMAILinfo@statswork.com