Internal Medicine: Open Access
Open Access

The significance of Prognostic modeling and its guidelines for formulating and validating prognostic rules

Johan Botha^{* *}Correspondence: Johan Botha, Department of Medicine, Yale University, New Haven, USA, Email:

Author info »

About the Study

Prognostic modelling is used to forecast outcomes in the absence of treatment, and predictive modelling is intended to anticipate treatment response, however the terms are frequently used interchangeably. The successful implementation of such models is crucial for identifying high-risk individuals, ordering necessary diagnostic tests, recognizing early-stage disease, and tailoring subsequent treatment regimens. Prognostic models are typically developed and implemented using standard multivariable regression models, more complex modern regression methods, and/or associated statistical tools, such as nomograms, and incorporate multiple factors, such as demographics, occupational or environmental exposures, genetics, and/or other biomarkers (although they may be based on a single measure, such as Prostate-Specific Antigen (PSA) for prostate cancer or CA-125 for ovarian cancer) into the model. Despite the tremendous clinical utility of prognostic models, major hurdles persist in their creation and validation, particularly for those that use moderate or high dimensional biomarker panels.

Erroneously optimistic outcomes are frequently the consequence of a variety of study design errors, such as prediction based on parameters that systematically differ by case status, which leads to considerable bias. In terms of progress towards validating and implementing potential prognostic rules, the vast majority of published studies on prediction modelling focus on developing or fitting the model, with many fewer studies assessing validation in a separate population. Almost no published studies assessed actual impact on physician behaviour or patient outcome.

Very few biomarkers have been found to have clinical relevance in terms of prognostic guidelines based on biomarkers. These severe limitations are most likely the result of both practical challenges associated with and/or failure to recognize the importance of specific stages of developing and validating biomarker panels or other prognostic rules, which leads to subsequent failure to advance promising prognostic rules through those stages of development and validation, as well as poor study design and reporting. Significant literature has been published criticizing even the most fundamental flaws of prognostic modelling, such as failing to justify the selection of predictor variables, failing to validate or even cross validate the model, and a lack of reproducibility in biomarker panel studies.

Conclusion

Many of the cited limitations and errors are entirely fixable (though many must be addressed prior to designing and conducting the study) by following existing guidelines for formulating and validating prognostic rules, conducting research in a reproducible manner, and implementing appropriate study designs for the given phase of research. Cross-sectional research, for example, cannot be used to validate and/or demonstrate clinical utility, as validation and illustration of true clinical utility can only come from a prospective study with randomization to use or non-use of the given model and assessment of effectiveness outcomes. Another readily solved but major issue is the proper usage of crucial words, such as the distinction between biomarkers as predictors and surrogate measures, the latter being a significantly more strict criterion. Lastly, to maximize the value of information obtained, more consistent incorporation of statistical knowledge unique to those modelling and related statistical difficulties is required. For example, despite the fact that it leads to extremely low statistical power, testing the differences between areas under the Receiver Operating Characteristic (ROC) curve remains a commonly used approach for measuring the improvement in classification accuracy between two nested models, and assessing the prognostic utility of new predictors is often better accomplished by more recently developed approaches, such as the net reclassification independence.