If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Inadvertent intraoperative hypothermia (core temperature <36°C) is a common surgical complication with several adverse events. Hypothermia prediction models can be a tool for providing the healthcare staff with information on the risk of inadvertent hypothermia. Our systematic review aimed to identify, demonstrate, and evaluate the available intraoperative hypothermia risk prediction models in surgical populations.
Design
This study is a systematic review of literature.
Methods
We systematically searched multiple databases (Ovid MEDLINE, Web of Science, Embase, and Cochrane Center Register of Controlled Trials). Two reviewers independently examined abstracts and the full text for eligibility. Data collection was guided by the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS checklist), and methodological quality and applicability were assessed by the Prediction model Risk Of Bias ASsessment Tool (PROBAST).
Findings
A total of 3,672 references were screened, of which eight articles were included in this study. All the models had a high risk of bias since most of them lacked model validation. Also, they failed to report the model performance and final model presentations, which restricted their clinical application.
Conclusions
The researchers should present models in a more standard way and improve the existing models to increase their predictive values for clinical application.
Inadvertent intraoperative hypothermia, defined as a core body temperature less than 36.0°C during a surgical procedure, is a common but preventable surgical complication.
Postoperative hypothermia and surgical site infection following peritoneal insufflation with warm, humidified carbon dioxide during laparoscopic colorectal surgery: A cohort study with cost-effectiveness analysis.
Also, hypothermia leads to more postoperative Intensive Care Unit (ICU) admissions, longer postanesthesia care unit (PACU) stays and prolonged postoperative hospital days.
Therefore, more importance should be attached to inadvertent hypothermia. In this era of personalized medicine, given that all patients do not have the same risk of hypothermia, the models developed to predict the risk of hypothermia are necessary.
Risk prediction models use predictors to evaluate the probability that a condition or disease will occur in the future.
One of the oldest and most famous prediction models in health care is the Apgar score, which provides a systematic and rapid assessment method to present newborns’ condition. Moreover, many multivariable prediction models have been widely developed to estimate individuals’ risk for cardiovascular disease events,
and so on. In the area of hypothermia, researchers identified the association between the occurrence of hypothermia and relevant variables such as age,
Prevalence and multivariable factors associated with inadvertent intraoperative hypothermia in video-assisted thoracoscopic surgery: A single-center retrospective study.
Although numerous studies examined the predictors and developed hypothermia prediction models, no studies have systematically reviewed the predictive performance and clinical applicability of these models.
Accordingly, the objective of this systematic review was to identify and evaluate current risk prediction models for intraoperative hypothermia in surgical populations. We attempt to present the characteristics of the models and appraise the predictive competency and applicability of these models to provide a reference for future research and clinical practice.
Method
Protocol and registration
We conducted this systematic review following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)
guidance (Supplemental 1) and present this review in accordance with the Cochrane methodology and CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS checklist).
This review protocol was also registered in the PROSPERO register of systematic reviews (NO. CRD42020206643).
Search strategy
Electronic databases (Ovid MEDLINE, Web of Science, Embase, and Cochrane Center Register of Controlled Trials) were searched from inception to August 2020 to identify original studies describing the construction and validation of hypothermia risk models. No language or date restrictions were applied, and articles were translated when necessary. The search strategy is shown in Supplemental 2. Briefly, the search included terms related to “hypothermia,” “intraoperative,” and “risk prediction.” Additional potential articles were located by manually searching the citations and reference lists of all included articles.
Eligibility criteria
Study inclusion criteria were as follows: (i) the model was developed or validated in surgical patients, (ii) the model involved at least two predictors, (iii) the outcome was inadvertent intraoperative hypothermia. Exclusion criteria were as follows: (i) nonhuman or animal studies, (ii) a cross-sectional study without any actual outcome analyses, (iii) the outcomes were perioperative hypothermia events (eg, the occurrence was not preoperative or postoperative).
Study selection
Two reviewers independently screened titles and abstracts to determine which articles could potentially be included. Results were pooled, and any discrepancies were resolved by consensus. The same procedure was applied to assessing the eligibility of the full-text articles. Finally, the citations and references of the included articles were examined to assure any additional eligible prediction models with the proposed characteristics were identified.
Data extraction
Two reviewers independently extracted information using standardized forms. The list of extracted items was based on the CHARMS checklist.
The information extracted from each study included country, study design (eg, retrospective, prospective cohort), surgery type (eg, abdominal surgery, thoracic surgery), final predictors, total/events (sample size/the number of positive events), temperature measuring site (eg, tympanic membrane, nasopharyngeal), model method (eg, logistic regression model, Cox proportional hazards model), model performance (eg, AUC: area under the receiver operating characteristic curve, H-L: Hosmer-Lemeshow test), model presentation (eg, equation, risk scale), and validation.
Risk of bias and applicability
The Prediction model Risk Of Bias ASsessment Tool (PROBAST) was used to estimate the risk of bias and applicability.
Risk of bias to occurs when shortcomings in study design, implementation, or analysis may lead to systematically distorted assessments of model values, which is compromised of four domains: participants, predictors, outcome, and analysis.
Applicability contains the first three areas. Risk of bias indicates whether systematic errors exist in the study, while applicability determines whether the model matches the research question of the systematic review.
A critical appraisal of the clinical applicability and risk of bias of the predictive models for mortality and recurrence in patients with oropharyngeal cancer: Systematic review.
Descriptive statistics of data were presented in tables to provide a synthesis of pivotal characteristics of the prediction models. Because of the heterogeneity among studies, we could not conduct a meta-analysis.
Results
Study selection
Through database searching and snowballing, a total of 3,672 articles were identified. After removing duplicates and excluding studies that did not meet criteria based on titles and abstracts, 35 reports remained for full-text evaluation. Eight articles were selected for a narrative synthesis after full-text assessment against the review criteria. The search process is shown in a PRISMA flow diagram (Figure 1).
Figure 1Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flowchart for studies included in the review. From: Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 6(7): e1000097. doi:10.1371/journal.pmed1000097. This image is available in color online at www.jopan.org.
Construcción y validación de un modelo predictivo de hipotermia intraoperatoria [Construction and validation of a model to predict intraoperative hypothermia].
Predictive factors of maternal hypothermia during Cesarean delivery: A prospective cohort study. Facteurs prédictifs d'hypothermie maternelle durant la Césarienne: une étude prospective de cohorte.
Construcción y validación de un modelo predictivo de hipotermia intraoperatoria [Construction and validation of a model to predict intraoperative hypothermia].
though surgeries with other types of anesthesia were also included, such as abdominal surgery, cesarean delivery, thoracic surgery, and open hysterectomy.
Predictive factors of maternal hypothermia during Cesarean delivery: A prospective cohort study. Facteurs prédictifs d'hypothermie maternelle durant la Césarienne: une étude prospective de cohorte.
The number of predictors per study ranged from 3 to 8, and 18 different predictors were identified in the prediction models. The most common predictors were age, baseline temperature, and weight. The number of participants per study varied between 104 and 1840, and the number of positive events per study varied between 51 and 472. Most of models measured tympanic membrane temperature (n = 6),
Construcción y validación de un modelo predictivo de hipotermia intraoperatoria [Construction and validation of a model to predict intraoperative hypothermia].
Predictive factors of maternal hypothermia during Cesarean delivery: A prospective cohort study. Facteurs prédictifs d'hypothermie maternelle durant la Césarienne: une étude prospective de cohorte.
Construcción y validación de un modelo predictivo de hipotermia intraoperatoria [Construction and validation of a model to predict intraoperative hypothermia].
Predictive factors of maternal hypothermia during Cesarean delivery: A prospective cohort study. Facteurs prédictifs d'hypothermie maternelle durant la Césarienne: une étude prospective de cohorte.
Predictive factors of maternal hypothermia during Cesarean delivery: A prospective cohort study. Facteurs prédictifs d'hypothermie maternelle durant la Césarienne: une étude prospective de cohorte.
developed three models, describing the incidence of intraoperative hypothermia in three different periods (1, 2, and 3 hours during surgery) and demonstrated goodness of fit with the coefficient of determination (R2) ranging from 0.631 to 0.665. The remaining studies did not report the model performance.
Construcción y validación de un modelo predictivo de hipotermia intraoperatoria [Construction and validation of a model to predict intraoperative hypothermia].
Predictive factors of maternal hypothermia during Cesarean delivery: A prospective cohort study. Facteurs prédictifs d'hypothermie maternelle durant la Césarienne: une étude prospective de cohorte.
is shown in Table 2. All studies had an overall high risk of bias (n = 8). In the domain of participants, most of studies had a low risk of bias (n = 6),
Construcción y validación de un modelo predictivo de hipotermia intraoperatoria [Construction and validation of a model to predict intraoperative hypothermia].
Predictive factors of maternal hypothermia during Cesarean delivery: A prospective cohort study. Facteurs prédictifs d'hypothermie maternelle durant la Césarienne: une étude prospective de cohorte.
The reason for poor performance was that some predictors (eg, duration of surgery, the volume of fluids) were not available at the moment of prediction. For outcome, the risk of bias was often low (n = 7). However, a high risk of bias was frequently scored in the analysis domain, mainly because these models lacked a validation process.
Table 2Risk of Bias Assessment Based on the PROBAST Tool
Risk of Bias
Applicability
Overall
Study
Participants
Predictors
Outcome
Analysis
Participants
Predictors
Outcome
Risk of Bias
Applicability
Kasai 2002
−
+
+
−
+
+
+
−
+
Rincon 2008
+
−
+
−
+
+
+
−
+
Kim 2014
+
+
+
−
+
+
−
−
−
Yang 2015
+
+
+
−
+
+
+
−
+
Desgranges 2017
+
−
+
−
+
+
+
−
+
Yi 2017
+
−
+
−
+
+
+
−
+
Emmert 2018
−
−
?
−
+
?
+
−
?
Chalari 2019
+
−
+
−
+
+
+
−
+
+ Indicates low risk of low concern regarding applicability, - indicates high risk of bias or high concern regarding applicability, and ? indicates unclear risk of bias or unclear concern regarding applicability.
This systematic review presented an overview of currently available hypothermia risk prediction models. We reported eight models specifically developed for common surgeries such as abdominal surgery, cesarean delivery, and thoracic surgery. The results showed that existing articles did not report model information in a complete and transparent way. Some studies failed to show pivotal modeling data, such as sample size considerations, the parameters of calibration and discrimination, and mathematical equations, which might lead to a high risk of bias.
The risk of bias assessment demonstrated that all studies, whether or not they conducted external validation, had a high risk of bias. The primary bias came from the domain of analysis due to inappropriate sample size and incomplete handling of missing data. This finding is in agreement with previous research.
From the perspective of methodology, although overall sample size matters, number of participants paired with the outcome (positive events) is even more significant. In prediction model studies, sample size calculations have usually been based on the number of Events Per Variable (EPV, positive events/candidate predictors).
The EPVs indicate not the number of predictors enrolled in the final model but rather the total number of predictors measured during any phase of the model implementation. For this reason, EPVs of at least 20 are less likely to have overfitting.
Overfitting can arise when the number of positive events is small compared with the number of final predictors, resulting in the incidence of an outcome event being underestimated in low-risk participants and overestimated in high-risk participants.
Issues related to missing data were also problematic. When a study does not mention missing data, patients with any missing information probably are omitted from analyses. Because statistical packages automatically eliminate samples with any missing data in the process of data analysis, this can undermine the validity of results by biasing estimation, lowing power, and ultimately reducing confidence in the models.
Overall, these shortcomings in the methodological quality increase the potential bias when evaluating the predictive performance of a model. Therefore, guidelines for the reporting of risk prediction research should be taken into account. The Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) Initiative is a checklist involving a minimum set of items that authors should report to inform the readers about how the study is designed and carried out.
This protocol can be a guidance to standardize the reporting way of prediction models. Moreover, researchers are encouraged to register their study on relevant websites, such as Clinical Trials.gov.
Our results showed that most studies did not include model validation, which might impact the comparison of these models’ predictive values. However, this finding is consistent with existing research.
Model validation is a crucial step to ensure that models perform stably in new participants, which can be divided into two categories (internal and external validation). Internal validation means that the same data are used for both development and validation, such as cross-validation and bootstrapping. External validation aims to evaluate existing models using data external to the construction sample (eg, from different populations).
Both of these methods can quantify the models’ bias. The model performance usually consists of discrimination (eg, the c-statistic) and calibration (eg, the H-L test). However, in our study, only three models used the AUC and/or H-L test for model performance to present the results of model validation. Meanwhile, only these three models showed the final models using an equation or a risk scale.
a lack of exact model presentations made external validation by other scholars difficult. Researchers may find it challenging to apply these models to clinical populations and cannot examine the models’ values. Therefore, to make the implementation of models within a healthcare setting more convenient, it is necessary to present the final models in a user friendly way. For example, a mathematical equation, a score chart, or a nomogram. Furthermore, these models undergoing the continued external data test also improve their clinical application.
Accordingly, the performance tests and presentations of models are required to report.
In our study, only three models performed well in the domain of predictors. The main reason was that some predictors were unavailable when the model would be intended to be applied, such as duration of surgery, the volume of fluids, or duration of anesthesia. Healthcare staff usually estimate the risk of intraoperative hypothermia before surgery. However, the above data can be obtained after the operation is over. In such situations, it is hard for medical staff to use these models for prediction, and the applicability of models would be limited.
For a prediction model to be applied to clinical settings, all enrolled predictors need to be obtained at the moment of prediction. Variables that cannot be obtained in time would inflate actual model values since such variables are measured closer in time to the outcome evaluation and are more likely to be strongly associated with the outcome.
Hence, this kind of flaw in the predictor domain needs to be mentioned and solved. Previous research suggested that only variables that were readily accessible to healthcare professionals can be considered in derivation, and variables that were challenging to determine objectively should be excluded.
However, this approach may undermine the predictive competency of models. For this shortcoming, we recommend consultation with clinical healthcare workers who have rich experience to identify relevant predictors, such as duration of surgery and the volume of fluids. Certainly, the accuracy of assessment depends on many factors, and the construction and validation of related assessment tools may be a reasonable starting point for further studies.
Recommendations
Given the high risk of bias surrounding the performance and methodology of published models, no models can be recommended for clinical practice at this time. However, the findings of this study can make valuable recommendations for the construction and improvement of further models. First, the sample size should be large enough to minimize overfitting. Thus, the EPV of at least 20 should be recommended and adopted. Second, samples with missing data should be handled appropriately instead of being directly excluded. Simply removing involved samples with incomplete information results in biased model performance since the analyzed individuals are a selective rather than a random sample of the original full sample. Hence, an appropriate method for handling missing data is recommended. According to Held et al.,
the multiple imputation with fixed or random intercepts for cohorts was an optimal method to impute the systematical missing data since it minimized biased results with correct P values and standard errors.
Third, future research should calibrate, extend, or update the existing models to strengthen their predictive performance rather than only constructing new models. And within this process, the additional evaluation value of certain predictors can be considered to enhance model performance. In addition, some new modeling methods can be explored, such as structural equation models and machine learning. Fourth, the EPVs should be clearly defined and accurately analyzed, and all final variables should be truly reported. Some assessment tools can be studied to assist researchers in predicting relevant variables. Finally, further prediction model research should be designed in a more user-friendly way, reported more clearly, and shown more completely. Widely recognized guidelines, including the TRIPOD Initiative
can be a standard model to offer guidance. Additionally, healthcare institutions can integrate the models into electronic medical records, which can be convenient for practitioners to apply models and further facilitate model validation.
Considering that no published hypothermia prediction model can be recommended for clinical use, future models are needed to determine the rigorous criteria of sample size to ensure the appropriate EPVs. Also, the complete model performance and presentation should be demonstrated following the standard modeling statements. Further studies are encouraged to validate and strengthen existing hypothermia risk prediction models rather than developing entirely new models.
Strengths and limitations
To our knowledge, this is the first systematic review that provides a comprehensive overview of hypothermia risk prediction models, evaluating their quality using the PROBAST tool. The results of this study can offer valuable insights for researchers to improve the models’ reporting quality. Nevertheless, some limitations of our review should be acknowledged and considered. First, we did not perform a meta-analysis due to the heterogeneity of the included research that met our study criteria, which limited the quantitative comparison of these models. Second, most research studies did not report model performance, which might impact these models’ predictive value assessment. These weakness are eagerly need improvement in further studies.
Conclusions
Our systematic review identified eight risk prediction models for inadvertent intraoperative hypothermia in surgical patients, and these models were reported following the guidance of the CHARMS and PROBAST statements. All the models had a high risk of bias, which undermines their prediction value in clinical practice. Accordingly, we encourage future studies to present models in a more standardized approach following relevant guidelines and include enough participants to validate models.
Postoperative hypothermia and surgical site infection following peritoneal insufflation with warm, humidified carbon dioxide during laparoscopic colorectal surgery: A cohort study with cost-effectiveness analysis.
Prevalence and multivariable factors associated with inadvertent intraoperative hypothermia in video-assisted thoracoscopic surgery: A single-center retrospective study.
A critical appraisal of the clinical applicability and risk of bias of the predictive models for mortality and recurrence in patients with oropharyngeal cancer: Systematic review.
Construcción y validación de un modelo predictivo de hipotermia intraoperatoria [Construction and validation of a model to predict intraoperative hypothermia].
Predictive factors of maternal hypothermia during Cesarean delivery: A prospective cohort study. Facteurs prédictifs d'hypothermie maternelle durant la Césarienne: une étude prospective de cohorte.
Funding: This research was supported by (i) Scientific Research and Innovation Project of postgraduates in Chongqing, China (no. CYS20196), (ii) National Key Clinical Specialist of the Ministry of Public Health, and (iii) the Medical Key Subjects in Chongqing province.
00122-2&id=gr1.jpg){kind=link}