Warning: fopen(/home/virtual/enm-kes/journal/upload/ip_log/ip_log_2024-03.txt): failed to open stream: Permission denied in /home/virtual/lib/view_data.php on line 88 Warning: fwrite() expects parameter 1 to be resource, boolean given in /home/virtual/lib/view_data.php on line 89 Development and Validation of a Risk Scoring System Derived from Meta-Analyses of Papillary Thyroid Cancer

Development and Validation of a Risk Scoring System Derived from Meta-Analyses of Papillary Thyroid Cancer

Article information

Endocrinol Metab. 2020;35(2):435-442
Publication date (electronic) : 2020 June 30
doi : https://doi.org/10.3803/EnM.2020.35.2.435
1Department of Internal Medicine, Dong-A University College of Medicine, Busan, Korea
2Department of Orthopaedic Surgery and Biomedical Research Institute, Pusan National University Hospital, Busan, Korea
3Department of Anatomy, Pusan National University School of Medicine, Yangsan, Korea
4Department of Nuclear Medicine and Biomedical Research Institute, Pusan National University Hospital, Busan, Korea
5Department of Nuclear Medicine, Chung-Ang University College of Medicine, Seoul, Korea
Corresponding authors: Kyoungjune Pak. Department of Nuclear Medicine, Pusan National University Hospital, 179 Gudeok-ro, Seo-gu, Busan 49241, Korea, Tel: +82-51-240-7389, Fax: +82-51-240-7442, E-mail: ilikechopin@me.com.
Ju Won Seok. Department of Nuclear Medicine, Chung-Ang University College of Medicine, 102 Heukseok-ro, Dongjak-gu, Seoul 06973, Korea, Tel: +82-2-6299-2891, Fax: +82-2-6299-2891, E-mail: ethmoid@daum.net
*These authors contributed equally to this work.
Received 2020 March 27; Revised 2020 May 02; Accepted 2020 May 27.

Abstract

Background

The aim of this study was to develop a scoring system to stratify the risk of papillary thyroid cancer (PTC) and to select the proper management.

Methods

We performed a systematic search of MEDLINE and Embase. Data regarding patients’ prognoses were obtained from the included studies. Odds ratios (ORs) with statistical significance were extracted from the publications. To generate a risk scoring system (RSS), ORs were summed (RSS1), and summed after natural-logarithmic transformation (RSS2). RSS1 and RSS2 were compared to the eighth edition of the American Joint Committee on Cancer (AJCC) staging system and the 2015 American Thyroid Association (ATA) guidelines for thyroid nodules and differentiated thyroid carcinoma.

Results

Five meta-analyses were eligible for inclusion in the study. Eight variables (sex, tumour size, extrathyroidal extension, BRAF mutation, TERT mutation, histologic subtype, lymph node metastasis, and distant metastasis) were included. RSS1 was the best of the analysed models.

Conclusion

We developed and validated a new RSS derived from previous meta-analyses for patients with PTC. This RSS seems to be superior to previously published systems.

INTRODUCTION

Thyroid cancer is the most common type of malignant endocrine cancer, and its incidence is continuing to rise worldwide [1]. Papillary thyroid cancer (PTC) is the most frequent type of differentiated thyroid carcinoma (DTC), which accounts for at least 70% of all follicular-cell derived thyroid malignancies [2]. Although the prognosis of PTC is generally good, a minority of patients eventually die of the disease and an even greater proportion face morbidity due to recurrence [2]. Therefore, both optimization of long-term health outcomes and education of individuals with thyroid cancer about their potential prognosis are of critical importance.

Over the years, multiple staging systems have been developed to predict the risk of mortality in patients with DTC [3]. Several studies have demonstrated that previous staging systems consistently provide the highest proportion of variance explained when applied to a broad range of patient cohorts, and they have been validated in retrospective studies and prospectively in clinical practice. However, none of the staging systems has been shown to be clearly superior to the others [2,3]. This relative inability to accurately predict an individual patient’s risk of death from thyroid cancer may be related to the failure of current staging systems to adequately integrate the risk associated with other potentially important clinicopathologic features [2]. Moreover, none of these systems is based on evidence from a comprehensive meta-analysis of published studies. This underscores the need to develop an accurate and practical prognostic algorithm for predicting patients’ disease progression.

The aim of this study was to develop a scoring system that integrates clinical information from meta-analyses of published studies, thereby helping to stratify the risk of PTC and to select the proper management. Therefore, we conducted a systemic literature review, collected data from previous meta-analyses, developed a risk scoring system (RSS), and validated it with data from The Cancer Genome Atlas (TCGA).

METHODS

Data search and study selection

We performed a systematic search of MEDLINE (from inception to February 2017) and Embase (from inception to February 2017) for English-language publications using the keywords “thyroid cancer,” “prognosis,” and “meta-analysis.” All searches were limited to human studies. The inclusion criteria were meta-analyses that investigated the prognostic value of risk factors in PTC. Abstracts and editorial materials were excluded. Two authors performed the searches and screening independently, and discrepancies were resolved by consensus.

Data extraction and risk scoring system

Data regarding disease-free survival, recurrence-free survival, disease-free interval, and progression-free survival were obtained from the included studies, and were redefined as event-free survival (EFS). Odds ratios (ORs) with statistical significance were extracted from the publications, and the following information was recorded: OR, 95% confidence interval (CI), and histology. To generate the RSS, ORs were (1) summed (RSS1) and (2) summed after the natural logarithmic transformation (RSS2). For example, a 63-year-old man with a 2.8-cm classical PTC with extrathyroidal extension (ETE), lymph node (LN) metastasis, BRAF mutation, with no TERT mutation or distant metastasis would be rated as (1.53+2.69+2.83+3.34+1+1+3.24+1=16.63) in RSS1, and as (ln1.53+ln2.69+ln2.83+ln3.34+ln1+ln1+ln3.24+ln1=4.8366) in RSS2.

Validation set

The primary and processed data were downloaded from the Genomic Data Commons Data Portal (https://portal.gdc.cancer.gov/) on January 2017. All TCGA data were available without restrictions for publications or presentations according to TCGA publication guidelines. We downloaded the data on somatic mutations and clinical information, which had most recently been updated in May 2016. Patients were categorized according to the eighth edition of the American Joint Committee on Cancer (AJCC) staging system [4] and the 2015 American Thyroid Association (ATA) guidelines for thyroid nodules and DTC [2]. RSS1 and RSS2 were calculated for each patient and categorized into two groups according to their respective cutoff value.

Statistical analysis

EFS was analysed with the log-rank test, and Kaplan-Meier survival plots were generated to validate the scoring systems. To evaluate the performance of the eighth edition of the AJCC system, the 2015 ATA guideline, RSS1, and RSS2, receiver operating characteristic (ROC) curve analysis was performed using the TCGA data, and the cutoff values of RSS1 and RSS2 were determined. Area under the ROC curve (AUC) was measured. For comparisons among models, the concordance index (C-index), Akaike information criterion (AIC), Bayesian information criterion (BIC), and Brier score were applied to quantify the predictive ability of a survival model [5], to select the statistical model [6], and to measure the accuracy of probabilistic predictions [7]. A higher C-index and a lower AIC, BIC, and Brier score indicated a better model for predicting outcomes. Statistical analyses were performed using GraphPad Prism 7 for Mac OSX (GraphPad Software Inc., San Diego, CA, USA), and R statistical software 3.5.0 (The R Foundation for Statistical Computing, Vienna, Austria, 2016).

RESULTS

Risk scoring system

The electronic search identified 841 articles. Non-English-language articles (n=42), non-human studies (n=36), conference abstracts (n=166), and 546 studies that did not meet the inclusion criteria based on their title and abstract were excluded. After reviewing the full text of 51 articles, five meta-analyses were eligible for inclusion in the study. The process is shown in more detail in Fig. 1. Eight variables derived from the meta-analysis were included in the RSS: sex [8], tumour size (>2 cm) [8], ETE (microscopic) [8], BRAF mutation status [9], TERT mutation status [10], histologic subtype [11,12], LN metastasis [8], and distant metastasis (Table 1) [8]. The scores of each patient ranged from 7.52 to 27.59 for RSS1, and from −0.65 to 7.32 for RSS2.

Fig. 1

Flowchart of the meta-analysis selection process.

Risk Scoring System

ROC curve analysis

We confirmed that cutoff values of 13.93 for RSS1 and 2.03 for RSS2 were useful for predicting the prognosis of patients with PTC. The composite score for sensitivity and specificity (AUC) determined by ROC analysis was measured for each model (RSS1, RSS2, eighth edition of the AJCC, and the 2015 ATA guideline). The AUC of RSS1 was not significantly different from that of RSS2 (0.751 vs. 0.745, P=0.5513). However, the AUC of RSS1 was higher than those of both the eighth edition of the AJCC (0.751 vs. 0.659, P=0.0349) and the 2015 ATA guideline (0.751 vs. 0.666, P=0.0171). The AUC of RSS2 was not significantly different from that of the eighth edition of the AJCC (0.745 vs. 0.659, P=0.0506), but was significantly higher than that of the 2015 ATA guideline (0.745 vs. 0.666, P=0.0292) (Fig. 2).

Fig. 2

Receiver operating characteristic curve analysis of RSS1 (area under the curve, 0.751), RSS2 (0.745), the eighth edition of the American Joint Committee on Cancer (AJCC, 0.659), and the 2015 American Thyroid Association (ATA) guideline (0.666). RSS, risk scoring system.

Validation with TCGA database

In total, 364 patients with PTC were included in this study (93 male, 271 female). Their mean age was 45.7 years. Of the 364 patients with PTC, 32 (8.8%) experienced recurrence/progression during the follow-up period (38.3±32.2 months). Patients’ characteristics are summarised in Table 2. Patients were dichotomised according to the RSS1 and RSS2 cut-offs. A survival analysis was conducted with the log-rank test for RSS1 (hazard ratio [HR], 4.241; 95% CI, 1.9541 to 9.6434; P<0.0001), RSS2 (HR, 6.9736; 95% CI, 3.4553 to 14.0743; P=0.0002), eighth edition of the AJCC (HR, 7.4592; 95% CI, 0.7834 to 71.0250; P<0.0001), and the 2015 ATA guideline (intermediate risk: HR, 1.9005; 95% CI, 0.9115 to 3.9628) (high risk: HR, 9.2530; 95% CI, 1.5019 to 57.0051; P<0.0001) (Fig. 3).

Patients’ Characteristics According to RSS1 and RSS2

Fig. 3

Kaplan-Meier survival plots of (A) RSS1 (hazard ratio [HR], 4.241; 95% confidence interval [CI], 1.9541 to 9.6434; P<0.0001), (B) RSS2 (HR, 6.9736; 95% CI, 3.4553 to 14.0743; P=0.0002), (C) eighth edition of the American Joint Committee on Cancer (AJCC) (HR, 7.4592; 95% CI, 0.7834 to 71.0250; P<0.0001), and (D) the 2015 American Thyroid Association (ATA) guideline (intermediate risk: HR, 1.9005; 95% CI, 0.9115 to 3.9628) (high risk: HR, 9.2530; 95% CI, 1.5019 to 57.0051; P<0.0001). The cutoff values were 13.93 for RSS1 and 2.03 for RSS2. RSS, risk scoring system.

Comparison of RSS1, RSS2, the eighth edition of the AJCC, and the 2015 ATA guideline

Among the models used, RSS1 showed the highest C-index and the second lowest AIC, BIC, and Brier score, making it the best model. The eighth edition of the AJCC had the second highest C-index, but the lowest AIC, and BIC. The 2015 ATA guideline showed the third highest C-index, the third lowest AIC and BIC, and the lowest Brier score (Table 3).

Comparison of RSS1, RSS2, the Eighth Edition of the AJCC, and the 2015 ATA Guideline

DISCUSSION

We developed and validated an RSS derived from previous meta-analyses to predict the prognosis of patients with PTC. Among the models used, RSS1 was shown to be better than the previously published systems.

At present, total thyroidectomy followed by radioactive iodine treatment together with life-long administration of thyroid hormone is the treatment strategy for most patients [2]. The biological behaviour of DTC is indolent and long-term survival is common, although the rate of recurrence is considerably higher [2]. Moreover, the overall risk of initial metastatic disease is 1% to 2% for PTC [13]. As a result, a number of studies have identified various clinicopathologic predictors for PTC and devised risk-group stratification or staging systems to select patients at high risk of cancer death for more aggressive surgical and adjuvant treatment, while those at low risk would be spared from the burden of aggressive treatment [2]. Individual therapy of a patient could be planned according to his or her risk group, and the survival of the patient would be estimated based on the appropriate therapy. This approach is known as stage-specific treatment.

Postoperative staging for thyroid cancer, as for other cancer types, is used (1) to provide prognostic information, which is of value when considering disease surveillance and therapeutic strategies, and (2) to enable a risk-stratified description of patients to facilitate communication among health care professionals, tracking by cancer registries, and research [2,3]. Several prognostic factors for DTC have been studied intensively since the 1980s and numerous staging classification systems have been proposed to predict outcomes for patients and to select individualized therapy, but there are some limitations of these systems [14]. Moreover, none of the existing systems is adequate for predicting outcomes in every population [14]. Undoubtedly, there is room for improvement, as none of the examined anatomic staging systems are able to account for the small proportion of cancer-related deaths in the so-called low-risk group. It is possible that more powerful prognostic biological factors and molecular markers could be added to existing staging systems in the future in order to improve survival prediction [3]. In this context, both the AJCC/Union for International Cancer Control staging system [4] and the ATA DTC management guidelines and RSS [2] were recently revised to improve their predictive power. Recently, molecular markers and clinical risk assessments with respect to DTC have been rigorously investigated, showing considerable evidence on prognostic significance [15]. As a result, the 2015 ATA guideline conveys a positive message, as it shows that mutational analysis of thyroid cancer has the potential to refine risk estimates [16]. This newly proposed system was shown to provide a more realistic estimate of prognosis for patients with PTC [17,18]. The newly developed scoring system in this study was shown to be more accurate in predicting patient outcomes than the eighth edition of the AJCC [4] and the 2015 ATA guideline [2]. Moreover, the predictors used in this model are readily available in routine clinical practice and easy to use.

A new staging classification should be developed in two steps: first, it should be generated and then it should be validated [19]. In the current study, a validated prognostic system following these two steps was presented. Our RSS was developed and then validated with 364 patients with full data on BRAF mutation status histology. Although there have been numerous attempts to compare the predictive power of staging systems for DTC, the majority of those studies either compared a limited number of systems or failed to make use of an objective comparative measurement to assess predictive power [3]. We used four different types of statistical analyses to compare the predictive accuracy of each staging system. All prognostic systems, including the present model, were valuable for identifying EFS in patients with PTC. Among the four methods, our meta-analysis-derived proposal was superior for determining the outcomes of patients with PTC. We think that the use of this scoring system will reduce overtreatment and its complications/sequelae, while providing relatively conservative cancer treatment with equally good outcomes.

This study has some limitations. All data were retrospectively collected, which limits the conclusions that can be drawn from the study. New, larger studies should be carried out to clarify the role of this tool. In addition, we could not analyse overall survival data, because only two patients died from PTC in the database from TCGA, reflecting the favourable prognosis of this condition. In addition, the eighth edition of AJCC was generated to predict the risk of survival (death), not the risk of recurrence; however, since the eighth edition of the AJCC and the 2015 ATA guideline are the most commonly used prognostic systems in clinical settings, we adopted both of them in this study. Furthermore, since we developed our RSS based on data from previous meta-analyses, we could not include risk factors that were not reported in the meta-analyses, such as mRNA [20] or miRNA markers [21].

In conclusion, we developed and validated a new RSS derived from previous meta-analyses for patients with PTC that incorporates information on sex, tumour size, ETE, BRAF mutation status, TERT mutation status, histologic subtype, LN metastasis, and distant metastasis. Based on a comparison with the eighth edition of the AJCC and the 2015 ATA guideline, this RSS appears to be superior to previously published systems.

Notes

CONFLICTS OF INTEREST

No potential conflict of interest relevant to this article was reported.

AUTHOR CONTRIBUTIONS

Conception or design: S.S., T.S.G., K.P., J.W.S., I.J.K. Acquisition, analysis, or interpretation of data: S.S., T.S.G., Y.H.K., S.O.O., K.P., J.W.S. Drafting the work or revising: S.S., K.P. Final approval of the manuscript: Y.H.K., K.P.

References

1. Cabanillas ME, McFadden DG, Durante C. Thyroid cancer. Lancet 2016;Dec. 3. 388(10061):2783–95.
2. Haugen BR, Alexander EK, Bible KC, Doherty GM, Mandel SJ, Nikiforov YE, et al. 2015 American Thyroid Association management guidelines for adult patients with thyroid nodules and differentiated thyroid cancer: the American Thyroid Association Guidelines Task Force on thyroid nodules and differentiated thyroid cancer. Thyroid 2016;26:1–133.
3. Lang BH, Lo CY, Chan WF, Lam KY, Wan KY. Staging systems for papillary thyroid carcinoma: a review and comparison. Ann Surg 2007;245:366–78.
4. Edge SB. American Joint Committee on Cancer. AJCC cancer staging manual 8th edth ed. New York: Springer; 2016.
5. Pitoia F, Jerkovich F, Urciuoli C, Schmidt A, Abelleira E, Bueno F, et al. Implementing the modified 2009 American Thyroid Association risk stratification system in thyroid cancer patients with low and intermediate risk of recurrence. Thyroid 2015;25:1235–42.
6. Palmer CR. Encyclopedia of biostatistics. BMJ 1999;318:542.
7. Ikeda M, Ishigaki T, Yamauchi K. Relationship between Brier score and area under the binormal ROC curve. Comput Methods Programs Biomed 2002;67:187–94.
8. Guo K, Wang Z. Risk factors influencing the recurrence of papillary thyroid carcinoma: a systematic review and meta-analysis. Int J Clin Exp Pathol 2014;7:5393–403.
9. Chen Y, Li Y, Zhou X. BRAF mutation in papillary thyroid cancer: a meta-analysis. Int J Clin Exp Med 2016;9:13259–67.
10. Yin DT, Yu K, Lu RQ, Li X, Xu J, Lei M, et al. Clinicopathological significance of TERT promoter mutation in papillary thyroid carcinomas: a systematic review and meta-analysis. Clin Endocrinol (Oxf) 2016;85:299–305.
11. Malandrino P, Russo M, Regalbuto C, Pellegriti G, Moleti M, Caff A, et al. Outcome of the diffuse sclerosing variant of papillary thyroid cancer: a meta-analysis. Thyroid 2016;26:1285–92.
12. Yang J, Gong Y, Yan S, Shi Q, Zhu J, Li Z, et al. Comparison of the clinicopathological behavior of the follicular variant of papillary thyroid carcinoma and classical papillary thyroid carcinoma: a systematic review and meta-analysis. Mol Clin Oncol 2015;3:753–64.
13. Mazzaferri EL, Jhiang SM. Long-term impact of initial surgical and medical therapy on papillary and follicular thyroid cancer. Am J Med 1994;97:418–28.
14. Yildirim E. A model for predicting outcomes in patients with differentiated thyroid cancer and model performance in comparison with other classification systems. J Am Coll Surg 2005;200:378–92.
15. Tavares C, Melo M, Cameselle-Teijeiro JM, Soares P, Sobrinho-Simoes M. Endocrine tumours: genetic predictors of thyroid cancer outcome. Eur J Endocrinol 2016;174:R117–26.
16. Pak K, Lee SH, Lee JG, Seok JW, Kim IJ. Comparison of visceral fat measures with cardiometabolic risk factors in healthy adults. PLoS One 2016;11:e0153031.
17. Lee SG, Lee WK, Lee HS, Moon J, Lee CR, Kang SW, et al. Practical performance of the 2015 American Thyroid Association guidelines for predicting tumor recurrence in patients with papillary thyroid cancer in South Korea. Thyroid 2017;27:174–81.
18. Nixon IJ, Wang LY, Migliacci JC, Eskander A, Campbell MJ, Aniss A, et al. An international multi-institutional validation of age 55 years as a cutoff for risk stratification in the AJCC/UICC staging system for well-differentiated thyroid cancer. Thyroid 2016;26:373–80.
19. Brierley JD, Panzarella T, Tsang RW, Gospodarowicz MK, O’Sullivan B. A comparison of different staging systems predictability of patient outcome. Thyroid carcinoma as an example. Cancer 1997;79:2414–23.
20. Pak K, Kim YH, Suh S, Goh TS, Jeong DC, Kim SJ, et al. Development of a risk scoring system for patients with papillary thyroid cancer. J Cell Mol Med 2019;23:3010–5.
21. Boos LA, Schmitt A, Moch H, Komminoth P, Simillion C, Marinoni I, et al. MiRNAs are involved in tall cell morphology in papillary thyroid carcinoma. Cancers (Basel) 2019;11:885.

Article information Continued

Fig. 1

Flowchart of the meta-analysis selection process.

Fig. 2

Receiver operating characteristic curve analysis of RSS1 (area under the curve, 0.751), RSS2 (0.745), the eighth edition of the American Joint Committee on Cancer (AJCC, 0.659), and the 2015 American Thyroid Association (ATA) guideline (0.666). RSS, risk scoring system.

Fig. 3

Kaplan-Meier survival plots of (A) RSS1 (hazard ratio [HR], 4.241; 95% confidence interval [CI], 1.9541 to 9.6434; P<0.0001), (B) RSS2 (HR, 6.9736; 95% CI, 3.4553 to 14.0743; P=0.0002), (C) eighth edition of the American Joint Committee on Cancer (AJCC) (HR, 7.4592; 95% CI, 0.7834 to 71.0250; P<0.0001), and (D) the 2015 American Thyroid Association (ATA) guideline (intermediate risk: HR, 1.9005; 95% CI, 0.9115 to 3.9628) (high risk: HR, 9.2530; 95% CI, 1.5019 to 57.0051; P<0.0001). The cutoff values were 13.93 for RSS1 and 2.03 for RSS2. RSS, risk scoring system.

Table 1

Risk Scoring System

Variable Odds ratio 95% CI Study No. of studies included in the meta-analysis
Sex Guo et al. 13
 Female 1 (2014) [8]
 Male 1.53 1.28–1.84

Tumour size, cm Guo et al. 6
 ≤2 1 (2014) [8]
 >2 2.69 2.06–3.50

Extrathyroidal extension (microscopic) Guo et al. (2014) [8] 12
 No 1
 Yes 2.83 2.32–3.44

BRAF mutation Chen et al. 8
 No 1 (2016) [9]
 Yes 3.34 2.36–4.73

TERT mutation Yin et al. 3
 No 1 (2016) [10]
 Yes 5.73 3.55–9.26

Histologic subtype
 Classical and other 1
 Follicular 0.52 0.34–0.80 Yang et al. (2015) [12] 8
 Diffuse sclerosing 3.19 1.86–5.49 Malandrino et al. (2016) [11] 7

LN metastasis
 No 1
 Yes 3.24 2.61–4.02 Guo et al. (2014) [8] 8

Distant metastasis
 No 1
 Yes 11.96 8.43–16.97 Guo et al. (2014) [8] 4

Table 2

Patients’ Characteristics According to RSS1 and RSS2

Variable All RSS1 RSS2


Low score (<13.93) High score (≥13.93) P value Low score (<2.03) High score (≥2.03) P value
Age, yr 45.8±15.0 45.8±14.6 45.7±16.1 0.79 46.6±14.5 45.1±15.4 0.27
Sex <0.01 0.04
 Female 271 216 (78.8) 55 (61.1) 130 (79.8) 141 (70.1))
 Male 93 58 (21.2) 35 (38.9) 33 (20.2) 60 (29.9)

Tumour size, cm <0.01 <0.01
 ≤2 145 130 (47.4) 15 (16.7) 90 (55.2) 55 (27.4)
 >2 219 144 (52.6) 75 (83.3) 73 (44.8) 146 (72.6)

Extrathyroidal extension <0.01 <0.01
 No 268 229 (83.6) 39 (43.3) 152 (93.3) 116 (57.7)
 Yes 96 45 (16.4) 51 (56.7) 11 (6.7) 85 (42.3)

BRAF mutation <0.01 <0.01
 No 148 141 (51.5) 7 (7.8) 117 (71.8) 31 (15.4)
 Yes 216 133 (48.5) 83 (92.2) 46 (28.2) 170 (84.6)

TERT mutation 0.25 1
 No 363 274 (100) 89 (98.9) 163 (100) 200 (99.5)
 Yes 1 0 1 (1.1) 0 1 (0.5)

Histologic subtype <0.01 <0.01
 Classical and other 282 196 (71.5) 86 (95.6) 88 (54.0) 194 (96.5)
 Follicular 81 78 (28.5) 3 (3.3) 75 (46.0) 6 (3.0)
 Diffuse sclerosing 1 0 1 (1.1) 0 1 (0.5)

LN metastasis <0.01 <0.01
 No 215 210 (76.6) 5 (5.6) 146 (89.6) 69 (34.3)
 Yes 149 64 (23.4) 85 (94.4) 17 (10.4) 132 (65.7)

Distant metastasis <0.01 0.03
 No 358 274 (100) 84 (93.3) 163 (100) 195 (97.0)
 Yes 6 0 6 (6.7) 0 6 (3.0)

Eighth edition of the AJCC <0.01 <0.01
 I/II 354 273 (99.6) 81 (90.0) 163 (100) 191 (95.0)
 III/IV 10 1 (0.4) 9 (10.0) 0 10 (5.0)

2015 ATA guideline <0.01 <0.01
 Low 226 203 (74.1) 23 (25.6) 144 (88.3) 82 (40.8)
 Intermediate 123 68 (24.8) 55 (61.1) 19 (11.7) 104 (51.7)
 High 15 3 (1.1) 12 (13.3) 0 15 (7.5)

Values are expressed as mean±standard deviation or number (%).

RSS, risk scoring system; LN, lymph node; AJCC, American Joint Committee on Cancer; ATA, American Thyroid Association.

Table 3

Comparison of RSS1, RSS2, the Eighth Edition of the AJCC, and the 2015 ATA Guideline

Variable C-index AIC BIC Brier score
RSS1 0.7413 335.0066 336.4723 0.1265
RSS2 0.5605 334.8337 336.2995 0.1490
Eighth edition of the AJCC 0.6454 340.9069 342.3727 0.1152
2015 ATA guideline 0.5899 337.1516 340.0831 0.1503

RSS, risk scoring system; AJCC, American Joint Committee on Cancer; ATA, American Thyroid Association; AIC, Akaike information criterion; BIC, Bayesian information criterion.