Prediction Score Models

This article aims to relate USMLE Step 1 scores using performance data from UWorld and Free 117/120 assessments, focusing on two regression models: linear and quadratic regression. The data used in this analysis was originally sourced from a Reddit-based scientific research dataset, which is no longer available. However, users can download the dataset through the link provided on the page below. The analysis was conducted using SPSS 25, and the results from both regression models were analyzed to determine the most effective predictors for Step 1 scores. The findings from this page highlight the effectiveness of both models, but suggest that linear regression may be the more reliable choice when predicting USMLE Step 1 scores based on the performance data.

You can also use our USMLE Step 1 Score Calculator to predict and compare your potential Step 1 scores. This tool allows you to enter your percentage of correct answers from UWorld's First Pass and NBME exams to estimate the Step 1 score you might achieve. The linear regression model for UWorld First Pass is included, which helps you visualize the score you could potentially reach in the future based on your current performance in the first pass of UWorld. Additionally, the calculator incorporates a separate tool based on NBME Percent Scores derived from the Reddit data. This tool can help you calculate your potential USMLE Step 1 score based on your NBME performance. You can access both calculators at the following link: USMLE Step 1 Score Calculator. 

The analysis compares two regression models to predict USMLE Step 1 scores based on the performance in the first pass of UWorld. The first model is a linear regression, and the second one is a quadratic regression.

Linear Regression Model

In the linear regression model, the independent variable is the UWorld 1st Pass Percent Correct score. The model shows a high R² value of 0.567, meaning that approximately 56.7% of the variation in the USMLE Step 1 scores can be explained by the UWorld 1st Pass Percent Correct score. 

The ANOVA results show a very high F-value of 764.704 with a p-value of 0.000, indicating that the model is statistically significant and that the UWorld 1st Pass Percent Correct is a good predictor of the USMLE Step 1 score.

The coefficients for the model indicate that for each 1% increase in the UWorld 1st Pass Percent Correct score, the USMLE Step 1 score increases by 1.127 points. The constant term is 164.516, suggesting that even with a 0% UWorld score, the predicted Step 1 score would be 164.5 points, though this does not represent a real-world scenario.

Quadratic Regression Model

In the quadratic regression model, we also include the square of the UWorld 1st Pass Percent Correct score. The model again shows an R² value of 0.567, similar to the linear model. However, despite the inclusion of the squared term, it does not provide a better fit than the linear model.

The ANOVA results for the quadratic model are significant, with an F-value of 382.611 and a p-value of 0.000, indicating that the model is statistically significant. However, the coefficient for the squared term is -0.003, and its p-value is 0.374, meaning that it does not significantly contribute to the model. This suggests that adding the quadratic term does not improve the prediction of Step 1 scores.

Conclusion

Both models provide a similar R² value. The linear regression model is simpler and more straightforward, with a clear and significant relationship between UWorld performance and Step 1 scores. The quadratic model, despite including a squared term, does not offer any additional predictive value and is less effective than the linear model in this case. Therefore, the linear model is the more appropriate choice for predicting Step 1 scores based on UWorld performance.

The analysis explores two regression models to predict USMLE Step 1 scores based on the UWorld 2nd Pass Percent Correct score, but with an important consideration of missing data. The first model is a linear regression, and the second one is a quadratic regression.

Linear Regression Model

The linear regression model uses UWorld 2nd Pass Percent Correct (if above 50% completed) as the independent variable. The R² value is 0.484, indicating that approximately 48.4% of the variability in the USMLE Step 1 scores can be explained by the UWorld 2nd Pass Percent Correct score. It is important to note that there is still a significant amount of unexplained variability, especially due to missing data.

The ANOVA results for this model show an F-value of 197.840 with a p-value of 0.000, which means the model is statistically significant. This suggests that the UWorld 2nd Pass Percent Correct score is a good predictor of Step 1 performance, even though the model could be improved by addressing the missing values.

The coefficients show that for every 1% increase in the UWorld 2nd Pass Percent Correct score, the USMLE Step 1 score increases by 1.222 points. The constant value is 140.359, meaning that if the UWorld score were 0%, the predicted Step 1 score would be about 140.4 points, although this is a hypothetical value since a 0% score does not represent a real-world situation.

Quadratic Regression Model

The quadratic regression model includes the square of the UWorld 2nd Pass Percent Correct score. The R² value for this model is slightly higher at 0.488, meaning that 48.8% of the variation in Step 1 scores can be explained by the model. However, this increase is minimal and does not significantly improve the model compared to the linear regression.

The ANOVA results show an F-value of 100.025 with a p-value of 0.000, indicating that the quadratic model is statistically significant. However, the coefficient for the squared term is very small (0.008) and not statistically significant (p = 0.204). This suggests that adding the squared term does not meaningfully improve the prediction of Step 1 scores.

Conclusion

Both the linear and quadratic regression models show significant relationships between the UWorld 2nd Pass Percent Correct score and USMLE Step 1 performance. However, the linear regression model is slightly more effective, with a more straightforward and significant relationship. The quadratic model, despite adding a squared term, does not offer a meaningful improvement in predictive power. Additionally, the presence of missing values affects the overall model performance, as shown by the slightly reduced model fit (compared to ideal scenarios), and addressing these missing data could further improve the results.

In conclusion, the linear regression model appears to be the most reliable for predicting USMLE Step 1 scores based on the UWorld 2nd Pass Percent Correct score, while the quadratic model does not add substantial predictive value.

This analysis compares two regression models to predict USMLE Step 1 scores using NBME Free 117/120 Percent Score as the independent variable. The models explored are linear regression and quadratic regression, with the consideration of a significant number of missing values.

Linear Regression Model

The linear regression model shows an R² value of 0.475, meaning that about 47.5% of the variation in the USMLE Step 1 scores can be explained by the NBME Free 117/120 Percent Score. This suggests a moderate relationship between the two variables, although much of the variability remains unexplained, likely due to the presence of missing data.

The ANOVA results for the linear model show an F-value of 365.042 and a p-value of 0.000, indicating that the model is statistically significant. This suggests that NBME performance is a reliable predictor of USMLE Step 1 scores.

The coefficients indicate that for each 1-point increase in the NBME Free 117/120 Percent Score, the USMLE Step 1 score increases by 1.448 points. The constant term is 124.451, meaning that if the NBME score were zero, the predicted Step 1 score would be about 124.5 points, which is a hypothetical value not typically observed in practice.

Quadratic Regression Model

The quadratic regression model includes both the linear term and the squared term of the NBME Free 117/120 Percent Score. The R² value for this model is slightly higher at 0.478, indicating that 47.8% of the variation in USMLE Step 1 scores can be explained by this model. Although this is a small increase over the linear model, it is not a significant improvement.

The ANOVA results for the quadratic model show an F-value of 183.815 and a p-value of 0.000, suggesting that the quadratic model is statistically significant. However, the coefficients reveal that neither the linear term (-0.364) nor the squared term (0.011) are significant predictors of the Step 1 scores (with p-values of 0.786 and 0.176, respectively). This suggests that the squared term does not add meaningful predictive power, and the quadratic model does not outperform the linear model.

Conclusion

Both models show that NBME Free 117/120 Percent Score is a significant predictor of USMLE Step 1 scores, but the linear regression model is slightly more effective, with a higher coefficient and a clear relationship between the variables. The quadratic model, while it includes an additional squared term, does not improve the prediction and does not provide significant additional information. Additionally, the presence of missing values is an important factor, as it reduces the amount of available data, which likely impacts the model fit.

In conclusion, the linear regression model is more reliable for predicting Step 1 scores based on the NBME Free 117/120 Percent Score, while the quadratic model does not contribute significantly to the prediction. Further analysis addressing the missing values could potentially improve model performance.