In this work we examined how different assumptions regarding soil carbon model calibration affect the resulting model performance. We found that how the litter inputs are set have a meaningful impact on the calibrated model parameters. Furthermore, two calibration methods produced parameter sets that differed meaningfully from each other but fit the validation dataset equally well. These results raise meaningful questions how we evaluate soil carbon model performance.