Accurate shortwave radiance computations are becoming increasingly important for some applications in atmospheric composition. In this work we propose a benchmark protocol and dataset to asses the accuracy and computing runtime of radiance calculations of radiative transfer models. It is applied to four models, showing the potential of this benchmark to evaluate the model performance under a variety of atmospheric conditions, viewing geometries, aerosol loading, and optical properties.