<p>Physical and heat limits of the semiconductor technology require the adaptation of heterogeneous architectures in supercomputers, such as graphics processing units (GPUs) with many-core accelerators and many-core processors with management and computing cores, to maintain a continuous increase of computing performance. The transition from homogeneous multi-core architectures to heterogeneous many-core architectures can produce “potential differences” that lead to numerical perturbations and uncertainties in simulation results, which could blend with errors due to coding bugs. The development of a methodology to identify the computational perturbations and secure the model correctness is a critically important step in model development on the computer system with new architectures. We have developed a methodology to characterize the uncertainties in the heterogeneous many-core computing environment, which contains a simple multiple-column atmospheric model consisting of typical discontinuous physical parameterizations defined by on-off switches, an efficient ensemble-based test approach, and a software tool applied to the GPU-based high-performance computing (HPC) and Sunway systems. Statistical distributions from ensembles of the heterogeneous systems show quantitative analyses of computational perturbations and acceptable error tolerances. The methodology explores fully understanding to distinguish between perturbations caused by platforms and discrepancies caused by software bugs, and provides encouraging references for verifying the reliability of supercomputing platforms and discussing the sensibility of Earth system modeling to the adaptation of new heterogeneous many-core architectures.</p>