<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Publishing DTD v3.0 20080202//EN" "https://jats.nlm.nih.gov/nlm-dtd/publishing/3.0/journalpublishing3.dtd">
<article xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="research-article" dtd-version="3.0" xml:lang="en">
<front>
<journal-meta>
<journal-id journal-id-type="publisher">GMD</journal-id>
<journal-title-group>
<journal-title>Geoscientific Model Development</journal-title>
<abbrev-journal-title abbrev-type="publisher">GMD</abbrev-journal-title>
<abbrev-journal-title abbrev-type="nlm-ta">Geosci. Model Dev.</abbrev-journal-title>
</journal-title-group>
<issn pub-type="epub">1991-9603</issn>
<publisher><publisher-name>Copernicus Publications</publisher-name>
<publisher-loc>Göttingen, Germany</publisher-loc>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="doi">10.5194/gmd-7-1247-2014</article-id>
<title-group>
<article-title>Root mean square error (RMSE) or mean absolute error (MAE)? – Arguments against avoiding RMSE in the literature</article-title>
</title-group>
<contrib-group><contrib contrib-type="author" xlink:type="simple"><name name-style="western"><surname>Chai</surname>
<given-names>T.</given-names>
</name>
<xref ref-type="aff" rid="aff1">
<sup>1</sup>
</xref>
<xref ref-type="aff" rid="aff2">
<sup>2</sup>
</xref>
</contrib>
<contrib contrib-type="author" xlink:type="simple"><name name-style="western"><surname>Draxler</surname>
<given-names>R. R.</given-names>
<ext-link>https://orcid.org/0000-0001-7081-9992</ext-link>
</name>
<xref ref-type="aff" rid="aff1">
<sup>1</sup>
</xref>
</contrib>
</contrib-group><aff id="aff1">
<label>1</label>
<addr-line>NOAA Air Resources Laboratory (ARL),  NOAA Center for Weather and Climate Prediction,  5830 University Research Court,  College Park, MD 20740, USA</addr-line>
</aff>
<aff id="aff2">
<label>2</label>
<addr-line>Cooperative Institute for Climate and Satellites, University of Maryland, College Park, MD 20740, USA</addr-line>
</aff>
<pub-date pub-type="epub">
<day>30</day>
<month>06</month>
<year>2014</year>
</pub-date>
<volume>7</volume>
<issue>3</issue>
<fpage>1247</fpage>
<lpage>1250</lpage>
<permissions>
<copyright-statement>Copyright: &#x000a9; 2014 T. Chai</copyright-statement>
<copyright-year>2014</copyright-year>
<license license-type="open-access">
<license-p>This work is licensed under the Creative Commons Attribution 3.0 Unported License. To view a copy of this licence, visit <ext-link ext-link-type="uri"  xlink:href="https://creativecommons.org/licenses/by/3.0/">https://creativecommons.org/licenses/by/3.0/</ext-link></license-p>
</license>
</permissions>
<self-uri xlink:href="https://gmd.copernicus.org/articles/7/1247/2014/gmd-7-1247-2014.html">This article is available from https://gmd.copernicus.org/articles/7/1247/2014/gmd-7-1247-2014.html</self-uri>
<self-uri xlink:href="https://gmd.copernicus.org/articles/7/1247/2014/gmd-7-1247-2014.pdf">The full text article is available as a PDF file from https://gmd.copernicus.org/articles/7/1247/2014/gmd-7-1247-2014.pdf</self-uri>
<abstract>
<p>Both the root mean square error (RMSE) and the mean absolute error
  (MAE) are regularly employed in model evaluation studies.
  Willmott and Matsuura (2005) have suggested that the RMSE is not a good indicator
  of average model performance and might be a misleading indicator of
  average error, and thus the MAE would be a better metric for that
  purpose.
  While some concerns over using RMSE raised by Willmott and Matsuura (2005) and
Willmott et al. (2009)  are valid, the proposed avoidance of RMSE
  in favor of MAE is not the solution.  Citing the aforementioned papers,
  many researchers chose  MAE over RMSE to present their model evaluation
  statistics when presenting or adding the RMSE measures could be more beneficial.
  In this technical note, we demonstrate that the RMSE is not
  ambiguous in its meaning, contrary to what was claimed by
Willmott et al. (2009).  The RMSE is more appropriate to represent model
  performance than the MAE when the error distribution is expected to
  be Gaussian.  In addition, we show that the RMSE satisfies the
  triangle inequality requirement for a distance metric,  whereas
  Willmott et al. (2009) indicated that the sums-of-squares-based
  statistics do not satisfy this rule.   In the end, we discussed
  some circumstances where using the RMSE will be more beneficial.
  However, we do not contend that the RMSE is superior over the MAE.
  Instead, a combination of metrics, including but certainly not limited
  to RMSEs and MAEs, are often required to assess model performance.</p>
</abstract>
<counts><page-count count="4"/></counts>
</article-meta>
</front>
<body/>
<back>
<ref-list>
<title>References</title>
<ref id="ref1">
<label>1</label><mixed-citation publication-type="other" xlink:type="simple">Chai, T., Carmichael, G. R., Tang, Y., Sandu, A., Heckel, A., Richter, A., and Burrows, J. P.: Regional NO&lt;sub&gt;x&lt;/sub&gt; emission inversion through a four-dimensional variational approach using SCIAMACHY tropospheric  NO&lt;sub&gt;2&lt;/sub&gt; column observations, Atmos. Environ., 43, 5046–5055, 2009.</mixed-citation>
</ref>
<ref id="ref2">
<label>2</label><mixed-citation publication-type="other" xlink:type="simple">Chai, T., Kim, H.-C., Lee, P., Tong, D., Pan, L., Tang, Y., Huang, J., McQueen, J., Tsidulko, M., and Stajner, I.: Evaluation of the United States National Air Quality Forecast Capability experimental real-time predictions in 2010 using Air Quality System ozone and NO&lt;sub&gt;2&lt;/sub&gt; measurements, Geosci. Model Dev., 6, 1831–1850, &lt;a href=&quot;http://dx.doi.org/10.5194/gmd-6-1831-2013&quot;&gt;https://doi.org/10.5194/gmd-6-1831-2013&lt;/a&gt;, 2013.</mixed-citation>
</ref>
<ref id="ref3">
<label>3</label><mixed-citation publication-type="other" xlink:type="simple">Chatterjee, A., Engelen, R. J., Kawa, S. R., Sweeney, C., and Michalak, A. M.: Background error covariance estimation for atmospheric CO&lt;sub&gt;2&lt;/sub&gt; data assimilation, J. Geophys. Res., 118, 10140–10154, 2013.</mixed-citation>
</ref>
<ref id="ref4">
<label>4</label><mixed-citation publication-type="other" xlink:type="simple">Horn, R. A. and Johnson, C. R.: Matrix Analysis, Cambridge University Press, 1990.</mixed-citation>
</ref>
<ref id="ref5">
<label>5</label><mixed-citation publication-type="other" xlink:type="simple">Huber, P. and Ronchetti, E.: Robust statistics, Wiley New York, 2009.</mixed-citation>
</ref>
<ref id="ref6">
<label>6</label><mixed-citation publication-type="other" xlink:type="simple">Jerez, S., Pedro Montavez, J., Jimenez-Guerrero, P., Jose Gomez-Navarro, J., Lorente-Plazas, R., and Zorita, E.: A multi-physics ensemble of present-day climate regional simulations over the Iberian Peninsula, Clim. Dynam., 40, 3023–3046, 2013.</mixed-citation>
</ref>
<ref id="ref7">
<label>7</label><mixed-citation publication-type="other" xlink:type="simple">McKeen, S. A., Wilczak, J., Grell, G., Djalalova, I., Peckham, S., Hsie, E., Gong, W., Bouchet, V., Menard, S., Moffet, R., McHenry, J., McQueen, J., Tang, Y., Carmichael, G. R., Pagowski, M., Chan, A., Dye, T., Frost, G., Lee, P., and Mathur, R.: Assessment of an ensemble of seven real-time ozone forecasts over eastern North America during the summer of 2004, J. Geophys. Res., 110, D21307, &lt;a href=&quot;http://dx.doi.org/10.1029/2005JD005858&quot;&gt;https://doi.org/10.1029/2005JD005858&lt;/a&gt;, 2005.</mixed-citation>
</ref>
<ref id="ref8">
<label>8</label><mixed-citation publication-type="other" xlink:type="simple">Savage, N. H., Agnew, P., Davis, L. S., Ordóñez, C., Thorpe, R., Johnson, C. E., O&apos;Connor, F. M., and Dalvi, M.: Air quality modelling using the Met Office Unified Model (AQUM OS24-26): model description and initial evaluation, Geosci. Model Dev., 6, 353–372, &lt;a href=&quot;http://dx.doi.org/10.5194/gmd-6-353-2013&quot;&gt;https://doi.org/10.5194/gmd-6-353-2013&lt;/a&gt;, 2013.</mixed-citation>
</ref>
<ref id="ref9">
<label>9</label><mixed-citation publication-type="other" xlink:type="simple">Taylor, M. H., Losch, M., Wenzel, M., and Schroeter, J.: On the sensitivity of field reconstruction and prediction using empirical orthogonal functions derived from gappy data, J. Climate, 26, 9194–9205, 2013.</mixed-citation>
</ref>
<ref id="ref10">
<label>10</label><mixed-citation publication-type="other" xlink:type="simple">Tukey, J. W.: Exploratory Data Analysis, Addison-Wesley, 1977.</mixed-citation>
</ref>
<ref id="ref11">
<label>11</label><mixed-citation publication-type="other" xlink:type="simple">Willmott, C. and Matsuura, K.: Advantages of the Mean Absolute Error (MAE) over the Root Mean Square Error (RMSE) in assessing average model performance, Clim. Res., 30, 79–82, 2005.</mixed-citation>
</ref>
<ref id="ref12">
<label>12</label><mixed-citation publication-type="other" xlink:type="simple">Willmott, C. J., Matsuura, K., and Robeson, S. M.: Ambiguities inherent in sums-of-squares-based error statistics, Atmos. Environ., 43, 749–752, 2009.</mixed-citation>
</ref>
</ref-list>
</back>
</article>