the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
Visual analysis of model parameter sensitivities along warm conveyor belt trajectories using Met.3D (1.6.0-multivar0)
Christoph Neuhauser
Maicon Hieronymus
Michael Kern
Marc Rautenhaus
Annika Oertel
Rüdiger Westermann
Abstract. Numerical weather prediction models rely on parameterizations for subgrid-scale processes, e.g., for cloud microphysics, which are a well-known source of uncertainty in weather forecasts. Via algorithmic differentiation, which computes the sensitivities of prognostic variables to changes in model parameters, these uncertainties can be quantified. In this article, we present visual analytics solutions to analyze interactively the sensitivities of a selected prognostic variable to multiple model parameters along strongly ascending trajectories, so-called warm conveyor belt (WCB) trajectories. We propose a visual interface that enables to a) compare the values of multiple sensitivities at a single time step on multiple trajectories, b) assess the spatio-temporal relationships between sensitivities and the trajectories' shapes and locations, and c) find similarities in the temporal development of sensitivities along multiple trajectories. We demonstrate how our approach enables atmospheric scientists to interactively analyze the uncertainty in the microphysical parameterizations, and along the trajectories, with respect to the selected prognostic variable. We apply our approach to the analysis of WCB trajectories within the extratropical cyclone "Vladiana", which occurred between 22–25 September 2016 over the North Atlantic.
Christoph Neuhauser et al.
Status: open (until 23 Jun 2023)
-
RC1: 'Comment on gmd-2023-27', Anonymous Referee #1, 20 Apr 2023
reply
## general comments ##
This paper presents an interactive visualization system that allows for the sensitivity analysis of model parameters along warm conveyor belt trajectories. For a set of WCB trajectories, the impact of 40 model parameters on the impact of QR is assessed visually in multiple ways. The research questions that motivate the work are well phrased and are addressed later by the method as well as in the case study. The work is of high importance for gaining a deeper understanding of NWP models.
pro:
+ insightful application of visualization methods to a complex meteorological problem (model parameter sensitivity during WCB ascent)
+ well-discussed case study that reveals interesting observations
+ the construction of the visualization components is described well
+ useful comparative tools are provided (similarity measure by time warping, temporal alignment)con:
- some visualization choices require more justification or discussion (regarding perception)
- the trajectory input could be described more formally to avoid ambiguitiesI enjoyed reading the paper. It is well written, addresses a relevant problem for model analysis, and describes its visualization system well, making the work reproducible. In summary, I recommend to:
* consider switching from a saturation-encoding to polar area charts for the spheres or discuss perceptual limitations
* discuss options for achieving scalability in the trajectory plots regarding many model parameters
* consider adding the suggested references
* make exposition improvements (size of figures, color map legends, axis labels, unambigious math symbols)
## specific comments ### Pie charts
The sensitivity to certain variables was color-coded by saturation at a certain time by placing a spherical glyph with one segment per variable at the respective location in space.
- The glyph was called a "pie chart", which suggests that a quantitative attribute is mapped to the slice angles, which was not the case. I would suggest using a different name for the glyph, in order to avoid the usually negative associations related to the inaccurate angle perception of pie charts.
- The meaning of the slice hue was not clear from the beginning. Please add color bars as in Fig. 8 also to Figs. 1 and 2.
- Using saturation for quantitative data is not an ideal encoding, since psychophysical experiments showed that it ranks rather low on the effectiveness scale (see the book "Visualization Analysis and Design" by Tamara Munzner).
- Instead of encoding a quantitative attribute by the saturation of a fixed-size pie segment, I would suggest to consider using a polar area chart instead. The polar area chart likewise uses pie segments with fixed angle, but the radius of the segment is varied according to the quantitative attribute. Thereby, the quantitiative data is mapped to a positional encoding (i.e., the radius), which ranks higher on the effectiveness scale, and where differences between different radii can be perceived easier. Since small values lead to tiny segments, it would be imaginable to use a full glyph with maximal radius, as it is done now, which is depicted with the desaturated hue. And then, each pie segment could be filled from the center towards the outside by a fully saturated segment where the radius of the fully-saturated segment encodes the quantitative data. The saturation of a pixel could be calculated in the fragment shader based on the distance to the glyph center and the quantitative value that is to be encoded. If I'm not mistaken, it should be straightforward to implement this shading on the existing spherical glyphs. I would leave it to the authors to decide if they want to try this alternative encoding or whether they would like to discuss the perceptual limitations of their current saturation-based encoding.# Preattentive encoding on 3D trajectories
- The distortion of the oriented patterns on the sphere impedes the effectiveness (Fig 6c). This is perhaps not a crucial feature and could possibly be removed.
- Along the 3D trajectories, multiple quantitative attributes are color-mapped, each with its own single-hue color map. While the individual encodings are pre-attentive, the task of finding locations in the image, where both encodings have high values at the same time, or where one is high and the other is low, is not pre-attentive anymore. If the user wants to find places, where two variables have high values at the same time, then it would be better to highlight those areas on the trajectory with a separate pre-attentive encoding, for example by high-lighting such regions. This could be considered if this task should be further supported. Since the displayed set of trajectories is usually relatively small, it is not a crucial improvement.# Curve plots
The temporal alignment is a nice idea. I also liked the time warping to find similar sequences. (Not a question, just a comment.)
- Please add little labels for the x-axis to indicate that this axis encodes time.
- The curves were simplified using LTTB if there are more vertices than pixels. It was mentioned that this reduces the performance penalty of drawing too many points. I would be curious by how much the performance is improved by this optimization.
- Instead of mean (using a curve) and the stdev (using color-coding), one could display the full distribution of all trajectories in a trajectory density plot.
- It was mentioned that the scalability of the approach is limited when there are too many model parameters to display. It would be imaginable to achieve scalability by including table lenses. A table lense distinguishes between selected and non-selected model parameters. The non-selected model parameters would encode less information in a box with smaller height, while the selected ones encode more information in a box with larger height. Implementing this is not a necessity, but it could be mentioned that standard approaches exist to achieve scalability of the presented method.# 3D View
- In Figure 1, I find it difficult to estimate the altitude of the surface precipitation. From the text, I would think that it should be on the ground, but since the drop shadows are not cast onto the surface precipitation it appears to float above the ground. Perhaps the drop shadows could also be cast onto the color-coding on the surface.
- Is the vertical scaling of the clouds and the WCB trajectories equal? The clouds seem rather flat compared to the high vertical ascent of the WCBs.# Meaning of derivative activations
- When we see on the trajectory that at a certain location the sensitivity to a certain variable is high, does this immediately mean anything, or could it be that this particular variable is actually not present? For example, for a variable to actually exhibit a change, many other conditions might have to be right. Couldn't it be that the change that would have a strong influence on QR would actually never occur in a certain weather situation? Or are certain changes on one variable easier to occur than certain changes on another variable? If this is the case, is there some room for an extension of the diagnostic tool in the future?# Literature on trajectory data visualization
The most relevant literature is already mentioned. The field of trajectory visualization is quite large, so there could always be additional references added. Here are two directions that could be interesting, a survey and a multi-parameter analysis method along trajectories.
- survey on movement data visualization: https://doi.org/10.3390/ijgi8020063
- time activity curves: https://doi.org/10.1109/VISUAL.2019.8933578 and https://doi.org/10.1111/cgf.14093# Formal description
- The input to the method could be described a bit more formally: Given is a set of trajectories X={x_1(t),..., x_n(t)} with attributes a_i^j(t), etc. This could make the dimensionality of inputs and outputs a bit clearer, emphasizing that this is about the analysis of multi-variate data along trajectories.
- The symbol "n" is ambigiously used, namely for the number of bands and for the normal (also in the appendix). I suggest adding a vector (\vec) over the normal and the tangent.
- Also consider using a single terminology, i.e., decide on "bands" or "sectors", instead of mixing the terms.
- In appendix B, I would suggest to call the "up" vector "\vec k" or something else with only one letter.# Case study
- It was said that "location and ascent rate" of WCB trajectories were used as distinction criteria for the clustering. Have those two scalars been added in a weighted combination of some sort? If so, what were the weights?
- The bending of lines in Figure 11 looks in some cases a bit odd. If there is a problem with this encoding along lines with high curvature, it might be worth mentioning this as limitation.
- Several figures could be made a bit larger (from Fig 10 onwards).
- In Figure 12, I find it difficult to see a difference in da_cnn_4 for the slantwise and convective trajectories. The green shades look rather similar to me.Citation: https://doi.org/10.5194/gmd-2023-27-RC1 -
AC1: 'Reply on RC1', Christoph Neuhauser, 05 May 2023
reply
Dear referee,
We would like to thank you for the in-depth review of our work. We respond to the individual points raised in the review in the attached supplement PDF. The comments from the review are highlighted in bold, followed by our responses. After the end of the discussion period, we will post a revised version of our manuscript which takes into account the recommended suggestions for improvement.
Best regards,
The authors
-
AC1: 'Reply on RC1', Christoph Neuhauser, 05 May 2023
reply
Christoph Neuhauser et al.
Data sets
Trajectory data with sensitivities to cloud microphysical parameters Maicon Hieronymus and Annika Oertel https://zenodo.org/record/7639184
Model code and software
Met.3D (1.6.0-multivar0) Christoph Neuhauser, Maicon Hieronymus, Michael Kern, and Met.3D Contributors https://zenodo.org/record/7636937
Video supplement
Video supplement Christoph Neuhauser and Maicon Hieronymus https://zenodo.org/record/7640203
Christoph Neuhauser et al.
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 274 | 72 | 9 | 355 | 3 | 2 |
- HTML: 274
- PDF: 72
- XML: 9
- Total: 355
- BibTeX: 3
- EndNote: 2
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1