the Creative Commons Attribution 4.0 License.
the Creative Commons Attribution 4.0 License.
| 20 Apr 2023
GeoPDNN: A Semisupervised Deep Learning Neural Network Using Pseudolabels for Three-dimensional Urban Geological Modelling and Uncertainty Analysis from Borehole Data
Xuechuang Xu
Xulei Wang
Mark Jessell
Vitaliy Ogarko
Zhibin Liu
Yufei Zheng
Abstract. Boreholes are one of the main tools for high-precision urban geology exploration and large-scale geological investigations. At present, machine learning based 3D geological modelling methods for borehole data have difficulty building a finer and more complex model and analysing the modelling results with uncertainty. In this paper, a semisupervised learning algorithm using pseudolabels for 3D geological modelling from borehole data is proposed. We establish a 3D geological model using borehole data from a complex real urban local survey area in Shenyang, and the modelling results are compared with implicit surface modelling and traditional machine learning modelling methods. Finally, an uncertainty analysis of the model is made. The results show that the method effectively expands the sample space, the modelling results perform well in terms of spatial morphology and geological semantics, and the proposed modelling method can achieve good modelling results for more complex geological regions.
- Preprint
(2853 KB) - Metadata XML
- BibTeX
- EndNote
Jiateng Guo et al.
Status: final response (author comments only)
-
CEC1: 'Comment on gmd-2023-11', Juan Antonio Añel, 06 May 2023
Dear authors,
Checking your manuscript, I have seen that the input data for your work is in .mdb format. This is a proprietary format, which can be opened only using proprietary software. This issue precludes the replicability of your work. For example, I can not check the data, as I do not have the necessary software for it, and although I could have access to it, it is not free software, which is against the principles of scientific reproducibility.
Therefore, please, we would thank you if you could share your input data in an open ISO format that is accessible to anyone and without the need to use specific software. This could be .dat, .csv, .ods, etc.
Regards,
Juan A. Añel
Geosci. Model Dev. Executive Editor
Citation: https://doi.org/10.5194/gmd-2023-11-CEC1 -
AC1: 'Reply on CEC1', Jiateng Guo, 06 May 2023
Dear Juan，
Thanks for your suggestion. Now, I have shared the input data in an open ISO format (.csv) accessible to anyone via the original data sets link: https://doi.org/10.5281/zenodo.7535214
Best regards,
Xuechuang and Jiateng
Citation: https://doi.org/10.5194/gmd-2023-11-AC1
-
AC1: 'Reply on CEC1', Jiateng Guo, 06 May 2023
-
RC1: 'Comment on gmd-2023-11', Anonymous Referee #1, 04 Jul 2023
This manuscript proposes an approach for geomodeling based on a deep neural network and a pseudo-labeling scheme to extract training data from borehole data. The method is tested on a shallow case study with a high density of boreholes, with a cross-validation to assess model performance. The method is qualitatively compared with two other implicit modeling approaches.
Overall the manuscript is difficult to follow, with a lot of approximate or inaccurate terms. The novelty is not clear, and the validation is not robust enough to support the conclusions.
Major comments:
Some words are used throughout the manuscript without clear definition of what they mean, although they are not commonly used by the geomodeling community. "geological semantics" is an important example: it is used in the method validation but it's unclear what that actually encompasses.
This constant use of inaccurate or approximate terms makes it difficult to assess how the method works and what is novel. I still can't figure out why the TIN mesh is needed or even why the pseudolabels themselves are really needed. On that note, I would question the use of semi-supervised learning here, which normally implies the use of a few labelled data and a lot of unlabelled data during training. Here all the data fed to the network are labelled if I'm not mistaken.
The cross-validation should be extended to include the implicit modeling approaches, otherwise it lacks robustness. Some qualitative, visual validation is fine, but it cannot be the main tool of comparison.
There is no mention of the exact architecture of the network, nor how it was developed and its impact on predictions. All that needs to be discussed in the paper.
I am wondering whether the application is representative of real conditions. I am more used to deeper applications, where such high density of boreholes is impossible. This is essential since most implicit methods are developed for larger scale, deeper applications. If the study targets a specific setting (small-scale, shallow), then it needs to be more clearly stated. Similarly, the targeted application needs to be clarified (geotechnical applications?) and the goal of building a model as well. Why do you need to build a 3D model in the first place with so many data available? And do you really need such advanced method to build a 3D model? This is where the lack of quantitative validation is a big issue.
There have been other studies on using deep learning for implicit modeling, e.g.:
https://doi.org/10.5194/gmd-2023-11
https://doi.org/10.5194/gmd-2022-290
How does this work compare to those? Both in terms of prediction errors and computational cost.Similarly, how does it compare with other modeling methods targeting quaternary deposits, e.g.:
https://doi.org/10.3389/feart.2022.884075Overall there is a lack of discussion around the advantages and limitations of the proposed approach, especially compared with other approaches. There is a long list of papers mentioned in the introduction, but many are not really linked to this study, and most are not really discussed.
There is also a lack of proper referencing, with more recent studies cited instead of the original ones (see implicit modeling or perceptron for instance).
Specific comments:
Abstract
Line 14: I'm not sure what "urban geology exploration" refers to.
Line 16: Why would we need more complex models? What we want is to support decision making, and more complexity doesn't necessarily means better support (on the contrary).
Line 16: "analysing the modelling results with uncertainty" I'm not sure about the phrasing there, maybe replace "with" by "and their"?
Line 17: "built" instead of "establish"?
Line 18: What makes this survey complex? The number of data? A fine stratigraphic layering? A folded and faulted structural setting?
Line 19: What is a traditional machine learning method? How is it different from the method proposed in the paper?
Lines 19-20: How is the uncertainty analysis performed?
Line 20: What does "expanding the sample space" mean?
Line 21: What does "geological semantics" mean?
Line 22: More complex regions than what region? The one from the case study? Is that something that was tested? If not, how do you know?
1. Introduction
Line 24: What do you mean by "Geological spatial distribution" exactly? Distribution of facies? Of rock properties?
Line 25: What do you mean by "underground situation"? What are the properties of interest? How far deep do we need them?
Line 26: How to determine whether a geological model is reasonable or not?
Line 26: What does "intuitive expression of geological features" mean?
Line 27: "revelation of the spatial distribution law" That should be part of the conceptual model used as basis to build the geological model (e.g., do we have a channelized sedimentary environment or a carbonate platform), not really of the geological model itself.
Line 28: "a long geological process" Usually it is the results of multiple processes (e.g., water flow and sediment transport in rivers, wave action, diagenesis, folding).
Line 29: We do have those laws, and they are implemented with various degrees of approximation in forward stratigraphic models in the case of sediment transport for instance. But using those models remain costly.
Line 31: "powerful computing power of computers" The phrasing is weird here, and this is true of any numerical method, not just deep learning.
Line 31: "complex fields" Which ones? What makes them complex?
Line 31: "increasingly attracted the attention of geological researchers, such as 3D modelling" Any reference?
Line 36: What does "intuitively" mean here?
Line 37: The distinction between explicit and implicit modeling was defined long before Wang et al. (2018).
Line 37: What is a geological semantic constraint?
Line 38: Which geological laws?
Line 42: Which implicit equations? Your explanation of implicit modeling is a bit confusing, and implicit modeling is not always based on basis functions.
Line 52: "has been developed as a method for boreholes" What does that mean?
Lines 48-54: The link between stochastic simulations and implicit modeling is unclear. And Lancaster & Bras (2002) is not a stochastic simulation method.
Line 60: What does "mined data" mean?
Line 64: What does it mean to "intelligently generate" a model? This sounds like an unsupported value judgment.
Line 67: "has been realized" This sounds like a long list of papers using deep learning just for the sake of making a list. What's their relation to this study?
Line 72: There are a lot of repetitions all along the introduction, which would benefit from a reorganization to be clearer, better introduce the context and problem statement, and better link to previous studies actually related to this work.
Line 75: How do those two approaches relate to all the methods mentioned before? How do they relate to explicit and implicit modeling.
Line 91: There needs to be a stronger and more explicit link to all the studies mentioned before. How does the previous paragraph relates to this study is unclear.
Line 93: "the pseudolabel data with high confidence" What are those data? Where do they come from? This is either too little or too much detail at this stage, and I'm quite confused about the objectives of the paper.
2. 3D Modelling Method Based on Deep Learning
Line 99: This is true all the time, not just with deep learning.
Lines 101-102: "the model at the borehole should be as consistent as possible with the stratum information revealed by the current borehole" I'm not sure what that means, is this about upscaling?
Line 104: "To increase the amount of data, the borehole data are upsampled" why is that needed?
Figure 1: I don't understand what is going on here, and the text doesn't help. What do the sections mean?
Line 124: "direct or indirect" Why "or"? What is their indirect role?
Line 124: What is "geological semantic information"? And what is "geological semantic information with high reliability"? How do you assess whether geological information is reliable or not?
Line 126: What is the Delaunay rule?
Line 126: I'm really confused about the reason we need all this.
Line 130: What is a "GTP-like section connection method"?
Line 131: "can simulate a variety of complex geological phenomena" What does that mean? How can a TIN simulate anything?
Line 132: "modelling scope of this study is mainly for a quaternary sedimentary surface" That needs to be specified in the introduction.
Line 133: "strata are deposited in chronological order" Geological layers are always deposited in chronological order.
Line 136: But by normalizing like that (x,y) and z don't follow the same scale anymore. How does that affect the results?
Line 144: I can't find any proof of that.
Line 146: Huang et al. (2012) didn't define the single-layer perceptron, this was done in the 40s and implemented in the late 50s.
Line 148: Activation functions are more important to capture non-linearity, and a single layer neural network is a universal approximator. So multiple layers are not always needed.
Line 152: "input index and output index" What are those indices? So it doesn't use the actual values?
Line 152: "which is a multilayer feedforward neural network" No the result is a prediction, the neural network is a model to get that prediction.
Line 156: The phrasing is confusing, there is a weight associated to each neuron.
Line 158: "The data in the data set are output after the multilayer perceptron" What does that mean?
Figure 2: It should be "prediction", not "pridiction".
Line 176: "borehole data tend to be dispersed" What does that mean? Aren't your data point cloud data in that case?
Line 179: But that upsampling can lead to imbalance and biases.
Line 184: "no specific mathematical law for the attribute of strata" I don't understand what that means.
Line 189: Which model is used here? It's not clear to me what is the advantage of doing that, why not using that model directly everywhere?
Line 222: "SVM method" Is that the HRBF mentioned above? What's the difference?
Lines 224-225: "The model established using the algorithm mentioned in the experiment is visualized with the developed visualization platform" I don't understand what that means.
3. Experimental method and verification
Line 218: What is the "rationality" of a model?
Line 221: At this stage there are still no explanation of what "geological semantics" means.
Line 235: That is a very high sampling rate (and unusually regular, but that might be because it is related to a geotechnical study?), do we really need advanced modeling methods there? There is no mention of why a 3D model is needed, so it is difficult to judge that here.
Line 245: I don't think "confused" is the right term here. "Missed"? Or "mislabeled"?
Line 245: Recall, precision, and F1 would be more adequate metrics than accuracy, especially if the layers have variable thicknesses.
Line 246: What does "depositional termination" mean?
Figure 5: Are the labels the different layers? It's unclear what "label" means here.
Figure 6: Same comment here: are the classes the different layers? And what are the micro- and macro-average ROC curves? Different averages for all the layers? It would be better to have them above the other curves, they are difficult to see.
Figure 6: There seems to be large differences between the different layers, why is that?
Figure 6: The use of the same color for multiple classes makes it difficult to analyze the plot, but are the areas right? Class 11 is said to have an area of 1, yet on the plot its true positive rate doesn't go straight to 1, so there seems to be a big issue there.
Line 259: It's still unclear to me what is the use of the TIN mesh.
Line 266: Is the cross-validation based on the borehole data or on the boreholes? This is important, because spatial correlations mean that the former will underestimate the prediction error.
Line 278: "three depositional terminations between any stratum and the surrounding boreholes" I don't understand what this means.
Lines 276-278: But by doing so you're creating a bias in your validation. In a real application you will miss locations that would be critical to understand the layers' distribution, and that is what needs to be tested to have a robust validation of the method.
Line 286: "excluded sample K1 test borehole data" Which one is that?
Line 290: What happens if you remove more than one borehole though?
Figure 10: What about the uncertainty here?
Line 296: What is the "rationality" of a model? And what is a "mature" modeling method?
Line 297: Any reference for this method?
Line 299: I'm not sure I understand what a vector model is. The implicit model can predict a value for the scalar field at any location, so why is there a need to transformed the predictions into a grid?
Line 303: There are so many data available that this is hardly surprising.
Line 310: "predicts [...] with high confidence, which has certain uncertainty" How can prediction have a high confidence yet have a certain uncertainty?
Line 321: "modelling results of the proposed method for complex geological conditions are significantly improved compared with those of the SVM method" This is a judgment call without actual proof. Why not apply the cross-validation to the HRBF and SVM models as well? This would make the results much stronger.
Line 323: The consistency of which feature? This is a very subjective and partial validation.
Line 325: "consistent with the geological semantics" What does that mean? How do you check that in an objective way?
Lines 327-328: You cannot conclude that from a single section in a single case study, this is too high a jump without quantified justification.
Figure 13: The colorscales don't make any sense, 0.3 on the left side looks like 0.4 on the right side.
Figure 13: There seems to be an issue with the labeling of the sub-figures, top and bottom are the same.
Line 349: What's a significant decrease? Based on the plots, the difference appears quite limited, and its really not clear that the semi-supervised method outperforms the supervised method. Also the validation here remains qualitative (comparing plots visually), it would be better to have summary statistics to compare the two methods in a quantitative way.
4. Discussion
Line 359: I'm not sure I understand the problem actually. From boreholes we get the horizons, so can directly interpolate between horizons, or sample data points within layers, which sounds like what is done here, but I would hardly qualify that as novel. And in the second case we loose the exact location of the horizons (unless one samples very finely), and looking back at the method section, it is not clear to me how this is tackled.
Line 367: I don't understand the comparison to MPS specifically. And a grid is used here too so I don't understand the point you're trying to make here.
5. Conclusion
Line 383: I think you mean "domain" instead of "scope".
Line 386: "for sampling data points" What do you mean by that? A randomized K-fold cross-validation? Was this mentioned before?
Line 392: "has more advantages in dealing with more complex geological phenomena" This is not supported by the results.
Citation: https://doi.org/10.5194/gmd-2023-11-RC1 -
RC2: 'Comment on gmd-2023-11', Anonymous Referee #2, 13 Oct 2023
This manuscript presents a semi-supervised deep-learning neural network method using pseudolabels to generate a 3D geological model from borehole data. The study provides a case study, used for comparison with implicit modelling and previous ML approaches. The corresponding uncertainties are visualised using the information entropy and confusion index methods. Overall, I found the approach interesting but the manuscript was difficult to read and the conclusions a bit too subjective, with many imprecisions and selective comparisons of the results. As such, I would recommend publication with major revisions to address the main comments.
The approach presented needs to be better explained, with a clear focus on what is novel and better comparison against other approaches (e.g., https://doi.org/10.5194/gmd-2022-290,https://doi.org/10.1007/s11004-021-09945-x, https://doi.org/10.1016/j.cageo.2020.104522, https://doi.org/10.3390/ijgi12030097), not just a single supervised method. The ML hyperparameters should all be mentioned (e.g NN architecture, number of neurons per layer) and the authors should explain or comment on the selected values. I didn’t quite understand the role of the TIN mesh, which is unfortunate as it is at the core of the method. What is the sensitivity of the results with respect to the triangulation selected? The supervised vs semi-supervised difference needs a bit more detail. How are the probabilities obtained that are used in the uncertainty analysis? What is being varied, and how, to compute those probabilities?
The comparison with implicit modelling needs more details about the pros and the cons, with a bit more objective pitch. The introduction should touch on the negative side-effects of ML approaches. The results shown (figure 11) display more geologically realistic output from the implicit method but the text only points out the reasonably good outcome of the ML approach, which sounds a bit defensive. Since the new method doesn’t seem to beat the implicit modelling in terms of result quality, a more complete and objective picture of the benefits needs to be presented. This includes commenting on the respective computational costs or suitability in terms of applications, for instance, to better illustrate which one should be used when. The example selected has a particularly large amount of boreholes, with some regular spacing, without drastic variability in borehole depths. The discussion should touch more on other applications with different conditions.
Some specific comments in the text:
- 24 “reasonably” for which criteria?
- 26 again using subjective words like “reasonable” and “intuitive”; please try to be more specific.
- 28 what is the purpose of mentioning the temporal distribution when there is no notion of time in the rest of the paper?
- 29: mechanical laws are rather well understood and can capture the stratum distribution well. The challenge is in applying them (with the appropriate distributions of material properties and boundary conditions).
- 52: more information is needed to understand “defining a random simulation path”. Which simulation? Why on a path? Why a random path?... Or, why mention that anyway? This is repeated in the text, so it needs to be really clear.
- 74: what other data than “spatial data” come from boreholes?
- 79: The lithology simulation is a spatial point simulation I guess, please rephrase
- 82: mentioning B-splines (2D curves) and (3D) voxels. Do you mean NURBS?
- 82 what is the criterion for those curves? I guess separation/clustering problem but it needs to be mentioned explicitly
- 83: why is the model more accurate?
- 84: how is the randomly selection of the curves reconciled with its automatic selection (l.82). In the absence of precision l.82, I had imagined the simplest option, which is deterministic. Why random?
- 85: what do you mean by “overthrow the order of strata”? Why is it lower in accuracy?
- 91: the link between that last paragraph of the introduction and the previous conclusion about lithology prediction is not explicit enough.
- 100: assuming vertical boreholes? How does the method work for arbitrary shapes of boreholes?
- 115: Hij doesn’t appear in eq.1.
- 130 “GTP” acronym undefined (I guess from Wu 2004)
- 167: rephrase “multiplying the weight matrix”
- 169: what are ‘high-dimensional features”?
- 2 “pridiction” typo
- 179-180: rephrase that (unreadable) sentence (“Only by … limited features”)
- 187: why the highest confidence and where (“in this range”)?
- 222: definition of mu_max: the probability of …
- What is the purpose of section 3.1? (What is the relevance of the weather information?)
- 237: how is the training accuracy evaluated? How much data is used for calibration vs testing?
- 4 The loss should be shown in log scale. (There’s also a typo “accurary”)
- 243 loss function is “close to zero”, not “poor”
- 8: Please show the TIN mesh as well. Is it made of a single connected component? (What if the 6 boreholes in the top left corner were further apart?)
- 274: what is the “geological semantic information”? This is an important concept that needs to be properly defined.
- 10: which cut lines?
- 290 “reasonable” is the real question: compared to what? Which features are you paying attention to? Can you get some quantitative measure? The comparison of figure 11 shows that the HRBF model is much more “reasonable” (except for the base)…
- 296: what is the “rationality”?
- 303-310: the comparison needs to be more explicit about the better results from HRBF, instead of mentioning “certain differences”
- 326 “however” is the wrong logical link
- 351, I don’t understand what is the model “with close raster accuracy”
- 13, colorbar legends of (b) and (c) are wrong
- 346 the comparison is not all that clear from the figures... It could be true if focusing on the front face of the model, but not if looking at the top face. Compute the difference in the whole model and plot that difference to better illustrate. Does the same conclusion still hold?
- 361: what do you mean by “artificial settings”?
- 370: which “predicted area”?
- 374 “thicker boreholes”? or boreholes showing with thicker units?
Citation: https://doi.org/10.5194/gmd-2023-11-RC2 -
EC1: 'Comment on gmd-2023-11', Thomas Poulet, 16 Oct 2023
Dear authors,
I appreciate your patience with this uncharacteristically long Discussion time and look forward to your response to the reviewers' major comments.
Best regards,
Thomas Poulet
Citation: https://doi.org/10.5194/gmd-2023-11-EC1
Jiateng Guo et al.
Data sets
Research borehole data Jiateng Guo and Xuechuang Xu https://doi.org/10.5281/zenodo.7535214
Model code and software
GeoPDNN 1.0 Jiateng Guo and Xuechuang Xu https://doi.org/10.5281/zenodo.7839508
Video supplement
Semisupervised Deep Learning Neural Network Using Pseudolabels for Three-dimensional Urban Geological Modelling and Uncertainty Analysis from Borehole Data Jiateng Guo and Xuechuang Xu https://drive.google.com/file/d/13VERDXM6YJmP7xMabQy3IjhCExuQSWzk/view?usp=sharing
Jiateng Guo et al.
Viewed
| HTML | XML | Total | BibTeX | EndNote | |
|---|---|---|---|---|---|
| 480 | 162 | 18 | 660 | 5 | 4 |
- HTML: 480
- PDF: 162
- XML: 18
- Total: 660
- BibTeX: 5
- EndNote: 4
Viewed (geographical distribution)
| Country | # | Views | % |
|---|
| Total: | 0 |
| HTML: | 0 |
| PDF: | 0 |
| XML: | 0 |
- 1