Implicit neural representation (INR) networks are emerging as a powerful framework for learning three-dimensional shape representations of complex objects. These networks can be used effectively to model three-dimensional geological structures from scattered point data, sampling geological interfaces, units, and structural orientations. The flexibility and scalability of these networks provide a potential framework for integrating many forms of geological data and knowledge that classical implicit methods cannot easily incorporate. We present an implicit three-dimensional geological modelling approach using an efficient INR network architecture, called GeoINR, consisting of multilayer perceptrons (MLPs). The approach expands on the modelling capabilities of existing methods using these networks by (1) including unconformities into the modelling; (2) introducing constraints on stratigraphic relations and global smoothness, as well as associated loss functions; and (3) improving training dynamics through the geometrical initialization of learnable network variables. These three enhancements enable the modelling of more complex geology, improved data fitting characteristics, and reduction of modelling artifacts in these settings, as compared to an existing INR approach to structural geological modelling. Two diverse case studies also are presented, including a sedimentary basin modelled using well data and a deformed metamorphic setting modelled using outcrop data. Modelling results demonstrate the method's capacity to fit noisy datasets, use outcrop data, represent unconformities, and efficiently model large geographic areas with relatively large datasets, confirming the benefits of the GeoINR approach.

© His Majesty the King in Right of Canada, as represented by the Minister of Natural Resources, 2023.

Understanding the geometry of the subsurface is of critical importance to a wide range of applications including earth resource estimation (e.g., mineral, hydrocarbon, geothermal, groundwater), subsurface storage (e.g., carbon sequestration, radioactive waste), urban planning, climate change, and education. Three-dimensional geological modelling provides a means of representing the geometry of the subsurface based on available geological point data, typically from boreholes and outcrop observations, sampling geological units, the interfaces between them, and orientations of various structural features (Wellmann and Caumon, 2018).

The two most common types of three-dimensional geological modelling approaches are differentiated between explicit and implicit surface representations. Explicit approaches (Caumon et al., 2009; Sides, 1997) characterize three-dimensional surface meshes between geological units and/or faults and rely on either (1) digitized wireframes interpreted by users possessing geological expertise – guided by primary geological observations – which are converted into Bézier or NURBS (non-uniform rational B-splines) curves and surfaces (de Kemp and Sprague, 2003; Sprague and de Kemp, 2005) or (2) minimizing the surface roughness on a carefully constructed initial surface mesh using discrete smooth interpolation (Mallet, 1992, 1997) and supplied geological observations. Although these approaches can produce excellent structural models – given sufficient modelling and geological interpretative skill – they can require extensive time to develop and are difficult to update and reproduce. Implicit approaches, on the other hand, represent geological surfaces as iso-surfaces in a three-dimensional scalar field, interpolated from surface interface points, orientations, and potentially off-surface information (Lajaunie et al., 1997; Frank et al., 2007; Hillier et al., 2014). These approaches directly consider stratigraphic continuities and allow for a more flexible updating process but give rise to new problems, as they can produce geological models with modelling artifacts in structurally complex settings. For more details on the different geological modelling approaches see Wellmann and Caumon (2018).

Classical implicit interpolation – that is, non-machine-learning estimation – has been thoroughly studied and developed over the last two decades with many extensions and enhancements (Boisvert et al., 2009; Calcagno et al., 2008; Caumon et al., 2012; Cowan et al., 2003; de la Varga et al., 2019; Grose et al., 2019, 2021a; Hillier et al., 2014; Irakarama et al., 2021; Laurent et al., 2016; Renaudeau et al., 2019; Yang et al., 2019). Although their extensions and enhancements are remarkable, the underlying mathematical models by which they have been developed are not flexible and scalable enough to incorporate large volumes of geological data and knowledge. Inequality constraints (Dubrule and Kostov, 1986; Frank et al., 2007; Hillier et al., 2014), for example, useful for incorporating

Geological models tend to converge towards subsurface reality as more geological data and knowledge are incorporated in the modelling process. For complex geological structures, it becomes increasingly difficult in comparison to simple structures (e.g., layer–cake stratigraphy) to develop accurate representations. For these scenarios, much more geometric and geological feature relationship information is needed to generate realistic models, and better approaches are required to use this information within the modelling process. Due to the inherent flexibility, efficiency, and scalability of deep-learning approaches (Emmert-Streib et al., 2020) to incorporate data and knowledge, they have the potential to provide an ideal framework for incorporating new geological data and knowledge constraints into the modelling process, enabling the modelling of complex geological structures and at scales (e.g., high resolution over mine, regional, and national scales) that were previously unfeasible. Beyond being able to expand on the types of geological constraints for structural modelling in deep-learning approaches, they also have potential for direct incorporation of relevant interdisciplinary datasets (e.g., fluid flow, mineralization) where there exist latent relationships to structural features. Collectively, we see potential for these approaches to provide a needed solution for data and knowledge integration within a single end-to-end manner, and thereby overcome the modelling limitations of existing methodologies, and that more accurate representations of three-dimensional geological structures are efficiently produced.

In recent years there has been increasing interest in deep-learning approaches for various geoscience applications including seismic data interpretation (Bi et al., 2021; Perol et al., 2018; Ross et al., 2018; Shi et al., 2019; St-Charles et al., 2021; Wang and Chen, 2021; Wang et al., 2022; Wu et al., 2018, 2019), spatial interpolation of geochemical and geotechnical data (Kirkwood et al., 2022; Shi and Wang, 2021), remote sensing (Ma et al., 2019), and implicit three-dimensional geological modelling (Hillier et al., 2021; Bi et al., 2022). It is also worth noting the machine-learning approach that casts implicit modelling as a multi-class classification problem by Gonçalves et al. (2017). While this is not a deep-learning-based approach, it supports continuous implicit modelling but not faulting or unconformities. Although deep-learning approaches to implicit three-dimensional geological modelling are promising, they are still in their infancy, and much more research and development is required for them to reach their full potential. For example, although the recently proposed deep-learning approach (Bi et al., 2022) can generate faulted three-dimensional geological models structurally consistent with the data, there exist limitations: it cannot currently model unconformities, there is ambiguity in how to properly annotate or set scalar constraints on horizon data, and it may suffer from edge effects that can generate spurious discontinuities.

In this paper, we advance an existing implicit neural representation (INR) approach to three-dimensional implicit geological modelling that used graph neural network (GNN) architectures (Hillier et al., 2021). In recent years, there has been substantial interest and advancements in using INR networks on a wide variety of problems including modelling of discrete signals in audio, image, and video processing; learning complex three-dimensional shapes; and solving boundary value problems (e.g., Poisson, Helmholtz) (Sitzmann et al., 2020). Moreover, mathematical connections to kernel methods have emerged (Jacot et al., 2020) to establish a foundation for numerical analysis. In the field of computer graphics, they are being effectively used to represent complex three-dimensional shapes (Park et al., 2019; Gropp et al., 2020; Atzmon and Lipman, 2020; Davies et al., 2021; Wang et al., 2021) and reminiscent of surface reconstruction methods using radial basis function interpolation (Carr et al., 2001). Here, our aim is to support more complex geological structures, in both very rich and sparse-data environments. To this end, we demonstrate INR networks can be used efficiently to incorporate a comprehensive set of inequality constraints on stratigraphic relations; support modelling of unconformities; improve data fitting characteristics; and reduce modelling artifacts when modelling complex geological structures with large, dense, noisy, or sparse data.

The remainder of this paper is organized as follows. Section 2 describes the proposed methodology using INR networks for modelling complex geological structures containing unconformities. Section 3 presents modelling results using the proposed methodology. Section 4 discusses modelling characteristics of the approach and comparisons with other approaches. In the last section, Sect. 5, conclusions are given.

To better support the geological relations and feature representations mathematically, we have employed specific symbology. For clarity, definitions and notations used throughout this paper are provided below.

First, the notations for scalar, vector, set/tuples, and matrix quantities are as follows: lowercase,

Second, the paper utilizes three types of geological point data: (1) sampled geological interfaces

Third, point sets in this paper are denoted by

Fourth, for scalar fields, the following notation is used to shorten expressions. Consider a three-dimensional point

Our objective is to use multilayer perceptron (MLP) neural networks to perform three-dimensional implicit modelling of complex geological settings having both conformable and unconformable structures, given a set of

Complex geological setting for three-dimensional implicit geological modelling. Inputs for modelling include scattered data constraints, a stratigraphic column, and set of geological rules (erodes,

Let

Let

Implicit neural representations, also known as coordinate-based representations (Tancik et al., 2020), are neural networks that parameterize implicitly defined functions

Neural network architecture for three-dimensional implicit geological modelling.

For structural geological modelling, interpolation constraints

Errors associated with interface (circles), geological unit (triangles), and orientation (black arrows) constraints at training iteration

Stratigraphic relations are defined, in terms of scalar field differences, to encapsulate

Given a point

Stratigraphic relations between specific interface–interface and geological unit–interface pairs and associated constraints. Constraints are colored according to their

To measure errors at some training iteration

The three loss functions for the

Loss functions associated with

The effect of

For interface data, there are four interpolation constraints. Firstly, the variance of all scalar field values

The other three constraints utilize the stratigraphic relations to enforce the

To constrain the implicit model with geological unit data

For an orientation data point

The loss function associated with orientation data

It is well established that a disadvantage of implicit approaches for structural geological modelling is that they can produce modelling artifacts, commonly referred to as “bubbly” artifacts, yielding geologically unreasonable models particularly in complex structural settings (de Kemp et al., 2017). One way to address this problem is to impose a global smoothness constraint over the modelling domain using energy minimization principles. Here, we use the eikonal constraint (e.g., a unit-norm constraint) (Gropp et al., 2020)

The resultant loss function, or total loss function

An important training aspect to our proposed INR networks is the geometrical initialization of network variables. The variables are initialized such that the resulting outputted scalar field represents a shape with reasonable starting geometry for a specific geological application, which will be evolved through training by fitting given data constraints. Using standard variable random initialization schemes (Glorot and Bengio, 2010; He et al., 2015), resulting output scalar fields can be far from an optimal starting point for training, especially as the network's complexity increases (Fig. 6a). Consequently, if the training algorithm is rerun many times using the same conditions (Fig. 6b), resulting structural models can exhibit large variance in modelled structures. To solve these issues, training starts with a geometrically reasonable scalar field by geometrical initialization of network variables. For intrusive-like modelling, network variables are initialized to produce a spherical geometry (Fig. 6c) (Atzmon and Lipman, 2020), whereas for stratigraphic modelling, they are initialized to produce a planar geometry (Fig. 6d).

Scalar fields generated from initialization of network variables.

To initialize our networks to produce a scalar field with a planar geometry, we first pre-train a MLP network for 1000 epochs with the same parameterization (

Another training aspect utilized in the proposed methodology is applying learning rate schedules in the Adam optimizer (Loshchilov and Hutter, 2017). Learning rate schedules adjust the learning rate during training by decreasing the rate according to a prescribed schedule. While the Adam optimizer does adapt the initialized learning rate on a per-parameter basis, there is a benefit to decreasing the adaptable learning rate with increasing training epochs. Empirically, we have found that applying either step decay or cosine annealing learning rate schedules yields much lower losses and consequently better data fitting characteristics.

After implicit scalar functions are constrained by training, gridded points sampling a geological volume are inputted to the trained MLP network to generate

Constructing geological domains and combining scalar fields.

Geological domains are spatial partitions created by boundaries defining discontinuous features (e.g., unconformities) within which continuous geological features (e.g., conformable stratigraphy) exist. In this paper, only unconformity boundaries are used to create geological domains, although the same approach can be used for faulting. See discussion (Sect. 4) for future work with incorporation of faulting. To construct geological domains, first mask arrays defining

Iso-surface extraction methods can be applied to specific regions of implicit scalar field volumes provided appropriate Boolean masks are given and can be useful for obtaining the geological horizons of conformal domains. However, obtaining unconformity interfaces using this approach will lead to the production of anti-aliasing artifacts in triangulated surfaces. To resolve this issue, we develop an algorithm (see Algorithm 1 in Appendix A) using the open-source library PyVista (Sullivan and Kaszynski, 2019) to generate all iso-surfaces that can be cut by unconformities. The algorithm first extracts the set of continuous iso-surfaces for each of the

Modelling results are presented for two real-world case studies to demonstrate proof of concept: (1) a sedimentary basin with large, dense, and noisy well data and (2) a deformed metamorphic setting with sparse outcrop data. For both case studies, the learnable variables of our network, called GeoINR, are initialized with pre-trained models with planar geometry (described in Sect. 2.5) using the model parameters summarized in Table 1. These variables are updated using the Adam optimizer within the PyTorch framework so that modelling errors are minimized during training through the backpropagation process. Moreover, the cosine annealing learning rate scheduler was used with this optimizer. The network architecture parameters (

GeoINR model parameters values for case studies.

Organizing geological point datasets first requires all necessary knowledge to be extracted from a stratigraphic column. Stratigraphic knowledge including the geological rule of the interface (e.g., erosional, or onlap (conformable)) and the set of interfaces above and below each interface and geological unit are tabulated (see Appendix B for tables associated with both case studies). Corresponding tables describe the set of stratigraphic relations (Sect. 2.4.1) and associate interfaces and units to a particular scalar field among the series. This information is used for implementation purposes so that associated loss functions can compute measuring errors between the stratigraphic constraints and the version of the model at some training iteration

As with any machine-learning algorithm, neural network inputs require normalization for the network to learn useful latent representations and yield accurate predictions. Inputs for INR networks, which are spatial coordinates in this case, are normalized to some range for each coordinate dimension. For the first case study, each coordinate dimension is normalized to the [

After the networks are trained using the supplied data and knowledge constraints, inference is performed on all the points within a voxel grid (e.g., grid corners) covering the volume of interest. At these points, predicted scalar field values and geological units are computed. Once computed, scalar fields and geological units are assigned to geological domains, followed by iso-surface extraction of modelled interfaces.

Results for both case studies were obtained using a high-end desktop PC with an Intel Core i9-9980XE CPU and a single NVIDIA RTX 2080 Ti GPU.

The first case study is of a provincial-scale sedimentary basin covering an area approximately 451 000 km

Modelling results for the Lower Paleozoic portion of the WCSB in Saskatchewan generated using the proposed GeoINR methodology. Data and modelled results use

Sampling intraformational units (smaller circles) along the well path.

GeoINR model performance metrics for case study 1.

The presented data and resulting models in Fig. 8 all have a vertical exaggeration of

Loss function plots of constraints used for network training for the two sedimentary basin models produced using interface data only

Resulting model performance metrics on both datasets used in this case study are summarized in Table 2. These metrics include loss function values at the last training iteration, computing times, mean distance residuals between modelled interfaces and associated point sets, and

Similarity between the GeoINR model (left) for the Lower Paleozoic portion of the WCSB and the Gocad model (right) for the same region.

In addition to the model performance metrics provided, we also present qualitative and quantitative comparisons of our three-dimensional geological model – constructed using interface and intraformational data – to a recent version of the model for the same region constructed using a hybrid implicit–explicit approach with Gocad/SKUA™ (geomodelling software) (Bédard et al., 2023). As shown in Fig. 11, it is clearly demonstrated that the developed methodology can produce geologically consistent modelling results since both models are so similar (96 %). The small differences (4 %) between them are attributed to the Gocad model using (1) updated top formation markers; (2) different interface relationships (e.g., erosional, onlap) for interfaces

The second case study utilizes a regional-scale outcrop dataset from central Baffin Island, Canada (de Kemp et al., 2001; Scott et al., 2002; St-Onge et al., 2002). It consists of data from a deformed metamorphic setting having Archean-aged structural domes, composed of primarily felsic gneisses and plutonic rocks, that are basement to Paleoproterozoic rocks (Fig. 12a). The region is associated with a Himalayan-scale collisional mountain belt – the Trans Hudson Orogen – and consequently geologically complex. The dataset consists of 23 planar orientations (e.g., normals) sampled from the structural map (Fig. 12b), 352 geological unit (e.g., intraformational) observations (Fig. 12c), and 6 interface observations (Fig. 12d). While the geological unit and orientation observations were taken in the field, the limited number of interface observations were randomly sampled from the geological map. For this case study, the objective is to demonstrate that the developed methodology can generate representative three-dimensional geological models from typical outcrop datasets: i.e., with limited interface data, moderate geological unit information, and orientation observations. The resulting three-dimensional geological models are validated by visually comparing the modelled objects with the generalized geological source map (Fig. 12c) of the structural domes (red – Na-g), the onlapping quartzite (yellow – Pp-PD), and the overlying units (blue – Pp-PL). To facilitate comparison, the three-dimensional modelling results (Fig. 12e, f (right)) are clipped at the topographic surface (Fig. 12e, f (left), g).

Modelled geological map patterns using the outcrop dataset in a deformed metamorphic setting containing structural domes (red) and onlapping quartzite (yellow).

Three sets of modelling results are presented. First, only the limited interface and moderate intraformational data are used (Fig. 12e). The resulting modelled map pattern with these data closely matches the expected pattern on the generalized map (Fig. 12c). Second, in Fig. 12f, the sampled orientation observations were added, resulting in an even better match: the quartzite (yellow) between the two domes (red) is no longer connected. Note that the addition of orientation data strongly influenced modelled geometries, which then better conforms to the observed orientational data. Third, in Fig. 12g, the addition of more interface points (sampled from map contacts) results in only minor model refinement. This case study, therefore, demonstrates the ability to successfully model a complex geological scenario with limited interface data, which is typical of outcrop datasets.

GeoINR model performance metrics for case study 2.

Quantitatively, model performance metrics for this case study are summarized in Table 3. Note that the global smoothness constraint

Our results show INR networks can be successfully applied to a diverse range of geological settings, using well and outcrop datasets. In the first case study (Sect. 3.1), these networks were shown to be capable of generating large-area basin-scale models containing numerous unconformities and conformable stratigraphic interfaces from large and noisy well datasets. While the intraformational constraints only provided incremental improvements to the basin model, they did help demonstrate their compatibility with the methodology. However, these types of constraints proved to have much more impact on modelling with outcrop datasets, which have significantly fewer interface points (Sect. 3.2). They could also provide a mechanism for better leveraging geological maps in the modelling process by incorporating points sampled within unit polygons and appropriately weighting them in loss functions. Finally, it is clear, though unsurprising, that orientation constraints can strongly influence and improve modelled geometries of geological structures, especially in highly deformed geological settings and sparse-data scenarios.

The ability of INR networks to process

Iso-values associated with modelled interfaces in the proposed methodology vary in every training iteration of the learning algorithm, so modelling results are independent of user-defined iso-values. While defining specific interface iso-values is a straightforward way to encode the stratigraphic sequence (e.g., larger values are younger than smaller values), it is not optimal. Assigning specific iso-values for interfaces heuristically (e.g., uniformly distributed between some numerical range) can negatively impact resulting modelled geometries. This is particularly evident when dealing with varying unit thicknesses across the modelling domain and many interfaces. However, the GeoINR algorithm avoids these issues by learning the optimal set of interface iso-values during training, thus permitting more complex geological structures to be modelled. It is important to note that the stratigraphic constraints (Sect. 2.4.1) embed the knowledge of the stratigraphic sequence, with resulting interface iso-values respecting that sequence.

Loss functions used to constrain resulting implicit scalar functions make frequent use of scalar field gradients

Although the MLP network architecture parameters (

Several interesting points arise from comparing the GeoINR and GNN deep-learning approaches (Hillier et al., 2021) for three-dimensional geological modelling. First, the generation of latent representations (e.g., embeddings, features) in GeoINR is at a minimum 2 orders of magnitude faster than in GNNs. Second, GeoINR does not require the generation of an unstructured volumetric grid (e.g., tetrahedral mesh), enabling the development of higher-resolution models over larger areas. For example, for the provincial case study, the GNN tetrahedral mesh with varying resolution required

In other future work, we aim to tackle various discontinuous features commonly found in more complex orogenic and shield terrains, such as faults and shear zones. Because neural networks with similar architectures have shown the capacity to approximate discontinuous functions (Llanas et al., 2008; Santa and Pieraccini, 2023), we believe GeoINR should support the modelling of these complex features with appropriate enhancements and modifications (e.g., discontinuous activation functions).

INR networks have been reported as underrepresenting high-frequency components of signals and shapes by underfitting these components (Mildenhall et al., 2021). Positional encodings are a common strategy for addressing this issue by transforming the coordinates of a point into a set of Fourier features, which are then fed into the hidden layers of the network (Tancik et al., 2020). Our preliminary tests indicate that while this technique improved local fitting of high-frequency detail when using either ReLU or Softplus activation functions, it can generate unsupported large wavelengths of folded features.

We have introduced GeoINR, a geological modelling approach founded on INR networks composed of MLPs. GeoINR advances an existing INR approach by incorporating unconformities, constraints for stratigraphic relations and global smoothness, and improved training dynamics from the geometrical initialization of network variables. These advances enable efficient modelling of more complex geology, improved data fitting, and reductions in the generation of modelling artifacts. Case studies demonstrate the effectiveness and validity of the approach in diverse geological settings, different-sized areas, and various data regimes. Future work will extend GeoINR to support modelling of even larger datasets in more complex geological settings involving faulting and intrusions.

Interface information for case study 1. n/a denotes not available.

The sequence of the

Formation unit information for case study 1. n/a denotes not available.

Interface information for case study 2. n/a denotes not available.

Formation unit information for case study 2. n/a denotes not available.

See

Note the following from loss plots shown in Fig. D1: (1) when no orientation data are used in training (Fig. D1a), early training is strongly influenced by the stratigraphic constraints (

Loss function plots as function of training epoch for individual constraints used in case study 2.

The source code for the GeoINR neural network developed in PyTorch and data can be freely downloaded from

MH conducted the research, implemented the modelling algorithms, and prepared the manuscript with contributions from all co-authors. FW supervised the research. FW, EdK, BB, and ES contributed to the conceptualization of the overarching research objectives and analysis of modelling results from a geological point of view. KB prepared the datasets used for modelling.

The contact author has declared that none of the authors has any competing interests.

Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims made in the text, published maps, institutional affiliations, or any other geographical representation in this paper. While Copernicus Publications makes every effort to include appropriate place names, the final responsibility lies with the authors.

This research is part of and funded by the Canada 3D initiative at the Geological Survey of Canada. We gratefully acknowledge and appreciate the collaborative work with Arden Marsh and Tyler Music at the Saskatchewan Geological Survey and the Geological Survey of Canada (GSC) for building a three-dimensional geological model of the Western Canadian Sedimentary Basin in Saskatchewan, Canada, and sharing the dataset used for this research and their extensive geological knowledge in Saskatchewan. Additionally, we are thankful for all the discussions with colleagues from RWTH Aachen University and the Loop 3D project. We also would like to thank both anonymous reviewers for their comments, which greatly improved the manuscript. Natural Resources Canada (NRCan) contribution number 20220381.

This research has been supported by the Canada 3D project at the Geological Survey of Canada.

This paper was edited by Thomas Poulet and reviewed by two anonymous referees.