Most SGMs use two types of constraint: value constraints, which represent a geological contact and specify the value of the underlying implicit field at specific locations, and gradient constraints that constrain the direction of the field's gradient to match structural (e.g. bedding) measurements . Standard loss functions (e.g., Mean Squared Error) can be used to fit neural field predictions to value constraints by penalising differences between predicted and observed values at the constraint locations. Non-standard loss functions e.g., involving the signed distance for geological contact information as used in can also be implemented for such constraints.

Spatial gradients of the neural field can be obtained through automatic differentiation with respect to the inputs (coordinates). As shown by , this gradient can be compared to structural measurements to penalise violations of gradient constraints (i.e. bedding measurements). This comparison can be signed (called gradient constraints), using e.g., Mean-Squared Error (MSE) loss on the normalised gradient vector, or unsigned (called tangent constraints), using absolute cosine similarity as a loss function . The former implies a known gradient (“younging”) direction, while the latter minimises the acute angle between the two vectors such that orientation is constrained to a known axis (strike and dip), but with unspecified younging direction.

Finally, as outlined in Sect. 3, we show that quantitative measurements of meaningful rock properties (e.g., features derived from geochemical, hyperspectral or petrophysical data) can be used to directly constrain neural field interpolations. The core assumption here is that an implicit field that accurately represents subsurface geology should explain a maximal amount of the variance in the measured properties. Property constraints can thus be implemented using a reconstruction loss, where predicted implicit field values are passed through a learned forward model (cf. Sect. 3.2) to reconstruct property measurements and penalise errors (using e.g., MSE loss).

As shown by , neural field interpolations can be formulated so that value constraints are replaced by constraints on the relation (equality and inequality) between pairs of observation points. Because the interpolated values in an SGM are typically arbitrary, such approaches find any (under-constrained) solution in which geological contacts have the same value i.e., equality constraints, and points in younger geological units are systematically greater than (or less than, depending on convention) older ones, i.e., inequality constraints. This representation has been suggested to enable significantly more flexibility than traditional value constraints .

The previously mentioned local and relational constraints can be used to generate models that fit a sparse set of data. However, the under-constrained nature of the problem means a typically infinite number of solutions can be found, many of which are geologically implausible. Global constraints are thus typically used to encourage smoother solutions over others, and integrate geological rules that penalise geologically unrealistic geometries.

Smoothness regularisation is used in discrete implicit modelling to constrain the evolution of the interpolated field between geological observations. In a discrete interpolation framework the implicit function is represented on a piece-wise support (e.g., tetrahedral mesh or cartesian grid) and geological constraints are applied by adding linear constraints to the element containing each observation. The smoothness constraints are applied globally to the interpolation function by adding additional constraints to locally minimize variations in the function gradient. This can be done by minimising the variations in the second derivatives of the implicit field using finite differences or by minimising the variation in function gradient projected across the shared elements . Data supported methods such as co-kriging or radial basis function (RBF) interpolation typically use globally smooth basis functions, which naturally yield smooth interpolants. The smoothness can be controlled by the shape parameters of the basis function used. In neural field approaches, the loss functions can be adjusted to implement smoothness regularisation. This is based either on the magnitude of the gradient vector e.g., or on the second derivative tensor (i.e., the hessian) of the implicit field. These adjustments require some global support (set of points) at which these losses are accumulated.

However, smooth implicit fields can still have geologically impossible isosurfaces. Most standard interpolators operate on the minimum curvature principle and are often implemented via biharmonic splines or discrete smooth interpolation. It implies that the resulting field approximates a solution to the biharmonic equation (∇4ϕ=0) . Biharmonic functions, while smooth, lack a maximum principle and therefore impose no limitations on the formation of local extrema. This leads the interpolator to generate closed isosurfaces (colloquially known as ”bubbles”), that contradict the fundamental principles of original horizontality . To encourage geologically realistic, bubble-free stratigraphy without allowing the network to collapse to a trivial solution, we have implemented a set of global geologically-informed loss functions (Sect. 3.2.2).

In many cases it is also desirable to enforce a constant (or approximately constant) gradient magnitude . Changes in gradient magnitude imply variations in the thicknesses of the sedimentary units (which correspondingly imply changes in volume during deformation events), which might be unwanted. For example, penalised gradient vectors of non-unit lengths to limit lateral variations in the thickness of interpolated sedimentary units. Such constant gradient constraints can be useful when interpolating true distance fields, such as the distance from a fault surface, but quickly become problematic when modelling units that change thickness (e.g., sedimentary series that change thickness, folds, etc.).

Many geological objects (folds, faults, intrusions) create discontinuities. Such discontinuities contradict the principles of global and local maximisation of smoothness. One approach to include such discontinuous geological features is to modify the interpolation framework and incorporate the geometries from separate implicit fields in a step-wise fashion, as in e.g., LoopStructural . LoopStructural models are built using a temporally successive approach where the most recent geological objects are built first, and these objects are used to constrain the geometry of older geological objects through kinematic reconstruction.

In the LoopStructural approach, faults are incorporated into the implicit model in two steps. First, constraints along the fault are used to interpolate an implicit field representing the fault surface. Then, this field is either used to split the interpolator into separate domains (i.e., domain boundary faults), as a step function in the polynomial trend or to constrain a kinematic operator for the fault . The kinematic approach provides the most control over the faulted surface, but requires a priori knowledge to constrain the displacement and slip direction.

Folds also pose challenges for smoothness regularisation. One approach is to modify the implicit interpolator by reducing the regularisation in the direction orthogonal to the axial surface and along the fold axis, such that high curvature in geologically plausible directions (i.e. fold hinges) is not penalised .

SGMs are defined in 2- or 3-D space. This low dimensionality presents a challenge for the neural fields, which can struggle to converge given the small number of input features. To help mitigate these limitations, proposed Random Fourier Feature (RFF) mappings, a method that maps input coordinates into a higher-dimensional feature space by projecting the input coordinates onto randomly chosen direction vectors and passing the resulting lengths through sine and cosine functions. This work was built on the contribution by , who proposed these feature mappings as approximators for large-scale kernel machines. This mapping (1) improves the representation of high-frequency signals in neural fields, (2) accelerates convergence during neural network training, avoiding the problem of spectral bias in neural networks , and (3) allows the network to learn a quasi-Fourier representation of the spatial variability. This approach is particularly effective to accurately capture geological complexities, especially in situations where geological structures display periodic geometries (i.e. folds).

In our implementation, input coordinates are projected onto a set of M 2- or 3-D vectors in which each component is randomly drawn from a Gaussian distribution with zero mean and unit variance. The resulting values are scaled by a frequency term and passed through a sine and a cosine function, to give 2M features ranging between -1 and 1, analogous to amplitudes in a sparsely sampled Fourier domain. The frequencies of the mapping are explicitly scaled with length scale parameters to match the expected scale of variability in the model being constructed. These features effectively seed the network and guide feature learning at desired scales.

For a 3D input, we get a 2M dimensional feature vector νs for every length scale ℓs given by 1νs=[sin⁡(2πWsr),cos⁡(2πWsr)],where Ws=ℓs-1WM×3 where r=[x,y,z]T, Wij∼N(0,1), and the sine/cosine transformations are applied element-wise followed by concatenation along the feature axis. Hence, for L length scales (ℓs∀s=1…L), we get a 2ML-dimensional feature vector, which is then fed into the subsequent multi-layer perceptron to get the implicit field value (Fig. a).

This RFF approach enhances the model's ability to learn high-frequency geological features. Furthermore, the encoding introduces inherent stochasticity into the interpolation process. Models trained with different seeds will learn different representations of the subsurface geometry for the same constraints, due to the random nature of the projection directions. Therefore, we can generate model ensembles by changing the seed of the random number generator to get different projections for each model. This becomes useful for stochastic uncertainty estimation, as multiple outputs can be generated without perturbing the input data as is typical in established approaches; e.g.,. In specific cases, it theoretically also allows known information on e.g., fold wavelength to be indirectly integrated as a model parameter by seeding the Fourier frequencies (and potentially directions) according to measured fold geometry.

The RFF encoding is followed by a standard multi-layer perceptron (MLP). The MLP block applies non-linear activation functions to all hidden layers. Because our framework computes derivatives via automatic differentiation (AD), the chosen activations must be C2 differentiable (as the network requires a second backward pass for the optimisation). Functions lacking this property, such as ReLU, produce abrupt edges in the resulting interpolation. Even among C2 differentiable options, performances may vary. For instance, the hyperbolic tangent (TanH) is stable, but its second-order derivatives are small and prone to saturation, which can impede convergence. Empirical testing showed that Swish SiLU, and Mish provided the best overall results. We have also experimented with options in which the Fourier features are directly connected to the predicted scalar field with a single layer of neurons (i.e. with no hidden layers and no activation functions). Early tests suggest that this minimalist neural field performs well in many situations, so we suggest that this architecture is used as a starting point (and extra complexity is added only if needed).

A neural field stores the implicit field within the weights and biases of the MLP block. These weights and biases are trained using a combination of local and global losses (the RFF projection remains fixed after initialisation). Local losses are accumulated at specific coordinates for which there is information on the implicit field value, its gradient, or its relation to other points in the model (see Sect. 2.2). Global losses are accumulated either over a grid of points that covers (or extends slightly beyond) the model domain, or (typically) on points that are randomly sampled from a grid during each training epoch. We have implemented loss functions in curlew such that each of these aspects can be introduced via corresponding hyperparameters. However, we recommend that any specific neural field should only use two to four of these loss functions (usually a local/relational loss coupled with a global loss), with the remaining terms disabled by setting their corresponding hyperparameter to zero. Every additional loss term adds an extra objective for the optimiser to minimise, quickly turning the interpolation into a complicated multi-objective optimisation problem (see Sect. 5.3).

The aforementioned sections explain the constraints and controls over the interpolation of a single, smooth, and continuous implicit field. However, most SGMs contain multiple geological events, separated by unconformable, intrusive or faulted contacts. Each of these events typically need to be represented using their own interpolated implicit field, before temporally guided intersections are used to derive a combined model .

In curlew, this is achieved using overprint and/or offset relations (Fig. b). Therefore, most neural fields in a model built using curlew represent a geological event (Table ) that (1) creates new units (generative events, such as e.g., depositions or intrusions) or (2) displaces older units (kinematic events, such as faults). Sheet intrusions (e.g., dykes) are an example of events that do both – generate a new unit (the dyke fill) while displacing the older units (due to opening along the dyke).

Generative events in curlew are combined with older ones using an inequality: implicit field values representing deposition above an unconformity “overprint” older values where they exceed a threshold value, and values defining a dyke overprint older values within a specified range (proportional to the dyke aperture). Conversely, kinematic events define vector displacements (based on the gradient-direction and/or tangent plane of the interpolated implicit field) that retro-deform the input coordinates used to constrain (or evaluate) older fields. Importantly these vector displacements are differentiable, allowing the combined training of multiple neural fields to learn additional unknown parameters (e.g., fault slip magnitude).

Overprint relations are implemented using a differentiable sigmoid function, such that values from a younger implicit field replace those of older ones based on an inequality. For unconformities, this inequality specifies a basement value below which the interpolated values are ignored (to retain predictions of basement lithology from older neural fields). An Event ID (eid, see Fig. b) is assigned to each overprinted location to keep track of the neural field that generated each implicit field value (and distinguish regions representing e.g., the interior of a dyke from those representing the surrounding stratigraphy). The geological operators outlining the implementation of kinematic events in curlew are outlined in the following sections, though terminology for these operations varies across the literature

The final structural element implemented in curlew is the domain boundary, which enables the segmentation of geological models into distinct sub-domains, each governed by its own sequence of (older) geological events. Domain boundaries thus partition the model into separate neural field chains, based on inequalities defined on the domain boundary implicit field, and can be combined to construct complex geological structures and relations.

A typical use case arises when sedimentary units are deposited above an unconformity but pinch-out laterally (onlap geometries). Because such deposits do not follow the generative logic of a conformal event (i.e., they are not stratigraphically contiguous with the unconformity), they cannot be represented within a single event chain. Instead, the unconformity is defined as a domain boundary, which splits the model into two stratigraphically and structurally independent domains: one below the unconformity and one above it.

Another key application of domain boundaries is in the modelling of major fault zones. When a fault induces large displacement, or juxtaposes basement rock against cover sequences, the stratigraphic relationship between the foot- and hanging-wall often becomes unresolvable. For such structurally complex settings, the fault is most simply defined as a domain boundary separating sub-models with different geological histories.

This formalism allows curlew to integrate geologically inconsistent or discontinuous regions within a unified framework. It removes the need for users to manually partition their models or construct separate simulations for each geological block, and facilitates instances where domain boundaries are affected by younger deformations (e.g. faults).

To illustrate the various loss terms described above, and to showcase the importance of the length scale parameter used during the Fourier feature projection, we have generated a simple test model containing a folded stratigraphic sequence (Fig. ). This model involves two generations of folding events, a small wavelength small amplitude folding overprinted on large wavelength large amplitude folds. A Fourier series was used to generate similar folds with relatively sharp noses (Fig. a). This sequence was then sampled along hypothetical boreholes (Fig. b) to simulate gradient, value and tangent constraints. Note that while this synthetic example is 2D for ease of visualisation, all of the aforementioned methods (and the successive results) could be applied in 3D too.

Only the simplest geological models can be represented by a single implicit field. Hence, we develop a second synthetic model that highlights curlew's approach to fault displacement, dyke injection and the use of domain boundaries to model intrusive and onlap relations (Fig. a). The implicit fields associated with the depositional and kinematic events of the model are generated using predefined analytic equations, allowing us to construct a completely synthetic model for testing. Seed locations within these implicit fields were used to define isosurfaces that mimic stratigraphic contacts (Fig. b). Value and gradient constraints were sampled from each implicit field along vertical drill holes, to be used as synthetic data (Fig. d–j) when fitting (training) an interpolated neural field based model. Every individual field in the model influences every field that came before it with displacements, overprinting, or a combination of both actions (Fig. c).

A neural field was parameterised (with different hyperparameters for various losses) for each set of constraints. The gradient loss term for all the implicit fields was given a higher weight, to give more importance to the shape of the reconstructed model. For the depositional events (s0 and s2), the value loss was given a very low weight (as the values are arbitrarily defined and may impede convergence), and the monotonicity loss enabled used to encourage a bubble-free geometry. For the dyke, as the values determine the aperture of the dyke, the value loss was enabled, along with the thickness loss, to ensure it retains a constant thickness in interpolated regions. Finally, the domain boundaries and faults were fit by prioritising value-loss in combination with a thickness loss that acts to keep a constant implicit field gradient. For this example fault-displacement was assumed to be known, so the principal shortening direction (s^) and slip magnitude were defined a-priori (though the slip magnitude can also be treated as a learnable parameter, as shown in the following section).

After 1000 epochs of training, the interpolated curlew model closely matches the original synthetic one, with a few visible differences (Fig. 7). The implicit values for the two models are very different (0–600 vs. 80–100 for s0), but this is coherent with the disabled value loss function for these two fields and does not necessarily influence the isosurface geometry (given we extract the isosurfaces for the lithologies based on seed positions).

Finally, to demonstrate that curlew can be applied to real (3D) data, we present a model generated from a digital outcrop in Australia. The digital outcrop consists of a faulted stratigraphic sequence intruded by a dyke. The Compass plugin for CloudCompare was used to extract (1) traces along the contacts between the stratigraphic units, dyke margin and fault surface, and (2) orientation measurements (strike and dip) of each of these geological structures.

Since these traces included a large number of points (due to the high resolution of the point cloud), they were subsampled to acquire evenly-spaced points along the stratigraphic contacts, the fault surface, and the dyke (Fig. a). While these traces could be directly used as relational constraints, we used the subsampled points as value constraints, with the values being the approximate height of the layer from the base of the digital outcrop. These constraints were then used to constrain three consecutive neural fields: (1) a stratigraphic field, (2) a fault field, and (3) dyke field.

After fitting each neural field separately to generate an initial geometry, we froze the geometries of the fault and dyke fields by fixing the weights of their corresponding neural fields. The fault slip and stratigraphic interpolation were then fit simultaneously, to find an optimal displacement for the fault (i.e. by finding the fault offset that results, after reconstruction, in the smoothest interpolation of the older stratigraphy field).

The resulting model (Fig. b) and solutions to the displacement fields induced by the fault and the dyke were then used to “undeform” the outcrop and retrieve pre-deformation locations for all of the points in the digital outcrop model (Fig. c). Note that while the dyke in the original outcrop is finite, the modelled dyke is infinite. Finite dykes (and faults) are planned to be implemented in future versions of curlew.

The inclusion of Random Fourier Feature (RFF) mapping allows curlew to train neural fields without using the pre-training protocol outlined by for network weight initialisation. Although there are several alternative approaches that could behave similarly e.g., SiREN;, the RFF mapping has the advantage that length-scales used by the interpolator can be carefully controlled, and potentially seeded for specific geological settings (e.g., in folded rocks with known amplitude and axial plane). However, this sensitivity can also be a disadvantage, as the manual selection of appropriate length scales can be quite challenging.

Interestingly, the randomness inherent to RFF mapping means multiple model realisations can be obtained by training models on the same data but with different RFF seeds (Fig. ). While model ensembles can also be derived from conventional neural fields, we suspect that the significant differences in (learned) geometric representation that result when randomly sampling different RFFs enable greater exploration of the potential solution space. The precise meaning of these variations remains a topic of future research, but it could facilitate methods for quantifying the epistemic uncertainty of the interpolated result.

For example, we have generated 25 random realisations of the folded stratigraphy model (see Sect. 4.1) and computed the information entropy of the resulting lithology predictions (Fig. f). This clearly highlights increased uncertainty near fold hinges and away from the constraints, aiding interpretation and potentially facilitating targeted data collection .

A full-variational Bayesian Neural Network BNN; provides a first order uncertainty quantification, as the weights and biases of the neural network are not fixed values and can be approximated as probability distributions. Thus, this more comprehensive representation of variability may help to capture epistemic uncertainty arising from the data. However, BNNs are computationally expensive, even more than the ensembles for highly optimised networks, and require more complex training procedures compared to standard neural networks with RFF encoding. Some studies propose Monte-Carlo dropout or Hamiltonian Monte-Carlo simulations within implicit neural representations as a simpler alternative to BNNs. These approaches are all theoretically compatible with our curlew architecture, and could be explored in future works.

As hinted at previously, the differentiability of the combined network of neural fields can also be leveraged to update the model geometry according to some additional loss that depends on the model as a whole (rather than the individual neural fields). This can potentially be applied to integrate unlabelled auxiliary data through an additional neural network (Fig. b), which learns the relationship between the outputs of the structural model (implicit field value and event ID; cf. Fig. b) and arbitrary measurements (e.g., geochemistry or petrophysics). The primary role of this neural network is not to accurately predict the measured properties (as it cannot account for lateral variability), but rather to encourage the model geometry to converge on a solution that explains as much property variance as possible (i.e. a common earth model). By jointly training this learnable forward model and the geological geometry fields, geometries that are most predictive of the measured properties will be favoured.

We illustrate this approach using a simple model containing a folded sequence, in which three arbitrary property measurements have been made along the boreholes. These are visualised using a ternary colour mapping that gives a false-colour representation of the measurements (Fig. a). An initial model, fit to sparse constraints from these drill holes, does a reasonable job at predicting the field geometry (Fig. b) given the available information. But, when this is coupled with the learnable forward model, the added unlabelled drillhole information (property measurements) result in a solution much closer to the original (Fig. c).

This approach potentially adds a powerful semi-supervised aspect to the modelling process: constraints are used to define an approximate (“supervised”) geometry, which is further refined to find solutions that best explains additional quantitative data or lithological logs, reducing interpretation bias and enabling the rapid integration of new data without explicit interpretation. A similar approach might also be used to integrate geophysical constraints on model geometry, albeit with physical forward models.

The parameterisation of geological models using neural networks opens up several exciting directions for future research and application. The introduction of Random Fourier Features, while powerful, necessitates research into the extraction of appropriate length scales from sparse data, instead of the trial-and-error approach used in the current work. Progressive implicit representations might also help in mitigating random noise generated by including high frequencies, while fitting sparse data better. Using the domain of the model and known sparsity of the constraints, one could also generate a power-spectrum of length scales to seed the model. This would effectively eliminate the need for a length scales hyperparameter. However, as the norms of the spatial wavenumbers follow a Chi distribution, the resulting distribution of wavenumbers includes a heavy tail. Careful optimisation is necessary to ensure that no spectral noise overtakes the interpolation process of the model.

We also note that our method of generating multiple model realisations by changing the Random Fourier Feature encoding has several similarities to the turning bands and spectral methods used to simulate Gaussian random fields , suggesting that a deeper stochastic link to other Gaussian process methods may be possible. The turning bands method uses the covariance matrix to seed the random field generation, which could possibly be adapted for the sparse datasets used as inputs to geological models. However, unlike the turning bands method, our method relies on Bochner's theorem , which states that a shift invariant kernel i.e., the interpolator that the model is trying to approximate, is defined by the Fourier transform of the probability distribution of the frequencies used to construct the kernel. Using a uniform distribution as suggested byfor turning bands simulations signifies that the interpolator will behave as a Sinc- or Bessel-type kernel, and could cause ringing artifacts. A careful examination of alternative distributions for drawing these frequencies is also an avenue for future research.

The flexibility of the loss function used to train neural fields opens the door to additional innovative constraints that better encode geological rules or prior knowledge, e.g., topological relationships of the model outputs . However, incorporating multiple physical and geometric constraints further increases the complexity of our already complex multi-objective optimisation problem. Because different constraints often have entirely different numerical scales and physical units, each additional loss component requires careful hyperparameter tuning to prevent one objective from dominating the network's gradient updates.

To mitigate this in curlew, we implement an optional initial loss normalization strategy in which each individual loss component is divided by its detached initial value at the very start of training. This initial scaling forces all the losses to begin at 1, and can be thought of as an assumption that the losses at initialisation are equally bad. Once the losses are mapped to this common (unitless) scale, users can apply a single, intuitive scaling hyperparameter to explicitly dictate the relative physical or geological importance of one constraint over another. Our tests show that setting the local loss scaling hyperparameters to 1, and the global loss scaling hyperparameters between 0.01 and 0.1 serves as a good starting point for most models.

Alternative approaches, such as Gradient Surgery and other gradient based algorithms , exist to automatically select and adapt hyperparameters during training. However, further work is needed to assess the extent to which these are able to replace manual hyperparameter optimisation in the context of curlew. We have implemented one of these, SoftAdapt , but with mixed results.