Pretraining system and method for seismic data processing using machine learning
View Patent ↗A method for processing waveform data for determining a characteristic inside an object includes receiving raw waveform data collected with waveform sensors over a given object; pretraining a first neural network with the raw waveform data so that structural information within the raw waveform data is extracted; receiving a first task to be performed with regard to the object, based on the raw waveform data; fine-tuning a second neural network, based on (1) pretrained parameters from the first neural network, and (2) the first task; and generating an image of the object using the second neural network with refined parameters.
1 . A method for processing waveform data for determining a characteristic inside an underground geophysical formation, the method comprising:
receiving raw waveform data collected with waveform sensors over the underground geophysical formation;
pretraining a first neural network with the raw waveform data so that structural information within the raw waveform data is extracted and pretrained parameters of the first neural network are generated;
receiving a first seismic data processing task to be performed with regard to the underground geophysical formation, based on the raw waveform data;
fine-tuning a second neural network by using the pretrained parameters from the first neural network, and generating refined parameters in response to the first task; and
generating an image of the underground geophysical formation using the second neural network with the refined parameters to map the raw waveform data for oil and gas exploration.
2 . The method of claim 1 , wherein the first neural network is different from the second neural network.
3 . The method of claim 1 , wherein the second neural network includes the first neural network and at least one more layer.
4 . The method of claim 1 , wherein the first neural network uses a sequence to sequence model, the raw waveform data is seismic data of the subsurface, and the characteristics is a presence of oil or gas.
5 . The method of claim 1 , wherein the first neural network maps the raw waveform data to the raw waveform data, and the traces of the raw waveform data are considered to form a sentence so that an order of the traces in the sentence constitutes the sequential and structural information.
6 . The method of claim 1 , wherein the second neural network uses as input the raw waveform data.
7 . The method of claim 1 , wherein the step of pretraining comprises:
encoding traces forming the raw waveform data with a first layer of a third neural network; and
encoding a position of the traces in the raw waveform data with a first layer of a fourth neural network,
wherein a result of the encoding steps is a trace embedding.
8 . The method of claim 7 , further comprising:
applying an attention layer of the first neural network to the trace embedding;
applying a feed forward layer of the first neural network to a result of the attention layer, which generates an optimized trace embedding; and
reconstructing an original raw trace from the optimized trace embedding.
9 . A computing device for processing waveform data for determining a characteristic of an underground geophysical formation, the computing device comprising:
an input/output interface configured to receive raw waveform data collected with waveform sensors over the underground geophysical formation; and
a processor connected to the input/output interface and configured to,
pretrain a first neural network with the raw waveform data so that sequential structural information within the raw waveform data is extracted and pretrained parameters of the first neural network are generated;
receive a first seismic data processing task to be performed with regard to the underground geophysical formation, based on the raw waveform data;
fine-tune a second neural network by using the pretrained parameters from the first neural network, and generating refined parameters in response to the first task; and
generate an image of the underground geophysical formation using the second neural network with the refined parameters to map the raw waveform data for oil and gas exploration.
10 . The computing device of claim 9 , wherein the first neural network is different from the second neural network.
11 . The computing device of claim 9 , wherein the second neural network includes the first neural network and at least one more layer.
12 . The computing device of claim 9 , wherein the first neural network uses a sequence-to-sequence model, the raw waveform data is seismic data of the subsurface, and the characteristics is a presence of oil or gas.
13 . The computing device of claim 9 , wherein the first neural network maps the raw waveform data to the raw waveform data, and the traces of the raw waveform data are considered to form a sentence so that an order of the traces in the sentence constitutes the structural information.
14 . The computing device of claim 9 , wherein the second neural network uses as input the raw waveform data.
15 . The computing device of claim 9 , wherein the processor is further configured to:
encode traces forming the raw waveform data with a first layer of a third neural network; and
encode a position of the traces in the raw seismic data with a first layer of a fourth neural network,
wherein a result of the encoding steps is a trace embedding.
16 . The computing device of claim 15 , wherein the processor is further configured to:
apply an attention layer of the first neural network to the trace embedding;
apply a feed forward layer of the first neural network to a result of the attention layer, which generates an optimized trace embedding; and
reconstruct an original raw trace from the optimized trace embedding.
17 . A non-transitory computer readable medium including computer executable instructions, wherein the instructions, when executed by a processor, implement a method for processing waveform data for determining a characteristic of an underground geophysical formation, the medium comprising instructions for:
receiving raw waveform data collected with waveform sensors over the underground geophysical formation;
pretraining a first neural network with the raw waveform data so that structural information within the raw waveform data is extracted and pretrained parameters of the first neural network are generated;
receiving a first seismic data processing task to be performed with regard to the underground geophysical formation, based on the raw waveform data;
fine-tuning a second neural network by using the pretrained parameters from the first neural network, and generating refined parameters in response to the first task; and
generating an image of the underground geophysical formation using the second neural network with the refined parameters to map the raw waveform data for oil and gas exploration.
18 . The medium of claim 17 , wherein the second neural network includes the first neural network and at least one more layer.
19 . The medium of claim 17 , wherein the first neural network maps the raw waveform data to the raw waveform data, and the traces of the raw waveform data are considered to form a sentence so that an order of the traces in the sentence constitutes the structural information.
20 . The medium of claim 17 , wherein the step of pretraining comprises:
encoding traces forming the raw waveform data with a first layer of a third neural network;
encoding a position of the traces in the raw waveform data with a first layer of a fourth neural network, wherein a result of the encoding steps is a trace embedding;
applying an attention layer of the first neural network to the trace embedding;
applying a feed forward layer of the first neural network to a result of the attention layer, which generates an optimized trace embedding; and
reconstructing an original raw trace from the optimized trace embedding,
wherein the raw waveform data is seismic data of the subsurface, and the characteristics is a presence of oil or gas.