IP Library › Granted Patent US 12,373,688
Granted Patent B2
US 12,373,688 · App. 17/827,362 · Granted Jul 29, 2025

Granular neural network architecture search over low-level primitives

Inventors: David Richard So (San Francisco, CA); Quoc V. Le (Sunnyvale, CA); Hanxiao Liu (Santa Clara, CA); Wojciech Andrzej Manke (Zürich, CH); Zihang Dai (Pittsburgh, PA); Noam M. Shazeer (Palo Alto, CA)
Assignee: Google LLC
G06N3/08
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,373,688
App. No.
17/827,362
Granted
Jul 29, 2025
Kind
B2
Abstract

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing a machine learning task on a network input to generate a network output. One of the systems includes an attention neural network configured to perform the machine learning task. The attention neural network includes one or more attentions layers that each include a squared ReLU activation layer, a depth-wise convolution layer, or both.

Claims (92)

1. A system for performing a machine learning task on a network input to generate a network output, the system comprising one or more computers and one or more storage devices storing instructions that, when executed by the one or more computers, cause the one or more computers to implement:

an attention neural network configured to perform the machine learning task, the attention neural network comprising one or more layers, each layer comprising an attention sub-layer and a feed-forward sub-layer, the attention sub-layer comprising (i) a query linear layer followed by a first depth-wise convolution layer and (ii) a key linear layer followed by a second depth-wise convolution layer, the attention sub-layer configured to:

receive an input sequence for the layer comprising a respective layer input at each of a plurality of positions; and

generate an attended input sequence at least in part by applying an attention mechanism over the input sequence for the attention sub-layer, the attended input sequence comprising a respective attended layer input at each of the plurality of positions, wherein generating the attended input sequence at least in part by applying the attention mechanism to the input sequence for the attention sub-layer comprises:

applying, by the query linear layer included in the attention sub-layer, a query linear transformation to the layer inputs or data derived from the layer inputs to generate a plurality of query vectors;

applying, by the first depth-wise convolution layer included in the attention sub-layer, a first depth-wise convolution function to the plurality of query vectors that have been generated by applying the query linear transformation to generate a plurality of modified query vectors;

generating a plurality of modified key vectors based at least on applying, by a key linear layer included in the attention sub-layer, a key linear transformation to the layer inputs or data derived from the layer inputs;

generating a plurality of modified value vectors based at least on applying, by a value linear layer included in the attention sub-layer, a value linear transformation to the layer inputs or data derived from the layer inputs; and

applying the attention mechanism over the respective layer inputs in the input sequence using (i) the plurality of modified query vectors that have been generated by applying both the query linear transformation and the first depth-wise convolution function and (ii) the plurality of modified key vectors and (iii) the plurality of modified value vectors, and the feed-forward sub-layer configured to:

receive the attended input sequence; and

generate an output sequence for the feed-forward sub-layer from the attended input sequence, the output sequence comprising a respective layer output at each of the plurality of positions.

2. The system of claim 1 , wherein generating the plurality of modified key vectors comprises:

applying, by the key linear layer included in the attention sub-layer, the key linear transformation to the layer inputs or data derived from the layer inputs to generate a plurality of key vectors;

applying, by a second depth-wise convolution layer included in the attention sub-layer, a second depth-wise convolution function to the plurality of key vectors that have been generated by applying the key linear transformation to generate the plurality of modified key vectors; and wherein applying the attention mechanism comprises:

applying the attention mechanism over the respective layer inputs in the input sequence using the plurality of modified key vectors that have been generated by applying both the key linear transformation and the second depth-wise convolution function.

3. The system of claim 1 , wherein generating the plurality of modified value vectors comprises:

applying, by the value linear layer included in the attention sub-layer, the value linear transformation to the layer inputs or data derived from the layer inputs to generate a plurality of value vectors;

applying, by a third depth-wise convolution layer included in the attention sub-layer, a third depth-wise convolution function to the plurality of value vectors that have been generated by applying the value linear transformation to generate the plurality of modified value vectors; and wherein applying the attention mechanism comprises:

applying the attention mechanism over the respective layer inputs in the input sequence using the plurality of modified value vectors that have been generated by applying both the value linear transformation and the third depth-wise convolution function.

4. The system of claim 3 , wherein the first depth-wise convolution layer, the second depth-wise convolution layer, and the third depth-wise convolution layer each have a respective convolution kernel with a limited size.

5. The system of claim 4 , wherein the limited size comprises a width of 3.

6. The system of claim 1 , wherein each layer of the attention neural network further comprises a normalization layer that applies layer normalization to the second transformed outputs generated by the feed-forward sub-layer.

7. The system of claim 6 , wherein the layer normalization comprises a modified layer normalization.

8. The system of claim 1 , wherein the machine learning task comprises:

an audio processing task, wherein the network input is a sequence representing a spoken utterance, and the network output is a piece of text that is a transcript of the utterance, an indication whether a particular word or phrase was spoken in the utterance, or an identification of a natural language in which the utterance was spoken;

a text to speech task, wherein the network input is text in a natural language or features of text in a natural language, and the network output is a spectrogram, a waveform, or other data defining audio of the text being spoken in the natural language;

a computer vision task, wherein the network input is an image or a point cloud, and the network output is a computer vision output for the image or point cloud, and optionally wherein the computer vision output is a classification output including a score for each of a plurality of categories, each score representing the likelihood that the image or point cloud includes an object belonging to the category;

an image generation task, wherein the network input is a conditioning input, and the network output is a sequence of intensity value inputs for the pixels of an image;

a neural machine translation task, wherein the network input is a sequence of text in one language, and the network output is a translation of the sequence of text into another language;

an agent control task, wherein the network input is a sequence of observations or other data characterizing states of an environment, and the network output defines an action to be performed by the agent in response to a most recent data in the sequence;

a health prediction task, wherein the network input is a sequence derived from electronic health record data for a patient, and the network output is a predicted diagnosis for the patient; or

a genomics task, wherein the network input is a sequence representing a fragment of a DNA sequence or other molecule sequence, and the network output is either an embedding of the fragment for use in a downstream task, or an output for the downstream task.

9. The system of claim 1 , wherein generating the output sequence for the feed-forward sub-layer comprises, for each of the plurality of positions:

generating a first transformed input, comprising applying a first linear transformation to the attended layer input at the position;

generating a squared ReLU activated input at the position by

applying a ReLU activation function to the first transformed input to generate a ReLU activated input, and

applying a square function to the ReLU activated input to generate the squared ReLU activated input;

generating a second transformed input, comprising applying a second linear transformation to the squared ReLU activated input at the position; and

generating the layer output at the position from the second transformed input.

10. One or more computer-readable storage media storing instructions that when executed by one or more computers cause the one or more computers to implement an attention neural network configured to perform a machine learning task on a network input to generate a network output, the attention neural network comprising one or more layers, each layer comprising an attention sub-layer and a feed-forward sub-layer, the attention sub-layer comprising (i) a query linear layer followed by a first depth-wise convolution layer and (ii) a key linear layer followed by a second depth-wise convolution layer, the attention sub-layer configured to:

receive an input sequence for the layer comprising a respective layer input at each of a plurality of positions; and

generate an attended input sequence at least in part by applying an attention mechanism over the input sequence for the attention sub-layer, the attended input sequence comprising a respective attended layer input at each of the plurality of positions, wherein generating the attended input sequence at least in part by applying the attention mechanism to the input sequence for the attention sub-layer comprises:

applying, by the query linear layer included in the attention sub-layer, a query linear transformation to the layer inputs or data derived from the layer inputs to generate a plurality of query vectors;

applying, by the first depth-wise convolution layer included in the attention sub-layer, a first depth-wise convolution function to the plurality of query vectors that have been generated by applying the query linear transformation to generate a plurality of modified query vectors;

generating a plurality of modified key vectors based at least on applying, by a key linear layer included in the attention sub-layer, a key linear transformation to the layer inputs or data derived from the layer inputs;

generating a plurality of modified value vectors based at least on applying, by a value linear layer included in the attention sub-layer, a value linear transformation to the layer inputs or data derived from the layer inputs; and

applying the attention mechanism over the respective layer inputs in the input sequence using (i) the plurality of modified query vectors that have been generated by applying both the query linear transformation and the first depth-wise convolution function and (ii) the plurality of modified key vectors and (iii) the plurality of modified value vectors, and the feed-forward sub-layer configured to:

receive the attended input sequence; and

generate an output sequence for the feed-forward sub-layer from the attended input sequence, the output sequence comprising a respective layer output at each of the plurality of positions.

11. The computer-readable storage media of claim 10 , wherein generating the plurality of modified key vectors comprises:

applying, by the key linear layer included in the attention sub-layer, the key linear transformation to the layer inputs or data derived from the layer inputs to generate a plurality of key vectors;

applying, by a second depth-wise convolution layer included in the attention sub-layer, a second depth-wise convolution function to the plurality of key vectors that have been generated by applying the key linear transformation to generate the plurality of modified key vectors; and wherein applying the attention mechanism comprises:

applying the attention mechanism over the respective layer inputs in the input sequence using the plurality of modified key vectors that have been generated by applying both the key linear transformation and the second depth-wise convolution function.

12. The computer-readable storage media of claim 10 , wherein generating the plurality of modified value vectors comprises:

applying, by the value linear layer included in the attention sub-layer, the value linear transformation to the layer inputs or data derived from the layer inputs to generate a plurality of value vectors;

applying, by a third depth-wise convolution layer included in the attention sub-layer, a third depth-wise convolution function to the plurality of value vectors that have been generated by applying the value linear transformation to generate the plurality of modified value vectors; and wherein applying the attention mechanism comprises:

applying the attention mechanism over the respective layer inputs in the input sequence using the plurality of modified value vectors that have been generated by applying both the value linear transformation and the third depth-wise convolution function.

13. The computer-readable storage media of claim 10 , wherein generating the output sequence for the feed-forward sub-layer comprises, for each of the plurality of positions:

generating a first transformed input, comprising applying a first linear transformation to the attended layer input at the position;

generating a squared ReLU activated input at the position by

applying a ReLU activation function to the first transformed input to generate a ReLU activated input, and

applying a square function to the ReLU activated input to generate the squared ReLU activated input;

generating a second transformed input, comprising applying a second linear transformation to the squared ReLU activated input at the position; and

generating the layer output at the position from the second transformed input.

14. A computer-implemented method for performing a machine learning task on a network input to generate a network output, wherein the method comprises

receiving, at an attention sub-layer of an attention neural network, an input sequence for the attention sub-layer comprising a respective layer input at each of a plurality of positions, wherein the attention sub-layer comprises (i) a query linear layer followed by a first depth-wise convolution layer (ii) a key linear layer followed by a second depth-wise convolution layer; and

generating an attended input sequence at least in part by applying an attention mechanism over the input sequence for the attention sub-layer, the attended input sequence comprising a respective attended layer input at each of the plurality of positions, wherein generating the attended input sequence at least in part by applying the attention mechanism to the input sequence for the attention sub-layer comprises:

applying, by the query linear layer included in the attention sub-layer, a query linear transformation to the layer inputs or data derived from the layer inputs to generate a plurality of query vectors;

applying, by the first depth-wise convolution layer included in the attention sub-layer, a first depth-wise convolution function to the plurality of query vectors that have been generated by applying the query linear transformation to generate a plurality of modified query vectors;

generating a plurality of modified key vectors based at least on applying, by a key linear layer included in the attention sub-layer, a key linear transformation to the layer inputs or data derived from the layer inputs;

generating a plurality of modified value vectors based at least on applying, by a value linear layer included in the attention sub-layer, a value linear transformation to the layer inputs or data derived from the layer inputs; and

applying the attention mechanism over the respective layer inputs in the input sequence using (i) the plurality of modified query vectors that have been generated by applying both the query linear transformation and the first depth-wise convolution function and (ii) the plurality of modified key vectors and (iii) the plurality of modified value vectors;

receiving, at a feed-forward sub-layer of the attention neural network, the attended input sequence; and

generating an output sequence for the feed-forward sub-layer from the attended input sequence, the output sequence comprising a respective layer output at each of the plurality of positions.

15. The method of claim 14 , wherein generating the plurality of modified key vectors comprises:

applying, by the key linear layer included in the attention sub-layer, the key linear transformation to the layer inputs or data derived from the layer inputs to generate a plurality of key vectors;

applying, by a second depth-wise convolution layer included in the attention sub-layer, a second depth-wise convolution function to the plurality of key vectors that have been generated by applying the key linear transformation to generate the plurality of modified key vectors; and wherein applying the attention mechanism comprises:

applying the attention mechanism over the respective layer inputs in the input sequence using the plurality of modified key vectors that have been generated by applying both the key linear transformation and the second depth-wise convolution function.

16. The method of claim 14 , wherein generating the plurality of modified value vectors comprises:

applying, by the value linear layer included in the attention sub-layer, the value linear transformation to the layer inputs or data derived from the layer inputs to generate a plurality of value vectors;

applying, by a third depth-wise convolution layer included in the attention sub-layer, a third depth-wise convolution function to the plurality of value vectors that have been generated by applying the value linear transformation to generate the plurality of modified value vectors; and wherein applying the attention mechanism comprises:

applying the attention mechanism over the respective layer inputs in the input sequence using the plurality of modified value vectors that have been generated by applying both the value linear transformation and the third depth-wise convolution function.

17. The method of claim 16 , wherein the first depth-wise convolution layer, the second depth-wise convolution layer, and the third depth-wise convolution layer each have a respective convolution kernel with a limited size.

18. The method of claim 17 , wherein the limited size comprises a width of 3.

19. The method of claim 14 , wherein each layer of the attention neural network further comprises a normalization layer that applies a modified layer normalization to the second transformed outputs generated by the feed-forward sub-layer.

20. The method of claim 14 , wherein generating the output sequence for the feed-forward sub-layer comprises, for each of the plurality of positions:

generating a first transformed input, comprising applying a first linear transformation to the attended layer input at the position;

generating a squared ReLU activated input at the position by

applying a ReLU activation function to the first transformed input to generate a ReLU activated input, and

applying a square function to the ReLU activated input to generate the squared ReLU activated input;

generating a second transformed input, comprising applying a second linear transformation to the squared ReLU activated input at the position; and

generating the layer output at the position from the second transformed input.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 13, 2022
From: SO, DAVID RICHARD; LE, QUOC V.; LIU, HANXIAO; MANKE, WOJCIECH ANDRZEJ; DAI, ZIHANG; SHAZEER, NOAM M.
To: GOOGLE LLC
Reel/Frame 060178/0694 →
Continuity (2)
Provisional Application 63194889 · May 28, 2021
Related Publication 20220383119A1 · Dec 1, 2022
References Cited (65)
US 10452978B2 · Shazeer et al. · 2019 [cited by applicant]
US 10997503B2 · Dohan et al. · 2021 [cited by applicant]
US 11144831B2 · Huang et al. · 2021 [cited by applicant]
US 20200320399A1 · Huang et al. · 2020 [cited by applicant]
US 20210150370A1 · McDonnell et al. · 2021 [cited by applicant]
Vaswani et al., Attention is All You Need, 2017 (Year: 2017). [cited by examiner]
Elfadel, On the Stability of Analog ReLU Networks, Dec. 2020 (Year: 2020). [cited by examiner]
Zhu et al., An Empirical Study of Spatial Attention Mechanisms in Deep Networks, 2019 (Year: 2019). [cited by examiner]
Jiang et al., An ConvBERT: Improving BERT with Span-based Dynamic Convolution, 2020, 34th Conference on Neural Information Processing Systems (NerIPS 2020) (Year: 2020). [cited by examiner]
Yang et al., GLoMo: Unsupervisedly Learned Relational Graphs as Transferable Representations, 2018 (Year: 2018). [cited by examiner]
Wu et al., Pay Less Attention With Lightweight And Dynamic Convolutions, 2019 (Year: 2019). [cited by examiner]
Xu et al., Efficient Transformer for Remote Sensing Image Segmentation, 2021 (Year: 2021). [cited by examiner]
Abadi et al., “TensorFlow: A System for Large-Scale Machine Learning,” Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation, Nov. 2016, 21 pages. [cited by applicant]
Adiwardana et al., “Towards a Human-like Open-Domain Chatbot,” arXiv, Feb. 27, 2020, 38 pages. [cited by applicant]
Ba et al., “Layer normalization,” arXiv, Jul. 21, 2016, 14 pages. [cited by applicant]
Baevski et al., “Adaptive Input Representations for Neural Language Modeling,” arXiv, Feb. 22, 2019, 13 pages. [cited by applicant]
Bender et al., “Can Weight Sharing Outperform Random Architecture Search? An Investigation With TuNAS,” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2020, pp. 14323-1433… [cited by applicant]
Brown et al., “Language Models are Few-Shot Learners,” Advances in Neural Information Processing Systems, 2020, 25 pages. [cited by applicant]
Cai et al., “ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware,” arXiv, Feb. 23, 2019, 13 pages. [cited by applicant]
Chelba et al., “One billion word benchmark for measuring progress in statistical language modeling,” Interspeech, Sep. 2014, pp. 2635-2639. [cited by applicant]
Dauphin et al., “Language Modeling with Gated Convolutional Networks,” Proceedings of the 34th International Conference on Machine Learning, 2017, 9 pages. [cited by applicant]
Devlin et al., “BERT: Pre-training of deep bidirectional transformers for language understanding,” arXiv, May 24, 2019, 16 pages. [cited by applicant]
Elsken et al., “Efficient Multi-objective Neural Architecture Search via Lamarckian Evolution,” arXiv, Feb. 26, 2019, 23 pages. [cited by applicant]
Elsken et al., “Neural architecture search: A survey,” Journal of Machine Learning Research, Mar. 2019, 20(55):1-21. [cited by applicant]
Fedus et al., “Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity,” arXiv, Jan. 11, 2021, 31 pages. [cited by applicant]
Gao et al., “Making Pre-trained Language Models Better Few-shot Learners,” arXiv, Dec. 31, 2020, 15 pages. [cited by applicant]
Gulati et al., “Conformer: Convolution-augmented Transformer for Speech Recognition,” Interspeech, Oct. 2020, 5 pages. [cited by applicant]
Hendrycks et al., “Bridging Nonlinearities and Stochastic Regularizers with Gaussian Error Linear Units,” arXiv, Nov. 4, 2016, 10 pages. [cited by applicant]
Kaplan et al., “Scaling laws for neural language models,” arXiv, Jan. 23, 2020, 30 pages. [cited by applicant]
Karnin et al., “Almost Optimal Exploration in Multi-Armed Bandits,” Proceedings of the 30th International Conference on Machine Learning, 2013, 9 pages. [cited by applicant]
Krotov et al., “Dense Associative Memory for Pattern Recognition,” Advances in Neural Information Processing Systems, 2016, 9 pages. [cited by applicant]
Li et al., “Hyperband: A novel bandit-based approach to hyperparameter optimization,” Journal of Machine Leaming Research, Apr. 2018, 18(185):1-52. [cited by applicant]
Li et al., “Random Search and Reproducibility for Neural Architecture Search,” Proceedings of The 35th Uncertainty in Artificial Intelligence Conference, 2020, 11 pages. [cited by applicant]
Liu et al., “DARTS: Differentiable architecture search,” arXiv, Apr. 23, 2019, 13 pages. [cited by applicant]
Liu et al., “RoBERTa: A Robustly Optimized BERT Pretraining Approach,” arXiv, Jul. 26, 2019, 13 pages. [cited by applicant]
Narang et al., “Do Transformer Modifications Transfer Across Implementations and Applications?,” arXiv, Sep. 10, 2021, 16 pages. [cited by applicant]
Patterson et al., “Carbon Emissions and Large Neural Network Training,” arXiv, Apr. 23, 2021, 22 pages. [cited by applicant]
Pham et al., “Efficient neural architecture search via parameter sharing,” arXiv, Feb. 12, 2018, 11 pages. [cited by applicant]
Radford et al., “Language models are unsupervised multitask learners,” Technical Report, OpenAI, 2019, 24 pages. [cited by applicant]
Raffel et al., “Exploring the limits of transfer learning with a unified text-to-text transformer,” arXiv, Jul. 28, 2020, 67 pages. [cited by applicant]
Ramachandran et al., “Searching for Activation Functions,” arXiv, Oct. 27, 2017, 13 pages. [cited by applicant]
Real et al., “AutoML-Zero: Evolving Machine Learning Algorithms From Scratch, ” Proceedings of the 37th International Conference on Machine Learning, 2020, 13 pages. [cited by applicant]
Real et al., “Large-Scale Evolution of Image Classifiers, ”Proceedings of the 34th International Conference on Machine Learning, 2017, 10 pages. [cited by applicant]
Real et al., “Regularized Evolution for Image Classifier Architecture Search,” Proceedings of the AAAI Conference on Artificial Intelligence, Jul. 17, 2019, pp. 4780-4789. [cited by applicant]
Schick et al., “It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners,” arXiv, Apr. 12, 2021, 14 pages. [cited by applicant]
Schmidhuber, “Evolutionary Principles in Self-Referential Learning. On Learning now to Learn: The Meta-Meta-Meta . . . —Hook,” Diploma thesis, Technische Universitat Munchen, Germany, May 14, 1987, 63 pages. [cited by applicant]
Shazeer et al., “Mesh-TensorFlow: Deep Learning for Supercomputers,” Advances in Neural Information Processing Systems, 2018, 10 pages. [cited by applicant]
Shazeer, “Glu variants improve transformer,” arXiv, Feb. 12, 2020, 5 pages. [cited by applicant]
So et al., “The Evolved Transformer,” Proceedings of the 36th International Conference on Machine Learning, 2019, 10 pages. [cited by applicant]
Stanley et al., “Designing neural networks through neuroevolution,” Nature Machine Intelligence, Jan. 7, 2019, 1:24-35. [cited by applicant]
Sutskever et al., “Sequence to Sequence Learning with Neural Networks,” Advances in Neural Information Processing Systems, 2014, 9 pages. [cited by applicant]
Tan et al., “EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks,” Proceedings of the 36th International Conference on Machine Learning, 2019, 10 pages. [cited by applicant]
Tan et al., “MnasNet: Platform-Aware Neural Architecture Search for Mobile, ” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun. 2019, pp. 2820-2828. [cited by applicant]
Tay et al., “Synthesizer: Rethinking Self-Attention in Transformer Models,” arXiv, Oct. 6, 2020, 13 pages. [cited by applicant]
Vaswani et al., “Attention is all you need,” arXiv, Dec. 6, 2017, 15 pages. [cited by applicant]
Vaswani et al., “Tensor2tensor for neural machine translation,” arXiv, Mar. 16, 2018, 9 pages. [cited by applicant]
Wang et al., “Entailment as Few-Shot Learner,” arXiv, Apr. 29, 2021, 15 pages. [cited by applicant]
Wu et al., “CvT: Introducing Convolutions to Vision Transformers,” arXiv, Mar. 29, 2021, 10 pages. [cited by applicant]
Xiong et al., “On layer normalization in the transformer architecture,” arXiv, Jun. 29, 2020, 17 pages. [cited by applicant]
Yang et al., “XLNet: Generalized Autoregressive Pretraining for Language Understanding,” Advances in Neural Information Processing Systems, 2019, 11 pages. [cited by applicant]
Yao, “Evolving artificial neural networks,” Proceedings of the IEEE, Sep. 1999, 87(9): 1423-1447. [cited by applicant]
Yu et al., “Evaluating The Search Phase of Neural Architecture Search,” ICLR, 2020, 16 pages. [cited by applicant]
Yu et al., “QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension,” ICLR, Feb. 15, 2018, 16 pages. [cited by applicant]
Zhang et al., “Root Mean Square Layer Normalization,” Advances in Neural Information Processing Systems, 2019, 12 pages. [cited by applicant]
International Search Report and Written Opinion in International Appln. No. PCT/US2022/031468, dated Oct. 14, 2022, 24 pages. [cited by applicant]
Cited By (1)
US 12,586,393