DEEP LEARNING USING LARGE CODEWORD MODEL WITH HOMOMORPHICALLY COMPRESSED DATA
A system and method for deep learning using a large codeword model with homomorphically compressed and dyadically encrypted data is disclosed. The system preprocesses input data, applies homomorphic-dyadic compression and encryption, tokenizes the compressed data into sourceblocks, and assigns codewords using a codebook. These codewords are processed through a machine learning core, which can be either a conventional transformer-based architecture or a latent transformer core utilizing a variational autoencoder. The system enables secure operations on encrypted data, preserving privacy while allowing complex computations. The processed output is decrypted, decompressed, and translated to match the input modality. A neural upsampler may further enhance the output. The machine learning core is continuously trained using the processed data and additional training data, improving performance over time.
1 . A computer system comprising:
a hardware memory, wherein the computer system is configured to execute software instructions stored on nontransitory machine-readable storage media that:
compress a plurality of input data sets by:
analyzing the input data sets to determine statistical properties and frequency distributions;
creating one or more transformation matrices based on the determined properties, wherein the transformation matrices are configured to preserve homomorphic properties of the input data sets;
transforming the input data sets using the transformation matrices to generate modified probability distributions;
generating (i) main data streams comprising the transformed data and (ii) secondary data streams comprising transformation information; and
compressing the main data streams while maintaining the homomorphic properties;
tokenize the compressed main data streams into a plurality of sourceblocks;
assign the plurality of sourceblocks a plurality of codewords, where each sourceblock is mapped to a particular codeword through a codebook;
process the plurality of codewords through a machine learning core to generate a codeword response;
translate the codeword response into a translated response which matches the modality of the inputs; and
decompress and decrypt the translated response.
2 . The system of claim 1 , wherein the machine learning core is a conventional transformer-based architecture comprising:
an embedding layer;
a positional encoding layer; and
a series of transformer layers.
3 . The system of claim 2 , further comprising a syntactic splitting component that splits the codewords into smaller units before processing through the conventional transformer-based architecture.
4 . The system of claim 1 , wherein the machine learning core is a latent transformer core comprising:
a variational autoencoder with an encoder and a decoder; and
a transformer that processes latent space vectors, wherein the transformer does not include an embedding layer and a positional encoding layer.
5 . The system of claim 4 , wherein processing the plurality of codewords through the machine learning core comprises:
generating a plurality of latent space vectors by processing the plurality of codewords through the variational autoencoder's encoder;
learning relationships between the plurality of latent space vectors by processing them through the transformer;
using the learned relationships to generate a plurality of output latent space vectors; and
generating the codeword response by passing the output latent space vectors through the variational autoencoder's decoder.
6 . The system of claim 5 , further comprising a syntactic splitting component that splits the latent space vectors into smaller units before processing through the transformer.
7 . The system of claim 1 , wherein compressing and encrypting the input data sets further comprises:
combining the compressed main data streams and the secondary data streams into output streams; and
implementing security measures to protect the output streams.
8 . The system of claim 7 , wherein the security measures comprise providing cryptographically secure random numbers for use in data transformation and implementing protections against side-channel attacks.
9 . The system of claim 1 , wherein transforming the input data into modified distributions comprises transforming the input data into dyadic distributions.
10 . The system of claim 1 , further comprising a neural upsampler that processes the codeword response to generate a reconstructed output containing more information than the translated response.
11 . A method for deep learning using a large codeword model with homomorphically compressed dyadically encrypted data, comprising the steps of:
compressing a plurality of input data sets by:
analyzing the input data sets to determine statistical properties and frequency distributions;
creating one or more transformation matrices based on the determined properties, wherein the transformation matrices are configured to preserve homomorphic properties of the input data sets;
transforming the input data sets using the transformation matrices to generate modified probability distributions;
generating (i) main data streams comprising the transformed data and (ii) secondary data streams comprising transformation information; and
compressing the main data streams while maintaining the homomorphic properties;
tokenizing the compressed main data streams into a plurality of sourceblocks;
assigning the plurality of sourceblocks a plurality of codewords, where each sourceblock is mapped to a particular codeword through a codebook;
processing the plurality of codewords through a machine learning core to generate a codeword response;
translating the codeword response into a translated response which matches the modality of the inputs; and
decompressing and decrypt the translated response.
12 . The method of claim 11 , wherein the machine learning core is a conventional transformer-based architecture comprising:
an embedding layer;
a positional encoding layer; and
a series of transformer layers.
13 . The method of claim 12 , further comprising splitting the codewords into smaller units before processing through the conventional transformer-based architecture.
14 . The method of claim 11 , wherein the machine learning core is a latent transformer core comprising:
a variational autoencoder with an encoder and a decoder; and
a transformer that processes latent space vectors, wherein the transformer does not include an embedding layer and a positional encoding layer.
15 . The method of claim 14 , wherein processing the plurality of codewords through the machine learning core comprises:
generating a plurality of latent space vectors by processing the plurality of codewords through the variational autoencoder's encoder;
learning relationships between the plurality of latent space vectors by processing them through the transformer;
using the learned relationships to generate a plurality of output latent space vectors; and
generating the codeword response by passing the output latent space vectors through the variational autoencoder's decoder.
16 . The method of claim 15 , further comprising splitting the latent space vectors into smaller units before processing through the transformer.
17 . The method of claim 11 , wherein compressing and encrypting the input data sets further comprises:
combining the compressed main data streams and the secondary data streams into output streams; and
implementing security measures to protect the output streams.
18 . The method of claim 17 , wherein the security measures comprise providing cryptographically secure random numbers for use in data transformation and implementing protections against side-channel attacks.
19 . The method of claim 11 , wherein transforming the input data into modified distributions comprises transforming the input data into dyadic distributions.
20 . The method of claim 11 , further comprising processing the codeword response through a neural upsampler to generate a reconstructed output containing more information than the translated response.