Parity token for error-localized block decoding
Aspects of the present disclosure involve blockwise token generation with selective error localization and repair. Aspects include generating, by a machine learning model, a structural plan comprising plan tokens that encode constraints expected to be satisfied by an output sequence. Aspects include parsing the plan tokens into a ledger of remaining constraints. Aspects include generating, by the machine learning model, the output sequence comprising a block of multiple candidate tokens. Aspects include computing local constraint signals for the block and comparing the local constraint signals to a corresponding subset of the ledger to produce a syndrome for token verification. Aspects include, based on the syndrome indicating a constraint violation, identifying one or more violating token positions via a token-to-constraint mapping. Aspects include generating, by the machine learning model, one or more replacement candidate tokens for the one or more violating token positions while retaining other candidate tokens in the block.
1 . A system for blockwise token generation with selective error localization and repair, comprising:
one or more processors; and
a non-transitory computer readable medium storing instructions that, when executed by the one or more processors, cause the system to:
generate, by a language processing machine learning model, prior to producing an output sequence, a structural plan comprising plan tokens that encode constraints expected to be satisfied by the output sequence;
parse the plan tokens into a ledger of remaining constraints;
generate, by the language processing machine learning model in a single forward pass, the output sequence comprising a block of multiple candidate tokens;
compute local constraint signals for the block and compare the local constraint signals to a corresponding subset of the ledger to produce a syndrome for token verification;
based on the syndrome indicating a constraint violation, identify one or more violating token positions via a token-to-constraint mapping;
generate, by the language processing machine learning model in an additional single forward pass, one or more replacement candidate tokens for the one or more violating token positions while retaining other candidate tokens in the block;
update the ledger to reflect one or more satisfied constraints; and
repeat block generation using the language processing machine learning model and syndrome-based token verification until the ledger indicates a terminal satisfied state or a termination token is produced.
2 . The system of claim 1 , wherein the updating of the ledger comprises decrementing, flipping, or clearing one or more ledger entries corresponding to the one or more satisfied constraints and storing metadata in association with the one or more ledger entries to support post-generation auditing and error analysis.
3 . The system of claim 1 , wherein the generating of the structural plan comprises generating parity indicators and bounded counters associated with delimiters, quote balance, terminators, and end-of-object flags.
4 . The system of claim 3 , wherein the generating of the structural plan further comprises generating delimiters, quote balance, terminators, and end-of-object flags associated with the parity indicators and the bounded counters.
5 . The system of claim 1 , wherein the parsing of the plan tokens produces a bounded-size ledger comprising bit fields and integer counters that are updated per block as constraints are satisfied.
6 . The system of claim 1 , wherein the generating of the block of the multiple candidate tokens comprises performing set block decoding with an entropy-bounded sampler to select a subset of tokens to reveal jointly based on predicted dependence.
7 . The system of claim 1 , wherein the computing of the local constraint signals comprises:
computing, over the block and adjacent context, parity values and counter deltas; and
comparing the computed parity values and the computed counter deltas to corresponding entries in the ledger to produce the syndrome.
8 . The system of claim 1 , wherein the identifying of the one or more violating token positions comprises applying a predefined token-to-constraint mapping that associates particular constraint fields with token locations or neighborhoods.
9 . The system of claim 1 , wherein the generating of the one or more replacement candidate tokens comprises:
masking only the one or more violating token positions and re-sampling the masked one or more violating token positions while retaining other candidate tokens in the block; and
invoking a grammar-constrained resampling fallback for one or more corresponding violating positions when a violation persists after a capped number of retries.
10 . A method for blockwise token generation with selective error localization and repair, comprising:
generating, by a language processing machine learning model, prior to producing an output sequence, a structural plan comprising plan tokens that encode constraints expected to be satisfied by the output sequence;
parsing the plan tokens into a ledger of remaining constraints;
generating, by the language processing machine learning model in a single forward pass, the output sequence comprising a block of multiple candidate tokens;
computing local constraint signals for the block and comparing the local constraint signals to a corresponding subset of the ledger to produce a syndrome for token verification;
based on the syndrome indicating a constraint violation, identifying one or more violating token positions via a token-to-constraint mapping;
generating, by the language processing machine learning model in an additional single forward pass, one or more replacement candidate tokens for the one or more violating token positions while retaining other candidate tokens in the block;
updating the ledger to reflect one or more satisfied constraints; and
repeating block generation using the language processing machine learning model and syndrome-based token verification until the ledger indicates a terminal satisfied state or a termination token is produced.
11 . The method of claim 10 , wherein the updating of the ledger comprises decrementing, flipping, or clearing one or more ledger entries corresponding to the one or more satisfied constraints and storing metadata in association with the one or more ledger entries to support post-generation auditing and error analysis.
12 . The method of claim 10 , wherein the generating of the structural plan comprises generating parity indicators and bounded counters associated with delimiters, quote balance, terminators, and end-of-object flags.
13 . The method of claim 12 , wherein the generating of the structural plan further comprises generating delimiters, quote balance, terminators, and end-of-object flags associated with the parity indicators and the bounded counters.
14 . The method of claim 10 , wherein the parsing of the plan tokens produces a bounded-size ledger comprising bit fields and integer counters that are updated per block as constraints are satisfied.
15 . The method of claim 10 , wherein the generating of the block of the multiple candidate tokens comprises performing set block decoding with an entropy-bounded sampler to select a subset of tokens to reveal jointly based on predicted dependence.
16 . The method of claim 10 , wherein the computing of the local constraint signals comprises:
computing, over the block and adjacent context, parity values and counter deltas; and
comparing the computed parity values and the computed counter deltas to corresponding entries in the ledger to produce the syndrome.
17 . The method of claim 10 , wherein the identifying of the one or more violating token positions comprises applying a predefined token-to-constraint mapping that associates particular constraint fields with token locations or neighborhoods.
18 . A non-transitory computer readable medium storing instructions that, when executed by one or more processors of a computing system, cause the computing system to:
generate, by a language processing machine learning model, prior to producing an output sequence, a structural plan comprising plan tokens that encode constraints expected to be satisfied by the output sequence;
parse the plan tokens into a ledger of remaining constraints;
generate, by the language processing machine learning model in a single forward pass, the output sequence comprising a block of multiple candidate tokens;
compute local constraint signals for the block and compare the local constraint signals to a corresponding subset of the ledger to produce a syndrome for token verification;
based on the syndrome indicating a constraint violation, identify one or more violating token positions via a token-to-constraint mapping; and
generate, by the language processing machine learning model in an additional single forward pass, one or more replacement candidate tokens for the one or more violating token positions while retaining other candidate tokens in the block;
update the ledger to reflect one or more satisfied constraints; and
repeat block generation using the language processing machine learning model and syndrome-based token verification until the ledger indicates a terminal satisfied state or a termination token is produced.