IP Library Granted Patent US 8,086,028
Granted Patent B2
US 8,086,028 · App. 12/648,933 · Granted Dec 27, 2011

Text-to-scene conversion

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 8,086,028
App. No.
12/648,933
Granted
Dec 27, 2011
Kind
B2
Abstract

The invention relates to a method of converting a set of words into a three-dimensional scene description, which may then be rendered into three-dimensional images. The invention may generate arbitrary scenes in response to a substantially unlimited range of input words. Scenes may be generated by combining objects, poses, facial expressions, environments, etc., so that they represent the input set of words. Poses may have generic elements so that referenced objects may be replaced by those mentioned in the input set of words. Likewise, a character may be dressed according to its role in the set of words. Various constraints for object positioning may be declared. The environment, including but not limited to place, time of day, and time of year, may be inferred from the input set of words.

Claims (49)

1. A method of generating a set of tuple-term pairs from a corpus of text, comprising:

compiling via a processor concordance lines associated with terms in the corpus;

identifying a set of verb-argument tuples and associated terms from the concordance lines;

selecting from the set each verb-argument tuple and associated term having a computed numerical quantification of strength of association greater than a threshold, wherein the selected verb-argument tuples and associated terms represent most likely actions associated with the terms; and

storing in a database each tuple-term pair in the set of tuple-term pairs that matches a verb in a sentence input to a text-to-scene conversion system that constructs an arbitrary three-dimensional scene from input text without using previously stored images.

2. The method of claim 1 , wherein each term describes an aspect of the environment of a scene.

3. The method of claim 2 , wherein the aspect of the environment of the scene denotes one of a name of: a location, an occupied space, an unoccupied space, a room, a time of day, and a season.

4. The method of claim 1 , wherein the verb-argument tuples are one of a verb-object tuple or a verb-preposition-object tuple.

5. The method of claim 1 , wherein the numerical quantification of the strength of association is computing according to steps comprising:

computing a ratio of a likelihood of a first and a second hypothesis, wherein:

the first hypothesis indicates that a probability of the verb-argument tuple occurring given the associated term is indistinguishable from a probability of the verb-argument tuple occurring given something other than the associated term, and

the second hypothesis indicates that the probability of the verb-argument tuple occurring given the associated term is distinguishable from the probability of the verb-argument tuple occurring given something other than the associated term.

6. The method of claim 1 , further comprising deleting duplicate concordance lines.

7. The method of claim 1 , further comprising filtering the set of concordance lines to remove collocations.

8. The method of claim 1 , further comprising computing a lemma of each verb in the set of tuple-term pairs.

9. The method of claim 1 , wherein verbs are in one of a base form and an inflected form.

10. The method of claim 1 , further comprising depicting an action described by the verb in the tuple-term pair in a context of an environment denoted by the term of the tuple-term pair.

11. An system for generating a set of tuple-term pairs from a corpus of text, the system comprising:

a processor;

a first module controlling the processor to compile via a processor concordance lines associated with terms in the corpus;

a second module controlling the processor to identify a set of verb-argument tuples and associated terms from the concordance lines;

a third module controlling the processor to select from the set each verb-argument tuple and associated term having a computed numerical quantification of strength of association greater than a threshold, wherein the selected verb-argument tuples and associated terms represent most likely actions associated with the terms; and

a fourth module controlling the processor to store in a database each tuple-term pair in the set of tuple-term pairs that matches a verb in a sentence input to a text-to-scene conversion system that constructs an arbitrary three-dimensional scene from input text without using previously stored images.

12. The system of claim 11 , wherein the numerical quantification of the strength of association is computing according to steps comprising:

computing a ratio of a likelihood of a first and a second hypothesis, wherein:

the first hypothesis indicates that a probability of the verb-argument tuple occurring given the associated term is indistinguishable from a probability of the verb-argument tuple occurring given something other than the associated term, and

the second hypothesis indicates that the probability of the verb-argument tuple occurring given the associated term is distinguishable from the probability of the verb-argument tuple occurring given something other than the associated term.

13. A non-transitory computer-readable medium storing instructions which, when executed by a computing device, cause the computing device to generate a set of tuple-term pairs from a corpus of text, the instructions comprising:

compiling via a processor concordance lines associated with terms in the corpus;

identifying a set of verb-argument tuples and associated terms from the concordance lines;

selecting from the set each verb-argument tuple and associated term having a computed numerical quantification of strength of association greater than a threshold, wherein the selected verb-argument tuples and associated terms represent most likely actions associated with the terms; and

storing in a database each tuple-term pair in the set of tuple-term pairs that matches a verb in a sentence input to a text-to-scene conversion system that constructs an arbitrary three-dimensional scene from input text without using previously stored images.

14. The non-transitory computer-readable medium of claim 13 , wherein the numerical quantification of the strength of association is computed according to steps comprising:

computing a ratio of likelihood of a first and a second hypothesis, wherein:

the first hypothesis indicates that a probability of the verb-argument tuple occurring given the associated term is indistinguishable from a probability of the verb-argument tuple occurring given something other than the associated term, and

the second hypothesis indicates that the probability of the verb-argument tuple occurring given the associated term is distinguishable from the probability of the verb-argument tuple occurring given something other than the associated term.

15. A method of inferring an environment from a sentence input to a text-to-scene conversion system, the method comprising:

constructing, via a processor, an arbitrary three-dimensional scene from a received sentence without using previously stored images, the sentence having a verb-argument tuple and a subject of an action included in the verb-argument tuple;

identifying an equivalent verb-argument tuple in a database of verb-argument tuples and environmental terms associated with the verb-argument tuples; and

placing the subject in a rendered environment via the text-to-scene conversion system according to the environmental term associated with the equivalent verb-argument tuple.

16. The method of claim 15 , wherein the verb-argument tuple is one of a verb-object tuple or a verb-preposition-object tuple.

17. The method of claim 15 , wherein the equivalent verb-argument tuple includes one of a lemmatized form of a verb in the verb-argument tuple, a base form of the verb in the verb-argument tuple, and an inflected form of the verb in the verb-argument tuple.

18. The method of claim 15 , wherein the environmental term includes one of a name of: a location, an occupied space, an unoccupied space, a room, a time of day, and a season.

19. A text-to-scene conversion system that infers an environment from a received sentence input, the system comprising:

a processor;

constructing, without using previously stored images, an arbitrary three-dimensional scene from a received sentence input, the sentence having a verb-argument tuple and a subject of an action included in the verb-argument tuple;

identifying an equivalent verb-argument tuple in a database of verb-argument tuples and environmental terms associated with the verb-argument tuples; and

placing the subject in a rendered environment via the text-to-scene conversion system according to the environmental term associated with the equivalent verb-argument tuple.

20. The system of claim 19 , wherein the environmental term includes one of a name of: a location, an occupied space, an unoccupied space, a room, a time of day, and a season.

Assignments (7)
CORRECTIVE ASSIGNMENT TO CORRECT THE REMOVE PATENT NUMBERS 10342096;10671117; 10716375; 10716376;10795407;10795408; AND 10827591 PREVIOUSLY RECORDED AT REEL: 58314 FRAME: 657. ASSIGNOR(S) HEREBY CONFIRMS THE ASSIGNMENT. Recorded Feb 29, 2024
From: RAKUTEN, INC.
To: RAKUTEN GROUP, INC.
Reel/Frame 068066/0103 →
CHANGE OF NAME Recorded Dec 6, 2021
From: RAKUTEN, INC.
To: RAKUTEN GROUP, INC.
Reel/Frame 058314/0657 →
CHANGE OF ADDRESS Recorded Dec 17, 2015
From: RAKUTEN, INC.
To: RAKUTEN, INC.
Reel/Frame 037751/0006 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Oct 26, 2012
From: AT&T INTELLECTUAL PROPERTY II, L.P.
To: RAKUTEN, INC.
Reel/Frame 029195/0519 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 14, 2012
From: AT&T PROPERTIES, LLC
To: AT&T INTELLECTUAL PROPERTY II, L.P.
Reel/Frame 028378/0961 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 13, 2012
From: AT&T CORP.
To: AT&T PROPERTIES, LLC
Reel/Frame 028369/0046 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 11, 2012
From: COYNE, ROBERT E.; SPROAT, RICHARD W.
To: AT&T CORP.
Reel/Frame 028353/0036 →