IP Library › Granted Patent US 12711368

Granted Patent B1

US 12711368 · App. 16/442,375 · Granted Aug 18, 2026

Neural network system for object identification

Inventors: Zheng Tang (Seattle, WA); Stan Birchfield (Sammamish, WA); William Heyward Hodge (Seattle, WA); FNU Ratnesh Kumar (Campbell, CA); Milind Naphade (Cupertino, CA); Jonathan Tremblay (Redmond, WA); Shuo Wang (Santa Clara, CA); Xiaodong Yang (Fremont, CA)

Assignee: NVIDIA Corporation

G06N3/08 G06F18/24 G06V20/54 G06V20/64

View Patent ↗

Loading inventors, assignments & file history…

Monitor This Case

Get email alerts when status or documents change.

Order Certified Copies

Most orders are placed with the USPTO same day — all within 24 business hours.

Order via The Patent Place →

Pre-filled with this patent's details

Quick Facts

Patent No.: US 12711368
App. No.: 16/442,375
Granted: Aug 18, 2026
Kind: B1

External Links

USPTO Patent Center↗ Full Patent PDF↗

Abstract

Apparatuses, systems, and techniques to identify objects in an image. In at least one embodiment, one or more neural networks identify the objects using pose information of the objects and visible properties of the objects.

Claims (68)

1 . A system comprising:

one or more processors to use one or more neural networks to identify one or more features of one or more objects within one or more first images, and to indicate whether the one or more objects appear within one or more second images, based, at least in part, on pose of the one or more objects and the one or more identified features, wherein the one or more neural networks comprise:

a first neural network to receive as input the one or more first images of the one or more objects and to output, based on the input of the one or more first images, the pose of the one or more objects and the one or more features of the one or more objects; and

a second neural network to receive as inputs the one or more first images of the one or more objects and the pose of the one or more objects and the one or more features of the one or more objects, and to identify the one or more objects based on the inputs of the second neural network.

2 . The system of claim 1 , wherein the pose of the one or more objects comprises at least one of a location of a portion of the one or more objects, or a likelihood of the location.

3 . The system of claim 1 , wherein the one or more identified features comprises at least one of a color of the one or more objects, or a type of the one or more objects.

4 . The system of claim 1 , wherein the one or more neural networks further comprise:

a feature learning network to receive the one or more first images of the one or more objects and a first portion of the pose of the one or more objects as feature learning network inputs and to output, based on the feature learning network inputs, a vector of image features; and

a classifier to receive the vector of image features and a second portion of the pose of the one or more objects as classifier inputs and to output, based on the classifier inputs, an identification of the one or more objects.

5 . The system of claim 4 , wherein the classifier is a first classifier, and wherein the one or more neural networks further comprise:

a second classifier to receive the vector of image features and the second portion of the pose of the one or more objects as second classifier inputs and to output, based on the second classifier inputs, a color of the one or more objects; and

a third classifier to receive the vector of image features and the second portion of the pose of the one or more objects as third classifier inputs and to output, based on the third classifier inputs, a type of the one or more objects.

6 . The system of claim 1 , wherein the one or more objects comprise a vehicle.

7 . The system of claim 6 , the one or more processors further to compare an identified vehicle to a previously identified vehicle to determine a comparison therebetween, and to store the comparison on a server.

8 . One or more processors, comprising:

circuitry to use one or more neural networks to identify one or more features of one or more objects within one or more first images, and to indicate whether the one or more objects appear within one or more second images, based, at least in part, on pose of the one or more objects and the one or more identified features, wherein the circuitry is to identify the one or more objects using:

a first neural network to receive as input of the one or more first images of the one or more objects and to output, based on the input of the one or more first images, the pose of the one or more objects and the one or more features of the one or more objects; and

a second neural network to receive as inputs the one or more first images of the one or more objects, the one or more features, and the pose of the one or more objects, and to identify the one or more objects based on the inputs of the second neural network.

9 . The one or more processors of claim 8 , wherein the pose of the one or more objects comprises at least one of a location of a portion of the one or more objects, or a likelihood of the location.

10 . The one or more processors of claim 8 , wherein the one or more identified features comprises at least one of a color of the one or more objects, or a type of the one or more objects.

11 . The one or more processors of claim 8 , wherein the one or more neural networks further comprise:

12 . The one or more processors of claim 11 , wherein the classifier is a first classifier, and wherein the one or more neural networks further comprise:

13 . The one or more processors of claim 8 , wherein the one or more objects are one or more vehicles.

14 . The one or more processors of claim 13 , the circuitry is further to compare an identified vehicle to a previously identified vehicle to determine a comparison therebetween, and to store the comparison on a server.

15 . A system comprising:

one or more circuits to train one or more neural networks to identify one or more features of one or more objects within one or more first images, and to indicate whether the one or more objects appear within one or more second images, based, at least in part, on pose of the one or more objects and the one or more identified features, wherein the training comprises:

identifying colors of the one or more objects based, at least in part, on the one or more identified features of the one or more objects; and

identify the one or more objects according to a loss function of the one or more objects and a loss function of the identified colors.

16 . The system of claim 15 , wherein the loss function of the one or more objects comprises a hard-mining triplet loss of the one or more objects and a cross-entropy loss of the one or more objects, and the loss function of the identified colors comprises a cross-entropy loss of the identified colors.

17 . The system of claim 15 , wherein:

the one or more objects includes visible properties; and

training the one or more neural networks includes identifying types of the one or more objects based, at least in part, on the pose of the one or more objects.

18 . The system of claim 17 , wherein the training further comprises training the one or more neural networks to identify the one or more objects according to a loss function of the one or more objects and a loss function of the identified types.

19 . The system of claim 18 , wherein the loss function of the one or more objects comprises a hard-mining triplet loss of the one or more objects and a cross-entropy loss of the one or more objects, and the loss function of the identified types comprises a cross-entropy loss of the identified types.

20 . One or more processors, comprising:

circuitry to generate parameters corresponding to one or more neural networks to identify one or more features of one or more objects within one or more first images, and to indicate whether the one or more objects appear within one or more second images based, at least in part, on pose of the one or more objects and the one or more identified features, wherein the generating further comprises generating the parameters according to a loss function of the one or more objects and a loss function of the colors, the loss function of the one or more objects comprises a hard-mining triplet loss of the one or more objects and a cross-entropy loss of the one or more objects, and the loss function of the colors comprises a cross-entropy loss of the colors.

21 . The one or more processors, of claim 20 , wherein:

the one or more neural networks indicate whether one or more objects within one or more first images appear within one or more second images is based on the pose and color of the one or more objects.

22 . The one or more processors, of claim 20 , wherein:

the pose comprises one or more visible properties.

23 . The one or more processors, of claim 22 , wherein the generating further comprises generating the parameters according to a loss function of the one or more objects and a loss function of types of the one or more objects.

24 . The one or more processors, of claim 23 , wherein the loss function of the one or more objects comprises a hard-mining triplet loss of the one or more objects and a cross-entropy loss of the one or more objects, and the loss function of the types comprises a cross-entropy loss of the types.

25 . A system comprising:

one or more circuits to use one or more neural networks to identify one or more features of one or more objects within one or more first images, and to indicate whether the one or more objects appear within one or more second images based, at least in part, on pose of the one or more objects and the one or more identified features, wherein the one or more neural networks further comprise:

26 . The system of claim 25 , wherein the one or more objects are detected by one or more motion activated cameras.

27 . The system of claim 25 , wherein the one or more objects are detected by one or more video cameras.

28 . The system of claim 25 , wherein the one or more objects are one or more vehicles.

29 . The system of claim 28 , the one or more circuits further to compare an identified one of the one or more vehicles to a previously identified one of the one or more vehicles to determine one or more comparisons therebetween.

30 . The system of claim 29 , the one or more circuits further to transmit the one or more comparisons for storage at a server.

31 . A method, comprising: using one or more neural networks to identify one or more features of one or more objects within one or more first images, and to indicate whether the one or more objects appear within one or more second images based, at least in part, on pose of the one or more objects and the one or more identified features, wherein the using one or more neural networks further comprises:

using a first neural network to generate, based on an input of the one or more first images, the pose of the one or more objects; and

using a second neural network to generate, based on the one or more first images of the one or more objects and the pose of the one or more objects, an identification of the one or more objects.

32 . The method of claim 31 , wherein the pose of the one or more objects comprises at least one of a location of a portion of the one or more objects, or a likelihood of the location.

33 . The method of claim 31 , wherein the one or more identified features of the one or more objects comprise at least one of a color of the one or more objects, or a type of the one or more objects.

34 . The method of claim 31 , further comprising receiving the one or more first images of the one or more objects, wherein the using one or more neural networks further comprises:

using a feature learning network to generate, based on the one or more first images and a first portion of the pose of the one or more objects, a vector of image features; and

using a classifier to generate, based on the vector of image features and a second portion of the pose of the one or more objects, an identification of the one or more objects.

35 . The method of claim 34 , wherein the classifier is a first classifier, and wherein the using one or more neural networks further comprises:

using a second classifier to generate, based on the vector of image features and the second portion of the pose of the one or more objects, a color of the one or more objects; and

using a third classifier to generate, based on the vector of image features and the second portion of the pose of the one or more objects, a type of the one or more objects.

36 . The method of claim 31 , wherein the one or more objects are one or more vehicles.

37 . The method of claim 36 , further comprising comparing an identified one of the one or more vehicles to a previously identified one of the one or more vehicles to determine a comparison therebetween, and to store the comparison on a server.