IP Library Granted Patent US 11,682,052
Granted Patent B2
US 11,682,052 · App. 17/510,248 · Granted Jun 20, 2023

Machine learning systems and methods for determining home value

Inventors: Mark Jayne (New York, NY); Gergely Svigruha (New York, NY); Raymond Liang (Brooklyn, NY); Gregory Aponte (Austin, TX)
Assignee: Orchard Technologies, Inc.
G06Q30/0278G06F18/214G06N3/042G06N3/045G06N3/08G06N5/01G06N20/20G06Q50/16G06V10/764G06V10/774G06V10/809G06V10/82
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,682,052
App. No.
17/510,248
Granted
Jun 20, 2023
Kind
B2
Abstract

Techniques for determining value of a home by applying one or more neural network models to images of spaces in the home. The techniques include: obtaining at least one image of a first space inside or outside of a home; determining a type of the first space by processing the at least one image of the first space with a first neural network model; identifying at least one feature in the first space by processing the at least one image with a second neural network model different from the first neural network model and trained using images of spaces of a same type as the first space; and determining a value of the home at least in part by using the at least one feature as input to a machine learning model different from the first neural network model and the second neural network model.

Claims (63)

1. A system, comprising:

at least one computer hardware processor; and

at least one non-transitory computer-readable storage medium storing processor executable instructions that, when executed by the at least one computer hardware processor, cause the at least one computer hardware processor to perform:

obtaining a plurality of images, the plurality of images including a first image of a first room inside a home and a second image of a second room inside the home;

determining a type of the first room by processing the first image of the first room with a first neural network model, the first neural network model comprising parallel first and second neural network sub-models each configured to receive the first image as an input and to provide a respective neural network sub-model output to a same prediction combination logic which is configured to provide a first output result indicating the type of the first room, wherein the first neural network sub-model has an average pooling layer and the second neural network sub-model has a max pooling layer instead of an average pooling layer, wherein processing the first image of the first room with the first neural network model comprises:

processing the first image using the first neural network sub-model to obtain first results representing the respective neural network sub-model output of the first neural network sub-model,

processing the first image using the second neural network sub-model to obtain second results representing the respective neural network sub-model output of the second neural network sub-model, and

combining, with the same prediction combination logic, the first and second results to obtain, from the first neural network, the first output result indicating the type of the first room;

determining a type of the second room by processing the second image of the second room with the first neural network model;

identifying at least one first feature in the first room by processing the first image with a second neural network model different from the first neural network model and trained using images of rooms of a same type as the first room;

identifying at least one second feature in the second room by processing the second image with a third neural network model different from the first neural network model and the second neural network model, the third neural network model trained using images of rooms of a same type as the second room; and

determining a value of the home at least in part by using the at least one first feature and the at least one second feature as input to a machine learning model different from the first neural network model, the second neural network model, and the third neural network model.

2. The system of claim 1 , wherein the first neural network model comprises one or more convolutional layers.

3. The system of claim 1 , wherein processing the second image of the second room with the first neural network model comprises:

processing the second image using the first neural network sub-model to obtain third results;

processing the second image using the second neural network sub-model to obtain fourth results; and

combining the third and fourth results to obtain a second output result for the first neural network model, the second output result indicating the type of the second room.

4. The system of claim 1 , wherein the first neural network model is a deep neural network.

5. The system of claim 1 , wherein the first neural network model comprises multiple convolutional layers.

6. The system of claim 1 , wherein the first neural network model is trained to determine the types for the first room and the second room using one or more images of the same types of rooms using a transfer learning technique.

7. The system of claim 1 , wherein the type of the first room is selected from the group consisting of: garage, living room, bedroom, kitchen, bathroom, dining room, family room, basement, attic, closet, laundry room, and mud room.

8. The system of claim 1 , wherein the processor-executable instructions further cause the at least one computer hardware processor to perform:

processing multiple images using the first neural network model to identify images for which the first neural network model output differs from labels produced by manual classification;

obtaining new labels for at least some of the multiple images; and

updating one or more parameters of the first neural network model by using the at least some of the multiple images with the new labels.

9. The system of claim 1 , wherein the second neural network model is a deep neural network model.

10. The system of claim 9 , wherein the second neural network model comprises one or more residual connections.

11. The system of claim 9 , wherein the second neural network model uses a bank of convolution kernels having different resolutions.

12. The system of claim 9 , wherein the second neural network model includes one or more convolutional layers.

13. The system of claim 9 , wherein the second neural network model is trained at least in part by using transfer learning.

14. The system of claim 1 , wherein the first room is a kitchen, and wherein identifying the at least one first feature comprises identifying a type of material of a countertop in the kitchen.

15. The system of claim 1 , wherein the first room is a kitchen, and wherein identifying the at least one first feature comprises identifying a finish of an appliance in the kitchen.

16. The system of claim 1 , wherein the machine learning model is a random forest model.

17. The system of claim 1 , wherein the first neural network model comprises a first plurality of layers and the second neural network model comprises a second plurality of layers, and wherein the first plurality of layers of the first neural network model includes at least one million parameters, and wherein the second plurality of layers of the second neural network model includes at least one million parameters.

18. The system of claim 1 , wherein the processor executable instructions, when executed by the at least one computer hardware processor, further cause the at least one computer hardware processor to perform:

obtaining the plurality of images including a third image of an outdoor space outside the home;

determining a type of the outdoor space by processing the third image with the first neural network model;

identifying at least one third feature of the outdoor space by processing the third image with a fourth neural network model different from the first neural network model, the second neural network model and the third neural network model, the fourth neural network model trained using images of a same type as the outdoor space; and

determining the value of the home at least in part by using the at least one first feature, the at least one second feature and the at least one third feature as input to the machine learning model different from the first neural network model, the second neural network model, the third neural network model and the fourth neural network model.

19. The system of claim 18 , wherein determining the type of the outdoor space is selected from the group consisting of: front yard, back yard, side yard and porch.

20. The system of claim 1 , wherein the first neural network sub-model has at least one deep neural network layer, a fully connected layer, and a softmax layer, and wherein the second neural network sub-model has at least one deep neural network layer, a fully connected layer, and a softmax layer.

21. The system of claim 1 , wherein the first result comprises a first plurality of output classes and the second result comprises a second plurality of output classes, and wherein the same prediction combination logic is configured to combine the first and second results at least in part by taking class averages across the first plurality of output classes and the second plurality of output classes and performing a renormalization operation.

22. A method comprising:

using at least one computer hardware processor to perform:

obtaining a plurality of images, the plurality of images including a first image of a first room inside a home and a second image of a second room inside the home;

determining a type of the first room by processing the first image of the first room with a first neural network model, the first neural network model comprising parallel first and second neural network sub-models each configured to receive the first image as an input and to provide a respective neural network sub-model output to a same prediction combination logic which is configured to provide a first output result indicating the type of the first room, wherein the first neural network sub-model has an average pooling layer and the second neural network sub-model has a max pooling layer instead of an average pooling layer, wherein processing the first image of the first room with the first neural network model comprises:

processing the first image using the first neural network sub-model to obtain first results representing the respective neural network sub-model output of the first neural network sub-model,

processing the first image using the second neural network sub-model to obtain second results representing the respective neural network sub-model output of the second neural network sub-model, and

combining, with the same prediction combination logic, the first and second results to obtain, from the first neural network model, the first output result indicating the type of the first room;

determining a type of the second room by processing the second image of the second room with the first neural network model;

identifying at least one first feature in the first room by processing the first image with a second neural network model different from the first neural network model and trained using images of rooms of a same type as the first room;

identifying at least one second feature in the second room by processing the second image with a third neural network model different from the first neural network model and the second neural network model, the third neural network model trained using images of rooms of a same type as the second room; and

determining a value of the home at least in part by using the at least one first feature and the at least one second feature as input to a machine learning model different from the first neural network model, the second neural network model, and the third neural network model.

23. At least one non-transitory computer-readable storage medium storing processor executable instructions that, when executed by at least one computer hardware processor, cause the at least one computer hardware processor to perform:

obtaining a plurality of images, the plurality of images including a first image of a first room inside a home and a second image of a second room inside the home;

determining a type of the first room by processing the first image of the first room with a first neural network model, the first neural network model comprising parallel first and second neural network sub-models each configured to receive the first image as an input and to provide a respective neural network sub-model output to a same prediction combination logic which is configured to provide a first output result indicating the type of the first room, wherein the first neural network sub-model has an average pooling layer and the second neural network sub-model has a max pooling layer instead of an average pooling layer, wherein processing the first image of the first room with the first neural network model comprises:

processing the first image using the first neural network sub-model to obtain first results representing the respective neural network sub-model output of the first neural network sub-model,

processing the first image using the second neural network sub-model to obtain second results representing the respective neural network sub-model output of the second neural network sub-model, and

combining, with the same prediction combination logic, the first and second results to obtain, from the first neural network model, the first output result indicating the type of the first room;

determining a type of the second room by processing the second image of the second room with the first neural network model;

identifying at least one first feature in the first room by processing the first image with a second neural network model different from the first neural network model and trained using images of rooms of a same type as the first room;

identifying at least one second feature in the second room by processing the second image with a third neural network model different from the first neural network model and the second neural network model, the third neural network model trained using images of rooms of a same type as the second room; and

determining a value of the home at least in part by using the at least one first feature and the at least one second feature as input to a machine learning model different from the first neural network model, the second neural network model, and the third neural network model.

Assignments (4)
SECURITY INTEREST Recorded Jan 27, 2025
From: ORCHARD TECHNOLOGIES, INC.
To: FIRST-CITIZENS BANK & TRUST COMPANY
Reel/Frame 070019/0461 →
SECURITY INTEREST Recorded Jan 27, 2025
From: ORCHARD TECHNOLOGIES, INC.
To: TRINITY CAPITAL INC.
Reel/Frame 070022/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 15, 2021
From: JAYNE, MARK; SVIGRUHA, GERGELY; LIANG, RAYMOND; APONTE, GREGORY
To: NOHO SOLUTIONS, INC.
Reel/Frame 058110/0700 →
CHANGE OF NAME Recorded Nov 15, 2021
From: NOHO SOLUTIONS, INC.
To: ORCHARD TECHNOLOGIES, INC.
Reel/Frame 058132/0156 →
Continuity (4)
Continuation PCTUS2020055473 · Oct 14, 2020
Continuation 16739286 · Jan 10, 2020
Provisional Application 62915257 · Oct 15, 2019
Related Publication 20220044285A1 · Feb 10, 2022