IP Library Granted Patent US 11,769,180
Granted Patent B2
US 11,769,180 · App. 16/739,286 · Granted Sep 26, 2023

Machine learning systems and methods for determining home value

Inventors: Mark Jayne (New York, NY); Gergely Svigruha (New York, NY); Raymond Liang (Brooklyn, NY); Gregory Aponte (Austin, TX)
Assignee: Orchard Technologies, Inc.
G06Q30/0278G06F18/214G06N3/042G06N3/045G06N3/08G06N5/01G06N20/20G06Q50/16G06V10/764G06V10/774G06V10/809G06V10/82
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,769,180
App. No.
16/739,286
Filed
Jan 10, 2020
Granted
Sep 26, 2023
Kind
B2
Art Unit
3629
USPC
705/306
Abstract

Techniques for determining value of a home by applying one or more neural network models to images of spaces in the home. The techniques include: obtaining at least one image of a first space inside or outside of a home; determining a type of the first space by processing the at least one image of the first space with a first neural network model; identifying at least one feature in the first space by processing the at least one image with a second neural network model different from the first neural network model and trained using images of spaces of a same type as the first space; and determining a value of the home at least in part by using the at least one feature as input to a machine learning model different from the first neural network model and the second neural network model.

Claims (77)

1. A system, comprising:

at least one computer hardware processor; and

at least one non-transitory computer-readable storage medium storing processor executable instructions that, when executed by the at least one computer hardware processor, cause the at least one computer hardware processor to perform:

obtaining a plurality of images, the plurality of images including a first image of a first room inside a home and a second image of a second room inside the home;

determining a type of the first room by processing the first image of the first room with a first neural network model, the first neural network model having a first plurality of layers comprising at least a convolutional layer, a pooling layer, a fully connected layer, or a softmax layer, the first plurality of layers including at least one million parameters, wherein the first image of the first room has a first resolution and wherein processing the first image of the first room with the first neural network model comprises:

generating, from the first image, a second image of the first room having a second resolution lower than the first resolution; and

processing the second image of the first room with the first neural network model;

determining a type of the second room by processing the second image of the second room with the first neural network model;

identifying at least one first feature in the first image of the first room by processing the first image with a second neural network model different from the first neural network model and trained using a first plurality of training images of rooms of a same type as the first room, the first plurality of training images including training images augmented by one or more transformations, the second neural network model having a second plurality of layers comprising at least first deep neural network layers, a reduction layer, second deep neural network layers, an average pooling layer, a fully connected layer, a dropout layer, or a softmax layer, the second plurality of layers including at least one million parameters;

identifying at least one second feature in the second image of the second room by processing the second image of the second room with a third neural network model different from the first neural network model and second neural network model, the third neural network model trained using a second plurality of training images of rooms of a same type as the second room, the second plurality of training images including training images augmented by one or more transformations, the third neural network model having a third plurality of layers comprising at least first deep neural network layers, a reduction layer, second deep neural network layers, an average pooling layer, a fully connected layer, a dropout layer, or a softmax layer, the third plurality of layers including at least one million parameters; and

determining a value of the home at least in part by using the at least one first feature and the at least one second feature as input to a machine learning model different from the first neural network model, the second neural network model, and the third neural network model.

2. The system of claim 1 , wherein the first neural network model comprises two neural network sub-models including a first sub-model having an average pooling layer and a second sub-model having a max pooling layer instead of the average pooling layer.

3. The system of claim 2 , wherein processing the first image of the first room with the first neural network model comprises:

processing the first image using the first sub-model to obtain first results;

processing the first image using the second sub-model to obtain second results; and

combining the first and second results to obtain an output result for the first neural network model.

4. The system of claim 1 , wherein the type of the first room is selected from the group consisting of: garage, living room, bedroom, kitchen, bathroom, dining room, family room, basement, attic, closet, laundry room, and mud room.

5. The system of claim 1 , wherein the processor-executable instructions further cause the at least one computer hardware processor to perform:

processing multiple images using the first neural network model to identify images for which the first neural network model output differs from labels produced by manual classification;

obtaining new labels for at least some of the multiple images; and

updating one or more parameters of the first neural network model by using the at least some of the multiple images with the new labels.

6. The system of claim 1 , wherein the second neural network model uses a bank of convolution kernels having different resolutions.

7. The system of claim 1 , wherein the first room is a kitchen, and wherein identifying the at least one first feature comprises identifying a type of material of a countertop in the kitchen and/or identifying a finish of an appliance in the kitchen.

8. The system of claim 1 , wherein the machine learning model is a random forest model.

9. The system of claim 1 , wherein the second plurality of layers comprises first deep neural network layers, a reduction layer, second deep neural network layers, an average pooling layer, a fully connected layer, a dropout layer, and a softmax layer.

10. The system of claim 9 , wherein processing the first image of the first room with the second neural network model comprises:

processing the first image with the first deep neural network layers to obtain first results;

providing the first results as input to the reduction layer to obtain second results;

providing the second results as input to the second deep neural network layers to obtain third results;

providing the third results as input to the average pooling layer to obtain fourth results;

providing the fourth results as input to the fully connected layer to obtain fifth results;

providing the fifth results as input to the dropout layer to obtain sixth results; and

providing the sixth results as input to the softmax layer to obtain an output result for the second neural network model.

11. A system, comprising:

at least one computer hardware processor; and

at least one non-transitory computer-readable storage medium storing processor executable instructions that, when executed by the at least one computer hardware processor, cause the at least one computer hardware processor to perform:

obtaining a plurality of images, the plurality of images including a first image of a first room inside a home, the first image having a first resolution;

determining a type of the first room by:

generating, from the first image, a second image of the first room having a second resolution lower than the first resolution; and

processing the second image of the first room with a first neural network model comprising:

a first neural network sub-model comprising a first plurality of layers comprising at least one million parameters, the first plurality of layers comprising at least deep neural network layers, an average pooling layer, a fully connected layer, or a softmax layer; and

a second neural network sub-model comprising a second plurality of layers comprising at least one million parameters, the second plurality of layers comprising at least deep neural network layers, a max pooling layer, a fully connected layer, or a softmax layer;

identifying at least one first feature in the first image of the first room by processing the first image with a second neural network model different from the first neural network model and trained using images of rooms of a same type as the first room, the second neural network model further having a third plurality of layers comprising at least a convolutional layer, a pooling layer, a fully connected layer, or a softmax layer, the third plurality of layers including at least one million parameters; and

determining a value of the home at least in part by using the at least one first feature as input to a machine learning model different from the first neural network model and the second neural network model.

12. The system of claim 11 , wherein:

the first plurality of layers of the first neural network sub-model comprises deep neural network layers, an average pooling layer, a fully connected layer, and a softmax layer; and

the second plurality of layers of the second neural network sub-model comprises deep neural network layers, a max pooling layer, a fully connected layer, and a softmax layer.

13. The system of claim 12 , wherein processing the second image of the first room with the first neural network model comprises:

processing the second image using the first neural network sub-model to obtain first output results;

processing the second image using the second neural network sub-model to obtain second output results; and

combining the first output results and second output results to obtain an output result for the first neural network model.

14. The system of claim 13 , wherein processing the second image using the first neural network sub-model comprises:

processing the second image with the deep neural network layers to obtain first results;

providing the first results as input to the average pooling layer to obtain second results;

providing the second results as input to the fully connected layer to obtain third results; and

providing the third results as input to the softmax layer to obtain the first output results.

15. The system of claim 11 , wherein the first resolution is 600×600 pixels and the second resolution is 300×300 pixels.

16. The system of claim 11 , wherein:

the second neural network model was trained using a plurality of training images of rooms of a same type as the first room, the plurality of training images including training images augmented by one or more transformations.

17. The system of claim 11 , wherein the second plurality of layers of the second neural network model comprises:

first deep neural network layers, a reduction layer, second deep neural network layers, an average pooling layer, a fully connected layer, a dropout layer, and a softmax layer.

18. The system of claim 11 , wherein the first plurality of training images and the second plurality of training images each comprise at least 10,000 training images.

19. A method, comprising:

using at least one computer hardware processor to perform:

obtaining a plurality of images, the plurality of images including a first image of a first room inside a home and a second image of a second room inside the home;

generating, from the first image of the first room, a second image of the first room having a lower resolution than the first image of the first room;

determining a type of the first room by processing the second image of the first room having a lower resolution than the first image of the first room with a first neural network model, the first neural network model having a first plurality of layers comprising at least a convolutional layer, a pooling layer, a fully connected layer, or a softmax layer, the first plurality of layers including at least one million parameters;

determining a type of the second room by processing the second image of the second room with the first neural network model;

identifying at least one first feature in the first image of the first room having a higher resolution than the second image of the first room by processing the first image of the first room with a second neural network model different from the first neural network model and trained using a first plurality of training images of rooms of a same type as the first room, the first plurality of training images including training images augmented by one or more transformations, the second neural network model having a second plurality of layers comprising at least first deep neural network layers, a reduction layer, second deep neural network layers, an average pooling layer, a fully connected layer, a dropout layer, or a softmax layer, the second plurality of layers including at least one million parameters;

identifying at least one second feature in the second image of the second room by processing the second image with a third neural network model different from the first neural network model and second neural network model, the third neural network model trained using a second plurality of training images of rooms of a same type as the second room, the second plurality of training images including training images augmented by one or more transformations, the third neural network model having a third plurality of layers comprising at least first deep neural network layers, a reduction layer, second deep neural network layers, an average pooling layer, a fully connected layer, a dropout layer, or a softmax layer, the third plurality of layers including at least one million parameters; and

determining a value of the home at least in part by using the at least one first feature and the at least one second feature as input to a machine learning model different from the first neural network model, the second neural network model, and the third second neural network model.

20. The method of claim 19 , wherein the first neural network model comprises two neural network sub-models including a first sub-model having an average pooling layer and a second sub-model having a max pooling layer instead of the average pooling layer.

21. The system of claim 20 , wherein processing the second image of the first room having a lower resolution than the first image of the first room with the first neural network model comprises:

processing the second image of the first room having the lower resolution than the first image of the first room using the first sub-model to obtain first results;

processing the second image of the first room having the lower resolution than the first image of the first room using the second sub-model to obtain second results; and

combining the first and second results to obtain an output result for the first neural network model.

22. The method of claim 19 , wherein the second plurality of layers comprises first deep neural network layers, a reduction layer, second deep neural network layers, an average pooling layer, a fully connected layer, a dropout layer, and a softmax layer.

Assignments (4)
SECURITY INTEREST Recorded Jan 27, 2025
From: ORCHARD TECHNOLOGIES, INC.
To: FIRST-CITIZENS BANK & TRUST COMPANY
Reel/Frame 070019/0461 →
SECURITY INTEREST Recorded Jan 27, 2025
From: ORCHARD TECHNOLOGIES, INC.
To: TRINITY CAPITAL INC.
Reel/Frame 070022/0001 →
CHANGE OF NAME Recorded Jan 13, 2021
From: NOHO SOLUTIONS, INC.
To: ORCHARD TECHNOLOGIES, INC.
Reel/Frame 054978/0461 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 21, 2020
From: JAYNE, MARK; SVIGRUHA, GERGELY; LIANG, RAYMOND; APONTE, GREGORY
To: NOHO SOLUTIONS, INC.
Reel/Frame 051570/0707 →
Continuity (2)
Provisional Application 62915257 · Oct 15, 2019
Related Publication 20210110439A1 · Apr 15, 2021