IP Library Granted Patent US 10,154,624
Granted Patent B2
US 10,154,624 · App. 15/231,505 · Granted Dec 18, 2018

Estimating nitrogen content using hyperspectral and multispectral images

Inventors: Wei Guan (Fremont, CA); Ying Xu (Boston, MA)
Assignee: The Climate Corporation
A01C21/007A01G22/00G06K9/00657A01G7/00
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,154,624
App. No.
15/231,505
Granted
Dec 18, 2018
Kind
B2
Abstract

In an approach, hyperspectral and/or multispectral remote sensing images are automatically analyzed by a nitrogen analysis subsystem to estimate the value of nitrogen variables of crops or other plant life located within the images. For example, the nitrogen analysis subsystem may contain a data collector module, a function generator module, and a nitrogen estimator module. The data collector module prepares training data which is used by the function generator module to train a mapping function. The mapping function is then used by the nitrogen estimator module to estimate the values of nitrogen variables for a new remote sensing image that is not included in the training set. The values may then be reported and/or used to determine an optimal amount of fertilizer to add to a field of crops to promote plant growth.

Claims (41)

1. A method comprising:

an agricultural intelligence computer receiving one or more band values for one or more pixels of one or more remote sensing images and, for each pixel of the one or more pixels, a corresponding nitrogen ground truth value of a nitrogen variable for the pixel;

the agricultural intelligence computer training a mapping function based on the one or more band values for the one or more pixels and, for each pixel of the one or more pixel, the corresponding nitrogen ground truth value, wherein the mapping function maps between a set of features related to the one or more band values of the one or more pixels and an estimated value of the nitrogen variable;

the agricultural intelligence computer receiving a new remote sensing image, wherein the new remote sensing image specifies, for one or more pixel of the new remote sensing image, the one or more band values for the pixel of the new remote sensing image;

the agricultural intelligence computer generating, for each pixel of the one or more pixels of the new remote sensing image, a respective estimated value of the nitrogen variable for the pixel using the mapping function;

the agricultural intelligence computer sending, for each pixel of the one or more pixels, the respective estimated value of the nitrogen variable for the pixel to a field manager computing device;

wherein the field manager computing device, in response to receiving respective estimated values of the nitrogen variable for one or more pixels, displays, on a computer display device, a map in which each pixel is assigned a color shaded based on the respective estimated value of the nitrogen variable generated for a pixel of the one or more pixels; wherein dark shadings in the map represent areas where leaf nitrogen content is high; wherein light shadings in the map represent areas where leaf nitrogen content is low.

2. The method of claim 1 , wherein the field manager computing device displays the respective estimated value of the nitrogen variable for each pixel of the one or more pixels using a heat map.

3. The method of claim 1 , wherein the agricultural intelligence computer trains the mapping function using linear regression.

4. The method of claim 3 , wherein the one or more remote sensing images are hyperspectral images and the agricultural intelligence computer trains the mapping function using linear regression with a dimension reduction based technique or a shrinkage based technique.

5. The method of claim 1 , wherein the agricultural intelligence computer trains the mapping function using a vegetation index.

6. The method of claim 5 , wherein the vegetation index is a two-band normalized difference index.

7. The method of claim 6 , wherein the one or more remote sensing images are hyperspectral images, the nitrogen variable is leaf nitrogen content, a first band of the two-band normalized difference index is approximately 574.5 nm and a second band of the two-band normalized difference index is approximately 514.1 nm.

8. The method of claim 6 , wherein the one or more remote sensing images are multispectral images, the nitrogen variable is leaf nitrogen content, a first band of the two-band normalized difference index is blue and a second band of the two-band normalized difference index is green.

9. The method of claim 1 , wherein the agricultural intelligence computer trains the mapping function using non-linear regression with additive model.

10. The method of claim 1 , wherein the new remote sensing image is an image of a field of crops, and further comprising the agricultural intelligence computer determining a recommend amount of nitrogen to add to the field to optimize a yield of the crops based on the respective estimated nitrogen value for the nitrogen variable for each pixel of the one or more pixels of the new remote sensing image and an optimal amount of nitrogen for the crops.

11. A system comprising:

one or more processors;

one or more non-transitory computer-readable storage mediums storing one or more instructions which, when executed by the one or more processors, cause the one or more processors to perform steps comprising:

receiving one or more band values for one or more pixels of one or more remote sensing images and, for each pixel of the one or more pixels, a corresponding nitrogen ground truth value of a nitrogen variable for the pixel;

training a mapping function based on the one or more band values for the one or more pixels and, for each pixel of the one or more pixel, the corresponding nitrogen ground truth value, wherein the mapping function maps between a set of features related to the one or more band values of the one or more pixels and an estimated value of the nitrogen variable;

a computer receiving a new remote sensing image, wherein the new remote sensing image specifies, for one or more pixel of the new remote sensing image, the one or more band values for the pixel of the new remote sensing image;

generating, for each pixel of the one or more pixels of the new remote sensing image, a respective estimated value of the nitrogen variable for the pixel using the mapping function;

sending, for each pixel of the one or more pixels, the respective estimated value of the nitrogen variable for the pixel to a field manager computing device;

wherein the field manager computing device, in response to receiving respective estimated values of the nitrogen variable for one or more pixels, displays, on a computer display device, a map in which each pixel is assigned a color shaded based on the respective estimated value of the nitrogen variable generated for a pixel of the one or more pixels; wherein dark shadings in the map represent areas where leaf nitrogen content is high; wherein light shadings in the map represent areas where leaf nitrogen content is low.

12. The system of claim 10 , wherein the field manager computing device displays the respective estimated value of the nitrogen variable for each pixel of the one or more pixels using a heat map.

13. The system of claim 11 , wherein the mapping function is trained using linear regression.

14. The system of claim 11 , wherein the one or more remote sensing images are hyperspectral images and the mapping function is trained using linear regression with a dimension reduction based technique or a shrinkage based technique.

15. The system of claim 11 , wherein the mapping function is trained using a vegetation index.

16. The system of claim 15 , wherein the vegetation index is a two-band normalized difference index.

17. The system of claim 16 , wherein the one or more remote sensing images are hyperspectral images, the nitrogen variable is leaf nitrogen content, a first band of the two-band normalized difference index is approximately 574.5 nm and a second band of the two-band normalized difference index is approximately 514.1 nm.

18. The system of claim 16 , wherein the one or more remote sensing images are multispectral images, the nitrogen variable is leaf nitrogen content, a first band of the two-band normalized difference index is blue and a second band of the two-band normalized difference index is green.

19. The system of claim 11 , wherein the mapping function is trained using non-linear regression with additive model.

20. The system of claim 11 , wherein the new remote sensing image is an image of a field of crops, and the steps further comprise determining a recommend amount of nitrogen to add to the field to optimize a yield of the crops based on the respective estimated nitrogen value for the nitrogen variable for each pixel of the one or more pixels of the new remote sensing image and an optimal amount of nitrogen for the crops.

21. One or more non-transitory computer-readable storage mediums storing one or more instructions which, when executed by one or more processors, cause the one or more processors to perform steps comprising:

receiving one or more band values for one or more pixels of one or more remote sensing images and, for each pixel of the one or more pixels, a corresponding nitrogen ground truth value of a nitrogen variable for the pixel;

training a mapping function based on the one or more band values for the one or more pixels and, for each pixel of the one or more pixel, the corresponding nitrogen ground truth value, wherein the mapping function maps between a set of features related to the one or more band values of the one or more pixels and an estimated value of the nitrogen variable;

a computer receiving a new remote sensing image, wherein the new remote sensing image specifies, for one or more pixel of the new remote sensing image, the one or more band values for the pixel of the new remote sensing image;

generating, for each pixel of the one or more pixels of the new remote sensing image, a respective estimated value of the nitrogen variable for the pixel using the mapping function;

sending, for each pixel of the one or more pixels, the respective estimated value of the nitrogen variable for the pixel to a field manager computing device;

wherein the field manager computing device, in response to receiving respective estimated values of the nitrogen variable for one or more pixels, displays, on a computer display device, a map in which each pixel is assigned a color shaded based on the respective estimated value of the nitrogen variable generated for a pixel of the one or more pixels; wherein dark shadings in the map represent areas where leaf nitrogen content is high; wherein light shadings in the map represent areas where leaf nitrogen content is low.

Assignments (5)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 23, 2026
From: CLIMATE LLC
To: MONSANTO COMPANY
Reel/Frame 075177/0751 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Mar 23, 2026
From: MONSANTO COMPANY
To: MONSANTO TECHNOLOGY LLC
Reel/Frame 075177/0908 →
CHANGE IN PRINCIPAL PLACE OF BUSINESS Recorded Sep 2, 2025
From: CLIMATE LLC
To: CLIMATE LLC
Reel/Frame 072809/0473 →
CHANGE OF NAME Recorded Nov 16, 2023
From: THE CLIMATE CORPORATION
To: CLIMATE LLC
Reel/Frame 065610/0571 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 11, 2016
From: GUAN, WEI; XU, YING
To: THE CLIMATE CORPORATION
Reel/Frame 039407/0928 →
Continuity (1)
Related Publication 20180035605A1 · Feb 8, 2018
Cited By (24)
US 12,193,350 US 12,216,472 US 12,225,846 US 12,245,549 US 12,250,905 US 12,271,196 US 12,284,934 US 12,295,288 US 12,298,767 US 12,302,791 US 12,310,286 US 12,329,050 US 12,329,065 US 12,329,148 US 12,358,493 US 12,386,354 US 12,419,220 US 12,422,847 US 12,444,511 US 12,450,669 US 12,495,733 US 12,520,759 US 12,550,802 US 12,582,035