IP Library › Granted Patent US 10,623,756
Granted Patent B2
US 10,623,756 · App. 15/680,761 · Granted Apr 14, 2020

Interpolating visual data

Inventors: Zehan Wang (London, GB); Robert David Bishop (London, GB); Wenzhe Shi (London, GB); Jose Caballero (London, GB); Andrew Peter Aitken (London, GB); Johannes Totz (London, GB)
Assignee: Magic Pony Technology Limited
H04N19/36G06K9/46G06K9/6215G06K9/66G06N3/04G06N3/049G06N3/0445G06N3/0454G06N3/08G06T3/40G06T3/4007G06T3/4046G06T3/4053G06T5/001G06T5/002G06T7/11H04N7/0117H04N19/117H04N19/142H04N19/154H04N19/172H04N19/177H04N19/31H04N19/33H04N19/46H04N19/59H04N19/80H04N19/86G06T2207/10016G06T2207/20081G06T2207/20084H04N19/176H04N19/87
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 10,623,756
App. No.
15/680,761
Granted
Apr 14, 2020
Kind
B2
Abstract

A method for enhancing lower-quality visual data using hierarchical algorithms, the method comprising the steps of: receiving one or more sections of lower-quality visual data; applying a hierarchical algorithm to the one or more sections of lower-quality visual data to enhance the one or more sections of lower-quality visual data to one or more sections of higher-quality visual data, wherein the hierarchical algorithm was developed using a learned approach; and outputting the one or more sections of higher-quality visual data.

Claims (34)

1. A method for enhancing lower-quality visual data using hierarchical algorithms, the method comprising:

receiving at least one section of lower-quality visual data;

selecting a hierarchical algorithm from a library of learned hierarchical algorithms based on metric data associated with the lower-quality visual data;

applying the hierarchical algorithm to the one or more sections of lower-quality visual data to enhance the one or more sections of lower-quality visual data to one or more sections of higher-quality visual data, wherein the hierarchical algorithm was developed using a learned approach; and

outputting the one or more sections of higher-quality visual data.

2. The method of claim 1 , wherein the hierarchical algorithm is developed using a machine learning technique.

3. The method of claim 1 , wherein the hierarchical algorithm includes a non-linear hierarchical algorithm.

4. The method of claim 1 , wherein the hierarchical algorithm includes at least one convolutional neural network.

5. The method of claim 1 , wherein the hierarchical algorithm performs image enhancement using a super resolution technique.

6. The method of claim 1 , wherein the hierarchical algorithm uses a spatio-temporal approach.

7. The method of claim 1 , wherein the receiving of the at least one section of lower-quality visual data and applying of the hierarchical algorithm to the at least one section of lower quality visual data occur simultaneously.

8. The method of claim 1 , wherein the at least one section of lower-quality visual data is generated from at least one section of original higher-quality visual data.

9. The method of claim 1 , wherein

the hierarchical algorithm includes a plurality of layers, and

the plurality of layers are at least one of sequential, recurrent, recursive, branching, and merging.

10. A method of removing image artefacts from an image or section of visual data communicated over a network from a first node to a second node, the method at the second node comprising:

receiving an image or section of visual data via a network;

selecting a hierarchical algorithm from a library of learned hierarchical algorithms based on metric data associated with the visual data, the hierarchical algorithm being developed using a learned approach;

applying the hierarchical algorithm to the image or section visual of data to enhance the visual data; and

determining a content type of the image or section of visual data;

determining a measure of artefact severity of the image or section of visual data;

selecting, based on the content type and the measure of artefact severity of the image or section of visual data, an example based model from a library of example based models, the selected example based model being operable to recreate an original image or visual data from the image or section of visual data; and

using the selected example based model to recreate the original image or section of visual data from the image or section visual data.

11. The method of claim 10 , wherein at least a portion of the library of example based models is stored at the second node.

12. The method of claim 10 , wherein at least a portion of the library of example based models is stored at the first node.

13. The method of claim 10 , wherein, if a usable example based model is not included in the library of example based models, a generic example based model is selected.

14. The method of claim 10 , wherein

the original image or section of visual data is a down-sampled image or section of visual data created from a higher quality image or section of visual data, and

the method further comprises using at least one further example based model to recreate the higher quality image or section of visual data from the down-sampled section of visual data.

15. The method of claim 14 , wherein a reference to the at least one further example based model is transmitted from the first node to the second node, the further example based model being stored in a library at the second node.

16. The method of claim 10 , wherein the artefacts include for at least a portion of visual data at least one of blurring, pixelation, ringing, aliasing, missing data, and other marks, blemishes, defects, and abnormalities in visual data.

17. The method of claim 10 , wherein a section of visual data includes at least one of a single frame of visual data, a sequence of frames of visual data, and a region within a frame.

18. The method of claim 10 , wherein the at least one section of lower-quality visual data has a greater amount of artefacts than the at least one section of higher-quality visual data.

19. The method of claim 10 , wherein enhancing a quality of visual data includes upscaling a quality of the visual data.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Sep 19, 2017
From: WANG, ZEHAN; BISHOP, ROBERT DAVID; SHI, WENZHE; CABALLERO, JOSE; AITKEN, ANDREW PETER; TOTZ, JOHANNES
To: MAGIC PONY TECHNOLOGY LIMITED
Reel/Frame 043618/0365 →
Priority Claims (8)
GB 1502753.5 · Feb 19, 2015 · national
GB 1503427.5 · Feb 27, 2015 · national
GB 1505544.5 · Mar 31, 2015 · national
GB 1507141.8 · Apr 27, 2015 · national
GB 1508742.2 · May 21, 2015 · national
GB 1511231.1 · Jun 25, 2015 · national
GB 1519425.1 · Nov 3, 2015 · national
GB 1519687.6 · Nov 6, 2015 · national
Continuity (2)
Continuation PCTGB2016050430 · Feb 19, 2016
Related Publication 20180130179A1 · May 10, 2018