IP Library › Granted Patent US 12,355,981
Granted Patent B2
US 12,355,981 · App. 17/980,156 · Granted Jul 8, 2025

Image processing device and a method for encoding a view area within an image frame of a video into an encoded video area frame

Inventors: Viktor Edpalm (Lund, SE); Song Yuan (Lund, SE); Toivo Henningsson (Lund, SE); Johan Palmaeus (Lund, SE)
Assignee: Axis AB
H04N19/159G06T7/11G06T9/00G06V10/25
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,355,981
App. No.
17/980,156
Granted
Jul 8, 2025
Kind
B2
Abstract

A and method encode a view area within a current image frame of a video into an encoded video area frame. The view area is a respective subarea of each image frame, each image frame comprising first and second image portions, and between previous and current image frames, the view area moves across a boundary between the first and second image portions. First and second encoders are encode image data of the first and second image portions, respectively. First, second and third portions of the view area are identified based on their respective location in the previous and current image frames. Image data of the first and third portions are inter-coded as first and third encoded slices/tiles. Image data of the second portion of the view area in the current image frame are intra-coded as a second encoded slice/tile. The encoded slices/tiles are merged into the encoded video area frame.

Claims (43)

1. A method for encoding a view area within a current image frame of a video comprising a plurality of image frames into an encoded video area frame, wherein the view area is a respective subarea of each image frame of the plurality of image frames, wherein each image frame of the plurality of image frames comprises a first image portion and a second image portion, wherein, between a previous image frame of the video and the current image frame of the video, the view area moves across a boundary between the first image portion and the second image portion, wherein a first encoder arranged in a first processing circuitry is configured to encode image data of the first image portion of each image frame of the plurality of image frames and a second encoder arranged in a second processing circuitry is configured to encode image data of the second image portion of each image frame of the plurality of image frames, the method comprising:

identifying a first portion of the view area that is located in the first image portion in both the previous image frame and the current image frame,

identifying a second portion of the view area that is located in the first image portion in the previous image frame and in the second image portion in the current image frame,

identifying a third portion of the view area that is located in the second image portion in both the previous image frame and the current image frame,

inter-coding, by the first encoder, image data of the first portion of the view area in the current image frame as a first encoded slice/tile by referring to reference image data corresponding to the previous image frame buffered in a first reference buffer arranged in the first processing circuitry,

intra-coding, by the second encoder, all image data of the second portion of the view area in the current image frame as a second encoded slice/tile and refraining from inter-coding of any image data of the second portion of the view area in the current image frame,

inter-coding, by the second encoder, image data of the third portion of the view area in the current image frame as a third encoded slice/tile by referring to reference image data corresponding to the previous image frame buffered in a second reference buffer arranged in the second processing circuitry, and

merging the first encoded slice/tile, the second encoded slice/tile, and the third encoded slice/tile into the encoded video area frame.

2. The method according to claim 1 , wherein, in the act of inter-coding, by the first encoder, the image data of the first portion of the view area in the current image frame are inter-coded as the first encoded slice/tile by referring to reference image data corresponding to the first portion of the view area in the previous image frame buffered in the first reference buffer, and wherein, in the act of inter-coding, by the second encoder, the image data of the third portion of the view area in the current image frame are inter-coded as the third encoded slice/tile by referring to reference image data corresponding to the third portion of the view area in the previous image frame buffered in the second reference buffer.

3. The method according to claim 1 , wherein, in the act of inter-coding, by the first encoder, the image data of the first portion of the view area in the current image frame are inter-coded as the first encoded slice/tile by referring to reference image data corresponding to the view area in the previous image frame buffered in the first reference buffer, and wherein, in the act of inter-coding, by the second encoder, the image data of the third portion of the view area in the current image frame are inter-coded as the third encoded slice/tile by referring to reference image data corresponding to the view area in the previous image frame buffered in the second reference buffer.

4. The method according to claim 3 , further comprising:

inter-coding, by the first encoder, dummy image data of a ghost portion of the view area as a ghost encoded slice/tile by referring to reference image data corresponding to the view area in the previous image frame buffered in the first reference buffer, wherein the ghost portion is an extra portion configured to have a shape such that the combination of the first portion and the ghost portion of the current image frame has the same shape as the reference image data in the first reference buffer, and wherein the ghost slice/tile is a slice/tile that is not merged into the encoded view area frame but is only encoded in order to enable an encoder to refer to all reference data corresponding to the view area in in the previous image frame buffered in the first reference buffer.

5. The method according to claim 1 , wherein the first image portion is captured by a first image sensor and wherein the second image portion is captured by a second image sensor.

6. A non-transitory computer-readable storage medium having stored thereon instructions, when executed by a device having processing capabilities, for implementing a method for encoding a view area within a current image frame of a video comprising a plurality of image frames into an encoded video area frame, wherein the view area is a respective subarea of each image frame of the plurality of image frames, wherein each image frame of the plurality of image frames comprises a first image portion and a second image portion, wherein, between a previous image frame of the video and the current image frame of the video, the view area moves across a boundary between the first image portion and the second image portion, wherein a first encoder arranged in a first processing circuitry is configured to encode image data of the first image portion of each image frame of the plurality of image frames and a second encoder arranged in a second processing circuitry is configured to encode image data of the second image portion of each image frame of the plurality of image frames, the method comprising:

identifying a first portion of the view area that is located in the first image portion in both the previous image frame and the current image frame,

identifying a second portion of the view area that is located in the first image portion in the previous image frame and in the second image portion in the current image frame,

identifying a third portion of the view area that is located in the second image portion in both the previous image frame and the current image frame,

inter-coding, by the first encoder, image data of the first portion of the view area in the current image frame as a first encoded slice/tile by referring to reference image data corresponding to the previous image frame buffered in a first reference buffer arranged in the first processing circuitry,

intra-coding, by the second encoder, all image data of the second portion of the view area in the current image frame as a second encoded slice/tile and refraining from inter-coding of any image data of the second portion of the view area in the current image frame,

inter-coding, by the second encoder, image data of the third portion of the view area in the current image frame as a third encoded slice/tile by referring to reference image data corresponding to the previous image frame buffered in a second reference buffer arranged in the second processing circuitry, and

merging the first encoded slice/tile, the second encoded slice/tile, and the third encoded slice/tile into the encoded video area frame.

7. An image processing device for encoding a view area within a current image frame of a video comprising a plurality of image frames into an encoded video area frame, wherein the view area is a respective subarea of each image frame of the plurality of image frames, wherein each image frame of the plurality of image frames comprises a first image portion and a second image portion, wherein, between a previous image frame of the video and the current image frame of the video, the view area moves across a boundary between the first image portion and the second image portion, the device comprising:

a first processing circuitry;

a second processing circuitry;

a first encoder arranged in the first processing circuitry configured to encode image data of the first image portion of each image frame of the plurality of image frames;

a second encoder arranged in the second processing circuitry configured to encode image data of the second image portion of each image frame of the plurality of image frames;

a first reference buffer arranged in the first processing circuitry;

a second reference buffer arranged in the second processing circuitry; and

device circuitry configured to execute:

a first identifying function configured to identify a first portion of the view area that is located in the first image portion in both the previous image frame and the current image frame,

a second identifying function configured to identify a second portion of the view area that is located in the first image portion in the previous image frame and in the second image portion in the current image frame,

a third identifying function configured to identify a third portion of the view area that is located in the second image portion in both the previous image frame and the current image frame,

a first inter-coding instructing function configured to instruct the first encoder to inter-code image data of the first portion of the view area in the current image frame as a first encoded slice/tile by referring to reference image data corresponding to the previous image frame buffered in the first reference buffer,

an intra-coding instructing function configured to instruct the second encoder to intra-code all image data of the second portion of the view area in the current image frame as a second encoded slice/tile and refrain from inter-coding of any image data of the second portion of the view area in the current image frame,

a second inter-coding instructing function configured to instruct the second encoder to inter-code image data of the third portion of the view area in the current image frame as a third encoded slice/tile by referring to reference image data corresponding to the previous image frame buffered in the second reference buffer, and

a merging function configured to merge the first encoded slice/tile, the second encoded slice/tile, and the third encoded slice/tile into the encoded view area frame.

8. The image processing device according to claim 7 , wherein the first inter-coding instructing function is configured to instruct the first encoder to inter encode the image data of the first portion of the view area in the current image frame as the first encoded slice/tile by referring to reference image data corresponding to the first portion of the view area in the previous image frame buffered in the first reference buffer, and wherein, the second inter-coding instructing function is configured to instruct the second encoder to inter-code the image data of the third portion of the view area in the current image frame as the third encoded slice/tile by referring to reference image data corresponding to the third portion of the view area in the previous image frame buffered in the second reference buffer.

9. The image processing device according to claim 7 , wherein the first inter-coding instructing function is configured to instruct the first encoder to inter-code the image data of the first portion of the view area in the current image frame as the first encoded slice/tile by referring to reference image data corresponding to the view area in the previous image frame buffered in the first reference buffer, and wherein the second inter-coding instructing function is configured to instruct the second encoder to inter-code the image data of the third portion of the view area in the current image frame as the third encoded slice/tile by referring to reference image data corresponding to the view area in the previous image frame buffered in the second reference buffer.

10. The image processing device according to claim 9 , wherein the device circuitry is further configured to execute:

a third inter-coding instructing function configured instruct the first encoder to inter-code dummy image data of a ghost portion of the view area as a ghost encoded slice/tile by referring to reference image data corresponding to the view area in the previous image frame buffered in the first reference buffer, wherein the ghost portion is an extra portion configured to have a shape such that the combination of the first portion and the ghost portion of the current image frame has the same shape as the reference image data in the first reference buffer, and wherein the ghost slice/tile is a slice/tile that is not merged into the encoded view area frame but is only encoded in order to enable an encoder to refer to all reference data corresponding to the view area in in the previous image frame buffered in the first reference buffer.

11. The image processing device according to claim 7 , further comprising:

a first image sensor for capturing the first image portion; and

a second image sensor for capturing the second image portion.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 3, 2022
From: EDPALM, VIKTOR; YUAN, SONG; HENNINGSSON, TOIVO; PALMAEUS, JOHAN
To: AXIS AB
Reel/Frame 061648/0979 →
Priority Claims (1)
EP 21211747 · Dec 1, 2021 · regional
Continuity (1)
Related Publication 20230171409A1 · Jun 1, 2023
References Cited (8)
US 10432970B1 · Phillips et al. · 2019 [cited by applicant]
US 20160014413A1 · Sato · 2016 [cited by applicant]
US 20180338156A1 · Morigami et al. · 2018 [cited by applicant]
US 20190200029A1 · Bangma et al. · 2019 [cited by applicant]
EP 3220642B1 · 2018 [cited by applicant]
EP 3346709A1 · 2018 [cited by examiner]
Skupin et al., “Compressed domain processing for stereoscopic tile based panorama streaming using MV-HEVC,” 2015 IEEE 5th International Conference on Consumer Electronics—Berlin (ICCE-Berlin), pp. 160-164 (2015). [cited by applicant]
Extended European Search Report dated May 4, 2022 for European Patent Application No. 21211747.7. [cited by applicant]