IP Library Granted Patent US 12,592,091
Granted Patent B2
US 12,592,091 · App. 18/015,088 · Granted Mar 31, 2026

Image and video processing circuitry and method using an image signature

Inventors: Lev Markhasin (Stuttgart, DE); Stephen Tiedemann (Stuttgart, DE); Stefan Uhlich (Stuttgart, DE); Bi Wang (Stuttgart, DE)
Assignee: SONY SEMICONDUCTOR SOLUTIONS CORPORATION
G06V20/70G06T1/0021G06V10/764H04L9/50G06T2201/005
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,592,091
App. No.
18/015,088
Granted
Mar 31, 2026
Kind
B2
Abstract

An image processing circuitry configured to: generate, based on obtained image data, a visual content word sequence indicative for a visual content of an image represented by the obtained image data; and generate, based on the generated visual content word sequence, an image signature for the image.

Claims (52)

1 . An image processing circuitry configured to:

generate, based on obtained image data, a visual content word sequence indicative for a visual content of an image represented by the obtained image data;

generate, based on the generated visual content word sequence, an image signature for the image;

generate, based on another obtained image data, another visual content word sequence indicative for a visual content of another image represented by the another obtained image data;

compare the visual content word sequence and the another visual content word sequence for at least one manipulation; and

output an alarm indication in response to the comparison between the visual content word sequence and the another visual content word sequence indicating the at least one manipulation.

2 . The image processing circuitry according to claim 1 , wherein the image processing circuitry is further configured to:

generate, based on the generated visual content word sequence, a visual content signature for the generated visual content word sequence, and wherein the image signature is generated by adding the generated visual content word sequence and the visual content signature to metadata of the image.

3 . The image processing circuitry according to claim 1 , wherein:

the image signature is generated by adding the generated visual content word sequence as a watermark to the image.

4 . The image processing circuitry according to claim 1 , wherein;

the image signature is generated by adding the generated visual content word sequence to a blockchain.

5 . The image processing circuitry according to claim 1 , wherein the image processing circuitry is further configured to:

output an indication whether the generated visual content word sequence and another visual content word sequence, generated based on another obtained image data, indicative for a visual content of another image represented by the another image data, are identical.

6 . The image processing circuitry according to claim 5 , wherein:

at least one of the indication, the image together with the generated visual content word sequence and the another image together with the another visual content word sequence is displayed on a display to a user.

7 . The image processing circuitry according to claim 1 , wherein:

the at least one manipulation is caused by artificial intelligence comprising at least one of a generative adversarial network or another deep neural network (DNN).

8 . The image processing circuitry according to claim 1 , wherein:

the visual content word sequence is generated by a neural network based on a convolutional neural network, a captioning network, an encoder-decoder network, or a long short-term memory (LSTM) network.

9 . An image processing method comprising:

generating, based on obtained image data, a visual content word sequence indicative for a visual content of an image represented by the obtained image data;

generating, based on the generated visual content word sequence, an image signature for the image;

generating, based on another obtained image data, another visual content word sequence indicative for a visual content of another image represented by the another obtained image data;

comparing the visual content word sequence and the another visual content word sequence for at least one manipulation; and

outputting an alarm indication in response to the comparison between the visual content word sequence and the another visual content word sequence indicating the at least one manipulation.

10 . The image processing method according to claim 9 , further comprising:

generating, based on the generated visual content word sequence, a visual content signature for the generated visual content word sequence, and wherein the image signature is generated by adding the generated visual content word sequence and the visual content signature to metadata of the image.

11 . The image processing method according to claim 9 , further comprising:

generating the image signature by adding the generated visual content word sequence as a watermark to the image.

12 . The image processing method according to claim 9 , further comprising:

generating the image signature by adding the generated visual content word sequence to a blockchain.

13 . The image processing method according to claim 9 , further comprising:

outputting an indication whether the generated visual content word sequence and another visual content word sequence, generated based on another obtained image data, indicative for a visual content present in another image represented by the another image data, are identical.

14 . The image processing method according to claim 13 , further comprising:

displaying at least one of the indication, the image and the another image on a display to a user.

15 . The image processing method according to claim 9 , wherein:

the at least one manipulation is caused by artificial intelligence comprising at least one of a generative adversarial network or another deep neural network (DNN).

16 . The image processing method according to claim 9 , wherein:

the visual content word sequence is generated by a neural network based on a convolutional neural network, a captioning network, an encoder-decoder network, or a long short-term memory (LSTM) network.

17 . A non-transitory computer-readable recording medium storing instructions configured to cause a processor to perform an image processing method, the method comprising:

generating, based on obtained image data, a visual content word sequence indicative for a visual content of an image represented by the obtained image data;

generating, based on the generated visual content word sequence, an image signature for the image;

generating, based on another obtained image data, another visual content word sequence indicative for a visual content of another image represented by the another obtained image data;

comparing the visual content word sequence and the another visual content word sequence for at least one manipulation; and

outputting an alarm indication in response to the comparison between the visual content word sequence and the another visual content word sequence indicating the at least one manipulation.

18 . The method according to claim 17 , wherein:

the at least one manipulation is caused by artificial intelligence comprising at least one of a generative adversarial network or another deep neural network (DNN).

19 . The method according to claim 17 , wherein:

the visual content word sequence is generated by a neural network based on a convolutional neural network, a captioning network, an encoder-decoder network, or a long short-term memory (LSTM) network.

20 . The method according to claim 17 , further comprising:

generating, based on the generated visual content word sequence, a visual content signature for the generated visual content word sequence, and wherein the image signature is generated by adding the generated visual content word sequence and the visual content signature to metadata of the image.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 1, 2023
From: MARKHASIN, LEV; TIEDEMANN, STEPHEN; UHLICH, STEFAN; WANG, BI
To: SONY SEMICONDUCTOR SOLUTIONS CORPORATION
Reel/Frame 062565/0851 →
Priority Claims (1)
EP 20186048 · Jul 15, 2020 · regional
Continuity (1)
Related Publication 20230252808A1 · Aug 10, 2023
References Cited (12)
US 20060020830A1 · Roberts · 2006 [cited by applicant]
US 20090256972A1 · Ramaswamy et al. · 2009 [cited by applicant]
US 20180288362A1 · Altenburger et al. · 2018 [cited by applicant]
US 20190370286A1 · Bertsch et al. · 2019 [cited by applicant]
WO 2020044052A1 · 2020 [cited by applicant]
International Search Report and Written Opinion mailed on Oct. 19, 2021, received for PCT Application PCT/EP2021/068987, filed on Jul. 8, 2021, 9 pages. [cited by applicant]
Xu et al., “Show, Attend and Tell: Neural Image Caption Generation with Visual Attention”, arXiv:1502.03044v3, [cs.LG], Apr. 19, 2016, 22 pages. [cited by applicant]
Hossain et al., “A Comprehensive Survey of Deep Learning for Image Captioning”, ACM Computing Surveys, arXiv:1810.04020v2 [cs.CV], Oct. 14, 2018, pp. 0:1-0:36. [cited by applicant]
Bhattacharjee et al., “Compression Tolerant Image Authentication”, IEEE, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269), Available Online At: https://ieeexplore.IEEE.org/ abstr… [cited by applicant]
Wang et al., “A Visual Model-Based Perceptual Image Hash for Content Authentication”, IEEE Transactions on Information Forensics and Security, vol. 10, No. 7, Available Online At: https://ieeexplore.ieee.org/document/70… [cited by applicant]
Lin et al., “A Robust Image Authentication Method Distinguishing JPEG Compression from Malicious Manipulation”, IEEE Transactions on Circuits and Systems of Video Technology, vol. 11, No. 2, Feb. 2001, Available Online … [cited by applicant]
Wang et al., “An Overview of Image Caption Generation Methods”, Computational Intelligence and Neuroscience, vol. 2020, Article ID 3062706, Available Online At: https://downloads.hindawi.com/journals/cin/2020/3062706.pd… [cited by applicant]