GENERATING A REACTIVE PROFILE PORTRAIT
A reactive profile picture brings a profile image to life by displaying short video segments of the target user expressing a relevant emotion in reaction to an action by a viewing user that relates to content associated with the target user in an online system such as a social media web site. The viewing user therefore experiences a real-time reaction in a manner similar to a face-to-face interaction. The reactive profile picture can be automatically generated from either a video input of the target user or from a single input image of the target user.
1 . A method comprising:
storing to a storage medium in association with a profile of a target user, a plurality of video segments depicting a portrait of a target user, wherein each of the plurality of video segments is associated with a different emotion and depicts the target user expressing the corresponding emotion, the plurality of video segments including an idle segment depicting the target user in a neutral expression;
providing to a client device of a viewing user, a presentation of the idle segment together with content associated with the target user;
detecting an interaction of the viewing user on the client device with the content associated with the target user;
analyzing, by a processor, the interaction to determine a sentiment associated with the interaction;
selecting, based on the sentiment, a reactive video segment from the plurality of video segments, the reactive video segment associated with an emotion corresponding to the sentiment of the interaction; and
presenting the reactive video segment to the client device.
2 . The method of claim 1 , wherein presenting the reactive video segment comprises:
blending a sequence of starting overlapping frames associated with the reactive video segment with a sequence of ending overlapping frames associated with the idle segment to generate a sequence of transition frames; and
presenting the transition frames to transition from the idle segment to the reactive video segment.
3 . The method of claim 2 , wherein blending the sequence of starting overlapping frames associated with the reactive video segment with a sequence of ending overlapping frames associated with the idle segment comprises:
determining a sequence of transformations to warp the sequence of ending overlapping frames to align locations of facial landmarks in the sequence of ending overlapping frames to locations of facial landmarks in the sequence of starting overlapping frames;
weighting the sequence of transformations with increasing weights over a duration of the sequence of ending overlapping frames and the sequence of starting overlapping frames to generate a weighted sequence of transformations;
applying the weighted sequence of transformations to the ending overlapping frames to generate the sequence of transition frames.
4 . The method of claim 1 , further comprising:
blending a starting set of overlapping frames associated with the idle segment with a set of ending overlapping frames associated with the reactive video segment to generate a set of ending transition frames; and
presenting the ending transition frames to transition from the reactive video segment back to the idle video segment.
5 . The method of claim 1 , wherein detecting the interaction comprises detecting a selection of an emoticon by the viewing user associated with the content associated with the target user, and wherein analyzing the interaction comprises:
identifying a predefined association between the emoticon and the sentiment.
6 . The method of claim 1 , wherein detecting the interaction comprises detecting a written post by the viewing user associated with the content associated with the target user, and wherein analyzing the interaction comprises:
performing a sentiment analysis of text of the written post; and
determining the sentiment from the sentiment analysis.
7 . The method of claim 1 , wherein detecting the interaction comprises capturing a video of the viewing user while viewing the content associated with the target user, and wherein analyzing the interaction comprises:
performing a facial analysis to detect an expression of the viewing user; and
determining the sentiment from the detected expression.
8 . A non-transitory computer-readable storage medium storing instructions executable by a processor, the instructions when executed causing the processor to perform steps including:
storing in association with a profile of a target user, a plurality of video segments depicting a portrait of a target user, wherein each of the plurality of video segments is associated with a different emotion and depicts the target user expressing the corresponding emotion, the plurality of video segments including an idle segment depicting the target user in a neutral expression;
providing to a client device of a viewing user, a presentation of the idle segment together with content associated with the target user;
detecting an interaction of the viewing user on the client device with the content associated with the target user;
analyzing the interaction to determine a sentiment associated with the interaction;
selecting, based on the sentiment, a reactive video segment from the plurality of video segments, the reactive video segment associated with an emotion corresponding to the sentiment of the interaction; and
presenting the reactive video segment to the client device.
9 . The non-transitory computer-readable storage medium of claim 8 , wherein presenting the reactive video segment comprises:
blending a sequence of starting overlapping frames associated with the reactive video segment with a sequence of ending overlapping frames associated with the idle segment to generate a sequence of transition frames; and
presenting the transition frames to transition from the idle segment to the reactive video segment.
10 . The non-transitory computer-readable storage medium of claim 9 , wherein blending the sequence of starting overlapping frames associated with the reactive video segment with a sequence of ending overlapping frames associated with the idle segment comprises:
determining a sequence of transformations to warp the sequence of ending overlapping frames to align locations of facial landmarks in the sequence of ending overlapping frames to locations of facial landmarks in the sequence of starting overlapping frames;
weighting the sequence of transformations with increasing weights over a duration of the sequence of ending overlapping frames and the sequence of starting overlapping frames to generate a weighted sequence of transformations;
applying the weighted sequence of transformations to the ending overlapping frames to generate the sequence of transition frames.
11 . The non-transitory computer-readable storage medium of claim 8 , wherein the instructions when executed by the processor further cause the processor to perform steps including:
blending a starting set of overlapping frames associated with the idle segment with a set of ending overlapping frames associated with the reactive video segment to generate a set of ending transition frames; and
presenting the ending transition frames to transition from the reactive video segment back to the idle video segment.
12 . The non-transitory computer-readable storage medium of claim 8 , wherein detecting the interaction comprises detecting a selection of an emoticon by the viewing user associated with the content associated with the target user, and wherein analyzing the interaction comprises:
identifying a predefined association between the emoticon and the sentiment.
13 . The non-transitory computer-readable storage medium of claim 8 , wherein detecting the interaction comprises detecting a written post by the viewing user associated with the content associated with the target user, and wherein analyzing the interaction comprises:
performing a sentiment analysis of text of the written post; and
determining the sentiment from the sentiment analysis.
14 . The non-transitory computer-readable storage medium of claim 8 , wherein detecting the interaction comprises capturing a video of the viewing user while viewing the content associated with the target user, and wherein analyzing the interaction comprises:
performing a facial analysis to detect an expression of the viewing user; and
determining the sentiment from the detected expression.
15 . A computer system comprising:
a processor; and
a non-transitory computer-readable storage medium storing instructions executable by the processor, the instructions when executed causing the processor to perform steps including:
storing in association with a profile of a target user, a plurality of video segments depicting a portrait of a target user, wherein each of the plurality of video segments is associated with a different emotion and depicts the target user expressing the corresponding emotion, the plurality of video segments including an idle segment depicting the target user in a neutral expression;
providing to a client device of a viewing user, a presentation of the idle segment together with content associated with the target user;
detecting an interaction of the viewing user on the client device with the content associated with the target user;
analyzing the interaction to determine a sentiment associated with the interaction;
selecting, based on the sentiment, a reactive video segment from the plurality of video segments, the reactive video segment associated with an emotion corresponding to the sentiment of the interaction; and
presenting the reactive video segment to the client device.
16 . The computer system of claim 15 , wherein presenting the reactive video segment comprises:
blending a sequence of starting overlapping frames associated with the reactive video segment with a sequence of ending overlapping frames associated with the idle segment to generate a sequence of transition frames; and
presenting the transition frames to transition from the idle segment to the reactive video segment.
17 . The computer system of claim 16 , wherein blending the sequence of starting overlapping frames associated with the reactive video segment with a sequence of ending overlapping frames associated with the idle segment comprises:
determining a sequence of transformations to warp the sequence of ending overlapping frames to align locations of facial landmarks in the sequence of ending overlapping frames to locations of facial landmarks in the sequence of starting overlapping frames;
weighting the sequence of transformations with increasing weights over a duration of the sequence of ending overlapping frames and the sequence of starting overlapping frames to generate a weighted sequence of transformations;
applying the weighted sequence of transformations to the ending overlapping frames to generate the sequence of transition frames.
18 . The computer system of claim 15 , wherein the instructions when executed by the processor further cause the processor to perform steps including:
blending a starting set of overlapping frames associated with the idle segment with a set of ending overlapping frames associated with the reactive video segment to generate a set of ending transition frames; and
presenting the ending transition frames to transition from the reactive video segment back to the idle video segment.
19 . The computer system of claim 15 , wherein detecting the interaction comprises detecting a selection of an emoticon by the viewing user associated with the content associated with the target user, and wherein analyzing the interaction comprises:
identifying a predefined association between the emoticon and the sentiment.
20 . The computer system of claim 15 , wherein detecting the interaction comprises detecting a written post by the viewing user associated with the content associated with the target user, and wherein analyzing the interaction comprises:
performing a sentiment analysis of text of the written post; and
determining the sentiment from the sentiment analysis.