Multimedia adaptation based on video orientation
View Patent ↗Technology for multimedia adaptation based on video orientation is disclosed. In an example, a server can include computer circuitry configured to: receive a device capability from a client device, wherein the device capability includes video orientation capability; and perform video orientation-aware multimedia adaptation based on the device's video orientation capability.
1. A mobile terminal (MT) for providing orientation-aware processing capability, comprising
a processor configured to:
determine a video orientation capability of the MT;
parse a media presentation description (MPD) metadata file to identify a presence of video orientation information in the third generation partnership (3GP) file or parse the 3GP file for embedded video orientation information;
determine an orientation correction term based on parsed video orientation information and a current orientation of the mobile terminal, and
correct a rendering orientation of a real-time transport protocol (RTP) stream for misalignment based on the determined orientation correction term; and,
a transceiver to transmit video orientation capability information to a server via a packet switched streaming service (PSS) device capability exchange signaling to enable the mobile terminal to:
access segments from the server via the PSS based on one or more of an orientation of the MT and the video orientation capability of the MT.
2. The mobile terminal of claim 1 , wherein the processor is further configured to embed video orientation information for a captured video with a specified video orientation for a specified media segment in a third generation partnership project (3GPP) file format (3GP) file as an instantiation of an International Organization for Standardization (ISO) base media file format (ISO-BMFF) file.
3. The mobile terminal of claim 2 , wherein the video orientation information is embedded in:
a video track or real-time transport protocol (RTP) hint track of the 3GP file;
a file-level ISO-BMFF box, wherein the file-level ISO-BMFF box is a ‘moov’ box initialization segment or a ‘moof’ box media segment;
a timed metadata track of the ISO-BMFF; or
a SampleEntry with video orientation parameters within the sample description box in an ISO file format box structure.
4. The mobile terminal of claim 2 , wherein:
the transceiver is further configured to:
upload the 3GP file for a hypertext transfer protocol (HTTP) stream or dynamic adaptive streaming over HTTP (DASH).
5. The mobile terminal of claim 1 , wherein:
the transceiver is further configured to:
transmit a video orientation attribute via a packet switched streaming service (PSS) device capability exchange signaling, wherein the video orientation attribute is a PSS common component that includes:
an Orientation attribute selected from a list of different video orientation states supported by the client,
a DefaultOrientation attribute that provides a default video orientation to be taken as reference for a content production or selection,
an OrientationModesCapture attribute selected from a list of different video orientation modes supported by the client for a capturing,
an OrientationModesDisplay attribute selected from a list of different video orientation modes supported by the client for a rendering and a display,
a DefaultOrientationCapture attribute that provides a default video orientation for capturing based on the installation of the camera sensor if an orientation sensor is disabled or not present,
a DefaultOrientationDisplay attribute that provides a default video orientation for rendering and display if the orientation sensor is disabled or not present,
an OrientationAngularGranularityCapture attribute selected from a number of equally-spaced angular granularity levels in a 90 degree interval at which a device sensor can detect video orientation changes during the capture,
an OrientationAngularGranularityDisplay attribute selected from a number of equally-spaced angular granularity levels in the 90 degree interval at which the device sensor can detect video orientation changes during the rendering and the display or both the capture and the rendering and display,
a NumOfCameras attribute selected from a number of cameras on a device, or
a HorizontalMirroringSupport attribute to indicate whether the device supports horizontal mirroring when the device is orientation-aware.
6. The mobile terminal of claim 1 , wherein the video orientation capability information is provided in a third generation partnership project (3GPP) long term evolution (LTE) packet switched streaming service (PSS) session, a dynamic adaptive streaming over hypertext transfer protocol (HTTP) (DASH) session, or an integrated multimedia subsystem (IMS) based PSS and multimedia broadcast and multicast service (MBMS) (IMS_PSS_MBMS) session.
7. The mobile terminal of claim 1 , further comprising:
an orientation sensor to determine an orientation of the mobile terminal.
8. The mobile terminal of claim 1 , wherein:
the transceiver is further configured to:
receive a real-time transport protocol (RTP) extension header for a RTP stream, or receive a third generation partnership project (3GPP) file format (3GP) file for a hypertext transfer protocol (HTTP) stream or dynamic adaptive streaming over HTTP (DASH); and
the processor is further configured to:
parse the media presentation description (MPD) metadata file to identify a presence of video orientation information in the 3GP file or parse the 3GP file for embedded video orientation information,
determine the orientation correction term based on the parsed video orientation information and a current orientation of the mobile terminal, and
correct a rendering orientation of the HTTP stream or DASH for misalignment based on the determined orientation correction term; or
parse a session description protocol (SDP) file to identify a presence of video orientation information in the RTP stream or parse a RTP extension header for the RTP stream for the embedded video orientation information,
determine the orientation correction term based on the parsed video orientation information and the current orientation of the mobile terminal, and
correct the rendering orientation of the RTP stream for misalignment based on the determined orientation correction term,
wherein correcting the rendering orientation compensates for rotation or flip of the orientation of the MT.
9. The mobile terminal of claim 1 , wherein the mobile terminal (MT) includes a client device, a user equipment (UE), or a mobile station (MS); and the mobile terminal includes an antenna, a camera, a touch sensitive display screen, a speaker, a microphone, a graphics processor, an application processor, internal memory, or a non-volatile memory port.
10. The mobile terminal of claim 1 , wherein a mobile terminal's current video orientation capability information overrides a default video orientation capability information via profile difference (Profdiff) signaling.
11. The mobile terminal of claim 1 , wherein the processor is further configured to instruct the mobile terminal to capture video with a specified video orientation for a specified media segment based on a device orientation.