System and method of tactile based display (image) adaptation of videoconference proceedings
A conferencing system and method detects one or more two-dimensional (2D) images in presented 2D data and/or objects shown during a videoconference and renders them on a participant's three-dimensional (3D) tactile screen. The system includes a conferencing server that is in communication with one or more participant devices. An image recognition engine is in communication with the conferencing server and identifies the one or more 2D images. An imaging processor is in communication with the image recognition engine, and may be in communication with a camera that recognizes objects shown during the videoconference. The imaging processor creates a 3D rendering of the 2D images and/or the objects and directly or indirectly sends them to the participant's 3D tactile screen.
1 . A conferencing system for detecting one or more two-dimensional (2D) images in conference data and rendering the images on a three-dimensional (3D) tactile screen, wherein the conferencing system comprises:
a conferencing server that is in communication with one or more participant devices and that is configured to (a) play a 2D conference that includes 2D data comprising the one or more 2D images, and (b) make the 2D data available to one or more participant devices;
an image recognition engine in communication with, or that is part of, the conferencing server and that electronically identifies the one or more 2D images and their position in relation to text in the 2D data;
an imaging processor in communication with the conferencing server and with the image recognition engine, and that includes imaging software configured to create a 3D rendering of the one or more 2D images contained in the 2D data;
an image description engine in communication with the imaging processor and/or the conferencing server, wherein the imaging description engine is programmed to recognize each of the one or more 2D images and generate a tactile description of each; and
a tactile 3D screen in communication with the imaging processor and the imaging description engine, wherein the tactile 3D screen is configured to receive the 3D rendering generated by the imaging processor, receive the tactile description generated by the imaging description engine, project the 3D rendering and the tactile description so they can be touched and perceived by a conference participant, and provide greater definition of the 3D rendering in response to touch commands by the conference participant on the 3D screen conference participant device.
2 . The conferencing system of claim 1 that further includes a video camera in communication with the imaging processor, wherein the video camera is configured to detect an object within a defined video conferencing field and communicate an image of the object to the imaging processor, wherein the imaging processor creates a 3D rendering of the object image, and the 3D rendering of the object image is transmitted to the tactile 3D screen to that it can be touched and perceived by the conference participant.
3 . The conferencing system of claim 2 , wherein the 3D rendering of the object is transmitted to the image description engine, which is configured to recognize the object, generate a tactile description of it, and transmit the tactile description to the tactile 3D screen of the participant device so that the tactile description can be touched and perceived by the conference participant.
4 . The conferencing system of claim 1 , wherein the tactile description is interposed within a Braille rendering of text presented in the 2D data.
5 . The conferencing system of claim 2 , wherein the object is recognized by the imaging processor using Auto ML Vision Object Detection.
6 . The conferencing system of claim 1 that further includes a text-to-Braille processor that converts text in the 2D data to Braille and the Braille is displayed on the tactile 3D screen so that the conference participant can touch and perceive it.
7 . The conferencing system of claim 6 , wherein the tactile 3D screen includes a refreshable Braille display.
8 . The conferencing system of claim 1 that further includes a database to store the 2D data and computerized speech of the 2D data text.
9 . The conferencing system of claim 1 that further includes a scanner to scan a 2D image and transfer the scanned 2D image to the imaging processor, which creates a 3D rendering of the 2D scanned image and is configured to transmit the 3D rendering to the tactile 3D screen where it can be touched and perceived by the conference participant.
10 . A computerized method for detecting one or more 2D images in 2D data and/or one or more 3D objects, converting each of the one or more 2D images and the one or more 3D objects to 3D renderings, and transmitting the 3D renderings to a 3D tactile screen, wherein the method comprises the steps of:
a conferencing server operating a 2D conference file that includes 2D data, wherein the 2D data comprises the one or more 2D images therein, and the conferencing server making the 2D data available to one or more conference devices, wherein each conference device is unique to a particular conference participant;
utilizing an imaging processor in communication with the conferencing server, creating a 3D rendering of at least one of the one or more 2D images embedded in the 2D data, and transmitting the 3D rendering to a tactile 3D screen;
utilizing an image description engine in communication with the imaging processor (a) recognizing the 3D rendering or the at least one of the one or more 2D images, (b) generate a tactile image description of the at least one of the one or more 2D images or the 3D rendering, and (c) transmit the tactile image description to the tactile 3D screen; and
utilizing the tactile 3D screen in communication with the imaging processor and the imaging description engine, receiving the 3D rendering from the imaging processor and receiving the tactile image description from the imaging description engine, and projecting them so they can be touched and perceived by the particular conference participant, and providing greater definition of the 3D rendering in response to touch commands by the conference participant on the 3D screen conference participant device.
11 . The method of claim 10 , wherein the imaging description engine includes image caption generator software that analyzes the 2D image or the 3D rendering and generates a tactile image description of it.
12 . The method of claim 10 that further includes the step of the tactile 3D screen refreshing when a new slide of 2D data is presented.
13 . The method of claim 10 , wherein (a) a first graphical user interface on the participation device for the unique conference participant includes a control that permits the unique conference participant to choose the duration for which the 3D rendering and the tactile image description remain on the tactile 3D screen, and (b) a second graphical user interface on the conferencing server that includes a second control that permits a conference host to choose the duration for which the 3D rendering and the tactile image description remain on the tactile 3D screen.
14 . The method of claim 10 , wherein one or more 3D objects are recognized by a video camera with a video conferencing field and that further includes the step of a conference host adjusting the video conferencing field to eliminate extraneous objects, wherein the step of adjusting is based on the location of the one or more 3D objects, the size of the one or more 3D objects, and/or the movement of the 3D objects.
15 . The method of claim 10 , wherein the participant device includes a GUI configured to permit the unique conference participant to provide comments via Braille.
16 . The method of claim 15 , wherein the unique conference participant can also input commands via Braille device utilizing the GUI of the participant device.
17 . A conferencing system for detecting one or more 2D images in 2D data and rendering the images on a three-dimensional (3D) tactile screen, wherein the conferencing system comprises:
a conferencing server that is in communication with one or more participant devices and that is configured to (a) play a 2D videoconference that includes the 2D data comprising the one or more 2D images, and (b) make the 2D data available to one or more participant devices;
an imaging processor in communication with the conferencing server and that includes imaging software configured to create a 3D rendering of the one or more 2D images contained in the 2D data;
an image description engine in communication with the imaging processor and/or the conferencing server, wherein the imaging description engine is programmed to recognize the each of the one or more 2D images and generate a tactile description of each;
a tactile 3D screen in communication with the imaging processor and the imaging description engine, wherein the tactile 3D screen is configured to receive the 3D rendering from the imaging processor and receive the tactile description from the imaging description engine and project them so they can be touched and perceived by a conference participant;
a conference device having a GUI configured to permit a unique conference participant to provide comments via Braille and input commands via Braille; and
one or more other conference devices each unique to one or more other conference participants, wherein the comments or commands of the unique conference participant create a vibratory motion on the one or more other conference devices so as to be recognized by the one or more other conference participants.
18 . The conferencing system of claim 17 , wherein the comment or command includes the name of the unique conference participant who sent it.
19 . The conferencing system of claim 17 , wherein emphasized portions of the conference text are presented with a vibratory motion on the one or more conference devices.