IP Library Granted Patent US 11,537,278
Granted Patent B2
US 11,537,278 · App. 17/240,532 · Granted Dec 27, 2022

Interactively presenting a visible portion of a rendering surface on a user device

Inventors: Haluk Burcin Tunali (North Bergen, NJ); Luiz do Amaral de Franca Pereira Filho (Jersey City, NJ); Etan Bukiet (New York, NY); Behnoosh Hariri (New York, NY); Norbert Zsolt Kenderesi (Jersey City, NJ); Igor Kopylov (Brooklyn, NY); Matthew Jay Isison (Jersey City, NJ); Kevin Winter (Metuchen, NJ); Olga Sergeyevna Saviano (New York, NY); Gregory George Galante (Little Silver, NJ); Mathieu Turcotte (New York, NY); Jacob Robert Voytko (New York, NY); Leeran Raphaely (New York, NY)
Assignee: Google LLC
G06F3/04845G06F3/0485G06F3/0482G06F3/04883
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,537,278
App. No.
17/240,532
Granted
Dec 27, 2022
Kind
B2
Abstract

The technology herein involves accessing content of an electronic document and a shared layout for the electronic document, wherein the shared layout provides a layout for user devices having different dimensions of display areas; rendering the content of the electronic document onto a primary rendering surface and a secondary rendering surface, wherein the primary rendering surface is associated with a native platform of a user device and the secondary rendering surface is associated with the shared layout of a server; applying operation commands to modify the primary rendering surface, the operation commands resulting from a user interaction with the electronic document; merging the operation commands into a merged operation call; and invoking the merged operation call to modify the secondary rendering surface, wherein the merged operation call reduces a number of operation calls that modify the secondary rendering surface and is shared with the server to modify the shared layout.

Claims (71)

1. A method comprising:

accessing, by a user device from a server, content of an electronic document and a shared layout for the electronic document, wherein the shared layout provides a layout for a plurality of user devices having different dimensions of display areas;

rendering the content of the electronic document onto a primary virtual rendering surface and a secondary virtual rendering surface, wherein the primary virtual rendering surface is associated with a native platform of the user device and the secondary virtual rendering surface is associated with the shared layout of the server;

applying, by the user device, a plurality of operation commands to modify the primary virtual rendering surface, the plurality of operation commands resulting from a user interaction with the electronic document;

merging, by the user device, the plurality of operation commands into a merged operation call, wherein the plurality of operation commands comprise relative coordinates that are specified relative to one or more parent nodes in a layout tree, and wherein merging the plurality of operation commands further comprise merging the relative coordinates into one or more absolute coordinates for the merged operation call, the one or more absolute coordinates corresponding to a position of at least a portion of the content of the electronic document relative to an entirety of the primary virtual rendering surface; and

invoking, by the user device, the merged operation call to modify the secondary virtual rendering surface, wherein the merged operation call reduces a number of operation calls that modify the secondary virtual rendering surface and is shared with the server to modify the shared layout.

2. The method of claim 1 , wherein the content of the electronic document comprises one or more of a word, a figure, a table, or an image that are represented by the one or more parent nodes of the layout tree for the electronic document.

3. The method of claim 1 , wherein the primary virtual rendering surface comprises a plurality of tiles and wherein applying the plurality of operation commands comprises determining rendering parameters of a subset of the plurality of tiles that are active in view of the user interaction, determining which active tiles in the subset are visible, caching non-visible tiles from the subset, and rendering visible tiles from the subset based on the rendering parameters.

4. The method of claim 2 , wherein the one or more absolute coordinates for the merged operation call are not specified relative to a parent node.

5. The method of claim 1 , wherein the secondary virtual rendering surface has a secondary stack of states for rendering the content of the electronic document, wherein the primary virtual rendering surface has a primary stack of states, and wherein the method further comprises:

maintaining one or more links between the primary stack of states and the secondary stack of states; and

determining, in view of the one or more links, that the plurality of operation commands have been applied to the primary virtual rendering surface and not to the secondary virtual rendering surface since a save command was applied to the second virtual rendering surface.

6. The method of claim 5 , further comprising:

obtaining an operation command of the plurality of operation commands relating to the electronic document; and

in response to determining that a command type of the operation command is a save command type or a transformation command type, applying the operation command to the primary virtual rendering surface to save a current state of the primary virtual rendering surface to the primary stack of states without applying the operation command to the secondary virtual rendering surface.

7. The method of claim 6 , further comprising:

obtaining a second and third operation command of the plurality of operation commands relating to the electronic document; and

in response to determining that a command type of one of the second or third operation command is a restore command type:

applying the respective operation command to the primary virtual rendering surface to restore a state from the primary stack of states to the current state of the primary virtual rendering surface; and

in response to determining that the links between the primary stack and the secondary stack comprise a link to the state of the primary stack that was restored to be the current state of the primary virtual rendering surface, applying the respective operation command to the secondary virtual rendering surface to restore a state from the secondary stack of states to a current state of the secondary virtual rendering surface and removing a state from the secondary stack of states that is linked to the state from the primary stack of states by the link.

8. The method of claim 7 , further comprising:

obtaining a fourth operation command of the plurality of operation commands relating to the electronic document;

determining that a command type of the fourth operation command is a draw command type; and

applying the fourth operation command to the primary virtual rendering surface to draw an object on the primary virtual rendering surface in view of the current state of the primary virtual rendering surface.

9. The method of claim 1 , wherein the native platform comprises any of a mobile operating system or a browser application of a computer.

10. The method of claim 1 , wherein the merging is performed by a layout component that is sharable to a second user device, wherein the second user device comprises a native platform that is different from the native platform of the user device.

11. A system comprising:

a memory that stores instructions; and

a processor configured to execute the instructions to:

access, at a user device from a server, content of an electronic document and a shared layout for the electronic document, wherein the shared layout provides a layout for a plurality of user devices having different dimensions of display areas;

render the content of the electronic document onto a primary virtual rendering surface and a secondary virtual rendering surface, wherein the primary virtual rendering surface is associated with a native platform of the user device and the secondary virtual rendering surface is associate with the shared layout of the server;

applying a plurality of operation commands to modify the primary virtual rendering surface, the plurality of operation commands resulting from a user interaction with the electronic document;

merge the plurality of operation commands into a merged operation call, wherein the plurality of operation commands comprise relative coordinates that are specified relative to one or more parent nodes in a layout tree, and to merge the plurality of operation commands comprises merging the relative coordinates into one or more absolute coordinates for the merged operation call, the one or more absolute coordinates corresponding to a position of at least a portion of the content of the electronic document relative to an entirety of the primary virtual rendering surface; and

invoke, by the user device, the merged operation call to modify the secondary virtual rendering surface, wherein the merged operation call reduces a number of operation calls that modify the secondary virtual rendering surface and is shared with the server to modify the shared layout.

12. The system of claim 11 , wherein the one or more absolute coordinates for the merged operation call are not specified relative to a parent node of the one or more parent nodes.

13. The system of claim 12 , wherein the secondary virtual rendering surface has a secondary stack of states for rendering the content of the electronic document, wherein the primary virtual rendering surface has a primary stack of states, and wherein the processor is further to:

maintain one or more links between the primary stack of states and the secondary stack of states; and

determine, in view of the one or more links, that the plurality of operation commands have been applied to the primary virtual rendering surface and not to the secondary virtual rendering surface since a save command was applied to the second virtual rendering surface.

14. The system of claim 13 , wherein the processor is further to:

obtain a first operation command of the plurality of operation commands relating to the electronic document;

in response to a determination that a command type of the first operation command is a save command type or a transformation command type, apply the first operation command to the primary virtual rendering surface to save a current state of the primary virtual rendering surface to the primary stack without application of the first operation to the secondary virtual rendering surface;

obtain a second operation command relating to the electronic document; and

in response to a determination that a command type of the second operation command is a restore command type:

apply the second operation command to the primary virtual rendering surface to restore a state from the primary stack to be the current state of the primary virtual rendering surface; and

in response to a determination that the links between the primary stack of states and the secondary stack of states comprise a link to the state that was restored to the current state of the primary virtual rendering surface, apply the second operation command to the secondary virtual rendering surface to restore a state from the secondary stack of states to a current state of the secondary virtual rendering surface and remove a state from the secondary stack of states that is linked to the state from the primary stack of states by the link.

15. The system of claim 14 , wherein the processor is further to:

obtain a fourth operation command of the plurality of operation commands relating to the one or more parent nodes;

determine that a command type of the fourth operation command is a draw command type;

apply the fourth operation command to the primary virtual rendering surface to draw an object on the primary virtual rendering surface in view of the current state of the primary virtual rendering surface;

determine that at least one fifth operation command having the transformation command type other than a translation command type has occurred since a last one of the save command type was applied to the secondary virtual rendering surface;

determine that the links between the primary stack of states and the secondary stack of states do not comprise a link from the secondary stack of states to the current state of the primary virtual rendering surface in the primary stack of states;

apply the save command type to the secondary virtual rendering surface to save the current state of the primary virtual rendering surface to the secondary stack of states;

add the link from the secondary stack of states to the current state of the primary virtual rendering surface in the primary stack of states; and

apply the fifth operation command and then the fourth operation command to the secondary virtual rendering surface.

16. A non-transitory processor-readable storage medium storing instructions that, when executed by a processor, cause the processor to:

access, by a user device from a server, content of an electronic document and a shared layout for the electronic document, wherein the shared layout provides a layout for a plurality of user devices having different dimensions of display areas;

render the content of the electronic document onto a primary virtual rendering surface and a secondary virtual rendering surface, wherein the primary virtual rendering surface is associated with a native platform of the user device and the secondary virtual rendering surface is associated with the shared layout of the server;

applying, by the user device, a plurality of operation commands to modify the primary virtual rendering surface, the plurality of operation commands resulting from a user interaction with the electronic document;

merge, by the user device, the plurality of operation commands into a merged operation call, wherein the plurality of operation commands comprise relative coordinates that are specified relative to one or more parent nodes in a layout tree, and wherein merging the plurality of operation commands further comprise merging the relative coordinates into one or more absolute coordinates for the merged operation call, the one or more absolute coordinates corresponding to a position of at least a portion of the content of the electronic document relative to an entirety of the primary virtual rendering surface; and

invoke, by the user device, the merged operation call to modify the secondary virtual rendering surface, wherein the merged operation call reduces a number of operation calls that modify the secondary virtual rendering surface and is shared with the server to modify the shared layout.

17. The non-transitory processor-readable storage medium of claim 16 , wherein the one or more absolute coordinates for the merged operation call that are not specified relative to a parent node.

18. The non-transitory processor-readable storage medium of claim 16 , wherein the secondary virtual rendering surface has a secondary stack of states for rendering the content of the electronic document, wherein the primary virtual rendering surface has a primary stack of states, and wherein the instructions are further to cause the processor to:

maintain one or more links between the primary stack of states and the secondary stack of states; and

determine that the plurality of operation commands have been applied to the primary virtual rendering surface and not to the secondary virtual rendering surface since a save command was applied to the secondary virtual rendering surface.

19. The non-transitory processor-readable storage medium of claim 18 , wherein the instructions are further to cause the processor to:

obtain an operation command of the plurality of operation commands relating to the electronic document; and

in response to a determination that a command type of the operation command is a save command type or a transformation command type, apply the operation command to the primary virtual rendering surface to save a current state of the primary virtual rendering surface to the primary stack of states without application of the operation command to the secondary virtual rendering surface.

20. The non-transitory processor-readable storage medium of claim 19 , wherein the instructions are further to cause the processor to:

obtain a third operation command of the plurality of operation commands relating to the electronic document;

determine that a command type of the third operation command is a draw command type; and

apply the third operation command to the primary virtual rendering surface to draw an object on the primary virtual rendering surface in view of the current state of the primary virtual rendering surface.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 10, 2021
From: TUNALI, HALUK BURCIN; FILHO, LUIZ DO AMARAL DE FRANCA PEREIRA; BUKIET, ETAN; HARIRI, BEHNOOSH; KENDERESI, NORBERT ZSOLT; KOPYLOV, IGOR; ISISON, MATTHEW JAY; WINTER, KEVIN; SAVIANO, OLGA SERGEYEVNA; GALANTE, GREGORY GEORGE; TURCOTTE, MATHIEU; VOYTKO, JACOB ROBERT; RAPHAELY, LEERAN
To: GOOGLE INC.
Reel/Frame 056812/0537 →
CHANGE OF NAME Recorded Jul 10, 2021
From: GOOGLE INC.
To: GOOGLE LLC
Reel/Frame 056821/0538 →
Continuity (5)
Continuation 15228387 · Aug 4, 2016
Provisional Application 62200920 · Aug 4, 2015
Provisional Application 62200793 · Aug 4, 2015
Provisional Application 62200979 · Aug 4, 2015
Related Publication 20210247891A1 · Aug 12, 2021