Intelligent placement of a browser-added user interface element on a webpage
Disclosed herein a browser that intelligently places a user interface (UI) element on a webpage such that the UI element appears to be integrated and coordinated with the webpage and not merely overlaid on the webpage. The intelligent placement ensures that the UI element neither obstructs the view of pertinent content displayed via the webpage nor interferes with the functionality of the webpage. Moreover, the intelligent placement ensures that the UI element is completely visible to a user. The placement of the UI element is done by the browser after the webpage has been retrieved from a web server. Consequently, the browser lacks the ability to control or alter the layout and appearance of the webpage. The UI element can be a selectable UI element that the browser associates with a function. In one example, the function can be an image-based search.
1. A method implemented by a browser operating on a computing device, comprising:
examining code for a webpage to be displayed via a display screen associated with the computing device;
identifying, based at least in part on the examining, a first layout position of an image that is to be displayed on the webpage;
determining, based at least in part on the examining, a rectangle that encloses a part of the image that is visible on the webpage;
implementing a hit test on the rectangle to identify a second layout position of an existing user interface element that is placed on the part of the image that is visible on the webpage, wherein the existing user interface element is part of the webpage;
applying a placement policy to identify a third layout position for a browser-added user interface element to be displayed on the webpage, wherein:
the third layout position is within the rectangle that encloses the part of the image that is visible and the third layout position avoids overlapping the second layout position of the existing user interface element; and
the placement policy allows for consistent placement of the browser-added user interface element across a plurality of images;
determining when a user-controlled pointer element hovers over the rectangle that encloses the part of the image that is visible on the webpage; and
displaying the browser-added user interface element in the third layout position when the user-controlled pointer element hovers over the rectangle that encloses the part of the image that is visible on the webpage.
2. The method of claim 1 , wherein the browser-added user interface element enables the browser to implement or initiate a function that is not part of the webpage.
3. The method of claim 2 , wherein the function comprises an image search function, and the method further comprises:
determining when the user-controlled pointer element hovers over the browser-added user interface element;
performing, based at least in part on the user-controlled pointer element hovering over the browser-added user interface element, the image search function based on characteristics of the image; and
displaying an image search user interface element when the user-controlled pointer element hovers over the browser-added user interface element, the image search user interface element providing results of the image search function.
4. The method of claim 1 , wherein:
the code for the webpage includes a Cascading Style Sheet (CSS) associated with the image; and
determining the rectangle that encloses the part of the image that is visible on the webpage comprises identifying, within the CSS, a fourth layout position of an ancestor element that overlaps the first layout position of the image, wherein the CSS defines a first z-index, in a z-order, for the ancestor element and a second z-index, in the z-order, for the image, the first z-index being greater than the second z-index thereby causing the ancestor element to crop the image.
5. The method of claim 4 , further comprising inserting the browser-added user interface element in the second z-index in the z-order for the image.
6. The method of claim 1 , wherein the placement policy includes a horizontal flow direction and a vertical flow direction.
7. The method of claim 6 , wherein at least one of the horizontal flow direction and the vertical flow direction is established based at least on one of a language of text in the webpage or a geographic location in which the computing device is located.
8. The method of claim 1 , wherein:
the hit test identifies a fourth layout position of an overlay element that is transparent to a viewer of the webpage and that is greater than a threshold size; and
application of the placement policy does not avoid overlapping the fourth layout position of the overlay element.
9. The method of claim 1 , wherein:
the rectangle defines points in a coordinate space; and
the hit test provides information regarding whether the user-controlled pointer element intersects with the part of the image that is visible at each of the points.
10. The method of claim 1 , wherein identifying the first layout position of the image includes identifying at least one of an alignment attribute for the image, a margin attribute for the image, a width attribute for the image, or a height attribute for the image.
11. A computing device comprising:
at least one processor; and
a computer-readable storage medium having computer-executable instructions stored thereupon which, when executed by the at least one processor, cause a browser of the computing device to perform operations comprising:
examining code for a webpage to be displayed via a display screen associated with the computing device;
identifying, based at least in part on the examining, a first layout position of an image that is to be displayed on the webpage;
determining, based at least in part on the examining, a part of the image that is visible on the webpage;
implementing a hit test on the part of the image that is visible to identify a second layout position of an existing user interface element that is placed on the part of the image that is visible on the webpage, wherein the existing user interface element is part of the webpage;
applying a placement policy to identify a third layout position for a browser-added user interface element to be displayed on the webpage, wherein:
the third layout position overlays the part of the image that is visible and the third layout position avoids overlapping the second layout position of the existing user interface element; and
the placement policy allows for consistent placement of the browser-added user interface element across a plurality of images; and
displaying the browser-added user interface element in the third layout position.
12. The computing device of claim 11 , wherein the browser-added user interface element enables the browser to implement or initiate a function that is not part of the webpage.
13. The computing device of claim 11 , wherein:
the code for the webpage includes a Cascading Style Sheet (CSS) associated with the image; and
determining the part of the image that is visible on the webpage comprises identifying, within the CSS, a fourth layout position of an ancestor element that overlaps the first layout position of the image, wherein the CSS defines a first z-index, in a z-order, for the ancestor element and a second z-index, in the z-order, for the image, the first z-index being greater than the second z-index thereby causing the ancestor element to crop the image.
14. The computing device of claim 13 , wherein:
the ancestor element crops the image such that a corner of the visible part of the image is rounded; and
the operations further comprise determining a rectangle that fits within the corner of the visible part of the image that is rounded, wherein the rectangle represent the visible part of the image on which the hit test is implemented and the browser-added user interface element is displayed.
15. The computing device of claim 13 , wherein the operations further comprise inserting the browser-added user interface element in the second z-index in the z-order for the image.
16. The computing device of claim 11 , wherein the placement policy includes a horizontal flow direction and a vertical flow direction.
17. The computing device of claim 16 , wherein at least one of the horizontal flow direction or the vertical flow direction is established based at least on one of a language of text in the webpage or a geographic location in which the computing device is located.
18. A computer-readable storage medium having computer-executable instructions stored thereupon which, when executed by at least one processor, cause a browser of a computing device to perform operations comprising:
examining code for a webpage to be displayed via a display screen associated with the computing device;
identifying, based at least in part on the examining, a first layout position of an image that is to be displayed on the webpage;
determining, based at least in part on the examining, a part of the image that is visible on the webpage;
implementing a hit test on the part of the image that is visible to identify a second layout position of an existing user interface element that is placed on the part of the image that is visible on the webpage, wherein the existing user interface element is part of the webpage;
applying a placement policy to identify a third layout position for a browser-added user interface element to be displayed on the webpage, wherein:
the third layout position overlays the part of the image that is visible and the third layout position avoids overlapping the second layout position of the existing user interface element; and
the placement policy allows for consistent placement of the browser-added user interface element across a plurality of images; and
displaying the browser-added user interface element in the third layout position.
19. The computer-readable storage medium of claim 18 , wherein the browser-added user interface element enables the browser to implement or initiate a function that is not part of the webpage.
20. The computer-readable storage medium of claim 18 , wherein:
the code for the webpage includes a Cascading Style Sheet (CSS) associated with the image;
determining the part of the image that is visible on the webpage comprises identifying, within the CSS, a fourth layout position of an ancestor element that overlaps the first layout position of the image, wherein the CSS defines a first z-index, in a z-order, for the ancestor element and a second z-index, in the z-order, for the image, the first z-index being greater than the second z-index thereby causing the ancestor element to crop the image; and
the operations further comprise inserting the browser-added user interface element in the second z-index in the z-order for the image.