IP Library › Granted Patent US 12,198,277
Granted Patent B1
US 12,198,277 · App. 17/556,510 · Granted Jan 14, 2025

Displaying a prioritized offscreen indicator

Inventors: Thomas G. Salter (Foster City, CA); Gregory Patrick Lane Lutter (Boulder Creek, CA); Rahul Nair (Santa Clara, CA); Devin William Chalmers (Oakland, CA); Ioana Negoita (San Jose, CA)
Assignee: APPLE INC.
G06T19/006G06F3/013G06F3/03G06T7/70G06T11/60
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,198,277
App. No.
17/556,510
Granted
Jan 14, 2025
Kind
B1
Abstract

A method is performed at an electronic device with a one or more processors, a non-transitory memory, and a display. The method includes presenting, on the display, a plurality of objects including a first object and a second object. The method includes obtaining a first display priority value that is associated with the first object. The method includes prioritizing the first object over the second object based on a function of the first display priority value. The method includes, in response to determining that each of the first object and the second object satisfies an offscreen criterion, displaying, on the display, a first offscreen indicator that is associated with the first object according to the prioritization.

Claims (56)

1. A method comprising:

at an electronic device including one or more processors, a non-transitory memory, and a display:

while presenting, via the display, a plurality of objects including a first object and a second object:

obtaining a first semantic label value associated with the first object;

determining a first display priority value that is associated with the first object, wherein determining the first display priority value is based at least in part on the first semantic label value; and

prioritizing the first object over the second object based on a function of the first display priority value; and

in response to determining that each of the first object and the second object satisfies an offscreen criterion, displaying, on the display, a first offscreen indicator that is associated with the first object according to the prioritization.

2. The method of claim 1 , wherein, in response to determining that each of the first object and the second object satisfies the offscreen criterion, foregoing display of a respective offscreen indicator associated with the second object according to the prioritization.

3. The method of claim 1 , further comprising, while presenting the plurality of objects, obtaining a second display priority value that is associated with the second object, wherein prioritizing the first object over the second object is also based on a function of the second display priority value.

4. The method of claim 3 , wherein prioritizing the first object over the second object includes determining that the first display priority value is higher than the second display priority value.

5. The method of claim 1 , further comprising:

obtaining a third display priority value that is associated with a third object of the plurality of objects;

prioritizing the third object over the second object based on a function of the third display priority value; and

in response to determining that the third object satisfies the offscreen criterion, displaying, on the display, a second offscreen indicator that is associated with the third object according to the prioritization.

6. The method of claim 1 , further comprising:

obtaining one or more object characteristic criteria; and

determining the first display priority value based on a function of the first object and the one or more object characteristic criteria.

7. The method of claim 6 , wherein the one or more object characteristic criteria include an object importance criterion.

8. The method of claim 1 , further comprising:

obtaining one or more user preference criteria; and

determining the first display priority value based on a function of the first object and the one or more user preference criteria.

9. The method of claim 1 , wherein determining the first display priority value includes determining a first engagement score that characterizes a level of user engagement with respect to the first object.

10. The method of claim 9 , wherein determining the first engagement score is based on eye tracking data that is indicative of a gaze position, including determining a spatial relationship between the gaze position and the first object.

11. The method of claim 9 , wherein determining the first engagement score is based on extremity tracking data that is indicative of an extremity position, including determining a spatial relationship between the extremity position and the first object.

12. The method of claim 1 , further comprising determining a second engagement score that characterizes a level of user engagement with respect to the second object, wherein prioritizing the first object over the second object is further based on a function of the second engagement score.

13. The method of claim 1 , wherein the first object represents a physical object, and wherein determining that the first object satisfies the offscreen criterion includes performing a computer-vision technique with respect to the first object.

14. The method of claim 1 , wherein determining that each of the first object and the second object satisfies the offscreen criterion includes:

determining that at least a portion of the first object is positioned at less than a threshold distance from a first edge of the display; and

determining that at least a portion of the second object is positioned at less than a threshold distance from a second edge of the display.

15. The method of claim 1 , wherein determining that each of the first object and the second object satisfies the offscreen criterion is in response to ceasing to present the first object and the second object.

16. The method of claim 1 , wherein the first offscreen indicator corresponds to a representation of the first object, and wherein displaying the first offscreen indicator includes replacing the first object with the first offscreen indicator.

17. The method of claim 1 , wherein at least one of the first object and the second object corresponds to a respective animation that satisfies the offscreen criterion.

18. The method of claim 1 , wherein the electronic device includes an input device, the method further comprising:

detecting, via the input device, a positional change input that corresponds to a positional change of the electronic device; and

determining that at least one of the first object and the second object satisfies the offscreen criterion based on the positional change input.

19. The method of claim 1 , further comprising:

while displaying the first offscreen indicator, detecting, via an input device, a positional change input that corresponds to a positional change of the electronic device; and

moving the first offscreen indicator to a different position on the display based on the positional change input.

20. The method of claim 1 , wherein the first semantic label value provides an understanding or meaning with respect to the first object, the method further comprising obtaining a second semantic label value associated with the second object, wherein the second semantic label value provides an understanding or meaning with respect to the second object, and wherein prioritizing the first object over the second object includes comparing the first semantic label value against the second semantic label value.

21. The method of claim 1 , wherein determining the first object satisfies the offscreen criterion includes identifying a last-displayed position of the first object on the display before the first object ceases to be presented, and wherein displaying the first offscreen indicator includes displaying a representation of the first object at the last-displayed position of the first object on the display.

22. An electronic device comprising:

one or more processors;

a non-transitory memory;

a display; and

one or more programs, wherein the one or more programs are stored in the non-transitory memory and configured to be executed by the one or more processors, the one or more programs including instructions for:

while presenting, on the display, a plurality of objects including a first object and a second object:

obtaining a first semantic label value associated with the first object;

determining a first display priority value that is associated with the first object, wherein determining the first display priority value is based at least in part on the first semantic label value; and

prioritizing the first object over the second object based on a function of the first display priority value; and

in response to determining that each of the first object and the second object satisfies an offscreen criterion, displaying, on the display, a first offscreen indicator that is associated with the first object according to the prioritization.

23. A non-transitory computer readable storage medium storing one or more programs, the one or more programs comprising instructions, which, when executed by an electronic device with one or more processors and a display, cause the electronic device to:

while presenting, on the display, a plurality of objects including a first object and a second object:

obtain a first semantic label value associated with the first object;

determine a first display priority value that is associated with the first object, wherein determining the first display priority value is based at least in part on the first semantic label value; and

prioritize the first object over the second object based on a function of the first display priority value; and

in response to determining that each of the first object and the second object satisfies an offscreen criterion, display, on the display, a first offscreen indicator that is associated with the first object according to the prioritization.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 5, 2022
From: SALTER, THOMAS G.; LUTTER, GREGORY PATRICK LANE; NAIR, RAHUL; CHALMERS, DEVIN WILLIAM; NEGOITA, IOANA
To: APPLE INC.
Reel/Frame 058557/0816 →
Continuity (1)
Provisional Application 63131539 · Dec 29, 2020
References Cited (20)
US 8971970B2 · Lindner · 2015 [cited by applicant]
US 10510191B2 · Powderly et al. · 2019 [cited by applicant]
US 10750245B1 · Zeiler · 2020 [cited by examiner]
US 11202117B2 · El Essaili · 2021 [cited by examiner]
US 11301684B1 · Kumar · 2022 [cited by examiner]
US 11698677B1 · Chiu · 2023 [cited by examiner]
US 11753029B1 · Gregory · 2023 [cited by examiner]
US 20010056427A1 · Yoon · 2001 [cited by examiner]
US 20120154619A1 · Lee · 2012 [cited by examiner]
US 20170052766A1 · Garipov · 2017 [cited by examiner]
US 20170069143A1 · Salter · 2017 [cited by examiner]
US 20190188868A1 · Bagnall · 2019 [cited by examiner]
US 20190362557A1 · Lacey et al. · 2019 [cited by applicant]
US 20200174262A1 · Godar · 2020 [cited by examiner]
US 20200250245A1 · Abhyankar · 2020 [cited by examiner]
US 20210201038A1 · Xia · 2021 [cited by examiner]
US 20210264226A1 · Lecue · 2021 [cited by examiner]
US 20210365113A1 · Chalmers · 2021 [cited by examiner]
US 20210398353A1 · Luo · 2021 [cited by examiner]
US 20220012283A1 · Richter · 2022 [cited by examiner]