Granted Patent B2

US 12,564,342 · App. 17/761,532 · Granted Mar 3, 2026

Inventors: Abdelhalim Abbas (San Jose, CA); Jeffrey Ford Garberson (Redwood City, CA); Nathaniel E. Bischoff (Mountain View, CA); Erik Beall (Palo Alto, CA)

Assignee: Cognoa, Inc.

A61B5/165A61B5/1114A61B5/1128A61B5/163A61B5/168A61B5/4088A61B5/4803A61B5/4836A61B5/681A61B5/6898A61B5/7264A61B5/7275A61B5/744G16H20/70A61B2503/06

Abstract

Claims (26)

1 . A computer-implemented method for automated audio or video assessment of an individual, said method comprising:

(a) providing a storytelling activity to said individual using a story told through one or more images shown via a user interface on a display of a computing device;

(b) receiving, via said user interface on said display of said computing device, feedback from said individual while said individual is provided said storytelling activity in (a), said feedback corresponding to said story of said storytelling activity;

(c) changing said story of said storytelling activity responsive to said feedback received from said individual in (b), thereby generating a changed story of said storytelling activity told through one or more changed images shown via said user interface on said display of said computing device;

(d) providing said storytelling activity having said changed story told through said one or more changed images shown via said user interface on said display of said computing device, wherein said storytelling activity is configured to elicit a plurality of behavioral units in said individual;

(e) receiving, with said computing device, input data comprising at least one of audio information of said individual or video information of said individual, wherein said input data is received from said individual in response to providing said storytelling activity having said changed story in (d);

(f) identifying, with said computing device, said plurality of behavioral units within said input data, wherein each behavioral unit of said plurality of behavioral units comprises a behavior that makes up a higher order behavior; and

(g) identifying, with said computing device, said higher order behavior based at least in part on at least one behavioral unit from said plurality of behavioral units.

2 . The method of claim 1 , wherein at least one of said plurality of behavioral units comprises a facial movement by said individual, a body movement by said individual, or a sound made by said individual.

3 . The method of claim 1 , wherein said higher order behavior comprises a verbal communication or a non-verbal communication.

4 . The method of claim 3 , wherein said non-verbal communication comprises a facial expression, posture, gesture, eye contact, or touch.

5 . The method of claim 1 , wherein said input data comprises said video information for said individual, and wherein identifying said higher order behavior in (g) comprises analyzing said video information using facial recognition to detect one or more facial movements.

6 . The method of claim 1 , wherein said input data comprises said audio information for said individual, and wherein identifying said higher order behavior in (g) comprises analyzing said audio information using audio recognition to detect one or more sounds, words, or phrases.

7 . The method of claim 1 , wherein identifying said higher order behavior comprises generating a timeline of said plurality of behavioral units.

8 . The method of claim 7 , further comprising generating a behavioral pattern based at least in part on said timeline of said plurality of behavioral units or said higher order behavior.

9 . The method of claim 1 , wherein said input data further comprises information or responses provided by a caretaker of said individual, wherein said information or responses comprise answers to questions.

10 . The method of claim 1 , further comprising generating a prediction comprising a positive classification, a negative classification, or an inconclusive classification with respect to one or more of: a behavioral disorder, a developmental delay, or a neurologic impairment.

11 . The method of claim 1 , further comprising obtaining said at least one of said audio information for said individual or said video information for said individual through a mobile computing device, wherein said mobile computing device comprises a smartphone, tablet computer, laptop, smartwatch or other wearable computing device.

12 . The method of claim 11 , wherein obtaining said at least one of said audio information for said individual or said video information for said individual comprises capturing one or both of video footage or an audio recording, wherein said one or both of said video footage or said audio recording correspond to one or both of said individual or interactions between a person and said individual.

13 . The method of claim 10 , wherein said one or more of a behavioral disorder, a developmental delay, or a neurologic impairment comprises one or more of: pervasive development disorder (PDD), autism spectrum disorder (ASD), social communication disorder, restricted repetitive behaviors, interests, and activities (RRBs), autism (“classical autism”), Asperger's Syndrome (“high functioning autism”), PDD-not otherwise specified (PDD-NOS, “atypical autism”), attention deficit disorder (ADD), attention deficit and hyperactivity disorder (ADHD), speech and language delay, obsessive compulsive disorder (OCD), depression, schizophrenia, Alzheimer's disease, dementia, intellectual disability, or learning disability.

14 . The method of claim 1 , wherein identifying said higher order behavior is performed using a machine learning software module.

15 . The method of claim 14 , wherein said machine learning software module is selected from nearest neighbor, naive Bayes, decision tree, linear regression, support vector machine, or neural network.

16 . The method of claim 1 , wherein said storytelling activity provides a virtual character guiding said individual through said story.

17 . The method of claim 1 , wherein said higher order behavior comprises one or more of: articulation of speech sounds, fluency, voice, ability to understand and decode language, or ability to produce and use language.

18 . The method of claim 1 , further comprising: dynamically modifying at least part of said story of said storytelling activity based at least in part on said input data.

19 . The method of claim 1 , wherein said story of said storytelling activity comprises an interactive story.

Assignments (3)

Continuity (2)

References Cited (233)