IP Library › Granted Patent US 11,307,667
Granted Patent B2
US 11,307,667 · App. 16/892,248 · Granted Apr 19, 2022

Systems and methods for facilitating accessible virtual education

Inventors: Ayan Banerjee (Tempe, AZ); Sandeep Gupta (Tempe, AZ); Prajwal Paudyal (Loveland, OH)
Assignee: Arizona Board of Regents on Behalf of Arizona State University
G06F3/017G06F3/015G06F17/18G06K9/00355G06K9/00375G06K9/6218G06N3/08G09B5/067G09B5/14
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 11,307,667
App. No.
16/892,248
Granted
Apr 19, 2022
Kind
B2
Abstract

Various embodiments of systems and methods for a learning environment for accessible virtual computer science education are disclosed.

Claims (24)

1. A method, comprising,

generating an expert reach set for a gesture as performed by a tutor wherein the expert reach set defines correctness for transient movement dynamics and hand shape for the gesture;

receiving a video feed of a user performing the gesture;

converting the gesture as performed by the user into a spatio-temporal evolution of continuous variables by:

generating an X and Y coordinate time series, wherein the positional X and Y coordinate time series is representative of positions of a right wrist and a left wrist of the user; and

extracting a hand shape of the user; and

determining a deviation of the gesture as performed by the user from the expert reach set for the gesture; and

providing correctness feedback to the user based on the deviation of the gesture as performed by the user from the expert reach set for the gesture,

wherein the expert reach set is a set of continuous states observed from simulating a hybrid system representative of the gesture over time for a bounded set of initial conditions

wherein the hybrid system is determined by:

segmenting the gesture as captured by the video feed of the user into a plurality of discrete modes:

clustering each mode of the plurality of discrete modes in accordance with their respective triggering mechanism;

obtaining one or more kinematic equations representative of the gesture by:

processing the gesture data using a multi-variate polynomial regression model or Fisher Information with Cramer Rao bound to obtain a set of flow parameters representative of mode transitions; and

clustering the gesture data using a density based approach to the set of flow parameters for each mode of the plurality of modes; and

deriving guard conditions for each cluster of the gesture.

2. The method of claim 1 , wherein the step of generating an expert reach set for a gesture further comprises:

deriving a plurality of initial conditions for the gesture as performed by the tutor by performing a statistical analysis of the tutor's speed, initial positions, and kinematic model parameters; and

performing a reachability analysis using the hybrid system representative of the gesture as performed by the tutor.

3. The method of claim 1 , wherein the X and Y coordinate time series is representative of position of the user's wrists over time and wherein the X and Y coordinate time series is extracted using a plurality of location buckets defined around a face and a chest of the user, as captured by the video feed of the user.

4. The method of claim 1 , further comprising:

training a neural network having a plurality of layers using gesture hand shapes;

processing the hand shape of the tutor and the hand shape of the user using the neural network; and

comparing respective outputs of a penultimate layer of the plurality of layers using a Euclidean distance metric, wherein each output is representative of the hand shape of the tutor and the hand shape of the user.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 13, 2021
From: PAUDYAL, PRAJWAL
To: ARIZONA BOARD OF REGENTS ON BEHALF OF ARIZONA STATE UNIVERSITY
Reel/Frame 056229/0963 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 8, 2020
From: BANERJEE, AYAN; GUPTA, SANDEEP
To: ARIZONA BOARD OF REGENTS ON BEHALF OF ARIZONA STATE UNIVERSITY
Reel/Frame 052870/0559 →
Continuity (2)
Provisional Application 62856571 · Jun 3, 2019
Related Publication 20200379575A1 · Dec 3, 2020
Cited By (1)
US 12,737,321