IP Library Granted Patent US 12,656,873
Granted Patent B1
US 12,656,873 · App. 19/006,448 · Granted Jun 16, 2026

Apparatus and method for developmental and/or rehabilitative sensory substitution

Inventors: Juan Aceros (St. Johns, FL); Matthew Cantwell (Orange Park, FL); John Prisco (Canton, GA)
Assignee: University of North Florida Board of Trustees
G06F3/016G06F3/011
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,656,873
App. No.
19/006,448
Granted
Jun 16, 2026
Kind
B1
Abstract

Described herein relates to an apparatus and method for developmental and/or rehabilitative sensory substitution to augment at least one low-functioning and/or non-functioning senses in at least one user. The apparatus may include an input module that may allow the user to perform various tasks designed to enhance development of or sensory recognition by the user. The apparatus may be capable of multiple different proportional controls and of being used across ages, sizes, and developmental stages. For example, the apparatus may include at least one joystick, button, lever, and accelerometer, such that a sensory feedback response (e.g., haptic/tactile, visual, and/or auditory) may be provided to the user across a plurality of exercises. Additionally, the apparatus may provide a new standard of care for individuals with special needs and/or those recovering from a traumatic physical and/or mental injury, regardless of ability level and age.

Claims (50)

1 . A method of augmenting sensory perception of at least one user, the method comprising:

providing a sensory substitution apparatus including:

a processor;

an input module communicatively coupled to the processor, the input module comprising at least one interface, wherein the at least one interface is configured to detect an engagement of the at least one user; and

an output module communicatively coupled to the processor, the output module comprising a haptic motor, wherein the haptic motor of the output module is configured to provide a sensory feedback response to the at least one user;

receiving an engagement from the at least one user through the input module; and

activating, via the processor, the output module to deliver the sensory feedback response proportional to changes in the engagement of the at least one user with the at least one interface, wherein the sensory feedback response includes directional haptic cues from the haptic motor to guide a movement of the housing of the sensory substitution apparatus during the changes in engagement by the at least one user.

2 . The method of claim 1 , wherein the input module includes at least one user interface configured to be engaged by the at least one user, via tactile, auditory, or visual stimuli.

3 . The method of claim 2 , wherein the sensory feedback response varies based on a change in a normal force applied to the at least one user interface of the input module by the at least one user.

4 . The method of claim 1 , wherein the output module includes at least one light array.

5 . The method of claim 4 , wherein the sensory feedback response is a visual signal with variable intensity, color, or both.

6 . The method of claim 1 , further comprising the steps of:

detecting a change in an orientation of the sensory substitution apparatus relative to a predefined area;

generating the sensory feedback response proportional to the detected change in orientation; and

outputting the sensory feedback response via the output module using a haptic stimulus, an auditory stimulus, or both.

7 . The method of claim 1 , further comprising the steps of:

transmitting, via the processor in electronic communication with a computing device, the engagement of the at least one user, the outputted sensory feedback response, or both to the computing device for analysis; and

adjusting sensory feedback response parameters of the output module based on received computational instructions in response to the transmitted engagement of the at least one user, the outputted sensory feedback response, or both.

8 . A method for rehabilitative training using a sensory substitution apparatus, comprising:

grasping a housing of the sensory substitution apparatus equipped with:

an input module configured to detect engagement by at least one user;

an output module comprising at least one haptic motor, the output module configured to provide a sensory feedback response; and

a processor communicatively coupled the input module and output module;

performing an activity, an exercise, or both while holding the housing of the sensor substitution apparatus;

detecting the engagement of the at least one user during the activity, exercise, or both, in real-time, via the input module; and

activating, via the processor, the output module to deliver the sensory feedback response to the at least one user from the at least one haptic motor that is proportional to the engagement detected by the input module, wherein the activity, the exercise, or both include maintaining balance using the sensory feedback response provided by the at least one haptic motor.

9 . The method of claim 8 , wherein the at least one haptic motor is disposed within the housing of the sensory substitution apparatus.

10 . The method of claim 8 , wherein the activity, the exercise, or both include dynamic motions such as lifting, tilting, or rotating the housing of the sensory substitution apparatus to adjust the sensory feedback response is, in real-time, accordingly.

11 . The method of claim 8 , further comprising the steps of:

setting at least one feedback sensitivity levels based on a capability of the at least one user; and

modifying the sensitivity level in response to progress of the at least one user over time.

12 . The method of claim 8 , wherein the sensory feedback response includes directional haptic cues to guide a movement of the housing of the sensory substitution apparatus, via the at least one user, during the activity, exercise, or both.

13 . A method for interactive sensory training, the method comprising:

providing a sensory substitution apparatus with:

an input module featuring a joystick, a pressure-sensitive surface, or both;

an output module with visual, auditory, haptic or tactile capabilities; and

a processor to process at least one input signal generated by the input module in response to the engagement of at least one user, via the joystick, the pressure-sensitive surface, or both;

engaging with the input module to perform a task requiring sensory coordination;

detecting input of the at least one user, via the input module, and translating, via the processor communicatively coupled to the input module and the output module, the input of the at least one user into proportional output signals;

guiding the at least one user toward a task objective with visual, auditory, or tactile sensory feedback response, wherein the sensory feedback response includes directional haptic cues to guide the at least one user toward the task objective; and

dynamically adjusting the sensory feedback response, in real-time, based on task performance by the at least one user.

14 . The method of claim 13 , wherein the task objective involves navigating a graphical interface using directional input detected by the input module.

15 . The method of claim 13 , wherein the sensory feedback response includes illuminating a light array to indicate task progression.

16 . The method of claim 13 , wherein the sensory feedback response includes producing audio signals with varying frequency or volume to indicate success or errors.

17 . The method of claim 13 , further comprising the steps of:

storing task performance data of the at least one user based on the sensory feedback response of the output module in a memory module communicatively coupled to the processor; and

analyzing the data to customize subsequent tasks or feedback.

18 . The method of claim 13 , further comprising the step of, integrating the sensory substitution apparatus with augmented reality tools to enhance user interaction.

19 . The method of claim 13 , wherein the sensory substitution apparatus delivers simultaneous multimodal feedback, including visual, auditory, and tactile responses.

20 . The method of claim 13 , wherein the sensory feedback response intensity increases as the at least one user approaches the task objective, providing motivational reinforcement.

Assignments (1)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Feb 28, 2025
From: ACEROS, JUAN; CANTWELL, MATTHEW; PRISCO, JOHN
To: UNIVERSITY OF NORTH FLORIDA BOARD OF TRUSTEES
Reel/Frame 070368/0203 →
Continuity (3)
Continuation 18474537 · Sep 26, 2023
Continuation 17939421 · Sep 7, 2022
Provisional Application 63241312 · Sep 7, 2021
References Cited (4)
US 11809629B1 · Segil · 2023 [cited by examiner]
US 20180081477A1 · Picciotto · 2018 [cited by examiner]
US 20210064143A1 · Stern · 2021 [cited by examiner]
US 20210186794A1 · Seim · 2021 [cited by examiner]