IP Library › Granted Patent US 12,526,765
Granted Patent B2
US 12,526,765 · App. 18/446,951 · Granted Jan 13, 2026

Machine learning model validation for UE positioning based on reference device information for wireless networks

Inventors: Muhammad Ikram Ashraf (Espoo, FI); Teemu Mikael Veijalainen (Espoo, FI); Mikko Säily (Espoo, FI); Oana-Elena Barbu (Aalborg, DK); Taylan Sahin (Munich, DE); Afef Feki (Massy, FR); Athul Prasad (Naperville, IL)
Assignee: Nokia Technologies Oy
H04W64/003H04W28/0226H04W28/0231
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,526,765
App. No.
18/446,951
Granted
Jan 13, 2026
Kind
B2
Abstract

A method may include receiving, by a first user device from a network node or a second user device, 1) a positioning measurement report including at least one positioning measurement measured by a reference device, and 2) reference positioning-related information to be used for testing and/or validating a machine learning model; determining estimated positioning-related information as outputs of the machine learning model based on at least a portion of the positioning measurement report as inputs to the machine learning model; determining a performance indication of the machine learning model based on the reference positioning-related information and the estimated positioning-related information, wherein the performance indication indicates a performance or accuracy of the machine learning model; and performing, by the first user device, an action based on the performance indication.

Claims (99)

1 . A method comprising:

performing at least one of the following:

receiving, by a first user device from a network node or a second user device, a request to test and/or validate a machine learning model for the first user device, wherein the machine learning model is to be used for positioning or to assist with positioning; or

transmitting, by the first user device to the network node or the second user device, a request to validate the machine learning model;

receiving, by the first user device from the network node or the second user device, 1) a positioning measurement report including at least one positioning measurement measured by a reference device based on reference signals, and 2) reference positioning-related information to be used for testing and/or validating the machine learning model;

determining, by the first user device, estimated positioning-related information as outputs of the machine learning model based on at least a portion of the positioning measurement report as inputs to the machine learning model;

determining, by the first user device, a performance indication of the machine learning model based on the reference positioning-related information and the estimated positioning-related information, wherein the performance indication indicates a performance or accuracy of the machine learning model; and

performing, by the first user device, an action based on the performance indication.

2 . A first user device comprising:

at least one processor; and

at least one memory including computer program code;

the at least one memory and the computer program code configured to, with the at least one processor, cause the first user device at least to:

perform at least one of the following:

receive, by the first user device from a network node or a second user device, a request to test and/or validate a machine learning model for the first user device, wherein the machine learning model is to be used for positioning or to assist with positioning; or

transmit, by the first user device to the network node or the second user device, a request to validate the machine learning model;

receive, by the first user device from the network node or the second user device, 1) a positioning measurement report including at least one positioning measurement measured by a reference device based on reference signals, and 2) reference positioning-related information to be used for testing and/or validating the machine learning model;

determine, by the first user device, estimated positioning-related information as outputs of the machine learning model based on at least a portion of the positioning measurement report as inputs to the machine learning model;

determine, by the first user device, a performance indication of the machine learning model based on the reference positioning-related information and the estimated positioning-related information, wherein the performance indication indicates a performance or accuracy of the machine learning model; and

perform, by the first user device, an action based on the performance indication.

3 . The first user device of claim 2 , wherein the performing an action based on the performance indication comprises:

making one or more changes to the machine learning model based on the performance indication.

4 . The first user device of claim 2 , wherein the performing an action based on the performance indication comprises:

transmitting, by the first user device to the network node or the second user device, the performance indication.

5 . The first user device of claim 4 , wherein the transmitting the performance indication comprises at least one of the following:

transmitting the performance indication if the performance indication is greater than a first threshold; or

transmitting the performance indication if the performance indication is less than a second threshold.

6 . The first user device of claim 2 , wherein the reference positioning-related information comprises at least one of:

a true or accurate line-of-sight/non-line-of-sight (LOS/NLOS) status of the reference device;

a true or accurate position of the reference device; or

a predicted output of the machine learning model, based on at least the portion of the positioning measurement report as inputs to the machine learning model, if the machine learning model is operating within a threshold level of accuracy.

7 . The first user device of claim 2 , wherein the estimated positioning-related information comprises at least one of:

an estimated line-of-sight/non-line-of-sight (LOS/NLOS) status of the reference device, based on at least the portion of the positioning measurement report as inputs to the machine learning model;

an estimated position of the reference device, based on at least the portion of the positioning measurement report as inputs to the machine learning model; or

an estimate of one or more measurements included within the positioning measurement report based on at least the portion of the positioning measurement report as inputs to the machine learning model.

8 . The first user device of claim 2 , wherein the performance indication of the machine learning model is based on a comparison between the reference positioning-related information and the estimated positioning-related information.

9 . The first user device of claim 2 , wherein the performance indication of the machine learning model comprises, or is based upon, at least one of the following:

a difference between the reference positioning-related information and the estimated positioning-related information; or

an error estimation between the reference positioning-related information and the estimated positioning-related information.

10 . The first user device claim 2 , wherein the performing an action based on the performance indication comprises:

transmitting, by the first user device to the network node or the second user device, the performance indication;

wherein the first user device is caused to:

receiving, by the first user device from the network node or the second user device, machine learning model change information based on the transmitted performance indication, to cause the first user device to perform one or more changes with respect to the machine learning model.

11 . The first user device of claim 10 , wherein the machine learning model change information comprises one or more of the following:

an indication that the machine learning model is inaccurate or does not meet a performance requirement;

information to trigger a deactivation and/or a reselection of the machine learning model;

a request to perform one or more changes or updates to the machine learning model;

a request to re-train the machine learning model;

a request to use a positioning method that does not use a machine learning model; or

a request to replace the machine learning model with a different machine learning model.

12 . The first user device of claim 10 , wherein the one or more changes performed by the first user device with respect to the machine learning model comprises one or more of the following:

perform one or more changes or updates to the machine learning model;

retrain the machine learning model;

deactivate the machine learning model;

select a different machine learning model;

activate a different or new machine learning model, or replace the machine learning model with a different machine learning model;

use a positioning method that does not use a machine learning model.

13 . The first user device of claim 2 , wherein the performing an action based on the performance indication comprises performing one or more of the following:

performing one or more changes or updates to the machine learning model;

retraining the machine learning model;

deactivating the machine learning model;

selecting a different machine learning model;

activating a different or new machine learning model, or replacing the machine learning model with a different machine learning model;

using a positioning method that does not use a machine learning model.

14 . The first user device of claim 2 , wherein the reference device comprises at least one of:

a third user device that operates as a positioning reference unit (PRU);

a third user device that is at a known location, has a known LOS/NLOS status, and/or has one or more known positioning measurements; or

a device that is trusted by the network node or trusted by one or more entities of a wireless network.

15 . The first user device of claim 2 , wherein the network node comprises at least one of:

a base station, eNB or gNB; or

a location management function (LMF).

16 . The first user device of claim 2 , caused to:

transmitting, by the first user device to the network node or second user device, information indicating one or more inputs of the machine learning model;

wherein the received positioning measurement report is based on, or includes information that is based on, the information indicating one or more inputs of the machine learning model.

17 . An apparatus comprising:

at least one processor; and

at least one memory including computer program code;

the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus at least to:

perform at least one of the following by a network node or a second user device:

transmit, by a network node or a second user device to a first user device, a request to test and/or validate a machine learning model for the first user device, wherein the machine learning model is to be used for positioning or to assist with positioning; or

receive, by the network node or the second user device from the first user device, a request to validate the machine learning model;

receive by the network device or the second user device from a reference device, 1) a positioning measurement report including at least one positioning measurement measured by a reference device based on reference signals, and 2) reference positioning-related information to be used for testing and/or validating the machine learning model;

transmit, by the network node or the second user device to the first user device, the positioning measurement report;

receive, by the network node or the second user device from the first user device, estimated positioning-related information, which are outputs of the machine learning model of the first user device based on at least a portion of the positioning measurement report as inputs to the machine learning model of the first user device;

determine, by the network node or the second user device, a performance indication of the machine learning model based on the reference positioning-related information and the estimated positioning-related information, wherein the performance indication indicates a performance or accuracy of the machine learning model;

perform, by the network node or the second user device, an action based on the performance indication.

18 . The apparatus of claim 17 , wherein the performing an action comprises:

determining machine learning model change information based on the performance indication; and

transmitting, by the network node or second user device to the first user device, the machine learning model change information to cause the first user device to perform one or more changes with respect to the machine learning model.

19 . The apparatus of claim 18 , wherein the machine learning model change information comprises one or more of the following:

an indication that the machine learning model is inaccurate or does not meet a performance requirement;

information to trigger a deactivation and/or a reselection of the machine learning model;

a request to perform one or more changes or updates to the machine learning model;

a request to re-train the machine learning model;

a request to use a positioning method that does not use a machine learning model; or

a request to replace the machine learning model with a different machine learning model.

20 . The apparatus of claim 17 , wherein the reference positioning-related information comprises at least one of:

a true or accurate line-of-sight/non-line-of-sight (LOS/NLOS) status of the reference device;

a true or accurate position of the reference device; or

a predicted output of the machine learning model, based on at least the portion of the positioning measurement report as inputs to the machine learning model, if the machine learning model is operating within a threshold level of accuracy.

Assignments (10)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 27, 2023
From: FEKI, AFEF
To: ALCATEL-LUCENT INTERNATIONAL S.A.
Reel/Frame 065666/0359 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 27, 2023
From: PRASAD, ATHUL
To: NOKIA OF AMERICA CORPORATION
Reel/Frame 065666/0393 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 27, 2023
From: BARBU, OANA-ELENA
To: NOKIA DENMARK A/S
Reel/Frame 065666/0413 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 27, 2023
From: SAHIN, TAYLAN
To: NOKIA SOLUTIONS AND NETWORKS GMBH & CO. KG
Reel/Frame 065666/0437 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 27, 2023
From: IKRAM ASHRAF, MUHAMMAD; VEIJALAINEN, TEEMU MIKAEL; SÄILY, MIKKO
To: NOKIA SOLUTIONS AND NETWORKS OY
Reel/Frame 065666/0462 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 27, 2023
From: ALCATEL-LUCENT INTERNATIONAL S.A.
To: NOKIA TECHNOLOGIES OY
Reel/Frame 065666/0483 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 27, 2023
From: NOKIA OF AMERICA CORPORATION
To: NOKIA TECHNOLOGIES OY
Reel/Frame 065666/0523 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 27, 2023
From: NOKIA DENMARK A/S
To: NOKIA TECHNOLOGIES OY
Reel/Frame 065666/0566 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 27, 2023
From: NOKIA SOLUTIONS AND NETWORKS GMBH & CO. KG
To: NOKIA TECHNOLOGIES OY
Reel/Frame 065666/0591 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 27, 2023
From: NOKIA SOLUTIONS AND NETWORKS OY
To: NOKIA TECHNOLOGIES OY
Reel/Frame 065666/0625 →
Continuity (2)
Provisional Application 63371211 · Aug 11, 2022
Related Publication 20240057022A1 · Feb 15, 2024
References Cited (25)
US 20230232266A1 · Li · 2023 [cited by examiner]
US 20230354254A1 · Sundararajan · 2023 [cited by examiner]
US 20240089905A1 · Gopalakrishnan · 2024 [cited by examiner]
US 20240187877A1 · Esswie · 2024 [cited by examiner]
WO 2022058020A1 · 2022 [cited by applicant]
WO WO2024027939A1 · 2024 [cited by examiner]
3GPP TSG RAN WG1 #109-e R1-2204838 e-Meeting Title: On potential specification impact of AI/ML for positioning, May 9-20, 2022 (Year: 2022). [cited by examiner]
3GPP TSG RAN WG1 #109-e R1-2205029 e-Meeting Title—Other aspects on AI-ML for positioning accuracy enhancement, May 9-20, 2022 (Year: 2022). [cited by examiner]
“3rd Generation Partnership Project; Technical Specification Group Radio Access Network; NG-RAN; NR Positioning Protocol A (NRPPa) (Release 17”, 3GPP TS 38.455, V17.1.1, Jun. 2022, pp. 1-198. [cited by applicant]
“3rd Generation Partnership Project; Technical Specification Group Radio Access Network; LTE Positioning Protocol (LPP) (Release 17)”, 3GPP TS 37.355, V17.0.0, Mar. 2022, pp. 1-340. [cited by applicant]
“Revised WID on NR Positioning Enhancements”, 3GPP TSG RAN Meeting #91e, RP-210897, Agenda: 9.7.25, Intel Corporation, Mar. 16-26, 2022, 6 pages. [cited by applicant]
“New SI: Study on Artificial Intelligence (AI)/Machine Learning (ML) for NR Air Interface”, 3GPP TSG RAN Meeting #94e, RP-213599, Agenda: 8.6.1, Qualcomm, Dec. 6-17, 2021, 6 pages. [cited by applicant]
“Sub use cases and spec impacts for AI/ML for positioning accuracy enhancement”, 3GPP TSG RAN WG1 #109-e, R1-2205081, Agenda: 9.2.4.2, Fujitsu, May 9-20, 2022, 2 pages. [cited by applicant]
“Discussion on AI/ML for positioning accuracy enhancement”, 3GPP TSG RAN WG1 Meeting #109-e, R1-2203145, Agenda: 9.2.4.2, Huawei, May 9-20, 2022, 6 pages. [cited by applicant]
“Use-cases and specification for positioning”, 3GPP TSG RAN WG1 #109-e, R1-2204798, Agenda: 9.2.4.2, Intel Corporation, May 9-20, 2022, pp. 1-21. [cited by applicant]
Sosnin et al., “NLOS Links Detection Algorithm for Improved 5G NR Indoor Positioning”, IEEE International Black Sea Conference on Communications and Networking (BlackSeaCom), May 24-28, 2021, 6 pages. [cited by applicant]
Jiang et al., “UWB NLOS/LOS Classification Using Deep Learning Method”, IEEE Communications Letters, vol. 24, No. 10, Oct. 2020, pp. 2226-2230. [cited by applicant]
“Msc-generator”, Sourceforge, Retrieved on Sep. 7, 2023, Webpage available at : https://sourceforge.net/projects/msc-generator/. [cited by applicant]
“3rd Generation Partnership Project; Technical Specification Group Radio Access Network; NR; Physical layer procedures for data (Release 17)”, 3GPP TS 38.214, V17.2.0, Jun. 2022, pp. 1-228. [cited by applicant]
“Discussion summary #1 of [109-e-R18-AI/ML-08]”, 3GPP TSG RAN WG1 Meeting #109-e, R1-2205229, Agenda: 9.2.4.2, vivo, May 9-20, 2022, pp. 1-54. [cited by applicant]
“Discussion on AI/ML for CSI feedback enhancement”, 3GPP TSG RAN WG1 #109-e, R1-2203939, Agenda: 9.2.2.2, NEC, May 9-20, 2022, pp. 1-7. [cited by applicant]
“3rd Generation Partnership Project; Technical Specification Group Radio Access Network; NG Radio Access Network (NG-RAN); Stage 2 functional specification of User Equipment (UE) positioning in NG-RAN (Release 17)”, 3GP… [cited by applicant]
Extended European Search Report received for corresponding European Patent Application No. 23180835.3, dated Jan. 4, 2024, 11 pages. [cited by applicant]
“Other aspects on AI-ML for positioning accuracy enhancement”, 3GPP TSG RAN WG1 #109-e, R1-2205029, Agenda 9.2.4.1, Qualcomm Incorporated, May 9-20, 2022, pp. 1-16. [cited by applicant]
“On potential specification impact of AI/ML for positioning”, 3GPP TSG RAN WGI #109-e, R1-2204838, Agenda 9.2.4.2, Fraunhofer IIS, May 9-20, 2022, 3 pages. [cited by applicant]