IP Library › Granted Patent US 12,689,958
Granted Patent B2
US 12,689,958 · App. 18/191,933 · Granted Jul 21, 2026

Method and apparatus for programmable and customized intelligence for traffic steering in 5G networks using open ran architectures

Inventors: Rajarajan Sivaraj (Plano, TX); Rahul Soundrarajan (Bengaluru, IN); Pankaj Kenjale (Plano, TX); Ankith Gujar (Sunnyvale, CA); Tarunjeet Singh (New Delhi, IN); Wasi Asghar (Bengaluru, IN)
Assignee: Mavenir Systems, Inc.
H04W36/13H04W36/0085H04W36/22
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 12,689,958
App. No.
18/191,933
Filed
Mar 29, 2023
Granted
Jul 21, 2026
Kind
B2
Art Unit
2683
USPC
455/436
Abstract

A method of optimizing traffic steering (TS) radio resource management (RRM) decisions for handover of individual user equipment (UE) in Open Radio Access Network (O-RAN) includes: providing an O-RAN-compliant near real time RAN intelligent controller (near-RT RIC) configured to interact with O-RAN nodes; and utilizing an artificial intelligence (AI)-based TS application xApp in the near-RT RIC to optimize TS handover control and maximize UE throughput utility. The TS xApp is configured utilizing a virtualized and simulated environment for O-RAN, which virtualized and simulated environment for O-RAN is provided by ns-O-RAN platform. The optimization problem to be solved is formulated as a Markov Decision Process (MDP), and a solution to the optimization problem is derived by using at least one reinforcement learning (RL) technique.

Claims (20)

1 . A method of optimizing traffic steering (TS) radio resource management (RRM) decisions for handover of at least one user equipment (UE) in Open Radio Access Network (O-RAN), comprising:

providing an O-RAN-compliant near real time RAN intelligent controller (near-RT RIC) configured to interact with O-RAN nodes; and

utilizing an artificial intelligence (AI)-based TS application in the near-RT RIC to optimize TS handover control and maximize UE throughput utility, wherein a data-driven AI-powered TS xApp in the near-RT RIC is utilized to optimize the TS handover control, and wherein the TS xApp is configured utilizing a virtualized and simulated environment for O-RAN.

2 . The method according to claim 1 , wherein the virtualized and simulated environment for O-RAN is provided by ns-O-RAN platform.

3 . The method according to claim 2 , wherein the optimization problem to be solved is formulated as a Markov Decision Process (MDP).

4 . The method according to claim 3 , wherein a solution to the optimization problem is derived by using at least one reinforcement learning (RL) technique.

5 . The method according to claim 4 , wherein the RL technique is utilized to select an optimal target cell for TS handover of the UE.

6 . The method according to claim 5 , wherein the RL technique is based on at least a Deep Q-Network (DQN) algorithm.

7 . The method according to claim 6 , wherein the DQN algorithm includes at least one of Conservative Q-learning (CQL) algorithm and Random Ensemble Mixture (REM) algorithm.

8 . The method according to claim 7 , wherein the RL technique is additionally based on Convolutional Neural Network (CNN) architecture.

9 . The method according to claim 8 , wherein the at least one of the CQL algorithm and the REM algorithm is used in conjunction with the CNN architecture to model a Q-function and the loss function.

10 . The method of claim 9 , wherein an offline Q-learning training is performed using the CQL algorithm, and the trained CQL algorithm is deployed in the TS xApp for at least one of online value iteration, inference derivation and handover control.

11 . The method according to claim 8 , wherein the RL technique enables control of multiple UEs using a single RL agent.

12 . The method according to claim 8 , wherein an offline Q-learning training is performed using the CQL algorithm, and the trained CQL algorithm is deployed in the TS xApp for at least one of online value iteration, inference derivation and handover control.

13 . The method according to claim 7 , wherein an offline Q-learning training is performed using the CQL algorithm, and the trained CQL algorithm is deployed in the TS xApp for at least one of online value iteration, inference derivation and handover control.

14 . The method according to claim 4 , wherein the Near-RT RIC with a TS xApp is integrated with a simulated environment on ns-3 for data collection and testing of at least one RL-based control policy.

15 . The method according to claim 2 , wherein the Near-RT RIC with a TS xApp is integrated with a simulated environment on ns-3.

16 . The method according to claim 2 , wherein the TS xApp in the near-RT RIC is evaluated for Key Performance Indicators (KPIs) including at least one of UE throughput, spectral efficiency, and mobility overhead.

17 . The method according to claim 16 , wherein the evaluation of the TS xApp for KPIs is performed on a simulated RAN network generated by an ns-O-RAN platform.

18 . The method according to claim 17 , wherein the ns-O-RAN platform includes a combination of ns-3 5G RAN module and an O-RAN-compliant E2 implementation.

Assignments (13)
RELEASE OF SECURITY INTEREST IN COLLATERAL RECORDED AT REEL 069113 AND FRAME 0558 Recorded Jul 31, 2025
From: GLAS USA LLC
To: MAVENIR SYSTEMS, INC.
Reel/Frame 072308/0172 →
RELEASE OF SECURITY INTEREST IN COLLATERAL RECORDED AT REEL 067565 AND FRAME 0678 Recorded Jul 29, 2025
From: WILMINGTON SAVINGS FUND SOCIETY, FSB
To: MAVENIR SYSTEMS, INC.
Reel/Frame 072263/0421 →
GRANT OF SECURITY INTEREST - PATENTS Recorded Jul 29, 2025
From: MAVENIR NETWORKS, INC.; MAVENIR SYSTEMS, INC.; ARGYLE DATA, INC.; MAVENIR, INC.; AQUTO CORPORATION; MAVENIR IPA UK LIMITED; MAVENIR SYSTEMS UK LIMITED; MAVENIR LTD.; MAVENIR US INC.
To: GLAS USA LLC
Reel/Frame 072245/0764 →
RELEASE OF SECURITY INTERESTS (SIDECAR) Recorded Jul 29, 2025
From: JPMORGAN CHASE BANK, N.A.
To: MAVENIR SYSTEMS, INC.
Reel/Frame 072263/0041 →
RELEASE OF SECURITY INTERESTS (SYNDICATED) Recorded Jul 29, 2025
From: JPMORGAN CHASE BANK, N.A.
To: MAVENIR SYSTEMS, INC.
Reel/Frame 072263/0121 →
SECURITY INTEREST Recorded Jul 28, 2025
From: MAVENIR NETWORKS, INC.; MAVENIR SYSTEMS, INC.; ARGYLE DATA, INC.; MAVENIR, INC.; AQUTO CORPORATION; MAVENIR IPA UK LIMITED; MAVENIR SYSTEMS UK LIMITED; MAVENIR LTD.; MAVENIR US INC.
To: BLUE TORCH FINANCE LLC
Reel/Frame 072268/0439 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT Recorded Oct 4, 2024
From: MAVENIR SYSTEMS, INC.
To: GLAS USA LLC
Reel/Frame 069113/0558 →
RELEASE OF SECURITY INTEREST Recorded Oct 4, 2024
From: WILMINGTON SAVINGS FUND SOCIETY, FSB
To: MAVENIR SYSTEMS, INC.
Reel/Frame 069113/0596 →
SECURITY INTEREST Recorded Aug 30, 2024
From: MAVENIR SYSTEMS, INC.
To: WILMINGTON SAVINGS FUND SOCIETY, FSB
Reel/Frame 068822/0966 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT SUPPLEMENT Recorded Jul 18, 2024
From: MAVENIR SYSTEMS, INC.
To: JPMORGAN CHASE BANK, N.A., AS COLLATERAL AGENT
Reel/Frame 068425/0126 →
INTELLECTUAL PROPERTY SECURITY AGREEMENT SUPPLEMENT Recorded Jul 18, 2024
From: MAVENIR SYSTEMS, INC.
To: JPMORGAN CHASE BANK, N.A., AS COLLATERAL AGENT
Reel/Frame 068425/0209 →
SECURITY INTEREST Recorded May 29, 2024
From: MAVENIR SYSTEMS, INC.
To: WILMINGTON SAVINGS FUND SOCIETY, FSB
Reel/Frame 067565/0678 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 18, 2023
From: SIVARAJ, RAJARAJAN; SOUNDRARAJAN, RAHUL; KENJALE, PANKAJ; GUJAR, ANKITH; SINGH, TARUNJEET; ASGHAR, WASI
To: MAVENIR SYSTEMS, INC.
Reel/Frame 064634/0517 →
Priority Claims (1)
IN 202221020576 · Apr 5, 2022 · national
Continuity (1)
Related Publication 20230319662A1 · Oct 5, 2023
References Cited (43)
US 20210012227A1 · Fang et al. · 2021 [cited by applicant]
US 20210326695A1 · Vitebsky et al. · 2021 [cited by applicant]
US 20220014963A1 · Yeh et al. · 2022 [cited by applicant]
US 20220159525A1 · Chou · 2022 [cited by examiner]
US 20220295309A1 · Akhtar · 2022 [cited by examiner]
WO 2021176092A1 · 2021 [cited by applicant]
IP.com search history (Year: 2025). [cited by examiner]
Extended European Search Report for corresponding European application No. 23166443.4; 8 pages, dated Aug. 24, 2023. [cited by applicant]
3rd Generation Partnership Project (3GPP) TS No. 38.901 . Study on channel model for frequencies from 0.5 to 100 ghz . 3GPP. (Version 16.1.0) Jan. 11, 2020; 3rd Generation Partnership Project, Valbonne, France. [cited by applicant]
3rd Generation Partnership Project (3GPP) TS No. 37.340. NR; multi-connectivity; overall description; stage-2 (Version 16.8.0); Dec. 23, 2021; 3rd Generation Partnership Project, Valbonne, France. [cited by applicant]
Abdalla, A. S., Upadhyaya, P. S., Shah, V. K., & Marojevic, V. . “Toward Next Generation Open Radio Access Network-What O-RAN Can and Cannot Do!” arXiv preprint arXiv:2111.13754 [cs.NI]; IEEE Network (Nov. 2022). [cited by applicant]
Agarwal, R., Schuurmans, D., & Norouzi, M. “An optimistic perspective on offline reinforcement learning” Proceedings of 37th International conference on machine learning (pp. 104-114).(2020). [cited by applicant]
Balasubramanian, B., et al. . Ric: “A ran intelligent controller platform for ai-enabled cellular networks” IEEE Internet Computing, 25(2), 7-17. doi: 10.1109/MIC.2021.3062487; (2021). [cited by applicant]
Bonati, L., D'Oro, S., Polese, M., Basagni, S., & Melodia, T. “Intelligence and Learning in O-RAN for Data-driven NextG Cellular Networks” IEEE Communications Magazine, 59(10), 21-27; (Oct. 2021). [cited by applicant]
Calabrese, F. D., Wang, L., Ghadimi, E., Peters, G., Hanzo, L., & Soldati, P. “Learning radio resource management in rans: Framework, opportunities, and challenges” IEEE Communications Magazine, 56(9), 138-145.(2018). [cited by applicant]
Challita, U., Ryden, H., & Tullberg, H. “When machine learning meets wireless cellular networks: Deployment, challenges, and applications” IEEE Communications Magazine, 58(6), 12-18.(Jun. 2020). [cited by applicant]
Chinchali, S., et al. “Cellular network traffic scheduling with deep reinforcement learning” In Proc. of thirty-second AAAI Conf. on Artificial Intelligence (pp. 766-774). New Orleans, LA.(2018). [cited by applicant]
D'Oro, S., Bonati, L., Polese, M., & Melodia, T. OrchestRAN: Orchestrating Intelligence in the Open RAN. Proceedings of IEEE Infocom; (2022). [cited by applicant]
Dryjanski, M., Kułacz, L., & Kliks, A. “Toward modular and flexible open RAN implementations in 6g networks: Traffic steering use case and O-RAN xapps” Sensors, 21(24).(2021). [cited by applicant]
Guo, D., Tang, L., Zhang, X., & Liang, Y. “Joint optimization of handover control and power allocation based on multi- agent deep reinforcement learning” IEEE Transactions on Vehicular Technology, 69(11), 13124-13138.(2… [cited by applicant]
Koda, Y., Yamamoto, K., Nishio, T., & Morikura, M. “Reinforcement learning based predictive handover for pedestrian-aware mmwave networks” IEEE conference on computer communications workshops (infocom wkshps) (p. 692-69… [cited by applicant]
Kumar, A., Zhou, A., Tucker, G., & Levine, S. “Conservative Q-Learning for Offline Reinforcement Learning.”In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, & H. Lin (Eds.), Advances in neural information processi… [cited by applicant]
Letaief, K. B., Chen, W., Shi, Y., Zhang, J., & Zhang, Y. A. “The Roadmap to 6G: AI Empowered Wireless Networks” IEEE Communications Magazine, 57(8), 84-90; (Aug. 2019). [cited by applicant]
Levine, S., Kumar, A., Tucker, G., & Fu, J.. “Offline reinforcement learning: Tutorial, review, and perspectives on open problems” arXiv:2005.01643 [cs.LG]. Retrieved from https://arxiv.org/abs/2005.01643 (2020). [cited by applicant]
Liu, Q., Kwong, C. F., Wei, S., Li, L., & Zhang, S.“Intelligent handover triggering mechanism in 5g ultra-dense networks via clustering-based reinforcement learning” Mobile Networks and Applications, 26(1), 27-39.(2021). [cited by applicant]
Mezzavilla, M., Zhang, M., Polese, M., Ford, R., Dutta, S., Rangan, S., & Zorzi, M. “End-to-end simulation of 5G mmWave networks” IEEE Communications Surveys Tutorials, 20(3), 2237-2263.(Apr. 2018). [cited by applicant]
Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., & Riedmiller,M. “Playing atari with deep reinforcement learning.” arXiv:1312.5602v1 [cs.LG] Dec. 19, 2013. [cited by applicant]
Mollahasani, S., Erol-Kantarci, M., & Wilson, R. “Dynamic CU-DU Selection for Resource Allocation in O-RAN Using Actor-Critic Learning” arXiv preprintarXiv:2110.00492 [cs.NI].(Oct. 2021). [cited by applicant]
Mollel, M. S., et al.“ Intelligent handover decision scheme using double deep reinforcement learning” Physical Communication, 42, 101133.(2020). [cited by applicant]
Muñoz, P., Barco, R., & de la Bandera, I. (2015). Load balancing and handover joint optimization in Ite networks using fuzzy logic and reinforcement learning. Computer Networks, 76, 112-125. [cited by applicant]
Mwanje, S., & Mitschele-Thiel, A. “Minimizing handover performance degradation due to Ite self organized mobility load balancing.” IEEE 77th vehicular technology conference (vtc spring). (2013). [cited by applicant]
O-RAN Software Community. sim-e2-interface repository. https://github.com/o-ran-sc/sim-e2-interface. (Accessed Mar. 2022). [cited by applicant]
O-RAN Working Group 1. “Use Cases Detailed Specification 6.0.” ORAN.WG1.Use-Cases-Detailed-Specification-v06.00 Technical Specification.(Jul. 2021). O-RAN Alliance, Alfter, Germany. [cited by applicant]
O-RAN Working Group 3. “Near-Real-time RAN Intelligent Controller E2 Service Model (E2SM) Key Performance Measurement (Kpm)” Oran. WG3.E2SM-KPM-v02.01.(Nov. 2021)O-RAN Alliance, Alfter, Germany. [cited by applicant]
O-RAN Working Group 3 “Near-Real-time RAN Intelligent Controller E2 Service Model (E2SM) Ran Control (Rc)” O-RAN.WG3.E2SM-RC-v01.01.O-RAN Alliance, (Nov. 2021) Alfter, Germany. [cited by applicant]
Polese, M., Bonati, L., D'Oro, S., Basagni, S., & Melodia, T. “Understanding O-RAN: Architecture, Interfaces, Algorithms, Security, and Research Challenges.” arXiv:2202.01032 [cs.NI]. Retrieved from https://arxiv.org/ a… [cited by applicant]
Polese, M., Giordani, M., Mezzavilla, M., Rangan, S., & Zorzi, M. “Improved handover through dual connectivity in 5g mmwave mobile networks.” IEEE Journal on Selected Areas in Communications, 35(9), 2069-2084.(2017). [cited by applicant]
Sana, M., De Domenico, A., Strinati, E., & Clemente, A. “Multi-agent deep reinforcement learning for distributed handover management in dense mmwave networks” IEEE international conference on acoustics, speech and signa… [cited by applicant]
Sivaraj, R., Larson, B., Soundrarajan, R., Asghar, W., & Kenjale, P. “Building the world's first o-ran-compliant, ai-powered, closed-loop near-rt ric.” Retrieved from https://www.mavenir.com/blog/building-the-worlds-fir… [cited by applicant]
Tayyab, M., Gelabert, X., & Jantti, R. “A survey on handover management: From LTE to NR” tIEEE Access, 7, 118907-118930.(2019). [cited by applicant]
Wang, S., Liu, H., Gomes, P., & Krishnamachari, B. “Deep reinforcement learning for dynamic multichannel access in wireless networks.” IEEE Trans. Cogn. Commun. Netw., 4(2), 257-265. doi: 10.1109/TCCN.2018.2809722; (Jun… [cited by applicant]
Wang, Z., Li, L., Xu, Y., Tian, H., & Cui, S. “Handover control in wireless systems via asynchronous multiuser deep reinforcement learning.” IEEE Internet Things J., 5(6), 4296-4307.(Dec. 2018). [cited by applicant]
Yajnanarayana, V., Rydén, H., & Hevizi, L. “5g handover using reinforcement learning” IEEE 3rd 5G world forum (5gwf) (p. 349-354).(2020). [cited by applicant]