IP Library Granted Patent US 9,830,373
Granted Patent B2
US 9,830,373 · App. 14/590,570 · Granted Nov 28, 2017

Data transfer requests with data transfer policies

Inventors: Shreya Prasad (Cambridge, MA); Indrajit Roy (Palo Alto, CA); Vincent Xu (Cambridge, MA); Sunil Venkayala (Cambridge, MA); Dan Huang (Cambridge, MA); Kyungyong Lee (Palo Alto, CA)
Assignee: ENTIT SOFTWARE LLC
G06F17/30584G06F17/30569H04L67/1014H04L67/1097Y10S707/964
View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,830,373
App. No.
14/590,570
Filed
Jan 6, 2015
Granted
Nov 28, 2017
Kind
B2
Art Unit
2458
USPC
709/202
Abstract

A method, system, and computer-readable storage device for transferring data from a database management system to a data analytics system is described herein. In one aspect, a database management node storing a data partition of a database table that is distributed across a plurality of database management nodes of a database management system receives a data transfer request from a data analytics node of a data analytics node cluster. The data transfer request may include a data transfer policy for splitting the data partition onto the data analytics node cluster. A data split may be identified from the data partition. The data split may be a subset of the data partition. The data split is communicated to the data analytics node according to the data transfer policy.

Claims (40)

1. A method comprising:

receiving, by a database management node storing a data partition of a database table that is distributed across a plurality of database management nodes of a database management system, a data transfer request from a data analytics node of a cluster of data analytics nodes, the data transfer request requesting loading of data from the database management system to the cluster of data analytics nodes and including a data transfer policy for splitting the data partition onto the data analytics nodes of the cluster,

wherein the data transfer request is a database select query including a parameter specifying the data transfer policy, a parameter identifying a table, and a parameter identifying a column of the table to load to the cluster of data analytics nodes, and

wherein the data transfer request further identifies a user defined function, and the parameter specifying the data transfer policy is a parameter of the user defined function;

identifying, according to the data transfer policy of the data transfer request, a data split from the data partition, the data split being a subset of the data partition and identified in response to the data transfer request to load the data split to the cluster of data analytics nodes; and

communicating, by the database management node in response to the data transfer request, the data split to the data analytics node.

2. The method of claim 1 , further comprising communicating additional data splits to the cluster of data analytics nodes according to the data transfer policy, the additional data splits and the data split representing the data partition.

3. The method of claim 1 , further comprising, while communicating the data split to the data analytics node, communicating an additional data split derived from the data partition to an additional data analytics node according to the data transfer policy.

4. The method of claim 1 , wherein the data transfer policy includes a parameter specifying a split size, and further comprising:

using the split size for splitting the partition;

splitting the data partition into a plurality of data splits including the data split and an additional data split according to the split size; and

communicating the additional data split to another data analytics node of the cluster of data analytics nodes.

5. The method of claim 1 , wherein communicating the data split comprises establishing a network connection with the data analytics node using a network address supplied as a parameter of the data transfer request.

6. The method of claim 1 , further comprising, responsive to receiving the data transfer request, activating a user defined function identified by a parameter of the data transfer request.

7. A first client device comprising:

a processor; and

a non-transitory computer-readable storage medium storing instructions that are executable on the processor to:

select a data transfer policy from a plurality of different data transfer policies, the selected data transfer policy specifying splitting of data partitioned across a plurality of database management nodes of a database management system;

communicate a data transfer request to a first database management node of the plurality of database management nodes, the data transfer request requesting loading of data from the database management system to a cluster of client devices including the first client device,

wherein the data transfer request is a database select query including a parameter representing the data transfer policy, a parameter identifying a table, and a parameter identifying a column of the table to load to the cluster of data analytics nodes, and

wherein the data transfer request further identifies a user defined function, and the parameter specifying the data transfer policy is a parameter of the user defined function; and

receive a data split identified by the first database management node in response to the data transfer request and based on the data transfer policy represented by the parameter in the data transfer request, the data split being a subset of a data partition of the data partitioned across the plurality of database management nodes.

8. The first client device of claim 7 , wherein the data transfer policy specifies transmitting subsets of a data partition managed by a database management node to a plurality of the client devices.

9. The first client device of claim 7 , wherein the instructions are executable on the processor to instantiate a listener thread with a network address specified by a network address parameter of the data transfer request.

10. The first client device of claim 9 , wherein the listener, responsive to receiving a response to the data transfer request, is to assign a worker thread from a thread pool to store the data split in a memory.

11. The first client device of claim 10 , wherein the data transfer request is sent via a first communication protocol and the response is received via a different second communication protocol.

12. A non-transitory computer-readable storage medium storing instructions that, when executed, cause a database management node to:

receive a data transfer request from a data analytics node of a cluster of data analytics nodes, the data transfer request including a data transfer policy for splitting a data partition of data distributed across a plurality of database management nodes of a database management system onto a first data analytics node and a second data analytics node of the cluster of data analytics nodes, the data transfer request requesting loading of data from the database management system to the cluster of data analytics nodes,

wherein the data transfer request is a database select query including a parameter specifying the data transfer policy, a parameter identifying a table, and a parameter identifying a column of the table to load to the cluster of data analytics nodes, and

wherein the data transfer request further identifies a user defined function, and the parameter specifying the data transfer policy is a parameter of the user defined function;

identify, according to the data transfer policy of the data transfer request, data splits from the data partition, the data splits being subsets of the data partition identified in response to the data transfer request to load the data splits to the cluster of data analytics nodes; and

communicate, in response to the data transfer request and in parallel, a first set of the data splits to the first data analytics node and a second set of the data splits to the second data analytics node.

13. The non-transitory computer-readable storage medium of claim 12 , wherein the first set of the data splits and the second set of the data splits are uniformly sized.

14. The non-transitory computer-readable storage medium of claim 12 , wherein the data transfer request further includes respective network addresses for the first data analytics node and the second data analytics node.

15. The non-transitory computer-readable storage medium of claim 12 , wherein the data transfer request further includes a user defined function identifier, and the instructions upon execution cause the database management node to further activate the user defined function identified by the user defined function identifier.

16. The method of claim 4 , wherein the split size is computed by the data analytics node based on a number of rows of the database table and a number of data analytics nodes in the cluster of data analytics nodes.

17. The method of claim 1 , wherein the data transfer request includes a parameter specifying the data transfer policy, the data transfer policy selectable from a plurality of different data transfer policies.

18. The first client device of claim 7 , wherein the data transfer request further includes a parameter specifying a split size for splitting the data partition, wherein the instructions are executable on the processor to:

compute the split size based on a number of rows of the table and a number of client devices in the cluster of client devices.

19. The non-transitory computer-readable storage medium of claim 12 , wherein the data transfer request further includes a parameter specifying a split size for splitting the data partition, wherein the split size is computed by the data analytics node based on a number of rows of the table and a number of client devices in the cluster of client devices.

Assignments (9)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded May 28, 2026
From: MICRO FOCUS LLC
To: ROCKET SOFTWARE, INC.
Reel/Frame 075795/0114 →
RELEASE OF SECURITY INTEREST REEL/FRAME 044183/0718 Recorded Feb 2, 2023
From: JPMORGAN CHASE BANK, N.A.
To: MICRO FOCUS LLC (F/K/A ENTIT SOFTWARE LLC); BORLAND SOFTWARE CORPORATION; MICRO FOCUS (US), INC.; SERENA SOFTWARE, INC; ATTACHMATE CORPORATION; MICRO FOCUS SOFTWARE INC. (F/K/A NOVELL, INC.); NETIQ CORPORATION
Reel/Frame 062746/0399 →
RELEASE OF SECURITY INTEREST REEL/FRAME 044183/0577 Recorded Feb 2, 2023
From: JPMORGAN CHASE BANK, N.A.
To: MICRO FOCUS LLC (F/K/A ENTIT SOFTWARE LLC)
Reel/Frame 063560/0001 →
CHANGE OF NAME Recorded Aug 8, 2019
From: ENTIT SOFTWARE LLC
To: MICRO FOCUS LLC
Reel/Frame 050004/0001 →
SECURITY INTEREST Recorded Oct 11, 2017
From: ATTACHMATE CORPORATION; BORLAND SOFTWARE CORPORATION; NETIQ CORPORATION; MICRO FOCUS (US), INC.; MICRO FOCUS SOFTWARE, INC.; ENTIT SOFTWARE LLC; ARCSIGHT, LLC; SERENA SOFTWARE, INC.
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 044183/0718 →
SECURITY INTEREST Recorded Oct 11, 2017
From: ENTIT SOFTWARE LLC; ARCSIGHT, LLC
To: JPMORGAN CHASE BANK, N.A.
Reel/Frame 044183/0577 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jun 9, 2017
From: HEWLETT PACKARD ENTERPRISE DEVELOPMENT LP
To: ENTIT SOFTWARE LLC
Reel/Frame 042746/0130 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Nov 9, 2015
From: HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P.
To: HEWLETT PACKARD ENTERPRISE DEVELOPMENT LP
Reel/Frame 037079/0001 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jan 6, 2015
From: PRASAD, SHREYA; ROY, INDRAJIT; XU, VINCENT; VENKAYALA, SUNIL; HUANG, DAN; LEE, KYUNGYONG
To: HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P.
Reel/Frame 034646/0555 →
Continuity (1)
Related Publication 20160197988A1 · Jul 7, 2016