IP Library Granted Patent US 9,178,935
Granted Patent B2
US 9,178,935 · App. 12/718,934 · Granted Nov 3, 2015

Distributed steam processing

View Patent ↗
Loading inventors, assignments & file history…
Monitor This Case
Get email alerts when status or documents change.
Order Certified Copies
Most orders are placed with the USPTO same day — all within 24 business hours.
Order via The Patent Place →
Pre-filled with this patent's details
Quick Facts
Patent No.
US 9,178,935
App. No.
12/718,934
Granted
Nov 3, 2015
Kind
B2
Abstract

A method and system for forming hybrid cluster to process log files are described. In example embodiments, a method configures a node to execute as a first slave node. The first slave node executes in a first operating environment. The method also adds the first slave node to a Hadoop cluster. The Hadoop cluster includes a second slave node that operates in a second and different operating environment.

Claims (63)

1. A computer-implemented system comprising:

at least one processor;

a configuration module that is executable by the at least one processor to:

configure a node to execute as a first slave node in a first operating environment that does not support Hadoop, the first slave node with access to a local file system;

add the first slave node to a Hadoop cluster that includes a second slave node that is configurable to operate in a second operating environment that supports Hadoop and with access to a network file system different from the local file system, the second slave node to operate in a second operating environment that is different from the first operating environment;

configure the first slave node of the first operating environment that does not support Hadoop to share the access to the network file system with the Hadoop cluster including the second slave node of the second operating environment that supports Hadoop; and

update the network file system to indicate the first slave node as being added to the Hadoop cluster; and

a communication module to:

receive a request from the first slave node of the first operating environment to access a log file from the network file system, the log file stored by the network file system operated within the second operating environment that supports Hadoop; and

request a server-side runner module to retrieve the log file through the network file system operated by the second operating environment that supports Hadoop.

2. The computer-implemented system of claim 1 , wherein the communication module is further to:

receive the log file retrieved by the server-side runner module; and

store a copy of the log file in a file system operated within the first operating environment.

3. The computer-implemented system of claim 1 , wherein the configuration module is to:

receive a client-side key from the first slave node;

store the client-side key at a master node;

receive a server-side key from the master node; and

store the server-side key at the first slave node.

4. The computer-implemented system of claim 1 , wherein:

the first slave node is configured to make an operating environment call that is natively supported by the second operating environment, and

the first operating environment executes an emulating program that emulates the second operating environment, the emulating program to perform the operating environment call made by the first slave node.

5. The computer-implemented system of claim 1 , wherein the Hadoop cluster processes log files of a content provider.

6. The computer-implemented system of claim 1 , wherein the first slave node executes on a workstation running a Windows operating environment.

7. A method comprising:

configuring a node to execute as a first slave node, the first slave node executing in a first operating environment that does not support Hadoop and accessing a local file system;

adding the first slave node to a Hadoop cluster that includes a second slave node operating in a second operating environment that supports Hadoop and with access to a network file system different from the local file system accessible to the first slave node;

configuring the first slave node of the first operating environment that does not support Hadoop to share the access to the network file system with the Hadoop cluster including the second slave node of the second operating environment that supports Hadoop;

updating the network file system to indicate the first slave node as being added to the Hadoop cluster;

receiving a request from the first slave node of the first operating environment to access a log file from the network file system, the log file stored by the network file system operated within the second operating environment that supports Hadoop; and

requesting a server-side runner module to retrieve the log file through the network file system operated by the second operating environment that supports Hadoop.

8. The method of claim 7 , further comprising:

receiving the log file retrieved by the server-side runner module; and

storing a copy of the log file, the copy being stored in a file system operated within the first operating environment.

9. The method of claim 7 , wherein the configuring a program to execute as the first slave node within the Hadoop cluster further comprises:

receiving a client-side key from the first slave node;

storing the client-side key at a master node;

receiving a server-side key from the master node; and

storing the server-side key at the first slave node.

10. The method of claim 7 , wherein:

the first slave node is configured to make an operating environment call that is natively supported by the second operating environment, and

the first operating environment executes an emulating program that emulates the second operating environment, the emulating program to perform the operating environment call made by the first slave node.

11. The method of claim 7 , wherein the Hadoop cluster processes log files of a content provider.

12. The method of claim 7 , wherein the first slave node executes on a workstation running a Windows operating environment.

13. A machine-readable medium storing instructions which, when executed by a machine, cause the machine to execute a method comprising:

configuring a node to execute as a first slave node, the first slave node executing in a first operating environment that does not support Hadoop and accessing a local file system;

adding the first slave node to a Hadoop cluster that includes a second slave node operating in a second operating environment that supports Hadoop and with access to a network file system different from the local file system accessible to the first slave node;

configuring the first slave node of the first operating environment that does not support Hadoop to share the access to the network file system with the Hadoop cluster including the second slave node of the second operating environment that supports Hadoop;

updating the network file system to indicate the first slave node as being added to the Hadoop cluster;

receiving a request from the first slave node of the first operating environment to access a log file from the network file system, the log file stored by the network file system operated within the second operating environment that supports Hadoop; and

requesting a server-side runner module to retrieve the log file through the network file system operated by the second operating environment that supports Hadoop.

14. The machine-readable medium of claim 13 , further including:

receiving the log file retrieved by the server-side runner module; and

storing a copy of the log file, the copy being stored in a file system operated within the first operating environment.

15. The machine-readable medium of claim 13 , wherein the configuring the program to execute as the first slave node within the Hadoop cluster further includes:

receiving a client-side key from the first slave node;

storing the client-side key at a master node;

receiving a server-side key from the master node; and

storing the server-side key at the first slave node.

16. The machine-readable medium of claim 13 , wherein:

the first slave node is configured to make an operating environment call that is natively supported by the second operating environment, and

the first operating environment executes an emulating program that emulates the second operating environment, the emulating program to perform the operating environment call made by the first slave node.

17. The machine-readable medium of claim 13 , wherein the Hadoop cluster processes log files of a content provider.

18. The machine-readable medium of claim 13 , wherein the first slave node executes on a workstation running a Windows operating environment.

Assignments (2)
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Jul 23, 2015
From: EBAY INC.
To: PAYPAL, INC.
Reel/Frame 036169/0680 →
ASSIGNMENT OF ASSIGNOR'S INTEREST Recorded Aug 17, 2010
From: CHIU, CHI-HSIEN; CRANE, PATRICK; NECKORCUK, ALYSSA; SINGH, GYANIT; SUNDARESAN, NEELAKANTAN
To: EBAY INC.
Reel/Frame 024851/0083 →