System and method for topics implementation in a distributed data computing environment

1. A messaging system comprising:

a distributed computing environment comprising a plurality of computer systems each comprising a processor and memory and interconnected by a network;

a plurality of storage nodes operating as a cluster on said plurality of computer systems;

a distributed cache comprising a plurality of partitions;

wherein each of the plurality of storage nodes maintains a separate partition of said plurality of partitions, said separate partition stored in the memory of the f computer system on which said each of the plurality of storage nodes operates;

a messaging system comprising a topic having a topic name operating on said cluster;

wherein said topic comprises an ordered plurality of messages stored on a plurality of numbered pages including a topic head page and a topic tail page;

wherein each particular page of the plurality of pages is mapped to a particular partition of the plurality of partitions using a hash function of the topic name and page number whereby the plurality of pages are distributed over said plurality of partitions of said plurality of storage nodes;

a plurality of publishers for publishing messages associated with said topic to the topic tail page of said plurality of pages wherein the plurality of publishers are configured to calculate a location of the tail page by performing the hash function of the topic name and page number associated with the tail page; and

a plurality of subscribers for consuming messages associated with said topic from said plurality of pages.

2. The messaging system of claim 1 , further comprising:

page metadata stored in said memory and associated with each of the plurality of pages.

3. The messaging system of claim 1 wherein the messaging system comprises a plurality of topics.

4. The messaging system of claim 3 wherein the plurality of topics each comprises a plurality of channels.

5. The messaging system of claim 4 wherein each channel of each topic comprises a head and tail page.

6. The messaging system of claim 1 , further comprising:

a distributed wait notify system.

7. The messaging system of claim 1 , further comprising:

a publisher application programming interface configured for flow control.

8. The messaging system of claim 1 , further comprising:

a subscriber application programming interface configured for flow control.

9. The messaging system of claim 1 , further comprising:

a partition-local garbage collection process for message deletion.

10. The messaging system of claim 1 , further comprising:

a unit-or-order system.

11. A method for supporting messaging in a distributed computing environment comprising a plurality of computer systems each comprising a processor and memory and interconnected by a network, the method comprising:

operating a plurality of storage nodes as a cluster on said plurality of computer systems;

providing, using said plurality of storage nodes, a distributed cache comprising a plurality of partitions;

wherein each of the plurality of storage nodes maintains a separate partition of said plurality of partitions in the memory of the computer system on which said each of the plurality of storage nodes operates; operating a messaging system comprising a topic having a topic name on said cluster;

storing, using the messaging system, an ordered plurality of messages associated with the topic on a plurality of numbered pages including a topic head page and a topic tail page;

wherein each particular page of the plurality of pages is mapped to a particular partition of the plurality of partitions using a hash function of the topic name and page number whereby the plurality of pages are distributed over said plurality of partitions of said plurality of storage nodes;

providing a plurality of publishers for publishing messages associated with said topic to the topic tail page of said plurality of pages wherein the plurality of publishers are configured to calculate a location of the tail page by performing the hash function of the topic name and page number associated with the tail page; and

providing a plurality of subscribers for consuming messages associated with said topic from said plurality of pages.

12. The method of claim 11 , further comprising:

storing page metadata stored in said memory and associated with each of the plurality of pages.

13. The method of claim 11 wherein the messaging system comprises a plurality of topics.

14. The method of claim 13 wherein the plurality of topics each comprises a plurality of channels.

15. The method of claim 14 wherein each channel of each topic comprises a head and tail page.

16. The method of claim 11 , further comprising:

providing a distributed wait notify system.

17. The method of claim 11 , further comprising:

providing a publisher application programming interface configured for flow control.

18. The method of claim 11 , further comprising:

providing a subscriber application programming interface configured for flow control.

19. The method of claim 11 , further comprising:

providing a partition-local garbage collection process for message deletion.

20. A non-transitory computer-readable storage media including instructions stored thereon for supporting messaging in a distributed computing environment comprising a plurality of computer systems each comprising a processor and memory and interconnected by a network, which instructions, when executed, cause the distributed computing environment to perform steps comprising:

operating a plurality of storage nodes as a cluster on said plurality of computer systems;

providing, using said plurality of storage nodes, a distributed cache comprising a plurality of partitions;

wherein each of the plurality of storage nodes maintains a separate partition of said plurality of partitions in the memory of the computer system on which said each of the plurality of storage nodes operates; operating a messaging system comprising a topic having a topic name on said cluster;

storing, using the messaging system, an ordered plurality of messages associated with the topic on a plurality of numbered pages including a topic head page and a topic tail page;

wherein each particular page of the plurality of pages is mapped to a particular partition of the plurality of partitions using a hash function of the topic name and page number whereby the plurality of pages are distributed over said plurality of partitions of said plurality of storage nodes;

providing a plurality of publishers for publishing messages associated with said topic to the topic tail page of said plurality of pages wherein the plurality of publishers are configured to calculate a location of the tail page by performing the hash function of the topic name and page number associated with the tail page; and

providing a plurality of subscribers for consuming messages associated with said topic from said plurality of pages.