Reliable Multicasting

Problem Description:

Currently, IP multicast communication is implemented using UDP. UDP does not provide guarantees that messages will arrive to the destinations. Therefore, IP multicasting cannot guarantee reliable transmissions.

Project Proposal:

Enable reliable many-to-many file transfers using TCP sockets connection on topthe Hypercast protocol for multicast communication. TCP, as opposed to UDP, provides reliable, connection-oriented communication.

Connection Issues:

TCP sockets should only be established between neighbors in the hypercube. This will minimize the amount of actual data transmissions that are made for each multicast transmission.

TCP connections should be left standing to minimize the amount of time spent initializing sockets. Only if the topology of the hypercubes changes will the sockets need to be reset.

Ensure that each pair of neighbors only maintains one TCP connection to the other. It is necessary to catch duplicate requests for redundant channels.

Provide a mechanism to detect when a TCP socket fails and handle the error gracefully.

Data Transmission Issues:

A node in a hypercube may be receiving TCP bytestreams from multiple senders that require forwarding via a common neighbor TCP connection. This may cause TCP packets from different senders to become interleaved in the output steam. We need to encapsulation essential information in each packet to enable reconstruction of the byte stream. This will allow interleaving of the outgoing byte streams with the ability to demultiplex the data at the next node.

We propose to add an additional header to all data that is being encoded. We need to partition the logical entity (file, etc.) into smaller units (refered to as hypercube packets) for transmission via TCP packets.

Each hypercube packet will have the following header:

This header is followed by X bytes of data (where X is specified by length field).

Proposed sequence of events for a many-to-many transfer

(note: We are assured that hypercube packets will arrive in correct order at each node because sequencing of packets is handled in the TCP layer. Thus, we do not need to consider the case where packets arrive out of order.)

The original sender:

• Determine the neighbors for forwarding (based upon logical address)
• Opens a socket to each neighbor
• Break the entity into chunks, setting the header fields appropriately
• Encodes each chunk as a string, and writes the string to each neighbor socket

The receiving node:

• Read the first 20 bits off the input stream of the socket (determine length of hypercube packet -- length field + remaining 33 bits of header )
• Reads rest of hypercube packet into a buffer.
• Decode chunk.
  1. perform entity reconstruction based upon this logical source (this is where demultiplexing by unique sender occurs)
     • write data to file creation buffer at current buffer pointer
       if (last packet bit)
       write buffer to file
       reset buffer and current buffer pointer
  2. determine which nodes this packet must be forwarded to
     • Read logical address of the source, and use to root a tree to determine neighbors
     • Re-encode the message to a string (with hypercube packet header - NOTE: Logical source does not change)
     • Write the encoded hypercube packet to each neighbor socket

Should bounded buffers be used or should memory dynamically be allocated?

Should we provide recovery in the case that hypercube become unstable? Or signal failure?

Timing considerations: How long should a node wait before it concludes that a TCP connection is down?

Should we maintain a persistant buffer at each node for recovery purposes?