Legion Research Group

Department of Computer Science

University Of Virginia

 

Michael Walker

To: Andrew Grimshaw

May 2000

 

Message Queues and POSIX

 

Q: Are message queues supported in the Portable Operating Systems Interface (POSIX) standard?

A: Yes. Message queues were added by the POSIX realtime standard 1003.1b-1993. The interface for POSIX message queues is different from System V message queues, and should not be confused. POSIX message queue interface commands begin with the prefix "mq_" (i.e. mq_open(), etc), and the interface can be found in <mqueue.h>.

 

Q: Is the POSIX message queue interface supported by the platforms supported by Legion?

A: Not all of our platforms support POSIX standard 1003.1b, so the short answer is no. Solaris supports it in full, but I have not found evidence that Linux offers support for it. Linux offers limited support for the POSIX realtime standard, but POSIX IPC does not seem to be implemented yet.

 

Q: Is the POSIX message queue interface really portable among all systems that support it?

A: Actually, not in full. Solaris 2.6 requires that the path name passed when creating a queue must start with "/", and forbids any additional slashes. It creates the queue in /tmp/pathname. However, Digital UNIX 4.0B creates the pathname in the filesystem, rather than putting it in /tmp. This can cause compatibility problems when "/" is passed as the pathname--it will work fine in Solaris, but the process will need write permissions in the root directory to work in Digital UNIX [1]. This might pose problems for us.

 

Q: Are message queues supported in UNIX system V?

A: Yes. The interface commands begin with "msg" (i.e. msgget(), etc.), and can be found in <sys/msg.h>.

 

Q: Is the UNIX system V message queue interface supported by the platforms supported by Legion?

A: I wouldn't think so; for instance, WinNT probably doesn't support it. The UNIX system V message queue interface is not designed to be portable; it is designed specifically for UNIX system V. Perhaps this is why you were interested in POSIX message queues specifically.

 

Q: What is the timing (i.e. bandwidth and latency) numbers one can expect when using message queues?

A: Rather than write my own program and "reinvent the wheel", I did a little research and found some precalculated benchmarks that are applicable. I apologize for the lousy scan quality; I am not yet a "master scanner". The first chart and graph pair shows the bandwidth for different IPC mechanisms on Solaris 2.6; the second shows the same mechanisms on Digital UNIX 4.0B. Both are taken from [1]. Generally, they show the relative bandwidths of different IPC mechanisms in terms of message size, including TCP and UDP connection bandwidth for network communication on the same machine.

 

What about latency? A chart from the same source follows.

 

Pipe

Posix message queue

System V message queue

Doors

Sun RPC TCP

Sun RPC UDP

TCP socket

UDP socket

UNIX domain socket

Solaris 2.6

324 us

584 us

260 us

121 us

1891 us

1677 us

798 us

755 us

465 us

DUnix 4.0B

574 us

995 us

625 us

 

1648 us

1373 us

848 us

639 us

289 us

 

Note that all TCP and UDP measurements are based on two processes residing on the same host.

 

Q: What conclusions can we draw from this data?

A: Firstly, let me make the assumption that your interest in these figures comes from the idea that two processes on the same host should not need TCP/UDP to communicate, and that other local IPC devices might be faster. However, Legion does not currently support local communication that bypasses the network layer. Thus, I was asked to investigate message queues to find if (1) they would be supported on all Legion architectures, making them valid for use; and (2) they would provide a significant performance advantage over the current communication methods for processes on the same host.

 

Given these assumptions, the data shows that message queues DO provide a bandwidth and latency performance increase over other TCP/UDP commincation methods, as would be expected. My previous investigation has NOT been able to show that either POSIX or System V message queues would be uniformly supported by all Legion architectures.

 

Q: What questions are still left unanswered?

A: These numbers are not Legion-architecture specific, although we could expect similar relationships between methods on different architectures. One remaining question is "what is the performance numbers on a PC_linux machine, an alpha_linux machine, WinNT, etc.?", rather than just "what are the relative performance measurements of message queues and other IPC mechanisms?". If you are still interested in message queues for local Legion IPC, I can implement my own test program and find this out. Please ask if interested.

 

P.S. I was never able to track down Marty and find the IPC book you referred to in your email (I believe he's out-of-town this week). I found the reference below to be well-suited for answering your questions.

 

References

[1] W. Richard Stevens. UNIX Network Programming, Volume 2: Interprocess Communication, 2nd edition. New Jersey: Prentice Hall PTR, 1998.