the cajo project
Free, simple, powerful: Transparent Distributed Computing
Common mistakes to avoid
This page will stand as a living document, fueled by the feedback of developers. It's easy to become highly proficient at this library, and completely forget all the initial frustrations in getting it to work the first time. Therefore, I would like to ask the community to please send in reports of odd things that have gone wrong. Often these 'pearls from tears' can be as useful for learning as the source documentation itself.
  1. Long Delays
  2. Network Realities
  3. Firewalled Clients
  4. Marshalling Objects
  5. Sending Proxies
  6. Configuring Remote   (corrected)
  7. Excessive Decomposition
  8. Method Visibility
  9. Reference Lifetime
1. Long delays: When each remote invocation takes a really long time to execute, there is a common problem. RMI attempts to do a reverse-DNS lookup before each invocation. In theory, this is for security purposes, so there is no way to disable it. The problem is; when trying to match IP addresses to host names, a DNS timeout can add up to several minutes to each invocation. If you do not have a DNS server to resolve the addresses to names, say on a private LAN for instance, it will be necessary to revert back to the 'old way' of the pre-DNS internet, i.e. the hosts file. This is a text file containing address/host name pairs, which the operating system will check first, before attempting a DNS lookup.

On *nix-type systems (Solaris/Linux/BSD/OSX...) the file is in: /etc/hosts   (ok, i know you guys knew that ;-)
On Windows systems, it is typically: \WINDOWS\system32\drivers\etc\hosts   (yet who knows where microsoft will move it next)

Here you would put the IP address/host name pairs for the machines that do not have DNS support.

2. Network Realities: While it is very simple to write code without regard to locality or remoteness of objects, it is very important to remember that any remote invocation can fail, or block for a very long time, if there is a problem with the network, or the target machine is unresponsive. The invoke interface is declared to throw an exception for two good reasons: One is logical, i.e. the invocation is invalid; the other is genuinely exceptional, i.e. something completely unexpected happened.

These realities can be managed in three different ways: First, when dealing with a remote item reference, your code should be prepared for unexpected network exceptions, and long transaction delays. Second, a proxy can be sent which manages these error conditions when calling its remote sender. (This makes life very nice for the user) Third, when it absolutely positively cannot go wrong, a proxy can be sent containing all of its necessary functionality, making little to no use of its remote link back to its sender.

    * NB: When using Transparent Item Proxies, e.g. the grail, it is highly recommended to have each method signature, in the client interface, declare that it throws java.rmi.RemoteException. Whilst this is technically not required; The benefit is that it will force wrapping remote invocations in a try/catch block, to manage when remote method invocations go bad, for network related reasons.

Lastly, I find it extremely helpful to print, frame, memorize, and post somewhere near your monitor, the eight fallacies of distributed computing. You could put it right next to your printout of the famous Internet RFC 1925: The Twelve Networking Truths.

3. Firewalled Clients: Very often a client item is operating behind a firewall that prohibits inbound socket connections. If server callbacks are necessary for the design, this posed a real problem. A solution to this issue is provided through two specialised classes in the gnu.cajo.utils.extra package. The ClientProxy object is used to locally represent a callable reference to the client, item in the server address space. A server would instantiate a ClientProxy, and pass a remote reference to it to the client. The client would then combine that reference, with a reference to the local item in an ItemProxy object. This transparent approach allows object design for both servers and clients, without regard for the firewall issue. To see a simple example application, please visit our firewalled clients wiki page.

4. Marshalling Objects: Sometimes, when sending large objects, particularly those which are a large graph of objects, exceptions can randomly occur during deserialisation. While this is not a defect in the framework, it can be very annoying. One particularly helpful technique is for the sender to encapsulate the object in a java.rmi.MarshalledObject. Then the receiver can retrieve the original object using its get() method. Encapsulating objects this way seems to make them far less prone to failure during transmission. The reason why this failure occurs, and even why this technique helps to fix it, is still a mystery. (see also item 6 below)

5. Sending Proxies: Proxy objects, when sent from remote JVMs, typically arrive in a Marshalled Object. There is a very important reason for this. A proxy is an object whose codebase is typically foreign to its hosting VM. When a proxy is extracted from the MarshalledObject, via its get method, its remote codebase will be automatically loaded over the network, from the location information contained within the MarshalledObject. The proxy now exists, and can be used, but does not contain information about where its codebase came from. If you try to send the proxy object itself to another VM, the receiver will most likely suffer a ClassNotFoundException. However, putting the proxy in a MarshalledObject first, will work correctly. MarshalledObjects are very special containers, which record codebase annotation, with which proxy objects can be freely sent between remote JVMs.

6. Configuring Remote: You should call Remote.config once, in the lifetime of the server, typically shortly after startup. It configures the TCP settings for network communication. Specifying port 1099, which is the default port for the Java RMIRegistry, eliminates the need to explicitly specify a port number, in a server lookup URL.

    *Update: (01-Jun-05) A convenience mechanism to provide a useful-guess at TCP configuration has now been added. If you do not explicitly configure Remote, it will now remote on an anonymous port. The actual port selected from the unused pool by the OS at runtime, can be determined by calling Remote.getServerPort. It is effectively an default call to Remote.config(null, 0, null, 0).

    *Update: (05-Aug-05) With the award of our official port number 1198, it is highly recommended to use this port when configuring remote for primary servers.

7. Excessive Decomposition: The interfaces to remote objects should tend to favor functional richnes, over logical abstraction. Unlike an all-local object application; a remote procedure call is a temporally expensive activity. This strongly favors the creation of much fewer interface objects, with more functionally complete methods, to be used when remoting. In other words: "Export trees, not leaves."

8. Method Visibility: When an object is made remotely accessible via Remote, all of its public methods, including its static ones, can be invoked by a remote JVM. This would make remoting an object that extended Thread for instance, a questionable choice, as its run() method could be repeatedly invoked remotely. Unless all possible clients are known to be well behaved, an object should have no public methods it does not want a client to be able to call. This gotcha is the corollary to the FAQ #14.

9. Reference Lifetime: When remote references to objects are created, normally they last for the life of the server JVM. Typically, this is what one would expect, when creating service objects. However, there are times when the lifetime of a remote reference is "too long."

Typically one of two scenarios come into play:
  • All of the remoted objects need to be shut down, typically needed by application server containers. To accomodate this, the Remote.shutdown() method is provided, it terminates all outstanding remote references. It is also recommended if the server would like to exit normally, without having to call System.exit().

  • Remote references to objects that are used by a client for a short-term, e.g. to monitor a transaction progress, whose lifetime is dictated by the clients, rather than the server. When remoting these objects, it is essential to use a special Remote method e.g.:
    return new Remote(clientMonitor).clientScope();
    This will prevent the memory leak that would occur, if remote references to short-term objects were returned. The method is named as such because, in this case, the lifetime of the remote reference is under the control of the clients, rather than the server. I.e. when the one (or more) clients no longer refer to the remote reference, it will then be disposed to the heapspace automatically. (Otherwise you would have to invoke Remote.unexport() on the reference manually)

A place to remember common mistakes, in order to make them less common.
© 2004 John Catherino, GNU FDL