Pattern: Store for Forward

Example

Imagine you are using an object/relational access layer and you are committing a transaction that contains some 100 or more changed objects that need to be written to the  central remote database. The updates will take say 20 seconds, even if you use bundled write, and you don't want to keep your user waiting for such a long time.

Problem

How do you prevent long waiting times when your user has changed many objects

Forces

Performance: Even if each of the update statements is processed with near optimal performance by the database, no user likes to wait for 3 or more seconds. You have to come up with a way to improve the performance that is felt by your user.

Correctness: On the other hand it might be necessary that all objects of the transaction you are committing are written into the central database before you can start a new transaction. In this case it seems your user has to wait.

Security: Keeping data on a well administered server is safer than keeping them on a personal computer even for minutes. On the other hand, the probability that a PC crashes is not much higher than the probability that somebody misspells something on a paper form and has to write that paper form again by hand, but. users are less tolerant with computers.

Cost and complexity: Whatever solution you are planning. It should be affordable and  simple.

Solution

Store your data in a local buffer and give control back to your user. Have a separate thread of execution forward your data from the buffer while your user works on the next task.

Structure

The user process (Thread 1) stores the data to be written in a local buffer. A background process (Thread 2, for example a bundle manager) forwards them to the remote database.

Consequences

Performance and user acceptance of a system: If you store your data locally, the speed you gain from this can come close to the I/O processing rate of the computer you are working on. This is more than fast enough for most business systems.

Correctness: You have to make sure, the next transaction your user is working on does not collide with data that you have stored for forward. The object sets should not contain any common objects unless you flush your client cache and risk a time stamp collision.

Security: Keeping data on a PC for a few seconds is save enough in most cases - even if host acolytes will give you a bad time for it.

Cost and Complexity: Most store for forward schemes can be made so simple that cost is affordable. If you use bundled write for example, your bundle manager can work in a separate thread, synchronized with your application. This is not much effort.

Variants

Replicated Databases must use a store for forward scheme in order to work properly. You can implement your own store for forward schemes by using synchronized flat files. You can also store your data to a local database and use a forwarding job in a second thread to do the forwarding for you.

Related Patterns

The pattern can be combined with bundled write.

Known Uses

The pattern has been used extensively in a fat client banking application by Genesys [Sta98 ]. Phoenix uses it for the forwarding of Error Protocols, in case the database connection is broken.