11.1 Multi-CPU Database Architecture

If CPUs are proximate, termed parallel architecture, else called distributed architecture (connected via network technology). The choices when there are multiple CPUs is what will be shared among them. The Architecture is the way the sharing is organized. Client-server architecture is differientiated from remote terminal in that the client machine does some local processing and then communicates with a "back end" server to obtain data. This takes the load of parsing the request and formatting the response off the database machine. Whether or not client software is employed, the database machine can still be organized with shared components.

11.2 CPU Cost vs Power Curve

This curve is exponential because the as we reach to high end of the power range we are also reaching the limits of available technology. It is always cheaper to use technology in the low end of the power range, but to get the same number of instructions per second, one has to devise ways to use multiple CPUs simultaneously (in parallel). For database applications where speed of response is paramount, we have to balance sharing duplicate copies of the data among the CPUs against the overhead of keeping the copies synchronized, or sharing a single copy of the data against the overhead of distributing data to the CPU that needs it (wherever it may be). Thus CPU cost scalability is not the only cost factor when determining database architecture.

The following elaborates the different architectures (posted separately as Parallel Architectures).

11.3 Shared-Nothing Database Architecture

This architecture is the most scalable but has the highest communications overhead. It is the only architecture that fits geographically dispersed databases (created to keep data where it is used). Tables are partitioned and parts are kept on different physical disks, but ORACLE is designed so it views the table as a whole.

Now we have to revisit how the ACID properties are preserved when updates involve different memories. Conceptually we give a global transaction monitor responsibility for understanding that this is a distributed transaction and it must be performed as many local transactions on CPUs connected to the different pieces of the table. Messages are exchanged between CPUs and the results are returned to the transaction monitor that then determines that the transaction completed if every local transaction completed.
Any transaction is in one of three states:

The system maintaining the transaction can be either functioning or crashed/recovering. We know that in recovery committed transactions are redone and uncommitted transactions are rolled back and Aborted.

Two-Phase Commit must be introduced to deal with the problem that when two different CPUs are processing parts of a transaction it is possible for one CPU to commit its part and the other CPU to abort its part. The consequence is that the database is inconsistent (money was being subtracted from one account and added to another by the two local transactions). A crash at any time can destroy attempts to coordinate commit processing (wiping the record of the transaction from the system except for the log entries).

The two-phase commit protocol introduces a new state for a transaction, the Prepared state to which distributed transactions go prior to committing or aborting. The prepared state has its own log and can be recovered if a crash occurs when a transaction is in this state.

  1. Phase one remote sites complete operations and enter prepared state, sending a message when prepared to TM. If TM does not receive notification that every CPU has achieved the prepared state, TM sends abort messages to all CPUs and the transaction is rolled back and retried.
  2. Phase two Commit authorization is distributed to all CPUs allowing the local transaction to complete processing. If system failure occurs after all CPUs are prepared, the recovery process will restore the failed CPU to prepared state, and the message processing system will deliver the commit authorization.

Problems DEADLOCK. Distributed transactions lock local rows or tables and seek to lock remote rows or tables. If two transactions in different CPUs begin by locking their local data and then seek a lock on each other's data, each will be told to wait (the cycle could involve many transactions and CPUs). Because no local scheduler has the ability to find the precedence cycle as it occurs in different transaction tables, the deadlock cannot be broken. The solution is the same as with Ethernet, timeout the transation and do a random delay before restarting it (thus allowing a different lock request sequence).

BLOCKING. While transactions are in the prepared state they retain their data locks. If there is a delay due to a failure somewhere in the system, these locks block other transactions. A three-phase commit has been proposed but not available commercially.

REPLICATED DATA. Instead of partitioning data (striping tables across different sites) we could replicate data. However, that means that to read a data item a TM would have to get a lock on one of the CPUs having a copy of that data, but to write an item it would have to get a lock on ALL of the CPUs having that item so that they could all be updated.

11.4 Query Parallelism

For read-only queries of distributed databases the system wants to distribute data collection across the database efficiently (Intra-Query Parallelism)
select Account.name, Account.phone from Account where Account.age>=65;
The distributed TM will generate k identical queries to send to each of the k sites on which the Account table is partitioned, and each will send back their data. If there were an ORDERED BY clause, the results would have to be merged. If the request was for a count(*), the results would have to be added.

Thus the distributed TM manager's query processor would have to be smart both in distributing the query and in consolidating the response. Suppose now that the query involved a join.

The Future

Our text ends here with just a glimpse of the problems that must be solved to scale up parallel and distributed database architectures, but you can be sure that the vendors are chipping away at these problems and developing commercial implementations of the solutions devised by researchers. To begin with, ORACLE has a distributed transaction module , and there is much advice on how to deploy it.

However, we are at a stage when there is a lot of controversy over what architecture serves the near term commercial marketplace. The accompaning articles written at the time of ORACLE 9i's release  are representative.

The reason for the controversy is that data is being captured far faster than current technology can incorporate it.   Datawarehousing scalibility issues are already upon us, and Acxiom is not alone in having a working data set that overflows ORACLE's maximums.