2 Using TimesTen and IMDB Cache

This chapter describes how TimesTen and IMDB Cache can be used to enable applications that require real-time access to data. It includes the following sections:

Uses for TimesTen

TimesTen can be used as:

  • The primary database for real-time applications. All data needed by the applications resides in the TimesTen database.

  • A data utility for accelerating performance-critical points in an architecture. For example, providing persistence and transactional capabilities to a message queuing system might be achieved by using TimesTen as the repository for the messages.

  • A data integration point for multiple data sources on top of which new applications can be built. For example, an organization may have large amounts of information stored in several data sources, but only subsets of this information may be relevant to running its daily business. A suitable architecture would be to pull the relevant information from the different data sources into one TimesTen operational database to provide a central repository for the data of immediate interest to the applications.

Uses for IMDB Cache

IMDB Cache can be used as:

  • A real-time data manager for specific tasks in an overall workflow in collaboration with a disk-based RDBMS like the Oracle Database. For example, a phone billing application may capture and store recent call records in the IMDB Cache database while storing information about customers, their billing addresses and credit information in an Oracle database. It can also age and keep archives of all call records in the Oracle database. Thus the information that requires real-time access is stored in the IMDB Cache database while the information needed for longer-term analysis, auditing, and archival is stored in the Oracle database.

  • A read-only cache. Oracle data can be cached in an IMDB Cache read-only cache group. Read-only cache groups are automatically refreshed when Oracle tables are updated. A read-only cache group provides fast access to reference data such as look-up tables and subscriber profiles.

  • An updatable cache. Oracle data can be cached in IMDB Cache updatable cache groups. Transactions on the cache groups can be committed synchronously or asynchronously to the associated Oracle tables.

  • A distributed cache. Oracle data can be cached in multiple installations of IMDB Cache running on different machines to provide scalability. You can configure a dynamic distributed cache in which records are loaded automatically and aged automatically.

TimesTen application scenario

This section describes an application scenario to illustrate how TimesTen can be integrated as part of a data management solution.

The application scenario is a Real-time quote service application. It uses TimesTen to store stock quotes from a data feed for access by program trading applications. Quote data is collected from the data feed and published on a real-time message bus. The data is read from the message bus and stored in TimesTen, where it is accessed by the program trading applications.

Real-time quote service application

A financial services company is adding a real-time quote and news service to its online trading facility. The real-time quote service will read an incoming news wire from a major market data vendor and make a subset of the data available to trading applications that manage the automated trading operations for the company. The company plans to build an infrastructure that can accommodate future expansion to provide real-time quotes, news, and other trading services to retail subscribers.

The real-time quote service includes a NewsReader process that reads incoming data from a real-time message bus that is constantly fed data from a news wire. Each NewsReader is paired with a backup NewsReader that independently reads the data from the bus and inserts it into a separate TimesTen database. In this way, the message bus is used to fork incoming data to two TimesTen databases for redundancy. In this scenario, forking the data from the message bus is more efficient than using TimesTen replication.

One NewsReader makes the stock data available to a trading application, while the other serves as a hot standby backup to be used by the application if a failure occurs. The current load requires four NewsReader pairs, but more NewsReader pairs can be added in the future to scale the service to deliver real-time quotes to other types of clients over the Web or cellular phone.

Figure 2-1 shows the configuration for capturing data from a message bus and feeding it to NewsReaders.

Figure 2-1 Capturing feed data from a message bus

Description of Figure 2-1 follows
Description of "Figure 2-1 Capturing feed data from a message bus"

As shown in Figure 2-2, the NewsReader updates stock price data in a Quotes table in the TimesTen database. Less dynamic earnings data is updated in an Earnings table. The Stock columns in the Quotes and Earnings tables are linked through a foreign key relationship.

The purpose of the trading application is to track only those stocks with PE ratios below 50, then use internal logic to analyze the current stock price and trading volume to determine whether to place a trade using another part of the trading facility. For maximum performance, the trading application implements an event facility that uses the TimesTen Transaction Log API (XLA) to monitor the TimesTen transaction log for updates to the stocks of interest.

To provide the fastest possible access to such updates, the company creates a materialized view, named PE_Alerts, with a WHERE clause that calculates the PE ratio from the Price column in the Quotes table and the Earns column in the Earnings table. By using the XLA event facility to monitor the transaction log for price updates in the materialized view, the trading application receives alerts only for those stocks that meet its trading criteria.

Figure 2-2 Using materialized views and XLA

Description of Figure 2-2 follows
Description of "Figure 2-2 Using materialized views and XLA"

IMDB Cache application scenarios

This section describes scenarios that illustrate how IMDB Cache can be used:

  • Call center application: Uses IMDB Cache as an application-tier cache to hold customer profiles maintained in an Oracle database.

  • Caller usage metering application: Uses IMDB Cache to store metering data on the activities of cellular callers. The metering data is collected from multiple TimesTen nodes distributed throughout a service area and archived in a central Oracle database for use by a central billing application.

Call center application

Advance Call Center provides customer service for Wireless Communications.

Figure 2-3 shows that the call center system includes a central server that hosts back-end applications and an Oracle database that stores the customer profiles.

Figure 2-3 Dynamically loading Oracle data to IMDB Cache nodes

Description of Figure 2-3 follows
Description of "Figure 2-3 Dynamically loading Oracle data to IMDB Cache nodes"

To manage a large volume of concurrent customer sessions, the call center has deployed several application server nodes and periodically deploys additional nodes as its customer base increases. Each node contains an IMDB Cache database. When a customer contacts the call center, the user is automatically routed to an available application server node and the appropriate customer profile is dynamically loaded from the Oracle database into the cache database.

When a customer completes a call, changes to the customer profile are flushed from IMDB Cache database to the Oracle database. Least recently used (LRU) aging is configured to remove inactive customer profiles from the IMDB Cache database.

If the same customer contacts the call center again shortly after the first call and is connected to a different application server node, the customer profile is dynamically loaded to the new node from either the Oracle database or from the first IMDB Cache node, depending on where the most recent copy resides. The IMDB Cache determines where the most recent copy resides and uses peer-to-peer communication to exchange information with other IMDB Cache databases in its grid. It also manages concurrent updates to the same data within its grid.

All of the customer data is stored in the Oracle database. The Oracle database is much larger than the combined IMDB Cache databases and is best accessed by applications that do not require the real-time performance of IMDB Cache but do require access to large amounts of data. Such applications may include a billing application and a data mining application.

As the customer base increases and demands to serve more customers concurrently increases, the call center may decide to deploy additional application server nodes. New IMDB Cache members can join the IMDB Cache grid with no disruption to ongoing requests in the grid. Similarly, failures or removal of individual nodes do not disrupt operations in the rest of the grid.

Caller usage metering application

Wireless Communications has a usage metering application that keeps track of the duration of each cellular call and the services used. For example, if a caller makes a regular call, a base rate is applied for the duration of the call. If a caller uses special features like roaming, extra charges are applied.

The usage metering application must efficiently monitor up to 100,000 concurrent calls, gather usage data on each call, and store the data in a central database for use by other applications that generate bills, reports, audits, and so on.

The company uses an IMDB Cache database to store the caller data that is of immediate interest to the usage metering application and to warehouse all of the other data in the Oracle database. The company distributes multiple installations of the usage metering application and IMDB Cache on individual nodes throughout its service areas. For maximum performance, each usage metering application connects to its local IMDB Cache database by an ODBC direct driver connection.

Figure 2-4 shows the configuration.

Figure 2-4 Distributed caching of usage data

Description of Figure 2-4 follows
Description of "Figure 2-4 Distributed caching of usage data"

A usage metering application and IMDB Cache are deployed on each node to handle the real-time processing for calls beginning and terminating at different geographical locations delineated by area code. For each call, the local node stores a separate record for the beginning and the termination of a call. This is because the beginning of a cellular call might be detected by one node and its termination by another node.

Transactions that impact revenue (inserts and updates) must be durable. To ensure data availability, each IMDB Cache database is replicated to a standby database.

Each time a customer makes, receives or terminates a cellular call, the application inserts a record of the activity into the Calls table in the IMDB Cache database. Each call record includes a timestamp, unique identifier, originating host's IP address, and information on the services used.

An IMDB Cache process periodically archives the rows in the Calls table to the Oracle database. After the call records have been successfully archived in the Oracle database, they are deleted from the IMDB Cache database by a time-based aging process.