The outages of the underlying software, hardware, communications, and storage layers can cause application execution to fail. In the worst cases, the middle-tier servers may need to be restarted to deal with the logon stormsFoot 1 . To overcome such problems, Oracle Database 12c Release 1 (12.1) introduces the Application Continuity feature that masks database outages to the application and end users are not exposed to such outages.
Note:
Application Continuity is a feature of the Oracle JDBC Thin driver and is not supported by JDBC OCI driver.Application Continuity provides a general purpose, application-independent solution that enables recovery of work from an application perspective, after the occurrence of a planned or unplanned outage. The outage can be related to system, communication, or hardware following a repair, a configuration change, or a patch application.
See Also:
Oracle Database Development Guide for more information about Application Continuity
Oracle Database Development Guide for more information about Transaction Guard
This chapter discusses the JDBC aspect of Application Continuity in the following sections:
Configuring Oracle Database for Application Continuity for Java
Identifying Request Boundaries in Application Continuity for Java
Registering a Connection Initialization Callback in Application Continuity for Java (optional)
Delaying the Reconnection in Application Continuity for Java
You must use either the oracle.jdbc.replay.OracleDataSourceImpl
or oracle.jdbc.replay.OracleConnectionPoolDataSourceImpl
data source to obtain JDBC connections. They are new Oracle JDBC data sources, and work similarly to the existing non-XA data sources, such as oracle.jdbc.pool.OracleDataSource
. You can use both in a standalone manner, or configure them as connection factories for a connection pool, such as Universal Connection Pool (UCP), or Oracle WebLogic Server connection pool.
The following code snippet illustrates their usage in a standalone JDBC application:
import java.sql.Connection; import javax.sql.PooledConnection; import oracle.jdbc.OracleConnection; import oracle.jdbc.replay.OracleDataSourceFactory; import oracle.jdbc.replay.OracleDataSource; import oracle.jdbc.replay.OracleConnectionPoolDataSource; ... { ...... OracleDataSource rds = OracleDataSourceFactory.getOracleDataSource(); rds.setUser(user); rds.setPassword(passwd); rds.setURL(url); ...... // Other data source configuration like callback, timeouts, etc. Connection conn = rds.getConnection(); ((OracleConnection) conn).beginRequest(); // Explicit request begin ...... // JDBC calls protected by Application Continuity ((OracleConnection) conn).endRequest(); // Explicit request end conn.close(); OracleConnectionPoolDataSource rcpds = OracleDataSourceFactory.getOracleConnectionPoolDataSource(); rcpds.setUser(user); rcpds.setPassword(passwd); rcpds.setURL(url); ...... // other data source configuration like callback, timeouts, and so on PooledConnection pc = rcpds.getPooledConnection(); Connection conn2 = pc.getConnection(); // Implicit request begin ...... // JDBC calls protected by Application Continuity conn2.close(); // Implicit request end ......
See Also:
"Data Sources and URLs" for more information about Oracle JDBC data sourcesYou must remember the following points while using the connection URL:
Always use the thin driver in the connection URL.
Always connect to a service. Never use instance_name
or SID because these do not direct to known good instances and SID is deprecated.
If the addresses in the ADDRESS_LIST
at the client does not match the REMOTE_LISTENER
setting for the database, then it does not connect showing services cannot be found
. So, the addresses in the ADDRESS_LIST
at the client must match the REMOTE_LISTENER
setting for the database:
If REMOTE_LISTENER
is set to the SCAN_VIP
, then the ADDRESS_LIST
uses SCAN_VIP
If REMOTE_LISTENER
is set to the host VIPs, then the ADDRESS_LIST
uses the same host VIPs
If REMOTE_LISTENER
is set to both SCAN_VIP
and host VIPs, then the ADDRESS_LIST
uses SCAN_VIP
and the same host VIPs
Note:
For Oracle clients prior to release 11.2, theADDRESS_LIST
must be upgraded to use SCAN, which means expanding the ADDRESS_LIST
to three ADDRESS
entries corresponding to the three SCAN IP addresses.
If such clients connect to a database that is upgraded from an earlier release through Database Upgrade Assistant, then you must retain the ADDRESS_LIST
of these clients set to the HOST VIPs. However, if REMOTE_LISTENER
is changed to ONLY SCAN
, or the clients are moved to a newly installed Oracle Database 12c Release 1, where REMOTE_LISTENER
is ONLY SCAN
, then they do not get a complete service map, and may not always be able to connect.
Set RETRY_COUNT
, CONNECT_TIMEOUT
, and TRANSPORT_CONNECT_TIMEOUT
parameters in the connection string. This is a general recommendation for configuring the JDBC thin driver connections, starting from Oracle Database Release 11.2.0.2. These settings improve acquiring new connections at runtime, at replay, and during work drains for planned outages.
The CONNECT_TIMEOUT
parameter is equivalent to the SQLNET.OUTBOUND_CONNECT_TIMEOUT
parameter in the sqlnet.ora
file and applies to the full connection. The TRANSPORT_CONNECT_TIMEOUT
parameter applies as per the ADDRESS
parameter. If the service is not registered for a failover or restart, then retrying is important when you use SCAN. For example, for using remote listeners pointing to SCAN addresses, you should use the following settings:
jdbc:oracle:thin:@(DESCRIPTION = (TRANSPORT_CONNECT_TIMEOUT=3) (RETRY_COUNT=20)(FAILOVER=ON) (ADDRESS_LIST =(ADDRESS=(PROTOCOL=tcp) (HOST=CLOUD-SCANVIP.example.com)(PORT=5221)) (CONNECT_DATA=(SERVICE_NAME=orcl))) REMOTE_LISTENERS=CLOUD-SCANVIP.example.com:5221
Similarly, for using remote listeners pointing to VIPs at the database, you should use the following settings:
jdbc:oracle:thin:@(DESCRIPTION = (TRANSPORT_CONNECT_TIMEOUT=3) (CONNECT_TIMEOUT=60)(RETRY_COUNT=20)(FAILOVER=ON) (ADDRESS_LIST= (ADDRESS=(PROTOCOL=tcp)(HOST=CLOUD-VIP1.example.com)(PORT=5221) ) (ADDRESS=(PROTOCOL=tcp)(HOST=CLOUD-VIP2.example.com)(PORT=5221) ) (ADDRESS=(PROTOCOL=tcp)(HOST=CLOUD-VIP3.example.com)(PORT=5221) )) (CONNECT_DATA=(SERVICE_NAME=orcl))) REMOTE_LISTENERS=CLOUD-VIP1.example.com:5221
See Also:
Oracle Database Net Services Reference for more information about local naming parameters
Oracle Real Application Clusters Administration and Deployment Guide
You must have the following configuration for Oracle Database to use Application Continuity for Java:
Use Oracle Database 12c Release 1 (12.1)
If you are using Oracle Real Application Clusters (Oracle RAC) or Oracle Data Guard, then ensure that FAN is configured with Oracle Notification System (ONS) to communicate with Oracle WebLogic Server or the Universal Connection Pool (UCP)
Use an application service for all database work. To create the service you must:
Run the SRVCTL
command if you are using Oracle RAC
Use the DBMS_SERVICE
package if you are not using Oracle RAC
Set the required properties on the service for replay and load balancing. For example, set:
FAILOVER_TYPE = TRANSACTION
for using Application Continuity
COMMIT_OUTCOME
= TRUE
for enabling Transaction Guard
REPLAY_INITIATION_TIMEOUT = 900
for setting the duration in seconds for which replay will occur
FAILOVER_RETRIES = 30
for specifying the number of connection retries for each replay
FAILOVER_DELAY = 10
for specifying the delay in seconds between connection retries
GOAL = SERVICE_TIME
, if you are using Oracle RAC, then this is a recommended setting
CLB_GOAL = LONG
, if you are using Oracle RAC, then this is a recommended setting
Do not use the database service, that is, the default service corresponding to the DB_NAME
or DB_UNIQUE_NAME
. This service is reserved for Oracle Enterprise Manager and for DBAs. Oracle does not recommend the use of the database service for high availability because this service cannot be:
Enabled and disabled
Relocated on Oracle RAC
Switched over to Oracle Data Guard
See Also:
Oracle Database Development Guide for more information on the operation and usage of Application Continuity.A Request is a unit of work on a physical connection to Oracle Database that is protected by Application Continuity. Request demarcation varies with specific use-case scenarios. A request begins when a connection is borrowed from the Universal Connection Pool (UCP) or WebLogic Server connection pool, and ends when this connection is returned to the connection pool.
Note:
You cannot borrow a connection from the Database Resident Connection Pool (DRCP) because DRCP does not work with Application Continuity. Refer to Chapter 23, "Database Resident Connection Pooling" for more information about DRCP.The JDBC driver provides explicit request boundary declaration APIs beginRequest
and endRequest
in the oracle.jdbc.OracleConnection
interface. These APIs enable applications, frameworks, and connection pools to indicate to the JDBC Replay Driver about demarcation points, where it is safe to release the call history, and to enable replay if it had been disabled by a prior request. At the end of the request, the JDBC Replay Driver purges the recorded history on the connection, where the API is called. This helps to further conserve memory consumption for applications that use the same connections for an extended period of time without returning them to the pool.
You do not need to make any changes to your application for identifying request boundaries, if the application uses connections from these Oracle connection pools, or from the oracle.jdbc.replay.OracleConnectionPoolDataSourceImpl
data source. However, for the connection pool to work, the application must get connections when needed, and release connections when not in use. This scales better and provides request boundaries transparently.
Applications that do not borrow and return connections from the Oracle connection pools should explicitly mark request boundaries. For example, if your application is using custom JDBC pools, then the beginRequest
method should be called at check-out and the endRequest
method should be called at check-in. These methods can also be used for standalone JDBC applications without a connection pool.
The APIs have no impact on the applications other than improving resource consumption, recovery, and load balancing performance. These APIs do not involve altering a connection state by calling any JDBC method, SQL, or PL/SQL. An error is returned if an attempt is made to begin or end a request while a local transaction is open.
Non-transactional session state (NTSS) is state of a database session that exists outside database transactions and is not protected by recovery. For applications that use stateful requests, the non-transactional state is re-established as the rebuilt session.
For applications that set state only at the beginning of a request, or for stateful applications that gain performance benefits from using connections with a preset state, one among the following callback options are provided:
In this scenario, the application builds up its own state during each request.
This scenario is applicable only to Universal Connection Pool (UCP) and Oracle WebLogic server. The application can be modified to take advantage of the preset state on connections. Connection Labeling APIs determine how well a connection matches, and use a callback to populate the gap when a connection is borrowed. All applications cannot use Connection Labeling because it requires re-coding to some extent.
In this scenario, the replay driver uses an application callback to set the initial state of the session during runtime and replay. The JDBC replay driver provides an optional connection initialization callback interface as well as methods for registering and unregistering such callbacks.
When registered, the initialization callback is executed at each successful reconnection following a recoverable error. An application is responsible for ensuring that the initialization actions are the same as that on the original connection before failover. If the callback invocation fails, then replay is disabled on that connection.
This section discusses initialization callbacks in the following sections:
To create a JDBC connection initialization callback, an application implements the oracle.jdbc.replay.ConnectionInitializationCallback
interface. One callback is allowed for every instance of the oracle.jdbc.replay.OracleDataSource
interface.
Note:
This callback is only invoked during failover, after a successful reconnection.The following example demonstrates a simple initialization callback implementation:
import oracle.jdbc.replay.ConnectionInitializationCallback; class MyConnectionInitializationCallback implements ConnectionInitializationCallback { public MyConnectionInitializationCallback() { ... } public void initialize(java.sql.Connection connection) throws SQLException { // Reset the state for the connection, if necessary (like ALTER SESSION) ... } }
The JDBC Replay Driver provides the registerConnectionInitializationCallback(ConnectionInitializationCallback cbk)
method in the oracle.jdbc.replay.OracleDataSource
interface for registering a connection initialization callback. One callback is allowed for every instance of the OracleDataSource
interface.
By default, when JDBC Replay Driver initiates a failover, the driver attempts to recover the in-flight work at an instance where the service is available. For doing this, the driver must first reestablish a good connection to a working instance. This reconnection can take some time if the database or the instance needs to be restarted before the service is relocated and published. So, the failover should be delayed until the service is available from another instance or database.
You need to use the FAILOVER_RETRIES
and FAILOVER_DELAY
parameters to manage reconnecting. These parameters can work well in conjunction with a planned outage, for example, an outage that may make a service unavailable for several minutes. While setting the FAILOVER_DELAY
and FAILOVER_RETRIES
parameters, check the value of the REPLAY_INITIAITION_TIMEOUT
parameter first. The default value for this parameter is 900 seconds. A high value for the FAILOVER_DELAY
parameter can cause replay to be canceled.
Parameter Name | Possible Value | Default Value |
---|---|---|
FAILOVER_RETRIES |
Positive integer zero or above | 30 |
FAILOVER_DELAY |
Time in seconds | 10 |
This section provides configuration examples for service creation and modification in the following subsections:
If you are using Oracle RAC or Oracle RAC One, then use the SRVCTL
command to create and modify services in the following way:
For Policy-Managed Environments
srvctl add service -d codedb -s GOLD -g Srvpool -j SHORT -B SERVICE_TIME –z 30 –w 10 -commit_outcome TRUE -e TRANSACTION -replay_init_time 1800 -retention 86400 -notification TRUE
For Administrator-Managed Environments
srvctl add service -d codedb -s GOLD -r serv1 -a serv2 -j SHORT -B SERVICE_TIME –z 30 –w 10 -commit_outcome TRUE -e TRANSACTION -replay_init_time 1800 -retention 86400 -notification TRUE
If you are using a single-instance database, then use the DBMS_SERVICE
package to modify services in the following way:
declare params dbms_service.svc_parameter_array; begin params('FAILOVER_TYPE'):='TRANSACTION'; params('REPLAY_INITIATION_TIMEOUT'):=1800; params('RETENTION_TIMEOUT'):=604800; params('FAILOVER_DELAY'):=10; params('FAILOVER_RETRIES'):=30; params('commit_outcome'):='true'; params('aq_ha_notifications'):='true'; dbms_service.modify_service('[your service]',params); end; /
A mutable object is a variable, function return value, or other structure that returns a different value each time that it is called. For example, Sequence.NextVal
, SYSDATE
, SYSTIMESTAMP
, and SYS_GUID
. To retain the function results for named functions at replay, the DBA must grant KEEP
privileges to the user who invokes the function. This security restriction is imposed to ensure that it is valid for replay to save and restore function results for code that is not owned by that user.
See Also:
Oracle Database Development GuideYou can work with mutables values by using the standard GRANT
and REVOKE
interfaces in the following way:
The DATE_TIME
and SYS_GUID
syntax is as follows:
GRANT [KEEP DATE_TIME | KEEP SYS_GUID].. [to USER] REVOKE [KEEP DATE_TIME | KEEP SYS_GUID] … [from USER]
For example, for EBS standard usage with original dates
Grant KEEP DATE_TIME, KEEP SYS_GUID to [custom user]; Grant KEEP DATE_TIME, KEEP SYS_GUID to [apps user];
The Sequence syntax can be of the following types:
ALTER SEQUENCE [sequence object] [KEEP|NOKEEP];
This command retains the original values of sequence.nextval
for replaying, so that the keys match after replay. Most applications need to retain the sequence values at replay. The ALTER SYNTAX
is only for owned sequences.
GRANT KEEP SEQUENCES.. [to USER] on [sequence object]; REVOKE KEEP SEQUENCES … [from USER] on [sequence object];
For example, use the following command for EBS standard usage with original sequence values:
Grant KEEP SEQUENCES to [apps user] on [sequence object]; Grant KEEP SEQUENCES to [custom user] on [sequence object];
The GRANT ALL
statement grants KEEP
privilege on all the objects of a user. However, it excludes mutable values, that is, mutable values require explicit grants.
Follow these rules while granting privileges on mutable objects:
If a user has KEEP
privilege granted on mutables values, then the objects inherit mutable access when the SYS_GUID
, SYSDATE
, and SYSTIMESTAMP
functions are called.
If the KEEP
privilege on mutable values on a sequence object is revoked, then SQL or PL/SQL blocks using that object will not allow mutable collection or application for that sequence.
If granted privileges are revoked between runtime and failover, then the mutable values that are collected are not applied for replay.
If new privileges are granted between runtime and failover, mutable values are not collected and these values are not applied for replay.
This section describes the following concepts:
If any application module uses a design that is unsuitable for replay, then the disable replay API disables replay on a per request basis. Disabling replay can be added to the callback or to the main code by using the disableReplay
method of the oracle.jdbc.replay.ReplayableConnection
interface. For example:
if (connection instanceof oracle.jdbc.replay.ReplayableConnection) { (( oracle.jdbc.replay.ReplayableConnection)connection).disableReplay(); }
Disabling replay does not alter the connection state by reexecuting any JDBC method, SQL or PL/SQL. When replay is disabled using the disable replay API, both recording and replay are disabled until that request ends. There is no API to reenable replay because it is invalid to reestablish the database session with time gaps in a replayed request. This ensures that replay runs only if a complete history of needed calls has been recorded.
By default, the JDBC replay driver replays following a recoverable error. The disable replay API can be used in the entry point of application modules that are unable to lose the database sessions and recover. For example, if the application uses the UTL_SMTP
package and does not want messages to be repeated, then the disableReplay
API affects only the request that needs to be disabled. All other requests continue to be replayed.
The following are scenarios to consider before configuring an application for replay:
Application Calls External PL/SQL Actions that Should not Be Repeated
Application Uses Time at the Middle-tier in the Execution Logic
During replay, autonomous transactions and external PL/SQL calls can have side effects that are separate from the main transaction. These side effects are replayed unless you specify otherwise and leave persistent results behind. These side effects include writing to an external table, sending email, forking sessions out of PL/SQL or Java, transferring files, accessing external URLs, and so on. For example, in case of PL/SQL messaging, suppose, you walk away in-between some work without committing and the session times out. Now, if you issue a Ctrl+C
command, then the foreground of a component fails. When you resubmit the work, then this side effect can also be repeated.
See Also:
Oracle Database Development Guide for more information about potential side effects of Application ContinuityYou must make a conscious decision about whether to enable replay for external actions or not. For example, you can consider the following situations where this decision is important:
Using the UTL_HTTP
package to issue a SOA call
Using the UTL_SMTP
package to send a message
Using the UTL_URL
package to access a web site
Use the disableReplay
API if you do not want such external actions to be replayed.
You can configure an application for replay if the application synchronizes independent sessions using volatile entities that are held until commit, rollback, or session loss. In this case, the application synchronizes multiple sessions connected to several data sources that are otherwise inter-dependent using resources such as a database lock. This synchronization may be fine if the application is only serializing these sessions and understands that any session may fail. However, if the application assumes that a lock or any other volatile resource held by one data source implies exclusive access to data on the same or a separate data source from other connections, then this assumption may be invalidated when replaying.
During replay, the driver is not aware that the sessions are dependent on one session holding a lock or other volatile resource. You can also use pipes, buffered queues, stored procedures taking a resource (such as a semaphore, device, or socket) to implement the synchronization that are lost by failures.
Note:
TheDBMS_LOCK
does not replay in the restricted version.In this case, the application uses the wall clock at the middle-tier as part of the execution logic. The JDBC replay driver does not repeat the middle-tier time logic, but uses the database calls that execute as part of this logic. For example, an application using middle-tier time may assume that a statement executed at Time T1 is not reexecuted at Time T2, unless the application explicitly does so.
If an application caches ROWIDs, then access to these ROWIDs may be invalidated due to database changes. Although a ROWID uniquely identifies a row in a table, a ROWID may change its value in the following situations:
The underlying table is reorganized
An index is created on the table
The underlying table is partitioned
The underlying table is migrated
The underlying table is exported and imported using EXP/IMP/DUL
The underlying table is rebuilt using Golden Gate or Logical Standby or other replication technology
The database of the underlying table is flashed back or restored
It is bad practice for an application to store ROWIDs for later use as the corresponding row may either not exist or contain completely different data.
In this case, the following are replayed during a replay:
Autonomous transactions
Opening of back channels separate to the main transaction side effects
Examples of back channels separate to the main transaction include writing to an external table, sending email, forking sessions out of PL/SQL or Java, writing to output files, transferring files, and writing exception files. Any of these actions leave persistent side effects in the absence of replay. Back channels can leave persistent results behind. For example, if a user leaves a transaction midway without committing and the session times out, then the user presses Ctrl+C, the foreground or any component fails. If the user resubmits work, then the side effects can be repeated.
SYSCONTEXT
options comprise a location-independent set such as National Language Support (NLS) settings, ISDBA
, CLIENT_IDENTIFIER
, MODULE
, and ACTION
, and a location-dependent set that uses physical locators. Typically, an application does not use the physical identifier, except in testing environments. If physical locators are used in mainline code, then the replay finds the mismatch and rejects it. However, it is fine to use physical locators in callbacks.
select sys_context('USERENV','DB_NAME') ,sys_context('USERENV','HOST') ,sys_context('USERENV','INSTANCE') ,sys_context('USERENV','IP_ADDRESS') ,sys_context('USERENV','ISDBA') ,sys_context('USERENV','SESSIONID') ,sys_context('USERENV','TERMINAL') ,sys_context('USERENV',ID') from dual
The JDBC Replay driver supports standard JDK logging. Logging is enabled using the Java command-line -Djava.util.logging.config.file=<file>
option. Log level is controlled with the oracle.jdbc.internal.replay.level
attribute in the log configuration file. For example:
oracle.jdbc.internal.replay.level = FINER|FINEST
where, FINER
produces external APIs and FINEST
produces large volumes of trace. You must use FINEST
only under supervision.
If you use the java.util.logging.XMLFormatter
class to format a log record, then the logs are more readable but larger. If you are using replay with FAN enabled on UCP or WebLogic Server, then you should also enable FAN-processing logging.
Following is the example of a configuration file for logging configuration.
oracle.jdbc.internal.replay.level = FINER handlers = java.util.logging.ConsoleHandler java.util.logging.ConsoleHandler.level = ALL java.util.logging.ConsoleHandler.formatter = java.util.logging.XMLFormatter
Following is the example of a properties
file for logging configuration.
oracle.jdbc.internal.replay.level = FINEST
# Output File Format (size, number and style) # count: Number of output files to cycle through, by appending an integer to the base file name: # limit: Limiting size of output file in bytes handlers = java.util.logging.FileHandler java.util.logging.FileHandler.pattern = [file location]/replay_%U.trc java.util.logging.FileHandler.limit = 500000000 java.util.logging.FileHandler.count = 1000 java.util.logging.FileHandler.formatter = java.util.logging.XMLFormatter
Footnote Legend
Footnote 1: "A Logon storm is a sudden increase in the number of client connection requests."