Oracle® Communications Data Model Reference 11g Release 2 (11.2) Part Number E15886-04 |
|
|
PDF · Mobi · ePub |
This chapter includes the following sections:
In Oracle Communications Data Model, reference and lookup tables store master, reference, and dimensional data; and the base, derived, and aggregate tables store transaction and fact data at different granularities. The base tables store the transaction data at the lowest level of granularity, while the derived and aggregate tables store consolidated and summary transaction data.
Two types of Extract, Transform, and Load (ETL) operations populate the tables with data. The source-ETL operations populate the reference, lookup, and base tables with data from the source On-Line Transaction Processing (OTLP) applications. Additional Intra-ETL operations populate the derived and aggregate tables with the data in the base, reference, and lookup tables. While the source ETL operations are not a part of Oracle Communications Data Model, the Intra-ETL operations are.
There are two categories of Intra-ETL operations (scripts):
Derived Population: A database package containing scripts that populate the derived tables based on the content of the base, reference, and lookup tables.
Aggregate Population: A database package containing scripts to refresh the Oracle Communications Data Model aggregate tables, mostly Materialized Views, based on the content of the derived tables and some reference tables.
Derived tables are implemented using Oracle tables, while the Aggregate tables are implemented using Materialized Views.
Note:
Do not make changes to the ETL as such changes are not supported.The INTRA_ETL_FLW is actually a process flow designed using the Oracle Warehouse Builder Workflow component which includes the dependency of each individual sub process flow and executes each process flow in the proper order. The result of each table loading is tracked in DWC_ control tables.
For more information, see "Intra-ETL Process Flows" and "Executing the Intra-ETL" and Oracle Communications Data Model Operations Guide.
Oracle Communications Data Model Value_Lookup values contains the Lookup tables and its values which are used in Intra-ETL mapping. Hardcoded values contains the list of tables and values which are used in Join conditions & Filter conditions in Intra-ETL mapping.
Table 7-1 Shows the lookup tables and values which are used in Intra-ETL mapping.
Table 7-1 Value Lookup Values for Intra-ETL Mapping
SI No. | Hard Coded Value Table Name | Hard Coded Value Column | Value used | ETL Program Name | ETL Usage Type |
---|---|---|---|---|---|
1 |
|
|
|
|
Hardcoded |
2 |
|
|
|
|
Hardcoded |
3 |
|
|
|
|
Hardcoded |
4 |
|
|
|
|
Hardcoded |
5 |
|
|
|
|
Hardcoded |
6 |
|
|
|
|
Hardcoded |
7 |
|
|
|
|
Hardcoded |
8 |
|
|
|
|
Hardcoded |
9 |
|
|
|
|
Hardcoded |
10 |
|
|
|
|
Hardcoded |
|
|
|
Hardcoded |
||
11 |
|
|
|
|
Hardcoded |
14 |
|
|
|
|
Hardcoded |
15 |
|
|
|
|
Hardcoded |
16 |
|
|
|
|
Hardcoded |
17 |
|
|
|
|
Hardcoded |
18 |
|
|
|
|
Hardcoded |
19 |
|
|
|
|
Hardcoded |
20 |
|
|
|
|
Hardcoded |
|
|
|
Hardcoded |
||
21 |
|
|
|
|
Hardcoded |
22 |
|
|
|
|
Hardcoded |
23 |
|
|
|
|
Hardcoded |
24 |
|
|
|
|
Hardcoded |
25 |
|
|
|
|
Hardcoded |
26 |
|
|
|
|
Hardcoded |
27 |
|
|
|
|
Hardcoded |
28 |
|
|
|
|
Hardcoded |
29 |
|
|
|
|
Hardcoded |
30 |
|
|
|
|
Hardcoded |
31 |
|
|
|
|
Hardcoded |
32 |
|
|
|
|
Hardcoded |
33 |
|
|
|
|
Hardcoded |
34 |
|
|
|
|
Hardcoded |
35 |
|
|
|
|
Hardcoded |
36 |
|
|
|
|
Hardcoded |
37 |
|
|
|
Hardcoded |
|
38 |
|
|
|
|
Hardcoded |
39 |
|
|
|
|
Hardcoded |
40 |
|
|
|
|
Hardcoded |
41 |
|
|
|
|
Hardcoded |
44 |
|
|
|
|
Hardcoded |
1 |
|
|
|
|
Value_Lookup |
2 |
|
|
|
|
Value_Lookup |
3 |
|
|
|
|
Value_Lookup |
4 |
|
|
|
|
Value_Lookup |
6 |
|
|
|
|
Value_Lookup |
7 |
|
|
|
|
Value_Lookup |
8 |
|
|
|
|
Value_Lookup |
9 |
|
|
|
|
Value_Lookup |
10 |
|
|
|
|
Value_Lookup |
16 |
|
|
|
|
Value_Lookup |
17 |
|
|
|
|
Value_Lookup |
18 |
|
|
|
|
Value_Lookup |
Shows the Oracle Warehouse Builder mapping to populate derived tables.
Populate target table DWD_ACCT_DEBT_DAY
. For more information, see ACCOUNT DEBT DAY DRVD.
Populate target table DWD_ACCT_PYMT_DAY
. For more information, see ACCOUNT PAYMENT DAY DRVD.
Populate target table DWD_ACCT_PYMT_MTHD_STAT_HIST
. For more information, see ACCOUNT PAYMENT METHOD STATUS HIST DRVD.
Populate target table DWD_ACCT_RFND_DAY
. For more information, see ACCOUNT REFUND DAY DRVD.
Populate target table DWD_ACCT_STAT
. For more information, see ACCOUNT STATUS DRVD.
Populate target table DWD_ACCT_STTSTC
. For more information, see ACCOUNT STATISTIC DRVD.
Table 7-7 DWD_ACCT_STTSTC__MAP ETL Mapping
Source Table Name |
---|
DWB_ACCT_BAL_HIST |
DWB_ACCT_CRDT_LMT |
DWB_ACCT_PYMT |
DWB_ACCT_STAT_HIST |
DWB_EVT_LYLTY_PROG |
DWB_PRTY_INTRACN_THRD |
DWL_AGE_BND |
DWL_AGE_ON_NET_BND |
DWL_ARPU_BAND |
DWL_CHRN_RSN |
DWL_CUST_RVN_BND |
DWL_DEBT_AGNG_BND |
DWR_ACCT |
DWR_ACCT_1 |
DWR_ACCT_PREF_PYMT_MTHD |
DWR_ADDR_LOC |
DWR_BSNS_MO |
DWR_CNRT |
DWR_CUST |
DWR_DAY |
DWR_PROD |
DWR_PROD_MKT_PLN |
DWR_PRTY |
DWR_PRTY_LYLTY_PROG_PRTCPTN |
DWR_SBRP |
DWR_SL_CHNL_RPRSTV |
Populate target table DWD_ARPU_BASE
. For more information, see ARPU BASE DRVD.
Populate target table DWD_CALL_CNTR_CALL_DAY
. For more information, see CALL CENTER CALL DAY DRVD.
Populate target table DWD_CALL_CNTR_CASE_DAY
. For more information, see CALL CENTER CASE DAY DRVD.
Populate target table DWD_CANBLZTN_DTL_DAY
. For more information, see CANNIBALIZATION DETAIL DAY DRVD.
Populate target table DWD_CMISN_DAY
. For more information, see COMMISSION DAY DRVD.
Populate target table DWD_CNCT_DSCNCT_DAY
. For more information, see CONNECT DISCONNECT DAY DRVD.
Contains connect and disconnect information of particular subscriber.
Populate target table DWD_CNRT_CHNG
. For more information, see CONTRACT CHANGED DRVD.
Populate target table DWD_COST_CUST
. For more information, see COST CUSTOMER DRVD.
Populate target table DWD_COST_ORG
. For more information, see COST ORGANIZATIONAL DRVD.
Populate target table DWD_CRDT_CTGRY
. For more information, see CREDIT CATEGORY DRVD.
Populate target table DWD_CUST_ACQSTN_SUMM_DAY
. For more information, see CUSTOMER ACQUISITION SUMMARY DAY DRVD.
Populate target table DWD_CUST_DEBT_COLLCTN
. For more information, see CUSTOMER DEBT COLLECTION DRVD.
Populate target table DWD_EXTRNL_DEBT_COLLCTN_DAY
. For more information, see EXTERNAL DEBT COLLECTION DAY DRVD.
Populate target table DWD_GIVE_AWAY_ITEM_DAY
. For more information see GIVE AWAY ITEM DAY DRVD.
Populate target table DWD_HNDST_SUBSDY_DAY
. For more information, see HANDSET SUBSIDY DAY DRVD.
Table 7-23 DWD_HNDST_SUBSDY_DAY__MAP ETL Mapping
Source Table Name |
---|
DWB_ACCT_CRDT_LMT |
DWB_CUST_ORDR |
DWB_CUST_ORDR_LN_ITEM |
DWB_CUST_ORDR_PYMT |
DWB_EVT_GFT_RDMPTN |
DWB_EVT_LYLTY_PROG_RDMPTN |
DWL_AGE_ON_NET_BND |
DWR_CNRT |
DWR_CUST |
DWR_DAY |
DWR_HNDST_MDL |
DWR_ITEM |
DWR_ORG_ITEM_SLNG_PRICE |
DWR_PROD |
DWR_PROD_MKT_PLN |
DWR_SBRP |
Populate target table DWD_INTRNL_DEBT_COLLCTN_DAY
. For more information, see INTERNAL DEBT COLLECTION DAY DRVD.
Populate target table DWD_INVC_ADJ
. For more information, see INVOICE ADJUSTMENT DRVD.
Populate target table DWD_LN_ACTVTN_TMNT_DAY
. For more information, see LINE ACTIVATION TERMINATION DAY DRVD.
Populate target table DWD_LYLTY_PROG_DAY
. For more information, see LOYALTY PROGRAM DAY DRVD.
Populate target table DWD_MKT_SHARE
. For more information, see MARKET SHARE DRVD.
Populate target table DWD_MKT_OPRTR_PRTNG
. For more information, see MARKET OPERATOR PORTING DERIVED.
The summary information about succeeded Number Porting between operators.
Populate target table DWD_PRPD_ACCT_STTSTC
. For more information, see PREPAID ACCOUNT STATISTIC DRVD.
Populate target table DWD_PRPD_ALWNCE_DAY
. For more information, see PREPAID ALLOWANCE DAY DRVD.
Populate target table DWD_PRPD_CALL_SUMM_DAY
. For more information, see PREPAID CALL SUMMARY DAY DRVD.
Populate target table DWD_PYMT_AGNG_DAY
. For more information, see PAYMENT AGING DAY DRVD.
Populate target table DWD_RDMPTN_DAY
. For more information, see REDEMPTION DAY DRVD.
Populate target table DWD_SBCRBR_CHRN_STTSTC
. For more information, see SUBSCRIBER CHURN STATISTIC DRVD.
Table 7-36 DWD_SBCRBR_CHRN_STTSTC__MAP
Source Table Name |
---|
DWB_ACCT_BAL_HIST |
DWB_ACCT_CRDT_LMT |
DWB_ACCT_PYMT |
DWB_EVT_LYLTY_PROG |
DWB_PRTY_INTRACN_THRD |
DWL_AGE_BND |
DWL_AGE_ON_NET_BND |
DWL_ARPU_BAND |
DWL_CUST_RVN_BND |
DWR_ACCT |
DWR_ADDR_LOC |
DWR_BSNS_MO |
DWR_CNRT |
DWR_CUST |
DWR_DAY |
DWR_HNDST_INSTNC |
DWR_PROD_MKT_PLN |
DWR_PRTY |
DWR_PRTY_LYLTY_PROG_PRTCPTN |
DWR_SBRP |
DWR_SBRP_ASGN |
DWR_SL_CHNL_RPRSTV |
Populate target table DWD_SHOP_EFFNCY_DAY
. For more information, see SHOP EFFICIENCY DAY DRVD.
Populate target table DWD_SHRD_PKG_USG_STTSTC_DAY
. For more information, see SHARED PACKAGE USAGE STATISTICS DAY DRVD.
Populate target table DWD_SL_DAY
. For more information, see SALES DAY DRVD.
Populate target table DWD_SL_RPRSTV_STTSTC_MO
. For more information, see SALES REPRESENTATIVE STATISTICS DRVD.
Populate the table DWD_SPLMNTR_SRVC_USG
. For more information, see SUPPLEMENTARY SERVICE USAGE DRVD.
Populate the table DWD_SUBSDY_AMT
. For more information, see SUBSIDY AMOUNT DRVD.
Populate target table DWD_VAS_SBRP_QCK_SUMM
. For more information, see VAS SUBSCRIPTION QUICK SUMMARY DRVD.
Shows the PL/SQL mapping to populate derived tables.
Populate target table DWD_DATA_USG_DAY
. For more information, see DATA USAGE DAY DRVD.
Populate target table DWD_VAS_USG_DAY
. For more information, see VAS USAGE DAY DRVD.
Populate target table DWD_VOI_CALL_DAY
. For more information, see VOICE CALL DAY DRVD.
The INTRA_ETL_FLW is the complete Intra-ETL process designed using Oracle Warehouse Builder, and is composed of individual sub-process flows to populate derived aggregate tables, and relational materialized views where the data originates from base, reference, and lookup tables. This process flow respects the dependency of each individual program.
Figure 7-1 shows the main process flow INTRA_ETL_FLW.
The process flow INTRA_ETL_FLW is initialized from START_PROCESS, and this checks if any previous process flows are running. If any process is running then START_PROCESS jumps to END_ERROR or START_PROCESS generate the process number from the sequence. This process number is sent as input to the Derived Flow.
In the DRVD_FLW when the START process is initiated this generates the process number and is sent as input to the Derived mapping. Once the number is generated it updates the status at backend (Control Tables). If derived mapping is successful then the derived mapping checks the status in control tables.
The DRVD_FLW sub-process flow contains all the Oracle Warehouse Builder mappings for populating derived tables, based on the content of the base, reference, and lookup tables. This sub-process flow has a dependency on the AGGR_FLW. If the DRVD_FLW is successful then it navigates to AGGR_FLW otherwise the process ends.
Figure 7-2 shows the DRVD_FLW sub-process flow for populating the derived tables.
Figure 7-2 Intra-ETL Derived Flow Sub-process (DRVD_FLW)
After the DRVD_FLW starts successfully, it moves to the fork. The sub-process FORK performs the derived mappings (these run in parallel). Once the activity is started then Start_Activity inserts one record in the control table and the state is set to 'Running'. The End_Activity updates the status in control tables (the state mapping is COMPLETED-SUCCESS or COMPLETED-ERROR) in the control tables. The AND activity specifies whether all the parallel mappings have been completed or not and then switches to the next activity, for example END_SUCCESS. This DRVD_FLW depends on the AGGR_FLW sub-process flow.
The AGGR_FLW sub-process flow contains PL/SQL code using Partitions Change Tracking Strategy for refreshing all the aggregate tables which are Materialized Views in Oracle Communications Data Model.
Figure 7-3 shows the AGGR_FLW sub-process flow for refreshing all the aggregate tables.
Figure 7-3 Intra-ETL Aggregate Flow Sub-process (AGGR_FLW)
After the AGGR_FLW is initiated and started successfully it is moved to the Fork. The FORK process makes the aggregates to run in parallel. The AND activity specifies that all the parallel aggregates have been completed or not and then switches over to the next activity, (for example, END_SUCCESS).
The OLAP_FLW sub-process flow triggers the OLAP package which can load data from Oracle Communications Data Model aggregate tables to Oracle Communications Data Model Analytical Workspace and calculate the forecast data. It reads OLAP ETL parameters from DWC_OLAP_ETL_PARAMETER table.
Figure 7-4 shows the OLAP_FLW sub-process flow that triggers the OLAP packages.
Figure 7-4 Intra-ETL OLAP Flow Sub-process (OLAP_FLW)
The MINING_FLW sub process flow triggers the data mining model.
Figure 7-5 shows the MINING_FLW sub-process flow.
Figure 7-5 Intra-ETL Mining Flow Sub-process (MINING_FLW)
One component of Oracle Communications Data Model is the INTRA_ETL_FLW process flow which is a complete Intra-ETL process composed of sub process flows to populate the derived and aggregate tables and relational materialized views with the data from the base, reference, and lookup tables. This process flow respects the dependency of each individual program. The INTRA_ETL_FLW process flow executes the programs in the proper order.
You can execute the Intra ETL mappings and packages by executing the INTRA_ETL_FLW from Oracle Warehouse Builder.
The INTRA_ETL_FLW is the complete Intra ETL process designed using Oracle Warehouse Builder, and is composed of individual sub-process flows:
DRVD_FLW - This sub-process flow contains all the OWB mappings for populating derived tables based on the content of the base, reference, and lookup tables.
AGGR_FLW - This sub-process flow contains PL/SQL code using Partitions Change Tracking Strategy for refreshing all the aggregate tables which are Materialized Views in our OCDM
OLAP_FLW - The OLAP sub-process flow triggers the OLAP package which can load data from OCDM aggregate tables to OCDM Analytical Workspace and calculate the forecast data. It reads OLAP ETL parameters from DWC_OLAP_ETL_PARAMETER table
MINING_FLW - This sub process flow triggers the data mining model.
Note:
The INTRA_ETL_FLW process flow provided with Oracle Communications Data Model assume that there is no data in the derived tables and aggregate tables and views. Typically, you perform this type of load only when you first create your data warehouse. Later, you need to add additional data to the tables and refresh your views.The AGGR_FLW sub process flow is executed only after the successful execution of all the mappings in the DRVD_FLW. If there is an error in any individual mapping in the derived process flow (DRVD_FLW), the execution of the Aggregate Process flow (AGGR_FLW) is skipped and only after a successful refreshing of all materialized views in AGGR_FLW process flow MINING_FLW can be executed.
Using the Intra-ETL involves the following tasks:
Executing the Intra-ETL for Oracle Communications Data Model
Monitoring the Execution of the Intra-ETL Process
Recovering an Intra_ETL Process
Monitoring the Execution of the Intra-ETL Process
Two control tables, DWC_INTRA_ETL_PROCESS and DWC_INTRA_ETL_ACTIVITY, monitor the execution of the Intra-ETL process. For more information on these tables, see Table A-3 and Table A-4.
Each normal run, as opposed to an error-recovery run, of a separate Intra-ETL execution performs the following steps:
Inserts a record into table DWC_INTRA_ETL_PROCESS with a monotonically increasing system generated unique process key, SYSDATE as process start time, RUNNING as the process status, input date range in the fields FROM_DATE_ETL and TO_DATE_ETL.
Invokes each of the individual Intra-ETL programs in the appropriate order of dependency. Before the invocation of each program, the procedure inserts a record into the Intra-ETL Activity detail table DWC_INTRA_ETL_ACTIVITY with a system generated unique activity key, the process key value corresponding to the Intra-ETL process, individual program name as the Activity Name, a suitable activity description, SYSDATE as activity start time, RUNNING as the activity status.
Updates the corresponding record in the DWC_INTRA_ETL_ACTIVITY table for the activity end time and activity status after the completion of each individual ETL program (either successfully or with errors. For successful completion of the activity, the procedure updates the status as COMPLETED-SUCCESS. If an error occurs, the procedure updates the activity status as 'COMPLETED-ERROR', and also updates the corresponding error detail in the ERROR_DTL column.
Updates the record corresponding to the process in the DWC_INTRA_ETL_ PROCESS table for the process end time and status, after the completion of all individual intra-ETL programs. If all the individual programs succeed, the procedure updates the status to 'COMPLETED-SUCCESS', otherwise it updates the status to 'COMPLETED-ERROR'. You can monitor the execution state of the Intra-ETL, including current process progress, time taken by individual programs, or the complete process, by viewing the contents of the DWC_INTRA_ETL_PROCESS and DWC_INTRA_ETL_ACTIVITY tables corresponding to the maximum process key. Monitoring can be done both during and after the execution of the Intra-ETL procedure.