Forum Stats

  • 3,824,768 Users
  • 2,260,416 Discussions
  • 7,896,309 Comments

Discussions

Ignoring all data load mappings but getting a successful DLR status

samlambert
samlambert Member Posts: 39 Green Ribbon
edited Mar 2, 2020 5:38AM in Financial Data Management

I have a DLR where I ignore mappings for all members. In the process details I get a success status for import and transform, but the DLR stops after that and the status fails. I can see where it's happening from the DLR's log (I've attached the validate step below).

I only want the DLR to fail if any mappings are not ignored. e.g. If COUNT <= 0 return SUCCESS, ELSE: RETURN FAIL.

So during the validate step, the script should see that there are no records and flag that as success, and then export and consolidate should also be successful if this condition is met.

I'm pretty new to FDMEE though so I'm not really sure how I should complete this task. I think an SQL command could do the trick. I was thinking if I create 'like' mappings and then add a script rule to each of the mappings that might work. Any assistance would be much appreciated!

2020-02-28 14:57:24,798 INFO  [AIF]: -------START VALIDATE STEP-------
2020-02-28 14:57:24,898 DEBUG [AIF]: CommData.validateData - START
2020-02-28 14:57:24,898 DEBUG [AIF]: Comm.executeScript - START
2020-02-28 14:57:24,899 INFO  [AIF]: Executing the following script: C:\Oracle\Middleware\user_projects\epmsystem1\FDMEE/data/scripts/event/BefValidate.py
2020-02-28 14:57:24,910 DEBUG [AIF]: Comm.executeVBScript - START
2020-02-28 14:57:24,911 DEBUG [AIF]: The following script does not exist: C:\Oracle\Middleware\user_projects\epmsystem1\FDMEE\data\scripts\event\BefValidate.vbs
2020-02-28 14:57:24,911 DEBUG [AIF]: Comm.executeVBScript - END
2020-02-28 14:57:24,911 DEBUG [AIF]: Comm.executeScript - END
2020-02-28 14:57:24,931 DEBUG [AIF]: CommMap.validateData - START
2020-02-28 14:57:24,938 INFO  [AIF]:
Validate Data Mappings for Period 'Oct-19'
2020-02-28 14:57:24,938 DEBUG [AIF]:
      SELECT COUNT(*) ROW_COUNT
      FROM TDATASEG
      WHERE LOADID = 536
      AND (PARTITIONKEY = 24 AND CATKEY = 5 AND PERIODKEY = '2019-10-31' AND RULE_ID = 39 AND VALID_FLAG = 'Y')
     
2020-02-28 14:57:24,941 WARN  [AIF]: Warning: No records exist for Period 'Oct-19'
2020-02-28 14:57:24,941 DEBUG [AIF]: CommData.updateWorkflow - START
2020-02-28 14:57:24,941 DEBUG [AIF]:
        UPDATE TLOGPROCESS
        SET PROCESSENDTIME=CURRENT_TIMESTAMP
           ,PROCESSSTATUS=12,PROCESSVAL=0,BLNVALDIRTY=0
           ,PROCESSVALNOTE='AIF_WF_VAL_VALIDATE_ERR;0'
        WHERE PARTITIONKEY=24 AND CATKEY=5 AND PERIODKEY='2019-10-31' AND RULE_ID=39
       
2020-02-28 14:57:24,952 DEBUG [AIF]: CommData.updateWorkflow - END
2020-02-28 14:57:24,953 INFO  [AIF]: Total Data Rows available for Export to Target: 0
2020-02-28 14:57:24,953 DEBUG [AIF]: CommMap.validateData - END
2020-02-28 14:57:24,957 INFO  [AIF]:
Validate Data Members for Period 'Oct-19'
2020-02-28 14:57:24,957 DEBUG [AIF]:
      SELECT COUNT(*) ROW_COUNT
      FROM TDATASEG
      WHERE LOADID = 536
      AND (PARTITIONKEY = 24 AND CATKEY = 5 AND PERIODKEY = '2019-10-31' AND RULE_ID = 39 AND VALID_FLAG = 'Y')
     
2020-02-28 14:57:24,960 ERROR [AIF]: Error: No records exist for Period 'Oct-19'
2020-02-28 14:57:24,960 DEBUG [AIF]: CommData.updateWorkflow - START
2020-02-28 14:57:24,962 DEBUG [AIF]:
        UPDATE TLOGPROCESS
        SET PROCESSENDTIME=CURRENT_TIMESTAMP
           ,PROCESSSTATUS=12,PROCESSVAL=0,BLNVALDIRTY=0
           ,PROCESSVALNOTE='AIF_WF_VAL_VALIDATE_ERR;0'
        WHERE PARTITIONKEY=24 AND CATKEY=5 AND PERIODKEY='2019-10-31' AND RULE_ID=39
       
2020-02-28 14:57:24,969 DEBUG [AIF]: CommData.updateWorkflow - END
2020-02-28 14:57:24,969 INFO  [AIF]: Total Data Rows available for Export to Target: 0
2020-02-28 14:57:24,981 FATAL [AIF]: Error in CommData.validateData
Traceback (most recent call last):
  File "<string>", line 4459, in validateData
RuntimeError: [u"Error: No records exist for Period 'Oct-19'"]

2020-02-28 14:57:24,981 DEBUG [AIF]: CommData.updateWorkflow - START
2020-02-28 14:57:24,981 DEBUG [AIF]:
        UPDATE TLOGPROCESS
        SET PROCESSENDTIME=CURRENT_TIMESTAMP
           ,PROCESSSTATUS=12,PROCESSVAL=0,BLNVALDIRTY=0
           ,PROCESSVALNOTE='AIF_WF_VAL_VALIDATE_ERR;0'
        WHERE PARTITIONKEY=24 AND CATKEY=5 AND PERIODKEY='2019-10-31' AND RULE_ID=39
       
2020-02-28 14:57:24,988 DEBUG [AIF]: CommData.updateWorkflow - END
2020-02-28 14:57:24,989 DEBUG [AIF]: Comm.executeScript - START
2020-02-28 14:57:24,991 INFO  [AIF]: Executing the following script: C:\Oracle\Middleware\user_projects\epmsystem1\FDMEE/data/scripts/event/AftValidate.py
2020-02-28 14:57:25,000 DEBUG [AIF]: Comm.executeVBScript - START
2020-02-28 14:57:25,001 DEBUG [AIF]: The following script does not exist: C:\Oracle\Middleware\user_projects\epmsystem1\FDMEE\data\scripts\event\AftValidate.vbs
2020-02-28 14:57:25,001 DEBUG [AIF]: Comm.executeVBScript - END
2020-02-28 14:57:25,001 DEBUG [AIF]: Comm.executeScript - END
2020-02-28 14:57:25,253 DEBUG [AIF]: Comm.finalizeProcess - START
2020-02-28 14:57:25,256 DEBUG [AIF]: CommData.updateRuleStatus - START
2020-02-28 14:57:25,257 DEBUG [AIF]:
    UPDATE AIF_BALANCE_RULES
    SET STATUS = CASE 'FAILED'
      WHEN 'SUCCESS' THEN
        CASE ( SELECT COUNT(*) FROM AIF_PROCESS_DETAILS pd WHERE pd.PROCESS_ID = 536 AND pd.STATUS IN ('FAILED','WARNING') )
          WHEN 0 THEN 'SUCCESS'
          ELSE ( SELECT MIN(pd.STATUS) FROM AIF_PROCESS_DETAILS pd WHERE pd.PROCESS_ID = 536 AND pd.STATUS IN ('FAILED','WARNING') ) END
      ELSE 'FAILED' END
    WHERE RULE_ID = 39
   
2020-02-28 14:57:25,264 DEBUG [AIF]: CommData.updateRuleStatus - END
2020-02-28 14:57:25,266 FATAL [AIF]: Error in COMM Validate Data
2020-02-28 14:57:25,268 DEBUG [AIF]: Comm.updateProcess - START
2020-02-28 14:57:25,287 DEBUG [AIF]: Comm.updateProcess - END
2020-02-28 14:57:25,297 DEBUG [AIF]: The fdmAPI connection has been closed.
2020-02-28 14:57:25,300 INFO  [AIF]: FDMEE Process End, Process ID: 536

Answers

  • user6692921
    user6692921 Member Posts: 179 Silver Badge
    edited Mar 2, 2020 4:27AM

    If there's nothing to load to the target the Validate step does not execute so you will get one gold fish.

  • samlambert
    samlambert Member Posts: 39 Green Ribbon
    edited Mar 2, 2020 4:50AM

    I'm getting one gold fish, but the whole DLR fails because there's nothing to load to the target. To be honest, I think I'm fine with the DLR failing, but I need to insert a script to tell me why the DLR is failing. i.e. is it failing because there's nothing to load to the target (which I would view as a success), or is it failing for some other reason (which I would view as a false positive)

  • SH_INT
    SH_INT Member Posts: 3,192 Bronze Crown
    edited Mar 2, 2020 5:38AM

    Lets take a step back. What are you actually trying to achieve with this DLR? As already mentioned, if at the validate stage your mappings are defaulting everything to IGNORE ("I have a DLR where I ignore mappings for all members"), FDMEEs native functionality will mean that you do not proceed past that step. I can't think of many reasons why you would want to do that so you need to be a bit clearer on exactly what you are trying to achieve

  • user6692921
    user6692921 Member Posts: 179 Silver Badge
    edited Mar 2, 2020 5:38AM

    If there is nothing to load to the target you will get one gold fish - ie it's been successful in importing and applying the maps and the Validate step is not necessary. The DLR will then fail if you have selected to Import and Export because it has not completed. You can create an AfterBatch script to check the status of your LOADID. Get the LOADID using the above SQL then get the status for the Import, Validate, Export, Check steps like this:

    import java.math.BigDecimal as BigDecimal

    stat = fdmAPI.getProcessStates(BigDecimal (sLoadID))

    strImpStatus = str(stat["IMPSTATUS"])

    strValStatus = str(stat["VALSTATUS"])

    strExpStatus = str(stat["EXPSTATUS"])

    strChkStatus = str(stat["CHKSTATUS"])