Forum Stats

  • 3,874,234 Users
  • 2,266,700 Discussions
  • 7,911,777 Comments

Discussions

ODI - Best approach??

AndyC1
AndyC1 Member Posts: 111
edited Nov 26, 2008 12:15AM in Data Integrator
Hi,
I have a req. to read data from a CSV file, perform some data transformations and store the transformed data into a DB(Oracle) . This mechanism should be then exposed as a WebService/DS which will be consumed by a BPEL process.
The BPEL process should only read the new records from the database table.

What will be the best approach to perform this?
Tagged:
«1

Answers

  • 566902
    566902 Member Posts: 809
    Create an interface which reads the CSV file, and loads the data into a database table.
    Put this into a package and generate the scenario.
    Use the ODIInvoke web service (see Public Web Services) to call this as a web service from ESB/BPEL.

    The last sentence is what I dont quite figure. Do you want to call the process to load from CSV file, or do you want to create a data service on the data in the CSV file?

    With ODI you can generate a Data Service on the table, which will give you access to all/some (by providing a fliter) of the data.

    If what you want is to be able to call one service which loads the data, and another which then allows you to read it, I suggest that you put the session ID into the data you load in a column, then you can call the second service requesting just the rows with that session ID.

    Does that make sense to you?
    566902
  • AndyC1
    AndyC1 Member Posts: 111
    What i want is that after the data has been transformed and loaded into the DB. the BPEL should be able to retreive the set of records from the DB.

    Like in the first instance their are 5 records(rows-R1,R2,R3,R4,R5) in the DB..and when the BPEL makes a call to Webservice/DS to retrieve records , it should get all the five records(R1-R5).

    And say after some time the BPEL process is made to run again, it should only retrieve new records added into the DB after transformation...say R6,R7 and not R1- R5.
  • AndyC1
    AndyC1 Member Posts: 111
    What i want is that after the data has been transformed and loaded into the DB. the BPEL should be able to retreive the set of records from the DB.

    Like in the first instance their are 5 records(rows-R1,R2,R3,R4,R5) in the DB..and when the BPEL makes a call to Webservice/DS to retrieve records , it should get all the five records(R1-R5).

    And say after some time the BPEL process is made to run again, it should only retrieve new records added into the DB after transformation...say R6,R7 and not R1- R5.

    The WebService should Extract (CSV-Oracle) and load the set of new data from the database(Oracle) for the BPEL.
  • 566902
    566902 Member Posts: 809
    It sounds like the approach I mentioned, putting in the session ID to the query may solve your problem.
    Another approach might be to run CDC on the Target Oracle Table, and use the ConsumeChangedxxx operation of the data service (generate the data service after turning on CDC and you will get this service automatically) to retrieve changes since the last.
    566902
  • AndyC1
    AndyC1 Member Posts: 111
    Hey,
    I have created Master and workin repo..now i guess next step is to create physical and logical schemas...can u ps guide me thru the steps involved in doing so...and if u have any document/tutorial for ODI , ps share the same..

    ty
  • AndyC1
    AndyC1 Member Posts: 111
    After creating an interface i dont get to see the Source Data Set in the flow panel(where we have to choose the LKM and IKM) i only see the staging+target area in the panel...

    i think im missing the KM creation/setup..ps lemme knw how to configure the LKM's for (Oracle-Oracle) and (CSV-Oracle) and the IKM...

    ty
  • 566902
    566902 Member Posts: 809
    This situation occurs when the source and target are in the same physical data server. ODI Understands when the two are accessible without the need for an LKM.
  • AndyC1
    AndyC1 Member Posts: 111
    So,
    How do i figure out about the IKM...
    ty
  • 566902
    566902 Member Posts: 809
    Have you imported any KMs into your project yet?
    If you click on th ecaption (Staging+Target) at the top of that box, it should open up the property pane for you to be able to select KMs from the drop-down.
  • AndyC1
    AndyC1 Member Posts: 111
    no, I hae not imported any of the KMs...actually the pt. is i've jus started with ODI so dnt hv a clear pic abut hw to go abt it..
    wat abt importin KM? do we need to create em...any docs for the same??
This discussion has been closed.