4 Replies Latest reply: Nov 25, 2012 8:57 PM by 976107 RSS

    ETL Tool Vs SOA

    526620
      Hello Experts,

      I would like to seek your valuable advice on following..
      One vendor is proposing a BI/DW solution in which he has replaced the entire ETL tool (ex OWB) by building a solution using SOA technology. Following advantages are proposed for this solution.
      1. It will be llight weight. ETL tool is having big footprint.
      2. Cost effective since no ETL tool is involved.
      3. Efficient data transfer as it will be online. (not based on a schedule with bulk-load like an ETL run during night time.)

      My queries are
      1. Whether anyone has worked/seen such design in BI/DW system.
      2. Will the online extract degrage the perfomance of source system.
      3. Whether such design will consider the use of surrogate keys. Late arriving fact entries. Conform dimensions.
      4. Will it land up in an ODS rather than a data warehouse.

      Please comment.

      ~ Ash
        • 1. Re: ETL Tool Vs SOA
          Cortanamo
          Well, it sounds like a lot of hype and promises with little substance to me. Let's look into it a bit closer:
          1. It will be llight weight. ETL tool is having big footprint.
          Anything browser-based is "light weight" or "client-less". I still have to see an environment where ETL tool footprint is a serious issue. In the end server components have to run on the server, adding a layer in between won't change that.
          2. Cost effective since no ETL tool is involved.
          Basic version of OWB is also "cost effective" as it comes for free with the database.
          If you were to code your loads in PL/SQL and schedule them using DBMS_JOBS that would be "cost effective" in regards of license, but a disaster in regards of maintainability. What does the maintainability of this SOA thing look like? How difficult / easy is it to design and debug jobs in it?
          3. Efficient data transfer as it will be online. (not based on a schedule with bulk-load like an ETL run during night time.)
          I'm afraid this line is a big mixup. Classical jobs (either PLSQL coded manually or generated by a tool like OWB) will be executed at the server. Data won't be transferred to the client and back, that'd be really bad for the performance. Using one or another tool to design PLSQL procedures doesn't influence the way PLSQL executes a tiniest bit.

          As of night time loads vs. some more frequent or near-real-time loads it all comes down to what you design, but you can implement all of those scenarios independently of the tool used to implement the load procedures.

          Overall I'd be very cautious with this vendor. Either you're talking to a salesman full of hot air who's not giving you relevant facts or the entire product is just a disaster waiting to happen to a gullible customer.
          • 2. Re: ETL Tool Vs SOA
            526620
            Hi Cortamano,

            Thanks for your comments.
            They are really valuable.

            ~ Ash
            • 3. Re: ETL Tool Vs SOA
              Nawneet_Aswal
              Yes I fully aggre with Cortanamo Comments.
              and was curious also how the big volumn of data can be managed by SOA.

              Cheers
              Nawneet
              • 4. Re: ETL Tool Vs SOA
                976107
                I experienced some "respectable SOA consultancies" using BPEL to execute ETL/ELT functionality with really disastrous results, only with intention to promote SOA SUITE instead finding the right "person for the job".