0 Replies Latest reply: Jan 27, 2013 6:43 PM by 987412 RSS

    Hybrid Data Warehouse: Oracle + MPP

    987412
      Hybrid Data Warehouse: Oracle + MPP

      We had Oracle for our Enterprise Data Warehouse for a while. But as data grows the Analytic applications and Aggregation related parts became less and less efficient. I was watching how Oracle was greedy to allocate enough resources for big sorts, joins, aggregations probably keeping in mind multiuser OLTP nature of this RDBMS.
      We tuned all that was possible and still had performance issues. Then we started to think about different platform. Unfortunately Exadata was out of consideration because of very high cost and lack of enthusiasm from Oracle sales (probably they had a long list of more attractive clients). We did POS testing for Greenplum and saw a huge difference in performance in those areas where Oracle was not great. POS time was limited we can not test everythin. So, we moved the whole data warehouse to Greenplum.
      Now we see that not everything is so great. What was black is white now, but what was white is black. Breefly:
      Greenplum is fast for big loads and huge queries like analytic ones.
      But it is pretty raw product: weak concurrency, weak backup / recovery, small queries are SLOW, and it is not very reliable. IMHO.
      I'm not sure that other MPP type product would be better solution for the whole system. Grennplum, Vertica, Netezza, etc. can beat Oracle in some dimensions but as a whole system would be weaker solutions. They can be a good ADDITIONS to existing system. Current data warehouses are not just load ones a day data and reporting systems. They are like 7x24 databases where users need to see what is happening right now and can drill down to small data set. And MPP is clumsy here.
      IMHO I see a hybrid system as a solution. OLTP + MPP or consolidated platform like Exadata.

      My question: who can share an experience of coexistence in the Data Warehouse of MPP system (Teradata, Greenplum, Vertica, Netezza, etc.) and OLTP system (Oracle, SQL Server, etc.).
      Pros. and Cons.

      This is a big topic but I think very interesting one.