4 Replies Latest reply on Jun 21, 2014 3:40 PM by mizzle

    Sparse and Dense dimension

    mizzle

      Dear all,

       

      We have a plan type including Accounts and Products as following,

       

      Header 1Header 2Header 3Header 4
      Products\AccountRateQuantityAmount
      P1
        P11
        P12
        P13
      P2
        P21
        P22
        P23

       

      There are about 30 members in Account dimension , while 500 members in Product dimension

      In Account dimension, most calculation like Amount=Rate*Quantity

      In Product dimension, most calculation are aggregation

       

      Usually account is dense, but this case we are not sure.

      I think make Product dense will reduce blocks, but  in which way performance is higher and calculate faster?

      Thanks.

       

      Best Regards

      Mizzle

        • 1. Re: Sparse and Dense dimension
          KKT

          Thats fine the number of member in your applications, with this you should not think much as Account should always be dense and your case Product should be Sparse.

           

          There are obvious reasons for the account structure to be dense -- most analytical applications run various formulas and such against different measures, so it only goes that in those cases, a dense dimension is better. There are execptions, of course (in some cases, you could say that the true 'measures' dimension isn't the account structure, but something else).


          Changing a dimension from sparse to dense will increase the block size by the number of stored members in the time dimension times the current block size. The size change impact could be minor or significant depending on the existing block size and number of blocks.

          This will also change the calculation order, which could affect calculated values.

           

          Let us know if you have more questions.

           

          Thanks,

          ~KKT~

          1 person found this helpful
          • 2. Re: Sparse and Dense dimension
            mizzle

            Thanks KKT !

             

            We have 2 Plan Types. In Plan 1 , Account is Dense, which are 800 members.

             

            Here we discuss Plan 2, as we mentioned , only 30 members in Account.

             

            My concern is, If we set Account dense in Plan 2, the block is too small, while the index will be too big.

             

            I prefer to set Product dense in Plan 2, that could reduce block numbers and index, and made the block size between 80k ~100k.  But I'm not so sure if it is the optimized way.

             

            Thanks and regards

            Mizzle

            • 3. Re: Sparse and Dense dimension
              Celvin Kattookaran

              What you are do is

              Make one dimension dense, load your data and check the block density - this will tell you which dimension is really dense.

               

              Regards

               

              Celvin

              • 4. Re: Sparse and Dense dimension
                mizzle

                Thanks Celvin , I think you are right.

                 

                Regards

                Mizzle