10 Replies Latest reply: Oct 18, 2011 12:03 PM by 895080 RSS

    Database storage vs. File system

    686410
      Hi,

      We'll be setting up a new Oracle UCM 10g* environment in a production scenario - we have access to high-speed SAN for file storage, but we can also use an Oracle 11g database for file storage (using the File Store Provider) - the database's tablespace files will anyway be stored on the SAN. We're not quite sure which way to go - assuming we have complete ha/dr for both SAN and database solutions, which way is the best?

      I'm also aware that if we don't select the webless option we'll anyway need a similar amount of storage on the file-system as the weblayout will contain a copy (or a web-rendition) of the checked in file - we could create a webless ucm, with inbound refinery only creating files in weblayout.

      Any input on the matter would be appreciated.

      *11g not chosen due to WebLogic licencing requirements for our client - anyone know if 11g will ever be released standalone?

      EDIT - re: 11g licensing - sorry about that i misunderstood the licensing issue of our client.

      Edited by: Etienne Azzopardi on 04-Feb-2011 13:44
        • 1. Re: Database storage vs. File system
          Jiri.Machotka-Oracle
          Sorry, but I will start my reaction from the last point:
          *11g not chosen due to WebLogic licencing requirements for our client - anyone know if 11g will ever be released standalone?
          What do you mean? UCM license contains a restricted license of Weblogic EE for the purpose of hosting UCM. In other words, WLS is there for free (only it cannot host anything else). 11g will never be standalone - it will only support other application servers, namely, Websphere and JBoss (it follows the logic for other non-Oracle middleware products).

          File system vs. database - there's been a couple of posts on this topic.

          In a nutshell:
          - with regard to the speed: unless you have really a lot of items (millions a month at least, maybe even millions a day), file system will be slightly faster or equal
          - in Oracle DB 11.2 or higher you can use SecureFiles, which can effectively work with unstructured content. It can squeeze the disk space needed for storage, speed up operations (checkins, but also e.g. backups)
          - with DB you will effectively have one system to manage - well, UCM still requires file system (shared storage for clusters), because some operations like conversions cannot run on the content in the database
          - DB gives you further options like partitioning, hierarchical storage management, advanced security, etc. Your SAN storage can have some of these too.
          • 2. Re: Database storage vs. File system
            580267
            Actually the database option is faster than filesystem always. Oracle has benchmarking data to back that up. Secure files in 11g db removes an inefficient layer of OS.
            • 3. Re: Database storage vs. File system
              746458
              It is not really true to suggest that one is always faster or always better.

              The real answer is it depends.

              Even if using SecureFiles you should consider the following general recommendations

              1)Storing Content in the DB as well as metadata and search index definitely simplifies the process of backup and recovery.
              2)Use of SecureFiles in 11g also allows (transparent to UCM) use of Deduplication, Compression, encryption and partitioning which can all help reduce the costs of storage

              As far as DB vs Filesystem...
              If you are storing losts of small images that tend to be accessed frequently (e.g. website assets, images, css, js) then file system really is the best place to serve these from in terms of performance -at least in all the data I have seen
              If your files are larger > 1MB and accessed less often (e.g. DocMan) then I think DB storage makes a lot of sense as the performance figures tend to get swapped.

              The FileStoreProvider can let you have the best of both worlds by allowing the definition of rules which would store the valut copy on the DB and a weblayout copy on the filesystem.

              Tim
              • 4. Re: Database storage vs. File system
                billycripe-BloomThink
                Just blogged about this here:
                http://cfour.fishbowlsolutions.com/2011/01/28/ucm-and-massive-storage/

                has links to the Oracle benchmarking reports and file store provider (FSP) how-to instructions too.

                Overall, usage patterns should govern your storage architecture. Think through both contribution/ingestion patterns as well as consumption patterns. Then use criteria driven FSP to direct content to the appropriate storage layer.
                Remember too that you can use a blended model where the DB holds the authority copy and the FS is used as a quick-access cache.

                Hope this helps.
                Warmly,
                Billy Cripe
                Fishbowl Solutions
                • 5. Re: Database storage vs. File system
                  751247
                  W/r to performance - we're running the latest version of UCM 11g against an 11g database.
                  We have set up some file storage rules to test performance of large files (2MB - 20MB) going into filesystem vs database storage.

                  Bottomline is we are seeing very* slow performance and timeout errors on anything above 2MB.
                  • 6. Re: Database storage vs. File system
                    746458
                    Oracogito,

                    thanks for your input but without knowing ANY details of your system this information is largely irrelevant.

                    There could be bottlenecks or problems just about anywhere - I have seen nothing to suggest that this will be an inherent UCM, DB or JDBC file storage issue. It is probably a local environment issue that you have.

                    I have deployed a large 11g instance on 11g DB w SecureFiles and have had no performance issues with files going up to ~ 1GB
                    • 7. Re: Database storage vs. File system
                      686410
                      I'm taking it you mean you're seeing slow performanace on the db ... better on the fs!?
                      • 8. Re: Database storage vs. File system
                        751247
                        Windows Server 2008 v6.0 (build 6002) SP2 64-bit
                        Oracle UCM 11gR1-11.1.1.4.0
                        Database Version:11.1.0.7.0 ---Oracle Database 11g Enterprise Edition Release --- - 64bit Production With the Partitioning.

                        We will continue to look at the environment.
                        The problem has been logged with Oracle but if you say you have deployed 1GB securefiles successfully using db storage than we must be
                        missing something not too obvious in our configuration.
                        • 9. Re: Database storage vs. File system
                          user10668485
                          I was wondering if you were able to fix your performance issue with large (2MB+) files being saved to the database? We are having the same issue with slow performance and timeouts with large files. Thank you.
                          • 10. Re: Database storage vs. File system
                            895080
                            that was always my understanding as well, good to see it backed up by someone else

                            Edited by: 892077 on Oct 18, 2011 10:03 AM