Re: our new X3 machines (in the data centers, yet to be installed).
I know that some of this was sort of covered in the earlier thread about DBFS. However, I do have slightly different questions...
I understand that the DBFS ASM diskgroup is mandatory and, because of the nature of how its size is calculated, the ACS consultants configure it at its minimum size. However, is it possible to (preferably at configuration time) increase the ASM diskgroup? Or is it meant to have a set size, no matter what?
Oracle have told me that I could put Flashback Logs into the DBFS ASM diskgroup (as long as I don’t configure DBFS to use it, obviously). I’m sure I read that Flashback Logs HAVE to be put into the db_recovery_file_dest (Fast Recovery Area or RECO diskgroup). Has anyone used DBFS (the ASM diskgroup) for such storage? I know it's not IDEAL given the fact it won't be spread across all storage cells, but I'm trying to figure out a way of separating the Flashback Logs from the Archive Logs because we monitor the Archive Log location for space usage (we get an alert if it gets too full and automatically kick off an archivelog backup/purge at a certain level of usage).
The first two Spinning Drives on each cell contain a 29 GB sized partition with the OS. The other 10 contain instead a 29 GB sized GridDisk from which the DBFS_DG diskgroup is built. In other words: The DBFS_DG diskgroup is also spread across all the cells but only on 10 drives (not 12 as DATA and RECO) on each cell.
This way, the GridDisks that make up DATA and RECO can have the same size on all spinning drives. And the DBFS_DG diskgroup will consume about 300GB on each cell for that reason, which can't be changed reasonably, I suppose.
You could set DB_RECOVERY_FILE_DEST='+DBFS_DG' and turn on Flashback Database, so that Flashback Logs are being generated there, while you set LOG_ARCHIVE_DEST_1 to the RECO diskgroup, so that Archivelogs get stored there. And with RMAN backups, you specify FORMAT '+RECO' to get the backups not into the Recovery Area (DBFS_DG in that example).
Ooooh, I do have one more question:
What sort of redundancy applies to the DBFS ASM diskgroup? For instance, our DATA and RECO diskgroups will be NORMAL as we think HIGH is overkill (plus, we need the space!). We also have the 'usable free space' (to withstand a storage cell lost and a disk lost and still maintain redundancy) calculation to figure out too.
It sounds like we can expect 290Gb from each of our three storage cells for a total of 870Gb in total for the diskgroup. I'm guessing we probably still halve that and have to take into account the usable free space stuff. At a guess, I'd be expecting 290Gb of usable space to be available.
Does that sound correct (ballpark figures)?
The DBFS_DG diskgroup is created with normal redundancy by default. It will be the location for the OCR and voting files unless you have a half or full rack with a high redundancy diskgroup. If you have an X3 with high capacity disks, the size of the DBFS_DG griddisks will be ~34.6GB. With the larger (3TB) high capacity disks, Oracle decided to give some more space on the cells to /var/log/oracle - which in turn makes the DBFS_DG griddisks larger as well.