This content has been marked as final. Show 4 replies
1. in place of dbfs filesystem
The space from the DBFS filesystem is not spread across all celldisks (due the need for operating system space) so isn't going to be evenly balanced; this is the reason why it is in a separate DBFS diskgroup rather than as part of your DATA diskgroup. I wouldn't recommend using it as such.
2. move RECO to use ZFS
Remember, RECO stores your archivelogs, so it's critical for recoverability. In a redo-intensive database it would have high I/O volumes too.
3. Move our DEV dbs to use ZFS instaed of HPC disks on exadata. What happens if we take backups of our production database and restore to DEV for development purposes?
I like this option better than the other two you've listed. You can definitely clone your production database to a ZFS Appliance-backed database. You won't be able to Exadata-specific features like smart scans though.
4. any other ways ZFS storage can help us save on being a new exadata appliance.
Look at why storage usage is increasing. Can you compress existing data (using HCC for example)? Perhaps create a separate archival database for older data?
If using the ZFS appliance for Exadata storage, make sure it's connected via InfiniBand of course.
We are having space issues
Oracle 22.214.171.124 on Exadata x2-2 on bp11.
All of this is prefaced with the that these are general recommendations and may not apply to your actual environment. Also, if connecting a ZFS storage appliance to Exadata, Infiniband connectivity is the only way you should do it.
I would recommend a ZFS storage appliance in place of using DBFS in most cases, just because DBFS seems to be cumbersome and it performs slow depending on what you're using it for. Even if you removed everything from the DBFS_DG diskgroup, Oracle does not recommend putting those griddisks into the DATA or RECO diskgroups, because the diskgroups would then become unbalanced. You could use DBFS_DG for things other than a DBFS filesystem, but remember that it's on the slowest area of the disks.
Would you place all of your RECO area on the ZFS, or just use it as a place for backups? I'd most likely suggest having a small FRA on the Exadata and backing up to the ZFS appliance, but it depends on how tight you are on space and what your workload looks like.
As for moving the development databases off of the Exadata storage and on to the ZFS, I don't really see the point in having them on the compute nodes at all, except for licensing reasons. It's going to be taking CPU and memory cycles away from databases that are open to utilizing Exadata storage. Keep in mind that the ZFS storage appliance will not present disks to be used within ASM, and aren't going to act like a database running on Exadata (with the exception of HCC). As long as you know that going in, you can certainly take databases running on the Exadata and restore them to the ZFS storage appliance.
As Marc said above, look into your storage needs and see what's using so much space. If it's historical data that doesn't get updated, it's a candidate for HCC.
We are going through a similar exercise to move backups to external storage (absolutely IB connected) but may not necessarily end up on ZFS. Exadata is an extremely expensive place to store database backups, and especially so if RECO is 60% of your DBM storage and even more so if you employ ASM High Redundancy. I agree with Marc in having a small FRA on the DBM and backing up to an external, dedicated, IB-connected storage unit.
DBFS adds complexity to a solution on Exadata and the default DBFS DG offers no flexibility in sizing. You are stuck with it, so leave it and use as you wish. If you want more space, you have to create a second DBFS DG. I do not see any problem with having this second DBFS DG on the same external storage.
866614 wrote:If you're going to use external storage, I wouldn't want to put it inside a DBFS. From my experience adding the extra layer of DBFS doesn't provide the much performance benefit. For external storage, directNFS is the way to go.
I do not see any problem with having this second DBFS DG on the same external storage.