Today, AI Data Platform (AIDP) supports mounting external catalogs such as Oracle, Snowflake, and MySQL as governable Master Catalog assets. This idea proposes extending that capability to Unity Catalog and Apache Iceberg catalogs.
As lakehouse architectures continue to become prevalent for enterprise analytics and AI workloads, Unity and Iceberg have emerged as two of the most widely adopted lakehouse oriented catalog standards across the industry. Supporting these catalogs as first-class AIDP Master Catalog assets would help customers bring externally managed data into AIDP without sacrificing discoverability, governance, or operational simplicity.
Why this matters
1. Align with industry-standard lakehouse architectures
Unity Catalog and Apache Iceberg are heavily used across modern data platforms, including Databricks, Snowflake, and other lakehouse ecosystems. Their open architecture enables interoperability across engines, clouds, and tools.
2. Improve discoverability and governance for existing data assets
AIDP's Spark engine can already query the underlying Delta and Iceberg storage formats associated with these catalogs. However, users must manually configure access and manage metadata outside of AIDP. Native catalog integration would make these assets easier to discover, govern, secure, and operationalize within the platform.
3. Reduce onboarding and operational overhead of multi-source lakehouse storage
Many customers already maintain centralized metadata, permissions, and data organization through Unity or Iceberg catalogs. Allowing those catalogs to be mounted directly into AIDP would eliminate duplicate catalog management efforts and accelerate time to value for new projects.
Reference:
See AIDP current state catalog mounting functionality: External catalogs in AI Data Platform: Going beyond the Oracle ecosystem
See Unity catalog mounting in ADW: Mount Databricks Unity Catalog Once and Query Many Tables from Oracle Autonomous AI Database | autonomous-ai-database
Expected outcome
Enable Databricks Unity Catalog and Apache Iceberg catalogs to be mounted and managed as AIDP Master Catalog assets, providing native governance, discovery, and access management capabilities while leveraging the catalog metadata customers already maintain elsewhere.
