Support Delta Share Consumption Configurable as a Managed Catalog Schema

DMD

Currently the primary way to consume a Delta Share in AIDP is to install the Delta Sharing python package and work with the Python delta share API in python based notebooks.

A more friendly and managed user experience would be to have the delta-share profile configured as a schema in the AIDP managed catalog. This would involve uploaded the Delta Share profile you recieved from another application to the Create Catalog form, which would create a catalog for this share schema.

Nice to haves as a part of this managed catalog experience:

(1) Implement a way to have filter predicates be handled through SQL conditions (i.e. Where COL1 = 'C', passed to a JSON predicate in the backend vs. manual JSON predicate handling). Reference: delta-sharing/PROTOCOL.md at main · delta-io/delta-sharing · GitHub

(2) Post-receipt of data, apply additional filtering client side to remove variability related to the "best effort" approach to data querying (i.e. after initial "best effort" data filtering is applied, apply a filter within AIDP client to return the actual intended filtering of data, all handeled in the catalog API backemd without user intervention).