Ability to integrate ORAAH Spark Session with Zeppelin and SparkR Spark Sessions
Description
Currently ORAAH Spark Sessions are exclusive and executed separately than other tools. An integration with other Spark Sessions could mean increased compatibility with Zeppelin and SparkRUse Case and Business Need
Currently ORAAH Spark Sessions are created for exclusive use of ORAAH.
In some cases IT departments prefer this, since they can see in isolation the usage of the Compute used by the Machine Learning algorithms from ORAAH as opposed to a mix of workloads from Zeppelin for example.
On the other hand, when running tools like Zeppelin Notebooks, where there is already a Spark Session running, ORAAH still requires its own additional Session and cannot interact transparently with other languages like Python or SparkSQL on the other paragraphs of the same Notebook directly. This also means that some limitations exist for using SparkR with ORAAH in combination.
Tagged:
12