Discussions
Categories
- 196.8K All Categories
- 2.2K Data
- 238 Big Data Appliance
- 1.9K Data Science
- 450.3K Databases
- 221.7K General Database Discussions
- 3.8K Java and JavaScript in the Database
- 31 Multilingual Engine
- 550 MySQL Community Space
- 478 NoSQL Database
- 7.9K Oracle Database Express Edition (XE)
- 3K ORDS, SODA & JSON in the Database
- 544 SQLcl
- 4K SQL Developer Data Modeler
- 187K SQL & PL/SQL
- 21.3K SQL Developer
- 295.8K Development
- 17 Developer Projects
- 138 Programming Languages
- 292.5K Development Tools
- 107 DevOps
- 3.1K QA/Testing
- 646K Java
- 28 Java Learning Subscription
- 37K Database Connectivity
- 155 Java Community Process
- 105 Java 25
- 22.1K Java APIs
- 138.1K Java Development Tools
- 165.3K Java EE (Java Enterprise Edition)
- 18 Java Essentials
- 160 Java 8 Questions
- 86K Java Programming
- 80 Java Puzzle Ball
- 65.1K New To Java
- 1.7K Training / Learning / Certification
- 13.8K Java HotSpot Virtual Machine
- 94.3K Java SE
- 13.8K Java Security
- 204 Java User Groups
- 24 JavaScript - Nashorn
- Programs
- 437 LiveLabs
- 38 Workshops
- 10.2K Software
- 6.7K Berkeley DB Family
- 3.5K JHeadstart
- 5.7K Other Languages
- 2.3K Chinese
- 171 Deutsche Oracle Community
- 1.1K Español
- 1.9K Japanese
- 232 Portuguese
Joining columns in a DataFrame with an Oracle table

Hello,
Here's our situation:
We have to replace existing pl/sql code which contains dblinks, since the enterprise standards have ruled dblinks to be insecure. Right now we are exploring various options. One option is to use Python but before we do that, need to figure out if it will be able to satisfy one of our use cases.
The use case is as follows:
1) Retrieve data from a driving table into a dataframe based on certain date criteria.
2) Parse pipe delimited data in one of the columns in the dataframe and create new derived columns (in memory)
3) Use the derived columns data in sql query to fetch additional data elements from another table in the source database
4) Finally once all the required data elements are populated in dataframe, update a target table (in a different database than source)
Some things to note:
1) We cannot create a temp table or any pl/sql procedures in source database. It is read only
2) We are trying to avoid looping through every record in dataframe and fetch from another source table, as performance may not be acceptable
3) Most Python examples I have seen online, suggest we get the data from both tables into separate dataframes and then merge (join?) this data in memory. However this may not work well, if the lookup tables (second table in use case above) is huge and has millions of records whereas we may need to retrieve only a few thousand based on criteria from first dataset.
Any thoughts?
Would appreciate suggestions from experts here, on how to implement this kind of logic in Python. Thank you.