Forum Stats

  • 3,769,900 Users
  • 2,253,029 Discussions
  • 7,875,233 Comments

Discussions

Storing scikit model in DB

IronScott-Oracle
IronScott-Oracle Member Posts: 6 Employee
edited Feb 27, 2020 8:42PM in Python

I’m running a scikit model on an Oracle Cloud environment. My final output is a Pipeline object. I need to be able to store this in a DB for a separate Python script to use later. I’ve been able to test the theory using pickle but storing the object on the file system isn’t going to be allowed once the code goes live. I thought a good approach would be to store the object as a blob in an Oracle DB. I can’t seem to find anything that works. The examples I’ve found have cx_Oracle error out with something like Pipeline object not supported.

Is this approach possible? Or some other approach recommended? Or am I just out of luck?

I’d like to be able to do the same thing with dataframe and objects as well, meaning storing objects as objects versus writing data to tables as columns and rows.

Sqlalchemy seems to be the route most people have success with but I don’t have access to it and cannot get it installed.

IronScott-Oracle

Best Answer

  • Anthony Tuininga-Oracle
    Anthony Tuininga-Oracle Member Posts: 37 Employee
    edited Feb 27, 2020 11:56AM Accepted Answer

    You can store the pickle in a BLOB in the database. Something like this should work:

    create table PickledObjects (

       id number(9) not null,

       pickled_obj blob not null

    );

    id_val = 1

    pipeline_obj = MyPipelineObject()

    cursor = connection.cursor()

    cursor.execute("insert into PickledObjects values (:1, :2)", [id_val, pickle.dumps(pipeline_obj)])

    Of course this assumes that you can pickle the object, but there are ways to force that, if needed. Whether this is the best approach or not, I can't say!

    IronScott-Oracle

Answers

  • Anthony Tuininga-Oracle
    Anthony Tuininga-Oracle Member Posts: 37 Employee
    edited Feb 27, 2020 11:56AM Accepted Answer

    You can store the pickle in a BLOB in the database. Something like this should work:

    create table PickledObjects (

       id number(9) not null,

       pickled_obj blob not null

    );

    id_val = 1

    pipeline_obj = MyPipelineObject()

    cursor = connection.cursor()

    cursor.execute("insert into PickledObjects values (:1, :2)", [id_val, pickle.dumps(pipeline_obj)])

    Of course this assumes that you can pickle the object, but there are ways to force that, if needed. Whether this is the best approach or not, I can't say!

    IronScott-Oracle
  • IronScott-Oracle
    IronScott-Oracle Member Posts: 6 Employee
    edited Feb 27, 2020 8:42PM

    Perfect! I got this to work using your guidance. Thank you very, very much.