3 Replies Latest reply on Oct 2, 2009 6:41 PM by Greybird-Oracle

    How feasible would it be to DIY BDB JE encyrption

      Hello All,
      I'm aware that BDB JE won't be supporting encryption.

      However, if I wanted to be bold/foolish enough to implement encryption myself for my project, what would the options be? I have encryption code (http://www.jasypt.org/). I have a small BDB JE database of less than a megabyte and plenty of RAM.

      Our client wants to host an application that deals with healthcare records with a 3rd party host and comply with HIPPA and departmental security and encryption policies. A hosting provider is handling the operating system and I cannot, in good faith, promise the client that their hosting provider wouldn't screw up an installation of the BDB native/C database.

      I see 2 options for this:
      1. Encrypting the payload and leaving the PK indexes unencrypted. This complies with regulations, but removes the query benefits of using BDB (we wouldn't be able to index confidential fields). This also makes the people we answer to nervous. I'd rather not do it this way.
      2. Doing all database operations in memory and manually saving, in encrypted form, to disk periodically as well as on shutdown. The app would decrypt from file on startup. I'd be interested in pursuing this if it is the best option.

      So I'll ask:
      1. Is there a strategy I didn't think of that would encrypt the database more reliably? Is there an API in the DB that I didn't think of that I could easily ensure encryption?
      2. Can the DB be run from memory? I assume it'd perform quite well. Would it's memory usage be reasonable? (I have < 1m of data and .75 GB of RAM for a small JEE app.
      3. If the DB can be run from memory, is there any reason why that would be a terrible idea?...beyond the 2 obvious concerns of the app shutting down without writing to the disk and storing more data than I have RAM to allocate (I have have a half gig of RAM, after app startup, to store 500k or so of data).

      Any strategic guidance would be greatly appreciated. I can implement the app in BDB 3.x or 4.x beta.


      PS: I realize that in the grand scheme, my strategy is flawed from the beginning....dealing with super-secret data on a 3rd party host, who I assume is barely competent, with a very low budget, but in this economy, we're happy to be employed. :) If this is just too much of a square peg in a round hole problem, I'll just use serialization and the collections API for storage and encrypt manually in the same way described above.

      An embedded, encrypted BDB JE install would be a great problem to solve as I like working with BDB JE much more than the BDB C version or a JPA + RDBMS solution, but am working with patient data for my next few projects.

      Edited by: JavaGeek_Boston on Oct 2, 2009 1:59 PM
        • 1. Re: How feasible would it be to DIY BDB JE encyrption
          Hi Steven,

          I think I need to understand a little more.

          There's something I'm missing about what you're thinking when you say: "I'll just use serialization and the collections API for storage and encrypt manually in the same way." Are you saying that you could implement payload encryption more easily if you don't use the DPL? How does this solve the problem that you can't index confidential keys?

          As far as running in memory and flushing to disk periodically, if this is possible for your app (you have enough memory and this form of durability is acceptable) then I have to ask why you need a database at all. In general, nobody is (or should be) using JE when the app can run under these conditions.

          • 2. Re: How feasible would it be to DIY BDB JE encyrption
            Hello Mark,
            The short answer as to why I'd use JE/DPL is querying and future expansion.

            If I didn't use the DPL, I'd encrypt the entire database, not just the payload. I could define an object with a few Maps as member variables and manually serialize and encrypt the entire object. That would satisfy the security needs of the client. It would also scale to the needs they've specified, but there's obvious limits to which that solution would scale.

            I wouldn't say I "need" a database yet, but if it didn't require too much effort, I'd prefer to use a DB so that it can be expanded and easily extended. They hope their programs become more successful and accommodate a greater number of users in the future over many years and we hope to reuse the application logic code for other projects where the native BDB version is an option. I'd encapsulate the logic so the hacked encrypted version could be replaced with JE or BDB native version with replacing a single class.

            Ultimately, the decision on architecture will be decided by either the client or someone above me. Half of this post was the hypothetical "what if" scenario. "If" I wanted to have a "coding adventure" and encrypt a BDB JE database in a secure manner, could it be done, how much effort would it require, and what would the potential negative consequences be? Are there means of encrypting it via any built-in APIs, perhaps on the serialization layer?


            PS - Deep down, I suspect the answer is "no...doing this with JE is a bad idea" but I wanted to be sure before I wrote my own in-memory pseudo-DB to accommodate the high-security cloud-computing requirement.
            • 3. Re: How feasible would it be to DIY BDB JE encyrption
              My confusion about "I'll just use serialization and the collections API for storage and encrypt manually in the same way" is that I thought you meant the JE collections API. You meant the built-in Java collections stuff in java.util.

              I don't think I have anything to add about the pure in-memory transitional approach you described.

              No, there are no built-in APIs that would work to implement full encryption in JE. Either you can encrypt each data record (payload you're calling it), or all keys and data record, individually. If you encrypt keys, then of course you lose sorting. If you decide you want to do this, I can make suggestions about how to do it with bindings.

              For DIY JE encryption, you would have to change the JE implementation. I suggest an off-line email discussion with Charles and myself if you want to explore that option.