This is curious, since a simple program similar to what is described is known to work. I've modified the standard sample program examples/cxx/EnvExample.cpp C++ to use an in-memory database, DB_SYSTEM_MEM, and DB_TXN_NOSYNC. The "Device busy" symptom occurs if the close of the environment handle is bypassed. I have not been able to reproduce the DB_RUNRECOVERY error.
How does the program's use of Berkeley DB different from what is provided in EnvExample.cpp?
Is it possible to send me the relevant portions of it?
Oracle Berkeley DB