Skip to Main Content

NoSQL Database

Announcement

For appeals, questions and feedback about Oracle Forums, please email oracle-forums-moderators_us@oracle.com. Technical questions should be asked in the appropriate category. Thank you!

Unable to add security to an existing Oracle NoSQL Database CE installation

User_GX5XXApr 6 2022 — edited Apr 7 2022

I deployed Oracle NoSQL Database CE on Minikube using this image: oracle/nosql:19.5-ce [https://github.com/oracle/docker-images/tree/main/NoSQL]
I followed this doc to make the db secure: https://docs.oracle.com/en/database/other-databases/nosql-database/19.5/security/single-node-secure-deployment.html#GUID-03A2067A-64CD-4CF4-99DC-D430DA8FD290
In the 3rd step, I used -pwdfile option since my db is CE version.
However, in the 7th step of this doc, I could not log in admin as anonymous as it should be. Instead, I was asked to provide a username and a password as you can see in the picture below:
Oracle_NoSQL_CE_Security_Problem.pngMy Oracle NoSQL Database deployment and service yaml:
oracleNosql-deployment.yaml (0 Bytes)
So, what is that I am missing here?

Comments

Tim Blackman-Oracle

First, in step 3, did you specify '-pwdmgr pwdfile' to use a password file? Note that the Oracle Wallet is not supported in EE.
In step 7, are you calling runadmin while running on the same machine as the SNA? The anonymous login is only permitted when logging in from the local machine.

User_GX5XX

Thanks for the quick reply.
Yes, I did specify pwdfile option as I said in my post above.
Yes, you can see in the picture that I'm entering the pod with "kubectl exec" and running the runadmin command inside the pod.

Tim Blackman-Oracle

Hmmm. Maybe there is something funny about the networking that is preventing the server from realizing that the call is coming from the local machine. Maybe you could try experimenting with this first in a simpler deployment environment to see if that might be the issue? If it is, we'll need to track it down, but it might be helpful to rule out other issues first.

User_GX5XX

What do you suggest as a simpler deployment environment?

Tim Blackman-Oracle

Oh, sorry, maybe I missed what you are trying to do. I guess your use case is specifically with our docker image, not just with the 19.5 release. I'll check with others about this.

User_GX5XX

Exactly, my use case is specifically with this docker image.
And I think it is important to point out that in the Dockerfile that I'm using to build my existing image, the version is 19.5.19 and not 19.5.25 which is the current one on Github (You can see the diff here: https://github.com/oracle/docker-images/commit/179caac4b43a484559334d38cb06e698aa1bb4b5#diff-ffc3de684b4fa40e629e7977082eeb3bae8300053414c948ebb38d829f450372 ).
Even this may cause the problem itself but I'm not sure.

Tim Blackman-Oracle

So the issue here is that the docker image has a non-secure kvlite configured in it by default. The directions in the documentation don't work for upgrading a kvlite from non-secure to secure mode. Instead, you should modify the Docker file to create a secure kvlite. We'll update the directions about this.

User_GX5XX

Exactly. That's the issue here.
Just to sum up:
I got stuck in the 7th step of the doc that I mentioned in my post because it does not login admin as anonymous right after I start the runadmin command (instead it asks me to provide a username and a password), and I need to be able to achieve to login admin as anonymous and go further in the doc to make my non-secure kvlite secure.
So, as I understand, I should wait for the update, right?

davega-Oracle

Those images are based on KVLite feature. KVLite is a simplified version of the Oracle NoSQL Database. It provides a single storage node, single shard store, that is not replicated. It runs in a single process without requiring any administrative interface.
https://docs.oracle.com/en/database/other-databases/nosql-database/19.5/kvlite/index.html
KVLite is intended for use by application developers who need to develop and unit test their Oracle NoSQL Database applications. It can be used as a development platform for developers to get familiar with Oracle NoSQL APIs, and test different ways of interacting with these APIs. KVLite runs on a single machine. It is not intended for production deployment, or for performance measurements.
The documentation that You mentioned does not apply here. We are using a specific version called KVLITE.
That said, You need to change the last line in the Dockerfile from disable to enable and you will have a secure configuration.
CMD ["java", "-jar", "lib/kvstore.jar", "kvlite", "-secure-config", "enable", "-root", "/kvroot"]
then

docker build -t oracle/nosql .
docker run -d --name=kvlite --hostname=kvlite  oracle/nosql

Then you can test using docker exec -ti kvlite

docker exec -ti kvlite /bin/bash
docker exec -ti kvlite java -Xmx64m -Xms64m -jar lib/kvstore.jar version
docker exec -ti kvlite java -Xmx64m -Xms64m -jar lib/kvstore.jar ping -host localhost -port 5000 -security /kvroot/security/user.security
docker exec -ti kvlite java -jar lib/kvstore.jar runadmin -host localhost -port 5000 -store kvstore -security /kvroot/security/user.security
docker exec -ti kvlite java -jar lib/sql.jar -helper-hosts localhost:5000 -store kvstore -security /kvroot/security/user.security

The real challenge here is to have access from another container because you need to copy the /kvroot/security/ from the kvlite container to the new one. Otherwise, you will have the following error

$ docker run --rm -ti --link kvlite:store oracle/nosql   java -jar lib/sql.jar -helper-hosts store:5000 -store kvstore -security /kvroot/security/user.security
Picked up _JAVA_OPTIONS: -Djava.security.egd=file:/dev/./urandom
Exception in thread "main" java.lang.IllegalStateException: /kvroot/security/user.security (No such file or directory)
        at oracle.kv.impl.security.util.KVStoreLogin.createSecurityProperties(KVStoreLogin.java:382)
        at oracle.kv.impl.security.util.KVStoreLogin.loadSecurityProperties(KVStoreLogin.java:164)
        at oracle.kv.impl.security.util.KVStoreLogin.updateLoginInfo(KVStoreLogin.java:139)
        at oracle.kv.util.shell.CommonShell$LoginHelper.updateStoreLogin(CommonShell.java:704)
        at oracle.kv.util.shell.CommonShell.connectStore(CommonShell.java:215)
        at oracle.kv.util.shell.CommonShell.connectStore(CommonShell.java:208)
        at oracle.kv.impl.query.shell.OnqlShell.init(OnqlShell.java:124)
        at oracle.kv.util.shell.CommonShell.start(CommonShell.java:1372)
        at oracle.kv.impl.query.shell.OnqlShell.main(OnqlShell.java:861)

And here, there are multiple options, but we need to understand the use case to provide accurate guidance.
You are talking about a simpler deployment environment. Could you help me understand why you need to activate the security for a development env? Why are you deploying using oracle/nosql:19.5-ce ? and most important, Which driver or SDK will you use for your developments?

User_GX5XX

Thank you for the explanation.
I'm developing an Oracle NoSQL scaler for KEDA (Kubernetes Based Event-Driven Autoscaling)[https://github.com/kedacore/keda#readme] and you can see my issue here: https://github.com/kedacore/keda/issues/2289
So, what this scaler basically does is that it connects to the Oracle NoSQL db and listens to it, and does some scaling if necessary. While connecting, it has to have an authentication mechanism like providing a username and a password. That's why I need to make the db secure and then see if my scaler works correctly.
I'm using this Go SDK (https://github.com/oracle/nosql-go-sdk) to connect to on-prem and cloud. For now, I only have Oracle NoSQL on Minikube which is on-prem, not cloud yet. So, the thing is, I need to have a secure db to be able to test my scaler and I cannot produce that environment right now.
Regarding why I'm using the oracle/nosql:19.5-ce image, it was the one I found at that time while I was searching for a Docker image to deploy the db on Minikube.
I hope this makes what I'm trying to do more clear.

davega-Oracle

You need the Oracle NoSQL Database Proxy to use the Go SDK drivers as explained here: https://github.com/oracle/docker-images/tree/main/NoSQL#oracle-nosql-database-proxy. The image that you are using (19.5) does not include this component.
The latest version of the image https://github.com/oracle/docker-images/tree/main/NoSQL/ce includes this component. But I am not currently supporting the secure option. I can work on it and provide you with all the instructions.
Otherwise, It is not clear to me what you are trying to do. If you want, we can try to work together to see a good solution for you. Maybe you can join our slack workspace http://bit.ly/odevrel-slack, and we can discuss it.
In this slack workspace, you can find a slack channel dedicated called oracle-nosql, or you can send me a direct message. My name is Dario VEGA.

User_GX5XX

I know that I need to run HTTP Proxy to use this SDK, but there is something strange now. I saw that in the commit message of the latest version of the image it says: "In this new version, we are also adding HTTP PROXY support", however, I was able to run HTTP Proxy and create a table with my Golang code with my image (19.5), too.
You can see that I'm running the proxy in the bottom right and see the audienceData table was created in the bottom left. Also see the port of proxy (80) was open in the top right:
OracleNoSQL.pngIf proxy was not supported in 19.5, then how I managed to connect to that and run my code succesfully? That's not the main focus here but I got curious now.
Of course, I would love to discuss with you on Slack and find my way around. I'll contact you.

davega-Oracle

The HTTP Proxy is supported and included in CE 19.5 binaries, but it does not start automatically in the container image. Sorry for the misunderstanding.
When I say supporting the secure option means starting automatically when the container starts. The idea is to have a container image that you run and work without needing manual configuration. It is why I am trying to understand what you are doing to see what you expect.
In the new version, you don't need to start the HTTP proxy manually by using: k exec -it (bottom-right window). When you start the container, everything starts automatically. See file start-kvlite.sh .

#!/bin/bash
#
# Copyright (c) 2022 Oracle and/or its affiliates.
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/

set -e

java -jar lib/kvstore.jar kvlite -secure-config disable -root /kvroot -host "$HOSTNAME" -port "$KV_PORT" -admin-web-port "$KV_ADMIN_PORT" -harange "${KV_HARANGE/\-/\,}" -servicerange "${KV_SERVICERANGE/\-/\,}" &
while java -jar lib/kvstore.jar ping -host $HOSTNAME -port $KV_PORT  >/dev/null 2>&1 ; [ $? -ne 0 ];do
    echo "Waiting for kvstore to start..."
    sleep 1
done
java -jar lib/httpproxy.jar -helperHosts "$HOSTNAME:$KV_PORT" -storeName kvstore -httpPort "$KV_PROXY_PORT" -verbose true
User_GX5XX

Now I got the point. It's about automating the processes.
So, I can use the latest version of the image and change the -secure-config option to enable, and then run the file start-kvlite.sh inside the pod. Finally, I'll have a secure db without having to do any additional configuration, right?

davega-Oracle

-secure-config option to enable secure allows having a KVlite secure configuration. Now we need to secure the HTTP proxy configuration as described in this documentation (at the end of the chapter you can find an example). In step 7, keep attention to the Note: Provide the hostname of the machine for CN if you are not running in localhost. <= This is our case
In the current image, there is no the Linux package OpenSSL (Step 7), So 2 options:
Create the certificate in your machine and then use the command docker cp or kubectl cp
Modify the container image to install this Linux package
Step 11 concerns the Java SDK, so you need to use the instructions for the go driver
In the current state, you cannot change -secure-config option to enable secure in the latest version and run start-kvlite.sh because it is running the HTTP proxy in a non secure configuration
Action plan.
You can use the current 19.5, change -secure-config option to enable and follow the instructions in this documentation https://docs.oracle.com/en/database/other-databases/nosql-database/19.5/admin/secure-proxy.html
Wait for me, this is what I called: "I am not currently supporting the secure option, I can work on it and provide you with all the instructions."

davega-Oracle

When running using 19.5, you will hit in this error https://community.oracle.com/tech/developers/discussion/4496143/no-http-proxyrequesthandler-class-in-httpproxy-jar
So, please use the following Dockerfile

# Copyright (c) 2020, 2022 Oracle and/or its affiliates.
# Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl/
#
FROM openjdk:14-oraclelinux7

LABEL org.opencontainers.image.source = "https://github.com/oracle/docker-images"

ENV VERSION="20.3.19" \
    KVHOME=/kv-20.3.19 \
    PACKAGE="kv-ce" \
    EXTENSION="zip" \
    BASE_URL="http://download.oracle.com/otn-pub/otn_software/nosql-database/" \
    _JAVA_OPTIONS="-Djava.security.egd=file:/dev/./urandom"

RUN yum -y install unzip openssl && \
    curl -OLs "${BASE_URL}/${PACKAGE}-${VERSION}.${EXTENSION}" && \
    unzip "${PACKAGE}-${VERSION}.${EXTENSION}" && \
    rm "${PACKAGE}-${VERSION}.${EXTENSION}" && \
    yum -y remove unzip && rm -rf /var/cache/yum/*

VOLUME ["/kvroot"]

WORKDIR "$KVHOME"

EXPOSE 5000 5001 5010-5020

CMD ["java", "-jar", "lib/kvstore.jar", "kvlite", "-secure-config", "enable", "-root", "/kvroot"]

Then build the image, start the container and run the following commands to secure the HTTP proxy

$ docker build -t oracle/nosql .
$ docker run -d --name=kvlite --hostname=kvlite -p 8080:8080 oracle/nosql
$ docker exec -it kvlite /bin/bash

Inside the container

bash-4.2# java -jar lib/sql.jar -helper-hosts localhost:5000 -store kvstore -security /kvroot/security/user.security

CREATE USER proxy_user IDENTIFIED BY "ProxyPass@@123";
exit

bash-4.2# mkdir -p /kvroot/proxy/
bash-4.2# java -jar lib/kvstore.jar securityconfig pwdfile create -file /kvroot/proxy/proxy.passwd
Created

bash-4.2# java -jar lib/kvstore.jar securityconfig pwdfile secret -file /kvroot/proxy/proxy.passwd -set -alias proxy_user -secret "ProxyPass@@123"
Secret created

bash-4.2# cp /kvroot/security/client.trust /kvroot/proxy/client.trust

bash-4.2# echo "oracle.kv.auth.username=proxy_user" > /kvroot/proxy/proxy.login
bash-4.2# echo "oracle.kv.auth.pwdfile.file=proxy.passwd" >> /kvroot/proxy/proxy.login
bash-4.2# echo "oracle.kv.transport=ssl" >> /kvroot/proxy/proxy.login
bash-4.2# echo "oracle.kv.ssl.trustStore=client.trust" >> /kvroot/proxy/proxy.login

#Please use the following password/pass phrase 123456 for the next 3 commands

bash-4.2# openssl req -x509 -days 365 -newkey rsa:4096 -keyout /kvroot/proxy/key.pem -out /kvroot/proxy/certificate.pem -subj "/C=US/ST=CA/L=San/CN=${HOSTNAME}/emailAddress=localhost@oracle.com"

bash-4.2# openssl pkcs8 -topk8 -inform PEM -outform PEM -in /kvroot/proxy/key.pem -out /kvroot/proxy/key-pkcs8.pem

bash-4.2# keytool -import -alias example -keystore /kvroot/proxy/driver.trust -file /kvroot/proxy/certificate.pem

#Trust this certificate? [no]:  yes

bash-4.2# java -jar lib/sql.jar -helper-hosts localhost:5000 -store kvstore -security /kvroot/security/user.security

CREATE USER driver_user IDENTIFIED BY "DriverPass@@123";
GRANT READWRITE TO USER driver_user;
GRANT DBADMIN TO USER driver_user;

bash-4.2# nohup java -jar lib/httpproxy.jar -storeName kvstore -helperHosts localhost:5000 -httpsPort 8080 -storeSecurityFile /kvroot/proxy/proxy.login \
-sslCertificate /kvroot/proxy/certificate.pem \
-sslPrivateKey /kvroot/proxy/key-pkcs8.pem \
-sslPrivateKeyPass 123456 \
-verbose true &

bash-4.2# cat nohup.out
Picked up _JAVA_OPTIONS: -Djava.security.egd=file:/dev/./urandom
Starting Proxy
Proxy creating SSL channel
Proxy started:
async=false
helperHosts=localhost:5000
httpPort=0
httpsPort=8080
idleReadTimeout=0
kvConsistency=NONE_REQUIRED
kvDurability=COMMIT_NO_SYNC
kvRequestTimeout=-1
numAcceptThreads=3
numRequestThreads=32
proxyType=KVPROXY
queryTrace=false
sslCertificate=/kvroot/proxy/certificate.pem
sslPrivateKey=/kvroot/proxy/key-pkcs8.pem
sslPrivateKeyPass=123456
sslProtocols=TLSv1.2,TLSv1.1,TLSv1
storeName=kvstore
storeSecurityFile=/kvroot/proxy/proxy.login
verbose=true
Proxy versions: 5.5.3kvclient version: 20.3.19


bash-4.2# exit

HERE is a program Node.js connecting to the kvlite container. I am reading the go driver, If I have specific instructions, I will post them here
For node.js, you need to retrieve the certificate from the container

docker cp kvlite:/kvroot/proxy/certificate.pem /home/opc/mylocalpath

"kvlite" hostname must be known when my program is running. It is the value used when creating the container docker run -d --name=kvlite --hostname=kvlite -p 8080:8080 oracle/nosql, you can change it. It depends on your configuration
to know the hostname to use, run

$ docker exec -it kvlite /bin/bash
bash-4.2# echo "$HOSTNAME"
kvlite
bash-4.2# exit

Here is the connection string

   return new NoSQLClient({
            serviceType: ServiceType.KVSTORE,
            endpoint: 'https://kvlite:8080'
   , auth: {
        kvstore: {
            user: "driver_user",
            password: "DriverPass@@123"
        }
   }

to execute a node.js program, you need to set this variable

export NODE_EXTRA_CA_CERTS=/home/opc/mylocalpath/certificate.pem

For a container image that starts all automatically, I need more time.
Hope that it helps and looking forward to discussing it soon with you on Slack

davega-Oracle

When using go, for your test you can use the suggested connection command in the https://github.com/oracle/nosql-go-sdk example. In production, you would use the certificate as shown above

cfg := nosqldb.Config{
    Endpoint: "https://kvlite:8080",
    Mode:     "onprem",
    Username: "driver_user",
    Password: []byte("DriverPass@@123"),
    // Specify InsecureSkipVerify
    HTTPConfig: httputil.HTTPConfig{
        InsecureSkipVerify: true,
    },
    // Alternatively, specify the CertPath and ServerName
    //
    // HTTPConfig: httputil.HTTPConfig{
    //     CertPath: "/path/to/certificate-used-by-server",
    //     ServerName: "nosql.mycompany.com", // set to the "CN" subject value from the certificate
    // },
}
User_GX5XX

I truly appreciate your efforts. Thank you very much Dario.
I'll do this all and let you know how it went.

davega-Oracle

In the meantime, here is a draft version
https://github.com/dario-vega/draft-docker-kvlite-secure

1 - 19

Post Details

Added on Apr 6 2022
19 comments
850 views