Forum Stats

  • 3,784,365 Users
  • 2,254,928 Discussions
  • 7,880,792 Comments

Discussions

Caching the entire weblogic database using coherence

706781
706781 Member Posts: 6
edited Jul 22, 2009 4:26AM in Coherence Support
I am trying to test a coherence based portal application for a database failure scenario by using the write-behind caching. I have created a sample cacheStore as given in the link:
http://wiki.tangosol.com/display/COH33UG/Sample+CacheStore

Also modifications are done in the coherence-cache-config.xml in coherence.jar as given in the link:
http://wiki.tangosol.com/display/COH33UG/Read-Through,+Write-Through,+Refresh-Ahead+and+Write-Behind+Caching#Read-Through%2CWrite-Through%2CRefresh-AheadandWrite-BehindCaching-CacheStore

This is my coherence-cache-config.xml:

<?xml version="1.0"?>

<!DOCTYPE cache-config SYSTEM "cache-config.dtd">

<cache-config>
<caching-scheme-mapping>
<cache-mapping>
<cache-name>dist-*</cache-name>
<scheme-name>distributed-rwbm</scheme-name>
</cache-mapping>
</caching-scheme-mapping>


<caching-schemes>
<distributed-scheme>
<scheme-name>distributed-rwbm</scheme-name>
<backing-map-scheme>
<read-write-backing-map-scheme>

<internal-cache-scheme>
<local-scheme/>
</internal-cache-scheme>

<cachestore-scheme>
<class-scheme>
<class-name>com.company.MyCacheStore</class-name>
<init-params>
<init-param>
<param-type>java.lang.String</param-type>
<param-value>{cache-name}</param-value>
</init-param>
</init-params>
</class-scheme>
</cachestore-scheme>
</read-write-backing-map-scheme>
</backing-map-scheme>
</distributed-scheme>
</caching-schemes>
</cache-config>

Questions:
1. How does the application interact with the cacheStore to cache data? Do I need to do any more configuration for the application to talk to the cacheStore by default? When I say CacheFactory.getCache("cacheName"), does the cacheStore gets called automatically?

2. CacheStore specifies which database table is to be cached. How to cache the entire database (Weblogic portal 10.3) using coherence 3.4.2?

Answers

  • robvarga
    robvarga Member Posts: 1,708 Gold Badge
    edited Jul 8, 2009 4:22AM
    Hi,
    user11266626 wrote:
    ...
    Questions:
    1. How does the application interact with the cacheStore to cache data? Do I need to do any more configuration for the application to talk to the cacheStore by default? When I say CacheFactory.getCache("cacheName"), does the cacheStore gets called automatically?
    The application does not interact directly with the cache store. The application interacts with the cache: reads and modifies the cache content. The cache store gets invoked on the cache misses and the changes to the cache content from Coherence.

    You don't need to do any more configuration, but your cache store needs to be able to go the correct database and the correct table based on the cache name.

    Your cache store is NOT called when you do a CacheFactory.getCache(), except that since you use wildcard cache mapping, it is instantiated if no records mapped to the particular storage node were accessed from or written to the cache with that particular name.

    >
    2. CacheStore specifies which database table is to be cached. How to cache the entire database (Weblogic portal 10.3) using coherence 3.4.2?
    This is a slight misconception. Coherence does not cache database tables. It caches objects returned by the cache store for the passed in cache keys. If the cache store maps to a database table, then this means that Coherence caches objects created from database rows corresponding to the keys to the specified entries. But Coherence does not cache an entire table. It caches only certain rows. It is your responsibility to ensure that the set of all entries correspond to the entire set of rows in the database. Also, Coherence will not load the entire data set into the cache on its own. If you want to do that, you have to implement it on your own. Coherence only provides transparent key-based access to the entire database table. E.g. Coherence does not provide QueryMap access to the entire table, it only provides QueryMap functionality on the already cached entries.

    Weblogic Portal is not a database, so I don't really understand your last question.

    Best regards,

    Robert
  • 706781
    706781 Member Posts: 6
    Hi,

    Thanks for the reply.

    By saying database, I mean the application database.

    I got some clarity but still do not understand when you say "cache store needs to be able to go the correct database and the correct table based on the cache name".

    This is my cache store implementation class where I am connecting to mysql database by name "test" which has a table called "userdetails".

    public class MyCacheStore implements CacheStore {

    protected String m_sTableName;
    protected Connection m_con;


    public MyCacheStore(String sTableName)
    {
    System.err.println("In MyCacheStore()");
    m_sTableName = sTableName;
    configureConnection();
    }

    /**
    * Set up the DB connection.
    */
    protected void configureConnection()
    {
    try
    {
    Class.forName("com.mysql.jdbc.Driver");
    m_con = DriverManager.getConnection("jdbc:mysql://localhost:3306/test", "root", "");
    m_con.setAutoCommit(true);
    }
    catch (Exception e)
    {
    e.printStackTrace();
    }
    }

    @Override
    public Object load(Object oKey) {
    Object oValue = null;
    Connection con = getConnection();
    String sSQL = "SELECT * FROM " + getTableName() + " WHERE id = ?";
    try
    {
    PreparedStatement stmt = con.prepareStatement(sSQL);
    stmt.setInt(1, new Integer(1));
    //stmt.setString(1, String.valueOf(oKey));

    ResultSet rslt = stmt.executeQuery();
    if (rslt.next())
    {
    oValue = rslt.getString(2);
    if (rslt.next())
    {
    throw new SQLException("Not a unique key: " + oKey);
    }
    }
    stmt.close();
    }
    catch (SQLException e)
    {
    e.printStackTrace();
    }
    return oValue;
    }

    I have modified my coherence-cache-config.cml file as follows:

    <?xml version="1.0"?>

    <!DOCTYPE cache-config SYSTEM "cache-config.dtd">

    <cache-config>
    <caching-scheme-mapping>
    <cache-mapping>
    <cache-name>userdetails</cache-name>
    <scheme-name>distributed-rwbm</scheme-name>
    </cache-mapping>
    </caching-scheme-mapping>


    <caching-schemes>
    <distributed-scheme>
    <scheme-name>distributed-rwbm</scheme-name>
    <backing-map-scheme>
    <read-write-backing-map-scheme>

    <internal-cache-scheme>
    <local-scheme/>
    </internal-cache-scheme>

    <cachestore-scheme>
    <class-scheme>
    <class-name>com.company.MyCacheStore</class-name>
    <init-params>
    <init-param>
    <param-type>java.lang.String</param-type>
    <param-value>*{cache-name}*</param-value>
    </init-param>
    </init-params>
    </class-scheme>
    </cachestore-scheme>
    </read-write-backing-map-scheme>
    </backing-map-scheme>
    </distributed-scheme>
    </caching-schemes>
    </cache-config>

    1. Please let me know where I am going wrong in connecting the cache store to the correct database and the correct table.
    How the cache store implementation class gets the table-name in its constructor?

    2. In the xml file, the {cache-name} has to be specified or is it taken by default the cache-name spcified in the xml?
  • robvarga
    robvarga Member Posts: 1,708 Gold Badge
    Hi,
    user11266626 wrote:
    Hi,

    Thanks for the reply.

    By saying database, I mean the application database.

    I got some clarity but still do not understand when you say "cache store needs to be able to go the correct database and the correct table based on the cache name".

    This is my cache store implementation class where I am connecting to mysql database by name "test" which has a table called "userdetails".

    public class MyCacheStore implements CacheStore {

    protected String m_sTableName;
    protected Connection m_con;


    public MyCacheStore(String sTableName)
    {
    System.err.println("In MyCacheStore()");
    m_sTableName = sTableName;
    configureConnection();
    }
    This is exactly the kind of error I hinted in my post. In the cache configuration you pasted in your previous post, the cache mappings were defined for cache names starting with "dist-", and that was defined to passed in to the constructor.

    The constructor would receive dist-... as the parameter, therefore you have to strip away the dist- prefix to arrive with what I assume you expected.
    /**
    * Set up the DB connection.
    */
    protected void configureConnection()
    {
    try
    {
    Class.forName("com.mysql.jdbc.Driver");
    m_con = DriverManager.getConnection("jdbc:mysql://localhost:3306/test", "root", "");
    m_con.setAutoCommit(true);
    }
    catch (Exception e)
    {
    e.printStackTrace();
    }
    }
    The JDBC connection is not thread-safe. You have to use a separate connection from each thread. If you use write-behind or refresh-ahead, or if you have a thread-pool configured for the cache service, then it is possible that multiple threads would attempt to use the connection which will lead to problems.

    Use a connection pool for your cache loaders / cache stores.

    @Override
    public Object load(Object oKey) {
    Object oValue = null;
    Connection con = getConnection();
    String sSQL = "SELECT * FROM " + getTableName() + " WHERE id = ?";
    try
    {
    PreparedStatement stmt = con.prepareStatement(sSQL);
    stmt.setInt(1, new Integer(1));
    //stmt.setString(1, String.valueOf(oKey));

    ResultSet rslt = stmt.executeQuery();
    if (rslt.next())
    {
    oValue = rslt.getString(2);
    if (rslt.next())
    {
    throw new SQLException("Not a unique key: " + oKey);
    }
    }
    stmt.close();
    }
    catch (SQLException e)
    {
    e.printStackTrace();
    }
    return oValue;
    }

    I have modified my coherence-cache-config.cml file as follows:

    <?xml version="1.0"?>

    <!DOCTYPE cache-config SYSTEM "cache-config.dtd">

    <cache-config>
    <caching-scheme-mapping>
    <cache-mapping>
    <cache-name>userdetails</cache-name>
    This is not what you would expect. Coherence handles wildcards only on the end of the cache name in the cache mapping, so you can have 3 kinds of cache name format specified:

    *
    something*
    something


    <scheme-name>distributed-rwbm</scheme-name>
    </cache-mapping>
    </caching-scheme-mapping>


    <caching-schemes>
    <distributed-scheme>
    <scheme-name>distributed-rwbm</scheme-name>
    <backing-map-scheme>
    <read-write-backing-map-scheme>

    <internal-cache-scheme>
    <local-scheme/>
    </internal-cache-scheme>

    <cachestore-scheme>
    <class-scheme>
    <class-name>com.company.MyCacheStore</class-name>
    <init-params>
    <init-param>
    <param-type>java.lang.String</param-type>
    <param-value>*{cache-name}*</param-value>
    This way you would end up with the constructor receiving a String starting with two asterisk (star) characters. You definitely don't need the asterisks here, at all.

    </init-param>
    </init-params>
    </class-scheme>
    </cachestore-scheme>
    </read-write-backing-map-scheme>
    </backing-map-scheme>
    </distributed-scheme>
    </caching-schemes>
    </cache-config>

    1. Please let me know where I am going wrong in connecting the cache store to the correct database and the correct table.
    How the cache store implementation class gets the table-name in its constructor?
    As mentioned above, the implementation class does not get the table name. It gets the cache name. You have to map the cache name to the table name or otherwise extract the table name from the cache name.

    2. In the xml file, the {cache-name} has to be specified or is it taken by default the cache-name spcified in the xml?
    If you don't specify an init-param without a param-name, then the no-argument constructor will be invoked. If you want the constructor with a String param invoked, you have to specify the init-param... As for what value is passed to the constructor is specified in the param-value element. You can use the {cache-name} macro, or you can use something totally different. But if you want to get the cache name, you have to use the {cache-name} macro.

    Best regards,

    Robert
    robvarga
  • 706781
    706781 Member Posts: 6
    Hi,

    The asterisks in the coherence-cache-config.xml is a typing mistake. I have not included this in the xml file.

    I understand that when the cache-store implementation class is mentioned in the coherence-cache-config.xml, the class has to be called by default. No further configurations are required as mentioned by you in the previous post.
    But my implementation class (atleast the constructor to start with) is not getting called as a result of which I am unable to test the database down scenario.

    Is there anything I have to code to call my implementation class?
    I have downloaded coherence-java-v3.4.2b411 from the oracle download site, which I understand is set to Grid editon by default in the tangosol-coherence.xml. Is it because by any way the Grid edition is disabled as a result of which I am unable to use write-behind caching? Do I need to do anything to enable write-behind caching for my application?
  • robvarga
    robvarga Member Posts: 1,708 Gold Badge
    Hi,

    one more thing.

    Please add the child element <autostart>true</autostart> to the <distributed-scheme> element.

    Also make sure that your Coherence node is not storage-disabled (you did NOT specify -Dtangosol.coherence.distributed.localstorage=false).

    Best regards,

    Robert
    robvarga
  • 706781
    706781 Member Posts: 6
    Hi,

    Thanks for the reply.

    Now my cacheStore implementation class is getting called by default. The overridden methods - store() and load() are also getting called properly.

    Now I shut down my application database (checking the continuous availability of data on DB failure through write behind caching as mentioned in the previous post), and run my application, I am unable to get the data from cache.

    Do I need to implement store() and load() methods in such a way that when the db is down, data is got from cache?

    Please suggest on this.
  • robvarga
    robvarga Member Posts: 1,708 Gold Badge
    Hi,

    to check the continuous availability you have to configure a write-behind timeout, as only in that case are your changes going to be reattempted to be written in the DB. Otherwise if you can't write the data into the DB it won't be written into the cache either (or it will, if you configure rollback-cachestore-failures to false, but in that case you will never get notified of the failure of the database write).

    To do that, you should configure a <write-delay> subelement to the <read-write-backing-map-scheme> element containing the length of the write delay period.

    This would also mean that instead of the store method, the storeAll method is likely to be invoked when there are multiple pending or to-be-retried changes to be written to the database.

    Best regards,

    Robert
    robvarga
  • 706781
    706781 Member Posts: 6
    Thanks for the reply. I have been able to cache a single row of database table using write-behind caching to check the db down scenario.
This discussion has been closed.