10 Replies Latest reply: Feb 26, 2013 9:23 AM by user5664922 RSS

    JDBC-ODBC on 64-bit, unaligned buffers, "Invalid string or buffer length"

    903195

      It seems that the JDBC-ODBC bridge on 64-bit platforms does not align buffers that it then passes on to ODBC. This leads to the following error:

      [Microsoft][ODBC Driver Manager] Invalid string or buffer length

      The code below demonstrates that the same code will sometimes work and sometimes will produce this error, only based on how much stuff is on stack (and therefore changes memory alignment of buffers created in the JdbcOdbcDriver).

      Is this a known bug? Anybody else able to reproduce the issue? At least this seems related: Re: sun JdbcOdbc bridge can not connect to ODBC DSN on win2008 64bit platform

      Environment:
      Windows Vista SP2 64-bit
      Java(TM) SE Runtime Environment (build 1.6.0_29-b11)
      Java HotSpot(TM) 64-Bit Server VM (build 20.4-b02, mixed mode)
      Microsoft Access Driver: ACEODBC.DLL 14.00.4760.1000

      Sample data:
      http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=19704

      Output:
      BLOCK1: read(): java.sql.SQLException: [Microsoft][ODBC Driver Manager] Invalid string or buffer length
      at sun.jdbc.odbc.JdbcOdbc.createSQLException(JdbcOdbc.java:6957)
      at sun.jdbc.odbc.JdbcOdbc.standardError(JdbcOdbc.java:7114)
      at sun.jdbc.odbc.JdbcOdbc.SQLGetDataString(JdbcOdbc.java:3907)
      at sun.jdbc.odbc.JdbcOdbcResultSet.getDataString(JdbcOdbcResultSet.java:5698)
      at sun.jdbc.odbc.JdbcOdbcResultSet.getString(JdbcOdbcResultSet.java:354)
      at JdbcOdbcBug.read(JdbcOdbcBug.java:77)
      at JdbcOdbcBug.main(JdbcOdbcBug.java:53)
      BLOCK2: read(int): ok

      Code:

      import java.sql.Connection; import java.sql.DriverManager; import java.sql.ResultSet; import java.sql.SQLException; import java.sql.Statement; /** * This class demonstrates a bug in Sun's JDBC-ODBC driver. * * MS Access 64-bit ODBC drivers expect the string buffers passed to it to be word-aligned. * However, Sun's JDBC-ODBC driver does not seem to align these buffers properly (if at all). * This leads to "Invalid string or buffer length" error message. * * It is necessary to run this example with 64 bit Java. * * Sample data used: * http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=19704 * * @author Jan Raszyk, Javlin, jan.raszyk@javlin.eu */ public class JdbcOdbcBug { private static final String INPUT_FILE = "C:\\Nwind.mdb"; private static final String DATABASE = "jdbc:odbc:Driver={Microsoft Access Driver (*.mdb, *.accdb)};DBQ=" + INPUT_FILE; private static final String selTable = "SELECT * FROM Shippers"; public static void main(String[] args) throws Exception { Class.forName("sun.jdbc.odbc.JdbcOdbcDriver"); /* * Two blocks of code follow, one of which will always succeed and the * other one will always fail with "Invalid string or buffer length". * * This is due to the fact that we offset the top of the stack by 32 bits * and the JdbcOdbcDriver will allocate a string buffer not aligned to 64 bit border, * but only to 32 bit border. This is invalid and explicitly mentioned * in ODBC API documentation that alignment should be done on word size * (64 bit on 64 bit platforms). * * It is probably not safe to say which of the two blocks will fail, * as the contents of the stack may be different on different * environments. On my machine it is the first block that always fails. */ // Optionally, you can also uncomment the following line to toggle // which of the two blocks will fail // int dummyVariable = 1; // BLOCK 1: no arguments try { System.err.print("BLOCK1: read(): "); read(); System.err.println("ok"); } catch (Exception e) { e.printStackTrace(); } // BLOCK 2: one integer argument, which means the top of the stack will // be offset by 32 bits try { System.err.print("BLOCK2: read(int): "); read((int) 0); System.err.println("ok"); } catch (Exception e) { e.printStackTrace(); } } private static void read() throws ClassNotFoundException, SQLException { Connection conn = DriverManager.getConnection(DATABASE, "", ""); Statement s = conn.createStatement(); ResultSet rs = s.executeQuery(selTable); while(rs.next()) { rs.getInt(1); rs.getString(2); rs.getString(3); } rs.close(); s.close(); conn.close(); } private static void read(int x) throws ClassNotFoundException, SQLException { read(); } }

      Edited by: 900192 on 5.12.2011 2:02

        • 1. Re: JDBC-ODBC on 64-bit, unaligned buffers, "Invalid string or buffer length"
          Joe Weinstein-Oracle
          You'd best talk to MS directly, in case they feel like fixing it...
          • 2. Re: JDBC-ODBC on 64-bit, unaligned buffers, "Invalid string or buffer length"
            jschellSomeoneStoleMyAlias
            900192 wrote:
            It seems that the JDBC-ODBC bridge on 64-bit platforms does not align buffers that it then passes on to ODBC. This leads to the following error:
            That is a conclusion that does not follow from what you presented.
            [Microsoft][ODBC Driver Manager] Invalid string or buffer length
            There are many ways that can occur. For example see the following thread. And search for "This seems a bug not in MyODBC, but in ADODB."

            http://bugs.mysql.com/bug.php?id=13776

            (Despite the conclusion above there are other ways the error can occur which are caused by user input errors.)

            It is possible that a fix to the bridge might solve some problem associated with this. But also possible that a fix to the MS Odbc Manager would fix it as well. Or just something odd with the driver (but MS Access is not the only driver that can exhibit this behavior.)
            • 3. Re: JDBC-ODBC on 64-bit, unaligned buffers, "Invalid string or buffer length"
              903195
              What leads me to believe this is related to memory alignment? Here http://msdn.microsoft.com/en-us/library/windows/desktop/ms711010(v=vs.85).aspx ODBC API Reference, SQLBindCol mentions: "An application must verify that alignment is valid.", and then here http://download.sybase.com/swr/12239/rdme_902_ebf_3804.html Sybase mention's in their log that 'Calling SQLBindCol() would have returned the error HY090 "Invalid string or buffer length", when trying to bind a user supplied buffer pointer that was not aligned to the data type requested on x86 platforms.'

              Also, the bug is fully reliably reproducable based on how many variables I add on the stack (by adding more arguments to the function call).

              The MySQL bug mentioned seems to be related to MySQL's LONGTEXT type, not to the problem I'm dealing with, I am reading from MS Access nvarchar(40) field.

              Edit:
              My further tests support my case - if I offset the stack by 32 bits (by using byte or int as arguments), it will alternatively crash and not crash, when offseting the stack by 64 bits (by using double type), it will either crash in all 4 cases or will pass in all 4 cases (I use additional 32 bit local int x; variable to switch between these two scenarios).

              So I really think the JdbcOdbcDriver on 64-bit platform is not aligning its buffers to the word size - 64-bit as ODBC requires in its specs, but only to 32-bit aligned addresses.

              Edited by: 900192 on 2.12.2011 7:00
              • 4. Re: JDBC-ODBC on 64-bit, unaligned buffers, "Invalid string or buffer length"
                jschellSomeoneStoleMyAlias
                900192 wrote:
                My further tests support my case - if I offset the stack by 32 bits (by using byte or int as arguments), it will alternatively crash and not crash, when offseting the stack by 64 bits (by using double type), it will either crash in all 4 cases or will pass in all 4 cases (I use additional 32 bit local int x; variable to switch between these two scenarios).
                Using code tags post the code example that works and doesn't work. And make it clear with comments in the code how you are making it fail.
                • 5. Re: JDBC-ODBC on 64-bit, unaligned buffers, "Invalid string or buffer length"
                  903195
                  jschell wrote:
                  Using code tags post the code example that works and doesn't work. And make it clear with comments in the code how you are making it fail.
                  Well, the code in my original post does exactly that. Anyway, I have edited the post, cleaned up the code and added comments explaining in as much details as possible what is going on.
                  I have no problems posting also other tests I have tried, but I don't see the point in doing that - the original code demonstrates the issue just fine and is quite clear.
                  • 6. Re: JDBC-ODBC on 64-bit, unaligned buffers, "Invalid string or buffer length"
                    903195
                    When I enable tracing in the ODBC Administrator, I am able to see the following in the log:
                    mdbdriver       1640-1684     ENTER SQLFetch 
                              HSTMT               0000000006DC8F90
                    
                    mdbdriver       1640-1684     EXIT  SQLFetch  with return code 0 (SQL_SUCCESS)
                              HSTMT               0000000006DC8F90
                    
                    mdbdriver       1640-1684     ENTER SQLGetData 
                              HSTMT               0000000006DC8F90
                              UWORD                        1 
                              SWORD                        4 <SQL_C_LONG>
                              PTR                 <unknown type>
                              SQLLEN                       SQLLEN                       SQLLEN *                     SQLLEN *           
                    mdbdriver       1640-1684     EXIT  SQLGetData  with return code 0 (SQL_SUCCESS)
                              HSTMT               0000000006DC8F90
                              UWORD                        1 
                              SWORD                        4 <SQL_C_LONG>
                              PTR                 <unknown type>
                              SQLLEN                       SQLLEN                       SQLLEN *                     SQLLEN *           
                    mdbdriver       1640-1684     ENTER SQLGetData 
                              HSTMT               0000000006DC8F90
                              UWORD                        2 
                              SWORD                        1 <SQL_C_CHAR>
                              PTR                 0x0000000006AEB050 (NYI) 
                               SQLLEN                       SQLLEN                       SQLLEN *                     SQLLEN *           
                    mdbdriver       1640-1684     EXIT  SQLGetData  with return code -1 (SQL_ERROR)
                              HSTMT               0000000006DC8F90
                              UWORD                        2 
                              SWORD                        1 <SQL_C_CHAR>
                              PTR                 0x0000000006AEB050 (NYI) 
                               SQLLEN                       SQLLEN                       SQLLEN *                     SQLLEN *           
                              DIAG [S1090] [Microsoft][ODBC Driver Manager] Invalid string or buffer length (0)
                    I should add that reading of the first row succeeds, it is the second row that fails (in the code block that fails, the other block is obviously ok).

                    Apparently, the PTR 0x0000000006AEB050 is aligned to 64-bit, so my theory crumbles. Sadly the contents of the SQLLEN and SQLLEN* params are not visible, I guess there will be some problem in there.

                    Edited by: 900192 on 5.12.2011 7:19
                    • 7. Re: JDBC-ODBC on 64-bit, unaligned buffers, "Invalid string or buffer length"
                      jschellSomeoneStoleMyAlias
                      Apparently, the PTR 0x0000000006AEB050 is aligned to 64-bit, so my theory crumbles.
                      Which certainly seems similar to the link that I posted and the conclusion there.
                      • 8. Re: JDBC-ODBC on 64-bit, unaligned buffers, "Invalid string or buffer length"
                        905053
                        I've been having the same issue and looking over all of these threads and elsewhere, and don't see a solution. I finally got around my problem by changing the order in which I retrieve the fields from the ResultSet. I changed it to retrieve all of the strings first, then the doubles. I only did this because initially if was failing on the rs.getDouble, and wanted to see whether it was specifically the double, or something about the order. I have no idea why this is working now and not before. My logs look similar to the earlier poster. Previously it was failing on either the first or third record of the set, now it processes completely. Hope this gives someone a clue or helps someone figure something out.
                        • 9. Re: JDBC-ODBC on 64-bit, unaligned buffers, "Invalid string or buffer length"
                          user5664922
                          I hope someone can shed some additional light on this problem. I am moving a Java application for a 32-bit JVM to the 64-bit JVM and have just (in February 2013) encountered the same "Invalid string or buffer length" problem noted in these posts. I am running 64-bit Java(TM) SE Runtime Environment (build 1.6.0_32-b05) on 64-bit Windows 7 Professional SP1. Office 2010 64 bit is installed as is accessdatabaseengine2010sp1-kb2460011-x64-fullfile-en-us. On this same test system, I have found that a small test program written in Perl works without problem using the same 64 bit driver for Access. This leads me to believe that the MS drivers for Access are working correctly. The "Microsoft Access Driver (*.mdb, .accdb)" is version 14.00.6015.1000. As far as I have found, this is the latest version. In the application I am porting, the ability to import/export to Access is a small, rarely used, but necessary component of a larger application. Efficiency is of little concern for this feature. This set of posts is more relevant to my problem than anything else I have found in many weeks of searching. I have not supplied code because it is the same simple calls shown in the posts above which get strings, doubles and ints from a ResultSet. As reported above, I can step through this code in Eclipse for as long as I want without failure. Running without the debugger, it fails on the first row. I changed the code to retrieve the strings first, ran without debugger and got through about 1000 rows out of 35000 before the error happened. The code runs without problem on a 64 bit system with 32-bit JVM. The original post makes a good case for the problem being buffer alignment not being on 64 bit boundaries as required by the MS drivers or at least that the problem is sensitive to shifting of arguments on the stack. If that is the case, it can only be fixed in the in java.sql. Has anyone been able to use an odbc connection to Access with a 64-bit JVM reliably? Does anyone have any more information on this? Thanks.
                          • 10. Re: JDBC-ODBC on 64-bit, unaligned buffers, "Invalid string or buffer length"
                            PhHein
                            Hi, please don't hijack years old threads. Please start a new one, feel free to link old threads if relevant.

                            I'm locking this one.