1 Reply Latest reply on Oct 19, 2018 9:25 PM by ChrisJenkins-Oracle

    java.lang.ClassNotFoundException: com.timesten.jdbc.TimesTenDriver

    3146278

      I have Timesten client on one machie and Timesten server on other machine.

       

      I am using apache spark using java and my connection string is : jdbc:timesten:client:dsn=[CLIENT's DSN];UID=<SERVER UID>;PWD=<SERVER PWD>

       

      I have also set LD_LIBRARY_PATH with lib location.

       

      still I am getting below error :

       

      Exception in thread "main" org.apache.spark.SparkException: Job aborted due to stage failure: Task 31 in stage 8.0 failed 4 times, most recent failure: Lost task 31.3 in stage 8.0 (TID 5301, 10.180.25.241, executor 5): java.lang.ClassNotFoundException: com.timesten.jdbc.TimesTenDriver

              at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

              at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

              at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

              at org.apache.spark.sql.execution.datasources.jdbc.DriverRegistry$.register(DriverRegistry.scala:45)

              at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$createConnectionFactory$1.apply(JdbcUtils.scala:55)

              at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$createConnectionFactory$1.apply(JdbcUtils.scala:54)

              at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.savePartition(JdbcUtils.scala:600)

              at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$saveTable$1.apply(JdbcUtils.scala:821)

              at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$saveTable$1.apply(JdbcUtils.scala:821)

              at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:929)

              at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:929)

              at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2067)

              at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2067)

              at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)

              at org.apache.spark.scheduler.Task.run(Task.scala:109)

              at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345)

              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

              at java.lang.Thread.run(Thread.java:745)

       

      Driver stacktrace:

              at org.apache.spark.scheduler.DAGScheduler.org$apache$spark$scheduler$DAGScheduler$$failJobAndIndependentStages(DAGScheduler.scala:1599)

              at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1587)

              at org.apache.spark.scheduler.DAGScheduler$$anonfun$abortStage$1.apply(DAGScheduler.scala:1586)

              at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)

              at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:48)

              at org.apache.spark.scheduler.DAGScheduler.abortStage(DAGScheduler.scala:1586)

              at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:831)

              at org.apache.spark.scheduler.DAGScheduler$$anonfun$handleTaskSetFailed$1.apply(DAGScheduler.scala:831)

              at scala.Option.foreach(Option.scala:257)

              at org.apache.spark.scheduler.DAGScheduler.handleTaskSetFailed(DAGScheduler.scala:831)

              at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.doOnReceive(DAGScheduler.scala:1820)

              at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1769)

              at org.apache.spark.scheduler.DAGSchedulerEventProcessLoop.onReceive(DAGScheduler.scala:1758)

              at org.apache.spark.util.EventLoop$$anon$1.run(EventLoop.scala:48)

              at org.apache.spark.scheduler.DAGScheduler.runJob(DAGScheduler.scala:642)

              at org.apache.spark.SparkContext.runJob(SparkContext.scala:2027)

              at org.apache.spark.SparkContext.runJob(SparkContext.scala:2048)

              at org.apache.spark.SparkContext.runJob(SparkContext.scala:2067)

              at org.apache.spark.SparkContext.runJob(SparkContext.scala:2092)

              at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1.apply(RDD.scala:929)

              at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1.apply(RDD.scala:927)

              at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)

              at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)

              at org.apache.spark.rdd.RDD.withScope(RDD.scala:363)

              at org.apache.spark.rdd.RDD.foreachPartition(RDD.scala:927)

              at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.saveTable(JdbcUtils.scala:821)

              at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:83)

              at org.apache.spark.sql.execution.datasources.SaveIntoDataSourceCommand.run(SaveIntoDataSourceCommand.scala:46)

              at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:70)

              at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:68)

              at org.apache.spark.sql.execution.command.ExecutedCommandExec.doExecute(commands.scala:86)

              at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:131)

              at org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:127)

              at org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155)

              at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)

              at org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152)

              at org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127)

              at org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:80)

              at org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:80)

              at org.apache.spark.sql.DataFrameWriter$$anonfun$runCommand$1.apply(DataFrameWriter.scala:654)

              at org.apache.spark.sql.DataFrameWriter$$anonfun$runCommand$1.apply(DataFrameWriter.scala:654)

              at org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:77)

              at org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:654)

              at org.apache.spark.sql.DataFrameWriter.saveToV1Source(DataFrameWriter.scala:273)

              at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:267)

              at com.sample.Transformation.main(Transformation.java:195)

              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

              at java.lang.reflect.Method.invoke(Method.java:498)

              at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)

              at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:879)

              at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:197)

              at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:227)

              at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:136)

              at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)

      Caused by: java.lang.ClassNotFoundException: com.timesten.jdbc.TimesTenDriver

              at java.net.URLClassLoader.findClass(URLClassLoader.java:381)

              at java.lang.ClassLoader.loadClass(ClassLoader.java:424)

              at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

              at org.apache.spark.sql.execution.datasources.jdbc.DriverRegistry$.register(DriverRegistry.scala:45)

              at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$createConnectionFactory$1.apply(JdbcUtils.scala:55)

              at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$createConnectionFactory$1.apply(JdbcUtils.scala:54)

              at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$.savePartition(JdbcUtils.scala:600)

              at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$saveTable$1.apply(JdbcUtils.scala:821)

              at org.apache.spark.sql.execution.datasources.jdbc.JdbcUtils$$anonfun$saveTable$1.apply(JdbcUtils.scala:821)

              at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:929)

              at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$29.apply(RDD.scala:929)

              at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2067)

              at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:2067)

              at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)

              at org.apache.spark.scheduler.Task.run(Task.scala:109)

              at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345)

              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)

              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)

              at java.lang.Thread.run(Thread.java:745)

       

      Any idea why this error is coming?

       

      Do I need to set anything on server as well?

       

      One more observation : when I am giving server's DSN in connection string then it gives different error but finds TimesTenDriver class, however in above scenario when i give client's DSN in connection string it cant find TimesTenDriver class.