Discussions
Categories
- 197K All Categories
- 2.5K Data
- 546 Big Data Appliance
- 1.9K Data Science
- 450.8K Databases
- 221.9K General Database Discussions
- 3.8K Java and JavaScript in the Database
- 31 Multilingual Engine
- 552 MySQL Community Space
- 479 NoSQL Database
- 7.9K Oracle Database Express Edition (XE)
- 3.1K ORDS, SODA & JSON in the Database
- 556 SQLcl
- 4K SQL Developer Data Modeler
- 187.2K SQL & PL/SQL
- 21.4K SQL Developer
- 296.4K Development
- 17 Developer Projects
- 139 Programming Languages
- 293.1K Development Tools
- 110 DevOps
- 3.1K QA/Testing
- 646.1K Java
- 28 Java Learning Subscription
- 37K Database Connectivity
- 159 Java Community Process
- 105 Java 25
- 22.1K Java APIs
- 138.2K Java Development Tools
- 165.3K Java EE (Java Enterprise Edition)
- 19 Java Essentials
- 162 Java 8 Questions
- 86K Java Programming
- 81 Java Puzzle Ball
- 65.1K New To Java
- 1.7K Training / Learning / Certification
- 13.8K Java HotSpot Virtual Machine
- 94.3K Java SE
- 13.8K Java Security
- 205 Java User Groups
- 24 JavaScript - Nashorn
- Programs
- 471 LiveLabs
- 39 Workshops
- 10.2K Software
- 6.7K Berkeley DB Family
- 3.5K JHeadstart
- 5.7K Other Languages
- 2.3K Chinese
- 175 Deutsche Oracle Community
- 1.1K Español
- 1.9K Japanese
- 233 Portuguese
Reading file and validate data issue

882656
Member Posts: 13
Hey all !
I'm developing an application where I get info from a file and I have to validate the data. What I want is to improve performance. Now I'm thinking about using a PRODUCER-CONSUMER approach to improve performance, having 1 thread reading data from file and having 1 thread validating data.
What do you think? Is there a better approach on this kind of problem?
Thanks in advance
I'm developing an application where I get info from a file and I have to validate the data. What I want is to improve performance. Now I'm thinking about using a PRODUCER-CONSUMER approach to improve performance, having 1 thread reading data from file and having 1 thread validating data.
What do you think? Is there a better approach on this kind of problem?
Thanks in advance

Answers
-
879653 wrote:Why? Have you measured it and determined that it does not meet your specific, documented requirements?
What I want is to improve performance.
Or is it that you have an assumption that something you are doing is inherently "slow" or "inefficient."Now I'm thinking about using a PRODUCER-CONSUMER approach to improve performance, having 1 thread reading data from file and having 1 thread validating data.Using multiple threads may or may not improve performance. The issue you have to consider before you even put multithreading on the table is whether the tasks you're planning to multithread can logically be run independently of each other. Certainly for the most basic form--a single producer thread reading from the file and inserting into the queue and a single consumer thread reading from the queue and processing the data--should be viable. But the next logical step--multiple consumer threads--may not be viable, if processing needs to occur in the same order as reading.
And of course, it doesn't make sense to invest in improving performance unless there is an actual performance problem in the first place.
And if the processing time after reading a chunk of the file is not on the same order of magnitude as the I/O to read that chunk, then you won't gain any noticeable performance by adding another thread.What do you think? Is there a better approach on this kind of problem?I think you haven't really defined an actual problem yet. -
I think that particular idea isn't likely to improve the performance (by which I assume you mean the elapsed time to completion) of your code much. But then that's just a guess; if you really want to improve the performance of a task then your first step should be to find out which parts of the task are the worst-performing (for whatever definition of "performance" you have in mind).
This discussion has been closed.