Anybody know how to execute a bunch of threads and then wait for them to finish or only execute 2 -8 threads while going through a while loop and then wait until the last item has finished running? There are roughly 40 iterations of the loop that take about 6minutes to process and cleanse 300megs of data and predicting a time based on megs won't work too well because this is run on different servers with different specs. Here is what i've got so far. That's pretty much the jist of it, i think i'm using the concurrent classes wrong and the executor stuff wrong. Any insite on this or a link that shows how to do this with an external file would be most helpful. If anyone wants all of the source code let me know but i can't give out any test files as the data is sensitive. These are 2 seperate classes also.
ExecutorService executor = Executors.newFixedThreadPool(3);
long waitTime = 360000; //6 minutes to wait for
while (iter.hasNext()) {
projectid = iter.next();
TextDataThread tdt = new TextDataThread(projectid, fileList, inputProjectRefs, fileout, ct);
executor.execute(tdt);
}
try {
Thread.sleep(waitTime);
executor.shutdown();
executor.awaitTermination
(waitTime, TimeUnit.MILLISECONDS);
} catch (InterruptedException ignored) {
}
//**** THe TextDataThread class parts
public TextDataThread(String projectidIn, FileList fileListIn, ConcurrentMap<String, ProjRefData> inputProjectRefsIn, FileOutputStream fileoutIn, CountDownLatch ctIn) {
//setters
}
public void run() {
ConcurrentMap<Double, Factmbr> memberList = new ConcurrentHashMap<Double, Factmbr>();
}
ExecutorService executor = Executors.newFixedThreadPool(3);
long waitTime = 360000; //6 minutes to wait for
while (iter.hasNext()) {
projectid = iter.next();
TextDataThread tdt = new TextDataThread(projectid, fileList, inputProjectRefs, fileout, ct);
executor.execute(tdt);
}
try {
Thread.sleep(waitTime);
executor.shutdown();
executor.awaitTermination
(waitTime, TimeUnit.MILLISECONDS);
} catch (InterruptedException ignored) {
}
//**** THe TextDataThread class parts
public TextDataThread(String projectidIn, FileList fileListIn, ConcurrentMap<String, ProjRefData> inputProjectRefsIn, FileOutputStream fileoutIn, CountDownLatch ctIn) {
//setters
}
public void run() {
ConcurrentMap<Double, Factmbr> memberList = new ConcurrentHashMap<Double, Factmbr>();
}