In this article, I will describe and present some example of how it is possible to run JUnit or TestNG test in multiple threads.

Let’s think about the situation that, that there is the test, which needs to validate 10 locales in 10 languages. Locales and languages are always dynamically taken from somewhere.
In this case, the most common way is:

for (Locale locale: locales) {
    for (Language language: languages) {
        assertStuff(locale, language);
    }
}

The problem here is that complexity will be always On^2.

So, what we can do with this?

We have multithreading! Bingo!

So, we can modify an example to something like:

for (Locale locale: locales) {
    Thread t = new Thread(() -> {
        for (Language language: languages) {
            assertStuff(locale, language);
        }
    }

    t.start();
}

Now it looks much better. But the problem is that we will not be seeing the error, because of assertions work in the main thread only.

Let’s modify the code a little bit and collect all the threads into the list:

List threads = new ArrayList<>();
for (Locale locale: locales) {
    Thread t = new Thread(() -> {
        for (Language language: languages) {
            assertStuff(locale, language);
        }
    }

    threads.add(t);
    t.start();
}

Alright, now we have the list of triggered threads. But what to with that?

After some researching about Java concurrency, I found the way how to implement Concurrent Assertions

    
void assertConcurrent(final String message, final List<? extends Runnable> runnables, final int maxTimeoutSeconds) throws InterruptedException {
        final int numThreads = runnables.size();
        final List exceptions = Collections.synchronizedList(new ArrayList<>());
        final ExecutorService threadPool = Executors.newFixedThreadPool(numThreads);
        try {
            final CountDownLatch allThreadsReady = new CountDownLatch(numThreads);
            final CountDownLatch afterInitBlocker = new CountDownLatch(1);
            final CountDownLatch allThreadsAreDone = new CountDownLatch(numThreads);
            for (final Runnable submittedTestRunnable : runnables) {
                threadPool.submit(() -> {
                    allThreadsReady.countDown();
                    try {
                        afterInitBlocker.await();
                        submittedTestRunnable.run();
                    } catch (final Throwable e) {
                        exceptions.add(e);
                    } finally {
                        allThreadsAreDone.countDown();
                    }
                });
            }
            assertTrue("Timeout during threads initializing threads.", 
                    allThreadsReady.await(runnables.size() * WAIT_MULTIPLIER, TimeUnit.MILLISECONDS));
            
            afterInitBlocker.countDown();
            assertTrue(message +" timeout! More than" + maxTimeoutSeconds + "seconds", 
                    allThreadsAreDone.await(maxTimeoutSeconds, TimeUnit.SECONDS));
        } finally {
            threadPool.shutdownNow();
        }
        assertTrue(message + "failed with errors" + exceptions, exceptions.isEmpty());
}

And then, the final test could be like:

List threads = new ArrayList<>();
for (Locale locale: locales) {
    Thread t = new Thread(() -> {
        for (Language language: languages) {
            assertStuff(locale, language);
        }
    }

    threads.add(t);
    t.start();
}
assertConcurrent("Failures are found", threads, 120); // 120 is max timeout of the test

The explanation of the Concurrent assertion is not difficult if You have an experience with Multithreading.
But even if not – feel free to use this code

Happy testing!