Multicore support for testing a large amount of archives at once

[AluminumHaste]

I was just testing 43 zip files to see if any of them had an issue. I selected all of them in Windows Explorer, right clicked one of them and picked Powerarchiver->Test.

I noticed right away that PA was doing 1 archive at a time and my CPU’s and HDD were basically doing nothing.
I have 4 cores and 8 threads (i7 4770k) and running this off a fast Intel SSD. Why not let PA do as many files at once as cores/threads?
Do 4 files at once or 8, what about people with Rizen or Threadripper? You could do 16 files at once or 32.
Is this possible?

[spwolf]

@aluminumhaste it is certainly possible to figure out few different ways it could be multithreaded. I have too check the speed when several are run at once and see what would be the speedup like.

[Brian Gregory]

Also there is a complication that while simultaneously testing several archives that are on an SSD would probably go faster if they were on a traditional rotating disk hard drive it could easily end up going much much slower than if they were tested one after another.

Plus testing certain kinds of archive already uses multiple threads just to test one archive.

[AluminumHaste]

@spwolf I took your suggestion of testing multiple instances at once.
So here’s my set up.
I have 200 zip files in one folder on a old, slow hard disk on my work computer.
I tested all 200 at once and got:

Time To complete: 1min 49sec
Disk Usage: ~12 MB/s
Queue Queue Length: Less than 1
CPU Usage: 25% CPU (So was using 1 core/thread)
Memory Usage: ~70MB

I then copied 50 zip files at a time into 4 sub directories and had their directories open in Explorer.
I then selected all the files in each window, then right clicked on one archive in window 1->Powerarchiver->Test.
then right clicked on one archive in window 2->Powerarchiver->Test.
then right clicked on one archive in window 3->Powerarchiver->Test.
then right clicked on one archive in window 4->Powerarchiver->Test.

Since I had to do this manually I would estimate that took between 5-8 seconds to get all 4 tests going.
The test results were:

Time To Complete: 1min 10sec (minus 5-8 seconds for the time it took to get 4 instances running)
Disk Usage: ~40MB/sec Disk Usage
Disk Queue Length: Less than or equal to 3
CPU Usage: ~90%
Memory Usage: ~240MB

So minus the time I wasted clicking on windows to get the other tests going, it shaved about 50 seconds off the test time.

Computer Stats:

Intel Core i3 2120 2 cores 4 threads (Sandy Bridge)
Seagate ST250DM000-1BD141 (https://www.seagate.com/files/staticfiles/docs/pdf/datasheet/disc/barracuda-ds1737-1-1111us.pdf)
Cheap and slow HDD.

[spwolf]

@aluminumhaste thank you for doing beautiful work with testing.
I have already tested it at the time that you wrote OP, or maybe 2-3 days later… in any case, yes it is faster.

There are two ways to implement this - test several archives at once, or test multiple files inside archives at once… i will see with @ivan whats possible, we were busy with pressing things about PA 2018.

[AluminumHaste]

Okay no problem, thank you for looking into it. With the proliferation of multi core CPUs, anything that uses more cores will be a boon.