Creation of large spanned archives is slow

dragon57

Hmm, different results, but definitely not optimal. After six minutes or so, PA created a 17GB tmp file (from compressing 30GB of source files). The progress bar was at 100% and PA continued to do something for 10 minutes or so. I then saw a z01 file appear in the destination directory (I specified to split the archive into DVD sized chunks). It continued to thrash for another 10 minutes, then a z02 file appeared. I finally canceled the operation as I had to shutdown the machine.

If this is as designed, more optimization is needed. For example, Winzip creates the z01, z02 files as the compression operation proceeds and you don’t need the intermediate tmp file (which basically requires you to have double the space) plus Winzip effectively does the operation in half the time since it doesn’t create the tmp file, then split up into the size specified in the options panel.

When creating a single zip file, I was able to create a zip from the same 30GB source with no problem (took about 6 1/2 minutes). I tried writing directly into the zip and not selecting that option and had no problems.

Hopefully this makes sense. Let me know if you need more info or want me to run more tests.

spwolf

thats completly different issue than original one, i noticed it was doing that, it was workaround as 2 days ago multivolumes were not working at all.

dragon57

@spwolf:

thats completly different issue than original one, i noticed it was doing that, it was workaround as 2 days ago multivolumes were not working at all.

Agreed. At least it is making multi-volumes now, just not very optimally :).

spwolf

@dragon57:

Agreed. At least it is making multi-volumes now, just not very optimally :).

especially on i7, where normal deflate is blazing through :-)

spwolf

move it to new thread, closing this one, thanks!

spwolf

err, closing old one :)

ivan

We will check progressbar issues, but currently the way spanned archives are created will not be changed in 11.50 for sure - they are generated in two steps - first in temp folder a complete zip is created and then that zip is spanned.

dragon57

If the splitting wasn’t so slow, this may be tolerable, even with the double disk space issue, but Winzip, even single threaded, makes split zip’s much faster using the same source files as the new PA does, even with the benefit of an i7 cpu on the front end.

spwolf

splitting itself is not slow, it is basically double action which is why it is slower than single action… its limited by the speed of hdd.

if it was done at once it would be much faster of course.

spwolf

unfortunatly right now we can cry about it and make it prettier :).

Question is how much of MV are you using vs single archive to see how much the problem affects you.

spwolf

we found some situations where there would be double the copying, and we made sure that does not happen now.

so it is still compressing and then splitting, but it might be faster for you now.