One feature most archive formats are lacking is archive integrity/recovery. RAR has had this capability from the start and thus remains the format of choice for those concerned with data loss from events such as storage media bit rot: https://en.wikipedia.org/wiki/Data_degradation
It would be a shame to not use this opportunity to also incorporate data integrity.
PAR1 and PAR2 specifications are available on the net; however, I’m not sure where to find the actual PAR3 specification, but maybe you could start here: https://www.livebusinesschat.com/smf/index.php?topic=2118.0 and here: https://multipar.eu/
PA needs a huge push on the deduplication department.
Further improvements to rep-fma will help but given that rar for example can completely destroy PA for some sets of data it would be a good idea to consider other options.
Hash based deduplication like rar5 can do could be a very good way to ensure no complete duplicate files are stored twice with almost complete disregard for dictionary sizes or how far apart they are.
Sorting the data before storing it could also be a good idea, compress all the files with the same extension secuentially, given that they are more likely to contain similar data.
these kinds of optimizations are all in all relatively cheap to do in terms of run time but can help for deduplication dramatically in cases plzma4 and rep-fma are failing
Thread for Jpeg codec for advanced codec pack / .pa format.
We already did our jpeg codec 6-7 years ago, maybe @eugene remembers what time exactly :). I believe our initial numbers were that it got compression gain of around 18% (vs 22%-24% of competition) but was 2x-3x faster than WinZip jpeg even in single thread state.
So @eugene gene started rewriting it earlier this year. It will be fun to see what specs we end up with.
Unsolved Add Data Integrity Support (e.g. Parchive) to .pa Format
-
One feature most archive formats are lacking is archive integrity/recovery. RAR has had this capability from the start and thus remains the format of choice for those concerned with data loss from events such as storage media bit rot: https://en.wikipedia.org/wiki/Data_degradation
It would be a shame to not use this opportunity to also incorporate data integrity.
PAR1 and PAR2 specifications are available on the net; however, I’m not sure where to find the actual PAR3 specification, but maybe you could start here: https://www.livebusinesschat.com/smf/index.php?topic=2118.0 and here: https://multipar.eu/
-
we could add support for .par to be generated after .pa.
It would not possible to add recovery directly into .pa until pa.v2. Right now .pa format structure is based on 7zip. Plan is to move to v2 version of .pa when we exaust current codec and filters options, so we know all the requirements needed when doing v2.
And we still have a way to go with codecs and filters. Right now minimum plan next is to add jpeg and speed up fma-rep1 deduplication filter. After that we could go on to mp3 and maybe some lz/dictionary based text codec.
-
@welhaux moved this to open forums.
-
what you could do in the mean time is add support for par3 if possible to powerarchiver?
-
I currently create Par2 files before I save everything to Blu-ray. It’s not a hard additional process but it would be cool if PA files had it included as a option.
-
It would be very nice to see some integrated parity/recovery record options for the .pa format.
So far history wise, I think only WinACE and WinRAR has/had that feature. ACE is dead and RAR is not my cup of tea over all.
PowerArchiver has grown a lot and the PA format is a nice development. It would really be great to see some sort of recovery data option, to add such directly to the archive, or just create PAR2/PAR3 data to the archive in one seamless process.
