• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

Compression Methods

L

llmercll

[H]ard|Gawd
Joined
Feb 17, 2010
Messages
1,369
I've got a ton of very large files (4-7gb each) that I want to compress, for storing. I believe I can shrink these file sizes considerably, and wondering what my options are, and which ones can give best compression. Speed doesn't matter.

I read about .rar and 7zip. Are they basically my best bet?

thanks!
 
Last time I checked, the .7z compression method was better than .rar, taking into account you use "maximum" or "ultra" Compression Levels and the LZMA method.

It's been a while though and 7z now has LZMA2 (in the beta) and perhaps WinRAR/Winzip have gotten better. There are also newer utilities like nanozip and KGB Archiver, etc. I'm too addicted to 7-zip to move away from it, though.
 
7zip has higher compression for multiple files such as binaries, source code, etc. For executables, WinRAR is still the best.

Also, WinRAR is far superior to 7zip when you want to ensure the integrity of your archives thanks to WinRAR's excellent recovery features.
 
Nighted said:
Also, WinRAR is far superior to 7zip when you want to ensure the integrity of your archives thanks to WinRAR's excellent recovery features.
Click to expand...

Unless it's been fixed, the problem with the recovery data Winrar creates is that if the recovery data gets any corruption, it can render all of it useless. i tested this quite some time ago and told Eugene about it, but he wasn't going to fix it at the time as he wanted the recovery records to match the max size of the other files you create (hence, it had no metadata, thus cannot tell what part of the recovery record is corrupt)...

I use Quickpar for this...

Again: maybe Winrar's been fixed now, but last I knew, it hadn't... if your data can become corrupt, so can your recovery/parity data... And last I knew, Winrar didn't have the capability to ignore corrupt recovery data...

(FWIW: you can corrupt your data, just one single bit, and then corrupt a bit in the first recovery record, and it will render your recovery data useless... It's that simple... So, test it yourself... if corrupting the first recovery record still allows you to recover your data, then Eugene's fixed it...)

I mention all this as I don't want people blinding trusting Winrar's recovery system... Unless it's been fixed...
 
The question has not been asked but it should be.

What type of files are you trying to compress? Please realize that some large files are already compressed and there will be no benefit if you compress them again.

For example, JPG, MP3, AVI, and WMV files are already compressed and do not benefit from file compression such as ZIP or RAR.
 
Deads0uls said:
The question has not been asked but it should be.

What type of files are you trying to compress? Please realize that some large files are already compressed and there will be no benefit if you compress them again.

For example, JPG, MP3, AVI, and WMV files are already compressed and do not benefit from file compression such as ZIP or RAR.
Click to expand...

Basically looking to compress h.264 video files and ISO image files.
 
Nighted said:
Also, WinRAR is far superior to 7zip when you want to ensure the integrity of your archives thanks to WinRAR's excellent recovery features.
Click to expand...

This and their interface is why I use them mainly and 7zip occasionally when the difference is significant and even then not on things of vital importance. I don't know I could be wrong if there is or isn't some kind of built in recovery if there is they really ought to advertise it. I spent a good time experimenting with both on the ultra highest solid settings and different algorithms. I found each had a slight lead depending on the files used. 7zip compressed slightly higher overall but not by a significant margin and also at a much longer pace. Winrar is very speedy and compresses almost as good. There are certain things though where 7zip definitely had a very good lead, at the moment I don't recall what it was though.

One advantage I found with 7zip for ex; when testing the PPMd algorithm I was able to consistently compress .mpg files down to 78-80% where Winrar did 88-90%.

I've tried so many different types of files and I have yet to find a compression advantage to bzip2. It does utilize all of the system threads including hyperthreads but even still LZMA has been faster in what I've tested while using only logical cores and it still compresses better.

To the OP though, I would download 7zip and Winrar as they are the most prominent right now and try both of there highest settings and see what the differences are. But in the end you mentioned doing this for files of storage, I do this often and I choose Winrar hands down because of the recovery methods. I'll trade a 5-10% compression ratio gain for archive integrity and the faster speed is just a bonus.
 
Last edited:
Deads0uls said:
Unfortunately those files are already compressed. You will not realize much gains from compressing them again.
Click to expand...

Yeah I noticed almost no compression in the files I tested. Is there nothing I can do about it? no other method or program that can farther compress these files?
 
llmercll said:
Yeah I noticed almost no compression in the files I tested. Is there nothing I can do about it? no other method or program that can farther compress these files?
Click to expand...

For the H.264, not really unless you re-encode with lower bitrate settings or tinker with the encoding options.
 
llmercll said:
Yeah I noticed almost no compression in the files I tested. Is there nothing I can do about it? no other method or program that can farther compress these files?
Click to expand...

H.264 is highly optimized for video. I don't think you will be able to find a better one.

devman makes a good suggestion. If space is a priority then choose to encode with a more lossy codec or lower the quality settings.
 
llmercll said:
Basically looking to compress h.264 video files and ISO image files.
Click to expand...

As has been said, h.264 is already highly compressed, and if anything you're probably going to increase the size slightly by using a naive binary compression on them. You can't really do any better without discarding some of the data (ie. lowering quality). ISO files will vary widely, some contain largely uncompressed data and will compress well (game/app images often fall in this category), others (such as DVD video images) are going to be highly compressed and you won't see much gain.
 
You must log in or register to reply here.
Back
Top