Best encoder to use to compress old movies for Plex server?

davidm71

[H]ard|Gawd
Joined
Feb 11, 2004
Messages
1,568
Hi,

Been a while since I dabbled in video encoding but was wondering whats the best video encoder to use to convert my old movies to an acceptable size within an acceptable time frame and have good quality to play on my Plex server? I have HandBrake standing by with a choice of Nvenc, H264, H265, + AVC.

Thanks
 
I'd use H265. Much smaller file sizes at the same quality level. The tradeoff is that it asks for more processing power from the client, so it's not quite as widely compatible as H264 yet. H265 works on all my devices though, and Plex would transcode it for you if needed I believe.
 
If hdd space is no concern, h264 is easily decoded on basically any hardware at this point and plex can stream without transcoding which takes a load off the server if you have multiple users accessing simultaneously. h265 really didn't get widespread adoption and all the big streaming services are set to move to av1 at some point...
 
What does your Plex server run on? If it is moderately powered or has some sort of hardware encoding (GPU of some sort, even QuickSync on an Intel IGP), H265 will get you smaller file sizes and transcode on the fly on a GPU. A lot of newer devices will handle H265 natively also. Even a newer Synology NAS will do H265 transcoding on the fly.

Nvenc is just using the GPU hardware to encode your video for you from native to whatever you pick (H264/H265, etc.). It's not a format per se. I have generally been doing encodes in an MKV container at H265 for Plex using the software encoder on my Ryzen 3600. I haven't had any issues.
 
If you care about squeezing every last bit of quality: Handbrake's GPU H.265 encoding isn't as good as their CPU H.264 (with most of the extra quality options turned on. the biggest thing being preservation of grain structure and/or fine textures). To get the most from your GPU encoding, you need to use FFMPEG with custom commands for H.264. Handbrake doesn't utilize all of them with their offshoot.
If you have 6 or more modern cores, the CPU encoding is fast enough to not be a chore.

However, GPU encoding is very fast. And H.265 GPU encoding is noticeably better than GPU H.264.
 
h265 all day long tiny file size, and good quality if they are only dvd type rips. Anything else may not really be that necesary, depending on how much of a videophile you may or may not be.
 
It depends:
Hardware encoders will use less encoding power than software encoders, and be much faster

h264 will be bigger than h265
X264 will be smaller than h264 but bigger than h265
X265 will be smaller than all the above

X/h265 uses more decoding power than x/h264 but a lot of hardware supports both.

My personal choice is to use NVENC h256 for personal content, it’s good “enough” and space wise it is pretty good
 
If you care about squeezing every last bit of quality: Handbrake's GPU H.265 encoding isn't as good as their CPU H.264 (with most of the extra quality options turned on. the biggest thing being preservation of grain structure and/or fine textures). To get the most from your GPU encoding, you need to use FFMPEG with custom commands for H.264. Handbrake doesn't utilize all of them with their offshoot.
If you have 6 or more modern cores, the CPU encoding is fast enough to not be a chore.

However, GPU encoding is very fast. And H.265 GPU encoding is noticeably better than GPU H.264.
and to expand on this:

I did some tests in Handbrake which I examined frame by frame. A Blu-ray processed through Handbrake with h.264 CPU with most of the quality settings cranked, looked better at 9,000kbps than both quckysnce H.265 and NVENC H.265, at 9,000kbps. and it took 12,000kps to get them real close. But still missing the fine grain structure. and the overall visual in motion, is more organic and pleasing, with the grain intact.
 
and to expand on this:

I did some tests in Handbrake which I examined frame by frame. A Blu-ray processed through Handbrake with h.264 CPU with most of the quality settings cranked, looked better at 9,000kbps than both quckysnce H.265 and NVENC H.265, at 9,000kbps. and it took 12,000kps to get them real close. But still missing the fine grain structure. and the overall visual in motion, is more organic and pleasing, with the grain intact.

apples aren’t apples though, newer versions of NVENC are substantially better
 
Hi,

Glad I checked back here. Wasn't getting notifications in my inbox. Anyhow after experimenting with different formats I found two methods of encoding that worked best. The best was hands down NVenc running off of a 2080 Touring GPU. The runner up was H265 10 Bit cpu encoding. Got to say that subjectively I found the NVenc/2080 encoder to look the best and considering it took under 20 minutes to convert 30 GB of bluray content compared to almost two to three hours off of CPU I am very impressed. My plex server runs off of a Z270 i5 7600K rig with a 1060 GPU and feel has enough horsepower in my opinion to serve each video file however using NVEnc with 1060 wasn't that great and looked blocky. Have yet to try how an AMD 5700 XT will work but read its not great.

Thanks
 
Plex only supports accelerated hardware transcoding with nvidia or intel quick sync fyi.
https://support.plex.tv/articles/115002178853-using-hardware-accelerated-streaming.

It sounds like he is encoding media on a different computer and then transferring them to his Plex server. The 1060 in his Plex server should be able to transcode anything well enough.

Hi,

Glad I checked back here. Wasn't getting notifications in my inbox. Anyhow after experimenting with different formats I found two methods of encoding that worked best. The best was hands down NVenc running off of a 2080 Touring GPU. The runner up was H265 10 Bit cpu encoding. Got to say that subjectively I found the NVenc/2080 encoder to look the best and considering it took under 20 minutes to convert 30 GB of bluray content compared to almost two to three hours off of CPU I am very impressed. My plex server runs off of a Z270 i5 7600K rig with a 1060 GPU and feel has enough horsepower in my opinion to serve each video file however using NVEnc with 1060 wasn't that great and looked blocky. Have yet to try how an AMD 5700 XT will work but read its not great.

Thanks

What format did you decide on? You're using the NVenc, but are you encoding in H264 or H265 with it?
 
Ah, I had read that wrong it looked like he was gonna try the 5700 XT in place of the 1060.
 
It sounds like he is encoding media on a different computer and then transferring them to his Plex server. The 1060 in his Plex server should be able to transcode anything well enough.



What format did you decide on? You're using the NVenc, but are you encoding in H264 or H265 with it?

All the videos I have in my library were done with HandBrake using H265 8 or 10 bit CPU encoded mostly on an 8 core 9900K system I have. Maybe a few done on the Plex Server but Bluerays take too long.

Anyhow as of yesterday I decided on NVenc using the 2080. Only thing is that machine is kind of sick and has unresolved issues but once I figure it out will switch to using the Turing based GPU for all my
transcoding needs. May even buy a 2060 to replace the 1060 in my Plex Server. Anyhow as fun as this all is I have a limited number of dvds and blurays in my collection and one day the fun will stop as
most of the videos we watch these days are streamed anyhow from other sources but considering what a nice job the 2080 did in respect to file size and quality matching that of the CPU I will start encoding
the whole collection all over again.

Thanks
 
I'd hold off till the end of the year to get ampere personally (though the 3060 likely is a 2021 release)
Thats what I'm waiting on to upgrade my transcode gpu
 
I do everything in H264 BUT everything is standard resolution, and 1080P or less.. My server is also an E6410 laptop with first gen I5..

If your talking old movies, 1080p or less, then that is what I recommend. A movie ends up at ~800mb or so the way I do it
 
For DVDs that may be fine. Blurays though are like twice that. Will try H264 thanks.
 
All the videos I have in my library were done with HandBrake using H265 8 or 10 bit CPU encoded mostly on an 8 core 9900K system I have. Maybe a few done on the Plex Server but Bluerays take too long.

Anyhow as of yesterday I decided on NVenc using the 2080. Only thing is that machine is kind of sick and has unresolved issues but once I figure it out will switch to using the Turing based GPU for all my
transcoding needs. May even buy a 2060 to replace the 1060 in my Plex Server. Anyhow as fun as this all is I have a limited number of dvds and blurays in my collection and one day the fun will stop as
most of the videos we watch these days are streamed anyhow from other sources but considering what a nice job the 2080 did in respect to file size and quality matching that of the CPU I will start encoding
the whole collection all over again.

Thanks

I know you were using NVenc but did you use it to encode in the h264 or h265? It wasn't clear from your description. Obviously you used your CPU software to encode to h265 as you described, but did you also use your hardware NVenc to h265 or did you switch it to h264? I do a lot of h265 transcoding but not a lot of h264. That's why I was wondering.

Also you don't need a 2060 level of card. You can get the good Turing NVenc on a 1650 super for around $150. The only reason you would want the higher level of card is if you were also gaming on it.
 
I personally use MAKEMKV and HandBrake. MAKEMKV strips it off the blu-ray or dvd, and then I use handbrake to convert it to a mp3 so it works good with my Synology Plex Server and not overload the CPU.
 
I know you were using NVenc but did you use it to encode in the h264 or h265? It wasn't clear from your description. Obviously you used your CPU software to encode to h265 as you described, but did you also use your hardware NVenc to h265 or did you switch it to h264? I do a lot of h265 transcoding but not a lot of h264. That's why I was wondering.

Also you don't need a 2060 level of card. You can get the good Turing NVenc on a 1650 super for around $150. The only reason you would want the higher level of card is if you were also gaming on it.

I used NVenc with h265 on both a 1060 card and a 2080. Good to know about the 1650 super. On the 1060 I’ve been only playing Doom Eternal.

Thanks
 
I used NVenc with h265 on both a 1060 card and a 2080. Good to know about the 1650 super. On the 1060 I’ve been only playing Doom Eternal.

Thanks

Obviously if you're gaming at all on it, then something more than a 1650 super might be better for you ;). I had a dedicated Plex server that was basically headless, so the video card was only about transcoding. Sounds like yours is more of a multi-use system.
 
Thats good to know about the 1650. Maybe one day I'll update the 1060. Been using it as a multi use system yes at my girlfriends house playing games when I have time and serving Plex for us to watch movies. So it can adequately handle H265 though wonder if my old raspberry pi would choke on that format..

Thanks
 
Thats good to know about the 1650. Maybe one day I'll update the 1060. Been using it as a multi use system yes at my girlfriends house playing games when I have time and serving Plex for us to watch movies. So it can adequately handle H265 though wonder if my old raspberry pi would choke on that format..

Thanks

Depends on how old it is. it shouldn't matter though because your server will transcode back to h264 if it needs it. Personally I just use a Roku. But I'm sure a lot of the newer streaming devices support h265 natively so you could have direct play.
 
Depends on how old it is. it shouldn't matter though because your server will transcode back to h264 if it needs it. Personally I just use a Roku. But I'm sure a lot of the newer streaming devices support h265 natively so you could have direct play.


You use a Roku to stream? I was referring to the R Pi as a Server.
 
You use a Roku to stream? I was referring to the R Pi as a Server.

I was thinking you were using it with something like Openelec receiving from your Plex server.

I don't know if a RPi would work or not. I tend to doubt it. I actually decommissioned my server for a Synology NAS. Works surprisingly well.
 
Back
Top