Best encoder to use to compress old movies for Plex server?

davidm71

[H]ard|Gawd
Joined
Feb 11, 2004
Messages
1,516
Hi,

Been a while since I dabbled in video encoding but was wondering whats the best video encoder to use to convert my old movies to an acceptable size within an acceptable time frame and have good quality to play on my Plex server? I have HandBrake standing by with a choice of Nvenc, H264, H265, + AVC.

Thanks
 

Epos7

Limp Gawd
Joined
Aug 31, 2015
Messages
405
I'd use H265. Much smaller file sizes at the same quality level. The tradeoff is that it asks for more processing power from the client, so it's not quite as widely compatible as H264 yet. H265 works on all my devices though, and Plex would transcode it for you if needed I believe.
 

darckhart

Limp Gawd
Joined
Jun 15, 2013
Messages
238
If hdd space is no concern, h264 is easily decoded on basically any hardware at this point and plex can stream without transcoding which takes a load off the server if you have multiple users accessing simultaneously. h265 really didn't get widespread adoption and all the big streaming services are set to move to av1 at some point...
 

kirbyrj

Fully [H]
Joined
Feb 1, 2005
Messages
26,261
What does your Plex server run on? If it is moderately powered or has some sort of hardware encoding (GPU of some sort, even QuickSync on an Intel IGP), H265 will get you smaller file sizes and transcode on the fly on a GPU. A lot of newer devices will handle H265 natively also. Even a newer Synology NAS will do H265 transcoding on the fly.

Nvenc is just using the GPU hardware to encode your video for you from native to whatever you pick (H264/H265, etc.). It's not a format per se. I have generally been doing encodes in an MKV container at H265 for Plex using the software encoder on my Ryzen 3600. I haven't had any issues.
 

chameleoneel

2[H]4U
Joined
Aug 15, 2005
Messages
3,290
If you care about squeezing every last bit of quality: Handbrake's GPU H.265 encoding isn't as good as their CPU H.264 (with most of the extra quality options turned on. the biggest thing being preservation of grain structure and/or fine textures). To get the most from your GPU encoding, you need to use FFMPEG with custom commands for H.264. Handbrake doesn't utilize all of them with their offshoot.
If you have 6 or more modern cores, the CPU encoding is fast enough to not be a chore.

However, GPU encoding is very fast. And H.265 GPU encoding is noticeably better than GPU H.264.
 

oldmanbal

2[H]4U
Joined
Aug 27, 2010
Messages
2,196
h265 all day long tiny file size, and good quality if they are only dvd type rips. Anything else may not really be that necesary, depending on how much of a videophile you may or may not be.
 

Keljian

[H]ard|Gawd
Joined
Nov 7, 2006
Messages
1,143
It depends:
Hardware encoders will use less encoding power than software encoders, and be much faster

h264 will be bigger than h265
X264 will be smaller than h264 but bigger than h265
X265 will be smaller than all the above

X/h265 uses more decoding power than x/h264 but a lot of hardware supports both.

My personal choice is to use NVENC h256 for personal content, it’s good “enough” and space wise it is pretty good
 

chameleoneel

2[H]4U
Joined
Aug 15, 2005
Messages
3,290
If you care about squeezing every last bit of quality: Handbrake's GPU H.265 encoding isn't as good as their CPU H.264 (with most of the extra quality options turned on. the biggest thing being preservation of grain structure and/or fine textures). To get the most from your GPU encoding, you need to use FFMPEG with custom commands for H.264. Handbrake doesn't utilize all of them with their offshoot.
If you have 6 or more modern cores, the CPU encoding is fast enough to not be a chore.

However, GPU encoding is very fast. And H.265 GPU encoding is noticeably better than GPU H.264.
and to expand on this:

I did some tests in Handbrake which I examined frame by frame. A Blu-ray processed through Handbrake with h.264 CPU with most of the quality settings cranked, looked better at 9,000kbps than both quckysnce H.265 and NVENC H.265, at 9,000kbps. and it took 12,000kps to get them real close. But still missing the fine grain structure. and the overall visual in motion, is more organic and pleasing, with the grain intact.
 

Keljian

[H]ard|Gawd
Joined
Nov 7, 2006
Messages
1,143
and to expand on this:

I did some tests in Handbrake which I examined frame by frame. A Blu-ray processed through Handbrake with h.264 CPU with most of the quality settings cranked, looked better at 9,000kbps than both quckysnce H.265 and NVENC H.265, at 9,000kbps. and it took 12,000kps to get them real close. But still missing the fine grain structure. and the overall visual in motion, is more organic and pleasing, with the grain intact.
apples aren’t apples though, newer versions of NVENC are substantially better
 

davidm71

[H]ard|Gawd
Joined
Feb 11, 2004
Messages
1,516
Hi,

Glad I checked back here. Wasn't getting notifications in my inbox. Anyhow after experimenting with different formats I found two methods of encoding that worked best. The best was hands down NVenc running off of a 2080 Touring GPU. The runner up was H265 10 Bit cpu encoding. Got to say that subjectively I found the NVenc/2080 encoder to look the best and considering it took under 20 minutes to convert 30 GB of bluray content compared to almost two to three hours off of CPU I am very impressed. My plex server runs off of a Z270 i5 7600K rig with a 1060 GPU and feel has enough horsepower in my opinion to serve each video file however using NVEnc with 1060 wasn't that great and looked blocky. Have yet to try how an AMD 5700 XT will work but read its not great.

Thanks
 

kirbyrj

Fully [H]
Joined
Feb 1, 2005
Messages
26,261
Plex only supports accelerated hardware transcoding with nvidia or intel quick sync fyi.
https://support.plex.tv/articles/115002178853-using-hardware-accelerated-streaming.
It sounds like he is encoding media on a different computer and then transferring them to his Plex server. The 1060 in his Plex server should be able to transcode anything well enough.

Hi,

Glad I checked back here. Wasn't getting notifications in my inbox. Anyhow after experimenting with different formats I found two methods of encoding that worked best. The best was hands down NVenc running off of a 2080 Touring GPU. The runner up was H265 10 Bit cpu encoding. Got to say that subjectively I found the NVenc/2080 encoder to look the best and considering it took under 20 minutes to convert 30 GB of bluray content compared to almost two to three hours off of CPU I am very impressed. My plex server runs off of a Z270 i5 7600K rig with a 1060 GPU and feel has enough horsepower in my opinion to serve each video file however using NVEnc with 1060 wasn't that great and looked blocky. Have yet to try how an AMD 5700 XT will work but read its not great.

Thanks
What format did you decide on? You're using the NVenc, but are you encoding in H264 or H265 with it?
 

Spartacus09

[H]ard|Gawd
Joined
Apr 21, 2018
Messages
1,419
Ah, I had read that wrong it looked like he was gonna try the 5700 XT in place of the 1060.
 

davidm71

[H]ard|Gawd
Joined
Feb 11, 2004
Messages
1,516
It sounds like he is encoding media on a different computer and then transferring them to his Plex server. The 1060 in his Plex server should be able to transcode anything well enough.



What format did you decide on? You're using the NVenc, but are you encoding in H264 or H265 with it?
All the videos I have in my library were done with HandBrake using H265 8 or 10 bit CPU encoded mostly on an 8 core 9900K system I have. Maybe a few done on the Plex Server but Bluerays take too long.

Anyhow as of yesterday I decided on NVenc using the 2080. Only thing is that machine is kind of sick and has unresolved issues but once I figure it out will switch to using the Turing based GPU for all my
transcoding needs. May even buy a 2060 to replace the 1060 in my Plex Server. Anyhow as fun as this all is I have a limited number of dvds and blurays in my collection and one day the fun will stop as
most of the videos we watch these days are streamed anyhow from other sources but considering what a nice job the 2080 did in respect to file size and quality matching that of the CPU I will start encoding
the whole collection all over again.

Thanks
 

Spartacus09

[H]ard|Gawd
Joined
Apr 21, 2018
Messages
1,419
I'd hold off till the end of the year to get ampere personally (though the 3060 likely is a 2021 release)
Thats what I'm waiting on to upgrade my transcode gpu
 

matt167

Gawd
Joined
Dec 18, 2016
Messages
917
I do everything in H264 BUT everything is standard resolution, and 1080P or less.. My server is also an E6410 laptop with first gen I5..

If your talking old movies, 1080p or less, then that is what I recommend. A movie ends up at ~800mb or so the way I do it
 

davidm71

[H]ard|Gawd
Joined
Feb 11, 2004
Messages
1,516
For DVDs that may be fine. Blurays though are like twice that. Will try H264 thanks.
 

kirbyrj

Fully [H]
Joined
Feb 1, 2005
Messages
26,261
All the videos I have in my library were done with HandBrake using H265 8 or 10 bit CPU encoded mostly on an 8 core 9900K system I have. Maybe a few done on the Plex Server but Bluerays take too long.

Anyhow as of yesterday I decided on NVenc using the 2080. Only thing is that machine is kind of sick and has unresolved issues but once I figure it out will switch to using the Turing based GPU for all my
transcoding needs. May even buy a 2060 to replace the 1060 in my Plex Server. Anyhow as fun as this all is I have a limited number of dvds and blurays in my collection and one day the fun will stop as
most of the videos we watch these days are streamed anyhow from other sources but considering what a nice job the 2080 did in respect to file size and quality matching that of the CPU I will start encoding
the whole collection all over again.

Thanks
I know you were using NVenc but did you use it to encode in the h264 or h265? It wasn't clear from your description. Obviously you used your CPU software to encode to h265 as you described, but did you also use your hardware NVenc to h265 or did you switch it to h264? I do a lot of h265 transcoding but not a lot of h264. That's why I was wondering.

Also you don't need a 2060 level of card. You can get the good Turing NVenc on a 1650 super for around $150. The only reason you would want the higher level of card is if you were also gaming on it.
 

NIZMOZ

[H]ard|Gawd
Joined
Oct 23, 2007
Messages
1,792
I personally use MAKEMKV and HandBrake. MAKEMKV strips it off the blu-ray or dvd, and then I use handbrake to convert it to a mp3 so it works good with my Synology Plex Server and not overload the CPU.
 

davidm71

[H]ard|Gawd
Joined
Feb 11, 2004
Messages
1,516
I know you were using NVenc but did you use it to encode in the h264 or h265? It wasn't clear from your description. Obviously you used your CPU software to encode to h265 as you described, but did you also use your hardware NVenc to h265 or did you switch it to h264? I do a lot of h265 transcoding but not a lot of h264. That's why I was wondering.

Also you don't need a 2060 level of card. You can get the good Turing NVenc on a 1650 super for around $150. The only reason you would want the higher level of card is if you were also gaming on it.
I used NVenc with h265 on both a 1060 card and a 2080. Good to know about the 1650 super. On the 1060 I’ve been only playing Doom Eternal.

Thanks
 

kirbyrj

Fully [H]
Joined
Feb 1, 2005
Messages
26,261
I used NVenc with h265 on both a 1060 card and a 2080. Good to know about the 1650 super. On the 1060 I’ve been only playing Doom Eternal.

Thanks
Obviously if you're gaming at all on it, then something more than a 1650 super might be better for you ;). I had a dedicated Plex server that was basically headless, so the video card was only about transcoding. Sounds like yours is more of a multi-use system.
 

davidm71

[H]ard|Gawd
Joined
Feb 11, 2004
Messages
1,516
Thats good to know about the 1650. Maybe one day I'll update the 1060. Been using it as a multi use system yes at my girlfriends house playing games when I have time and serving Plex for us to watch movies. So it can adequately handle H265 though wonder if my old raspberry pi would choke on that format..

Thanks
 

kirbyrj

Fully [H]
Joined
Feb 1, 2005
Messages
26,261
Thats good to know about the 1650. Maybe one day I'll update the 1060. Been using it as a multi use system yes at my girlfriends house playing games when I have time and serving Plex for us to watch movies. So it can adequately handle H265 though wonder if my old raspberry pi would choke on that format..

Thanks
Depends on how old it is. it shouldn't matter though because your server will transcode back to h264 if it needs it. Personally I just use a Roku. But I'm sure a lot of the newer streaming devices support h265 natively so you could have direct play.
 

davidm71

[H]ard|Gawd
Joined
Feb 11, 2004
Messages
1,516
Depends on how old it is. it shouldn't matter though because your server will transcode back to h264 if it needs it. Personally I just use a Roku. But I'm sure a lot of the newer streaming devices support h265 natively so you could have direct play.

You use a Roku to stream? I was referring to the R Pi as a Server.
 

kirbyrj

Fully [H]
Joined
Feb 1, 2005
Messages
26,261
You use a Roku to stream? I was referring to the R Pi as a Server.
I was thinking you were using it with something like Openelec receiving from your Plex server.

I don't know if a RPi would work or not. I tend to doubt it. I actually decommissioned my server for a Synology NAS. Works surprisingly well.
 
Top