video format suggestions?

[skeptic]

A couple times now I've thought I had the container/codec question figured out, but I'm back to the questions stage.  Ignoring the issue of which program(s) to use, which settings, etc. what container and codecs are recommended (by users, not necessarily officially)?  What I THINK I want is:

codecs:
video codec: H.264 for the video for good quality and smaller file size,
sound codec: AC3 sound to I can pass it through to a 5.1 stereo when I get to that point
subs: forced subtitle support either burned in or automatically played. 

testing with these containers gives me:
AVI: limited options and functionality, and the program I'm using either crashes or locks up while trying to transcode as a test. 
mp4: does not officially support AC3, and while the program I'm using to transcode will create an mp4 with x264/ac3 there is no sound on playback with mplayer, and xine just crashes.
mkv: does not play natively on my Archos, limiting my ability to drag and drop movies, but I can live with that.  movies play in mplayer/xine but within lmce they have green blocks that grow and shrink from the left side of the screen, often taking up over half the screen and making it unwatchable.  I am also unable to skip forward/back properly, different symptoms with different players, but none of them work well enough to use.  pause/play is as good as I've been able to get.

If it matters, I'm trying to rip/decode from my DVD collection.

[colinjones]

Welcome to the dark art that is codecs/containers! I don't even pretend to understand it - just when you think you have it all worked out, something happens that throws you again

I find that ImTOO MPEG Encoder 3 is reasonably good for transcoding, still has issues and the results are often unexpected, but it is easy to use and fast. The Java tool ProjectX is very good, but is incredibly difficult to understand because it is for real gurus (has so many options)

I have also noticed the green on the left you talk about with x264 .mkv files - just put up with it so far (thinking it was the souce, but obviously not)

[skeptic]

Can you seek in mkv files?  The fact that the green blocky stuff doesn't show up in mplayer/xine outside of lmce gives me hope that it will be fixed soon if it's not already.  I haven't tried going from UI2 transparent to UI2 overlay, or any other things that may fix it.  I'm running 710 B3 still, and I see that RC1 is out so perhaps there is an updaged lmce specific xine or will be by 710 final.  However, if the seek issue can't be resolved then mkv files are worthless.  I'm hoping it's a setting and within my control.

On the other hand, I went back to an older version of HandBrakeCLI (0.9.1) and it IS able to create x264/ac3 AVI files within the limitations of the container.  No subtitles without a hack, no chapters without a hack, questionable B frames support, questionable variable frame rate for those movies that switch.  On the other hand, it does handle x264 + ac3 and so far this is the only container that I've been able to get to work that does.

I don't want to get into a discussion about which program to use, at least not until I know the formats I want to get the movies into, there are plenty of those threads on that subject already.  Suffice it to say, I'm fairly happy with HandBrakeCLI and I eventually want to plug it (or some other command line program) into the lmce rip script so when I rip a movie it will also transcode it.  I have 300+ DVDs waiting for ripping, but after that I want to be able to just pop one in a MD and have it happen automagically.

[colinjones]

very interested in the auto-transcode you mention in the last para... Yes, I'm pretty sure I can seek in mkv, will have to check later (nudge me if I forget to get back to you). I'm using B4 at the moment in UI2 Masking...

[skeptic]

I'll try to remind you to let me know if you can seek in mkv files.  If you can, I'll try to figure out why I can't, and why I get the green blocks.  BTW, I switched to UI2 overlay and I still see the green stuff.

As to the auto-transcode stuff, the script that does the ripping is /usr/pluto/bin/ripDiskWrapper.sh.  In that file there is a section of code (below) that looks like a good place to stuff a transcode command.  Right before the exit 0; I will put my HandBrakeCLI command probably followed by removing the .dvd file.  It's not going to show progress of the transcode, so that will kinda suck.  I may try to play with the ProgressOutput stuff (code above my C&P below where it does the rip), but it will require the output to be in a specific format of which I have no clue.

if [[ "$diskType" == 2 ]]; then
        if eval "$command"; then
                echo "Ripping successful"
                touch "$targetFileName.dvd.lock"
                mv -f "$targetFileName.dvd"{.in-progress,}
                chmod 664 "$targetFileName.dvd"
                exit 0;

I may even change some stuff to avoid the DiskCopy all together and go straight to HandBrakeCLI with a modified ProgressOutput to get the HandBrakeCLI output in format to let the progress stuff work.  But I'm getting way ahead of myself, I don't even know for sure what container/codecs I'm going to commit to.

[skeptic]

Mini-update:  I'm getting the green blocks with an x264/ac3/avi file, so the issues seems to be around x264 not the container.  It does seek properly though.  Again, the green blocks are only present while playing from lmce.  Playing with xine or mplayer on my desktop is just fine.

[colinjones]

I am able to fast forward/rewind, relative and absolute seek, etc without any problem in these files. The specs for one of them are:

               ViDEO: x264 at 2750 kbps                               
               AUDiO: AC3 5.1 384kbps 48khz                 
               ASPECT RATiO: 16:9                                                 
               RESOLUTiON: 1280x720                                             
               FRAME RATE: 25 fps                                               
               ARCHiVE SiZE: 1100MB: 24x50MB

[daniel.welch]

Hi,
You have probably checked this out, but heres whats happening for me...

I am using the latest handbrake "HandBrake 0.9.2"

http://handbrake.fr/?article=download

It allows you to rip direct from dvd disc, and supports new things like the ahem Windows version:

"### Source Options-----------------------------------------------------------

    -i, --input <string>    Set input device
    -t, --title <number>    Select a title to encode (0 to scan only,
                            default: 1)
    -L, --longest           Select the longest title
    -c, --chapters <string> Select chapters (e.g. "1-3" for chapters
                            1 to 3, or "3" for chapter 3 only,
                            default: all chapters)"

-s, --subtitle <number> Select subtitle (default: none)
    -U, --subtitle-scan     Scan for subtitles in an extra 1st pass, and choose
                            the one that's only used 10 percent of the time
                            or less. This should locate subtitles for short
                            foreign language segments. Best used in conjunction
                            with --subtitle-forced.
    -F, --subtitle-forced   Only display subtitles from the selected stream if
                            the subtitle has the forced flag set. May be used in
                            conjunction with --subtitle-scan to auto-select
                            a stream if it contains forced subtitles.
    -N, --native-language   Select subtitles with this language if it does not
          <string>          match the Audio language. Provide the language's
                            iso639-2 code (fre, eng, spa, dut, et cetera)
    -m, --markers           Add chapter markers (mp4 output format only)

I am using the Mp4 encoding and just saving the file as .avi cos I cant be bothered with any other file types.
It allows AC3 (5.1) and supports down mixes to whatever stereo you have ... 2.1 or 2 channel etc.

Maybe this new version will solve your problems?
I rip at about 2000Bitrate and i get a good picture on my projector, compared to 2500 bitrate for Handbrake 0.9.1 - so theres definitely an improvement. I take the files as large, because thats where i want to view, on my projector, in best possible scenario. For the kids cartoon dvd's i rip at about 1200 although that may be lower, but still looks ok on the projector.
For hi quality animation I use the 1800 to 2000 bit rate.

PS: I have not looked into the seek/chaptering (tend to get ready and watch from beginning to end!)- but the ability to rip from DVD disc is nicer than using dvddecrypter to .Iso image the disc first, then encode.

I also have a small script that will strip every title off, in case there are multi-title/movies on the one disc.
It also seems to overcome most of the encoding issues with encryption (of all my discs so far I have only had a few issues - One of which was "The Lion King" I too am rip/coding all my dvd's I am forced to .ISO using dvd decryptor as no matter what i ripped, there was no audio!

I saw little difference with using the x.264 and when i checked, the MP4 method produced a slightly smaller file

I have it scripted to save typing, but havent implemented it automatically yet - on disc insertion/via menu - that would be cool

[skeptic]

I am able to fast forward/rewind, relative and absolute seek, etc without any problem in these files. The specs for one of them are:

               ViDEO: x264 at 2750 kbps                               
               AUDiO: AC3 5.1 384kbps 48khz                 
               ASPECT RATiO: 16:9                                                 
               RESOLUTiON: 1280x720                                             
               FRAME RATE: 25 fps                                               
               ARCHiVE SiZE: 1100MB: 24x50MB

Hmm...  I wonder if my videos don't have key frames??  I'm not specifically setting that, just assuming HandBrakeCLI does it with sane default values.  If it's not an mkv playback issue it must be with my files and the key frames is the only thing I know of that will affect seeking.  Doesn't make sense to me as the avi version is ok, but the mkv version isn't with the same settings other than container.

Daniel - I tried HandBrakeCLI 0.9.2 as well as the SVN version.  There was a known bug in 0.9.2 that was supposedly fixed in svn, but for me I still have the same issues.  If by mp4 do you mean the ffmpeg option?  Right now I'm on a crappy old (480p HD) rear projection TV, so I don't feel like I can really compare the quality differences.  Eventually I'll be replacing it with a 1080i 60"+ plasma and I know the limitations of DVD will show up, so I want the best codec I can use without resorting to straight DVD rips (disk space).  I'm going to keep fighting with this until 710 is officially released.  At that point if I still don't have a fully working x264 (seek/no green blocks) I'll probably go back to a different video codec.


edit:  Figured out the issue with seeking.  HandBrake does not include cue markers in mkv files.  The official line is "the mkv spec does not require it and there is no intention of adding it."  I hate to rag on HandBrake as it seems to be such a nice program in general, but that's a pretty lame stance when it's such a well known limitation and causes all kinds of messed up playback everywhere. 

However, there is a quick and easy solution.  Using mkvmerge -o newfile.mkv originalfile.mkv fixes it, and seems to go almost as fast as just copying the file.

[colinjones]

skeptic - did you every think about looking further into putting that transcode command in?

[skeptic]

Sorry, no I haven't.  More and more of my newer DVDs just don't work well with the default lmce ripping or handbrake.  For now I'm just ripping on my desktop with handbrake and the ones that don't work I'm putting to the end of the list to figure out later.  I have over 300 movies to rip, which is a small collection to some of you, but it's still time consuming.  Once I get through the ones that work out of the box I'm going to work on a method of getting the others ripped.  I want to use handbrake, even though I can't get H.264+AC3 working right, because it does a really good job of picking the right title to rip as well as catching and "burning in" forced subs. 

Once I figure out a way of ripping/transcoding the way I want from a command line I'll work on getting it into LMCE.  I don't know how efficient it will be though, I really want to stick with handbrake for the transcoding, which requires a DVD or DVD image.  I tried ddrescue/dd_rescue to rip the DVD thinking it would be an acceptible way to get an error free image for handbrake to use, but even letting it run for 15 hours I never got a full usable image.  I probably didn't have the right flags set, but for now I don't have a programmatic way to rip/transcode any DVD.

I'm about 150 movies in (a little under half way).