Monthly Archives: July 2006

147 episodes a day

Much to my surprise, I had a few people ask me how my encoding adventures were coming along.  I wrote a blog post earlier about it, but I didn’t like it that much, so I set it to private.  I just set it back to published, because I’m too tired to think of something coherent to share and it also includes my magical mencoder settings I decided on.

Following up on that last post though, things are going really well.  These poor machines have literally been encoding non-stop for a couple of days now.  Maybe even a week, I’m not sure.  When I went to bed last night I remembered I had 450 episodes to encode.  When I started writing this post, it was down to 303.  I’m only doing the cartoons right now though, so the length of most of these episodes is either 7 minutes or 22.  It seems like it’s going pretty fast.  I would try and do the math, but that wouldn’t work out too well.

After this I have to re-encode all the live-action shows that are variable framerate as well.  I’m a little nervous testing on those, because the results haven’t been nearly as positive as it has been on the cartoons.  I really cannot say enough on how nice those look — the picture is really clear, and I seriously cannot tell that it’s a ripped copy.  One of the nice things about forcing the output fps, is that mencoder doesn’t screw around with the video bitrate, so I really am getting about 2200 kbps this time.  Before, if I left them to encode to 29.97, it would drop to around 1700.  I think that extra 500 or so has a lot to do with the quality.  They look really good.

Leave a comment

Filed under Computers, MPlayer, Multimedia

i'm only happy when it rains

I woke up this morning, and it was raining. I love the rain. It’s one of Mother Nature’s nicer way of cleaning things up. It makes the air nice and cool, and there are breezes that just make you feel great. I love it.

It got me thinking about how it might be a time for a change, too. I’m thinking about moving. Seems like it’s time for a change, and Utah doesn’t really feel like it fits me. It never did feel like it did, really. For one thing, I can’t relate to you guys too well (I grew up in Missourri, Indiana and Georgia … things are quite different here).

So, now I’m thinking about moving on, learning something new, trying something else out. If I had my way, I’d love to live in Europe. But I’ll probably stay in the States. Ideally, I’d like to move to either the Northwest coast or New England. I hear it rains a lot there.

Leave a comment

Filed under General

absolute insanity

Uncle Scrooge always had a great saying that he said someone taught him — “Work smarter, not harder.” Gosh, I still remember that Duck Tales episode from fifteen years ago or so. Now that’s crazy.

That’s what I’m doing though — I’ve gotten my encoding settings finalized, I’ve scrapped all my old encodings, and now I’m ripping everything in a nice organized little manner. Right now my floor is a huge mess, with DVDs literally everywhere, as I’ve got them stacked in piles and divvied out to three computers to rip and encode.

The other day, I quietly changed the backend class and switched the main encoder from transcode to mencoder. The main reason is that mencoder can handle variable framerates just fine, of which more than 90% of these DVDs are. That, combined with pretty much the same filters and settings I was using on transcode, the end results looks really, really nice. Every sample I’ve watched so far there has been no stutter, no weird skips, no odd artifacts. In fact, I’ve though to myself, “wow, this looks just like the DVD.” They are that smooth, so I’m very happy.

For the record, here’s my encoder settings I’m using on most all of my TV DVDs:

mencoder foo.vob -o bar.avi -ovc xvid -xvidencopts bitrate=2200 -oac copy -vf pullup,softskip -ofps 24000/1001

Pretty simple, huh? The only drawback is that I like the new settings so much that I scrapped every encoding I’ve done up to now except for a few shows. That was a lot of time down the drain, but I have a new system now which has helped me quite a bit to do this much quicker.

One thing I did is I divvied up all the discs between my three amd64s, and there are stacks all over my living room. This way I don’t have to hunt down a box or remember what’s next. When the tray pops out, I just stick the next one in, and start that one up. Right now I’m ripping about three discs every 15 minutes.

The other idea I had that I like a lot is I took a sample track from each series, and dumped it to a folder where they won’t be ovewritten. With those, I do all my encoding testing, and that’s how I found out those mencoder settings above really do work on everything I’ve thrown at it so far. Especially the cartoons, those look gorgeous.

The encoding is gonna take some time, that’s for sure. I just counted, and theres 18 more DVDs to rip, and *then* I’ll be halfway done with ripping (I haven’t even started on the live-action shows yet). My queue manager says my estimated encoding time is currently 142.34 hours. Yeeks.

Leave a comment

Filed under bend / dvd2mkv, Computers, MPlayer, Multimedia, Transcode

aspect ratios

A comment on yesterday’s blog entry got me thinking and doing some research about how the aspect ratios work, and something finally clicked. I finally understand how it works. :)

The video aspect ratio isn’t just a simple calculation of height/width, it’s the ratio at how you want the playback displayed. Fullframe is 1.33:1, flat is 1.85:1 and scope is 2.35:1. It all makes sense, now. I didn’t need to scale my videos down to 640×480 to force the resolution to 1.33, I just needed to either tell mplayer to playback at a different aspect, or fix Matroska so the aspect ratio was correct. I ended up doing the second one.

Turns out it wasn’t (necessarily) a bug in mkvtoolnix, just me not passing in another option. Actually, I didn’t even know about it tell I looked at the man page some more. Turns out there’s an –aspect-ratio flag you can use when setting it for different video streams (have I mentioned that Matroska can hold multiple video and audio streams? To say nothing of attachments, chapters, DVD menus and subtitles … why are you still using AVI?). By not setting that flag (this is where I think the bug is) it defaulted it to 1.5, even though the AVI had it set at 1.33. Using the argument –aspect-ratio 0:4/3 fixed everything though. And I don’t have to rescale my video anymore, I’m glad about that one.

Leave a comment

Filed under Computers, MPlayer, Multimedia

rambling about encoding

Well, I’ve finally settled into an encoding configuration that is presentable enough to use on most everything. It’s not the best, but it’s acceptable, and looks good on my TV, and I’m tired of not doing any encoding lately … so I’m going with it.

I actually did end up using mencoder as the default encoder instead of transcode. Well, it’s the default anytime there is something odd like variable framerate or transcode just can’t get it quite right. If the framerate is standard on it though, I run a two-pass encode on transcode instead.

I actually did run into a funky bug either with mencoder or mkvtoolnix (Matroska tools), but I’m not sure which one it is. The DVDs are encoded at 720×480 resolution, or a 1.5 ratio, but whenever I would encode them … mencoder would save them as a 1.33 ratio. The AVI would play fullscreen, but when I wrapped it into Matroska, it probably used the incorrect ratio, and rescaled it on playback so now I had black bars at the bottom and top.

I tried using -vf dsize=1.5 for a quick hack with mencoder to force it to 1.5 instead of 1.33, and I could have sworn that worked, but when I tried it later it wasn’t. So, instead, I’ve resorted to the less friendly but still functional option of rescaling the video to 640×480 instead. Not really a big deal since I’m not moving it around that much, but still not desirable. I think that’s probably the reason now that transcode would chop off a few pixels on each side when encoding, was to make sure that the aspect ratios were correctly calculated. I guess.

Another thing that’s interesting is that I could rescale it down to 320×240 if I wanted to. I’m not sure, but I think that’s what my television’s highest resolution is anyway (no HDTV for me). The video looks just as good, as far as I can tell. The only problem (there’s always something) is that mencoder always chooses to ignore my bitrate options that I’m telling it to use (2200) and encodes it at 700 kbps instead, which is pretty annoying. Even at 2200 and rescaling to 640×480, it still only encodes at around 1800. I don’t know what’s up with that. I was looking at the mailing lists yesterday, and I couldn’t find any way to force mencoder to use CBR with XviD. Bleh. Another reason I prefer transcode, right there. I eventually did find out though that you pretty much have to either double or quadruple the bitrate in your config file or arguments for it to actually downgrade to where you really want it.

I’ve also started playing with some video filters on mencoder. The cartoons still look much better with transcode, but that’s most likely because I’m passing it through three filters. Right now I’m trying to find the mencoder equivalents. It’s not all that big of a deal though, and I’m just glad I’ve found something that gives pretty good acceptable quality for now. The live-action shows are easy to encode, its the cartoons that you really notice the problems because of all the action and the cool colors. I’m planning on coming back to doing more research when I’m all finished encoding, but for now I think I’m finally set. :)

1 Comment

Filed under Computers, MPlayer, Multimedia, Transcode

so, did I just waste another three weeks of research?

I’m still doing testing on what works best for what series so I can rip and encode them all (which, there is an awful lot), and tonight I started playing around with mencoder. The thought occurred to me … why don’t you just use that instead of transcode?

Um. That’s a good question.

One of the cool things about mencoder is I don’t have to worry about selecting the AC3 track, I don’t have to worry about variable framerates, I don’t have to resize, crop, filter, post- or pre-process the video for it to look good. In fact, this is *all* I have to do:

mencoder dvd_track.vob -o xvid.avi -ovc xvid -xvidencopts bitrate=2200 -oac copy

Compare that to the average tests I run using transcode.

So, why don’t I use it? I honestly don’t remember.

I think the main reason is probably the same as for using Gentoo. Why use the pansy tools that do all the work for you when you can spend all your free-time figuring out for yourself what the heck you need to do. Besides, you get to learn new terms like “telecine,” “deinterlace” and “3:2 pulldown.” There’s a certain grace to learning how to use something that confuses the crap out of you, and you can use it later to impress other people and sound like you know what you’re doing all at once. It’s great.

I just ran an xvid one-pass encoding on the trickiest DVD set I could find, and mencoder handled it with flying colors. Am I gonna switch? I dunno yet. I might add a “meh, this is too hard to figure out how to encode, so use mencoder and forget it” flag to the database, but that’s it. I’m enjoying the four extra db columns just for tweaking the transcode settings. :)

Leave a comment

Filed under Computers, MPlayer, Multimedia, Transcode

more fun with framerates

Well, I found a way to convert a video from variable framerate to a standard one, and it’s actually pretty simple. Not only that, I found the example about it on transcode’s wiki entry for Matroska. Obviously the last place I would have been looking for it, since I know all about those two things, right? :)

The wiki says to pre-process the video with mencoder, by forcing the output fps to a certain rate. The command is simply something like this:

mencoder input.vob -ofps 24000/1001 -oac copy -ovc copy -o output.avi

That’s copying the audio and video directly, but dropping frames on the segments that are not 23.97. You would think that this would drop a lot of frames, but from most cases I’ve seen so far, there are only a few segments where it jumps back to 29.97, so it turns out pretty well, and the video looks good. There’s only a few problems, though.

First of all, mencoder doesn’t (or can’t, I’m not sure yet) create an AVI index in output.avi, so you can’t seek the file at all. You have to rebuild it first:

mplayer output.avi -saveidx avi.idx

That will do two things: build the index before playing, and use it while playing. Another way to play the file later is to load the index while playing the file:

mplayer output.avi -loadidx avi.idx

Then you can seek again.

The second problem is that it’s visibly noticable that you are dropping frames. On playback there are weird black blocks that show up when you do any kind of seeking. A small pain, I know, but that’s the price you pay. It’s really not that annoying, actually. My bigger concern is the index.

I’m playing with ways to get both fixed at once, and I’m hoping that transcode will either re-index the AVI and/or cleanup the funky blocks. Now that the video is all the same framerate, at the very least it won’t have any A/V sync issues. That’s the great part.

Another option is to get mencoder to somehow rebuild the index, but I’m going to have to do some research to find out that stuff. While I started on that though, I did find this great documentation on how to deal with progressive framerate video. Essentially, it looks like I’m on the right track. The bummer part is that there’s a little more to it than I thought, and I might have to do research for each TV series to see how they are encoded, as far as telecine and interlacing goes.

Either way, all this research boils down to one simple principle that I keep harping on: there’s really no solution that is going to work for every video you throw at it. You can come close, of course, with some generic encoding options that will fix 90% of what you have and make it viewable. But if you want best results, smallest filesize, and cleanest video, you’re just going to have to do some research. Personally, I think it’s worth it.

Leave a comment

Filed under Computers, Multimedia, Transcode

transcoding cartoons

I’ve been very cautious about posting any progress reports about my transcoding adventures, because the arguments I use keep changing while I test things. In fact, I deleted the last two posts about transcode because I later proved myself to be completely wrong.

I have, however, found some settings when it comes to cartoons that has worked well on everything I’ve thrown at it so far, minus one. The only exception was Roger Ramjet, where I do a two-pass on it because it looks better.

Cartoon DVDs are the worst culprits of trying to encode video though. All the DVDs are variable framerate or 23.97 fps or detected incorrectly, and with some being so old, it’s easy to get visual artifacts if you’re not careful. Here’s what I’ve been using though:

transcode -x vob,vob –hard_fps -M 2 -R 3 -w 2 -B 3,9,16 -y xvid -A -N 0x2000 -a 0 -b 128,0,0 –write_pid transcode.pid -f 0,4 –export_frc 1 -J ivtc -J decimate -J 32detect=force_mode=5:chromathres=2:chromadi=9 -i $1 -o cartoon.avi

Ninety percent of that is from the framerate.txt doc that comes with the transcode page. The only thing that I added myself was to copy over the Dolby Digital audio tracks instead of encoding them to MP3, and then I added --write-pid which obviously isn’t going to affect anything.

If you look at the man page (which is very nicely laid out, btw) and lookup each argument I’m using, you’ll see what I’m doing here. If you don’t feel like it, I’ll cover it briefly.

Basically I’m forcing transcode to read it at 29.97 frames per second, but forcing it to export it to 23.97 instead. It sounds strange to specifically set those, I know, but doing just that won’t save your skin. It’s the filters I’m using as well that fix the framerate and deinterlace the video that fix it up as well and make it all clean and pretty. And even better, the audio and video are in sync — which, by the way, is really hard to look for on a cartoon.

Everytime I tried to do two passes on any cartoon, it really made it look nasty. The lines would be blurry, and it just looked low quality overall. Using this one pass makes them all look pretty darn nice.

One other magic trick that I do is I tell XviD to encode it as a cartoon. From what I’ve read, and seen, doing this increases the quality a little bit. To do that, just install xvid4conf, and save a vanilla file to ~/.transcode/xvid4.cfg. Then, just copy that to the directory where your ripped DVD’s VOB files are at, and edit it so that cartoon = 1. If you’re feeling lazy, here’s how to do it with sed: sed --in-place -e s/cartoon\ =\ 0/cartoon\ =\ 1/ xvid4.cfg

Once you’re ready, go ahead and encode it with transcode. I must say, I like the results that I’ve gotten every time. And for me not to complain … well, you know that’s a step in the right direction, since I’m super picky.

Encoding live action shows is a completely different beast altogether. Essentially, there is no magic bullet, but I’ll cover that in a later post. I’m cautiously optimistic about these settings though, at least for cartoons, and would tenatively recommend them for now.

Leave a comment

Filed under Computers, Multimedia, Transcode

online board games

I figured I should post this in case anyone is interested.  I went looking for an online version of Monopoly the other day.  I knew it was around somewhere since I had played it before, but it was hard to find.

Anyway, Atari is hosting it now on their games site.  Here’s the linky: http://games.atari.com/playgames/home.jsp

They’ve got a lot of other fun stuff on there, like Sorry, Battleship and Scrabble.  Have fun. :)

1 Comment

Filed under Computers

Notes: variable framerate

I’m having the hardest time getting things to work right with encoding variable framerate DVDs, so I’m going to do a braindump on this page and update it as I find stuff out. Please feel free to e-mail me and correct me where I’m wrong!!

Post any comments here. Thanks.

Introduction

Here’s what I’ve found out, understand so far: The problem is in the fact that the framerate of some DVDs are not standard across the whole stream. Basically, your movie will start out at 29.97 frames per second, and then switch to 23.97. Then, it starts to switch back and forth again.

This is problematic for a number of reasons. For one, it throws the audio out of sync because when the encoder “probes” the video for the first time, it only looks at the framerate once, and uses that variable to decode / encode the entire file. This makes it really difficult to get the audio and video in good sync.

The only real way to “fix” this is to drop some frames, and force it to encode at a certain framerate. It’s troublesome finding just the right settings, and as far as I can tell so far, it’s also best to try your options on a case-by-case basis. There doesn’t seem to be any magic bullet that will fix all your source videos with this issue.

Transcode

The encoder I use the most is transcode. Not for any particular reason other than I like it a lot, it’s a tool just for encoding video, and I’ve had a lot of success in the past.

Transcode’s docs come with a page that talks just about framerates. If you unpack the tarball, you’ll see framerate.txt in the docs/ directory. At the end it has a tiny section about VFR. Unfortunately, the doc is showing it’s age, and so I don’t know how reliable the example given is, because some of the options are now deprecated, and some are default.

Here’s a copy of it though:

transcode -i vobs/ -V -x vob,vob -f 0,4 -M2 -R3 -w2 –export_frc 1 \
-J ivtc -J decimate -B 3,9,16 –hard_fps –print_status 10 \
-J 32detect=verbose=1:force_mode=5:chromathres=2:chromadi=9 \
-y xvid -o next1.avi

The only problem I have with it is that it encodes the audio portion to MP3. I’d prefer getting an AC3 stream instead, but everytime I try to change it, it throws the sync off by about 20 to 40 seconds. The only stuff that’s really *new* in that example that I haven’t used before is using the constant quantizer (-R 3 -w 2) and the plugins (ivtc, decimate, 32detect). I know from watching the output of the file that it’s doing something with the interlacing, but sadly, that is something I’ve never understood very well. The video quality does look good after that, but still, it’s been a hit an miss as far as a/v sync goes with me.

The solution I’ve come up with so far is to test the video by running transcode 6 times on the same file, each time with different settings, and then seeing which one works out best for that DVD. Here’s the script I’m using currently:

#!/bin/sh
if ! [ -a tc-test1.avi ]; then
transcode -a 0 -b 128,0,0 -i “$1″ -w 2200,250,100 -A -N 0x2000 -M 2 -Y 4,4,4,4 -B 1,11,8 -R 0 -x vob,vob -y xvid4 -c 1-15000 -o tc-test1.avi
fi
if ! [ -a tc-test2.avi ]; then
transcode -a 0 -b 128,0,0 -i “$1″ -w 2200,250,100 -A -N 0x2000 -M 2 -Y 4,4,4,4 -B 1,11,8 -R 0 -x vob,vob -y xvid4 -c 1-15000 -f 23.976,1 -o tc-test2.avi
fi
if ! [ -a tc-test3.avi ]; then
transcode -a 0 -b 128,0,0 -i “$1″ -w 2200,250,100 -A -N 0x2000 -M 2 -Y 4,4,4,4 -B 1,11,8 -R 0 -x vob,vob -y xvid4 -c 1-15000 -f 23.976,1 –export_fps 23.976,1 -o tc-test3.avi
fi
if ! [ -a tc-test4.avi ]; then
transcode -a 0 -b 128,0,0 -i “$1″ -w 2200,250,100 -A -N 0x2000 -M 2 -Y 4,4,4,4 -B 1,11,8 -R 0 -x vob,vob -y xvid4 -c 1-15000 -f 23.976,1 –export_fps 23.976,1 –hard_fps -o tc-test4.avi
fi
if ! [ -a tc-test5.avi ]; then
transcode -a 0 -b 128,0,0 -i “$1″ -w 2200,250,100 -A -N 0x2000 -M 2 -Y 4,4,4,4 -B 1,11,8 -R 0 -x vob,vob -y xvid4 -c 1-15000 –hard_fps -o tc-test5.avi
fi
# vfr framerate redecimate, or something
if ! [ -a tc-test6.avi ]; then
transcode -i “$1″ -x vob,vob -f 0,4 -M 2 -R 3 -w 1 –export_frc 1 -J ivtc -J decimate -B 3,9,16 –hard_fps -J 32detect=force_mode=5:chromathres=2:chromadi=9 -y xvid -c 1-15000 -o tc-test6.avi
fi

As you can see, just a simple little bash script. Checks to see if the outpfile file exists first (tc-test 1-6.avi) and then transcodes them. The “-c 1-15000″ section tells it to only encode the first 150 thousand frames, which comes out to around 5 or 10 minutes of video. I can’t remember, do the math yourself. :) It does take about 7 minutes to run through each pass though. A lot of time, I know, but it’s worth it if you can pinpoint what’s gong to work well.Anyway, I’m still currently in experimental, research and learning mode with this thing, so take everything on here with a little grain of salt. And if you find something that helps, let me know.

Helpful references


–hard_fps, not –force_fps

variable framerate

Leave a comment

Filed under Computers