First of all, mplayer. The latest revision is 1.0_rc2_p27120. The snapshot itself is about a month old, and I don't know of anything particularly major that's in there (I haven't had time to follow development for the past month anyway). It's been masked for testing, not because upstream has done anything funky, but because there are some new use flags, which have helped me to go back and clean up some stuff that I should have done a long time ago. I haven't heard a peep from anyone on bugzilla so either no one's tried it, or it works great.

The big changes that I can think of off the top of my head are this: first of all, there's an ewarn added if you enable the cpudetection flag. It finally dawned on me that this one might not make sense. Here's what it *doesn't* do -- magically make the build detect and build for your CPU architecture. The build system already automatically detects and installs with the correct instruction set by default, using stuff like MMX, SSE, and friends. The only reason (that I can think of) that you would use that, is if you were deploying the binary package on another computer, since it is a runtime feature, not a buildtime one. So, you'll get a little warning if you enable it, because, in general, desktop users don't need it. I also fixed the use.local.desc to be a bit more specific as well.

Secondly, there's a new use flag: custom-cpuopts which should finally solve the long-standing dance of trying to outwit users who always break their system but still allow real tweakers to do exactly what they want. Here's how it works. MPlayer's build system, again, by default, will automatically detect the CPU optimizations that are supportd for your platform (MMX, SSE, again, etc.). That means that, again, if you leave everything by default -- it should work fine.

Originally the problem we would have is that people would have their CPU optimization use flags turned off (like mmx), which would normally break the build. The ebuild is written to only disable them if they are turned off, so my solution for a long time was to tell people to just turn them all on and let MPlayer *not* disable them, therefore, revert back to autodetection. This would then cause problems, though, for those few who were building MPlayer for some other box, and they wanted specific CPU opts on or off. This use flag will fix just that.

What custom-cpuopts will do, when enabled, is forcibly enable or disable the actual CPU optimization use flags you set. This will help out those on certain profiles and setups who need to tweak their build for whatever reason. For normal desktop users, you don't need to enable it -- if it's turned off, the ebuild will completely ignore the other use flags (for CPU optimizations) and just use the default checks ... which work great. It also means that your build will be optimized the best it can, yay!

I know I really went into depth on that one, but it's been an issue with the build and users for a long time and I'm glad I finally got a cleaner solution working. Now everyone has the options to do whatever they want, but by default, everything should build and be optimized wonderfully.

Oh yes, one last thing -- if someone wants to fix bug 228799, and submit a patch upstream, I'd be extremely grateful. That would clean up the LINGUAS hack in our ebuild, which I've heard breaks on paludis anyway.

As far as transcode -- I've been out of the loop, and the media-video team has been taking things over and cleaning up (I don't even know who, sorry guys, haven't taken the time to look into that either -- I know loki_val fixed up some imagemagick / ffmpeg dep crap that I didn't want to touch, thanks man!). 1.0.5 is back in the tree, and should be the next stable candidate. I also added another rc for 1.0.6, and I have to add the 1.10 beta sometime.

That's about it for now. Have fun. Go break your box.

I've mostly been cleaning up really small stuff, small bugfixes that have been annoying me for a while.

GPNL / Packages

For the packages website, I finally fixed it so that you can search by just package name again. It's still messed up where it searches way too much stuff by default, but that'll be fixed soon. It was originally searching by atom and description, so stuff like package$ would break.

I did, however, put the basic search I want to add to the packages website into GPNL: search by atom, package, category, description or all. I need to add changelog to that list. It's not much, I know, but it's a start on the long road to getting an advanced search going. I also cleaned up the front page a bit, and added a link to the nightly database dumps.

I also cleaned up the bash script to import the data. It actually has the beginnings of some error checking now, so you shouldn't be seeing blank pages anymore. And the database is vacuumed correctly, and on a regularly basis, so things should be slightly snappier. I'm also importing the entire portage tree 3x daily now instead of twice. The import script is actually a nice cleanup for me, because if something breaks, I can run parts of it partially, instead of having to manually fix it. It's much nicer.

My next big thing is getting RDEPEND searchable.  In the database, it's combined with the DEPEND variable, so I have to separate the two out.  Once that's done, we can finally dynamically query the tree to see where ebuilds need to be fixed for binary packages.

MPlayer / Transcode

Looking better, closed like eight bugs the other day for mplayer. Finally fixed some asinine bugs of mine for transcode, have one more to go.

Took out the masked libdvdnav because it will conflict and break libdvdread. I already wrote about how I put it in my overlay so if someone wants to use it, they can.

Sword ebuilds

I finally got pretty much all the main ones in the tree that I wanted to get in. There's still two LDS ones that I have to make myself. Shouldn't be too hard. I hope. In all, there are 150 sword ebuilds in the tree. Freak. That reminds me of something else I fixed on the packages website: it lists the number of results. That's something else that's been annoying me for a while.

I still need to remove the old sword-modules ebuild and add a new virtual-type one that will pull in all of the ones based on which language they are written in. Not hard to do, just slightly tedious. Should be done soon.

lds-scriptures 3.0

Believe it or not, I'm actually planning on getting this finished this week. The actual data has been finished for a very, very long time... it's writing the documentation that I am extremely particular about because I plan on this being the final release.

That's about it for now, that I can remember.

I haven't had any luck getting dvd::rip to work with the newer branch (though I haven't updated my snapshot lately, either), but I know it's support is planned.

Anyway, give them both a try ... both very mature, good pieces of software.  I always encourage people to give transcode a try as an alternative to using MEncoder or ffmpeg.  There's a learning curve, but well written man pages will help you out immensely.

It won't work with dvd::rip, though, but that doesn't mean you can't play around with it. Besides, look at this as a perfect chance to get familiar with finally learning some CLI arguments and really learning to command your applications. That's where all the fun is, anyway.

After a quick glance, it looks like the man page has a few errors. Nothing major, just small parts of it are incomplete. Actually, everything seems much cleaner, really. The output of transcode is a little nicer, and the man page seems trimmed down and more organized. I noticed while writing the ebuild that a few options were dropped in this branch, and a few other things added as well. I'm interested to take a look at it sometime and get a closer look at what's improved. I haven't played with encoding stuff in a long time, since I've kind of given up on my idea of having a massive library of encoded files on my mythbox. Truth be told, I don't mind changing discs at all. I'm sure my position would change, though if I had some rugrats who enjoyed putting DVDs in the toaster or something like that. All in good time, though.

One thing is for certain -- writing ebuilds of development snapshots is not nearly as easy as I thought. Configuration options change quite a bit, and you have to test things pretty thoroughly. I'm glad for the experience, though, that's for sure. I feel like I'm getting a little better at it after each one.

I've been focusing mostly on just multimedia applications lately, since other Gentoo stuff has been annoying me, and this is the most fun to work on. Last week I went through the sound bugs and cleaned up some crufty old bugs that were simple fixes to ebuilds. I think that's actually the first time I've fixed almost anything in the sound herd since I joined oh so long ago. There's still a lot of stuff to be done there, but from a quick glance most of it seems pretty trivial. There's a lot of requests for version bumps in there, which I'll get around to sooner or later, usually depending on how tricky the ebuild is.

Here's a helpful note to users filing bugs, btw. If you can attach a working ebuild, that helps a lot. I usually have to go in and tweak them anyway and fix errors, but most of the time they are just cosmetic in nature, or are missing a DEPEND flag or something. Even a little effort helps, though. And if you don't know how to write ebuilds, learn! It's not that hard for most of the packages that use a simple configure, make and make install. I recommend checking out the excellent devmanual for guidance on getting started.

Or, if you're already good at writing ebuilds, try to find a proxy maintainer to put the ebuilds in the tree for you, while you do all the hard work (I really need to write more about that when I get some time). Or maybe get them into one of the Sunrise overlays. I've already dragged one user into the fray and put some of his multimedia applications in the tree. Proxy maintenance seems to work really well for me. I'm already doing it for a few applications. All I have to do is look over the ebuilds, fix them up, test the program and I'm done. The maintainer gets to do all the hard work like write the original ebuild and bug me when the version gets bumped.

Now that the wordpress security saga is finally over (it's hard masked), I've been able to get back to my own ebuilds as well, too. Here's some of the stuff that is new and I'm hoping people will check out and test.

IVTV released a new version, 0.10.1 which works across three different kernels this time (2.6.18 through 2.6.20). I upgraded my mythbox last weekend to the new kernel and version, and while it may be my imagination, I'll swear that the picture is a little cleaner than before. The ebuilds just barely got unmasked because there was a bug in pre-2.6.20.2 kernels that would bork loading the firmware for certain devices. So if you are going to use a 2.6.20 kernel with IVTV, be sure to use either gentoo-sources-2.6.20-r1 (or higher) or vanilla-sources-2.6.20.2. Those are patched, so you should be good to go.

I'm thinking about masking the other, older IVTV ebuilds in a bit here, since technically they are all deprecated now by upstream because of this new release. I probably won't remove them from the tree, though, because I'm all too familiar of the symptom of some funky version being the only one working on your hardware for some ungodly reason. Please do migrate and move onto the newer versions, though, since if any bugs come in on the older ones, I'm just going to start ignoring them.

In other IVTV news, the drivers are going to be in the mainline kernel starting with 2.6.22. I'm pretty excited about that too, since that means I'll just have to maintain a set of userland tools after that. Speaking of which, 0.10.x have a few more than before, so be sure to check them out if you're looking for some help. And please, for the love, read the README if you have any questions about the package. It contains lots of useful information.

Something else I bumped the other day was dvd::rip. It seems to have stabled up pretty good (I haven't heard any complaints). I was a bit worried at first about removing the 0.5.x series from the tree (the old GTK+ v1 releases), but things seem to be okay. I don't even remember what's new in this version of dvd::rip. I remember glancing at the changelog, but nothing exciting is coming to mind, so who knows.

I also did a CVS snapshot of transcode, which is in the tree now. Their last point release was November of 2005 (eek!) and their website recommends running the CVS version instead. A lot of multimedia applications tend to do that it seems like (mythtv, mplayer being some other examples), and while it usually works just fine (except for myth of course, where upgrading is like russian roullette ... sorry, I just can't resist ripping on Myth ) it makes it difficult for users that don't want to or don't know how to get CVS snapshots.

Transcode actually has two CVS branches as well, one for stable bugfixes, which would be the 1.0.3 release, and one for the next development release of 1.1.0. I put a snapshot of 1.0.3 in the tree, and I'll get around to doing one for 1.1.0 later on. In the meantime, I've created a forum post if you find any issues with the snapshot, I'll see what I can do.

I should mention that I have a secret love affair with transcode. I know I fawn over MPlayer and MEncoder constantly, but I love using transcode, especially for DVD encoding. The controls are a bit different, and may seem somewhat unorganized at first glance, but once you read through the entire man page, it's not bad at all. It's a great little encoding app, and I'll put a cap on my fanboyism and just suggest giving it a try in your free time. I tested it with the latest dvd::rip and it seemed just fine, so go ahead and be daring and do the same.

One thing I did notice that was new in this release was that if you started transcode and canceled the process (ctl-c), it would kill the pipes as well. That's actually very nice, since it was somewhat annoying before having to stop up to four processes just to kill transcode completely.

Other stuff that's new, there's a new version of OGMRip in the tree. I really like this slim frontend to MEncoder. The things I like the most is that it handles subtitles for me (I'm still really unclear on how those work, I need to do some manual encoding some time), wraps the output straight into Matroska, and of course is GTK+. I would say it's been pretty stable for a good while now, so if you're looking for a good ripper, I would recommend that one as well. So there you go, two recommendations for two rippers that use different backends (transcode and mencoder). There are more of course, but if you want the simplification of a GUI, you've got a few good options.

Speaking of subtitles, I somehow got roped into maintaining all the subtitle programs in the portage tree some time ago. I bumped gnome-subtitles again this week (thanks again to proxy maintenance), and there's another ebuild in bugzilla for a new one that I'll be looking at sometime soon. And I must in good conscience mention gaupol, your third option for subtitlely goodness.

I also got roped into helping out on ALSA now that Diego has left us. I'm still not sure how I got suckered into that one. I'm not making any claims to fixing things up nice and pretty, though. At the very least I hope to keep us on track to staying current with upstream and fixing some minor bugs. I'm still not convinced that I can be of any help on the team, but I'll see what happens. Just don't expect me to fix any major borkage.

Last but not least is mplayer. I finally got a new version of mplayer-bin in the tree, though that's already old news. Thankfully the biggest complaint I've gotten so far is that I've ripped out AAC support. I'm actually glad for that, because it was a royal pain in the pooty pants getting that thing even working in the first place. As far as missing media libraries go, though, that is not not really my area of expertise. I'll see what I can do about it sometime, but to be honest I'm not really in a huge hurry. I figure if you can play either the audio or video stream in the native mplayer / mencoder, then just separate the two and use mencoder-bin to do the other half. That's poor reasoning, I know, but that's why it's not high on my list to fix right away. I should also mention that there's a RealVideo stream I watch on a regular basis, half of which will only work in each player. So, I can sympathize. And yes, I do download it and then re-encode it with mplayer-bin just so I can watch it in mplayer.

I've also been working with Luca (lu_zero) on a new mplayer SVN snapshot. It's coming along quite nicely, I'd say, and hopefully should be ready and in the tree real soon now.

That's it for now, I guess there was more stuff to write about than I imagined. Sheesh. I hope the multimedia experience on Gentoo is working well for everyone. As I've said before, it was one of the main things that made me pick and stick with this distribution years ago, and I'm glad that I can finally get the chance to help out and give back at least a little bit.

As a final note, there really are a lot of bugs for sound, video and cdr stuff in the tree (well, not so much media-optical). They almost all register pretty much the same as far as importance goes to me, so if there's something that's been just nagging you for a while now, feel free to ping me on IRC (beandog on Freenode), or e-mail me (beandog at gentoo dot org) and I'll see if there's something I can't help move along a bit. No promises, though.

One thing I committed to the tree yesterday was the latest version of ogmrip. If you haven't heard of it before, I'd recommend you check it out if you're looking for a good little DVD ripping program. It's similar to dvd::rip except that instead of using transcode, this uses MEncoder. That, and ogmrip is a lot simpler. I like it a lot, though. And I know how hard it is to find a cool ripping program out of the probable dozens that are in the portage tree, so check this one out.

One of the cool features that it has that I really like is it can encode your files and wrap them straight into Matroska. Woots. Now if he would only add chapter support too, that would be really cool.

It also handles subtitles pretty easily, which I consider "the final frontier" since there's so many ways to get them, and it's all very confusing, imo. It's one of the few things I prefer leaving to a GUI when it comes to encoding video.

Working on ogmrip though got me thinking a bit about how life was like before I was a developer too, and some of the other frustrations that I had as a user. One of the problems with learning so much stuff is that I tend to forget what it was like, and what problems I wanted to fix. I am starting to remember a few of them though, and here's what I can recall:

Everything was always marked unstable, and never seemed to get marked stable.

I always hated that -- it seemed like all the multimedia stuff (which I still play around with a lot) is all marked ~arch and so you never knew if it was really unstable or just been sitting in the tree for years and no one has bothered to look at it.

Recently I joined the amd64 herd so I could hopefully help out on problems like that. So far I haven't done jack squat because I've been sitting in the shadows just observing how things go on, for fear of breaking everyone's boxes. But I have noticed one thing -- a lot of the stabilization requests that come across that get neglected are for ebuilds that are are of just totally fringe programs. I mean, pretty much all of the bugs that I've seen have been for stuff I've never even *heard* of (and I've been using Gentoo a long time, too). I suppose that's one reason that some of this stuff isn't getting around to being marked stable, is because nobody uses it in the first place. I know when I see this stuff, I immediately think to myself "I'm not gonna test it, I don't even know what it does, and I obviously don't need it if I never heard of it." I don't know if that's anyone else's take on stuff or not. But it definately explains why my fringe amd64 stable requests pretty much just sit in the tree.

Anyway, I'll see what I can do about that, at least for the multimedia stuff that I'm familiar with. It was a bit of a bumpy road getting dvd::rip stabilized since it had so many deps across a few arches that needed to be stabilized first, but it was kind of a cool sense of accomplishment once it was done. (Actually, dvd::rip is stable on all the arches except for x86, but that's because of xvid-1.1.0 bugs. Just downgrade to xvid-1.0.0 and you'll be superb.)

New ebuilds never get into the tree.

That was another thing that always bothered me about being a user, is ebuild requests would just sit out there in limbo and I had no idea why. Now I understand a little bit better. Its kind of the same situation as before, in that if there aren't any devs that are interested in using it, they're not going to support it.

I was poking around the media-tv bugs today, and I saw a few programs that actually looked pretty cool. I'm going to try and get them into the tree (provided they are stable, and work well) since it looks like they would be pretty beneficial. Then I started looking at other ones that I wasn't interested in using, but they still looked pretty helpful. It got me thinking about something I read in this great article, Myths, Lies, and Truths about the Linux kernel.

The myth was this: "My driver is only for an obscure piece of hardware, it would never be accepted into the main kernel." And the awesome response was this:

This just is not true at all. We have a whole sub-architecture that only has 2 users in the world out there. We have drivers that I know have only one user, as there was only one piece of hardware ever made for it. It just isn't true, we will take drivers for anything into our tree, as we really want it.

Thinking about that really got me thinking about how I've always perceived Gentoo as well -- just totally willing to take this fringe stuff that no one else has heard of and integrate it into the tree. Just replace "drivers" with "software" in that quote, and that's the feeling I've always had, and I think that's what makes Gentoo really cool. Sure, Debian, Suse, Fedora and Ubuntu might not package your really odd software in their tree, but Gentoo will! I've just always found it really cool that the tree has so much stuff in there that I have never heard of and never would have been able to find out about on my own.

I think that keeping these tiny, well-written and working applications in the tree just increases the cool factor, and I'm going to try and help out some more on that too.

I can think of some other stuff that bugged me, but I'm running out of steam so I better stop there. Gentoo is some great stuff, and that's all there is to it.

Anyway, since the cartoons will finish encoding today, now I'm left with a completely different beast -- live-action shows. The same generic mencoder settings I'm using for cartoons work fine on these as well, but they don't look quite as nice. It's probably going to be back to the drawing board for a few days as I resume testing and research.

A while back, I thought I had the perfect solution. Since all I really needed to do to fix them so that transcode could process them was fix the variable framerate, I would just encode them, copying the audio and video, and then forcing the output fps. That actually worked great until you go to encode them *after* that.

To be honest, I've only tested that idea on two series so far, and they were both on TV shows from Universal Studios (A-Team and Murder, She Wrote in case you were curious). I don't know if I've mentioned this before, but Universal makes pretty much the worst discs out of the entire lot. The Incredible Hulk is the first series I've gotten from them where I could actually play all the discs. And I'm not just talking about my computer can't play them, they send the DVD firmware for a wild loop as well.

Depending on which DVD drive I'm using, some of them might be able to actually pull the entire file off. My Pioneer and Lite-On drives work great, while the Sony one would lock up my computer completely, and I'd have to hardboot it. In fact, I've been pleasantly surprised by the results with my Pioneer drives. They throw a lot of error messages to the system log, but if it really freaks out, all I have to do is forcibly open the tray, and the drive actually resets itself! Normally when I have to rip a disc out myself, the drive won't be able to read until I reboot the computer to kind of clear the drive's head. The Pioneer drives, however, still work. After I take them out, the open/close button still works, and I'm able to keep on ripping. I'm impressed.

So, my sampling of testing the worst discs in the bunch may not be the best approach. When I tried the A-Team one, it worked fine, but you could tell where it was switching framerates. Well, I need to clarify a little bit. The first copy/copy pass where I just used -ofps, *that* video looked great. It was when I transcoded from that that it looked like crap. Part of the problem might have been that I had to reindex the video because the AVI stream was broken.

One thing I could do is do a near-lossless encode on the audio or video so it at least processes the media file, albeit lightly, so the index is working without having to rebuild it. The audio would be pretty simple ... just use 'pcm' as the output codec, pretty much copying it as a wav file. The video would be a little harder. I suppose I could export it as an MPEG2 stream again, and see what that does. Hmm, I hadn't thought of that. Another idea is to give up on the AC3 stream and encode to the audio to MP3 or Vorbis or something. Doing any kind of encoding will keep the index intact. I'm just trying to find a solution that will leave it as near the original as possible.

I think that broken AVI index is what's causing the awful artifacts though. Well, either that or the fact that it's dropping (or duplicating, depending on what your target fps is) frames on the first encode. What happens though is you take that re-indexed video with the forced standard framerate, and *then* you run it through transcode. Again, I haven't done much testing at all, just tried a two-pass encode on two different files. On Murder, She Wrote, the second file I tested, it actually looked really good for about the first twenty minutes. Then the entire thing was offset horizontally by about 60%, and there was a nasty green overlay. I wish I still had that original transcode, I could have posted a snapshot.

Whoops ... one thing I just noticed is that my little reindex script was outputting the framerate to 29.97 (30000/1001) instead of 23.97 (24000/1001). That definately wouldn't work well. I wonder if that's how I was doing it on those first tries. I don't think I was, but that might explain quite a few things. Time for more testing!

Well, this post ended up much longer than I had intentioned, but it definately got the wheels turning in my head a little bit. I've got a few ideas now that I can't wait to try out. There may just be a perfect solution yet.

That's what I'm doing though -- I've gotten my encoding settings finalized, I've scrapped all my old encodings, and now I'm ripping everything in a nice organized little manner. Right now my floor is a huge mess, with DVDs literally everywhere, as I've got them stacked in piles and divvied out to three computers to rip and encode.

The other day, I quietly changed the backend class and switched the main encoder from transcode to mencoder. The main reason is that mencoder can handle variable framerates just fine, of which more than 90% of these DVDs are. That, combined with pretty much the same filters and settings I was using on transcode, the end results looks really, really nice. Every sample I've watched so far there has been no stutter, no weird skips, no odd artifacts. In fact, I've though to myself, "wow, this looks just like the DVD." They are that smooth, so I'm very happy.

For the record, here's my encoder settings I'm using on most all of my TV DVDs:

mencoder foo.vob -o bar.avi -ovc xvid -xvidencopts bitrate=2200 -oac copy -vf pullup,softskip -ofps 24000/1001

Pretty simple, huh? The only drawback is that I like the new settings so much that I scrapped every encoding I've done up to now except for a few shows. That was a lot of time down the drain, but I have a new system now which has helped me quite a bit to do this much quicker.

One thing I did is I divvied up all the discs between my three amd64s, and there are stacks all over my living room. This way I don't have to hunt down a box or remember what's next. When the tray pops out, I just stick the next one in, and start that one up. Right now I'm ripping about three discs every 15 minutes.

The other idea I had that I like a lot is I took a sample track from each series, and dumped it to a folder where they won't be ovewritten. With those, I do all my encoding testing, and that's how I found out those mencoder settings above really do work on everything I've thrown at it so far. Especially the cartoons, those look gorgeous.

The encoding is gonna take some time, that's for sure. I just counted, and theres 18 more DVDs to rip, and *then* I'll be halfway done with ripping (I haven't even started on the live-action shows yet). My queue manager says my estimated encoding time is currently 142.34 hours. Yeeks.

I actually did end up using mencoder as the default encoder instead of transcode. Well, it's the default anytime there is something odd like variable framerate or transcode just can't get it quite right. If the framerate is standard on it though, I run a two-pass encode on transcode instead.

I actually did run into a funky bug either with mencoder or mkvtoolnix (Matroska tools), but I'm not sure which one it is. The DVDs are encoded at 720x480 resolution, or a 1.5 ratio, but whenever I would encode them ... mencoder would save them as a 1.33 ratio. The AVI would play fullscreen, but when I wrapped it into Matroska, it probably used the incorrect ratio, and rescaled it on playback so now I had black bars at the bottom and top.

I tried using -vf dsize=1.5 for a quick hack with mencoder to force it to 1.5 instead of 1.33, and I could have sworn that worked, but when I tried it later it wasn't. So, instead, I've resorted to the less friendly but still functional option of rescaling the video to 640x480 instead. Not really a big deal since I'm not moving it around that much, but still not desirable. I think that's probably the reason now that transcode would chop off a few pixels on each side when encoding, was to make sure that the aspect ratios were correctly calculated. I guess.

Another thing that's interesting is that I could rescale it down to 320x240 if I wanted to. I'm not sure, but I think that's what my television's highest resolution is anyway (no HDTV for me). The video looks just as good, as far as I can tell. The only problem (there's always something) is that mencoder always chooses to ignore my bitrate options that I'm telling it to use (2200) and encodes it at 700 kbps instead, which is pretty annoying. Even at 2200 and rescaling to 640x480, it still only encodes at around 1800. I don't know what's up with that. I was looking at the mailing lists yesterday, and I couldn't find any way to force mencoder to use CBR with XviD. Bleh. Another reason I prefer transcode, right there. I eventually did find out though that you pretty much have to either double or quadruple the bitrate in your config file or arguments for it to actually downgrade to where you really want it.

I've also started playing with some video filters on mencoder. The cartoons still look much better with transcode, but that's most likely because I'm passing it through three filters. Right now I'm trying to find the mencoder equivalents. It's not all that big of a deal though, and I'm just glad I've found something that gives pretty good acceptable quality for now. The live-action shows are easy to encode, its the cartoons that you really notice the problems because of all the action and the cool colors. I'm planning on coming back to doing more research when I'm all finished encoding, but for now I think I'm finally set.

Um. That's a good question.

One of the cool things about mencoder is I don't have to worry about selecting the AC3 track, I don't have to worry about variable framerates, I don't have to resize, crop, filter, post- or pre-process the video for it to look good. In fact, this is *all* I have to do:

mencoder dvd_track.vob -o xvid.avi -ovc xvid -xvidencopts bitrate=2200 -oac copy

Compare that to the average tests I run using transcode.

So, why don't I use it? I honestly don't remember.

I think the main reason is probably the same as for using Gentoo. Why use the pansy tools that do all the work for you when you can spend all your free-time figuring out for yourself what the heck you need to do. Besides, you get to learn new terms like "telecine," "deinterlace" and "3:2 pulldown." There's a certain grace to learning how to use something that confuses the crap out of you, and you can use it later to impress other people and sound like you know what you're doing all at once. It's great.

I just ran an xvid one-pass encoding on the trickiest DVD set I could find, and mencoder handled it with flying colors. Am I gonna switch? I dunno yet. I might add a "meh, this is too hard to figure out how to encode, so use mencoder and forget it" flag to the database, but that's it. I'm enjoying the four extra db columns just for tweaking the transcode settings.

The wiki says to pre-process the video with mencoder, by forcing the output fps to a certain rate. The command is simply something like this:

mencoder input.vob -ofps 24000/1001 -oac copy -ovc copy -o output.avi

That's copying the audio and video directly, but dropping frames on the segments that are not 23.97. You would think that this would drop a lot of frames, but from most cases I've seen so far, there are only a few segments where it jumps back to 29.97, so it turns out pretty well, and the video looks good. There's only a few problems, though.

First of all, mencoder doesn't (or can't, I'm not sure yet) create an AVI index in output.avi, so you can't seek the file at all. You have to rebuild it first:

mplayer output.avi -saveidx avi.idx

That will do two things: build the index before playing, and use it while playing. Another way to play the file later is to load the index while playing the file:

mplayer output.avi -loadidx avi.idx

Then you can seek again.

The second problem is that it's visibly noticable that you are dropping frames. On playback there are weird black blocks that show up when you do any kind of seeking. A small pain, I know, but that's the price you pay. It's really not that annoying, actually. My bigger concern is the index.

I'm playing with ways to get both fixed at once, and I'm hoping that transcode will either re-index the AVI and/or cleanup the funky blocks. Now that the video is all the same framerate, at the very least it won't have any A/V sync issues. That's the great part.

Another option is to get mencoder to somehow rebuild the index, but I'm going to have to do some research to find out that stuff. While I started on that though, I did find this great documentation on how to deal with progressive framerate video. Essentially, it looks like I'm on the right track. The bummer part is that there's a little more to it than I thought, and I might have to do research for each TV series to see how they are encoded, as far as telecine and interlacing goes.

Either way, all this research boils down to one simple principle that I keep harping on: there's really no solution that is going to work for every video you throw at it. You can come close, of course, with some generic encoding options that will fix 90% of what you have and make it viewable. But if you want best results, smallest filesize, and cleanest video, you're just going to have to do some research. Personally, I think it's worth it.

I have, however, found some settings when it comes to cartoons that has worked well on everything I've thrown at it so far, minus one. The only exception was Roger Ramjet, where I do a two-pass on it because it looks better.

Cartoon DVDs are the worst culprits of trying to encode video though. All the DVDs are variable framerate or 23.97 fps or detected incorrectly, and with some being so old, it's easy to get visual artifacts if you're not careful. Here's what I've been using though:

transcode -x vob,vob --hard_fps -M 2 -R 3 -w 2 -B 3,9,16 -y xvid -A -N 0x2000 -a 0 -b 128,0,0 --write_pid transcode.pid -f 0,4 --export_frc 1 -J ivtc -J decimate -J 32detect=force_mode=5:chromathres=2:chromadi=9 -i $1 -o cartoon.avi

Ninety percent of that is from the framerate.txt doc that comes with the transcode page. The only thing that I added myself was to copy over the Dolby Digital audio tracks instead of encoding them to MP3, and then I added --write-pid which obviously isn't going to affect anything.

If you look at the man page (which is very nicely laid out, btw) and lookup each argument I'm using, you'll see what I'm doing here. If you don't feel like it, I'll cover it briefly.

Basically I'm forcing transcode to read it at 29.97 frames per second, but forcing it to export it to 23.97 instead. It sounds strange to specifically set those, I know, but doing just that won't save your skin. It's the filters I'm using as well that fix the framerate and deinterlace the video that fix it up as well and make it all clean and pretty. And even better, the audio and video are in sync -- which, by the way, is really hard to look for on a cartoon.

Everytime I tried to do two passes on any cartoon, it really made it look nasty. The lines would be blurry, and it just looked low quality overall. Using this one pass makes them all look pretty darn nice.

One other magic trick that I do is I tell XviD to encode it as a cartoon. From what I've read, and seen, doing this increases the quality a little bit. To do that, just install xvid4conf, and save a vanilla file to ~/.transcode/xvid4.cfg. Then, just copy that to the directory where your ripped DVD's VOB files are at, and edit it so that cartoon = 1. If you're feeling lazy, here's how to do it with sed: sed --in-place -e s/cartoon\ =\ 0/cartoon\ =\ 1/ xvid4.cfg

Once you're ready, go ahead and encode it with transcode. I must say, I like the results that I've gotten every time. And for me not to complain ... well, you know that's a step in the right direction, since I'm super picky.

Encoding live action shows is a completely different beast altogether. Essentially, there is no magic bullet, but I'll cover that in a later post. I'm cautiously optimistic about these settings though, at least for cartoons, and would tenatively recommend them for now.

I have this strange little psychological tick that turns on anytime I encode video from a professionally formatted source (basically anything I didn't make myself). I always think I can see that the A/V is out of sync. In reality, I can't tell if it is or not. But after staring at people's lips for a few hours straight, you'll probably start to believe anything.

There actually is one legitimate problem though, and I know I've encountered these issues a long time ago and decided on using what I am now (transcode over ffmpeg / mencoder, an "fps" flag in the database, etc.) but for the life of me I can't remember why.

The problem is that DVDs come in three different framerates: 29.97, 23.97 and a mix between the two. To handle the matter, I've got a trinary flag in the database. The default value will let transcode decide how to process the framerate. In most cases, that works just fine for me. Most DVDs are either 29.97 fps, or start off for a second or two in 23.97 and then switch to 29.97 for good.

Then there are my good friends that just switch back and forth between scenes from one framerate to the other -- variable framerate. I know it sounds crazy, so here's an example:

Starting playback...
VDec: vo config request - 720 x 480 (preferred colorspace: Planar YV12)
VDec: using Planar YV12 as output csp (no 0)
Movie-Aspect is 1.33:1 - prescaling to correct movie aspect.
VO: [xv] 720x480 => 720x540 Planar YV12
A:  44.8 V:  44.8 A-V: -0.007 ct:  0.076  16/ 13 ??% ??% ??,?% 0 0
demux_mpg: 24000/1001fps progressive NTSC content detected, switching framerate.
A:  62.3 V:  62.3 A-V: -0.002 ct:  0.296 203/200 11%  0%  2.0% 0 0
demux_mpg: 30000/1001fps NTSC content detected, switching framerate.
Warning! FPS changed 23.976 -> 29.970  (-5.994005) [4]    2.0% 0 0
A:  62.7 V:  62.7 A-V:  0.024 ct:  0.325 215/210 11%  0%  2.0% 0 0
demux_mpg: 24000/1001fps progressive NTSC content detected, switching framerate.
A:  63.9 V:  63.9 A-V:  0.009 ct:  0.386 246/240 11%  0%  2.0% 0 0
demux_mpg: 30000/1001fps NTSC content detected, switching framerate.
Warning! FPS changed 23.976 -> 29.970  (-5.994005) [4]    2.0% 0 0
A:  64.4 V:  64.4 A-V:  0.038 ct:  0.421 261/253 10%  0%  1.9% 0 0
demux_mpg: 24000/1001fps progressive NTSC content detected, switching framerate.
A:  74.3 V:  74.3 A-V:  0.013 ct:  0.182 672/664 11%  1%  4.5% 0 0
Exiting... (Quit)

Now, I can't remember what I used to do about this problem. I think I used to force transcode to save the new file to 23.97 frames per second. I'd rather drop some frames on the 29.97 than try to stretch out the other ones, after all. Testing this stuff isn't easy though. You have to stare at a lot of lips and try and convince yourself that you really can see a difference of a few milliseconds.

To solve my little problem (or to put my mind to rest, whichever's easiest), I have three ideas:

1. Tell mplayer to fix the a/v sync on playback

MPlayer has an "autosync" feature (see the man page) that you can use to tweak how often it looks to see if the audio is delayed. I tried flipping it on on one pass, and I thought I could see an improvement (although, it was a cartoon so who knows if the lip sync was ever correct in the first place), and then I tried it while watching a live-action show with it turned on, and it seemed like the sync was off. So, who knows. One thing I've learned from all of this is that it's not likely there's going to be a setting that will work for every scenario.

2. Use some funky transcode flags

In transcode's docs directory, theres a framerate.txt file that talks just about adjusting framerates. It even mentions the exact problem I'm running into ... "Some video sources are not of constant framerate. This is mostly true for Anime and some DVD Releases of TV series." Doh!

They then have this huge example using flags I've never even read up on before, so that seems kinda scary. There is one, though, that I noticed in the man page, --hard-fps, which will "disable smooth dropping (for variable fps clips)" and is off by default. That might be my ticket, too.

3. Don't do anything

There's always the exciting option of assuming that the developers of these multimedia applications are just a twinge more genius than I am, and that by using the defaults on everything, that things might just turn out fine. I'm not sure where the fun is in that, though.

When I'm encoding more than a thousand episodes of TV shows ranging from 6 to 60 minutes in length, I have a tendency to be a little picky. Eventually though, I'm sure I'll just settle in and start watching the shows and not even realize that it's transcoded from the original and that the A/V is off by 3/1000th of a second. And hopefully I'll forget all about their lips, too.

I spent all weekend repairing my MythTV box (which, btw, I have a lot of issues with the project as a whole ... but that's another post), and I finally got it to where I want it -- working partially. Anyway, I just started playing around with mencoder to see how hard it would be to convert these files to something other than mythtv's special nuv format (must ... contain ... rant...), and it actually turned out to be spectacularly simple. All I really have to do is just re-encode them so the AVI index is rebuilt.

The really cool part is coming up next though. I wanted to strip out the commercials myself, and at first I started thinking of how I could do that with mythtv's commercial flagging. After all, it does just store the cutlist in the database, right? For the record: yes, it does, its in the recordedmarkup table, which took me a while to find out. Well, after my average research time of 2 seconds, I couldn't find an easy intuitive way to use that to tell mencoder when to cut the video into pieces. I had heard of avidemux before, but last time I tried it (back when I was using Mandrake ... eek. Nothing wrong with MDK, that was just a loooong time ago), it only crashed and burned horribly. I figured this was a good a time as any to give it a swirl.

And man, it works great. There's this great feature it has "skip to next black frame" which pretty much finds the commercials for you. Then just set marker A and B at the start and stop of each break, cut it, and save the file and you're done! All you need is about 15 minutes each to re-encode each NUV and then maybe 2 minutes to manually cut them, but you've got a great perfect looking pristine TV episode. Sweet.

I should mention that I already know about nuvexport, but I've never had much luck with that either. If anyone's interested, here's a raw howto I actually got this far:

1. Set mythtv to keep original recordings (they'll be saved as foo.nuv.old) or just don't transcode at all
2. Reencode from an MPEG4 NuppelVideo to an MPEG4 AVI, video only: mencoder foo.nuv.old -ovc lavc -oac copy
3. Run avidemux2 on the AVI, cut out commercials and reencode the audio to mp3 (usually PCM audio works too, but sometimes it gets scratchy).

The good thing though is that it's down to mostly fixing bugs and fixing code far more than it is adding features and just plain getting it to work. That's a good milestone, I'd say.

One thing I like a lot about it is I'm just plain learning to write some better code. This is my first really interactive php shell script, and writing the backend, I keep finding better ways to clean it up and make the code tighter.

Just running an interactive script by itself is very cool. I was really surprised when I found out how easy it was.

The hardest part in all of this has been the transcode arguments. Actually, the first 85% or so of it was really simple, using dvd::rip as a reference and then reading up on the excellent transcode man pages. Transcode comes with some really good tools, and I still prefer it over MEncoder even though it (mplayer) would make things generally *so* much simpler. For instance, I already prefer tcprobe to midentify as getting me some really good valid data. That's not a fair comparison, I know, since they both return different types of data. I'll probably end up using them both. Right now I rely heavily on them both. Mplayer to rip the VOB, transcode to encode it to an XviD, and in the future tcprobe to make sure I'm doing things right.

Good stuff.