The bug has once again bitten me to get all my multimedia on demand, even though it is an enormous time-consuming and costly process. This is probably round three or four for me trying to do this. This time around, things are a little different. Instead of ripping and saving everything from my entire collection, I’m going to try and just archive the stuff that I consider a novelty, or any cartoons that are short in nature. I’m doing it as a bit of a test run for a couple of reasons — to see how much time it will take, to work out a few bugs in the system, and to find out if I’ll actually bother watching anything.
So, right now, I’m working on both Looney Tunes and all my Walt Disney Treasures DVDs. Even that stuff alone is going to take up a massive amount of space, since I don’t re-encode the video (read archives for why, long story) or audio. Right now, Ive only got a little over 500 gigs to spare, with close to another terabyte that I can slap in if I think this is going to work. In the end, I’m going to have to buy either 750g or 1tb harddrives to hold everything, but I’m trying not to think about that just yet.
Anyway, onto the joy of ripping, encoding and muxing. I’ve dregged out my old dvd ripping project, bend, and dusted it off and started poking at it again. I can’t believe how much work I put into this thing to start with, it works *amazingly* well if I do say so myself. It makes ripping and archiving TV shows on DVD incredibly simple. I only had to add on a few small features as issues cropped up with my new goals.
First of all, I had always had it hard-coded to re-encode the video to MPEG4. In the database, I had an option to leave it alone as MPEG2, but the code wasn’t using that. So, I added that and set the default to not re-encode. The next problem I ran into was that on the Looney Tunes discs, I found some tracks that multiple audio tracks: both different languages and commentaries. This would cause problems because when you rip the VOB from the DVD to the harddrive, I use mplayer along with -dumpstream -dumpfile, and it snags all the audio tracks. Using that method, there’s no way to single out just one audio track (to be more specific, passing -aid 128 or -alang en doesn’t do anything). The only way to do that is to actually re-encode the video (or audio) and specifically select the audio track, which I don’t want to do, since it causes A/V sync issues on cartoons because of variable framerates.
To solve the problem, I poked around at Matroska and the mkvmerge tool a little bit. I had remembered seeing once that you can tell it which audio tracks to use when creating a new video file. Finding the track ids is simple, just run “mkvmerge -i” on the MPEG-2 .vob file. Here’s a sample output of one with multiple tracks:
steve@flick ~/media/dvds/Looney_Tunes $ mkvmerge -i season_1_disc_4_track_15.vob
File ‘season_1_disc_4_track_15.vob’: container: MPEG 2 program stream (PS)
Track ID 0: video (MPEG-2)
Track ID 1: audio (AC3)
Track ID 2: audio (AC3)
Track ID 3: audio (AC3)
Now, you’ll see there’s only one video track, so that’s not a problem … matroska will just use that one by default. But since there are three audio tracks to choose from, it also uses the default, being the first, which is always another language or a commentary track (so far). In fact, my experience has shown that in every case where there are multiple audio tracks, it’s always the last one that I want. There have been some corner cases on a few individual episodes where that rule of thumb doesn’t apply, but it’s been something like one for each disc, which would be something like one out of every 15 to 20 episodes. As a matter of QA, I have to individually go back and play each muxed video to make sure the default of using the last audio track was what I wanted. Slightly time consuming, but not really a big deal, and I don’t know of any other way to get more details on which audio tracks are which. It just occurred to me that I should probably look into using mplayer with some verbosity flags. Oh well.
Another problem I ran into was that mkvmerge itself is picky as to the order of passing arguments. If I didn’t do things in the right order, then all the audio tracks would be selected. Kind of annoying. I vaguely recall reading something about that in the man page before, but I didn’t bother looking through there again. Instead, I figured up mmg, the wxwindows GUI to mkvmerge and watched how it did it. For the record, here’s a sample of a correct execution:
$ mkvmerge -o foo.mkv -a 2 season_1_disc_1_track_3.vob
In that example, the -a argument is for the audio track, followed by the number.
I also changed the database around, adding a new table called ‘episode_tracks’. When I rip the DVD for the first time, bend will check the number of audio tracks, and store them in the database. On the web frontend, where you edit the track titles, I also added an option to select the audio tracks on a per-episode basis. You can also set the default track # for the entire disc, if you want, to change all the episodes at once.
I should note that in all the DVDs I’ve seen, Looney Tunes seems to be an exception to many rules. Most DVD series aren’t even going to have multiple language tracks or commentaries. And even if they did, I don’t think they would have so many. I’m just glad that I happened to use this one first, so I could get the workarounds coded.
Something else I changed in the code, when muxing into Matroska, is I now pass the –title call along with the series and episode titles, so that they get written into the container as well. I’m not sure if I can get MPlayer to display that, but it might be kind of nice sometime if I wanted to recall what the title was, and just hit a button on my remote — I wouldn’t have to do something funky like check the filename, instead it would just be all right there.
Here was something else that was funky. My DVD drive in my desktop box was having issues ripping discs with MPlayer. I was getting some IDE error messages like this:
Buffer I/O error on device hdb, logical block 20278
Buffer I/O error on device hdb, logical block 20279
hdb: media error (bad sector): status=0x51 { DriveReady SeekComplete Error }
hdb: media error (bad sector): error=0x30 { LastFailedSense=0x03 }
ide: failed opcode was: unknown
ATAPI device hdb:
Error: Medium error — (Sense key=0x03)
(reserved error code) — (asc=0x11, ascq=0x05)
The failed “Read 10” packet command was:
“28 00 00 00 4f 36 00 00 02 00 00 00 00 00 00 00 ”
end_request: I/O error, dev hdb, sector 81112
Buffer I/O error on device hdb, logical block 20278
First, I looked at my kernel to make sure my config was okay, and I flipped on a few more generic flags to see if that would help, then rebooted into the new kernel. It didn’t do anything. The drive would read for a few megabytes, then freeze on some titles. It seems like also it wouldn’t matter which track or disc it was on … it would just poop out after a certain amount of ripping.
I thought about heading down to PC Club and dropping $50 on an SATA DVD burner, thinking that having a drive that didn’t use IDE in the first place might be the solution. I can’t afford to buy more hardware right now though, so I kept poking around. At the same time, on the Walt Disney Treasures DVDs I was ripping (Tomorrow Land rocks, by the way), a lot of them have introductions by Leonard Maltin. I’ve got nothing against the guy, but I certainly don’t want his or anyone else’s face on my screen telling me what I’m about to see and then showing off the coolest scenes right before I’m going to watch them. So, I had to add another option to dvd-bend to let me say which chapter I wanted to start the ripping at. Incidentally, once the code was finished, I tried it on my desktop … and my IDE issues went away. As a result, I went ahead and hard-coded it as well to default to add ” -chapter 1 ” to the mplayer dumpstream command (if none was specified) so that my drive would always work. So far, it’s done great — no more ripping issues.
Once again, just for the record, here’s what I used as a workaround to my faulty drive to properly rip a DVD track:
$ mplayer dvd://3 -dumpstream -dumpfile track_3.vob -chapter 1
That’s pretty much it so far, as far as changes. That alone didn’t take too much time at all, just a few hours. I’m still working on setting up subversion on my live server so I can share the code if anyone wants to look at it. In the meantime, here’s the output of bend as I add a new series to the database, and rip it, so you can see how it works