wonkablog

A bug in MythVideo inspired me to work on fixing mplayer-resume tonight, so that it can properly handle movies with filenames.  I don’t know why I didn’t think about this before, but it’s simple if the file is properly escaped or quoted.  And so, mplayer-resume v1.5 is released, with support for spaces in filenames, finally, and also one other cool little thing: it works with playlists now, to a degree.

The playlists thing is kind of hard to explain, and it’d be easier to point you to the documentation that I’ve already written.  Instead, I’ll just describe what it is I’m going to use it for.

One thing I’ve been wanting to add to my MythVideo setup is some playlists so that I can randomly play something.  I have a lot of cartoons and videos and movies, and sometimes I don’t feel like picking something myself — one of the nice things about TV in general is you are genuinely surprised when you’re channel surfing and something cool just happens to crop up.  That’s kinda what I like, and what I wanted to do.  But, I wanted to take it a step further.  If I started playing $random_episode, then if I quit, I want to be able to resume playback of that same show.  Up until now, mplayer-resume wouldn’t work that way, since if you’re randomly picking something from a playlist file, there’s no real way to seek back to the same one.

That’s fixed now.  The script will read the filename of the movie you are playing when you exit (once you setup .lircrc correctly), and checks to see if that’s the same file you started playing.  So if I play random.pls and it plays Tarzan.mkv, and I exit, then when I go back to watch Tarzan, it will resume in the same place.  Basically, it saves the file position for Tarzan instead of the playlist file.  Pretty cool.

So, there you go.  I’ll put it in portage shortly as well.  Enjoy.

Now that I’ve got my mythbox all setup properly (I screwed up LVM2 and lost all my data, twice, trying to remove harddrives … meh), I’m back to ripping my DVDs again.  This time I’m using dvd2mkv, my custom little script I wrote, to do all the heavy lifting for my movies.  But, there’s one thing I didn’t really put in there the first time around, and that’s support for alternate audio tracks.

Originally I wrote it simply to check automatically for the highest number of tracks and best audio format.  As a general rule, that chooses the first channel that’s in English with 6 channels, which is always (in every case I’ve seen so far), Dolby Digital.  If there is a DTS track, it’s always the second or third track behind it, but never gets selected automatically since it’s not the first one on the list.  I can, however, select it if I run the program interactively.  Not really ideal, of course, but it’ll have to work for now.

My real question though is, why aren’t there more movies with DTS audio tracks to them?  Back in the day when I was working at a movie theater, one of my managers would swear up and down that DTS was better quality than Dolby.  He would even make the projectionist screen the movies for him in the DTS  theaters if the movie was equipped for it.  He was quite the audio and videophile so I took his word for it.  Now, though, you hardly see it anywhere.  The only DVDs I’ve seen them on are some Paramount and Fox titles, and even then it’s only the newer ones that have it.

What’s also really interesting, and I kind of assumed this, is that SDDS, Sony’s 8 channel format, is completely missing from a home theater setup.  Good ol Sony, going off and making their own standard yet again.

Anyway, when I listen to DVDs with both tracks, I really can’t tell a difference myself.  My receiver supports both DTS and Dolby, so I figure … why not, I’ll rip em anyway and see if it really is any nicer.

I was lucky enough to catch Martin’s post on Planet Larry today about undvd, a script he’s working on — very cool, Martin, I love it.  I’m actually working on one of my own, or have been for a while, and I’ve been toying with the idea of cleaning it up and releasing it for public consumption.

I actually started writing my own because, as Martin says, all the other ones out there are too complex, and I prefer a simple command-line app that does exactly what I need it to do.  I call mine dvd2mkv, since I extract everything I want (video, audio, subtitles, chapters) and dump it straight into a Matroska file.

In fact, the script is already done, and it works great.  It automates the entire process — selects the longest video track, the widescreen one if there if full frame is on there as well, grabs the English (or preferred language track) with the highest number of channels (Dolby or DTS), plus the English subtitles if they exist, and finally the chapters.  All I have to do is put in the title of the movie, and even then if the disc ID is the title, then you can skip past that as well.

In fact, here’s the output of a movie I just ripped a few minutes ago, Return to Me.  Great movie, btw.

steve@charlie ~/dvd $ dvd2mkv
[DVD] Disc title: RETURN_TO_ME
Enter a movie title: [Return To Me]
[Video] Track number: 5
[Video] Aspect ratio: 16/9
[Video] Length: 115.79
[Audio] Track: 128
[Audio] Format: Dolby Digital
[Audio] Channels: 6
[DVD] Subtitles: None
[DVD] Ripping MPEG-2
[DVD] Ripping chapters
[MKV] Creating Matroska file

And that’s it!  Pretty simple. :)  Mine doesn’t have support (right now) for re-encoding the movies, since I’ve already gone into quite a bit of length on why I don’t like doing that, but it would be simple to add.  In fact, my shortcut method would just be to have the user setup mencoder profiles in the config file and call those directly.

Anyway, I like the idea of cleaning it up and throwing it out there, so I’ll probably be doing that fairly soon here.  If it works good enough for me (picky as I am), it’s sure to help out someone. 

The bug has once again bitten me to get all my multimedia on demand, even though it is an enormous time-consuming and costly process. This is probably round three or four for me trying to do this. This time around, things are a little different. Instead of ripping and saving everything from my entire collection, I’m going to try and just archive the stuff that I consider a novelty, or any cartoons that are short in nature. I’m doing it as a bit of a test run for a couple of reasons — to see how much time it will take, to work out a few bugs in the system, and to find out if I’ll actually bother watching anything.

So, right now, I’m working on both Looney Tunes and all my Walt Disney Treasures DVDs. Even that stuff alone is going to take up a massive amount of space, since I don’t re-encode the video (read archives for why, long story) or audio. Right now, Ive only got a little over 500 gigs to spare, with close to another terabyte that I can slap in if I think this is going to work. In the end, I’m going to have to buy either 750g or 1tb harddrives to hold everything, but I’m trying not to think about that just yet.

Anyway, onto the joy of ripping, encoding and muxing. I’ve dregged out my old dvd ripping project, bend, and dusted it off and started poking at it again. I can’t believe how much work I put into this thing to start with, it works *amazingly* well if I do say so myself. It makes ripping and archiving TV shows on DVD incredibly simple. I only had to add on a few small features as issues cropped up with my new goals.

First of all, I had always had it hard-coded to re-encode the video to MPEG4. In the database, I had an option to leave it alone as MPEG2, but the code wasn’t using that. So, I added that and set the default to not re-encode. The next problem I ran into was that on the Looney Tunes discs, I found some tracks that multiple audio tracks: both different languages and commentaries. This would cause problems because when you rip the VOB from the DVD to the harddrive, I use mplayer along with -dumpstream -dumpfile, and it snags all the audio tracks. Using that method, there’s no way to single out just one audio track (to be more specific, passing -aid 128 or -alang en doesn’t do anything). The only way to do that is to actually re-encode the video (or audio) and specifically select the audio track, which I don’t want to do, since it causes A/V sync issues on cartoons because of variable framerates.

To solve the problem, I poked around at Matroska and the mkvmerge tool a little bit. I had remembered seeing once that you can tell it which audio tracks to use when creating a new video file. Finding the track ids is simple, just run “mkvmerge -i” on the MPEG-2 .vob file. Here’s a sample output of one with multiple tracks:

steve@flick ~/media/dvds/Looney_Tunes $ mkvmerge -i season_1_disc_4_track_15.vob

File ’season_1_disc_4_track_15.vob’: container: MPEG 2 program stream (PS)
Track ID 0: video (MPEG-2)
Track ID 1: audio (AC3)
Track ID 2: audio (AC3)
Track ID 3: audio (AC3)

Now, you’ll see there’s only one video track, so that’s not a problem … matroska will just use that one by default. But since there are three audio tracks to choose from, it also uses the default, being the first, which is always another language or a commentary track (so far). In fact, my experience has shown that in every case where there are multiple audio tracks, it’s always the last one that I want. There have been some corner cases on a few individual episodes where that rule of thumb doesn’t apply, but it’s been something like one for each disc, which would be something like one out of every 15 to 20 episodes. As a matter of QA, I have to individually go back and play each muxed video to make sure the default of using the last audio track was what I wanted. Slightly time consuming, but not really a big deal, and I don’t know of any other way to get more details on which audio tracks are which. It just occurred to me that I should probably look into using mplayer with some verbosity flags. Oh well.

Another problem I ran into was that mkvmerge itself is picky as to the order of passing arguments. If I didn’t do things in the right order, then all the audio tracks would be selected. Kind of annoying. I vaguely recall reading something about that in the man page before, but I didn’t bother looking through there again. Instead, I figured up mmg, the wxwindows GUI to mkvmerge and watched how it did it. For the record, here’s a sample of a correct execution:

$ mkvmerge -o foo.mkv -a 2 season_1_disc_1_track_3.vob

In that example, the -a argument is for the audio track, followed by the number.

I also changed the database around, adding a new table called ‘episode_tracks’. When I rip the DVD for the first time, bend will check the number of audio tracks, and store them in the database. On the web frontend, where you edit the track titles, I also added an option to select the audio tracks on a per-episode basis. You can also set the default track # for the entire disc, if you want, to change all the episodes at once.

I should note that in all the DVDs I’ve seen, Looney Tunes seems to be an exception to many rules. Most DVD series aren’t even going to have multiple language tracks or commentaries. And even if they did, I don’t think they would have so many. I’m just glad that I happened to use this one first, so I could get the workarounds coded.

Something else I changed in the code, when muxing into Matroska, is I now pass the –title call along with the series and episode titles, so that they get written into the container as well. I’m not sure if I can get MPlayer to display that, but it might be kind of nice sometime if I wanted to recall what the title was, and just hit a button on my remote — I wouldn’t have to do something funky like check the filename, instead it would just be all right there.

Here was something else that was funky. My DVD drive in my desktop box was having issues ripping discs with MPlayer. I was getting some IDE error messages like this:

Buffer I/O error on device hdb, logical block 20278
Buffer I/O error on device hdb, logical block 20279
hdb: media error (bad sector): status=0×51 { DriveReady SeekComplete Error }
hdb: media error (bad sector): error=0×30 { LastFailedSense=0×03 }
ide: failed opcode was: unknown
ATAPI device hdb:
Error: Medium error — (Sense key=0×03)
(reserved error code) — (asc=0×11, ascq=0×05)
The failed “Read 10″ packet command was:
“28 00 00 00 4f 36 00 00 02 00 00 00 00 00 00 00 ”
end_request: I/O error, dev hdb, sector 81112
Buffer I/O error on device hdb, logical block 20278

First, I looked at my kernel to make sure my config was okay, and I flipped on a few more generic flags to see if that would help, then rebooted into the new kernel. It didn’t do anything. The drive would read for a few megabytes, then freeze on some titles. It seems like also it wouldn’t matter which track or disc it was on … it would just poop out after a certain amount of ripping.

I thought about heading down to PC Club and dropping $50 on an SATA DVD burner, thinking that having a drive that didn’t use IDE in the first place might be the solution. I can’t afford to buy more hardware right now though, so I kept poking around. At the same time, on the Walt Disney Treasures DVDs I was ripping (Tomorrow Land rocks, by the way), a lot of them have introductions by Leonard Maltin. I’ve got nothing against the guy, but I certainly don’t want his or anyone else’s face on my screen telling me what I’m about to see and then showing off the coolest scenes right before I’m going to watch them. So, I had to add another option to dvd-bend to let me say which chapter I wanted to start the ripping at. Incidentally, once the code was finished, I tried it on my desktop … and my IDE issues went away. As a result, I went ahead and hard-coded it as well to default to add ” -chapter 1 ” to the mplayer dumpstream command (if none was specified) so that my drive would always work. So far, it’s done great — no more ripping issues.

Once again, just for the record, here’s what I used as a workaround to my faulty drive to properly rip a DVD track:

$ mplayer dvd://3 -dumpstream -dumpfile track_3.vob -chapter 1

That’s pretty much it so far, as far as changes. That alone didn’t take too much time at all, just a few hours. I’m still working on setting up subversion on my live server so I can share the code if anyone wants to look at it. In the meantime, here’s the output of bend as I add a new series to the database, and rip it, so you can see how it works

It’s far from glamorous, but I’ve setup a directory where you can download the latest version of mplayer-resume, in all of its 2.7k glory! Sweet.

I also submitted the “project” to freshmeat, so hopefully that will give it a little added exposure. I searched high and low for a solution, so I know how hard it can be to find something like this.

Downloads are available here: http://spaceparanoids.org/downloads/mplayer-resume/

Oh, and for the record, the documentation is included as comments in the code.

Uncle Scrooge always had a great saying that he said someone taught him — “Work smarter, not harder.” Gosh, I still remember that Duck Tales episode from fifteen years ago or so. Now that’s crazy.

That’s what I’m doing though — I’ve gotten my encoding settings finalized, I’ve scrapped all my old encodings, and now I’m ripping everything in a nice organized little manner. Right now my floor is a huge mess, with DVDs literally everywhere, as I’ve got them stacked in piles and divvied out to three computers to rip and encode.

The other day, I quietly changed the backend class and switched the main encoder from transcode to mencoder. The main reason is that mencoder can handle variable framerates just fine, of which more than 90% of these DVDs are. That, combined with pretty much the same filters and settings I was using on transcode, the end results looks really, really nice. Every sample I’ve watched so far there has been no stutter, no weird skips, no odd artifacts. In fact, I’ve though to myself, “wow, this looks just like the DVD.” They are that smooth, so I’m very happy.

For the record, here’s my encoder settings I’m using on most all of my TV DVDs:

mencoder foo.vob -o bar.avi -ovc xvid -xvidencopts bitrate=2200 -oac copy -vf pullup,softskip -ofps 24000/1001

Pretty simple, huh? The only drawback is that I like the new settings so much that I scrapped every encoding I’ve done up to now except for a few shows. That was a lot of time down the drain, but I have a new system now which has helped me quite a bit to do this much quicker.

One thing I did is I divvied up all the discs between my three amd64s, and there are stacks all over my living room. This way I don’t have to hunt down a box or remember what’s next. When the tray pops out, I just stick the next one in, and start that one up. Right now I’m ripping about three discs every 15 minutes.

The other idea I had that I like a lot is I took a sample track from each series, and dumped it to a folder where they won’t be ovewritten. With those, I do all my encoding testing, and that’s how I found out those mencoder settings above really do work on everything I’ve thrown at it so far. Especially the cartoons, those look gorgeous.

The encoding is gonna take some time, that’s for sure. I just counted, and theres 18 more DVDs to rip, and *then* I’ll be halfway done with ripping (I haven’t even started on the live-action shows yet). My queue manager says my estimated encoding time is currently 142.34 hours. Yeeks.

I must be on a roll. I just got my other big problem with mplayer solved: resuming playback.

It turns out that MPlayer has a lot more slave / LIRC events than I knew about, and one of them is ‘get_time_pos’ which prints out the current playback position to stdout. With that, I was able to write a php shell script to save that value to a file, and read it on playback and seek to that position. Pretty cool stuff.

Here’s the script, I’ve added it to my SVN repository for bend: http://spaceparanoids.org/trac/bend/wiki/mplayer-resume

Right now the only problem it has is that it’s PHP5 dependent, and only then because I use file_put_contents() and file_get_contents(). I haven’t been using PHP4 for a long time, so I was too lazy to make it backwards compatible, but I’ll fix it later.

I’ve documented how to use it in the script, so everything is already covered there. I’ve been testing it and it works great. Now I can seek chapters on my video files and leave and come back anytime I want. I’m pretty excited about that.

One of my biggest beefs about media players (if you haven’t noticed yet) is the lack of support for my favorite media wrapper, Matroska. If you don’t know what that is, just ask me next time I’m around, and watch me shift into extreme-fanboyism explanation mode. Simply put, though, its similar in concept to AVI, but far superior in features.

My favorite part of it is you can have chapters in any Matroska file just like you can in an DVD. The reason I want something like that so badly is for two reasons. One is that once I have a feature in one format, I’ll go stir crazy until I have it another. The second one is that the chances of me sitting through one movie in an entire sitting are pretty low. My threshold is at about an hour, max.

So, if I leave and come back often … I want to be able to *quickly* get to where I left off. Either by resume playback completely (I’m working on that one) or skipping to the next chapter. No, I don’t want to skip ahead 5 seconds / minutes / hours. I want to skip to the next chapter.

My whole point of this is that it finally occurred to me to actually ask for this feature on the MPlayer mailing list instead of constantly complaining that it didn’t have it. Well, much to my surprise, someone much smarter than me posted a patch just a few hours later, and to make a long story short … it works great.

I’m hoping it gets included into the main program soon. That would be really cool. I’m still just really stoked that my most wished-for mplayer feature got added. Not to mention that it was really easy just to ask for it. I really should try out this strange thing called “communication” some more.

I’ve run out of interesting titles for my ‘bend’ updates, so I figure the changeset # is just as non-entertaining.

I hit a critical point today in the coding — I’m finally at a point (again) where I’d use the current snapshot to go ahead and rip / encode my DVDs. Yay! Basically all I did was finish up the encoding part so it outputs to the correct filename: 104._High_Diving_Hare.mkv. That would be season one, episode 4 of Looney Tunes. And that is also one of the funniest cartoons ever. I love Bugs Bunny.

Now all I need is a media player for Linux that can resume playback for my movies. Nevermind that my DVD players have had that feature for at least the past 7 years. I’m not bitter about it.

Actually, I did find something, but it’s not working right. And, it’s written in perl, so I’m up the river on trying to fix it myself. Theres a project called mplayer-tools that has a bunch of little wrapper scripts around mplayer. I can get it working half the time, and the other half it just skips back a few seconds on resuming. Very odd. If I could figure out how to write a shell script to capture the output of mplayer, then I could write my own, but I can’t even get that far. Ah well, on with searching / tweaking.

I only worked on ‘bend’ a little bit today. The main new feature is that ripping works 100% now, and I’ve cleaned up a lot of old bugs. Encoding is the last thing left, and then I can finish up dvd2mkv.

One interesting little feature I added is that now you can rip the same disc in one or more session at the same time. Something I don’t recommend in the least, since it increases the disc readtime quite a bit. However, sooner or later, I’ll have it so you can pass bend –rip in one shell, and bend –rip –dvd_device /dev/dvd1 in the other. That can come in handy if you have more than one DVD drive and want to rip more stuff at once.