Category Archives: Multimedia

reading, ejecting, ripping and polling dvd devices in linux; also, notes on my dvd library

It has been a looong time since I both posted in my blog and worked on my custom DVD ripper scripts.  Apparently the last time I worked on the code was last June, and even then I didn’t make many updates.

I’ve been spurred onto building up my DVD library again by a couple of things. First, I realized that my Blu-ray player has  support for Matroska videos with VobSub and SRT subtitle support!  I was not expecting that.  In fact, it’s way better than what my PS3 can playback, which is … depressing.

I put away my HTPC about two years ago, when I was living in my previous apartment.  I moved into a place that was probably about 550 square feet.  Pretty tiny, and I liked it, but no room for a fantabulous multimedia setup.  So I sacked it for a while and was okay with that.  The fact is I actually spent more time getting it up and running and customizing it than using it.  Which is weird.  Actually I spent even *more* time ripping the DVDs and then not watching them.  But that’s okay.  It wasn’t until recently that I found a setup I think I’d like even more.

For now, I’m preferring having *less* hardware, and so just sticking a small 8GB USB thumb drive in my Blu-ray player with a smattering of samples of shows suits me just fine.  It’s no amazing thundershow of hardware and multimedia, but it *does* get me actually watching the content, so there.  I imagine if (and when) I have a house where I can properly get loud without upsetting neighbors, that’s when I’ll whip the big speakers back out and deck it out properly.  Some day. :)

In the meantime, today, I’ve been working on my DVD scripts.  I call it dart for “dvd archiving tool.”  It’s a complex set of scripts that I’ve been putting together for years, and it is highly customized for my own setup, with a CLI tool to read and access DVDs, then archive them in a database.  I also have a web frontend that I use to tag tracks, titles, episodes, etc. and so on.  If it wasn’t so unwieldly I’d throw the source out there, but the thought of having to explain to *anyone* how to get it up and running makes my head hurt.  So, if you want a good DVD ripper, here’s my advice: use Handbrake.

One problem I was trying to solve tonight was checking for these three statuses of my DVD drive: is the tray open, is the tray closed, is there media in the tray (while closed).  I have to use different tools for each one, but the problem that I always run into is this: it’s impossible (as far as I have been able to discover) to know when a DVD tray is both closed and ready to access.

The problem is that you can run eject just fine to close the tray, but once the command exits successfully, that doesn’t mean the drive can be accessed.  That is, running “eject -t /dev/dvd” and then “mplayer dvd://” in sequence, mplayer will complain that there’s no DVD device.

What’s the solution to all this?  Well, wait four seconds after running “eject.”  That’s simple, but I still spent hours today trying to find out if there was another way to do it.  While I never did (and ended up using ‘sleep’), I did find some cool stuff for polling and reading DVD devices.


blockdev basically displays some interesting information about the block devices — in this case, /dev/dvd.  Now, for my library, one thing I have been doing lately is storing the size of the DVD in my database, so I can get an accurate number of how much HDD space I need when I want to archive the UDF or rip it.

You can use blockdev to get the amount of bytes like this:

blockdev –getsize64 /dev/dvd

Now if you want to see that in megabytes, just divide it by 1024

expr `blockdev –getsize64 /dev/dvd` / 1024


Next up is udisks, which can get information about the DVD device itself.  In this instance, I use it to see if there is media (a DVD) in the tray or not.

Running “udisks –show-info /dev/dvd” spits out all kinds of interesting information, but what I’m looking for is the “has media” field.

udisks –show-info /dev/dvd | grep “has media”
has media:                   1 (detected at Wed Jul  3 23:21:23 2013)

Now, that will say 1 *if* the both the disc tray is closed and there is something in there.  And if the DVD drive has stopped spinning enough for the command to work (again, sleep 4 seconds after closing the tray).

It will display a zero if there is no media *or* if the DVD tray is open.  Here’s a simple command to get just the number:

udisks –show-info /dev/dvd | grep “has media” | awk ‘{print $3}’


This is an old small command-line tool I’ve used in the past.  It polls the drive to see if there’s something in there or not, and if the tray is open or not.  Sounds great, right?  It should do everything I want, solving all my problems … except that it doesn’t build on my system (Ubuntu 12.10 with gcc 4.7.2).  It used to, on my older setup, which would have been about 2.5 years ago.

It’s just a small C script, just over 500 lines, you can find it here on Freshmeat.  If someone wants to patch it to get it working, I’ll personally deliver you a plate of brownies.  Mmmm, brownies.

I actually *do* have an old 64-bit binary that I built way back when, because I kept a copy of my old development filesystem.  So I have a working blob, but it kind of breaks.  So I kind of rely on it.  I can only use it if the tray is open or if the tray is closed and empty.  So the way I check if a device is empty and closed in my script is I’ll first poll it to see if it has media with udisks, and if it doesn’t, then I’ll run this one.  If I run it with a disc in there, it pukes on me, and so I have to work around it.  It’s a hack, I know, but whatever.

qpxtool and readdvd

This is the project I ran into today, and I am super, super excited about it.  The QpxTool project is full of way cool little utilities for accessing your drive settings.  Honesty, I didn’t look at the other ones, because I was so hyperfocused on ‘readdvd’.

From the man page, “readdvd reads even a corrupted dvd and writes the the result into a new image file on your harddisk.”  This is awesome, because it’s the first utility I’ve found *specifically* for creating an exact image of a DVD filesystsem (UDF).  In the past, I’ve always used dd, but now I’m onto this one.  It skips over bad sectors and gets the image squeaky clean off of there, and I could not be happier.  This one ranks up there with Handbrake in both awesomeness and must-have-ness.  I should add that it’s also in Ubuntu’s default repos, so have fun.

Just run “readdvd -o movie.iso /dev/dvd”.  Pretty simple.

That’s pretty much it for now.  There are other great tools out there: lsdvd also ranks in the “must have” category.  I couldn’t do anything without it.

I mentioned dd earlier, and I actually use pv with it to give me a nice progress bar (also in Ubuntu repos).  It works just fine, I’ve been using this approach for years.

pv -ptre /dev/dvd | dd of=movie.iso

One more thing I wanted to mention.  Sometimes, some errors get thrown to the syslog because either an application or the DVD drive itself is being fussy.  I haven’t quite narrowed down which it is, but I’m betting it’s the firmware on the DVD drive complaining since some brands (Memorex) complain, and some do not (BenQ).  By far, the best-quality DVD drive I’ve had to date was actually a Sony BD-ROM drive.  At least, I think it was Sony.  Here’s some of the errors I get sometimes:

Jul 3 18:37:04 localhost kernel: [11955.073772] sr 0:0:0:0: [sr0]
Jul 3 18:37:04 localhost kernel: [11955.073784] sr 0:0:0:0: [sr0]
Jul 3 18:37:04 localhost kernel: [11955.073795] sr 0:0:0:0: [sr0]
Jul 3 18:37:04 localhost kernel: [11955.073808] sr 0:0:0:0: [sr0] CDB:
Jul 3 18:37:04 localhost kernel: [11955.073826] end_request: I/O error, dev sr0, sector 4096
Jul 3 18:37:04 localhost kernel: [11955.074286] Buffer I/O error on device sr0, logical block 512

To avoid issues like this, I run a small command to just decrypt the CSS on the DVD so it can kind of clear its head a bit.  Just run mplayer on it, watching about 60 frames (or 2 seconds worth of video), but just ignore it and dump it out.  The whole point of it is to decrypt the DVD, and move on with your life.  And here you are:

mplayer dvd:// -dvd-device /dev/dvd -frames 60 -nosound -vo null -noconfig all

I don’t pretend to understand how or why that helps, but I know it does.  If someone knows why the drives are doing that, I’d love to know.

The only other app I can think of right now off the top of my head is ‘dvdxchap’, which is part of ‘ogmtools’.  I know ogmtools is old, and the OGM container isn’t popular anyway (that I’ve seen), but it’s perfect for getting the chapter information out.  Although I may use something else now (lsdvd?).  I can’t remember, and I haven’t had to mess with chapters lately.

That’s it for me.  Have fun, rip away, and watch some cool Super Friends DVDs.  There are a LOT of seasons out there.  It’s great. :)

Leave a comment

Filed under bend / dvd2mkv, MPlayer, Multimedia

rebooting my mini-itx

It’s been a long time since I’ve worked on much anything computer-related as a hobby.  Things have changed quite a lot in the past year.  I moved to a much smaller apartment in Salt Lake, which is about a third the size of my old place.  The idea was to trim the fat and focus on going back to school, which is my major direction in life these days.  When I moved in, I didn’t have room for setting up a desktop computer anywhere, so it’s been just my netbook and me.  That suits me plenty fine, though, I wasn’t really using it that much either.  I had just upgraded to a six-core so I could rip DVDs much faster, and now it was sitting headless wherever I could find room, and even then, only used occasionally.

It’s not just at home that things have been changing.  At work I got to make the transition from programmer to full-time sysadmin, and I’m absolutely loving it.  I knew I was getting tired of coding, and I had always enjoyed just taking care of servers, and now I get to do that all day long. When I initially started as a sysadmin, I didn’t think our small company would have enough work for me to do after a few months.  In actuality, I’m kept busy all the time.  The part I like the most is that part of my job is doing research, how to do things better, more efficiently, anything to make the workload easier.  It’s fun.

On top of all that, my school attendance is starting to ramp up more, and I’ve been consistently drifting to adding more classes to my workload.  All this stuff has basically booted Linux out of my life as a hobby, and so now I need things to “just work” without hassle, so I leave my installations alone.

One thing I’d been neglecting a little bit was my entire HTPC setup.  I hadn’t been using it much lately just because I would mostly stream some Netflix (yay, Doctor Who!).  My setup has been a beast though, normally running for months on end without the slightest hiccup.  What started to happen though is that I would come back to using it, switching my HDMI input over, and the box would be powered off for some reason.  Most of the time, I would either power it back on and go on with life or just ignore it.  Until one day it wouldn’t power on at all, and I just shrugged it off and determined to look at it later.

Well, later turned out to be finals week, when my brain has been working overtime, and I seriously needed a hobby.  I pulled out my main frontend and started looking at it to see what was going on.  It was plugged in properly and everything looked legit, but when I hit the power, the CPU fan would start up for a second and then everything would stop.  After fiddling with it for a bit, I started to notice that something was smelling burnt.  Once that happened, I abandoned my diagnosis.  Even if I did manage to get it working, I didn’t want it to catch everything on fire.

At the same time, my external USB drive enclosure died on me.  So even if I could have gotten it working, I still wouldn’t have had a way to watch my shows.  Them giving out on me hasn’t bothered me in the least — the entire setup has been running flawlessly for years, and I’d managed to get a lot of mileage out of them.

Now I had to decide what I was going to do.  I have a lot of hardware, but in pieces.  I have four mini-ITX boards altogether, two of them are VIA C7 chipsets, and the other two are Zotac boards both running low-powered Celeron CPUs (around 35W if I remember correctly).  The power supplies for the VIA boards use 20-pin connectors and only run at about 80W, and aren’t enough to handle the Zotac boards which use 24-pin connectors.  So I have this mix of hardware, and nothing powerful enough to act as a frontend.

There are some great packaged systems out there now where for between $200 to $300 you can get an entire package in one go that does exactly what I’m putting together myself. I considered the idea of just starting over, but I decided that it’d be cheaper to just salvage what I could.

So this week I ordered a new USB HDD enclosure, and I also ordered a new power supply for the main Zotac board.  I found a site that sells really small power supplies for mini-ITX boards, called picoPSU.  The design eliminates a lot of the hardware that I would normally need to get all the power to my box.  I was really skeptical about them when I first heard of it, but did some looking around and it looks like it’s exactly what I need.

In the meantime, I ripped out my motherboard out of my desktop, and put both Zotac boards in there to make sure they still work, and thankfully they do.  I got the old setup pieced together using my desktop case, and fired up the old system to play around with it.

I had started to forget how much time I put into this thing.  I forgot that I had put countless hours stitching this thing together, running a custom build of Gentoo suited to run on small environments.  On top of that I made hacks to mythvideo and got those working to polish off some rough edges.  It just started to come back to me how much I’d worked on this … and how much fun it was. :)

I played around with my frontend a little bit, and fired up a few movies just to try out the surround sound.  It was awesome.  I’d forgotten how nice it was to have that huge library on demand, too.

So I’m excited now to get things up and running.  It’s been a good little while.

Leave a comment

Filed under Hardware, Multimedia

multimedia reference guide: x264

It seems a little weird to me to post something on my blog that I already posted on our blog at work, but whatever. I figured it’d get more visibility if I wrote about it, since I already cover multimedia stuff sometimes, plus I’m excited about this thing anyway. :)

At work, I get to do all kinds of stuff, and working with video is one of them. I threw together an x264 reference guide on my devspace for what the settings of each preset covers, compared to the defaults. I’ve even translated it to spanish! Vamos, che!

The thing I like about this, is that it helps me see which areas to start tweaking to get higher quality gains, and which ones to stay away from. It kind of sheds light on where the best places to start tweaking are. For instance, the settings that are changed on the ultrafast preset should never be messed with at all, if you want a good outcome. And on the flipside, the ones under the placebo preset are going to slow down the encode greatly if you start beefing them up.

Generally speaking, though, it’s a best approach to use presets set by developers. Every now and then I get the idea in my head that I can somehow make things better just by tweaking a few of the variables. That never works out too well. I always end up spending like 60 minutes to encode a 5 minute video, and then I can’t tell a difference after that. Whoopsie fail.

Next, I want to put together a similar type guide for Handbrake presets, both to compare their presets to each other, and then how to duplicate the same x264 settings using both the x264 cli encoder, and libav. The reason being that, a lot of times I really like the output that Handbrake delivers, and I want to duplicate that using other encoders, but I’m not sure how. That’s what I’m planning to target.

Leave a comment

Filed under Gentoo, Multimedia

uphpu presentation: ripping dvds

I gave a short presentation last night at uphpu on ripping and encoding DVDs under Linux. I wanted to type up a few notes on some stuff I mentioned during the meeting.

Short version: Use Handbrake (available on Linux, Mac OS X and Windows). I’ve gone through a lot of applications through the years trying to find that “perfect rip”, and Handbrake beats everything hands down. I’m a videophile who likes everything looking as nice as it can, and this is what I use regularly.

Accessing DVDs

The first things you’ll need installed on your box is libdvdread and libdvdcss. libdvdread will access the DVD filesystem, and libdvdcss will decrypt the DRM on the disc. These should both ship with your distro, or if they don’t, should be relatively easy to acquire.

I use a lot of command line tools to access DVDs, so my focus was on that. The reason for that is because I have a lot of shell scripts to rip my massive library of discs, and I store data about them in a database.

lsdvd is the first stop. A very small app that uses libdvdread to display human and script-friendly output of the details of your disc.

mplayer dvd:// and mplayer dvdnav:// plus the optional track (dvd://3) or optional -dvd-device argument to specify the location of the device or ripped ISO.

I use dd to copy the contents of the entire disc to the harddrive and then rip it from there, to avoid wear and tear on the DVD drive, and also because I can do things like simultaneously rip one disc in the background and encode another at the same time.

When accessing the drive, you’ll want to use mplayer or some other program to access the drive and decrypt the CSS so that the drive itself caches the access. Otherwise, it may lock up.

I use pv to dump the disc and give me a pretty progress bar with ripping ETA and disc read speed: pv -ptre -w 80 /dev/dvd | dd of=movie.iso

If you want to extract the chapters, you can use dvdxchap, which is part of the ogmtools package.

DVD Tracks

Every movie I’ve run into has one track specifically for the video. Using lsdvd you can find the longest track, time-length wise, and that will be the movie.

TV shows on DVD often put many episodes into one track, where one or more chapters may contain an episode. You’ll need to examine these yourself to find out where each one starts and each one ends.

Another caveat is that they are not always ordered on the track or on the DVD in the same order that shows up in the menu. The best advice when accessing tracks and titles on DVDs is to never assume anything, if you are trying to preserve order.

DVD Subtitles

Subtitles on DVDs come in two formats: VobSub and Closed Captioning. VobSub subtitles are images that are overlaid onto the picture. Closed captioning is a text format.

If you have a ripped VOB (DVD MPEG-2 video), you can extract the subtitles using a program called ccextractor. It will save the subtitles to SRT, a subtitle format that can be muxed into your final video.

Encoding DVDs

Handbrake is a very “smart” application that will make things extremely easy on you. It supports encoding using three codecs: x264, ffmpeg and theora. x264 is an actively-developed H.264 video codec, and is used professionally by many companies. It also has a lot of options that you may find useful in your encode, based on your target playback device (ipod, ipad, iphone, apple tv, htpc, computer, etc.).

Handbrake supports muxing (or putting audio and video into a wrapper format) to two containers: MP4 and Matroska. MP4 is a very popular standard used in lots of places. It’s an open-specification. Audio is usually AAC. Matroska is an open-source openly developed container format that has a standardized specification. Support for it is growing in a lot of devices. Matroska can handle multiple video streams, multiple audio streams, multiple subtitles, stores metadata, chapters and attachments. Strictly speaking, either one will work fine for most people’s needs, but for those looking for advanced usage and tagging, Matroska will fit your needs. On linux, the tools for muxing and accessing information about a Matroska file are found in the mkvtoolnix package.

Deinterlacing and Decombing

Most movies will be progressive video, meaning that each frame is one full picture. Production studios and DVD authors may have their video interlaced. Interlaced video is where half of the frame is shown in quick succession where the second half is shown. On older TVs (like the CRT tube TVs) display at a native resolution of 480i, so you won’t notice the lines. Progessive displays (computer monitors, HDTVs, etc.) will display the lines and it can be a visual annoyance.

Handbrake uses filters to convert interlaced video to progressive video, called deinterlacing. Decombing is also a feature that Handbrake supports, which is basically “smart” deinterlacing. It is safe to turn on the decombing filter to the default setting for all your encodes, and let Handbrake automatically convert your video for you. Don’t use the deinterlacing filter unless you specifically know what you are doing.

Please see for a quick explanation of video deinterlacing.

Other Handbrake Features

Handbrake has lots of handy features. It has support for “profiles” which is a combined set of preferences for container format, video codec, audio codecs, preferred subtitle settings, post-processing, deinterlacing, decombing, and all the H.264 settings that come with x264.

One nice thing it will do as well is auto-crop your feature film for you. This is a boon if you have a letterbox source video, where the movie itself is in widescreen, but the presentation is in fullframe.

You can also “queue” up your encodes, so Handbrake can be working on multiple files. You don’t need to wait for one to finish before assigning it it’s next task.

Handbrake is also multi-threaded when using x264 to encode your video. With a six-core desktop, I have been able to encode video using the “Normal” profile between 160 to 180 frames per second. Meaning a 25 minute video will take about 2 1/2 minutes to encode.

PHP Classes

I’ve got some small classes that I use in my shell scripts for Matroska and Handbrake (v0.95 required). You may or may not find these helpful. I should warn that these are just used on my own set of scripts, so they may or may not be very clean or can have small bugs.


If you want more information, here are some good places to go:

Multimedia wiki
Handbrake forums

Also, feel free to contact me if you have any questions, and I’ll be glad to help.

Leave a comment

Filed under Matroska, MPlayer, Multimedia, Programming

wrapper script for disc_id

I wrote a little wrapper script for disc_id tonight, available here. disc_id is a little binary that ships with libdvdread, or at least, it used to in older versions.

I use disc_id to give me a unique 32-character string of a DVD, so I have an identifier to track them by in my database of DVDs.

I don’t know if it’s just me or not, but my DVD drives have issues trying to poll the devices. Once I insert a disc, it will take a few seconds for it to register completely so I can access it. However, binaries that access it will think it’s ready to respond sooner than it is able, and will die unexpectedly. So what I needed was a way to get the disc id and not worry about whether or not the drive has finished registering or not.

I just call my little script dvd_id and it is simply a small wrapper that checks the exit code of the disc_id binary. If it doesn’t work the first time, it sleeps for one second and tries again, then repeats the process until it gets a successful exit code of zero.

That’s it. Pretty simple, but like all little scripts, you really tend to depend on them.
if [[ -z $DEVICE ]]; then

if [[ ! -b $DEVICE ]]; then
echo "Device $DEVICE doesn't exist" >&2
exit 1

while [[ $EXIT_CODE != 0 ]]; do
/usr/local/bin/disc_id $DEVICE 2> /dev/null

if [[ $EXIT_CODE != 0 ]]; then
sleep 1

Leave a comment

Filed under bend / dvd2mkv, Gentoo, Multimedia, Programming

web media frontend

I have always wanted to tweak my HTPC frontend quite a lot to add extra functionality, but the entry barrier to learning a GUI language has been way too high for me.  I’ve had success though, in patching MythFrontend to do some things a little better for me, but I’ve always wanted to get my own going if I could.

Recently, I was thinking about how LIRC can capture IR events and map them to X keyboard events.  Basically, you can control X applications with your remote control.  I started to reason that if that were possible, then I could just use my web development skills and create a webpage frontend for my HTPC that would run on a lightweight browser, and listen for keystrokes.

Just playing around with it tonight, I actually made some really great progress thanks to a combination of a good friend, my humble jQuery beginnings, and my laughable CSS skills.  This is the result so far. :)

I’m really stoked about the implementation so far.  You’ll most likely need Firefox to get that working properly.  It will capture the arrow key presses (up, down, left, right) and use that for navigation.  I realize that the beginnings are rather crude, but the fact that I could throw this together, so quickly, while I’m just barely learning my way around jQuery seems pretty impressive to me.  I’m actually quite proud, though, that I got the navigation to work properly, too, so wrapping around rows and columns works. :)

This is certainly going to be a fun project to hack on.  If I could get this working, this would open up all kinds of possibilities for me for displaying metadata and new options for navigation.

For comparison, here’s a screenshot of what my frontend looks like right now.  As you can see, I’m trying to imitate the style as closely as possible.

There’s a lot of advantages to having it web-based — not that I’m going to serve up anything remotely or anything, this is solely for my LAN.  It’ll just allow me to build out stuff much faster.

The hard part is going to be doing testing on the frontends.  They are both running off of tiny installations, and it’s not easy building and porting software to run on them.  Sounds like a challenge that’s extremely hard, going to take a lot of time, and will have marginal benefit and at the same time increase my workload and opportunity to own more of my software stack when things go wrong.  That’s just right up my alley. :)

1 Comment

Filed under Entertainment, Gentoo, Multimedia, MythTV, Programming

handbrake ebuild

My life has been uncharacteristically busy lately, which is a really new experience for me, because I usually have so much free time that I don’t know what to do with myself.  Anyway.  As a result of lack of time, it’s been interesting to see how I deal with the crunches when there’s stuff I want to work on.  So far though, my adaptation has been nothing more than watching tasks I want to do be ignored for weeks on end.

So, in an attempt to get one task of many moved forward, I’m just going to do a brain dump of my thoughts into a blog post and hope that someone can take it running from here.

To start with, I totally love the video encoding tool Handbrake.  It is an aboslute godsend, one that makes it possible for me to actually encode all my DVDs to MPEG4 using x264, and have me happy on every count.  (If I’ve talked about this already before … oh well.  I can’t remember these things anymore.)

There’s a lot of reasons for it’s awesomeness, but I’ll write those up in a later post.  The simplest summary is probably to say that it passed the Star Trek test with flying colors — which was always assumed to be an impossible task.  So, saying I’m happy is putting it mildly.  It’d be more accurate to say I feel like a schoolgirl on crack who is dancing on the rain.  Or something.

Anyway.  I’d like to roll an ebuild for it and get it into portage, if possible, but because of the build system, there’d be some things that need to change first.

The build system used in Handbrake downloads sources from their website and unpacks them during the building stage.  While that’s fine if you’re building it yourself, and if you wanted to roll your own ebuild (which, in fact, there are some already in our multimedia overlay), it wouldn’t be good from a QA stand for Gentoo.

So, what needs to be done (this is where I start whining about how busy I am, and how this is your job to fill in the gap) is the Makefile needs to be modified so it won’t download and unpack the remote sources.  It can still access them, but it needs to be up to the ebuild to do those in its own stages — like moving the tarballs into SRC_URI and using src_unpack to unpack them.

I haven’t looked closely at the build system, but I imagine it wouldn’t be too difficult to patch.  If someone wanted to take it from there, I could run the last few legs and see about cleaning up the ebuild and possibly getting it included in the tree.

If anyone’s up for the challenge, follow this bug.  Thanks :D


Filed under Gentoo, Multimedia