Category Archives: Transcode

mplayer and transcode fixes

Since things are finally starting to settle down (hey, I’ve had Internet for a week now, cool), I’ve started getting back to poking at stuff, slowly. Today I cleaned up some mplayer and transcode stuff in the portage tree, and I just thought I’d mention what’s new.

First of all, mplayer. The latest revision is 1.0_rc2_p27120. The snapshot itself is about a month old, and I don’t know of anything particularly major that’s in there (I haven’t had time to follow development for the past month anyway). It’s been masked for testing, not because upstream has done anything funky, but because there are some new use flags, which have helped me to go back and clean up some stuff that I should have done a long time ago. I haven’t heard a peep from anyone on bugzilla so either no one’s tried it, or it works great.

The big changes that I can think of off the top of my head are this: first of all, there’s an ewarn added if you enable the cpudetection flag. It finally dawned on me that this one might not make sense. Here’s what it *doesn’t* do — magically make the build detect and build for your CPU architecture. The build system already automatically detects and installs with the correct instruction set by default, using stuff like MMX, SSE, and friends. The only reason (that I can think of) that you would use that, is if you were deploying the binary package on another computer, since it is a runtime feature, not a buildtime one. So, you’ll get a little warning if you enable it, because, in general, desktop users don’t need it. I also fixed the use.local.desc to be a bit more specific as well.

Secondly, there’s a new use flag: custom-cpuopts which should finally solve the long-standing dance of trying to outwit users who always break their system but still allow real tweakers to do exactly what they want. Here’s how it works. MPlayer’s build system, again, by default, will automatically detect the CPU optimizations that are supportd for your platform (MMX, SSE, again, etc.). That means that, again, if you leave everything by default — it should work fine.

Originally the problem we would have is that people would have their CPU optimization use flags turned off (like mmx), which would normally break the build. The ebuild is written to only disable them if they are turned off, so my solution for a long time was to tell people to just turn them all on and let MPlayer *not* disable them, therefore, revert back to autodetection. This would then cause problems, though, for those few who were building MPlayer for some other box, and they wanted specific CPU opts on or off. This use flag will fix just that.

What custom-cpuopts will do, when enabled, is forcibly enable or disable the actual CPU optimization use flags you set. This will help out those on certain profiles and setups who need to tweak their build for whatever reason. For normal desktop users, you don’t need to enable it — if it’s turned off, the ebuild will completely ignore the other use flags (for CPU optimizations) and just use the default checks … which work great. It also means that your build will be optimized the best it can, yay!

I know I really went into depth on that one, but it’s been an issue with the build and users for a long time and I’m glad I finally got a cleaner solution working. Now everyone has the options to do whatever they want, but by default, everything should build and be optimized wonderfully.

Oh yes, one last thing — if someone wants to fix bug 228799, and submit a patch upstream, I’d be extremely grateful. That would clean up the LINGUAS hack in our ebuild, which I’ve heard breaks on paludis anyway.

As far as transcode — I’ve been out of the loop, and the media-video team has been taking things over and cleaning up (I don’t even know who, sorry guys, haven’t taken the time to look into that either — I know loki_val fixed up some imagemagick / ffmpeg dep crap that I didn’t want to touch, thanks man!). 1.0.5 is back in the tree, and should be the next stable candidate. I also added another rc for 1.0.6, and I have to add the 1.10 beta sometime.

That’s about it for now. Have fun. Go break your box. :)

Leave a comment

Filed under Gentoo, MPlayer, Transcode

what i'm working on

Just another update on what I’ve been poking at lately.

I’ve mostly been cleaning up really small stuff, small bugfixes that have been annoying me for a while.

GPNL / Packages

For the packages website, I finally fixed it so that you can search by just package name again. It’s still messed up where it searches way too much stuff by default, but that’ll be fixed soon. It was originally searching by atom and description, so stuff like package$ would break.

I did, however, put the basic search I want to add to the packages website into GPNL: search by atom, package, category, description or all. I need to add changelog to that list. It’s not much, I know, but it’s a start on the long road to getting an advanced search going. I also cleaned up the front page a bit, and added a link to the nightly database dumps.

I also cleaned up the bash script to import the data. It actually has the beginnings of some error checking now, so you shouldn’t be seeing blank pages anymore. And the database is vacuumed correctly, and on a regularly basis, so things should be slightly snappier. I’m also importing the entire portage tree 3x daily now instead of twice. The import script is actually a nice cleanup for me, because if something breaks, I can run parts of it partially, instead of having to manually fix it. It’s much nicer.

My next big thing is getting RDEPEND searchable.  In the database, it’s combined with the DEPEND variable, so I have to separate the two out.  Once that’s done, we can finally dynamically query the tree to see where ebuilds need to be fixed for binary packages.

MPlayer / Transcode

Looking better, closed like eight bugs the other day for mplayer. Finally fixed some asinine bugs of mine for transcode, have one more to go.

Took out the masked libdvdnav because it will conflict and break libdvdread. I already wrote about how I put it in my overlay so if someone wants to use it, they can.

Sword ebuilds

I finally got pretty much all the main ones in the tree that I wanted to get in. There’s still two LDS ones that I have to make myself. Shouldn’t be too hard. I hope. In all, there are 150 sword ebuilds in the tree. Freak. That reminds me of something else I fixed on the packages website: it lists the number of results. That’s something else that’s been annoying me for a while.

I still need to remove the old sword-modules ebuild and add a new virtual-type one that will pull in all of the ones based on which language they are written in. Not hard to do, just slightly tedious. Should be done soon.

lds-scriptures 3.0

Believe it or not, I’m actually planning on getting this finished this week. The actual data has been finished for a very, very long time… it’s writing the documentation that I am extremely particular about because I plan on this being the final release.

That’s about it for now, that I can remember.

3 Comments

Filed under Computers, General, Gentoo, MPlayer, Religion, Transcode

transcode and dvd::rip

New versions of transcode and dvd::rip just went into the tree.  Transcode saw its first stable bugfix release (1.0.3) for the first time in a year and a half.  Woots!  The 1.1.0 release of transcode is apparently just around the corner, too, so hopefully we’ll see that soon.

I haven’t had any luck getting dvd::rip to work with the newer branch (though I haven’t updated my snapshot lately, either), but I know it’s support is planned.

Anyway, give them both a try … both very mature, good pieces of software.  I always encourage people to give transcode a try as an alternative to using MEncoder or ffmpeg.  There’s a learning curve, but well written man pages will help you out immensely.

1 Comment

Filed under Gentoo, Multimedia, Transcode

transcode snapshots

I just put a new version of transcode in the tree, this time a CVS snapshot of upstream’s development branch (v1.1.0).

It won’t work with dvd::rip, though, but that doesn’t mean you can’t play around with it. Besides, look at this as a perfect chance to get familiar with finally learning some CLI arguments and really learning to command your applications. That’s where all the fun is, anyway. :)

After a quick glance, it looks like the man page has a few errors. Nothing major, just small parts of it are incomplete. Actually, everything seems much cleaner, really. The output of transcode is a little nicer, and the man page seems trimmed down and more organized. I noticed while writing the ebuild that a few options were dropped in this branch, and a few other things added as well. I’m interested to take a look at it sometime and get a closer look at what’s improved. I haven’t played with encoding stuff in a long time, since I’ve kind of given up on my idea of having a massive library of encoded files on my mythbox. Truth be told, I don’t mind changing discs at all. I’m sure my position would change, though if I had some rugrats who enjoyed putting DVDs in the toaster or something like that. All in good time, though.

One thing is for certain — writing ebuilds of development snapshots is not nearly as easy as I thought. Configuration options change quite a bit, and you have to test things pretty thoroughly. I’m glad for the experience, though, that’s for sure. I feel like I’m getting a little better at it after each one.

Leave a comment

Filed under Gentoo, Multimedia, Transcode

multimedia bumpage

I don’t usually like talking about writing about stuff that I’m just bumping in the tree, but since there’s been a lot of it lately, I figure if I roll it all into one post it might be somewhat interesting.

I’ve been focusing mostly on just multimedia applications lately, since other Gentoo stuff has been annoying me, and this is the most fun to work on. Last week I went through the sound bugs and cleaned up some crufty old bugs that were simple fixes to ebuilds. I think that’s actually the first time I’ve fixed almost anything in the sound herd since I joined oh so long ago. There’s still a lot of stuff to be done there, but from a quick glance most of it seems pretty trivial. There’s a lot of requests for version bumps in there, which I’ll get around to sooner or later, usually depending on how tricky the ebuild is.

Here’s a helpful note to users filing bugs, btw. If you can attach a working ebuild, that helps a lot. I usually have to go in and tweak them anyway and fix errors, but most of the time they are just cosmetic in nature, or are missing a DEPEND flag or something. Even a little effort helps, though. And if you don’t know how to write ebuilds, learn! It’s not that hard for most of the packages that use a simple configure, make and make install. I recommend checking out the excellent devmanual for guidance on getting started.

Or, if you’re already good at writing ebuilds, try to find a proxy maintainer to put the ebuilds in the tree for you, while you do all the hard work (I really need to write more about that when I get some time). Or maybe get them into one of the Sunrise overlays. I’ve already dragged one user into the fray and put some of his multimedia applications in the tree. Proxy maintenance seems to work really well for me. I’m already doing it for a few applications. All I have to do is look over the ebuilds, fix them up, test the program and I’m done. The maintainer gets to do all the hard work like write the original ebuild and bug me when the version gets bumped.

Now that the wordpress security saga is finally over (it’s hard masked), I’ve been able to get back to my own ebuilds as well, too. Here’s some of the stuff that is new and I’m hoping people will check out and test.

IVTV released a new version, 0.10.1 which works across three different kernels this time (2.6.18 through 2.6.20). I upgraded my mythbox last weekend to the new kernel and version, and while it may be my imagination, I’ll swear that the picture is a little cleaner than before. The ebuilds just barely got unmasked because there was a bug in pre-2.6.20.2 kernels that would bork loading the firmware for certain devices. So if you are going to use a 2.6.20 kernel with IVTV, be sure to use either gentoo-sources-2.6.20-r1 (or higher) or vanilla-sources-2.6.20.2. Those are patched, so you should be good to go.

I’m thinking about masking the other, older IVTV ebuilds in a bit here, since technically they are all deprecated now by upstream because of this new release. I probably won’t remove them from the tree, though, because I’m all too familiar of the symptom of some funky version being the only one working on your hardware for some ungodly reason. Please do migrate and move onto the newer versions, though, since if any bugs come in on the older ones, I’m just going to start ignoring them.

In other IVTV news, the drivers are going to be in the mainline kernel starting with 2.6.22. I’m pretty excited about that too, since that means I’ll just have to maintain a set of userland tools after that. Speaking of which, 0.10.x have a few more than before, so be sure to check them out if you’re looking for some help. And please, for the love, read the README if you have any questions about the package. It contains lots of useful information.

Something else I bumped the other day was dvd::rip. It seems to have stabled up pretty good (I haven’t heard any complaints). I was a bit worried at first about removing the 0.5.x series from the tree (the old GTK+ v1 releases), but things seem to be okay. I don’t even remember what’s new in this version of dvd::rip. I remember glancing at the changelog, but nothing exciting is coming to mind, so who knows.

I also did a CVS snapshot of transcode, which is in the tree now. Their last point release was November of 2005 (eek!) and their website recommends running the CVS version instead. A lot of multimedia applications tend to do that it seems like (mythtv, mplayer being some other examples), and while it usually works just fine (except for myth of course, where upgrading is like russian roullette … sorry, I just can’t resist ripping on Myth ;) ) it makes it difficult for users that don’t want to or don’t know how to get CVS snapshots.

Transcode actually has two CVS branches as well, one for stable bugfixes, which would be the 1.0.3 release, and one for the next development release of 1.1.0. I put a snapshot of 1.0.3 in the tree, and I’ll get around to doing one for 1.1.0 later on. In the meantime, I’ve created a forum post if you find any issues with the snapshot, I’ll see what I can do.

I should mention that I have a secret love affair with transcode. I know I fawn over MPlayer and MEncoder constantly, but I love using transcode, especially for DVD encoding. The controls are a bit different, and may seem somewhat unorganized at first glance, but once you read through the entire man page, it’s not bad at all. It’s a great little encoding app, and I’ll put a cap on my fanboyism and just suggest giving it a try in your free time. I tested it with the latest dvd::rip and it seemed just fine, so go ahead and be daring and do the same.

One thing I did notice that was new in this release was that if you started transcode and canceled the process (ctl-c), it would kill the pipes as well. That’s actually very nice, since it was somewhat annoying before having to stop up to four processes just to kill transcode completely.

Other stuff that’s new, there’s a new version of OGMRip in the tree. I really like this slim frontend to MEncoder. The things I like the most is that it handles subtitles for me (I’m still really unclear on how those work, I need to do some manual encoding some time), wraps the output straight into Matroska, and of course is GTK+. :) I would say it’s been pretty stable for a good while now, so if you’re looking for a good ripper, I would recommend that one as well. So there you go, two recommendations for two rippers that use different backends (transcode and mencoder). There are more of course, but if you want the simplification of a GUI, you’ve got a few good options.

Speaking of subtitles, I somehow got roped into maintaining all the subtitle programs in the portage tree some time ago. I bumped gnome-subtitles again this week (thanks again to proxy maintenance), and there’s another ebuild in bugzilla for a new one that I’ll be looking at sometime soon. And I must in good conscience mention gaupol, your third option for subtitlely goodness.

I also got roped into helping out on ALSA now that Diego has left us. I’m still not sure how I got suckered into that one. I’m not making any claims to fixing things up nice and pretty, though. At the very least I hope to keep us on track to staying current with upstream and fixing some minor bugs. I’m still not convinced that I can be of any help on the team, but I’ll see what happens. Just don’t expect me to fix any major borkage. :)

Last but not least is mplayer. I finally got a new version of mplayer-bin in the tree, though that’s already old news. Thankfully the biggest complaint I’ve gotten so far is that I’ve ripped out AAC support. I’m actually glad for that, because it was a royal pain in the pooty pants getting that thing even working in the first place. As far as missing media libraries go, though, that is not not really my area of expertise. I’ll see what I can do about it sometime, but to be honest I’m not really in a huge hurry. I figure if you can play either the audio or video stream in the native mplayer / mencoder, then just separate the two and use mencoder-bin to do the other half. That’s poor reasoning, I know, but that’s why it’s not high on my list to fix right away. I should also mention that there’s a RealVideo stream I watch on a regular basis, half of which will only work in each player. So, I can sympathize. And yes, I do download it and then re-encode it with mplayer-bin just so I can watch it in mplayer.

I’ve also been working with Luca (lu_zero) on a new mplayer SVN snapshot. It’s coming along quite nicely, I’d say, and hopefully should be ready and in the tree real soon now.

That’s it for now, I guess there was more stuff to write about than I imagined. Sheesh. I hope the multimedia experience on Gentoo is working well for everyone. As I’ve said before, it was one of the main things that made me pick and stick with this distribution years ago, and I’m glad that I can finally get the chance to help out and give back at least a little bit.

As a final note, there really are a lot of bugs for sound, video and cdr stuff in the tree (well, not so much media-optical). They almost all register pretty much the same as far as importance goes to me, so if there’s something that’s been just nagging you for a while now, feel free to ping me on IRC (beandog on Freenode), or e-mail me (beandog at gentoo dot org) and I’ll see if there’s something I can’t help move along a bit. No promises, though. :)

1 Comment

Filed under Gentoo, IVTV, MPlayer, Multimedia, Transcode

gentoo, multimedia, and other shtuff

I think its always interesting how the more you get into a community project, it can take you in directions that you never anticipated going towards in the first place. For instance, I wanted to be a gentoo dev for a very long time, so that I could help out with the tree, and ebuilds and things like that. I always figured that if that ever happened, I’d be working on stuff like documentation and who knows what else. What ended up happening is I’m doing something completely different — taking care of Planet Gentoo, helping out with user relations, and maintaining multimedia ebuilds and a little qa on the side. It’s nowhere near what I thought I’d be working on, but I’m very happy nontheless. Working on Gentoo and helping to improve things is great fun.

One thing I committed to the tree yesterday was the latest version of ogmrip. If you haven’t heard of it before, I’d recommend you check it out if you’re looking for a good little DVD ripping program. It’s similar to dvd::rip except that instead of using transcode, this uses MEncoder. That, and ogmrip is a lot simpler. I like it a lot, though. And I know how hard it is to find a cool ripping program out of the probable dozens that are in the portage tree, so check this one out.

One of the cool features that it has that I really like is it can encode your files and wrap them straight into Matroska. Woots. Now if he would only add chapter support too, that would be really cool. ;)

It also handles subtitles pretty easily, which I consider “the final frontier” since there’s so many ways to get them, and it’s all very confusing, imo. It’s one of the few things I prefer leaving to a GUI when it comes to encoding video.

Working on ogmrip though got me thinking a bit about how life was like before I was a developer too, and some of the other frustrations that I had as a user. One of the problems with learning so much stuff is that I tend to forget what it was like, and what problems I wanted to fix. I am starting to remember a few of them though, and here’s what I can recall:

Everything was always marked unstable, and never seemed to get marked stable.

I always hated that — it seemed like all the multimedia stuff (which I still play around with a lot) is all marked ~arch and so you never knew if it was really unstable or just been sitting in the tree for years and no one has bothered to look at it.

Recently I joined the amd64 herd so I could hopefully help out on problems like that. So far I haven’t done jack squat because I’ve been sitting in the shadows just observing how things go on, for fear of breaking everyone’s boxes. :) But I have noticed one thing — a lot of the stabilization requests that come across that get neglected are for ebuilds that are are of just totally fringe programs. I mean, pretty much all of the bugs that I’ve seen have been for stuff I’ve never even *heard* of (and I’ve been using Gentoo a long time, too). I suppose that’s one reason that some of this stuff isn’t getting around to being marked stable, is because nobody uses it in the first place. I know when I see this stuff, I immediately think to myself “I’m not gonna test it, I don’t even know what it does, and I obviously don’t need it if I never heard of it.” I don’t know if that’s anyone else’s take on stuff or not. But it definately explains why my fringe amd64 stable requests pretty much just sit in the tree.

Anyway, I’ll see what I can do about that, at least for the multimedia stuff that I’m familiar with. It was a bit of a bumpy road getting dvd::rip stabilized since it had so many deps across a few arches that needed to be stabilized first, but it was kind of a cool sense of accomplishment once it was done. (Actually, dvd::rip is stable on all the arches except for x86, but that’s because of xvid-1.1.0 bugs. Just downgrade to xvid-1.0.0 and you’ll be superb.)

New ebuilds never get into the tree.

That was another thing that always bothered me about being a user, is ebuild requests would just sit out there in limbo and I had no idea why. Now I understand a little bit better. Its kind of the same situation as before, in that if there aren’t any devs that are interested in using it, they’re not going to support it.

I was poking around the media-tv bugs today, and I saw a few programs that actually looked pretty cool. I’m going to try and get them into the tree (provided they are stable, and work well) since it looks like they would be pretty beneficial. Then I started looking at other ones that I wasn’t interested in using, but they still looked pretty helpful. It got me thinking about something I read in this great article, Myths, Lies, and Truths about the Linux kernel.

The myth was this: “My driver is only for an obscure piece of hardware, it would never be accepted into the main kernel.” And the awesome response was this:

This just is not true at all. We have a whole sub-architecture that only has 2 users in the world out there. We have drivers that I know have only one user, as there was only one piece of hardware ever made for it. It just isn’t true, we will take drivers for anything into our tree, as we really want it.

Thinking about that really got me thinking about how I’ve always perceived Gentoo as well — just totally willing to take this fringe stuff that no one else has heard of and integrate it into the tree. Just replace “drivers” with “software” in that quote, and that’s the feeling I’ve always had, and I think that’s what makes Gentoo really cool. Sure, Debian, Suse, Fedora and Ubuntu might not package your really odd software in their tree, but Gentoo will! I’ve just always found it really cool that the tree has so much stuff in there that I have never heard of and never would have been able to find out about on my own.

I think that keeping these tiny, well-written and working applications in the tree just increases the cool factor, and I’m going to try and help out some more on that too.

I can think of some other stuff that bugged me, but I’m running out of steam so I better stop there. Gentoo is some great stuff, and that’s all there is to it. :)

5 Comments

Filed under Gentoo, Multimedia, Transcode

onto live-action next, baby!

My cartoons are almost done. I took a look at the queue last night, and there were 99 episodes left…. 97 of them all on one machine. I still haven’t coded it so that you can (easily) reassign the episodes in the queue from one machine to another. All it would really take, though, is moving the files from one machine to another and then a funky sub-select query to update the queue. I’d go into details, but it would be extremely boring. Needless to say, I just need to get off my duff and finish polishing up the frontend admin.

Anyway, since the cartoons will finish encoding today, now I’m left with a completely different beast — live-action shows. The same generic mencoder settings I’m using for cartoons work fine on these as well, but they don’t look quite as nice. It’s probably going to be back to the drawing board for a few days as I resume testing and research.

A while back, I thought I had the perfect solution. Since all I really needed to do to fix them so that transcode could process them was fix the variable framerate, I would just encode them, copying the audio and video, and then forcing the output fps. That actually worked great until you go to encode them *after* that.

To be honest, I’ve only tested that idea on two series so far, and they were both on TV shows from Universal Studios (A-Team and Murder, She Wrote in case you were curious). I don’t know if I’ve mentioned this before, but Universal makes pretty much the worst discs out of the entire lot. The Incredible Hulk is the first series I’ve gotten from them where I could actually play all the discs. And I’m not just talking about my computer can’t play them, they send the DVD firmware for a wild loop as well.

Depending on which DVD drive I’m using, some of them might be able to actually pull the entire file off. My Pioneer and Lite-On drives work great, while the Sony one would lock up my computer completely, and I’d have to hardboot it. In fact, I’ve been pleasantly surprised by the results with my Pioneer drives. They throw a lot of error messages to the system log, but if it really freaks out, all I have to do is forcibly open the tray, and the drive actually resets itself! Normally when I have to rip a disc out myself, the drive won’t be able to read until I reboot the computer to kind of clear the drive’s head. The Pioneer drives, however, still work. After I take them out, the open/close button still works, and I’m able to keep on ripping. I’m impressed.

So, my sampling of testing the worst discs in the bunch may not be the best approach. When I tried the A-Team one, it worked fine, but you could tell where it was switching framerates. Well, I need to clarify a little bit. The first copy/copy pass where I just used -ofps, *that* video looked great. It was when I transcoded from that that it looked like crap. Part of the problem might have been that I had to reindex the video because the AVI stream was broken.

One thing I could do is do a near-lossless encode on the audio or video so it at least processes the media file, albeit lightly, so the index is working without having to rebuild it. The audio would be pretty simple … just use ‘pcm’ as the output codec, pretty much copying it as a wav file. The video would be a little harder. I suppose I could export it as an MPEG2 stream again, and see what that does. Hmm, I hadn’t thought of that. Another idea is to give up on the AC3 stream and encode to the audio to MP3 or Vorbis or something. Doing any kind of encoding will keep the index intact. I’m just trying to find a solution that will leave it as near the original as possible.

I think that broken AVI index is what’s causing the awful artifacts though. Well, either that or the fact that it’s dropping (or duplicating, depending on what your target fps is) frames on the first encode. What happens though is you take that re-indexed video with the forced standard framerate, and *then* you run it through transcode. Again, I haven’t done much testing at all, just tried a two-pass encode on two different files. On Murder, She Wrote, the second file I tested, it actually looked really good for about the first twenty minutes. Then the entire thing was offset horizontally by about 60%, and there was a nasty green overlay. I wish I still had that original transcode, I could have posted a snapshot.

Whoops … one thing I just noticed is that my little reindex script was outputting the framerate to 29.97 (30000/1001) instead of 23.97 (24000/1001). That definately wouldn’t work well. I wonder if that’s how I was doing it on those first tries. I don’t think I was, but that might explain quite a few things. Time for more testing!

Well, this post ended up much longer than I had intentioned, but it definately got the wheels turning in my head a little bit. I’ve got a few ideas now that I can’t wait to try out. There may just be a perfect solution yet. :)

1 Comment

Filed under Computers, MPlayer, Multimedia, Transcode