wonkablog

I’ve put this off for a long time, but I’m finally rebuilding my desktop from scratch. Actually, my one at work, too. Today is a state holiday here in Utah, and it’s great taking a day off in the middle of the week. I honestly have no idea what time it even is right now.

Anyway, at home my box is a dual-core AMD Athlon64 something or other. I’ve stopped building boxes a while ago after realizing that while it may be only around $300 to get started, it ends up costing another $400 for all the stuff I really wanna get on there, like a decent video card and quiet power supply, so this has been my box for a while. In fact, I’m running the exact same motherboard in my media server as well. It’s actually really nice knowing that I can swap hardware between the two boxes in a pinch (which has already happened) without worrying about the specs.

This box has been doing great, and I really don’t use it for much more than just surfing the web, writing email and of course Gentoo development. I’ve had problems on and off with XFCE and X. Exiting out would kill all input, and I could never figure out the cause. It wouldn’t matter which video driver or video card I used, so I just worked around it — I’d switch to the shell if I wanted to kill X, instead of just logging out, etc. But, one thing I sorely miss is using tuxonice, so today I finally decided to take the plunge and scratch it all.

It has been a long time coming, too. I can’t even remember the last time I installed the system on here. Generally speaking, I usually reinstall anyway just to clear out cruft and what not about every 18 months or so. It’s also a good way to keep me on my toes with new installs — my biggest problem is *still* forgetting to change my root password from a fresh stage3 install.

I’ve gotten pretty far, as I’m writing this in Seamonkey right now, and I started this morning. I’ve already been going through the list of stuff I had installed and it’s amazing how many applications I try out and then completely forget about. I’m actually excited to try out the new jobs and resume options in portage, see how that changes things around.

Actually, the thing that kicked off the reinstall at home was that I finally broke down and decided to do my one at work, first. The box is also a dual-core 64-bit machine, but it’s an Intel Pentium D. Now, for some reason, I have *never* had much luck with any of the Intel boxes. I always have some stupid problem where they just end up unresponsive for some reason that I’ve never had on the AMD boxes. I’ve never been able to pin it down, and so I’ve been methodically getting rid of any Intel-based boxes I have around.

This one at work is particularly painful though, though it’s mostly my fault. It’s only got 512 megs of RAM in there which is the obvious bottleneck in today’s modern desktop scenario — especially for a Linux workstation like mine where running a dozen applications all at once is the norm. At first I tried to resist doing anything about it so I changed my CFLAGS to -Os to save on memory (which I probably should have done in the first place), and while that helped, everything of course slowed down by a huge magnitude. Still, at least I wasn’t having jerkiness — it just took everything twice as long to load and save.

I finally broke down this week and shelled out $50 to buy 2 gigs of RAM for the machine. I haven’t been watching computer prices at all recently, so I was quite surprised by how cheap it was. In fact, I’m tempted to put more in my box at home. I’m sure that’ll fix things up alright. In the meantime, it’s reinstalling right now, chugging along, and seeming to do fine. I think I needed my head cleared of all the crap more than that box did, really. Sometimes, even though we technically don’t need to reinstall, it’s just a good mental relief knowing that you’ve done everything you can to make things run better. That’s a hard belief to argue against.

One thing I also did on my box at home is I finally switched over to ~amd64. I’m running unstable, which may not seem that big of a deal, but my policy over the last six years or so of using Gentoo, on my desktop at least, was to use the stable tree. Slowly, though, one by one, on all my other boxes, I’d switch them over to unstable and just run that tree, and I found that more often than not, it runs just fine without a hitch. I first switched over with my myth box, when I realized that my package.keywords file was longer than my arm — if there was one thing I always wanted the very latest bleeding edge code of, it was all the bugfixes and improvements for the multimedia software. The myth box is an obvious choice, and even then, you get it to a working point and then just leave it alone for an eternity. Those last two sentences just contradicted each other, I realize, but anyway, the point was — well, I forget.

Ooh, this –jobs option is pretty cool. Good work, Zack. Gentoo rocks, what more can I say.  After all this time, I’m still a fan.

Since things are finally starting to settle down (hey, I’ve had Internet for a week now, cool), I’ve started getting back to poking at stuff, slowly. Today I cleaned up some mplayer and transcode stuff in the portage tree, and I just thought I’d mention what’s new.

First of all, mplayer. The latest revision is 1.0_rc2_p27120. The snapshot itself is about a month old, and I don’t know of anything particularly major that’s in there (I haven’t had time to follow development for the past month anyway). It’s been masked for testing, not because upstream has done anything funky, but because there are some new use flags, which have helped me to go back and clean up some stuff that I should have done a long time ago. I haven’t heard a peep from anyone on bugzilla so either no one’s tried it, or it works great.

The big changes that I can think of off the top of my head are this: first of all, there’s an ewarn added if you enable the cpudetection flag. It finally dawned on me that this one might not make sense. Here’s what it *doesn’t* do — magically make the build detect and build for your CPU architecture. The build system already automatically detects and installs with the correct instruction set by default, using stuff like MMX, SSE, and friends. The only reason (that I can think of) that you would use that, is if you were deploying the binary package on another computer, since it is a runtime feature, not a buildtime one. So, you’ll get a little warning if you enable it, because, in general, desktop users don’t need it. I also fixed the use.local.desc to be a bit more specific as well.

Secondly, there’s a new use flag: custom-cpuopts which should finally solve the long-standing dance of trying to outwit users who always break their system but still allow real tweakers to do exactly what they want. Here’s how it works. MPlayer’s build system, again, by default, will automatically detect the CPU optimizations that are supportd for your platform (MMX, SSE, again, etc.). That means that, again, if you leave everything by default — it should work fine.

Originally the problem we would have is that people would have their CPU optimization use flags turned off (like mmx), which would normally break the build. The ebuild is written to only disable them if they are turned off, so my solution for a long time was to tell people to just turn them all on and let MPlayer *not* disable them, therefore, revert back to autodetection. This would then cause problems, though, for those few who were building MPlayer for some other box, and they wanted specific CPU opts on or off. This use flag will fix just that.

What custom-cpuopts will do, when enabled, is forcibly enable or disable the actual CPU optimization use flags you set. This will help out those on certain profiles and setups who need to tweak their build for whatever reason. For normal desktop users, you don’t need to enable it — if it’s turned off, the ebuild will completely ignore the other use flags (for CPU optimizations) and just use the default checks … which work great. It also means that your build will be optimized the best it can, yay!

I know I really went into depth on that one, but it’s been an issue with the build and users for a long time and I’m glad I finally got a cleaner solution working. Now everyone has the options to do whatever they want, but by default, everything should build and be optimized wonderfully.

Oh yes, one last thing — if someone wants to fix bug 228799, and submit a patch upstream, I’d be extremely grateful. That would clean up the LINGUAS hack in our ebuild, which I’ve heard breaks on paludis anyway.

As far as transcode — I’ve been out of the loop, and the media-video team has been taking things over and cleaning up (I don’t even know who, sorry guys, haven’t taken the time to look into that either — I know loki_val fixed up some imagemagick / ffmpeg dep crap that I didn’t want to touch, thanks man!). 1.0.5 is back in the tree, and should be the next stable candidate. I also added another rc for 1.0.6, and I have to add the 1.10 beta sometime.

That’s about it for now. Have fun. Go break your box.

Something else I’ve been meaning to do for a while, is get the main site up for the larrythecow.org domain since I started hosting stuff there other than planet. Today I got the idea that it might be fun to just have a page of resources of community pages that people are working on. So, that’s what I did.

If I missed anything, just let me know.

Edit: If you are getting redirected to Planet, then clear out your browser’s cache.  I originally had a permanent redirect going there, which should have been temporary to start with.  Whoops.  The old RSS feed links still work though, for everyone who never updated their entries.

I’m still working on getting ebuild dependencies into the database so I can query them with GPNL.  Getting the actual deps was easy, and that part I just barely wrapped up tonight.  The hard part is getting a fine-grained look at which dependencies are which.

Right now, I’m not too sure what the SQLite database generated by portage is doing, as far as setting DEPEND and RDEPEND.  It *looks* like it’s dumping all of RDEPEND into DEPEND, which makes sense as far as pulling in which packages to install, but that makes it hard on me to determine what the actual run time deps are — assuming that’s what it’s doing.  The fact is, there’s a number of possibilities (no DEPEND, no RDEPEND, empty DEPEND, empty RDEPEND, packages in both, etc.) that really the best way to determine which is which is to do something I should have done a long time ago, which was to write something that will parse the ebuild files directly for the variables.  Then I could easily see which ones are set, and which ones are included.

The fact is, though, that I wouldn’t have been able to do that until recently anyway, and the first time I tried it was when I first started on the package.  At that time, it was a such a disastrous mess trying to parse them as text files because you have random comments and straying lines and unquoted variables that it was just too much of a hassle to deal with.  Now that I’ve got regular expressions setup to figure out the difficult stuff like breaking down atoms into every version component and finding the package name from any atom, it should be much simpler to handle.

Until then, though, at least I’ve got dependencies, so I can add support to GPNL real soon now so we can dynamically query which packages at least depend on others.

In setting up the system, I’ve run into some interesting scenarios as well, as is usually the case.  I’m always thrown a few curve balls that I never would have expected and they either turn out to be QA issues or gentoo culture that I never could have accounted for.  One of them I hit this time was that I kept having ebuilds with dependencies for packages that were no longer in the tree.  At first I was confused, as it seemed like an obvious QA bug, but then looking at it closer, (almost) all of them are blockers — that is, don’t include this package in RDEPEND or DEPEND.  It leads to an interesting question like, how long should that be in there once the package is removed from the tree?  Who knows.  It’s fine to leave them in, I guess, but still, it’s interesting to run into stuff like that.

There’s still a lot more work to do, besides breaking down the actual RDEPEND and DEPEND variables more closely.  One thing I didn’t realize until tonight is that included in the variable is all the dependencies of the eclasses.  So if you have an ebuild using the kde eclass, then the ebuild will also have a dep for system packages like make and libtool.  I need to find a way to clean those out as well, obviously.

Fun stuff.  I hope to have an additional section soon for GPNL on the ebuild page that will display the dependencies, and in turn a possibility to display them in reverse.

I’m still getting overdue stuff taking care of, this time in Planet Larry. I’ve finally cleaned up the scripts and the configs to make it much, much simpler to debug and run the update process. Not only that, now it runs about 15 times faster (under one minute, actually).

Something interesting I realized the other day is that Universe gets about 5% of the traffic that Planet does. As a result, I changed it from updating twice an hour to once every 2 hours. Planet, on the other hand, I changed it from twice an hour to every fifteen minutes, and I might make it update even more frequently than that.

Anyway, all that’s changed is a bunch of boring backend stuff, which was some much needed cleanup that I’ve been putting off for a very, very long time. The fact that planet has been a maintenance nightmare is what has mostly kept me from dragging my feet on getting anything done, but now that that headache is gone, I feel hugely relieved.

There’s still more stuff to do, but I think I’m over the worst of it. What I’d like most is to work on next is getting the actual frontend cleaned up, and get a new design. That’s not really my thing, so if someone can help out, that’d be much appreciated.

Just another update on what I’ve been poking at lately.

I’ve mostly been cleaning up really small stuff, small bugfixes that have been annoying me for a while.

GPNL / Packages

For the packages website, I finally fixed it so that you can search by just package name again. It’s still messed up where it searches way too much stuff by default, but that’ll be fixed soon. It was originally searching by atom and description, so stuff like package$ would break.

I did, however, put the basic search I want to add to the packages website into GPNL: search by atom, package, category, description or all. I need to add changelog to that list. It’s not much, I know, but it’s a start on the long road to getting an advanced search going. I also cleaned up the front page a bit, and added a link to the nightly database dumps.

I also cleaned up the bash script to import the data. It actually has the beginnings of some error checking now, so you shouldn’t be seeing blank pages anymore. And the database is vacuumed correctly, and on a regularly basis, so things should be slightly snappier. I’m also importing the entire portage tree 3x daily now instead of twice. The import script is actually a nice cleanup for me, because if something breaks, I can run parts of it partially, instead of having to manually fix it. It’s much nicer.

My next big thing is getting RDEPEND searchable.  In the database, it’s combined with the DEPEND variable, so I have to separate the two out.  Once that’s done, we can finally dynamically query the tree to see where ebuilds need to be fixed for binary packages.

MPlayer / Transcode

Looking better, closed like eight bugs the other day for mplayer. Finally fixed some asinine bugs of mine for transcode, have one more to go.

Took out the masked libdvdnav because it will conflict and break libdvdread. I already wrote about how I put it in my overlay so if someone wants to use it, they can.

Sword ebuilds

I finally got pretty much all the main ones in the tree that I wanted to get in. There’s still two LDS ones that I have to make myself. Shouldn’t be too hard. I hope. In all, there are 150 sword ebuilds in the tree. Freak. That reminds me of something else I fixed on the packages website: it lists the number of results. That’s something else that’s been annoying me for a while.

I still need to remove the old sword-modules ebuild and add a new virtual-type one that will pull in all of the ones based on which language they are written in. Not hard to do, just slightly tedious. Should be done soon.

lds-scriptures 3.0

Believe it or not, I’m actually planning on getting this finished this week. The actual data has been finished for a very, very long time… it’s writing the documentation that I am extremely particular about because I plan on this being the final release.

That’s about it for now, that I can remember.

An interesting post came up the other day on the Gentoo forums about how to rip Blu-Ray discs on Linux. Short summary: I have no idea if it’s possible, and the original poster is still investigating. It has gotten me thinking though. The Blu-Ray player that I want to get it is $600, and it looks like it’s being phased out of production anyway, so why not get a disc drive instead and rip the movies? It’d save me some money, and I’d eventually buy one anyway.

Well, the questions that come to mind are, will the software actually work, will the drive firmware let me do that, and am I going to have to use Windows?

I haven’t done any research at all, mostly because I can’t afford to buy a DVD drive right now, but the whole thing does have me curious. I always assumed there was no way to rip the stuff under Linux, but I haven’t gone looking for possible solutions either. The only thing I am sure about though, is that once ripped, you can play the content just fine. At least, I think so. I’m not positive about the HD audio codecs, pretty sure about the video ones though.

I tend to buy hardware first and figure out how to get it working second, but because the DRM is so finicky in this case, I really don’t want to take that approach and be out a couple hundred bucks.  In the meantime, I really wish I could at least demo the stuff at home.  That would be cool.  The only 1080p content I’ve seen so far is the movie trailers I’ve downloaded from Apple’s website.  I gotta say that stuff looks pretty good.

I took a few minutes today and cleaned up a few small bugs on Planet Larry and friends.

One thing I get asked for every now and then is if I have archives of past posts. Well, I do now. I just copy the HTML file of the last post to $date one each run. A simple and unelegant solution. I’ll be doing the same thing for Planet Gentoo soon.

Also, fixed the FeedBurner link on the main page — I didn’t even realize it was broken. While I was at it, I created one for Universe as well.

On the packages website, I finally fixed it so you can search against just packages again. That’s been annoying me for a while. By default the search is way too wide, I think. It will search the full atom, the package description and the package name. I have to do the package name twice because of regular expressions (starting with, ending with, exact matches, etc.). And there’s still no simple way to search for packages containing multiple words, which is also an annoying little bug. Advanced searches for GPNL and Packages has been something I’ve wanted to do for a long time, but have been putting off since I started the projects. Sheesh. Every time I sit down and start to poke at it, though, I realize just how big a beast it is, based on what I’d like to accomplish. I really need something for the interim, though.

Anyway, I better quit before this post gets any more boring. One last thing — we can use more users who are Gentoo users and have a blog on Planet Larry. Just drop me an e-mail and I’ll get you setup.

I’ve made an awkward and difficult decision that I hope doesn’t make anyone feel bad: effective immediately I’ve removed any feeds from Planet Larry / Larry the Universe that were from developers who have retired from the Gentoo project.

I setup the planet feeds mainly so that regular users of Gentoo Linux could talk about and share their experiences, and I want to get back to keeping it with them primarily in mind. I tend to think that ex-developers have more weight with their posts, which tends to cause some unbalance that I was never comfortable with.

Speaking of users, I am always looking for new bloggers to get added, so if you are a Gentoo user who blogs about computer experiences, be sure to sign up.

That’s all for now.

Edit: I originally took this post down, and am now restoring it. I still haven’t decided if this is a permanent change or not. I might create a planet just for ex-developers, or reintegrate them somehow. I dunno. Comments and suggestions are welcome.

There has been some recent activity again on the libdvdnav front, and so I’ve been playing around with it again.  I finally created an SVN ebuild for MPlayer which I can use myself instead of just manually updating and reconfiguring the repo myself.

Generally speaking, I don’t like the idea of ebuilds that hit upstream’s SVN servers directly, so please be kind and don’t do anything asinine like upgrade every single day or something.

So now I have live subversion ebuilds for both libdvdnav and mplayer.  This is the important thing to know about the libdvdnav ebuild: it has some changes to libdvdread so you will have to unmerge the one that is already uninstalled.  On top of that, since it is a new API version, everything you previously compiled against libdvdread (k3b, etc.) will have to be re-emerged … and even then, there are zero guarantees that it will work at all.

In short, these ebuilds are only designed for a select few: those people who are using mplayer exclusively and want to have dvdnav support at the risk of breaking anything else that needs DVD access.  Obviously, that scenario fits for someone with a media frontend, but doesn’t make sense for general desktop usage.

Also worth noting is that these are the only two ebuilds that will work together.  Previously, the mplayer in the tree would have detected and worked with the libdvdnav ebuild, but that’s not the case anymore.  It’s these two in the overlay, or the mplayer in the tree.  Pick one set and choose.

If you have problems with the ebuilds, let me know.  I’m still not an expert at layman so I can’t go about giving you instructions on how to install this stuff directly.  Have fun, though.

Oh, and last but not least — I tested them, and it works for me. :)  Just emerge libdvdread first, then mplayer.