So, if you are sincerely interested in running it, shoot me an email at beandog at gentoo dot org and I'll let you know everything that's involved (hint: not much).

Edit: I should add, this doesn't require being a Gentoo developer.  The Planet was never an officially sanctioned Gentoo project, and it was never intended to be.  Anyone with the ganas can run it.

Edit: Thanks to all the people who have offered to help.  I'll say the same thing I said to all of them: I'm going to wait a few more days to see who else steps up and has ideas for the site.  Also, no, I haven't decided what to do with Znurt for now.  I don't have any plans of giving up ownership yet.

I'm adding an ebuild for libbluray to the gentoo multimedia overlay if someone wants to access it.  It's something I plan on pushing into the mainline tree soon enough, once it's properly finished.

If you are building MPlayer from SVN, it will automatically detect the new library, and build against it.  You can use the -9999 ebuild in the portage tree.

To playback some of your Blu-Ray content, you will first need to extract it to your harddrive.  I use MakeMKV, also in the multimedia overlay, to accomplish that.

Here's a simple way using the CLI to dump the contents:

$ makemkvcon backup --decrypt disc:/mnt/bluray/ <location to dump content>

The syntax for playback is:

$ mplayer br:// -bluray-device <path to dumped content>

By default, it will play the longest playlist (I think).  If you can get the list of playlists available, you can pass that as an optional parameter to br:// (fex: list_titles /home/steve/bluray/src; mplayer br://5 -bluray-device /home/steve/bluray/src).

libbluray also ships with a few example programs that do basic stuff like listing the titles (list_titles), dumping information about the playlists (mpls_dump), and a few more (sound_dump, index_dump, mobj_dump, libbluray_test, bdsplice, clpi_dump).

Have fun with it.

I haven't found a way to embed this in my browser yet, but I haven't really looked either, so this is for all the CLI geeks.

$ mplayer $(youtube-dl -b -g http://www.youtube.com/watch?v=9IfEInQ7aic)

And thar ya go.

Oh, and did you know that Flash Gordon is on Blu-Ray now?  Flash!  Aaaaaa-ah!

Decrypting Blu-Ray discs is a really confusing process ... I'm still not even sure of all the steps that are involved.  Everything I understand has been cobbled together from posts on the doom9 forums.  While the forums are a great resource, it's not a comprehensive one at times.

I was playing around with aacskeys (from doom9 forums, available in portage), and it managed to decrypt / find the keys / whatever it's doing / work successfully on most of my movies.  I'm not sure how to get them off after that, though, or why that's important yet, but I do know it's a good sign.

For now I'm taking the simple route of using shareware to access my movies.  There's two programs I've used so far to rip my Blu-Rays, AnyDVDHD and MakeMKV.  They are both nice programs with some good features, but MakeMKV is the only one that has a Linux port.

The last time I tried MakeMKV, it couldn't decrypt all my discs, so I had to use my PS3 to rip the ISOs, and then use AnyDVDHD.  This time, though, using the most recent version (1.5.6), it managed to decrypt all of my discs.  I was going through my Blu-Rays to see if it could handle all of them, but I gave up after the 15th one, since it was working on every single one.

While AnyDVDHD will extract the original, unencrypted files to your harddrive, MakeMKV will additionally mux them at the same time into Matroska.  I kinda wish I could still have the originals, but I'm not going to be picky. (Edit: you can, see comments)

So, no real plans after this except to play around and post my results.  I really don't have that much interest in playing with Blu-Rays on Linux other than curiosity.  I don't wanna rip them and stream them to my HTPC just yet since I don't have the storage space, and because my frontend isn't quite as HD-ready as I'd like it to be (I still need to update some software and tweak settings ... lots of testing, meh).

I am going to be looking at some other tools and see if I can get them in portage or our multimedia overlay, which reminds me, I just added MakeMKV to there this morning if someone else wants to try it out.

I'll put it up here as a small reference, but I imagine I'll probably be the one using it the most.

local: git init

remote: mkdir project.git
cd project.git
git init --bare

local: git remote add origin ssh://server/~steve/project.git

Specifically, and I'm not picking on Ars in general, I notice in popular journalism a technique to claim that many people are supporting a view, but then to provide only *one* source that supports that view.  That doesn't mean that many people support it ... it means that at least one person does.

For example:

"Some critics of VP8 contend that its design is sufficiently similar to H.264 to warrant concern. One such critic is Jason Garrett-Glaser, a software developer who works on x264, a well-known open source implementation of H.264. In a lengthy analysis of VP8, he attacks On2's claim that the format is superior to H.264 and says that the format's legal status is too dubious for companies to trust."

There are no other references to "some critics" anywhere else in the article.

Again, here's a second example:

"MPEG LA's threats at this stage appear to be little more than self-serving saber rattling, but others who have analyzed the technology seem to believe that there could be serious patent risks on the horizon."

There is a reference earlier to MPEG LA's own remarks, the original piece of which makes its own conclusions as well.

Looking at that piece, the whole article is based around *one* question that he shared:

He writes:

"Here’s an excerpt from my email exchange with him:

JP: Let me ask you this: Are you creating a patent pool license for VP8 and WebM? Have you been approached about creating one?

Larry Horn: Yes, in view of the marketplace uncertainties regarding patent licensing needs for such technologies, there have been expressions of interest from the market urging us to facilitate formation of licenses that would address the market’s need for a convenient one-stop marketplace alternative to negotiating separate licenses with individual patent holders in accessing essential patent rights for VP8 as well as other codecs, and we are looking into the prospects of doing so."

That's the other thing I don't like about journalism ... I would call it a pet peeve, but really it's just a matter of not being able to trust the reporting when all we get is excerpts.  His entire article is written around one excerpt of an email exhange.  Why don't journalists ever post the entire exchange?  Lack of transparency, to me, just gives the impression that they are trying to present a biased view.

I realize, of course, that in only including excerpts here that I'm doing the same thing in a sense, but at least I'm providing references to the full sources I have available so that anyone else can do their own analysis and come to their own conclusion.

If you wanted to see his own conclusions, just read the article.  First of all, the headline is: "Google's "Royalty-Free" WebM Video May Not Be Royalty-Free for Long".  There's no way to draw that conclusion from the article.

I wonder if the editors come up with the titles of the articles themselves.  It  could easily have said "MPEG LA may create a patent pool for VP8", and that would be more accurate.  Compare that possible title to the other one when reading the author's assumption after the excerpt:

"It would seem, then, that VP8 may end up subject to the same licensing issues as H.264. If MPEG LA does create a patent pool license for the standard, the free lunch Google promised yesterday may not be free after all."

That's an obvious conclusion, and I could come to the same one as well -- If this, then that.

We can see again, even in this article, that he uses the same tactic of using one source and pretending it's many:

"As a number of observers have already noted VP8 isn’t free from patent liability."

Again, it's not a number of observers ... it's one blog post ... the same one that Ars referenced as well!  Jason is a great multimedia dev, but he's not a patent lawyer last I checked.  I'd be equally bothered if someone took my opinion, on any piece of my blog, and quoted me as the expert who knows which way the industry in Linux is going to go, or what legal battles it has to deal with in the future.

My take on the whole thing is this -- first of all, I thought Jason's original piece was very well written, and it was exactly what he set it out to be: a technical write-up of the codec.  He made some comments in passing about patents, but the focus of his post was how VP8 is better than Theora, not as good as x264 (and I would agree).  I would imagine that the poor guy didn't expect his blog post to get as much attention as it did, and that it will probably affect future blog posts, if any.

My opinion on the MPEG LA stance, reading just the excerpt above -- and not the author's own conclusion -- is that their business stance is completely normal and reasonable.  The way I read it is not that MPEG LA is claiming anything, but that some other companies might be wanting their own assurances of patent protection, and looking to their company to make sure they have their licensing ducks in a row.  That could be it, maybe not.  Either way, we don't have any information from them to really speculate.

Personally, I'm not too worried about the whole thing.  I think VP8 will emerge just fine, there may be *some* licensing involved somewhere, but in the end, open source tools will go on just like it has for years and support the standards, and consumers will still win out with more options.

As far as journalism goes, I think we're going to see more FUD pieces about the whole thing.  It's a common tactic used by big bullies (anyone remember SCO?).  I'm not saying the concerns are illegitimate, but I sure wish people would use critical thinking and analysis when writing their articles, instead of trying to spin up hype and paranoia for .. whatever reasons they may have.

It's obvious that my attitude is that modern journalism has completely lost its credibility, and that's the reason I don't like writing about it -- is because I get into rant mode. And I apologize for that.  Also, sorry that the post kinda bounces back and forth between my points ... it's the nature of a rant, I suppose.

One last comment (this is one of those posts that has the misfortune of never ending), that I wanna make sure I clarify is that it's not my intention to disprove, stir controversy or anything like that ... my only goal is to encourage critical thinking which seems to be a missing element in reporting these days.  I'm personally tired of how research becomes whittled down to conclusions.  It's like statistics -- you can often make the numbers say anything you want.  But, yah, not trying to hand out pitchforks or anything, I just think it's a good idea to be honest in reporting, present the facts, and let people come to their own conclusions.  That's all.   Have a donut.

For one, LIRC should actually build again, both on 2.6.32 and 2.6.33 kernels (both gentoo- and zen- tested, for the record).  Use either lirc-0.8.6-r3 or the new 0.8.7_pre1 if you like, and open or comment on existing bugs if you hit any.

I also fixed ivtv-utils so it'd build, and split v4l2-ctl into a separate ebuild so it's simpler to access for those who need just that handy little tool.

Finally, amid all the VP8 excitement, which I'll write up my own thoughts sometime later, there is already an ebuild for libvpx in bugzilla.  I tested it on my own overlay, and it looks good, so it should just be a matter of time before it gets included.

Just to clarify, I don't deserve any credit here other than actually doing some small amount of legwork and putting the stuff in the tree.  In each of the above cases, there were both ebuilds and patches provided by users, without which it probably never would have either gotten done or as quickly.  Thanks, guys.

I found a few bugs in my ripper this weekend, one of them was that I was only storing one possible subtitle type in my Matroska rips.  If a DVD had both VobSub and Closed Captioning, it'd only mux the first one I added.  Fixing it was fun, since it was one of those moments where you open up the code trying to find the reason for it, and you find a big comment labeled "FIXME: Add this feature here."  Heh.  So, now it muxes both, if available.  Woots.

There is still one DVD subtitle format that I am having absolutely zero luck in finding anything about -- English SDH (Subtitled for the Deaf and Hard of hearing).  According to Wikipedia, it's basically closed captioning with color.  I can play / watch / rip closed captioning just fine (watching: mplayer -subcc dvd://, ripping: ccextractor), but not SDH.  And I haven't seen anything that can even play them yet, although in fairness I've only been playing with Linux applications.  And everytime I try to explain to someone what I'm trying to do, they think I'm talking about VobSub subtitles.  Usually I get tired of trying to explain the difference and give up searching.  I could try finding some Windows apps to rip / play them, but if I can't get something in Linux that's scriptable to access them, then it doesn't matter anyway.  So, if someone knows of something ... plz to drop me a line, kthx.

Speaking of subtitles and MPlayer, I've come to the conclusion that MPlayer's support for them is just plain sub-par.  The options to play them back (or force them off) are buggy and inconsistent across the bar.  For example, here's a small roundup:

- Flagging a subtitle track as "default" when muxing a Matroska stream means that, if you turn on subtitles in the viewer, that should be the first one to show up.  It does not mean "these are forced subtitles, so display them automatically."  That's why Matroska has a "forced" tag.  default != forced.  If you're still lost, look at the original audio and video tracks, and you'll see they are also muxed with the "default" flag fipped on.  It's purpose makes more sense with video with multiple audio tracks -- if there's more than one, which one do you play by default?  The one with the "default" flag!  Same principle should apply with subtitles when you turn them on.

- MPlayer can't load Matroska subtitles externally.  You can, if you wish, mux just subtitle streams into a Matroska wrapper (ex: mkvmerge subtitles.{idx,srt} -o subtitles.mks).  But using "mplayer -sub subtitles.mks" won't work.  Bummer.   I understand that in this case, the Matroska stream could contain more than one subtitle stream (VobSubs and CC in my example), and it generally expects just one (-sub subtitles.idx, fex), but still, it'd be a fancy feature.

- MPlayer can't dump CC to SRT, even though it can play them (mplayer -subcc).  Bummer.

- Random rant about -noforcedsub and -nosub and -sub are conflicting / confusing, but too lazy to put together data about it, and it's mostly related to the Matroska one above.

I just had to get that stuff off my chest.   I have faith in MPlayer eventually improving in said areas, and filing bugs would probably be good on my part.  I generally don't deal with subtitles much anyway, so for me it's kind of a "would be nice to have" set of features.  Meaning, I've already worked around the bugs and they don't bother me as much anymore.  I would be curious to get SDH read support though.

I'm starting to notice a general trend here -- I complain a lot about certain issues and bugs in detail, but never go out of my way to report them.  I'm becoming the kind of user that as a developer I totally hate!  Oh noes!

In reality, I like being able to be on both sides of the coin, and I'd have to agree with the assessment of most user complaints I see, that are: the barrier to entry to reporting bugs is too hard.  I could go into detail about that, but I don't really want to, as I don't wanna focus on the negative.  But generally speaking, sometimes it's too much of a hassle to easily report a bug.  If it means me creating yet another user account on a bug tracker or subscribing to a mailing list, I weigh that against the strain of just ignoring or working around the bug.

I am, of course, to blame for my laziness, and I completely understand that developers (such as myself) need a detailed report with contact information along with the ability to quickly index reports.  I wonder if there's some magical middle ground, though, where users who aren't regular bug reporters can just easily report their issues and be on their way.  I know in Gentoo, we tend to use the forums as a poor-man's bugzilla sometimes, and maybe that's one way to do it.  Interesting stuff to think about.  Drive-by bug reporters, kinda thing.  They'll come by once or twice, but not regularly.

Anyway, I can't think of any other interesting DVD stuff I ran into this weekend.  Other than I bought season three of Taxi and it wasn't as entertaining as I remembered it to be.  Oh well.  You win some, you lose some.

I'm going on my eighth year or so of using it.  I don't know the exact timeline, but I'm pretty sure I've used it since around April of 2002 or so.  I'm also coming up on my fourth year of being a Gentoo developer.  That just blows me away.  I've also started recently getting a resurgence of interest in working on the development side again, something I'm really glad to see, since I have been progressively phasing myself out over the past few months.  Quite recently, though, I've managed to get myself much more organized and that has given me a much bigger picture of areas where I want to focus (and ones I want to drop interest), and so that has made it working on Gentoo fun again.   For the record, that's how it should always be.

There is one argument in particular I wanted to rehash here, and that is the one about stability as a distro.

I've often heard the argument, when debating Linux distributions, someone say something like, "Why would anyone run Gentoo .. it's not stable!"  I never have a really good retort to that answer when people bring it up, other than something simple like "Well, the obvious reason would be that I've used it for years and if I manage it, you won't have any problems."  Not to say that I don't have problems with my Gentoo installs, but these days they are usually stupid things ... like forgetting my root password or accidentally firewalling myself out of the box.

I don't really care about the "distro wars" though.  My motto has long been, "to each his own."  Find what works for *you* and apply it.  I could argue all day to someone about the merits of Gentoo, but really, all that matters is that it fits my needs well and does exactly what I want it to do, and how I want it to do things.  Everyone else can go find their own.  I've got mine.

But the fallacy in that original argument "Gentoo's not stable!" is a weak one.  It implies that the distribution should be the one providing all the support for keeping things constantly maintained, running and efficient.  To me, those tasks sounds like the description of a systems administrator job position.

My point is that, it shouldn't matter what Linux distribution you are going to use -- you need someone to keep it up and running.  I think Gentoo is great because it removes the veil from saying, "just run these versions of the software and you'll be totally fine."  Bugs creep in all the time.  Binary distributions stick you with a set of packages, that if,  you want to break out of that pigeon hole, it may be completely impossible to do.  With Gentoo, the definition of "stable" is left up to the user, the maintainer, the systems administrator.  I love it.

Go Gentoo.

I also can't believe I'm still a fanboi, after all these years.

So, the backstory is that the other day I was making a note of all the projects I have to do,  creating what I call my project tree.  It's basically an ordered list of general projects (Gentoo, home theater system, etc.), and then abstract projects underneath that.  It's nice because I can get a birds eye view of all the stuff I'm working on without going into specifics about any of them.  One of the home theater ones was to get at least ISO disc rips of the Blu-Ray movies I already have, even if I can't decrypt them yet, so I can at least play around with my options as they become available.

Well, I had moved my server setup around not too long ago, so I had two 750 GB harddrives just collecting dust.  I put one of them in an external USB drive, and plugged it into my PS3, which was already running Linux (see this pervious post for all the fun details).

I formatted my external USB drive as NTFS, so that I could read/write to it with Windows as well, and then I would insert a disc and just dump it to an ISO file.  That's easy enough:

$ cat /dev/sr0 > KFP.iso

I grabbed a couple of them (which took awhile, don't let me kid you on that part) so that I could get a good sampling in case I had more luck with one than another.

Once that was done, I trotted the little drive and plugged it into my netbook, running Windows XP, and installed both Virtual CloneDrive and AnyDVDHD.  The first one lets me mount an ISO as an actual disc drive, and the second actually decrypts the disc for me and dumps the contents back to the harddrive.  So, that's two passes now on all the data, which is making this take a long time.  But that's okay, it's fun.

Now that I have the actual contents, the MPEG2 transport streams, I can play it back with MPlayer, ffplay or whatever.  I'm having limited success with latest MPlayer on the files with VDPAU playback support, but it could just be my video card (GeForce 8600 GTS).  On my HTPC frontend, it works almost perfectly on there, with a nicer video card (GeForce 9300).  My desktop just craps out, though.  I can still watch it with Xv video out, though.

mkvtoolnix doesn't support m2ts files right now, so I don't have many options if I wanna change things around.  I'm still in a proof-of-concept stage, so I don't really care all that much.  Plus, my options are already limited.  AnyDVDHD is shareware that will expire in 21 days, and while it's amazing and works great, it's really expensive -- something like $200 for a lifetime license.  Eek.  With that, I'll keep trying my options on Linux.

The second piece of shareware I ran into (which also has a limited evaluation license, though this time for 30 days) is MakeMKV.  The Linux port is always a little more difficult to find, so here's a direct link to the details on the lastest version.  Boy, I never thought I'd be talking about shareware on my blog.

MakeMKV works really nice, too.  It's supposed to be able to do the same job as AnyDVDHD, I believe, but since I don't have the actual Blu-Ray drive, everything I've tried has never worked when just mounting the ISO direcly and trying to access that.  I believe that part of cracking the key involves having access to the drive.  I'm really not sure.  I've read a bit about the whole process, but it's still really confusing to me still.

Anyway, the software will let you access it directly through the decrypted contents, and that's what I did.  The interface is actually really simple and nice, and I would actually consider buying this one (it's much cheaper, at $50 for a lifetime license).  I'm trying to remember the last time I paid for a software license.  One that comes to mind, is that I actually have a valid registration key for Legend of the Red Dragon, the *really* old BBS door game.  Wow.  I think it cost me something like $15.

MakeMKV is pretty nice, though.  It snags the subtitles I select, and already includes the chapters as well as the HD audio formats.

So, that's it.  The whole process is pretty tedious, but it works, and I'm happy.  I don't really care about decrypting it *too* much right now, since I don't wanna go through the pain of trying to play them back over my HTPC just yet.  I'd need to do a lot of tweaks and upgrades to my system, and I really don't care that much.  It's not worth the hassle.  Especially, uh, since I just bought a new Blu-Ray player last month.

Really, though, I'm just doing this for the exercise since when I get bored, often times I'll want to play around with media files and formats and see what I can do with them.

Eventually I'll buy a BD-ROM drive and see what I can do, but for now I'm trying to save some $$$ and the whole point of this was to see if I could rip some discs with just the hardware available, and I could.   Woots.

On a sidenote, I posted aacskeys to the portage tree today, which is one of the tools users need to decrypt the keys on their discs.  Hopefully we can get some more hackers interested in poking at it.  That's always good.

Last but not least, here's an actual screenshot from the final rip.

I love TMNT.   Boy that's a post for another day ... which reminds me, I should get a copy of my home-made videos some day, that I made with my action figures.  Oh man, that'd be awesome.

First of all, I get a lot of ideas to do a lot of projects.  There's lots of cool stuff I want to do, and I have a hard time saying to myself "I have enough projects already in the works to finish, better not start another one," but I do anyway.  I tend to quickly overload myself sometimes that way, which can be bad for everything.  However, one thing I'm getting more strict on is only picking up projects that I'm sure I want to complete, that I'll see through until the end.  I very rarely, if ever, completely drop a project that I've started.  I will tend to put them on hold for a while -- sometimes years -- but I'll eventually revisit the idea (heck, the packages website is a perfect example of that).

I have a ton of projects I'm "working on," though.  So many, that I'm honestly afraid to write them all down for fear of being totally overwhelmed by the responsibility I put on myself for them.  I do, however, plan on getting them all done, and they circle around in my head on a regular basis, and often times I think of ways to integrate two projects (for example, adding an option to search gentoo planet(s) from the packages site).  I get a lot of interesting ideas all the time, but I really have to be careful not to overextend myself.

One thing I've been trying to do recently (as in the past year) is slowly shutter off some of the support I've been providing for the Gentoo tree directly, and ebuilds / herds I've in the past taken close care of.  It occurred to me way back when that it'd be a more efficient use of my time if I built out some project websites (like the packages one) rather than trawling the tree looking for ebuilds to fix, bump and repair (for example).  Not that I mind doing that, mind you, in fact I find it rather relaxing at times, but what's happened is that I've overextended my responsibilities again, and I'm trying to cut back.  Basically, my thought is that while I want to still work on Gentoo for a while, I don't want to make a career out of it.

Oddly enough, though, part of the reason I'm doing these community projects is so that I can more efficiently do other ones.  For example, at times I like to go through the multimedia packages and just check them to make sure we aren't missing version bumps, and go fix small bugs that I can take care of and just little stuff that isn't really important (in a sense of package popularity) but still relevant to a few users.  Those are fun.  But it'd make my life easier if I could more quickly track what has been neglected, more easily see what available version bumps are available (I still wanna hook into GnomeFiles and track their changes, for example), and stuff like that.  A lot of the tree-fixing stuff in Gentoo development is just monotonous, which is why it's hard to find volunteers to do it.  There's a good chunk of it that is just boring work!  And I'd like to help streamline that a bit.  That's one of my big goals.

With that goal in mind, a huge reason for doing the packages site was just so I can have a simple interface to get all the information I need, and finally a standardized set of data for categories, packages and versions.  That's mostly done, or at least the framework is, so now I can get going on the *really* cool stuff.  What I've done so far is really just the tip of the iceberg.

Anyway, I didn't wanna talk about just the packages site.  There's lots of other stuff I have going on.  It's interesting, even to me, to see which ones I'll want to juggle at a time.  I switch between them on a regular basis.  Sometimes I'll be working on the packages site, then my DVD ripper, then my scriptures stuff, then I'll work on theology ebuilds, then sound ones, then I'll look after ALSA, then mplayer, then I'll go back to tweaking MythVideo a bit, and round and round and round it goes.  I'm always working on *some* project, that's for sure.  It might do me some good to try and get a bit more organized, but I don't even do a good job of keeping track of bugs in my own projects.  I just track them internally for the most part.

So, I apologize for the epic behind status that I'm always in.  I'm starting to recognize more and more how much I'm holding people up on some projects, so I'm doing my best to gracefully exit those areas so someone else can come in and take over.  I'm still fumbling a bit at the best way to do that, but at this point in my life I have at least recognized the few areas that I'm sure I'm not passionate about anymore, and shouldn't be lazing around just pretending to commit once in a while -- of which, there are actually really few.  In fact, I can only think of one off the top of my head.

One thing that might be cool that I just thought of -- have a status indicator on my blog or something that displays the current project I'm working on.  That'd be fun.   Sounds like work, though.  I'm gonna go watch a movie.

The big thing I've been working on with the packages site now is making it more efficient.  The first step in that has been gathering some data on how often certain things are being called to see where optimizations are most needed.  So, the other day, I added a counter to the constructor of each class that would just tally each time the class was instantiated, and then I'd dump out the counter at the end of an import run.

One thing that surprised me is how often one particular class was being called -- PortageTree.  It's a really simple class, and all it does is set down some really simple variables that aren't going to change at all once they are declared, such as the location of the portage tree and it's metadata cache on the filesystem.  Pretty much used across the board on a lot of other classes that need to know the filesystem location of files (PortageCategory, PortagePackage, etc.).

Well, being still pretty new and fuzzy to the OOP approach, I thought it made sense to just extend the PortageTree class on PortageCategory and call the parent constructor to get the variables set.  That ended up in that class being created a huge magnitude of times,  all for the same pretty much unchanging variables.

So, I switched it this morning to use a singleton instance instead, so the class is only being created once and referenced thousands of times each import.  Much nicer already.

It's stuff like that that makes me wish I knew more about OOP.  I am studying it on and off, but there's still some concepts that I just can't wrap my brain around at times, like exceptions.  In my procedurally-attuned programming frame of mind, every time someone explains them to me, I think ... "Well, if something *breaks* why don't you just work with the return codes and work around that?"  So, yah.  Some stuff is still lost on me.  I'm trying to figure it out though.  Maybe it's one of those things that doesn't make sense so much when you apply it to PHP and it's general usage of websites.  A lot of the stuff I read about, I think how it would make much more sense if it were an actual application running.

Anyway.

On a totally different note, one thing I want to look at getting into the packages website is tracking a changelog of all the package's keywording history.  Right now, the import process is pretty simple -- if the content of the ebuild has changed, then the old one is marked for removal and an entirely new ebuild record is created in the database.  The reason for that is because that is far easier to do than it would be to examine all the myriad of data that is associated with one ebuild, track the changes, and then flag those.  Instead, I just dump the old one and treat the new one as a completely new record.

There's a tradeoff in the compromise, though, because instead of tracking ebuild modifications, I have to do all this coding to flag packages and ebuilds that things have changed and to treat them as an update instead of a new one.  That was tricky to get setup right, and getting that stuff in there in fact was one of the main things that pushed the initial launch back.  It was just one of those things that I couldn't run into the bugs until I started actually doing  a sequence of import runs, since they wouldn't show up until then anyway.

But, I'd like to start at least tracking the ebuild keyword status changes.  The reason is because that is really valuable data that can provide an excellent set of reports.  For instance, we can see which categories / packages / herds are getting ignored historically as far as stabilization.  Plus you can do cool stuff like import results from a statistics tracker as far as what people have installed, and you can start to see where maybe the tree could use a little more love.  And, it would help contributors who want to help out, but are overwhelmed by the enormity of bugs and packages and issues that need to be addressed.  I could see it being helpful saying, "here's an area that is suffering from neglect *and* is popular."  That would be cool.  And that's my goal.  In fact, that's *been* my goal for years.  I'm just now getting to the point where it's becoming possible, though.

Fun stuff.  I gotta hone my coding skills as I go, though.

I feel bad about how things have gone so terrible since the initial launch of the site. I really was not ready for the massive load, and my interim solutions were just slow and clunky. Hopefully things should be much happier now.

There's still a lot of silly bugs in the code that I need to fix. I just found another one this morning where the caching is breaking if you change your architecture selection around. Oy. I'd like to get to them, but I've been pretty swamped for time lately, between starting a new job this month and dividing my remaining time doing consulting work for two other companies.

Having a break from it though has been kind of good. I've already thought of a few optimizations that I can throw in there that are kind of like, "well, duh" type stuff I can't believe I didn't think of. For example, one way that I check to see if an ebuild is new is to see if the file mtime has changed. I don't know why it never occurred to me to just read the Manifest file and see if any of the hash sums have changed. That'd save me a lot of time.

I've been poked a few times about getting the code in a live repo somewhere, too. I guess that's coming soon, assuming I can get around to it. Personally, I don't like the idea of doing it when I *know* my code is in some ugly stages, but whatever. I need to learn how to setup a git repo anyway.

Oh yes, that reminds me. I also moved all the Planet Larry stuff onto the same server. Everytime I poke at the site, all I can think about is how much of an overhaul the whole thing needs. I'm totally embarrassed that I haven't even switched over to using Gravatar yet.

My goal is to ditch the planet software and write my own software to pull in the feeds, drop them in a database, and have the whole thing searchable. Then build a user admin section as well so users can manage their feeds themselves, and stop waiting on me. I'm planning on making that my next project, once Znurt gets to a better stage of stability.

Right now, though, I just did some minor tweaks. I got rid of the subdomains, and all the other projects on the site that I let atrophy, so planet is just available now at http://larrythecow.org/

El sitio esta aqui: http://es.znurt.org/

Translation: I'm all out of cookies.  May I date your daughter?

You may or may not know, but Znurt is the name of another Gentoo mascot of ours -- he's the little UFO / whatever flying around that you can see on Gentoo's homepage.

I'm not real sure what else to say about the site, other than it's been a lot of work, and I'm really proud of it.  If I had any idea of how many hours it'd take to put together, I never would have started on it, that's for sure.  I think it's safe to say I've put in at least 200 hours working on it in the past 2 months or so.  I think it's turned out really well, though.  It's a far better improvement over my old packages site which was thrown together quite hastily, and the code was pretty nasty.  This time around, everything is object-oriented which made it really go by quickly.

I've tried to put in some of the best features for launch as possible, but some just had to make the cut due to constraints.  The ones I really wanted was a page for new ebuilds, and a chance to have a feed of those, and customized by arch.  Same for new packages as well -- a view for each arch.  The new packages section almost got cut completely, though, because of some issues I ran into at the last minute, but fortunately that one made it.

I'll still be adding more stuff, but probably not at such a clipped pace as I have been for the past month.  You can track the planned features I already want to implement here.  If there's other ideas you have, let me know, and I'll see if I can get them in.

Which reminds me, I'd like to give a huge shout-out to all the beta-testers that helped out with the site development.  Every single person that got involved had a unique idea or perspective that I hadn't even considered, and it really helped clean things up quite a bit.  Thanks, guys, I owe you one.

I hope you all enjoy the new site.  Please contact me about any bugs you find -- hopefully I didn't miss any obvious glaring ones.  Also, if there are any XML wizards out there, I'd really appreciate some feedback on the Atom and RSS feeds, in case I missed something.  Not really a strong point of mine.

The thing I like the most about working on the site is it really revitalized my interest in Gentoo, and working on the project as a whole.  I had been losing interest in it for a while, but I realize now that that was partly fueled by my slip-shod attempts at user websites (planet included).  Taking a weak thing and making it strong, though, has really helped to push me in the opposite direction, though.  Not to say I'm going to make a career out of Gentoo development or anything, but it sure does help fuel my passion when you have some work you can be really proud of.

In fact, working on this has inspired me to do a rewrite of Planet Larry as well.  Writing the XML feeds wasn't nearly as difficult as I imagined, and though I don't understand them too well, it's definitely something I'd like to learn more about.  So, I'm going to ditch the planet software we are using and write my own, and finally give Planet Larry some features and love that it really needs.

Oh, also, there's a twitter feed that I use just for the website development, if you want to see what I'm working on, there.  Have fun.

The new design is in place, though, and it looks awesome, in my opinion.  I can't wait to show it off.   Right now, only the testers get to see it -- if you'd like to do some as well, just lemme know.  In the meantime, here's a little preview:

Now who is that little guy?

I've also got a Twitter feed set up now where I'm sending updates about the site progress, if you're interested in following.

I would cover a list of new features again, but this time I think it's embarassingly short since I've been just fixing bugs for the past few days.  I did get really basic RSS feeds added, but that's about it.

Here's some of the planned new features, though:

• display changelog with syntax highlighting
• display ebuild source inline
• show use flags, dependencies, reverse deps on ebuilds
• show open bugs on ebuilds (thanks to Mike for his help on this one)
• RSS feeds: new packages, version bumps
• Compact, verbose views
• Text-friendly design, for CLI browsers (elinks, etc.)

Like I said, no idea how long this stuff is gonna take, but it should make the site much more friendly and usable.  Yee-har.  Lemme know if there's something else you'd like to see, and I'll fit it in if I can, and it's reasonable.  Thanks, guys.

I have been working on this thing almost non-stop all week, because this site is by far the most complex one I have going for me.  Well, that's a personal project, at least.  I forgot how much work goes into this thing, as I've still got small laundry list of to do items.  But!  The good news is it should be ready and online really, really soon now.  Hopefully this weekend if I can squeeze more blood out of me.

I got to see the new design this morning, and I gotta say, I am absolutely floored by how amazing it is.  My brother-in-law was kind enough to do an original design for the rewrite, and I gotta say ... it's just spanky.  I love it.   He also redid the design for Planet Larry if you want an idea of his skill level.  Thanks, David!

Anyway, I'm not gonna open up the site just yet, but I do wanna start getting a list compiled of people who are interested in doing some beta testing for me.  Just send me an email if you'd like to help out.  It's not really a big deal if you wanna do it -- I just need you to poke around, use it like you normally would, but (and this is important) actually send me feedback about any bugs you find or suggestions you have.  Lemme know.

This is an incomplete list, and I've just started jotting it down yesterday afternoon as I started working on the site some more.  Also, it's a brain dump, so excuse the randomness.

Roadmap:

• package masks
• website; new design
• load testing
• bash script w/import options
• expanded versions
• options to use find all ebuilds updated recently
• status column for updates to happen in background
• db classes to access properties

I'll explain quickly what some of those mean.

Finding out whether a package is masked or not is a real pain in the butt.  The reason is you have so many ways a package can be masked.  I'm not gonna go into the coding required to checking it.  It's possible, and it's nice when it's done, but it's painful to write out.

New website -- yes, I'm rewriting it from scratch.  Well, the backend.  The functionality is going to be there 100%, so nothing is going to be lost.  If anything, there's gonna be a lot more stuff.  I'm also hoping to get a new design done before launching it.  Also, I'm probably going to do a closed testing invite session before launching it, so I can get some serious feedback first.

Okay, the status column thingie -- that's gonna eliminate one really annoying feature of the last website.  It would shut down for about 5 minutes while the site was updating, because it would delete stuff and add stuff all over the place.  The new one is just going to insert all the new stuff in the background, but won't flip it on to be actually visible until the import is completed.  So, it'll be completely transparent to the user, and the site will always have populated data, and any updates will just show up all of a sudden.

Now, some of the feature requests:

• RSS feeds
• XML API
• import all metadata
• parse changelogs
• profile masks
• tags
• track deleted ebuilds, packages
• screenshots
• user ratings
• twitter feed
• GLSA integration

A quick disclaimer -- no idea if and/or when any of those are going to be done, because they are all unnecessary to the actual launch of the site itself, which is what I'm working on now.  Another quick recap on what I'm planning, though.

The RSS feeds are going to be much better this time, and I'll have a larger selection: new ebuilds, recently updated ebuilds, per-arch feeds, etc.  And they'll have all the same info on the website too.

The XML API, which I've mentioned before, isn't really going to be that fancy to start with.  You'll just be able to do something like browse to website/app-admin/foobar/xml and it will have an XML printout of all the data that I have on that package.  Same for category pages, too.  Nothing fancy to start with.  Oh yes, I'll also produce database dumps as usual.

I need to import all metadata too to get GPNL back on its feet.  I already *have* it all in the database, but just as the raw original strings.  I need to sort through it and get it into its individual tables.  Again, I'm just doing the bare minimum right now to get the site up and running.

Parsing the changelogs, ugh.  That's a new feature I tried adding in the old website, and it never quite worked out exactly how I planned.  I'll get that one in eventually.

Profile support is one feature I'm really excited about, and one I wanted to keep a lid on for as long as possible.  Oh well.  Now you know.   Basically, I'm going to add support to the site to browse the keyword statuses for a specific profile instead of just the default one.  Once you change your profile, it'll affect package masks ... and, that's about it for now.  Still I think it'd be a handy feature.

Tags, screenshots, user ratings -- just some features that I'll eventually add to make the site actually accept user input.  That's gonna be a long way off because of all the work involved in user accounts and preferences and uploads.

Tracking deleted ebuilds -- I'm still debating whether I want to do this one or not.  I mean, I'll track them in the database, but it could be inaccurate for a number of reasons.  Not sure if I'm going to have a display for it on the website or not.  Would be kinda nice, though.

GLSA integration will hopefully be easy enough, I haven't looked at it at all.   Just track what GLSA notices there have been for a certain package and display them when you view it.  Nothing really fancy.

Alright, so that's about it.  I know I've gotten some feature requests from people, and if you don't see it listed here, then I've forgotten about it.  Please do me a favor and ping me on IRC or email and let me know, and I'll get it added to the list.

I still don't have a timeline to get the old site back up and running.  The good news is it is using a lot less CPU than I thought it'd be, so that expands my options for hosting.  The coding for the original roadmap is coming along at a clipped pace.  Everything I need to do has either already been done and just needs to be rewritten for the new backend, or is possible without too much problem.  In other words, there are no major roadblocks.  I still see it taking a few weeks, though.

I'll keep you posted.

I've been working on the packages website rewrite over the holidays, and it is really starting to come together.  I've decided to short circuit the process and instead of going for a full-featured site that I'd like to have, just duplicate the bare minimum of the old site to get it up and running as fast as possible.

I don't want to raise hopes too much, but it's gonna be a really cool site -- not the first launch, but what it'll eventually become.  I've rewritten all the code to access the portage tree, and I have classes to access just about everything.  Doing that has given me a tremendous amount of flexibility, and it just makes things easier to bang out.

There have been some rough spots that I have run into that are some hurdles.  Right now I'm working on determining masked packages.  Always a bit tricky, but thank goodness that's the hardest part I'm running into right now.

I don't have a timeline for when I expect the site to be up.  I'm actually ready to start rewriting the frontend for it right now.  I'm looking at getting a new design theme for it, and I'll probably wait to get that before launching it.  I already have a new domain for it, too.  Much shorter, easier to remember and type.  Realistically speaking, I think the site is probably going to be going live somewhere near the end of December.

Hosting is still a problem.  I don't know what I'm going to do with it, partially because I'm not sure how much CPU or traffic load it's going to generate.  I have three options right now -- host it at home, on my Linode or on a shared hosting account -- neither of which I'm really excited about since I think they'll all have different things they'll choke on, network traffic, CPU usage and speed, respectively.  So it's gonna be very likely that it gets bumped around a bit at first.

One last thing I wanted to mention.  I almost decided to completely abandon the project completely, since it seemed like no one really cared that I had even taken it offline.  I have had a few people poke me though, and ask about it's status, and I've seen it mentioned in a few other places.  This is one of those projects that I'm never sure if anyone ever finds it useful, so lack of feedback makes me want to drop it completely.  Inversely, any feedback makes me want to work on it and keep it going.  So, I'm really curious to hear what you liked about the old site.

Also, along that same vein, something else I'm going to do with the new site is build out features based on requests only.  I have a lot of ideas that I'd like to put in there, but instead of doing that, I'm going to hold off on them and just build out what the users want.  Everytime someone tells me how they use it, it surprises me because I never imagined it being accessed in that way.  So, again, feedback is critical.

Well, that's all for now.  Again, I don't know how soon I'll have a super slick website up.  Chances are, that I'll put an XML API up first (another new feature) or the RSS feeds since that doesn't require any nice eye candy.  We'll see.

The new site will also run on my dedicated Linode, where I think I've finally correctly managed the apache issues, so that means there will be less arbitrary downtime as I screw around with my box here at home.  I really hate running servers at home that other people are dependent on, because I like the freedom to change things around without affecting anyone.  Right now, the old site is so CPU intensive, that I can't move it over to the VPS.

The code for the new site is much cleaner.  The entire thing is rewritten in OOP classes to access the portage tree, which makes my job incredibly easier.  Not to mention it's a lot faster.  It'll still be a bit before I get it online, but killing it will inspire me to push it along.  I'm tired of having this thing limp around when it's just a dead albatross around my neck right now.  So, farewell.  The new one will be better.