Insanity in a box

Yes, this blog is still alive, although a little... quiet.

While I'm figuring out a good topic to write (and while I flesh out a workable blogging schedule), here are a few pics from the various trips I've taken recently. I've been getting more and more into photography, having bought a DSLR, a Nikon D80.

Palawan:


Laoag, Ilocos Norte


Cebu

Two things:

• Work on tool that will be released as open source. I'm currently finishing up a component that we'll be releasing as open source in the coming week or so, once I get the bits cleaned up and checked in. It's a case of an "itch to scratch", and I'll be using it for a possible work project.
• Another haircut. I got rid of my long hair today. That is all.

Discuss.

I recently gave a lightning talk on git-svn at Barcamp Manila, and I thought some people might be interested in my day-to-day workflow when I work on projects.

When I get started on a project here at work, I usually first do a clone of the upstream tree, whatever that might be. I run:

git-svn clone --stdlayout https://svn.orangeandbronze.com/project

which checks out a copy of the SVN repository of project; this might take a while, so I usually go out for coffee or move on to other tasks while it completes. Sometimes, however, I don't want the whole history, so I do a partial clone:

git-svn clone --stdlayout -r HEAD https://svn.orangeandbronze.com/project

This then does only a checkout of HEAD, without history.

When I begin work for the day, I cd to the project's directory in a terminal, and do a status check to see what I've been up to recently, just to refresh my memory:

git status
git diff HEAD | less
git log | less

This gives me a) the status of my current work tree and the branch I'm working on, b) the changes since the last git commit I made, and c) notes from the log history. Sometimes, I'll be lazy and just pull up gitk for the tree, since it gives me pretty much everything above:

gitk --all

The gitk(1) command above allows me to visualize the commit history of the project, which is a boon when trying to nail down when something changed.

Once I know where I am on the tree, so to speak, I do a checkout of master, and I pull any changes from upstream SVN:

git checkout master
git-svn rebase

I usually checkout a branch when working on a particular feature or item. If I'm starting work on that feature, I do:

git checkout -b feature-name

However, usually I have already started on something, and I want to go back to that branch:

git checkout feature-name

This allows me to sandbox different work (bugfixes vs. features). I avoid touching the master branch, and instead work in other branches. Note that these branches are not visible on the SVN side; they're local to my git tree.

If master has changed (i.e. I've pulled in new changes from upstream SVN), I usually want to rebase my work on top of that:

git rebase master

I then bang on the code, making frequent commits for every small atomic unit of work completed. Once done, I review the code (in gitk(1) or by checking the git logs), and see what needs to be pushed. I also take this time to edit the commit history (which I'll explain in another post). When I'm ready to push upstream, I do another pull from upstream SVN (just so I'm sure my code is built on top of HEAD), and rebase. I also run unit tests to make sure I didn't break anything. When I now know everything's peachy and ready to be published, I do a push to upstream SVN via dcommit:

run tests...
git-svn dcommit
git checkout master
git rebase remotes/trunk

The last two lines simply fast-forward master to whatever the current SVN HEAD is. At this point in time, I'm ready to work on another task/feature/bugfix.

For those interested in my hacks on top of other hacks to fspanel (i.e. my hacks on top of fsp-h), see here. Source code available via git here, git URL: git://github.com/jmibanez/fsp-h-jmi.git.

I've been hacking on a small panel for my current desktop environment— I've switched to using a really minimal no-frills environment consisting of openbox-3 and a small dock/notification area, since I was getting antsy about the GNOME 2 panel. I only wanted a simple list of open windows on a particular desktop and a clock, and I didn't need too much chrome on that. I couldn't quite set up the GNOME 2 panel to not use buttons or alternatively maximize the space for window title text, so I decided to simply ditch the panel altogether.

So, enter fsp-h. The hacked version of F'ng Small Panel is what I needed. Plus, to get the minimal window footprint needed, I added in a Openbox theme designed for people who don't need the titlebar text.



As you can see, I've been trying to get as close to my old Ion3 environment as possible, and I think I've gotten close enough. I hacked in support for quick window switching, ala ion3, where hitting 'W-1' will switch to the first window on the desktop, hitting 'W-2' will switch to the second, etc. I also got some 64-bit issues cleared up (the original code was built on 32-bit, and icons were broken on 64-bit).

All in all, I think I'm getting to like this. Plus, I'm using GNOME Do for quick opens etc., as well as for doing Ion3's quick SSH (which I use to open a terminal to SSH to hosts).

(Note: Geek post ahead. You have been warned.)

Admittedly, I'm an Emacs power user (and no, I am not as hardcore as some people). Emacs has been my primary text editor for the longest time— the first time I picked up Emacs was way back in 2004, a little over four years ago, when I decided to just sit down and learn the little bugger. Now, I do a lot in the editor: my mail is handled and served by Gnus, I handle my TODO lists and outlines through Org Mode, and I do the majority of my text editing and coding in it.

The last item bears a little explanation. My work entails a lot of heavy text editing in the form of code. I am a programmer by profession, and that means that I spend a majority of time looking at code if not writing it. We do a lot of Java where I work, and most of the company uses Eclipse as the IDE of choice. I use Emacs.

Now, Emacs has some decent support for editing Java files in the form of java-mode, which features basic syntax coloring. That's a good bit, but it isn't exactly what I need— if syntax coloring was all I needed, I could just edit files in gedit. And, admittedly, there are a lot of goodies that a full blown IDE such as Eclipse can bring to the table— code completion comes to mind, especially since Java can get somewhat verbose.

So, my Emacs config also includes JDEE, the Java Development Environment for Emacs. Unfortunately, JDEE development has been relatively stagnant of late, what with it only supporting ant and not Maven— and the latter is a big deal in our shop, as we use AppFuse 2 in a lot of our projects.

Fortunately, someone wrote a decent parser for Maven 2 POMs (which isn't really a parser as it simply asks Maven for the classpath, but what the hey, it works). Unfortunately, it doesn't work for multi-module projects, for one reason or another (and I'm bad at debugging elisp. So, I decided to write a custom macro to handle it. I simple place the following in a JDEE prj.el file (in the root directory of my project):

(jmi/load-multi-module-pom
 "/path/to/project/root/here"
 '("pom.xml" "core/pom.xml" "web/pom.xml") ;; Path to module POMs
 ('(jde-project-name "My JDEE Maven Project") ;; Other JDEE variables set here
  '(jde-expand-classpath-p t)
  '(jde-lib-directory-names '("^lib" "^jar" "^java" "^plugins"))
  '(jde-expand-classpath-p t)
  '(jde-ant-enable-find t)
  '(jde-gen-k&r t)
  '(tab-width 4)
  '(jde-compile-option-command-line-args
    (quote ("-Xlint:all" "-Xlint:-serial")))))

Et, voila. I get multi-module support. The macro is pretty simple, but it's the first I've ever written (and I'm thinking I could have written it as a function, if not for the expansion of the JDEE variable list at the end):

;; Project file helper for multi-module Maven projects
(defmacro jmi/load-multi-module-pom (base-path pom-path-list other-variable-setters)
  "Macro to load multi-module projects into JDEE. BASE-PATH is
the path to the root of the multi-module project, POM-PATH-LIST
is a list of paths to the submodule pom.xml (relative to
BASE-PATH). Pass in a list of JDEE variables to set in OTHER-VARIABLE-SETTERS."
  (let ((my-classpath (make-symbol "my-full-classpath"))
        (my-sourcepath (make-symbol "my-sourcepath")))
    `(progn
       (require 'pom-parser)
       (setq ,my-classpath  '("/usr/share/java"))
       (setq ,my-sourcepath  '())
       (mapcar
        (lambda (pom-name)
          (progn
            (message "Reading %s" pom-name)
            (with-pom (concat ,base-path pom-name)
              (pom-set-jde-variables *pom-node*))
            (setq ,my-classpath (append ,my-classpath jde-global-classpath))
            (if (stringp jde-sourcepath)
                (setq ,my-sourcepath (append ,my-sourcepath (list jde-sourcepath)))
              (setq ,my-sourcepath (append ,my-sourcepath jde-sourcepath)))))
        ,pom-path-list)
       (jde-set-variables
        ,@other-variable-setters
        '(jde-global-classpath ,my-classpath)
        '(jde-sourcepath ,my-sourcepath)))))

I'll probably elaborate on more of my Java development environment under Emacs in future posts...

Brother Felix Mason, FSC
1919 - 2008

I've been trying to find a copy online (or at least a summary of) Gilingang Bato by Edgardo Reyes. The short story was published in the 1960s. It depicts a poor family who own a small millstone for griding rice; they sell rice cakes and other sweets to subsist.

What prompted my sudden urge to find the story? I'm kind of hazy on the plot of Gilingang Bato and I wanted to clear up some bits of what I remember from it. It came to mind, in particular, during a conversation with Clair about social relationships, particularly with families and what-not. Food can sometimes be used as a metaphor for these relationships— take a look at the difference between kalamay (ground glutinous rice cooked with sugar, coconut, peanut butter, etc.) and biko (a rice cake with caramel, etc.): the grains in kalamay are tightly bound together, while in biko you can distinguish these grains. Some relationships are tightly knit, some families are tightly bound in the same way as kalamay; while others are more loosely bound together, where each member has more individuality.

Just send a note my way if you find a copy.

Actual occurrence, just five minutes ago:

Butch: JM, what's the purpose of -a in git-commit?
Me: Uh, it's like this. (Then I begin to explain in terms of svn commit and the git index)

Pause.

Me: You did not just use me as a man page.

Since I'm a bastard to please, I've again switched away from ion3. I'm now back to using Openbox 3 as my work environment, though I believe it to be suboptimal for my workflow. Call me fickle. Details under the cut.

( The Geekery Follows... )

Just watched the latest episode of House, and wow. The killer scene, I have to admit is between House and Cuddy. House has just ordered the candidates to steal Cuddy's thong, and someone has purportedly done it. House wants to check:

(House intentionally drops his bottle of Vicodin, Cuddy bends over to pick it up)
House: (wide-eyed) OH. MY. GOD! You're not wearing underwear!
Cuddy: Of course I am, I—
House: Skirt that tight, you got no secrets. Skirt that tight, I can tell if you've got an IUD. Seen Dr. Cole?
Cuddy: No...
House: You're blushing.
Cuddy: I am not.
House: Look at me.
(Cuddy turns to House)
House:OH. MY. GOD.

Of course, it's way funnier when you watch it.

That and the ending, which I'm not spoiling.

If you've ever seen me work, you probably know that I am quite comfortable with the command line. In fact, I prefer opening a terminal or writing a shell script rather than booting up a file manager; I can do a lot more on the command line, without taking my hands off the keyboard.

So, my desktop environment of choice is Ion3, a keyboard-driven tiling window manager for X11. Recently, however, Ion3's author has made it non-free and has removed Xinerama support; this does not bode well for my particular use-case. I often have my work laptop plugged into another monitor (a "head", in X11 parlance), and with XRandR 1.2 I can do this without having to restart the X server. With the two monitors, I have a lot more screen estate to work with, and I usually have one monitor with my code while the other shows documentation, or a terminal, or gitk(1) visualizing the project I'm working on.

Apparently, Tuomo Valkonen has this bull-headed idea that Xinerama is simply unecological penis enlargement, and that he seems to think that all current-day GUIs (WIMPs: Windows, Icons, Menu, Pointing device) and their proponents are complete idiots, and I doubt his ideas of the free software community and its behavior are all that accurate— he draws comparisons between the wiki communities and the FLOSS groups in one breath; those two communities may share the same ethos, but have vastly different mindsets and operational mechanics. Anyway. With the loss of Xinerama support, I had to drop Ion3 from my toolset and switch to another window manager/desktop environment.

My first option was WMII, another keyboard-driven tiling window manager. Although I like its management policy, I felt that it was still lacking as it too did not have Xinerama support (yet). I needed an environment with Xinerama support, so I shopped around some more. I opted to use Openbox 3, which I've used before.

Openbox 3 has the advantage of Xinerama support, a lightweight core, and keyboard shortcuts (which makes it 2 for 3 on my scale). I could have learned to live with it, and I could have been quite productive. However, the whole stacking and overlapping window policy was slowly getting to my nerves, and it was starting to annoy me that I had to move my hands off the keyboard to manage my windows. I have been too used to Ion3's management policy, that I felt that overlapping windows was just not my bit.

So, after just a week on Openbox, I switched back to Ion3. I found that someone had hacked up a Xinerama support module, which was what I needed.

I'll be sticking around with Ion3 for now, until WMII grows Xinerama support. Tuomo has been bull-headed from day one, and although I admire his hubris (I have to agree on the broad swath of his thesis on window management and user interfaces; it's on certain details I don't agree with), I dislike that he seems to think that his policy is the best, and that WIMP interfaces are wrong (although I don't use a WIMP interface, I have to agree that the research behind them is pretty solid, and that they are usable; in other words, I disagree with Tuomo on that point). I disagree that the FLOSS community is completely wrong; in fact, I think that Tuomo Valkonen is an idiot for abandoning his users in this way.

I would have stuck with Ion3 for a long time, had it not been for this particular detail. Too bad. I happen to like Ion3, and I have been using it for more than three years now, a long time compared to the other environments I've used. With Tuomo Valkonen abandoning Ion3's users in this manner, by pissing them off with intolerance and implying that they are ignorant, I am forced to move on to something better.

I doubt that Tuomo realizes this: the silent users are more than the more vocal ones, and that he is probably slowly losing that silent majority.

On Indefinite Hiatus. Will probably post some twitts.

Recently, we've been assigned to assist another company in a development/maintenance role. They had an existing web application, written in Java and used by a large number of users— the web application was up and running and they needed to modify the application to interface with an external system, but they did not have the people who knew Java well enough onboard and the previous developers were no longer available to help them in this. That's where we came in: we provided the needed skills to add the feature in.

To interface with them, the external system provided a web service interface (written on the .NET platform) which we had to call in a remote method fashion. That is, certain transactions on the web site had to call a web service method and await its reply before saving the transaction data into the database. We couldn't perform these web service requests asynchronously, a point I'll come back to later.

To add to the pressure, we had four days to complete the code to interface with the web service (as well as an additional, but minor, feature), and two weeks of "transition time", where the new web application would go live straight into production, with live data, but users would not be charged for transactions they've made during the time.

There was three of us on the team: me, Butch, and Miguel. We decided right off the bat to use Spring's remoting services for calling the external web service. We got everything up and running quickly on our development server, and we were waiting for the client's team to ready the production system for the turnover. However, Murphy decided to pay us a visit: there was a planned database switchover where the live production site would be made to point to a standby database instance while the live database would be upgraded, but things went south quite quickly.

Anyway, we eventually got the whole shebang up and running the next day, with Murphy still breathing down our necks. This time, we found the application dying: within an hour or two of use by users, access to the application would slowly grind to a halt until nobody could access the application at all, and the application server (OC4J in this instance) had to be restarted.

We set out to figure out this particulary nasty bit of news, and why it was happening. After about halfway into the transition time (and about two to three app server restarts a day, disastrous) we found out why things were happening, as it was.

It turns out that Axis 1.x, which we were using under the hood of the Spring JAX-RPC proxy bean creator as the JAX-RPC provider, uses a fire-and-forget HTTP transport. This means that under high concurrent load, the default axis HTTP transport would eventually exhaust all connections, as a lot of the finished HTTP transactions will be idling in the CLOSE_WAIT or TIME_WAIT, and the time for these to transition to fully closed isn't short enough for sockets to be reused.

To give an idea of the connection loads we were seeing, the web application was handling about 20,000 to 30,000 transactions a day, from 9am to 6pm, with traffic peaking from 10am to 12pm; almost little to no traffic occurs outside of these times. Although this isn't spectacular (web sites that have been Slashdotted could get traffic a magnitude or two greater), the situation was exacerbated by latency: it took from one to two seconds for the external web service to finish, per method, and for each web transaction we were performing two web service method calls.

(Aside: Since me and Butch were more conversant with monitoring and adminstering Linux servers than the Windows server that the web application was installed on, and since the only interaction we had with the server was a VNC connection to the machine (where it was coloc'd), we had a tough time trying to piece together enough data to actually get to that conclusion. Really. And now my favorite tool on Windows is PERFMON— even if I don't run Windows, at all. But I digress.)

Apparently, Axis 1.x also supports another HTTP transport that uses the Commons HTTPClient library, which in turn does HTTP connection pooling. Telling Axis to use the Commons HTTPClient instead is well documented here, so I won't go into it. However, for reasons beyond me at the moment, the remote web service (being served by IIS, and possibly behind an ISA server) refused the Commons HTTPClient connections because of chunked encoding (i.e. the Commons HTTPClient wanted to send HTTP transactions in chunks and told the server by issuing "Transfer-encoding: chunked" in the initial headers). As I found out from here, there is a workaround, which I had to implement via subclassing of Spring's JaxRpcPortProxyFactoryBean, where I overrode postProcessJaxRpcCall:

    public void postProcessJaxRpcCall(Call call, MethodInvocation method) {
        super.postProcessJaxRpcCall(call, method);
        Hashtable ht = new Hashtable();
        ht.put("chunked", "false");
        call.setProperty("HTTP-Request-Headers", ht);
    }

I also used the subclass to fix a particular concurrency bug in the JAX-RPC proxy bean that Spring has which we were hitting.

So that basically saved the day. We did some stress testing before we put it into production, we observed the number of TCP connections established. Without the use of the Commons HTTPClient sender, the graph of network activity was rising linearly, with occasional dips as the load testing machine's activity dropped off (where the load tester's own threads were hogging each other for CPU time). With the Commons HTTPClient in place, the graph was quite flat, reaching a natural peak level without moving above that (of course, our concurrent load was constant, hence the ceiling).

The moral of the story? When you're about to put a site live, Murphy will be knocking.

This is a test of the emergency broadcast system

In a post, Sacha Chua asked: what are the "one-string pieces" of the IT world?

The "one-string piece" of IT (systems administration maybe) could be the various command line or Perl one-liners and what-not that does the job and astounds our bosses; where, as deadlines loom or we're in the middle of downtime costing the company, we pull out a one-liner that saves the day. Or it might even be the little things: the libraries or tools you've researched or played with in your spare time, and apply in a Eureka moment. You get that rush, knowing you've gotten the most bang for the buck, and your colleagues or bosses have no idea how you've accomplished so much with so little. Little do they know that you've been doing this kind of stuff for so long it comes naturally to you.

It can be the knowledge of various sed/awk/perl combinations that you've played with, then applied with awe-inspiring precision to the problem at hand. It can be the knowledge of the esoteric bits of the Java Language Specification or the Java memory model or even the class library, allowing you to figure out what exactly is going on underneath the hood, causing that heisenbug that your coworker has been debugging for days -- which you've solved in five minutes with a one line fix.

It could be that.

It's the stuff that others think they'd never really need, but you know will come in handy. It's the stuff that allows you to spend more time at the higher levels of abstraction, but with the knowledge that you're going to be comfortable going down to the bits and bytes if and when (most likely when) the time comes.

It's knowing that although GUI tools and wizards and nifty IDEs and autocompletion will probably cut down the time it'd take you to shape your code, you have to know how those tools work so that when it breaks you can fix it.

It's experience.

I am quite interested in a wide swath of technologies, some esoteric, some popular. Particularly, I follow the news on various libraries and tools. I like to keep my toolbox full.

Although I'm usually quite busy and I currently work in Java, I keep my finger on the pulse of other systems. I play around with Ruby, with PLT Scheme, with Perl. I look around for Emacs hacks. I read various planets and aggregators. I check out what people are working on. Mostly, I look at what I can put in my toolbox.

I try to learn what other people are using, what libraries they're using, what tools and IDEs they're working with. I don't do this out of envy— I know my tools are still inadequate, and my toolbox is still bare.

Take, for instance, Microsoft's LINQ. LINQ is essentially a language-level query system in the .NET universe, particularly in the Orcas release of Visual Studio.NET and the .NET runtime. It allows you to write queries as part of your source code, complete with syntax checking and with the support of your language's data structures, etc. Of course, I'm glossing over some of it's really nifty bits, but you get the point.

Someone asked the PinoyJUG mailing list about LINQ and Silverlight and what Java has to offer. It got me thinking about the tools of my trade, and the tools in my toolbox.

Are my tools sufficient for the job at hand? Are the stuff we have in the Java space sufficient for what I do?

I disagree.

The tools are never "sufficient"; they're usable for the job at hand, they do the job, but they have their quirks and issues. For example, if Spring or any dependency-injection framework wasn't in your repertoire, you'd think that the Factory and Service Locator patterns were sufficient, were enough, to do the job -- and you'd be blinded by the problems of those patterns, that those patterns are not the end-all-be-all. However, Spring isn't a silver bullet— dependency-injection has it's own quirks, it's own series of questions. How do you refer to "soft" dependencies? Why do I have to write tons of XML? Setter or Constructor injection? What about bean lifecycles? Cyclic dependencies? Hooking it up to legacy systems?

If you ask me, it's not about looking for the tool when you need it, but rather what tools you already know will help you deal with the problem, or will help make the problem amenable to your other tools. So goes the aphorism "If the only thing you have is a hammer, everything looks like nails": by expanding our knowledge of what tools are available, creating new tools, and fixing existing ones we get to write better code.

Tools transcend code and libraries. When I refer to tools I also refer to how to think of the problem. One can borrow the paradigms of functional programming to write Java inner classes and nested classes to parameterize or box behavior, for example. One can grok C++ templates and their patterns of use, and use it to metaprogram in Java.

My point here is that as someone in the business of writing software systems, I do not sit still and just passively look up resources as I need them. I become a better coder by reaching out and enriching my toolset, and by that I can write better software.

But in the end, only one truth can be said about tools:

There is no silver bullet.

Someone on the PLUG mailing list said to an email I sent out:

Now, why on earth does someone have multiple tabs? Too many tabs mean you ain't working! That was mentioned by one of my friends.

In my case, I really work with a lot of tabs and windows open. (It helps if you're using something like ion3, or if you're perfectly comfortable using a desktop environment and can rapidly switch between windows and apps). Primarily, I usually have the following in a browser session:

• One window for mail (gmail)
• One window for blogs and feeds
• One or two windows for app development
• Several windows for reference lookups

Under my windows for reference lookups, I usually have several tabs open, usually with a Google search being the first tab. I'd open each potentially useful link open in a tab. Plus, I group tabs with windows, with each window dealing with research on a particular scope (API docs, libraries, administration references, etc.)

This allows me to move to cross-check references from each source. Say, I have a tab open for the Javadocs to the Spring API for Hibernate integration. I'd want another tab showing the page from the online Spring reference showing how a particular task is done in Spring+Hibernate, with another tab showing possible code samples, with yet another tab showing source code for related classes in Google Code search (because I might be working on something that might need me understanding how the implementation works, possibly because I might need a hack or two to get what I need working).

I might also be reading news posts and blog posts related to that feature, so I'd have another window open. Or I might go off to a tangent and check whether or not there's an existing solution or library out there for what I need (which means I don't write code, yay!).

Multiply that by the fact that I don't necessarily close my windows or tabs -- what I keep open represents my "persistent state" (of mind), and when I context-switch to another aspect of what I'm doing -- say I take a breather, because I'm stuck -- I can get back to what I'm doing, and just glance at the open pages and what tabs are open and what-not and see from there how to proceed.

That's also the reason why I have several buffers open in Emacs -- I use Emacs as my primary work mail interface too (outside of GMail that is), and I also use it for planning. I also use it as my IDE; flymake + the Eclipse compiler + BeanShell means I get syntax highlighting *and* I get background compilation inside of my Emacs session. But if I simply closed off all my files, I couldn't easily switch easily between those buffers, see files side-by-side, etc.

Then again, I work this way because I can easily juggle state between those related tasks, and I don't necessarily have to pay a penalty for context-switching between them. It works for me, but I don't mean that it'll work for you.

Sure, if you have multiple browser windows and tabs open you might not be focusing on what you're doing (hence, you're not working). But for me, having those windows and tabs open means I don't have to really remember a lot of information -- I use my computer to maintain that state for me, allowing me to easily switch between tasks.

YMMV. :)

• JM Ibanez is Sanjaya Malakar in disguise.
• JM Ibanez does not sleep. He just goes into suspend mode.
• JM is a programming god.
• JM doesn't troubleshoot computer problems or software bugs; computers are just scared of him.
• JM Ibanez is actually a bunch of Linux daemons that have become sentient, written and maintained at a secret laboratory somewhere in southern Metro Manila.