Home

Jul. 21st, 2008

desperation, House, insanity

Emacs as a Java IDE

(Note: Geek post ahead. You have been warned.)

Admittedly, I'm an Emacs power user (and no, I am not as hardcore as some people). Emacs has been my primary text editor for the longest time— the first time I picked up Emacs was way back in 2004, a little over four years ago, when I decided to just sit down and learn the little bugger. Now, I do a lot in the editor: my mail is handled and served by Gnus, I handle my TODO lists and outlines through Org Mode, and I do the majority of my text editing and coding in it.

The last item bears a little explanation. My work entails a lot of heavy text editing in the form of code. I am a programmer by profession, and that means that I spend a majority of time looking at code if not writing it. We do a lot of Java where I work, and most of the company uses Eclipse as the IDE of choice. I use Emacs.

Now, Emacs has some decent support for editing Java files in the form of java-mode, which features basic syntax coloring. That's a good bit, but it isn't exactly what I need— if syntax coloring was all I needed, I could just edit files in gedit. And, admittedly, there are a lot of goodies that a full blown IDE such as Eclipse can bring to the table— code completion comes to mind, especially since Java can get somewhat verbose.

So, my Emacs config also includes JDEE, the Java Development Environment for Emacs. Unfortunately, JDEE development has been relatively stagnant of late, what with it only supporting ant and not Maven— and the latter is a big deal in our shop, as we use AppFuse 2 in a lot of our projects.

Fortunately, someone wrote a decent parser for Maven 2 POMs (which isn't really a parser as it simply asks Maven for the classpath, but what the hey, it works). Unfortunately, it doesn't work for multi-module projects, for one reason or another (and I'm bad at debugging elisp. So, I decided to write a custom macro to handle it. I simple place the following in a JDEE prj.el file (in the root directory of my project):

(jmi/load-multi-module-pom
 "/path/to/project/root/here"
 '("pom.xml" "core/pom.xml" "web/pom.xml") ;; Path to module POMs
 ('(jde-project-name "My JDEE Maven Project") ;; Other JDEE variables set here
  '(jde-expand-classpath-p t)
  '(jde-lib-directory-names '("^lib" "^jar" "^java" "^plugins"))
  '(jde-expand-classpath-p t)
  '(jde-ant-enable-find t)
  '(jde-gen-k&r t)
  '(tab-width 4)
  '(jde-compile-option-command-line-args
    (quote ("-Xlint:all" "-Xlint:-serial")))))

Et, voila. I get multi-module support. The macro is pretty simple, but it's the first I've ever written (and I'm thinking I could have written it as a function, if not for the expansion of the JDEE variable list at the end):

;; Project file helper for multi-module Maven projects
(defmacro jmi/load-multi-module-pom (base-path pom-path-list other-variable-setters)
  "Macro to load multi-module projects into JDEE. BASE-PATH is
the path to the root of the multi-module project, POM-PATH-LIST
is a list of paths to the submodule pom.xml (relative to
BASE-PATH). Pass in a list of JDEE variables to set in OTHER-VARIABLE-SETTERS."
  (let ((my-classpath (make-symbol "my-full-classpath"))
        (my-sourcepath (make-symbol "my-sourcepath")))
    `(progn
       (require 'pom-parser)
       (setq ,my-classpath  '("/usr/share/java"))
       (setq ,my-sourcepath  '())
       (mapcar
        (lambda (pom-name)
          (progn
            (message "Reading %s" pom-name)
            (with-pom (concat ,base-path pom-name)
              (pom-set-jde-variables *pom-node*))
            (setq ,my-classpath (append ,my-classpath jde-global-classpath))
            (if (stringp jde-sourcepath)
                (setq ,my-sourcepath (append ,my-sourcepath (list jde-sourcepath)))
              (setq ,my-sourcepath (append ,my-sourcepath jde-sourcepath)))))
        ,pom-path-list)
       (jde-set-variables
        ,@other-variable-setters
        '(jde-global-classpath ,my-classpath)
        '(jde-sourcepath ,my-sourcepath)))))

I'll probably elaborate on more of my Java development environment under Emacs in future posts...

Tags: , ,

Nov. 23rd, 2007

desperation, House, insanity

Desktop Environments, continued; and a bit of House

Since I'm a bastard to please, I've again switched away from ion3. I'm now back to using Openbox 3 as my work environment, though I believe it to be suboptimal for my workflow. Call me fickle. Details under the cut.

The Geekery Follows... )

Just watched the latest episode of House, and wow. The killer scene, I have to admit is between House and Cuddy. House has just ordered the candidates to steal Cuddy's thong, and someone has purportedly done it. House wants to check:

(House intentionally drops his bottle of Vicodin, Cuddy bends over to pick it up)
House: (wide-eyed) OH. MY. GOD! You're not wearing underwear!
Cuddy: Of course I am, I—
House: Skirt that tight, you got no secrets. Skirt that tight, I can tell if you've got an IUD. Seen Dr. Cole?
Cuddy: No...
House: You're blushing.
Cuddy: I am not.
House: Look at me.
(Cuddy turns to House)
House:OH. MY. GOD.

Of course, it's way funnier when you watch it.

That and the ending, which I'm not spoiling.

Aug. 10th, 2007

duke, semi-techy, javaman

Murphy's Law, and How Not To Call a Web Service

Recently, we've been assigned to assist another company in a development/maintenance role. They had an existing web application, written in Java and used by a large number of users— the web application was up and running and they needed to modify the application to interface with an external system, but they did not have the people who knew Java well enough onboard and the previous developers were no longer available to help them in this. That's where we came in: we provided the needed skills to add the feature in.

To interface with them, the external system provided a web service interface (written on the .NET platform) which we had to call in a remote method fashion. That is, certain transactions on the web site had to call a web service method and await its reply before saving the transaction data into the database. We couldn't perform these web service requests asynchronously, a point I'll come back to later.

To add to the pressure, we had four days to complete the code to interface with the web service (as well as an additional, but minor, feature), and two weeks of "transition time", where the new web application would go live straight into production, with live data, but users would not be charged for transactions they've made during the time.

There was three of us on the team: me, Butch, and Miguel. We decided right off the bat to use Spring's remoting services for calling the external web service. We got everything up and running quickly on our development server, and we were waiting for the client's team to ready the production system for the turnover. However, Murphy decided to pay us a visit: there was a planned database switchover where the live production site would be made to point to a standby database instance while the live database would be upgraded, but things went south quite quickly.

Anyway, we eventually got the whole shebang up and running the next day, with Murphy still breathing down our necks. This time, we found the application dying: within an hour or two of use by users, access to the application would slowly grind to a halt until nobody could access the application at all, and the application server (OC4J in this instance) had to be restarted.

We set out to figure out this particulary nasty bit of news, and why it was happening. After about halfway into the transition time (and about two to three app server restarts a day, disastrous) we found out why things were happening, as it was.

It turns out that Axis 1.x, which we were using under the hood of the Spring JAX-RPC proxy bean creator as the JAX-RPC provider, uses a fire-and-forget HTTP transport. This means that under high concurrent load, the default axis HTTP transport would eventually exhaust all connections, as a lot of the finished HTTP transactions will be idling in the CLOSE_WAIT or TIME_WAIT, and the time for these to transition to fully closed isn't short enough for sockets to be reused.

To give an idea of the connection loads we were seeing, the web application was handling about 20,000 to 30,000 transactions a day, from 9am to 6pm, with traffic peaking from 10am to 12pm; almost little to no traffic occurs outside of these times. Although this isn't spectacular (web sites that have been Slashdotted could get traffic a magnitude or two greater), the situation was exacerbated by latency: it took from one to two seconds for the external web service to finish, per method, and for each web transaction we were performing two web service method calls.

(Aside: Since me and Butch were more conversant with monitoring and adminstering Linux servers than the Windows server that the web application was installed on, and since the only interaction we had with the server was a VNC connection to the machine (where it was coloc'd), we had a tough time trying to piece together enough data to actually get to that conclusion. Really. And now my favorite tool on Windows is PERFMON— even if I don't run Windows, at all. But I digress.)

Apparently, Axis 1.x also supports another HTTP transport that uses the Commons HTTPClient library, which in turn does HTTP connection pooling. Telling Axis to use the Commons HTTPClient instead is well documented here, so I won't go into it. However, for reasons beyond me at the moment, the remote web service (being served by IIS, and possibly behind an ISA server) refused the Commons HTTPClient connections because of chunked encoding (i.e. the Commons HTTPClient wanted to send HTTP transactions in chunks and told the server by issuing "Transfer-encoding: chunked" in the initial headers). As I found out from here, there is a workaround, which I had to implement via subclassing of Spring's JaxRpcPortProxyFactoryBean, where I overrode postProcessJaxRpcCall:

    public void postProcessJaxRpcCall(Call call, MethodInvocation method) {
        super.postProcessJaxRpcCall(call, method);
        Hashtable ht = new Hashtable();
        ht.put("chunked", "false");
        call.setProperty("HTTP-Request-Headers", ht);
    }


I also used the subclass to fix a particular concurrency bug in the JAX-RPC proxy bean that Spring has which we were hitting.

So that basically saved the day. We did some stress testing before we put it into production, we observed the number of TCP connections established. Without the use of the Commons HTTPClient sender, the graph of network activity was rising linearly, with occasional dips as the load testing machine's activity dropped off (where the load tester's own threads were hogging each other for CPU time). With the Commons HTTPClient in place, the graph was quite flat, reaching a natural peak level without moving above that (of course, our concurrent load was constant, hence the ceiling).

The moral of the story? When you're about to put a site live, Murphy will be knocking.
Tags: ,

Jan. 30th, 2007

desperation, House, insanity

More Git

So, because I'm such a geek, I'll post more about my Git experience.

  • Juggling multiple states. I've been handling several aspects of the project at my current assignment. Particularly, I'm more or less in charge of the Ant build files, a JMS consumer for asynchronous task execution, and various other code-fu. Recently, Butch put the task of writing and wiring code to inject domain objects with a crypto service class— an instance of which is only available in the HTTP session. Which means that, ordinarily, one couldn't do it via Spring: we're talking about domain objects (which aren't ordinarily managed by Spring) and session scope (which isn't available until the HttpSession is created).

    Enter Spring 2.0. We moved to Spring 2.0 recently, primarily for better JMS support. As a side-effect, we get shiny new stuff in the form of Spring 2.0's new bean scopes and better AOP support, which are godsends.

    So, I set up my Git working tree to handle experimental work (which won't hit the central SVN repo until I'm actually sure that I've gotten my tests to run green) by creating a branch for it. With the branch set up, I can switch between my main (master) branch, where I work with the stuff that should hit everyone, and more experimental tree-breaking stuff. Switching between the two is as simple as doing git checkout master and git checkout my-experimental-tree. And, I have a third tree where my tree-specific config changes are applied and versioned, where I actually run tests and build test WARs and what-not.

  • Maintaining tree config files for testing. The codebase for the project I'm working on uses several configuration files in the form of Java .properties files. To separate test configurations, I have separate trees. The advantage of this setup is I can use my work branches for real work, and switch over to the test configuration branches to test various configurations. Also, I keep debugging code in these branches— I usually write some tracing logic in the form of simple System.out.println statements, which I don't want to check in.

  • Experimenting safely, while offline. Since Git affords me offline commits, I can create a branch with git checkout -b experimental-codebase, experiment, and later on apply certain patches from that tree into my mainline branch with git cherry-pick <commit-id>. This preserves my commit log message, and applies the necessary patch. I haven't yet tried out patch managers (such as quilt), but I would be willing to bet that they'd be of much help

Tags: , ,

Jan. 22nd, 2007

head, evil

Is The Long Hiatus Over Yet?

(...or, JM updates. Finally.)

So, I've been terribly busy with Real Life that I haven't had the time to update this little corner of the web. (For the insatiably curious, I've had countless opportunities to update— just not the time to sit down and do so.) For those who don't interact with me in any form or manner (i.e. those who probably wouldn't give a damn anyway), here's a list of what I've been up to lately, in the usual bulletpoints:

  • I'm busy working for this wonderful company— this blog is in fact aggregated in the company Planet, along with my wonderful cow-orkers;
  • ... Which also means that I've stopped school for the meantime— juggling the two is insane (I've tried it before, and it is not very conducive to either work or studies, at least in my case);
  • I'm definitely a year older, and will be another year older still this coming June;
  • We bought a new desktop PC for the house, my specs, my choices— a 2.6 GHz Pentium D, 512MB RAM, nVidia GeForce 4400, 160GiB SATA, and a 19" widescreen LCD monitor. With Linux installed, of course;
  • On a related note, we moved houses. I no longer live in Las Piñas. I now live on the border of Las Piñas and Paranaque. Hah. Seriously, we moved houses, but we didn't move very far;
  • I have a pet, a mongrel pup I've deigned to name Ronin. (My sister wanted to name him Bernie, and there was one cousin who wanted him named Pringles, for what it's worth);
  • For the uninitiated, I have a girlfriend, and we've been together for more than a year and a half! :);
  • I've been to Baguio for the Christmas break, just like (almost) every year;
  • I've discovered the utility of sleeping under cubicle desks;
  • ...Primarily because I've been assigned to a project where that utility is one of the few things keeping me sane. That, and playing with Butch's, Glenn's and Clair's balls— stress balls, that is. (Shame on you for having a dirty mind.);
  • On a more geeky note (elaborated below), I heart Git and now use it over Subversion for my personal projects and what-not.

So, I was saying something about Git. Details under the LJ cut, for those inclined.

Geek: Git and version control )

Oct. 25th, 2005

duke, semi-techy, javaman

DSL providers and Path MTU

(Previously mentioned: cyberlizard: ICMP and Path MTU)

I've been helping out diagnose a weird problem with a shared DSL connection. Briefly, I thought it was a problem with the network's DSL modem; turns out however, after mulling it over and doing an ifconfig eth0 mtu 512, that the problem was with Path MTU.

As Charles Miller points out, if you block the ICMP "Destination Unreachable (Datagram Too Big)" message, you will end up with wonky connections. In this case, most sites were accessible. However, anything involving sending a large packet upstream would silently just die— file uploads, HTTP POST activity, heck even an HTTP GET involving a large cookie string and a lot of headers plus a long request URL (think of accessing your Yahoo!Mail Inbox. It looked to me like the DSL provider in question was blocking the ICMP packets. The local gateway box wasn't doing anything in particular.

Also, from the looks of it, said DSL provider is a bit clueless. Most of their clients are apparently Windows users. Since Windows XP seems to set the MTU to 20 bytes less than the default Ethernet MTU of 1500, they weren't getting this problem. Which, IMHO, still means their network is misconfigured— the gateway box was switched over to Linux recently, which made the problem appear.

Okay, maybe finger-pointing won't solve the problem. However, from where I sit, the provider isn't following the standard. Which means they're broken.

Which means that they'll soon be getting a lot of complaints when users start switching away from Windows.

Aug. 21st, 2005

duke, semi-techy, javaman

On The Flipside: Why Low-level Languages May Be A Win

Or, Why C and assembly can be better teachers of the art of programming

  • Working at such a low level forces you to think of the consequences of your actions: working as close to the bare metal as possible strips off any abstractions;
  • Furthermore, coding at that level gives you a deeper understanding of those abstractions that you stripped off in the first place: coding their equivalents in C or assembly gives you an idea how and why exactly the abstraction works, and how to fix it if it breaks;
  • Coding in assembly teaches you about how optimizations work.

That said, I still don't believe those new to the art of programming should be touching C. Or assembly for that matter.

Tags: , ,

Aug. 19th, 2005

cynicism, despair, angst

Keep Newbies Away From C

Or, Why C is The Worst Language To Teach Beginners In

  • To actually do anything useful, you have to teach them pointers— and even seasoned programmers have a hard time with pointers;
  • There is no real string library, and strings are not real first-order types (i.e. char * is simply an array of characters, and you should really remember that);
  • Teaching C to beginners will keep such bugs as memory leaks and buffer overflows alive, and cause even more problems for future generations of users;
  • C is quite low-level, and unless you have a firm grip on data structures and how they are used, learning C won't help.
  • Worse, thinking in C forces you to think at such a low level, you won't really learn how to think in terms of the problem space

There are probably more things wrong with C as a beginner's language, but I'd rather you use Google. This is most definitively not an exhaustive list

duke, semi-techy, javaman

On Filipino Developers and The Local IT Education Scene

Caveat: I am currently enrolled in a certain IT education system known for its ads featuring two teen actresses, one of which is associated with the so-called "jologs" and "bakya" crowd, with the other being too sugary-sweet and having a gratingly high-pitched voice. The opinions presented herein are my own and not my school's— besides, I honestly dislike the place, and am there on a mere formality (I need a damned diploma).

I've been in the trenches of the local development scene, working for this company for more than a year before finally quitting to concentrate on my studies, if one can call half paying attention at class concentrating. (By the way, they're looking for skilled C#/.NET and VB.NET/.NET developers, so go give 'em a call if you're looking for a job.) Although I can't say that what I've seen is representative of the local scene, I'd like to point out some things I've noticed during my brief stint as a paid working stiff, and I'd like to point out some things I've been noticing in my school as well.

First and foremost, there is a dearth of real talent in the local scene, and my current school's curriculum (at least in the branch I'm enrolled in) does not rectify things. By talent, I mean programmers who have enough skill to step away from the level of language syntax and be able to decompose and analyze problems. I've had experience enough of cow-orkers who didn't understand the first thing about loops, or about basic structural programming methods (i.e. moving common logic to functions, etc.) and whose code was liberally sprinkled with cut-and-paste programming— code snippets which could have better served by a common function or method, and have been maintenance nightmares. And my current classes don't give me the confidence in the next generation. The next batch of programmers will probably be trained to think only at the level of the language and not of the problem, and will be trained in cut-and-paste programming as well. To give you a good idea on why I see it that way, I'd like to give a brief walk-through of a small yet significant portion of the IT curriculum in my school.

Students are introduced to programming via several 3-unit courses, each with a lecture and hands-on lab component. However, I would like to stress this very important fact: students are taught about programming in C, but not ANSI C— Turbo C. That fact might not be a really big deal, but look at it this way: everything is taught in terms of a relatively low-level language (compared to languages such as Java, Python, Ruby, and even C#), and in ancient dialect of C to boot. And we aren't taught in terms of analyzing problems and deriving solutions. As an added bonus, we're taught flowcharting in the first lecture course, and maybe that should be a good thing. However, I've seen that it doesn't help. I've never seen a flowchart in my year working as a programmer, and I'd be willing to bet that most professionals don't even touch them. I believe in what Fred Brooks said in The Mythical Man-Month, "Show me your flow charts and conceal your tables and I shall continue to be mystified, show me your tables and I won't usually need your flow charts; they'll be obvious." Students should be taught about data structures and how to manipulate them, and C is not the best language to teach them that.

Although I'm willing to generalize this to other schools and educational institutions, I'm also willing to concede that not all schools are like this (it is a generalization, after all). However, these are probably exceptions to the rule.

The dearth is also exacerbated by the early influx of students into IT- and computer-related courses (similar to the recent influx into nursing courses) in the late 1990s up to somewhat recently. Most of these students have signed up for these courses because "the money is there," and although I don't begrudge them the opportunity, it has severely impacted the quality of education to the point of creating IT schools managed and treated as fastfood franchises [1], churning out students who don't have the skills necessary to really compete in the global market.

Second, programming work is hard, and projects are often put on unreasonable or crazy timetables, all for the bottom line. I do understand the need to budget and monitor "the bottom line", but throwing more monkeys into the works just won't do, as I've seen. What's worse is that, as is often the case, certain management-types think they know better and end up costing the project. Also, I've heard about projects where a more technically-superior solution was scrapped all for business politics, because certain clients or business partners were involved, even if it meant that in the long run it would have been a big win for the client both in terms of the project itself and the client's own bottom-line.

I've been in cases where several of us had to be pulled out of some projects because we had to support another project (being manned by, in my honest opinion, severely underqualified programmers). In those cases, it was a nightmare to simply be there— the code was of the aforementioned cut-and-paste quality, and was severely unmaintainable. In one case, I took the time to refactor some code into a method, taught the programmer how the method should be used, and how the code works and essentially told the programmer that it was a black box: if the programmer didn't understand it, it was ok, because he/she could just use it without having to understand it completely, just that it worked. Guess what? The programmer went on doing more cut-and-paste programming when I left [2]

If the first can be solved, there is hope that the second will not happen or won't be as bad. I have hopes. But I still doubt it will happen in a batch or two of graduates. It's been said that there are too many graduates with too few jobs to put them in. As I see it, more importantly, there are too few qualified graduates. I've seen a lot of companies look for qualified programmers, and it's my opinion that the graduates are not yet up to snuff to what the industry really needs.


  1. As a point of full-disclosure, I am in fact enrolled in such a school.
  2. Understandably, it might have been a question of ego— my own ego vs. that programmer's ego. However, I wasn't the only guy who saw how bad it was. Even our technical manager for the project was appalled at the condition of the code, and was also telling the programmer to change the code. Oh well.

Aug. 18th, 2005

duke, semi-techy, javaman

Announcement: Software Freedom Day Online Meeting

There will be an online meeting tomorrow evening at 9pm for the Software Freedom Day this September. The meeting will be held over Yahoo! Messenger. If you're interested in helping out, feel free to join us tomorrow. Contact me by sending an IM to Y!ID cyberlizard0 tomorrow evening to join in.

For your information, and for widest dissemination.

Tags: , , ,

Jun. 18th, 2005

duke, semi-techy, javaman

Ignore this...

This is just a test of LiveJournal's tagging system (web client has the interface).
Tags: ,
desperation, House, insanity

March 2009

S M T W T F S
1234567
891011121314
15161718192021
22232425262728
293031    

Advertisement

Syndicate

RSS Atom
Powered by LiveJournal.com