Quality Equality

Trying to compare two arrays for equality without regard to order in a unit test is often very annoying. In Java, it would generally take four or five lines of code to express this in a test, completely obscuring the aspect of the code that I was trying to test. Ruby's Test::Unit turned that comparison into a one-liner, (expected - actual).empty?, a definite improvement but still a bit ugly. I was testing this in RSpec a couple days ago and figured there had to be a better way, and there is: actual.should =~ expected. I probably would have written a custom matcher for this rather than overload =~ if I was designing it myself, but either way, it's a big improvement in test expressiveness and readability.

Rails Parameter Filters

Secure programming 101 dictates that you should never write sensitive information to log files, and Rails makes this easy by allowing you to specify a list of sensitive fields in the filter_parameters property of your application configuration object (defined in the config/application.rb file). It's even nice enough to give you a sensible default (:password).

I got tripped up by this a couple days ago when a non-sensitive parameter that I didn't want to get filtered out of my logs was getting caught by the filter. I struggled with this for a while until I found the code responsible for the filtering, which clearly states in its comments that it filters out any parameter whose name matches the regular expression /<filter_param>/i. In other words, any parameter that contains one of your filter parameter strings will get filtered. It doesn't even have to be in the same case. Here's some example code demonstrating what's going on

The other thing I discovered is that other code that you include in your project (i.e. gems) can modify the list of filter parameters. For example, the clearance gem adds :token and :password to the filter parameters. To get the definitive list of all of the filter parameters in your application, launch the rails console and inspect the value of <ProjectName>::Application.config.filter_parameters.

BASIC Training

I was intrigued by this recent tweet from Roger Ebert for an article entitled "Why Johnny can't code". The article, which was written by science fiction author David Brin almost four years ago, is disappointing. The entire article is built on the false assumption that there are no freely available line number BASIC interpreters available for modern personal computers. The author concludes that because of this, the kids of today will never learn how computers work and therefore, as a society, we've all but guaranteed our descent into a digital dark age.

A quick Google search invalidates the author's base assumption. He's correct in that most of today's programmers cut their programming teeth on BASIC (myself included), but to argue that it's the classical Greek or Latin of computer science is highly delusional.

Of course, even if computers came loaded with an interpreter for a modern BASIC-derived programming language he still wouldn't be happy because he's completely fixated on the original line numbered version of BASIC. He seems to think that line number BASIC is closer the actual machine code than other languages or BASIC dialects that omit line numbers while at the same time being the only language that is easy enough for a beginner to grasp. Line number basic is no closer or further from the metal than any other variant of BASIC. I don't think it's any easier to learn line number BASIC than a more modern dialect, and relying on line numbers leads to really bad programming habits and obscures a lot of the mathematical elegance that he talks about.

His dismissal of widely available modern scripting languages like Perl and Python is also confusing. He claims that they are too high-level to allow you to follow the logical flow of the program. If anything, languages like Python that provide an interactive shell are even easier to experiment with than BASIC since you can execute your program one line at a time and see exactly what is happening inside the computer.

I don't know if the world is headed for a shortage of computer programmers, but if it is, the availability (or perceived lack thereof) of line number BASIC interpreters on modern personal computers is neither the solution to nor the cause of this problem.

Boxing Day

Any guesses as to what this program prints when you run it? l = 0? l = 100? Something else?

public class Foo {
  static long e(long v) { return 0L; }

  static <T> T e(T v) { return null; }

  static void f(long l) { 
      System.out.println("l = " + l);
  }

  public static void main(String[] args) {
      Long l = 100L;
      f(e(l));
  }
}

As it turns out, this program fails with a NullPointerException. There are two version of the e() method, one that takes a primitive long and one that takes a generic Object reference. Since l is a Long instead of a primitive long, the compiler is able to resolve the overload without needing to do unbox l, so it resolves it to the e() method that takes a generic Object reference. The result of this method is then auto-unboxed when it is passed into f(), and since the Object reference version of e() returns null, it fails with a NullPointerException. The solution is pretty easy, just call l.longValue() to convert the value into a primitive long, or better yet, declare l as a primitive long instead of a Long in the first place.

While this example of auto-unboxing and overloading confusion may look contrived and frivolous I've run into this error a couple of times in real life using the EasyMock#eq() methods to set up mock object test case expectations.

Autoboxing and unboxing is a fairly controversial language feature in some circles. I think the benefits outweigh the drawbacks, but, as I've illustrated above, there are definitely some drawbacks. I like the Scala approach better, but there's no way to completely get rid of primitive types at the language level in Java, so autoboxing and unboxing is as good (or as bad, depending on your opinion) as it's going to get.

The Little Engine That Could?

Now that Google has added Java support to the App Engine, I decided to finally give it a try. I decided to go the Eclipse plug-in route since I already use Eclipse, but the plug-in didn't seem to include the development server (as the docs lead me to believe), so I couldn't test the sample application locally. Rather than try and figure out what was wrong, I downloaded the standalone App Engine SDK and installed that as well. I got a bunch of errors when I started the development server because it requires Java 6 and I'm running Mac OS X 10.4, which doesn't have an official Java 6 SDK (remind me again why I thought switching to Mac was a great idea for Java development?). I tried the ancient pre-release Java 6 JDK that Apple put out a few years ago but it failed with a bunch of weird XML parser errors, so I tried SoyLatte JDK 6 1.0.3 and that finally worked. I logged into App Engine to register my account (which can only be done via SMS) and entered my mobile #, but I have I yet to receive my activation code text message so I'm stuck for the time being. From what I've read, others appear to have had more luck with it, so as always YMMV.

Block Expressions

I discovered a weird Java collection initialization idiom whilst reading this post a couple days ago.

List<String> l = new ArrayList<String>() {{
   add("A");
   add("B");
   add("C");
}};

It took me a minute to figure out what was going on. For those who haven't seen this before, the first and last curly brace define an anonymous inner class. The inner curly braces and the code therein comprise an instance initialization block for the anonymous inner class. So this code effectively create a subclass of ArrayList that adds three elements ("A", "B", and "C") to itself upon creation. It's kind of interesting, but I'm not sure how useful it is. The only place I could see using something like this would be in a class that needs a wrapped static final collection. Using this pattern would eliminate the need to build and populate a temporary collection in a static initializer and then wrap it and assign it to the static final member.

As it turns out, one of the proposals on the table for Java 7 is something called Block Expressions, which would allow you to define temporary local variables within expressions. One of the use case given for this feature is identical to the use case I outlined above.

public static final Map<Integer,Integer> primes = (
   Map<Integer,Integer> t = new HashMap<Integer,Integer>();
   t.put(1, 2);
   t.put(2, 3);
   t.put(3, 5);
   t.put(4, 7);
   Collections.UnmodifiableMap(t));

Block expressions have other uses beyond this so it may be a good idea to add them to the language, but I'm actually partial to the anonymous inner class with instance initializer approach in this situation.

public static final Map<Integer,Integer> primes = Collections.unmodifiableMap(
   new HashMap<Integer,Integer>() {{
       put(1,2);
       put(2,3);
       put(3,5);
       put(4,7);
   }});

Stream of Consciousness

I've been taking a look at functional programming languages in general and Scala in particular as of late. Partially because I've been sucked up into the hype surrounding FP & Scala and partially because I've never really done any FP. I read this post about infinite lists in Scala a couple months ago and decided that I wanted to try and do something similar with files. My goal was to create a function that takes a directory and returns a lazily evaluated list that can be used to recurse through every file and directory under the directory. Here's what I came up with:

import java.io.File

def makeFilestream(filelist: Stream[File]) : Stream[File] = {
  if (!filelist.isEmpty) {
    val file = filelist.head
    if (file.isDirectory) {
      Stream.cons(file, makeFilestream(file.listFiles.toStream.append(filelist drop 1)))
    } else {
      Stream.cons(file, makeFilestream(filelist drop 1))
    }
  } else {
    Stream.empty
  }            
}

def filestream(root: File) : Stream[File] = {
  val filelist:Stream[File] = root.listFiles.toStream
  makeFilestream(filelist)      
}

So what's the point of all this? By using Scala's Stream class, the list is lazily evaluated so it doesn't have to scan the entire directory tree before returning the first element. Furthermore, since Stream behaves like any other Collection class, all of the normal Collection operations like foreach, map, reduceLeft/Right, etc. are supported, eliminating the need to load the directory tree into memory and storing it in a List before operating on it.

Here are some examples of the cool things you can do with a file stream

Print everything in the directory /tmp

val tmpdir = filestream(new File("/tmp"))
tmpdir.foreach(println)

Print the first 10 directories under ~

val homedir = filestream(new File("/Users/username"))
homedir.filter(f => f.isDirectory).take(10).foreach(println)

Find the largest file under ~

val homedir = filestream(new File("/Users/username"))
val biggestFile = homedir.reduceLeft(  
      (a, b) => if (a.length > b.length) a else b)

Find the first Java source file under ~

val homedir = filestream(new File("/Users/username"))
val firstJavaFile = homedir.find(f => f.getName().endsWith(".java"))

Pretty much any file searching, selecting, or transformation operation you can think of can be expressed as a one-liner or chain of one-liners and thanks to the Stream, any operation that doesn't need to scan the entire directory tree is fast and efficient.

LRU For Dummies

I needed a Least Recently Used data structure for some work I was doing so I went out to see what I could find in the open source world. I have used the LRUMap from Commons Collections before, but I was hoping to find one that makes use of generics. I checked out Google Collections, but they don't have any LRU data structures (yet). Fortunately, I check out the JavaDocs for LinkedHashMap on a hunch and was happy to discover that Sun already implemented an LRU algorithm in LinkedHashMap that can be enabled by setting a constructor parameter. They even provided a protected method that can be overriden in a subclass to create a fixed-size LRU map in fewer than 10 lines of code. The JavaDocs explain it all if you're interested.