1 2 3 Previous Next

forax

101 posts
Hi everyone, This is my last post on java.net, I've decided to move my blog to github: http://forax.github.io/ I hope to see you soon, bye.  

Java 8 will introduce (or introduces it depends if you have already played with the betas or not) a new full featured feature, lambdas. Instead of explaining the syntax, the semantics, how it's integrated with java.util, yada yada. Let's take an example and see how lambdas can be used by users and developers of APIs.

Let suppose I want to design a JUnit/TestNG replacement, basically these two libraries are designed to take an existing code and write unit tests to the code. I want library which is designed in the opposite way, the Test Driven Development way. I want to first write tests and use them as a specification to write code. Note that this idea is far from new, it's basically a rip of Ruby's RSpec in Java.

JaySpec

So let me introduce JaySpec, which allows to describe executable tests that can be used as specification. In language like Scala or Groovy, the parser is enough flexible to be able to parse that text and see it as code, so they are able to understand/execute code like

describe ArrayList
  given an empty list
    ArrayList<String> list = new ArrayList<>();

    it should have a size == 0
      verify that list.size() is equals to 0

In Java, even with the lambda syntax you need to add extra characters. Here is the equivalent code in Java 8

describe(ArrayList.class, it -> {
  given("an empty list", () -> {
    ArrayList<String> list = new ArrayList<>();

    it.should("have a size == 0", verify -> {
      verify.that(list.size()).isEqualTo(0);
    });
  });
});

Ok, less readable but not that bad :) 'describe', 'given' and 'should' are methods that takes a textual description and a lambda that correspond to the executable part of the spec. The lambda acts as way to delay the execution of a code, so the code above is seen as a tree (with a node 'describe' that contains a node 'given' ...) by JaySpec. In term of vocabulary, for the rest of this text and for JaySpec, 'describe' defines a spec, 'given' defines an example and 'should' defines a test.

The expressions that you can use after verify.that() is the one provided by the fest-assert library (I don't want to re-invent the wheel here), which is a nice library (at least the 2.x version, I've not taken a look to the older versions).

I've cheated a little by saying that the code above was a Java code because I've omitted the class declaration, so the real executable Java code for our small example is

public interface ExampleTest {
  public static void main(String[] args) {
    new JaySpec() {{
      describe(ArrayList.class, it -> {
        given("an empty list", () -> {
          ArrayList<String> list = new ArrayList<>();

          it.should("have a size == 0", verify -> {
            verify.that(list.size()).isEqualTo(0);
          });
        });
      });
    }}.run();
  }
}

I use an interface instead of a class at top-level so nobody can create an instance of ExampleTest (you can have public static method in interface in Java 8). I abuse of the inner-class syntax (hence the double mustache after new JaySpec()) to be able to use 'describe' or 'given' as if they were functions. There are in fact of methods of the class JaySpec.

What JaySpec does if to take all the tests (the one that starts with given), execute them in parallel (using fork-join) and in this case write the results in a terminal (the code the execution of the tests and the output are separated so JaySpec can be embedded in any UI easily).

Here is an example of run of a similar but a little more complex code (see ExampleTest.java)

class java.util.ArrayList
  given a list of one element
    it should have a size == 1
    it should get the item at index 0
    it should not return a valid index for a different item
  given an empty list
    it should have a size == 0

The cool thing is that JaySpec.java is one only file, containing less than 200 lines of code (with no comment). Because the is not a lot of codes that use lambda out there (at least at the time when I write this lines), I suppose that it a good idea to explain a little bit the code of JaySpec.java.

JaySpec does 3 different things, the first one is to create the tree describing what to execute, the second one if to run all the examples (in parallel, we have several cores on our computers now) and at the end, the result (the report) needs to be collected and organized by spec and by example.

Creating the tree

Technically, we don't need to maintain the forward links between a spec and all the examples of the spec but the only the backward links from a test (the things that starts by "it should") to its example and from an example to its spec. Unlike in Ruby, there is no scope object in Java, so the standard way to emulate dynamic scope object is to use thread local variables.

Running the examples in parallel

This part is really easy because JaySpec uses the new java.util.stream API, so the code is just:

examples.parallelStream().flatMap(example -> { 
    ArrayList<R> reportList = new ArrayList<R>();
    // execute each test of the example
    return reportList.stream();
}).collect(Collectors.toList()); 

That takes all examples, from the previous step, and for each of them in parallel, it will generate several reports (one by test defined in the examples) that will be gathered in a List. The underlying implementation of the parallel stream will use a fork-join to distribute the execution of the tests on all available cores. Note that each test for one example will be executed sequentially so JaySpec can test code that use objects that are not thread safe.

Gathering the reports

Once the reports are all created, in parallel, they need to be grouped first by example and then by the spec of the example, again this is something easy to write with the new stream API:

Map<Spec, Map<Example, List<Report>>> map = ... .stream().collect(
    Collectors.groupingBy(report -> report.getExample().getSpec(),
        Collectors.groupingBy(Report::getExample)
    ));

The Collector API is a really powerful but complex, I guess that i will take a little more time for people to use it than the other parts of the Stream API.

And at the end, the result of each report is printed using the newly introduced method Map.forEach which is far easier to use that using a for loop on an entrySet as we used to do before Java 8.

map.forEach((spec, exampleMap) -> {
  ...
  exampleMap.forEach((example, reports) -> {
    ...
    reports.forEach(report -> {
      ...
    });
  });
});

The last calls to forEach can be also written using a for loop, but I have not yet determined if the lambda syntax should be used or not in this case.

cheers, Remi

Wow, I should have blog more this year. I was too lazy and fairly busy. Let say I will write more posts next year. Anyway, I will visit the nice city of Lille next Tuesday evening for a session about Lambdas and Java 8. I will post a translation of my slides in English soon after. If you want to attend in person, the registration is free :) http://chtijug.org/session-java-8-avec-remi-forax-le-11-decembre-2012/ cheers, R

Hi all,
I have three talks scheduled at the beginning of this year.

The first one at FOSDEM in the Free Java Devroom is titled Java Lambdas for Hackers. It will explore the details of what can be the implementation of the lambda in Java 8
The agenda of the Free Java sessions this year is great, a kind of perfect mix between old friends and newcomers.
http://fosdem.org/2012/schedule/track/free_java_devroom

The two other talks are in French.
The first one, Java.next() at the Paris Jug hosted by Julien Ponge and Alexis Moussine-Pouchkine, I will also talk about lambdas but more from the user point of view that from the VM side.
http://www.parisjug.org/xwiki/bin/view/Main/WebHome

And the last talk will be at Devoxx France
http://devoxx.fr/display/FR12/Speakers
where I will show how you can implement your dynamic language on top of the JVM.

C U soon,
Rémi

Sometimes I want a express that a static field is unlikely to change, so the VM should consider it has a constant,   but it may changed.   And because we all live in a threaded world, if the static field is changed   I want that all threads that want to read the field value to notice that the value has changed like   a volatile field. Take by example the default Locale that you can obtain using Locale.getDefault(),   it's unlikely that this value will change, but it may change, there is a Locale.setDefault().

The idea is that default Locale is mostly a constant, so  when a code that call getDefault is JITed, the VM should consider it as a constant i.e. the JIT should replace the call to getDefault() by a direct reference to default Locale object.  And if a code calls setDefault(), the VM should drop the JITed code and re-optimize it later.

Suppose that you have a magic class named   AlmostFinalValue,   a value wrapped in this class acts as a constant for the VM, so a code that call   getDefault will be replaced by a constant by the VM.   And if setValue() is called, the VM will deoptimize the code that use  getDefault if   the code was JITed and change the value of the default locale.   If the code still use getDefault, it will be re-optimized by with the new constant. 
Note that this deopt/reopt takes time so it's not a good idea to use an AlmostFinalValue   for storing a value which changes frequently.

public class Locale {
  ...
  private static final AlmostFinalValue<Locale> DEFAULT_LOCALE =
      new AlmostFinalValue<Locale>() {
        @Override
        protected Locale initialValue() {
          return new Locale("default");
        }
      };
  private static final MethodHandle DEFAULT_LOCALE_GETTER =
      DEFAULT_LOCALE.createGetter();
  
  public static Locale getDefault() {
    try {
      return (Locale)(Object)DEFAULT_LOCALE_GETTER.invokeExact();
    } catch (Throwable e) {
      throw new AssertionError(e.getMessage(), e);
    }
  }
  
  public static void setDefault(Locale locale) {
    DEFAULT_LOCALE.setValue(locale);
  }
}

 

How AlmostFinalValue works ?

All modern VMs, V8-cranksaft (Javascript/Google), IonMonkey (Javascript/Mozilla), Hostspot (Java/Oracle), J9 (Java/IBM)   are able to de-optimize a JITed code if an optimistic assumption doesn't hold anymore. 
In Java, we go a little bit further, JSR 292, foolishly you may say,   gives access to VM optimization/deoptimization to the mass* by providing a silver bullet named the   SwitchPoint.

A SwitchPoint can be seen as a volatile boolean which is true by default and that can be switched off once.   On a SwitchPoint, you can create a guard, an 'if' statement seen as a method handle** that takes two method handles as parameter   and execute the former if the boolean is true or the later otherwise. The JIT optimizes a SwitchPoint by considering that the boolean value is unlinkely to change so it will   remove the guard to always call the first method handle and if the SwitchPoint   is switch off, the VM will deoptimize the JITed code. In fact, in Hotspot, this is done lazily the code is just marked   as must be dropped when it will be called. So later, when code will be called again,   the VM will call the second method handle. The second method is the slow path, the path which is infrequenly called when there is an update,   it get the new value, see it as a constant method handle and protect it with a new SwitchPoint.   Then this new method handle is installed as the new target of a mutable method handle (a MutableCallSite + a dynamic invoker in fact).   So the new constant is installed and will be called when the DEFAULT_LOCAL_GETTER will be called. The method hande DEFAULT_LOCAL_GETTER is called using invokeExact which calls the method handle with no runtime conversion. As a game, I will offer a free beer at FOSDEM or Devoxx France to the first that will explain why you need the two casts in front of invokeExact ***.

The full code is here, and works well with latest jdk8 and jdk7u2.   With jdk7, the SwitchPoint was not fully optimized so the VM will fail to transform the default local value to a constant.

* In fact, JSR 292 was intended to be used by developers of dynamic language runtime (Jython, JRuby, Groovy, etc)   but when you provide a golden hammer, everybody will want to use it. 
** A MethodHandle is runtime typed function pointers.
*** People that have written the JSR 292 spec can't participate, sorry John, Dan and Fredrik :)

forax@univ-mlv.fr

AsyncSwing Blog

Posted by forax@univ-mlv.fr Oct 1, 2011
forax@univ-mlv.fr

FOSDEM 2011 Blog

Posted by forax@univ-mlv.fr Feb 7, 2011

Filter Blog

By date: