1 2 Previous Next


27 posts

50.000 times too slow? Blog

Posted by herkules Jul 18, 2010

Recently I was in urged to do a web project with the latest and coolest web framework Ruby on Rails.

That's just one line of code grabbed from somewhere:

      next if entry =~ /^\./

Ruby is said to read like a natural language, but for my eyes this is just a cryptic sequence. Maybe I have the wrong natural language. So to clarify what it means I translated it to some assembly language:

      cmp (entryptr),'.'
      beq next

Yes: if the entry begins with '.', skip it.

The latter code might take 2 CPU cycles to execute.

How much will the Ruby code take? There is the line to be interpreted (syntax check, AST creation,... no idea), than a regexp engine is started which starts parsing the expression before it tries to find a match. All this is accompanied by lots of heap activity. My guess: 100.000 cycles. At least.

So it's 50.000 times slower than it needs to be to get the work done. Who needs a cluster of 50.000 machines to serve his application? Well, all others should be well of with a single server, even a poor one, if it only was programmed with care.

I know it's unfair and untrue, so please don't comment on that. But ... to some respect it is true.

How often have I read about cool new technologies where, after all, the programmer can now 'fully concentrate on the business logic'.

This meanwhile happens since at least 20 years so I wonder why still anybody does something else than concentrating on the 'business logic'?

One reason might be that it always has been a lie. Using any web framework still ends with tracking and analyzing HTTP requests or reading server logs, using persistence layers ends with monitoring the database, network traffic and such and so on. The promised abstraction just doesn't hold.

But - the main question is: Who - as a software developer - really wants to deal with business logic? Isn't business logic the most boring thing after all?  What is the business logic of a webshop e.g.? Displaying a list, maybe filtered, putting things of that list onto another list (aka 'shopping cart'), sending an email, telling the backend. +1 here, -1 there. Nobody studied computer science for that. Business is all about lists and simple calculations. Let's face it: business logic is boring. Period.

I cannot imagine that our brightest minds cannot come up with a suitable solution for a shop in 20 years. Each year we have at least dozens of new-and-cool frameworks to solve that list-thing. +1. -1. Once again. How stupid.

Nevertheless programming computers can be exciting. That's why people try to get rid of doing business logic and do frameworks and technology. That's by far more appealing. Just to get that funded, they claim that later some poor guy can 'fully concentrate on the business logic'.

When reading http://www.eclipsezone.com/articles/what-is-iadaptable/ I got the impression that certain solutions always come back, are quite universal and independant of domain, language or environment.
The question here is about getting certain aspects of an object that are not statically declared in a yet typesafe manner. The Eclipse world came up with IAdaptable, as it seems.

public interface IAdaptable {
  Object getAdapter(Class clazz);

public class HashMap implements IAdaptable {
  public Object getAdapter(Class clazz) {
    if (clazz == java.util.List.class) {
      List list = new ArrayList(this.size());
      return list;
    return null;
// ...

To be used like that:

IAdaptable adaptable = new HashMap();
List list = (List)adaptable.getAdapter(java.util.List.class);

Well, this immediately brought back good-old COM, Microsofts Component Object Model back to my mind. I alway felt comfortable with COM because COM objects are just as plain C++ objects and thus very fast.
The basics are the same. Every object is just a IUnknown (which transforms into an IAdaptable)

interface IUnknown {
  HRESULT _stdcall QueryInterface([in] GUID* rrid, [out] void** ppvObj);
  unsigned long _stdcall AddRef();
  unsigned long _stdcall Release();

For this is nothing useful, derived interfaces can be defined.

class IStandardMathFunctions : public IUnknown
  STDMETHOD(Add)(long, long, long*)
  STDMETHOD(Sub)(long, long, long*)

class IAdvancedMathFunctions : public IUnknown
  STDMETHOD(Fibonacci)(short, long*)

But despite static inheritance, every COM object can deliver all of its capabilities (which is, all the interfaces it implements) at runtime from the object using QueryInterface() which is pretty much the same approach as with IAdaptable.

Interesting enough, when I designed an abstraction layer for my pet project http://www.flyingguns.com describing the static 3D world, my solution was similar again.

public interface Feature

public interface SceneObject extends Disposable
   * Provide a certain feature if available.
  <T extends Feature> T getFeature( Class<T> clazz );

public interface Movable extends Feature
  void set( Vector3f position, Quat4f orientation );

public interface Damageable extends Feature
  void setDamage( float damage );
  void destroy();

public interface Aircraft extends Feature
  SceneObject[] getEngines();

  void setRudder();
  void setAileron();
  void setElevator();

The approach has a lot of advantages.

  • 100% typesafe, no casts due to generics
  • features can be combined deliberately per object
  • features may even change over time
  • makes delegation easy, good for code reuse
  • no runtime overhead
  • small footprint: resources for unsused features don't need to be loaded e.g. no fire and smoke if nobody wants to damage an object.

Sample code looks like that:

SceneObject fokkerobject = scene.create( DefaultDatabaseSetup.FOKKER_DR1 );

Movable fokkermover = fokkerobject.getFeature( Movable.class );
fokkermover.set( new Vector3f( 0f, 700, 0f ), new Quat4f( 0,0,0,1 ) );

WeaponSystem fokkerweapons = fokkerobject.getFeature( WeaponSystem.class );
SustainedFire machinegun = fokkerweapons.getWeapons()[0].getFeature(SustainedFire.class);


SceneObject bullet = machinegun.createProjectile();
Movable bulletmove = bullet.getFeature( Movable.class );
bulletmove.set( machinegun.getNuzzlePosition(), new Quat4f( 0,0,0,1 ) );

Projectile hitter = bullet.getFeature( Projectile.class );
Projectile.Hit hit = hitter.findHit( 50 );
if( null != hit )
       System.out.println( hit.getTarget() + " at " + hit.getLocation() );



In my famous company innoQ I currently have the opportunity to work on a real cool tool: Bundle-Bee. It claims to be able to take any OSGi bundle and distribute the computational load to an ad-hoc grid (e.g. all machines in an office) without special setup or configuration.

We just released version 0.5.3 which is still very restricted and far from feature complete - we don't even know what feature completeness might mean - but is already quite useful when it comes to doing computationally intensive things. Like fractals ... for example. Fractal computing is known to be computationally intensive and it may take long times to render the fascinating images. On the other hand, the nature of the problem gives a straight forward way to compute the 2D area in separated tiles and thus - in parallel. 

Bundle-Bee is about taking any OSGi bundle an distributing parts of that bundles computational code over a grid with (close-to) zero configuration. We did so e.g. on one of our company's meeting, where we ran the fractal computation on the attendees notebooks in parallel without any hassle. Just let them all start the Bundle-Bee equipped OSGi container (not a big deal) and they immediately make up an ad-hoc grid. Even one iPhone has been part of the game. The fractals jar is automagically flooded to the grid and execution load is split among all nodes.

Hence Bundle-Bee does the grid distribution transparently, there is no special notion of grid computing in the code at all. The brute-force Mandelbrot implementation is just like that:

public class MandelbrotAlgorithm {

   private static int iterate(Complex z0, int maxdepth) {
      Complex z = z0;
      for (int t = 0; t <= maxdepth; t++) {
         if (z.absSquared() >= 4.0)   return t;
         z = z.times(z).plus(z0);
      return maxdepth;

   public int[][] computeArea( Complex z0, Complex dz, int width, int height, int maxdepth ) {
      int[][] res = new int[width][height];
      for( int i=0; i<width; i++ ) {
         for( int j=0; j<height; j++) {
            Complex z = new Complex( z0.re()+i*dz.re(), z0.im()+j*dz.im() );
            res[i][j] = iterate(z, maxdepth);
      return res;

Just the main loop needs to go multithreaded which is always a bit tricky when dealing with an UI, but the pseudocode looks easy like that:

MandelbrotAlgorithm algo = new MandelbrotAlgorithm();

onUpdateRequest(area) {
  Collection<Tile> tiles = divideToTiles(area);
  foreach(tile : tiles) {
    runAsThread() {
      int[][] result = algo.computeArea(tile);


Letting Java2D fly a bit, a kind-of-nice UI is not far away:



Now, by just packing that code into an OSGi bundle (which basically means to augment it with a handful of entries in the MANIFEST.MF), it is ready to be run on a Bundle-Bee grid with nearly unlimited computing power. 


Sounds too easy, eh? But it actually is....





RMI .... cool somehow Blog

Posted by herkules Dec 31, 2009

Happy New Year folks!

In 2002, I wrote a highly specialized, very small Q&D tool for my brother to support him in his oncological doctoral's practice. They are three physicians sharing a common room with some unique ultrasonic device. They needed something showing them at their desk wether the room is available or is currently occupied by somebody else. The name 'dokma' is a German pun.


Admittedly not very pretty and I definately need to give it an UI overhaul.

Those days, I used RMI (still with 'rmic' to generate the stubs, supported by a NetBeans RMI plugin). The tool worked flawlessly since then. Yesterday, I did a small extension (allow for more physicians) and thereby upgraded to the new, stub-less RMI scheme.

While doing so, I remembered how cool RMI is. Today people tend to think of web technologies for everything that sounds like networked operation, starting with a tomcat as a bare minimum. But in this case, RMI is way easier and way cooler, for it allows to transparantly use callbacks for 'room-occupied' notifications and a tiny, integrated chat function.

     String serverurl = "rmi://" + servername + "/DokmaServerInterfaceImpl";
     mServer = (DokmaServerInterface)Naming.lookup( serverurl );
     mServer.registerCallback( mCallback );

Very hard and cumbersome to achieve with web techniques. And RMI comes with the JRE, so the whole application still fits into a 50kb jar file and zero installation and configuration effort.

A point worth discussing: is RMI still the way to go today? What do the experts say?





iRant Blog

Posted by herkules Nov 27, 2009
Do you know that feeling? Something annoys you and you have to tell somebody? This is the bare reason for this blog entry. Don't read it. It is just for me to make me feel better. When I joined my new company in January, they let my choose my weapons freely. For I've heard so much about these Apple machines and how cool they are and so far ahead of the Windows world, I was curious and ordered one for myself. MacBook Pro 17". Very new, very expensive. Must be great. As my grandma used to say (or was it Forrest Gump?): if you say something, try to say something kind. The display, the battery and the touchpad are great. Outstanding, excellent.  The rest was quite disappointing. First, besides the lack of some keys (PgUp/Dn....), many very important key labels are just missing (at least on the German keyboard). []{}~|\  ... try to find them. Very annoying on a coders machine. And why? My old, cheap discounter notebook had them. The aluminium case looks very stylish but unfortunately is quite thin-skinned and soon had a first bump which no longer looks so cool. But worse, the bottom edge is very sharp and tries to shave my wrist. Which might be part of the plan, for it often gets quite hot on the lower left side which might be easier to bear with if your skin is properly shaved. Software ... well. The darkest part maybe (I'm an UI guy, so maybe I'm too sensitive here). Try to find a svn client. You have to buy it or go back to the commandline which always gives me that nice feeling of the 80th ... spooky. The Finder needs to go back to usability lab. Even Mac-enthusiasts do say that. Explorer is far ahead! The basic desktop handling is clearty motivated by technical reasoning instead of by a users perspective. Use cmd-tab to switch process, than use cmd-shift-' (German keyboard) to switch applications windows ... what the f**k? Users deal with windows, they don't care for processes. What is the difference between closing, hiding and minimizing an applications window? Very subtle. I never found out.... And so on and so on ... Anyway, often it is very hard to get that far. Instead, I admire the spinning beach ball of death. This always seems to happen if MacOS runs out of physical memory. Which is ... always. Or the camera doesn't work any more. For that, Apple very frequently publishes system updates that require a reboot. I assume this is a trick... Finally, Java applications are somewhat different than they used to be on windows. They feel sluggish and less responsive. I use VirtualBox with Windows if I want to work with NetBeans e.g. (not a joke!). I don't say its a bad thing overall, it's ok, some things are really nice. The blog just emphasizes on the annoying parts. But next time I'll save my companies money, and will buy a common notebook for less than half. I have no idea where Macs reputation comes from but I suspect it is kind of reality distortion and wishful thinking. It's a linux for the rich. Did I mention that I also ordered an iPhone? :)

Head banging... Blog

Posted by herkules Sep 29, 2007
Are you doing engineering using Java? Or even science? Than you sometimes might need to calculate how objects move under           the impression of forces and torques. The following might be for you then...        
        Maybe somebody of you, dear reader, has already tried out my flight simulator           FlyingGuns which is part of the           Distributed RealTime Simulation project on SourceForge.           The flight model of motion is very simple.         I call it physically motivated instead of physical because it relies on some heuristics           that are not modeled using forces and torques. That is appropriate for a game but not for a true simulator.        
        There is no framework to plug in forces that are then correctly treated. This is the main reason why           the airplane cannot land, because there is just no mean to model the forces caused by the undercarriage on the ground.        
        In order to change this, I started to develop a simple dynamics framework.           It currently is located in the projects sandbox           because the API is not fully settled yet.           With this package, it is easy to define bodies that behave physically correct. This is a           piece of sample code to setup a 5kg body with gravity G and a Spring:        
Body body = new Body();

KinematicState state = new KinematicState();

CompositeForce f = new CompositeForce();
f.addForce(new G(body));

Vector3d springcenter = new Vector3d(0,0,5);
Spring s = new Spring(springcenter, 5 );
f.addForce( s );

Dynamic dynbody = new Dynamic(body, state, f);
                I call it dynamics stack to keep it apart from a physics engine like           ODEJava or JOODE.           A physics engine is a far more complex beast dealing a lot with collision detection           and - as a main task - computing the appropriate forces. But any physics engine           needs some kind of dynamics code under the hood.        
                There is no secret in how to do that. The math is known for hundreds of years meanwhile.           Yet some aspects, esp. those concerning rotational motion are highly unintuitive           (it took me 2 years to develop kind of mental image).         Having this thoroughly solved makes the value of this package.        


                Why is it a stack? Because on each level of the stack there is a set of classes implementing the interface of the next lower level.           Each level of the stack can be used on its own, omitting higher levels as desired.           These are the stacked components:                
  • intergrator
  • kinematics
  • dynamics
  • physics
  • application, e.g. flight model


                Integration has the task to solve ordinary differential equations (ODE). It gets some initial           values and an object that can calculate the resp. derivatives in time d/dt. The integrator has no idea           what the values do mean - it has no notion of e.g.  position or speed.        
        There are several well-known ways to do that with different quality and performance.           My package implements Euler and RungeKutta. While the algorithms are           not easy to understand, the resp. code is quite simple because the algorithms are very           well described in various textbooks           and there are many samples on the net.           Cash-Karp (see Numerical Recipes in C)           is currently under development.        
                The Euler integrator is the most simple one basically perfoming:        
                x_new = x_old + v*dt        
                Thats very fast, but only works if forces are small and do hardly ever change.           A common spring already may blow that approach.         RungeKutta is much more advanced but also takes (at least) four times more CPU power.                


                  Kinematics gives a meaning to terms like position or speed and can transform them to           the array of values needed by the integrator. It also respects that we are dealing with second           order differential equations (eg. position is order 0, velocity is order 1, acceleration is order 2)           and transforms them into a system of equations of first order:           
        One second order equation        
               position = f( velocity, acceleration )
                          makes 2x first order equation:           
               position = f(velocity)
               velocity = f(acceleration)
                  On the kinematics level, the interface Acceleration plays the key role. On this level,           there still is no notion of a phyical body having mass properties.                


                  Dynamics introduces the phyical body having mass and inertia. A class implementing           the kinematics interface Acceleration converts forces and torques acting on a           physical body into acceleration. While this is trivial for translational properties (just divide           by mass - F=ma - remember?), the rotational parts needs some consideration concerning world-           and body-fixed coordinate systems and the transforms between them.                 On the dynamics level, the interface Force plays the key role.                   


                  On this level, forces and torques are defined. It is not truely a level in the stack, but more           a collection of utilities. There are predefined forces like gravity G or a spring.           Conceptually harder is a class called RotatingPart which allows to model parts           within a body that do rotate themselves. Think of propellers or turbines as an example.           Again, this rotational things are not easy to deal with on an intuitive level.                


                Now how to test dynamics? I took two approaches. One is the classical unit test comparing the results of           the integration with a know analytical solution. Here is one taken from a textbook:

// problem 10-32 in my edition of Classical Dynamics by Marion+Thornton
public void spinningPlate() 
    Body b = new Body();

    // I1 = 1, I2 = 2*I1, I3 = I1 + I2
    b.setInertia(new Matrix3d(1.0, 0.0, 0.0, 0.0, 2.0, 0.0, 0.0, 0.0, 3.0));

    KinematicState state = new KinematicState();
    state.omega(new Vector3d(Math.sqrt(3), 0, 1));

    // with no forces and torques, w_body_y(t) should be |w| cos(a) tanh(|w| t sin(a)).
    // a = 30 degrees, so cos(a) = sqrt(3)/2 and sin(a) = 1/2.  the expected result is
    // thus sqrt(3) * tanh(t).          

    Dynamic d = new Dynamic(b, state, Force.NONE );

    double dt = 1.0/1000.0;
    double t = 0;
    for( int i = 0; i < 300; i++ )
        d.progress((long)t, dt);
        t += dt;
//          System.out.println(i + " " + Math.sqrt(3)*Math.tanh(t) + " " +  state.omega().y);
        assertEquals(Math.sqrt(3)*Math.tanh(t), state.omega().y, 0.05 );

    The other approach is piecewise comparison with a human judging from the visuals. E.g. assuming euler      integration works, I run the system with a RungeKutta integration. The results have to be      comparable. The same can be done to show that the calculation of acceleration from forces work correct      or that a rotating body behaves like one that does not rotate, but has a rotating part with same      inertia.    



                     Things to come (besides an application for a new flight model for FlyingGuns) is some           kind of exception handling in case of overstress. Sometime, forces may go beyond their limits           that e.g. may destroy the structure. This could be implemented using exceptions.                


The NetBeans source code editor is not known the be the ultimate one these days. From what I heard and saw, IDEA seems to be #1 in this respect today. But sometimes even small things have big effects and make work more enjoyable. Sandip Chitale created a set of linetools as a NetBeans module that nobody should miss. It gives a liteweight way to work with lines. It's very easy to move lines around or duplicate them (and then move around). The same works with multiline selections. Very handy, for it avoids prior selection of text in many cases.

Working with that reminded me of an editor feature I created in the glorious times when I was allowed to work in the  Forth programming language (my all-time favorite).

Those days I had a line- and a character stack. A single keystroke allowed to swallow or copy lines or characters to the stack and spit them out at another place. This was one of the features you'll never miss again once getting used to it. Much better than the common cut/copy/paste based on selections.

Sandip, please, can you help me (again)?

Filter Blog

By date: