Cloudlab / Benchmarking

This post formerly appeared on the Cloudlab Blog. Cloudlab was a SAAS product that failed to get traction and wound-up in June '12. It is reproduced here for posterity.

In trying to build a good replacement for other numerical processing applications, we've needed to benchmark the performance of cloudlab versus other places in which JavaScript can be run to process numbers. The following table lists the result of five successive runs of Benchmark.go

Browser 1 2 3 4 5 Avg Total
Win7 Chrome 16 8.44 8.53 8.33 8.59 8.44 8.466 42.33
Win7 Safari 5.1 11.51 10.15 10.19 10.27 10.51 10.526 52.63
Win7 Firefox 9 116.49 110.00 109.51 109.50 110.70 111.24 556.20
Mac 10.7 Chrome 16 8.15 8.08 8.11 8.12 8.11 8.114 40.57
Mac 10.7 Safari 5.1 7.04 7.08 7.04 7.05 7.06 7.054 35.27
Jurassic (2.2) 566.12 563.16 563.31 566.12 565.50 564.842 2824.21
var Benchmark = {
  js: function() {
    'use strict';
    var _sts = (new Date()).getTime();

    var arr = [];
    for(var i = 0; i < 1000; i++) {
      arr[i] = i;
    }

    var _sum = 0;
    for(i = 0; i < 1000; i++) {
      for(var j = 0; j < 1000; j++) {
        _sum += arr[j];
      }
    }

    var _ets = (new Date()).getTime();
    return (_ets - _sts);
  },
  go: function() {
    'use strict';
    var _jsTotal = 0.0;
    var _start = (new Date()).getTime();

    for(var i = 0; i < 100; i++) {
      _jsTotal += this.js();
    }
    return {
      'Javascript': _jsTotal/100,
      'Total': (new Date()).getTime() - _start
    };
  }
};

No real surprises here, although we expected more from the latest Firefox. Now, if instead we use cloudlab's built-in Vector type with it's native-backed math, we get a different result:

Graph Results

This graph represents 250 runs of Benchmark.go using the native math provider we've hooked-up to Jurassic. Performance is variable, but excluding the two outliers, it ranges from as fast as the browsers to ~33% faster. So it's mostly faster (we're working on that), but you also get some awesome math:

native: function() {
    'use strict';
    var _sts = (new Date()).getTime();

    var vec = new Vector(1000);
    for(var i = 0; i < 1000; i++) {
      vec.v(i,i);
    }

    var _sum = 0;
    for(i = 0; i < 1000; i++) {
       _sum = vec.sum();
    }

    var _ets = (new Date()).getTime();
    return (_ets - _sts);
  },

Summary

We love math and Javascript sorely needs more of it. We're hoping to really make hard math easy and let you process 'Big Data' using the tools of the web.