Distributed Raytracer

This was an experiment in distributing computation across browsers connected to a website. Using a simple map reduce algorithm to split the raytracing of an image, and distributing the jobs to browsers via ajax, it was possible to render the simple test image in a parallel fashion.

At its peak, when a spike of traffic from Hacker News hit, images were rendered impressively fast.

Unfortunately, the code was inextricable from my website, which was written in django, and as I've since moved this site over to a new system, maintaining the demo was infeasible.

So for now, you will have to settle for raytracing an image on your own:

Original Content:

This is Distributed Ray Tracing over an Ajax powered Map-Reduce

Yeah! Buzz words!

But seriously, this is pretty cool. Raytracing is computationally intensive, as well as easily decomposable into discrete elements. Perfect as a benchmark for a distributed processing system.

Since reading about the concept of a browser based map-reduce a while ago, I'd mulled over the idea. After hearing that it had been used in The Engine Yard's hash cracking contest, I began to seriously consider the idea of crowd-sourced computation.

Javascript is getting faster and faster — there is no reason it can't become as fast as any other interpreted language. Although it's not ideal for computation, the fact that there are millions of nodes capable of running it with processor cycles to spare, means that it has serious potential for distributed computation.

The Internet is the biggest supercomputer you could ever hope to run your code on.

Implementation

This is a 2 day project. Meaning that for 2 days I've sweat caffeine as I typed. It's very much a proof of concept, and will doubtless break if too many people use it.

The Javascript is of course viewable. The small amount of server code is so unbelievably hackish and inefficient that it is probably best that you can't see it.

Features Include…

You'll notice that the resultant data is encoded pretty tightly. This is because I'm using mysql temporarily to store the data on the back-end. Yes this should be in memcache, or ideally in couchdb, but I haven't got round to that yet. As a result I had to pull some cool tricks like run length encoding the arrays.

The raytracer is of course my project from a few days ago. It's not changed much, but one of these day's I'm going to finish the anti-aliasing and add some more primitives.

I've disabled job spawning for now - the map-reduce backend interface is sufficiently generic that it should be fairly trivial to switch computations. There's a lot of kinks to work out first though.

Big Ideas

I'll probably keep working on this until something more interesting comes along. The backend needs a lot of attention — I'll release the code as soon as I'm not embarrassed of it. If somebody wants to implement protein folding in Javascript, then I'd love to do some meaningful computation with the thing.

Yep, like all my experimental, browser based demos, this won't work in crippled browsers like IE. Upgrade. Works very nicely in Chrome, a little slower in Firefox 3.5+, and fairly slowly in Safari.

    </li>

</ol>