RIA

I was recently pointed to wonderwall’s new site, which uses some simple maths along with a bitmap distortion feature new to Flash Player 10 to create a visually interesting way to browse a portfolio. The site is an excellent example of how simple dynamics can elevate a design, and i’d like to go through the way it was (likely) built, to give insight into how little it takes to put together a visually complex system.

It all begins with the point

Most visuals can be described in some form in terms of points; be they in 3D space, on a 2D plane, or even along a 1D line segment. A point can be where something is located. At the lowest level, it can be where a particularly colored pixel is located. It can be in relation to two other points to describe a simple curve. Typical Bezier curves for instance consist of 4 points. A center point and a radius point can describe a circle.

But alone, with no strings attached, a point is simply a location with no purpose.

Here is a single point. Not much going on here!

However, consider that any layout is essentially a collection of points, or locations. For instance the position of an image

Any layout can be considered to consist of such points, describing positions for all layout elements.

Armed with rudimentary trigonometry, a point, being a simple position, can easily be transformed into other positions and, for the purposes of a dynamic system, over time.

For instance we can follow the mouse position (which is just another point)

Or we can escape it

With a little fidgeting, we can make combined behaviors, like this push/pull sort of behavior, where the point is forced away from its original position, but always attempts to go back to where it started.

Knowing where the point is now, and knowing where it originally was, we can calculate things like distances and angles to produce dynamic, interesting visual feedback to the point transformation.

For instance this variation, where the closer the point is to the mouse cursor, the larger its radius.

On their own, these behaviors can be fun. A notorious Flash effect in the late 90′s and early 00′s was to create socalled mouse trailers; simply graphic positions tracking the mouse as shown above. These simple systems are easy to put together, and with a few minutes of pen and paper sketching and a clear idea of what you want (and the occasional Google outing) nearly any conceivable interactive transform can be produced with a minimal amount of code.

Emergent behavior (lite)

Quoting Wikipedia

“In philosophy, systems theory and science, emergence is the way complex systems and patterns arise out of a multiplicity of relatively simple interactions“.

I’m absolutely in love with the term. What it’s referring to is what happens when you endow a simple object with a few rules for its behavior, make it somewhat aware of what’s going on around it, and then place a bunch of them together in the same context.

Let’s take our first couple of examples, and duplicate them a couple of hundred times.

Hmm, simply having every point follow the mouse isn’t very attractive or useful. Not on its own at least.

Now however, we’re getting somewhere. Pushing the points around looks almost like sand-painting

With all the points trying to keep a certain distance from the mouse, but also return to where they were, we get an effect almost like a sphere moving under a blanket.

By rendering our points with a little more information about their position, we enhance the sphere effect and get something approximating a 3D look.

A computer science classic of emergent behavior is Craig Reynolds’ Boids, a system where a few points each with a set of simple behavioral rules and environmental awareness can approximate bird flocking in a scarily accurate way. An excellent Flash example can be found here.

What can we do with our transformed points?

We established earlier that any element of a layout is, at it’s absolute most primitive, a location. It’s a perverse thought, but you could conceivably build a layout in which every single element’s position can be pushed or tugged by its relation to other elements, and have the whole shebang be affected by gravity, wind or fluid simulation.

When it comes to points; if you’ve worked out a good behavior for a point, the visual complexity of your system only grows with the number of points you introduce. And for every transformed point, you have a new position at which to place or deform graphics.

Particles

Points used in systems such as these are typically referred to as particles, and systems managing a large number of particles are typically called, simply, particle systems. Particle systems are used in video games and computer graphics to simulate effects such as smoke, fire, water, cloth, explosions, or any other visual effect involving a large number of graphics moving together.

Following a talk by Keith Peters at Flash on the beach ’08, i sat down and created what is called a verlet integration system. Verlet integration is, in short, a particle system where particles self-manage their movement based on comparing their current position to the one they last inhabited, as opposed to the traditional way of handling particle movement, which is to have a separate movement vector for every particle. It’s a very stable system used for its computational efficiency and for its constraints, which are relationship rules for how particles interact.

A typical constraint is one that tells one particle to always be the same distance from another. By chaining particles and these distance constraints, you can simulate a cloth-like structure such as this:

Notice the points and constraints forming squares? Hmmmm…

Wonderwall

To be perfectly honest, I can only guess at how they handle their constraints or behaviors, but Wonderwall’s system is in the end a collection of triangles described by associated particles.

Flash can easily take sets of such triangles and use them to distort images in a uniform way, allowing skewing, stretching, rotation or any such distortion.

Click here to see the previous cloth example used to distort a single image.

The work involved thus boils down to the following steps

1. Decide on the intended effect
2. Write a pushing/pulling particle behavior for a single particle
3. Plot the particles onto the canvas and associate them in such a way that they form triangles
4. Run the program and observe their collective behavior
5. Take the resulting triangles and use them to distort images

In terms of the actual programming for an intermediate developer with a little experience in particles, this is an absolute cakewalk, easily implemented in a day. The time sink is tweaking and adjusting the final look and feel.

Summary

My intent was to show that visually complex systems and behaviors can be built from small, simple ones that don’t require a huge effort to write. In the end, as with all development, it comes down to the specification and the desired end result. The idea.

My hope is that any front-end designers, interaction designers or concept developers working with Flash developers will feel less anxious about suggesting visually complex dynamic interaction after reading this.

I also hope developers reading this, in the rare case that they haven’t already fussed about with particle systems and emergent behaviors, will become inspired to learn more about what you can do with the most basic element of geometry and start playing. Things like bitmap distortion if you have no prior experience requires a little Googling and browsing the Actionscript documentation, but is in the end fairly straightforward.