Ambient Motion Painter

It began as Sky Painter

The first idea was narrow: the phone samples the colour of the sky and a computer paints brush-like strokes from it. Early camera tests showed the system responded meaningfully to any surrounding light — a lit room, a shadow on a wall, a bright screen — so the sky framing was dropped in favour of ambient colour.

That reframing opened the design space the rest of the project worked in, and eventually renamed the project.

Phone sketch

A p5.js sketch that samples every fourth pixel of a 160×120 video stream, averages the values into one ambient colour, computes brightness from it, and derives motion intensity from changes in acceleration magnitude.

Transport

One-directional publish–subscribe over MQTT: phone → broker → desktop, with a compact payload of averaged colour, brightness and motion intensity.

Desktop sketch

A second p5.js sketch that subscribes to the stream and draws one semi-transparent stroke per message onto a persistent canvas — the desktop is the canvas and the temporal memory.

Expressive rather than functional

Five decisions the material forced

The project developed through branching experiments rather than a plan. Five decisions did the structural work — each one arrived because the system behaved unexpectedly, not because it was scheduled.

01 — Beyond the sky

Colour sampling responded to any ambient light, so the concept broadened from Sky Painter to ambient colour mapping.

02 — Dropping particles

Particle systems behaved unpredictably and fought the calm, atmospheric quality I wanted. Committing to layered semi-transparent ellipses gave colour and motion something coherent to act on.

03 — Smoothing the accelerometer

Raw accelerationIncludingGravity data produced chaotic visuals. Interpolating between readings and constraining ranges turned motion from raw input into a material that could be shaped.

04 — Renaming the project

The prototype painted rooms, shadows and screens, not the sky. Ambient Motion Painter named what it actually was, and that clarified the interaction design.

05 — A clean export

Hiding the interface before saving — a two-frame rendering trick plus hiding the DOM overlays — made the output an exhibitable artwork rather than a screenshot.

let mag = sqrt(accX * accX + accY * accY + accZ * accZ);
let delta = abs(mag - prevMag);
motion = lerp(motion, constrain(delta, 0, MAX_MOTION), 0.15);
prevMag = mag;

Motion intensity is the change between consecutive acceleration readings, smoothed with lerp so the visual response stays expressive rather than chaotic.

Two minutes. The desktop canvas builds as the phone moves.

What it does not do

Averaging the camera image reduces detail: the system trades fidelity for atmosphere, deliberately, but it cannot represent. Motion input varies between people and environments, so responsiveness is not consistent.

There is no explicit state control. Painting begins as soon as camera access is granted, motion becomes active only once motion permissions are given, and the canvas has no reset or undo — every session is irreversible.

Next: multiple phones contributing to one shared canvas, richer brush models through shaders or textures, and exposing the mapping parameters during interaction so the expressive space can be explored live.