Assistive Rocket Adventure
Game Development and Physical Fabrication
NYU IMA Project


Challenge: Design and develop a physical-therapy-oriented technology-based experience for a 5 year old boy with Cerebral Palsy.

Solution: Create an interactive physically-controlled game that reinforces ongoing physical therapy to keep his legs vertical while sitting.

Role: I was fully responsible for digital programming/design and helped with physical hardware. 3 student project team, with the other two focusing on the physical hardware exclusively.

Design Process

We began from the perspective of a five year old: what would we find entertaining and relevant at that age, and what limitations would that age provide on the complexity of the gameplay? It was from this discussion that we decided on many of the game dynamics:

  1. Have a '1D' movement system with no side to side avoidance or backwards sliding. 2D movement with the need for 2-leg independent movement would probably be too complicated for the child at the current moment, and backwards movement upon a lack of button presses would just be a frustrating experience.
  2. Make the graphics playful and colorful enough to be engaging to very young children, but not in an overly flashy way that could over-stimulate a child with sensory sensitivity.
  3. Divide the game into specific themed levels to give a sense of progress and to create reproducable goals for the child's trainer to ask them to reach.

After these discussions, we split the project into three different tasks: the programming of the digital game, the assembly of the physical interface, and the creation of a head-mounted system for reinforcing head-up posture through LED or game feedback. The first prototype presented was a basic tech demo of the digital game connected via arduino to simple buttons, and this is the feedback we received from the child's trainer and our classmates:

Feedback Cards


Taking into account these suggestions, I added more HUD elements for time and stage progress. I also overhauled the visual design of these elements to give them a more rounded and playful feel. Planets were added into the background to give a sense of progress and level design. Below is a video of a test run of the game with the internal logic sped up to better show the different levels.

Simultaneously to the work being done towards the digital game, there was also work towards a physical interface for the child. We were given an enclosure template which would allow us to make a base that can fit into the child's existing chair design. From this template we drilled out holes for the buttons and added a section to the bottom to house electronics.

The wiring was probably the hardest part of this stage. In order for the switch adapter to work properly, the buttons needed to be wired into a 3.5mm mono aux jack. There wasn't really much information available online for how to wire up the connection to the iPad, so there was plenty of trial and error to get to that stage. This is the wiring configuration that we ended up with:

Programming Diagram 2

And below are images of the in-progress and final construction. We decided on layered cardboard and tape for the final project not out of convienience but rather mostly becuase it's freely available around the ITP floor. We had to conform to strict budgetary requirements and wanted to cut down in any way possible to make it easier for the family to afford the product. Most of the Adapative Design Associations products are made out of the material for the same reason, though they were a bit better in construction quality than we were.

Project Wiring

Final Product

Taking into account the hardware interfaces, below is the diagram and code base behind that prototype. It is a Node server formatted with Johnny-Five to take data form an Arduino Uno connected over serial, then send that data to the client upon GET requests so that the client can handle the gameplay locally with JQuery animations and effects.

Programming Diagram

NodeJS Server (Hardware Input)
Client Javscript (Game Logic)
// Node Server Setup Code
// Module Requirements
var express = require('express');
var path = require('path');
var app = express();

var isLeftButtonHeld = false;
var isRightButtonHeld = false;

// Set public folder for client-side access

// Send index.html at '/'
app.get('/', function(req, res){
  res.sendFile(path.join(__dirname + '/views/index.html'));

//Send AJAX data stream at '/data'
app.get('/data', function(req,res) {
  // Compile individual variables into object
    var dataToSendToClient = {
      'isLeftButtonHeld': isLeftButtonHeld,
      'isRightButtonHeld': isRightButtonHeld

  // Convert javascript object to JSON
    var JSONdata = JSON.stringify(dataToSendToClient);

  //Send JSON to client

//Set app to port 3000

//Log start of app
console.log("App Started");

// Johnny-Five Code
var five = require("johnny-five");

board = new five.Board();

board.on("ready", function() {

// Button Initialization
  var leftButton = new five.Button("8");
  var rightButton = new five.Button("9");

// Left Button Setup
  leftButton.on("press", function() {
    console.log( "Left Button Pressed" );
    isLeftButtonHeld = true;

  leftButton.on("release", function() {
    console.log( "Left Button Released" );
    isLeftButtonHeld = false;

// Right Button Setup
  rightButton.on("press", function() {
    console.log( "Right Button Pressed" );
    isRightButtonHeld = true;

  rightButton.on("release", function() {
    console.log( "Right Button Released" );
    isRightButtonHeld = false;