Gyro Cube

This week I started experimenting with incorporating some physical interaction into an augmented reality experience. I'm still not 100% sure what I'll do for the midterm next week, but something that makes use of bridging the physical world with augmented reality. Going for a bit of a reversal from last week where I went from the AR world to changes in the physical world. For this experiment I used a gyroscope to send rotational data to cubes in a scene, and when a user taps on one of these cubes it toggles an LED light on and off.

Building off the Node server I created last week, I used Socket IO and its associated Swift package to communicate between the microcontroller, the local network, and the iOS app. To trial some different types and directions of data exchange I wired up both an IMU (to get gyroscopic rotational tracking data) and an LED to the Particle Photon I have. I set up a Node server to communicate with Particle's web API. I also set it up to create a Socket IO server on the local network to talk to wirelessly the phone. It's not too important as it's not swift code, but here's what it looks like:

const board = new Board({
  io: new Particle({
    token: process.env.PARTICLE_TOKEN,
    deviceId: process.env.PARTICLE_DEVICE_ID,
  }),
});

// Init Node server
const app = Express();
const server = HTTP.createServer(app).listen(PORT, () => {
  console.log(`📮 SocketIO listening at ${ip.address()}:${process.env.PORT}`);
});
const io = Socket.listen(server);

// =================================================
// Johnny Five logic
board.on('ready', () => {
  let socketClient;
  const imu = new IMU({
    controller: 'MPU6050',
  });

  const led = new Led('D2');

  io.on('connection', (socket) => {
    console.log('Socket connected!');
    socketClient = socket;
    socket.on('tap', () => {
      led.toggle();
    });
  });

  imu.on('change', () => {
    if (socketClient) {
      socketClient.emit('gyroData', [
        imu.gyro.pitch.angle,
        imu.gyro.roll.angle,
        imu.gyro.yaw.angle,
      ]);
    }
  });
});

Now for the Swift code. There was surprisingly little that I needed to change to get this to work, just needed to get the gyro data, map it to radians, then iterate through all the cubes in the scene and apply it as a transformation.

override func viewDidLoad() {
        super.viewDidLoad()
        
        let socket = io.defaultSocket
        
        socket.on(clientEvent: .connect) {data, ack in
            print("socket connected")
        }
        
        socket.on("gyroData"){data, ack in
            self.gyroData = data[0] as! [Double]
            let gyroDataRad: [Float] = self.gyroData.map { Float($0 * (-.pi /  180)) }
            
            self.childCubes.forEach{cubeName in
                let cube = self.originAnchor.findEntity(named: cubeName)
                cube?.transform = Transform(pitch: gyroDataRad[0], yaw: gyroDataRad[2], roll: gyroDataRad[1])
            }
        }
        
        socket.connect()
...

The only change elsewhere in the file is that I used an array to keep track of the names of all the cubes so I could iterate through them upon receiving data.

self.childCubes.append(goldBox.name)

And sending out an emit event when a box is tapped:

if let hitEntity = self.myARView.entity(at: touchInView) {
            print("➡️ TAPPED A ENTITY:", hitEntity.parent!.name)
            
            io.emitAll("tap")
...