​Controlling Pendubot with a Kinect
​(2015)
I wanted to take the Pendubot a step further and the robotics lab on campus happened to have a few extra Microsoft Kinects laying around. What better fun that controlling a robot by waving your hands! There were a few design considerations that had to be fleshed out however.
  1. The robot hardware was all setup with a computer running Matlab r2012b which makes implementing data acquisition for the kinect sensor a much more tedious process involving 3rd party solutions. Matlab r2013+ introduced much more simple implementations.
  2. I had access to a computer running Matlab r2013b, but conducting the data acquisition on a different PC meant communicating that information over a network.
  3. Assuming the network connection was operational in Matlab, there were still two functions that needed to execute simultaneously... 1) Simulink model controlling the robot... and 2) the client script receiving updated info from the server. Also the robot controller would still have to run lightning fast, whereas the updated commands from the server could come in relatively slowly (~20/second). Without being able to put the call in the control loop itself, the solution solution would have to involve two asynchronously parallel function. Of course Matlab doesn't lend itself to simple asynchronous parallelism. Although there are some workarounds.[I later discovered this was unnecessary, as the model was running externally and a simpler solution would suffice]


Drafting a Client + Server to Acquire + Retrieve Kinect Data

Designing an Asynchronous Parallel Client

After spending a great deal of time experimenting with asynchronous parallelism in Matlab (including getting into some java threading behind the scenes) it dawned on me that the controller was actually being run externally. I felt quite silly for wasting so much time but it was good experimentation nonetheless. In the end providing instructions to the model transformed into a dramatically simple task. To take care of the updates I used a "parameter set" command in the client that targeted a variable in the model and executed on update from the server.