Low-latency web to serial bridge

I am going to use a Zigbee based wireless control mechanism for my robot having drive-and-steer capabilities. But that's a lot of overhead and a significant deviation from scalar & vectoral control algorithm that I am focusing on. Is there a solution that can offer me similar features without going to the lengths of developing the hardware and firmware for it?

  • Input: Parameters for scalar & vectoral control algorithm.
  • Output: Serial commands to the Arduino board (hardware controller) to drive the motor controller.

I already have a Raspberry Pi on board. I can leverage it's horsepower to host a web server to interact and collect user input for the control engine and I can use the serial port on board as an output and connect it to Arduino board.

Of course this would be high-latency solution compared to a dedicated Zigbee for the same purpose but since the web server won't be going out of my LAN and there will be at the most 1 client, this can work.

I can serve a php page to the user and then make AJAX calls back to the server which can transport data directly to hardware layer via something like php-Serial library.

But I do want to make sure that there is a probing point on the data path for debugging and analytics. On the Zigbee implementation, the idea was to let Raspberry Pi probe the serial lines for incoming wireless commands. This probing can take place via a local SQL database which is good for offline analytics because of data persistency. But going through a database can introduce latency and again, analytics is not the primary goal here.

So, it is safe to replace the database as a probing point to something way simpler. I will do with a dedicated directory which will serve as a bucket between web server and hardware. The php scripts will dump the commands into this bucket.

Ok, let's cross the bridge.

Now that we have a bucket-full of commands, it is easy to come up with something that parses the commands, formats it the way the Arduino hardware controller expects it and ships it out. This being a Raspberry Pi board, the apparent and wise choice to do this is in Python.

All right, let's put all this in a picture. A picture something something words something something.

In terms of improving to reach my final Zigbee goal, these are the iteration cycles marking major milestones:

  • Iteration 0*: Implement directory based web-to-serial bridge on Raspberry Pi
  • Iteration 1: Replace out directory based probing with a SQL database-based probing.
  • Iteration 2: Replace web-to-serial bridge with RF Zigbee bridge.


Instead of collecting user input from a plain form input, let's choose something dynamic to stress out the bridge. Here is my choice:

  • Transmit the mouse pointer coordinates as (x,y) pairs; where (x,y) ∈ [0,255].
  • Enable transmission on mouse down (left click press & hold). Disable transmission on mouse up (left click release)

Building further:

  • Layer0: The javascript provides all the client side processing to format data in the format compliant with the server's expectation (php).
  • Layer1: This data is sent over to server via AJAX calls where a php script parses it and dumps it in the data bucket in the format compliant with the next layer.
  • Layer2: A python script waiting on data drop in the bucket picks up the incoming data, parses it and sends it out via onboard serial port of Raspberry Pi.

Observations and issues:

  • The data bucket needs to be a FIFO bucket.
  • If there is no timestamp on the data, the python script should also ensure that it sends data out to serial that is relatively new. Additionally, it can also notify upstream that data was dropped in case it chooses to NOT send out data that got stale.
  • Some of the design of layer2 is dependent on how the receiver on the the side of serial port looks like. If it is a high-speed serial bus and data processing is not an issue, then the inclination would be to let it decide what to do with the data packets bearing timestamps. On the other hand, if it is a simple controller, then heavy lifting should be done by python script as noted in previous point.

In Action

References & Further Reading:

Related but unrelated:

  • Because, counting starts as 0
  • In case you end up using a database as a probing tool, please make sure you are using a good quality database sanitizer: