Back in 2012, technology websites were abuzz with news of the Lytro: a camera that was going to revolutionize photography thanks to its innovative light fields technology. An array of microlenses in front of the sensor let it capture a 3D image of a scene from one point, allowing the user to extract depth information and to change the focus of an image even after capturing it.
The technology turned out to be a commercial failure however, and the company faded into obscurity. Lytro cameras can now be had for as little as $20 on the second-hand market, us [ea] found out when he started to investigate light field photography. They still work just as well as they ever did, but since the accompanying PC software is now definitely starting to show its age, [ea] decided to reverse-engineer the camera’s firmware so he could write his own application.
[ea] started by examining the camera’s hardware. The main CPU turned out to be a MIPS processor similar to those used in various cheap camera gadgets, next to what looked like an unpopulated socket for a serial port and a set of JTAG test points. The serial port was sending out a bootup sequence and a command prompt, but didn’t seem to respond to any inputs.
Now able to send commands directly to the camera’s CPU, [ea] wrote a Python library and a set of tools to operate the camera remotely, and enabled several new features. The Lytro can now function as a webcam, for example, or be operated remotely with full control over its zoom and focus mechanisms. All those functions can be accessed via the built-in WiFi interface, so there’s no need to solder wires to the CPU’s serial port.
With the low-level functions now out in the open, we’re curious to see what hidden potential there is still in Lytro’s technology. Perhaps these cameras can be repurposed to make more advanced 3D capture systems, similar to Google’s research presented in 2018. If you need a primer on light field technology, check out Alex Hornstein’s presentation from the 2018 Supercon.