 | |||||||||||||||||||||||||||
|
A transient camera system based on the PMDTechnologies CamBoard nano plus a custom light source. In this document, I will be collecting information to help you build your own capture setup. There is an infinite number of paths to the destination, one being, of course, reading our SIGGRAPH 2013 paper and giving someone with mixed-signal, RF expertise a year or so to build a well-engineered system using the information provided and a sensor datasheet. Our own solution originates from a reasonably good connection to the PMD Technologies company (good enough so they would share datasheets and sell us nonstandard filterless chips; not good enough so they would build custom hardware, firmware, and drivers), along with some experience in digital electronics and endless amounts of tinkering. | |||||||||||||||||||||||||||
IntroOur work draws inspiration from the "Femto-Photography" direction of work out of the MIT Media Lab (here's an alternative, much nicer project page from their collaborators in Zaragoza). Fascinated by their ultrafast optical impulse responses of real-world scenes, we set out to capture similar data maybe at a reduced resolution, but at a much lower price point and in a more practical way. Our device of choice are time-of-flight imagers that capture a coded version of the desired response. By combining a multi-frequency capture with a numerical reconstruction process (solving a linear inverse problem), we showed that such devices can in fact be used to film light in flight. Our results were presented at SIGGRAPH 2013, both as a Technical Paper and a live demo at Emerging Technologies. At CVPR 2014, we showed how basically the same setup (with just a few collimation lenses added to the laser diodes) can be used to look around corners.This document aims to provide you with the information needed to recreate our system, or one "like ours". Our system costs less than $1000 in parts to build, that is, if you don't screw up. I'm not formally trained in electronics, but I did start into the project with several years of experience running a little business, plus a supervisor who basically gave me carte blanche for breaking stuff. We must have burned about $6000 until the first fully working setup. The instructions given below are based on my experience after building roughly 12 cameras across 4 hardware revisions (9 of those cameras are actually operational). I've developed a certain confidence in the process - you will see that some of the steps take a bit of a daredevil attitude to perform. DisclaimerThis is an attempt to document procedures that are to some degree un-documentable. Treat this page as work in constant progress. If in doubt about a certain step or about the big picture, e-mail me (hullin at cs dot uni dash bonn dot de).High-level requirements and available hardwareWe will assume that you are familiar with the basic principles of time-of-flight imaging (CW/periodically modulated, not pulse-based) if you read this document. In order to turn a ToF camera into a transient imager, the following basic requirements have to be met:
None of the systems can be used to capture transient images as they come off the shelf; either because the hardware doesn't allow it or because the API is locked down. Solutions range from (1) asking your favorite contact at Microsoft to measure data for you via (2) hacking existing hardware/software to do what you need to (3) developing the whole system from scratch. Given the current situation regarding off-the-shelf products, solution (3) would be the cleanest if you invest an EE staff for a year or so, full-time. We opted for (2) and implemented it in a modular way. Our camera, based on the PMD CamBoard nano, also contains the signal generator that produces modulation signals for sensor and light source. Our light source is a bank of laser diodes with inputs for RF modulation (BNC/50 Ohm), power and trigger signal. Recreating our systemStatus quo and solution approachThe CamBoard nano is a reference design for the PMD 19k-S3 ToF sensor that runs off the USB port and is reasonably affordable. Unfortunately, it can't do some of the things we need for transient imaging. We need to be able to change the modulation frequency, and we need to set arbitrary relative phases between illumination and reference modulation signal. The nano modulates at a fixed 25MHz (way too slow) and takes phase images in 90° steps (way too coarse). Even if we had access to its FPGA to change the modulation frequency and phase, the LED would still cut off at around 25MHz (above which the modulation contrast goes down).
Materials needed
Building the electronicsAs always, three options:
Self-assembling the circuit boardsOrder PCBs and solder paste stencil, as in the EAGLE files provided below, from a circuit board manufacturer. I use PCBPool, who are not the cheapest but none the more professional, quite fast, they are prepared for EAGLE files and they offer free steel stencils. You will need 1x cameraboard, 1x laserboard_mother and 4x laserboard_square. Get some more of each to allow for breakage. Specs for cameraboard and laserboard_square are 4 layers, 1.6mm thick, 0.2mm min. drill, 4mil lithography. For laserboard_mother, 2 layers, 1.6mm thick, 0.3mm drill, 6mil lithography. Don't be shocked - a full set of boards may end up costing USD 300-400 - they get drastically cheaper in large volumes. If your board house doesn't accept data in EAGLE format, you WILL need experience in CAM (Gerber) export, since 4-layer designs are rarely used by hobbyists for obvious reasons. Maybe there are tutorials for 4-layer Gerber export from EAGLE somewhere out there, though.Squeegee solder paste across steel stencil (alignment marks fit nail-shaped wire pins of type Mill-Max 9083-0-00-15-00-00-38-0), and use tweezer to populate SMD parts as indicated in bill of materials. Reflow solder in toaster oven - observe closely and turn off oven when all solder has molten across the board. If you have never done SMD assembly by hand, search the web for tutorials. Some of the parts used by us are fine pitch (0.5mm) -- a steady hand will definitely be required for this.
Firmware installation and testing
Assembling and adjusting the light source
Modifying the CamBoard nanoIf the above wasn't tricky enough for you, this is the chapter where you can show what you've got. We start by removing the CamBoard from its aluminum mount using the Torx drivers. Clip off the USB cable using side cutters (before you do, take note of wire colors), and unconnect the LED from the flex board extension. By default, the nano comes with a board lens that is glued on. Carefully soften glue with a heat gun; gently pull off lens. To remove residual glue, repeat heating and scrape off with a wooden toothpick until the sensor is clean on all sides. Yes, this is as little fun as it sounds. Now take a closer look at the sensor from the "north" side, as seen above. You will find that it is in itself a printed circuit board with the silicon die sitting inside a recessed area and covered by a IR-pass window. (The tiny capacitors probably serve to filter some of the supply voltages.) You can see that most pads of the package are connected to the top through tiny vertical connections, so-called vias. Pin 1, the modulation line, is marked with an arrow.  The drawing above illustrates a cut through sensor package (top) and CamBoard circuit board (bottom) at Pin 1 of the sensor. Red(dish) parts are conductive and carry the modulation signal from the source through the solder pad of pin 1 (3) and a via (2) to the top of the package (1). Points 1-3 are the only ones where the signal can be reached; it is covered by sensor and FPGA elsewhere. Use the cutter knife to carefully scratch off the black varnish layer above point (1), until you see the bare copper. Using the continuity tester, check that point 1 and 3 are electrically connected. Now, with a steady hand, start drilling into the package from the side (2) to interrupt the modulation line. Make sure not to go any deeper than 2mm, and stop immediately once the connection between 1 and 3 is interrupted (no more beeping from the continuity tester).  
Files(firmware and capture tool will be posted once someone starts a serious attempt to build a system)
CitationIf this information turns out useful for your research, I would appreciate if you cite this page:
@misc{Hullin2014,
Author = {Matthias B. Hullin},
Year = {2014},
Note = {http://pulsr.info},
Title = {Building a Transient Camera}
}
|
In short, our solution is
If you are lucky, your modulation board should now look roughly like above. Give it a thorough round of beeping with a continuity tester, to make sure there are no shorts.
As the very last step, get the soldering iron and add the through-hole parts. The male 2x6-pin connectors are attached to the side of the laser boards in a somewhat unorthodox way.
You need