Future Image Sensors

Developments in 2012: The advance I describe here is starting to occur! The upcoming generation of sensors from Sony will move the readout circuitry behind the light detector, under the name “stacked CMOS” (a term I only just learned). One generation after that, there should be room for the digital accumulator too, and the shutterless camera design that I describe here will become possible. It should take 5 years at most, allowing time for other needed developments to occur. I’m calling it for 2017 or before!

A sudden revolution in sensor technology could occur when imaging chips have room for a fast A/D converter and a digital accumulator at each pixel. That seems as though it ought to be easy with three-dimensional circuits by putting the extra gear behind the sensor proper, and it may be possible sooner (it depends on hardware stuff that I don’t understand). With that, cameras can switch to a simpler and more capable shutterless design.

Instead of accumulating the image through the entire exposure, each pixel will accumulate light for a short time, read it out and add the value to the accumulator. In effect, the sensor is digitally adding up short exposures to make a long exposure. (Or similarly, with short enough minor exposures, use an “it’s full” counter instead of a proper D/A. In the limit, it’s a photon counter.) The short exposures might be synchronous across the sensor or independent for each pixel. Advantages:

1. There is no need for a shutter. To start the exposure, zero the accumulators (or simply read out their initial values). To finish it, electronically stop accumulating (or read out the final values and subtract).
2. Dynamic range can be extreme, because it depends on the number of bits in the digital accumulators rather than on the capacity of the analog sensor well. In other words, overexposure will be history. If the sun is in your frame, you can count sunspots (provided the short exposures are short enough and there are enough bits).
3. The camera becomes in essence a high-speed always-on movie camera. Cute features are possible with extra hardware. With two accumulators per pixel, you can have the shutter button decide when the exposure ends rather than when it begins. The next step is to have it adjust for your slowness and choose the right moment after you recognize it. With communication between the pixels, you can do image stabilization electronically in real time.

Astronomers already make images by stacking. Adding hardware support for everyday use is obvious to me, but then, my sense of obviousness is skewed, so who knows. I haven’t seen this idea elsewhere, but I’d be surprised if it were absolutely new.

I’m not holding my breath. I think the implementation should be easy with 3D chips, but my impression is that 3D chips are far off. Lithography is naturally flat, and all the proposed near-term replacements that I’ve seen are also flat. I suspect that the economic tradeoffs support shrinking over layering for years to come. Ideas like self-assembly and nanomechanical assembly support 3D designs, but they won’t be possible any time soon.

Original version, June 2009. Added here March 2010. Last updated June 2012.