Optics in Computing at IOTA
Parallel Processors for real-time image processing
Groupe Physique des Images directed by
French résumé available here
Teamwork : A.Cassinelli,
Ph. Lalanne, J.C.Rodier, P.Chavel, Groupe Physique des Images,
Laboratoire Charles Fabry de l'Institut d'Optique, CNRS URA 14, IOTA.
Partners : A.Dupret, E.Belhaire and P.Garda at
Institut d’Electronique Fondamentale, CNRS URA 22, France.
I.Glaser and A.A.Friesem at the Weizmann Institute, Physics of
Complex Systems, Rehovot 76100, Israel.
In the last decade, our group has grown some experience
in the field of optoelectronic cellular automata for massive parallel processing
tasks [1-7]. In the past few years, we focused
on the possibiliy to integrate
optoelectronic stochastic parallel processors
(OSPP) within real-time low-level image processing systems [10-18].
The key device under study is a Smart Pixel Array (sometimes called
artificial retina) composed out of a regular matrix of simple processing
elements (PEs), each playing the role of a neuron whose binary state evolves
as a function of a short-range semi-shift-invariant neighborhood. The data
to be processed is introduced either optically or electronically into the
OE-VLSI circuit. The firing of each neuron-like processing element is made
thanks to a laser-speckle-based
random number generator [8,9],
giving the possibility for the device to run complex vision algorithms
simulated annealing optimization techniques. The OSPP
is therefore capable of efficiently implement
vision tasks in real time (i.e. standard video rate) [13, 15-18].
Typical exemples of these operations are halftoning, non-linear
cleaning noise operations  and motion detection [17,18]
in a sequence of gray level images (see below).
Electronically interconnected PE array
A first VLSI prototype (SPIE600) was built in collaboration
with IEF on standard silicon technology (CMOS 0,8 microns) , providing
mesh-like electronic linking between a 24x24 PE array (see Fig.1,
left). The underlying principles of the circuit are described extensively
elsewhere [4,6,10-12]. In short, each processing element is defined by
its binary state and is provided with a pair of photo-detectors; all a
PE has to do is to update its state accordingly to a very simple rule:
if the collected contribution of its neighbours plus the corresponding
pixel value of the optically dual-rail encoded data projected onto the
chip (a sum we will call "force") is positive, then the PE will choose
the on-state, otherwise, it will choose the off-state. This operation is
performed by simply thresholding the local force by a standard electronic
comparator (see Fig.1, right).
Fig.1: SPIE600 VLSI CMOS chip. The data to be processed is optically
projected, dual-rail encoded onto the 24x24 PEs of the (electronically)
The nature of such deterministic operation
is turned probabilistic thanks to a laser speckle generator which
projects a time-varying speckle light over two photodetectors attached
to both positive and negative inputs of the comparator, so that the decision
threshold is “randomly shaken” (see Fig.2). Characterization of these "stochastic
comparators" (or "random number generators") is given in detail in [8,9].
Fig.2 : optical random number generation based on differential detection
Using SPIE600 we succesfully simulated the relaxation
of a two-dimensional spin network (2-D Ising problem) ,
a problem strongly related with the minimization of a non-convex, non-quadratic
many-variables "cost" function, a common problem in low-level image processing.
Using a binary ferroelectric spatial light modulator an a simple coherent
imaging system, we also succeded in demonstrating video-rate cleaning
noise in a sequence of binary corrupted images .
Optically interconnected PE array
The SPIE600 chip has optical inputs, but lacks of optical
outputs. Iindividual PEs are electronically interconnected to each other.
We explored (both experimentaly and theoretically) the all optical interconnection
issue, and showed it to be a powerful solution for providing both extension
and easy reconfiguration of the interconnection pattern. Experimental
results using a prototype demonstrator based on the existing silicon SPIE600
chip, two spatial light modulators and a convolution setup relying on interchangeable
Dammann gratings (in charge of the interconnection pattern, see Fig.5)
enabled us to successfully demonstrate the cleaning noise operation using
an optical mesh-interconnection kernel, and also
motion detection in
a sequence of gray level images using a larger interconnection kernel
Fig.5 : a 4-f optical convolver and an electronic feed-back loop simulates
SPA optical intra-chip interconnects. Our CGHs are 2-level phase Dammann
gratings developed on photoresist.
Motion detection using the optically interconnected OSPP prototype
Motion detection algorithms and devices provide an output
consisting of a binary map indicating motion or absence of it at every
pixel of a noise-corrupted gray-level image sequence. There is evidence
that motion detection can be correctly achieved using a statistical regularization
model based on spatio-temporal Markov Random Fields (MRF), an approach
which lends naturally to the need of intensive stochastic computation.
Motion detection is therefore an interesting example for our optoelectronic
stochastic parallel processor.
Figure 6 is a view of the complete setup. First
binary ferroelectric liquid crystal SLM-A projects onto the chip the preprocessed
image sequence at video-rate. For each image being projected, a whole simulated
anneling procedure is started: control PC drives the current of the random
number generator laser diode, while a secondary SLM (SLM-B) and a Dammann
grating implement the PE's interconnection pattern through an electronic
feed-back loop. This feed-back runs a hundred times, and the chip keeps
updating in parallel. When "thermal equilibrium" at the current speckle
laser power is achieved, the process resumes with the speckle laser power
being decreased as required to implement the "temperature" variation for
annealing. The processed data is recovered electronically from the SPIE600
Fig. 6 : (a) optical architecture of the optically interconnected
(b) View of the complete optically interconnected setup.
It is appropriate here to mention the difference
between the reconstructed
motion detection field and the mere subtraction
of successive frames in a time sequence (the so called change detection
field): a non-zero difference between two consecutive images may indicate
motion, but it may as well be due to time variable noise or a change in
illumination conditions; on the other hand, a null difference may be the
result of subtracting consecutive pixel values at two different locations
in the body of a uniformly lit moving object (see Fig.7).
Fig. 7 : Motion detection exemple on a synthetic sequence using the optically
Hybrid CMOS/SEED-based prototype
Overall performances of the optically interconnected
prototype using SPIE600 turned out to be rather poor: while video rate
was compatible with the components, non optimal operation of the Windows
computer interface operating with the driving PC slowed the process to
about 5 seconds per image when using the four nearest neighbor interconnecting
hologram, and to about 24 seconds in the case of an height nearest neighbor
interconnecting hologram. We elaborate then on the advantages of using
a hybrid CMOS/SEED Smart Pixel Array to monolithically integrate photodetectors
and modulators on the same chip, providing compact, high bandwidth intra-chip
optoelectronic interconnects (see Fig.8). We have modeled the functioning
of this monolithic processor, clearly showing a huge improvement of the
system performance (the prototype is able to process more than ten thousands
images per second) . Equally important is the fact that the size of
the whole system would be compatible with the standards of electronic packaging
Fig. 8 : architecture principles of the CMOS/SEED based OSPP.
The OSPP represents one of the possible pathways for
the introduction of novel optoelectronic devices in dedicated image processing
systems; nevertheless, application of an OSPP to any connectionist-based
signal processing system can be contemplated if only local (i.e.
shift-invariant) and real-time reconfigurable intra-chip interconnects
were made somehow available within the SPA - an issue worth to be investigated.
Early stages of this work were supported by the European Commission
under contract ERBCI1*CT93-0004.
...on optoelectronic cellular automata and
optical-scale parallel computing :
1. J. Taboury, J.M. Wang, P. Chavel, F.
Devos and P. Garda, "Optical cellular processor architecture. 1: Principles",
Appl.Opt. Vol.9, pp.1643-1650, (1988).
2. J. Taboury, J.M. Wang, P. Chavel,
F. Devos, "Optical cellular processor architecture. 2: Illustration and
system considerations", Appl.Opt. Vol.28, pp.3138 (1989).
3. I.Seyd-Darwish, P.Chavel, J.Taboury, F.Devos,
R.Reynaud, T.Maurin, "Opto-electronic automata for lattice-gas", Optics
in Complex Systems, SPIE 1319, pp.173, (1990).
4. Ph. Lalanne et al., "Optoelectronic devices
for Boltzmann machines and simulated annealing", Opt.Eng. 32, pp.1904-1914
5. G. Prémont, P.Lalanne, P.Chavel,
P.Heremans, M.Kuijk, "Photothyristor-based stochastic elementary processor",
Proc. from Opt.Comp.Int.Conf., Edinburgh, UK., August (1994).
6. A. Dupret, E. Belhaire, J.C. Rodier, Ph.
Lalanne, D. Prévost, P. Garda and P. Chavel, "An optoelectronic
CMOS circuit implementing a simulated annealing algorithm ", IEEE Journal
of Solid-State Circuits 31, pp.1046-1050, (1996).
7. T.Grudzien, P.Koppa, P.Chavel, J.L.Oudar,
R.Kuszelewicz, T.Rivera, "Optical adress recognition by optical bistables
and vertical cavity surface emitting lasers", Optical Computing 1998, Brugge,
Belgium, 17-20 June 1998, SPIE Proc.Vol.3490, P.Chavel, D.A.B.Miller, H.Thienpont,
Ed., pp.397-400, (1998).
...on optical random number generation &
stochastic processing :
8. Ph. Lalanne, E. Belhaire, J.C. Rodier, A.
Dupret, P. Garda and P. Chavel, "Gaussian random number generation by differential
detection of speckles", Optical Engineering, Vol. 34, No.6, p.1835-1837,
9. G. Prémont, Ph. Lalanne, P. Chavel,
M. Kuijk and P. Heremans, "Generation of sigmoid probability functions
by clipped differential speckle detection", Opt. Comm. 129, pp 347-352
...on optoelectronic parallel processor for vision
10. Ph. Lalanne, G. Prémont, D. Prévost,
P. Chavel, "Stochastic Optoelectronic Retinae for Vision Tasks", Optical
Computing`94, Edinburgh, August 1994, Proceedings, Institute of Physics
Conference Series 139, 295-313 (1995).
11. P.Chavel, Ph.Lalanne, "On parallel algorithms
for optical image processors", Optical Computing 1994, Edinburgh, August
1994, B.S.Wherrett, Ed., pp.11-16, IoP Publishing, London, (1995).
12. D. Prévost, Ph. Lalanne, J.C.
Rodier and P. Chavel, "Video-rate simulated annealing for stochastic artificial
retinas", Opt. Comm. Vol.132 pp.427-431 (1996).
13. A.Cassinelli, P.Lalanne, P.Chavel, I.Glaser,
"Demonstration of video-rate optoelectronic parallel processors for noise
cleaning in binary images by simulated annealing", Optical Computing 1998,
Brugge, Belgium, 17-20 June 1998, SPIE Proc.Vol.3490, P.Chavel, D.A.B.Miller,
H.Thienpont, Ed., pp.163-166, (1998).
14. Ph.Lalanne, G.Prémont, D.Prévost,
P.Chavel, "Optoelectronic implementation of stochastic artificial retinas",
Ann.Phys. Fr.24, No.3, (1999).
15. A.Cassinelli, P.Chavel, "Video-Rate Optoelectronic
Parallel Processors for Image Processing using Simulated Annealing", III.
Euro Amer.Workshop on Optoelectronic Information Processing, pp.27, 31
May-2-June 1999, Colmar, France (1999).
16. P. Chavel, A. Cassinelli and I. Glaser,
"Optoelectronic cellular automata for video real time vision", Optics in
Computing, OC'2000, Canada, Quebec18-23 juin 2000, SPIE 2000, pp.374-381,
conférence invitée, (2000).
17. P.Chavel, A.Cassinelli, I.Glaser, "Optoelectronic
implementation of cellular automata for complex vision algorithms", ROMOPTO
2000, 6th Conf. on Optics, September 4-7, Bucharest, Romania, conf. invitée,
18. A.Cassinelli, P.Chavel and M.P.Y.Desmulliez,
"Dedicated Optoelectronic Stochastic Parallel Processor (OSPP) for real-time
image processing: motion detection demonstration and design of an hybrid
CMOS/SEED based prototype", in preparation.
Last updated 07/11/2000, email@example.com
Laboratoire Charles Fabry de l'Institut d'Optique, firstname.lastname@example.org