Optics in Computing at IOTA 

Optoelectronic Stochastic Parallel Processors for real-time image processing

Groupe Physique des Images directed by Pierre CHAVEL

French résumé available here


Teamwork : A.Cassinelli, Ph. Lalanne, J.C.Rodier, P.Chavel, Groupe Physique des Images,
                        Laboratoire Charles Fabry de l'Institut d'Optique, CNRS URA 14, IOTA.

Partners :  A.Dupret, E.Belhaire and P.Garda at the Institut d’Electronique Fondamentale, CNRS URA 22, France.
                   I.Glaser and A.A.Friesem at the Weizmann Institute, Physics of Complex Systems, Rehovot 76100, Israel.

Introduction

    In the last decade, our group has grown some experience in the field of optoelectronic cellular automata for massive parallel processing tasks [1-7]. In the past few years, we focused on the possibiliy to integrate optoelectronic stochastic parallel processors (OSPP) within real-time low-level image processing systems [10-18]. The key device under study is a Smart Pixel Array (sometimes called stochastic artificial retina) composed out of a regular matrix of simple processing elements (PEs), each playing the role of a neuron whose binary state evolves as a function of a short-range semi-shift-invariant neighborhood. The data to be processed is introduced either optically or electronically into the OE-VLSI circuit. The firing of each neuron-like processing element is made stochastic thanks to a laser-speckle-based random number generator [8,9], giving the possibility for the device to run complex vision algorithms relying on simulated annealing optimization techniques. The OSPP is therefore capable of efficiently implement intensive, low-level vision tasks in real time (i.e. standard video rate) [13, 15-18]. Typical exemples of these operations are halftoning, non-linear cleaning noise operations [13] and motion detection [17,18] in a sequence of gray level images (see below).

Electronically interconnected PE array

    A first VLSI prototype (SPIE600) was built in collaboration with IEF on standard silicon technology (CMOS 0,8 microns) [6], providing mesh-like electronic linking between a 24x24 PE array (see Fig.1, left). The underlying principles of the circuit are described extensively elsewhere [4,6,10-12]. In short, each processing element is defined by its binary state and is provided with a pair of photo-detectors; all a PE has to do is to update its state accordingly to a very simple rule: if the collected contribution of its neighbours plus the corresponding pixel value of the optically dual-rail encoded data projected onto the chip (a sum we will call "force") is positive, then the PE will choose the on-state, otherwise, it will choose the off-state. This operation is performed by simply thresholding the local force by a standard electronic comparator (see Fig.1, right).
 

  Fig.1: SPIE600 VLSI CMOS chip. The data to be processed is optically projected, dual-rail encoded onto the 24x24 PEs of the (electronically) mesh-interconnected array.

     The nature of such deterministic operation is turned probabilistic thanks to a laser speckle generator which projects a time-varying speckle light over two photodetectors attached to both positive and negative inputs of the comparator, so that the decision threshold is “randomly shaken” (see Fig.2). Characterization of these "stochastic comparators" (or "random number generators") is given in detail in [8,9].
 

Fig.2 : optical random number generation based on differential detection of speckles.

    Using SPIE600 we succesfully simulated the relaxation of a two-dimensional spin network (2-D Ising problem) [12], a problem strongly related with the minimization of a non-convex, non-quadratic many-variables "cost" function, a common problem in low-level image processing. Using a binary ferroelectric spatial light modulator an a simple coherent imaging system, we also succeded in demonstrating video-rate cleaning noise in a sequence of binary corrupted images [13].

Optically interconnected PE array

    The SPIE600 chip has optical inputs, but lacks of optical outputs. Iindividual PEs are electronically interconnected to each other. We explored (both experimentaly and theoretically) the all optical interconnection issue, and showed it to be a powerful solution for providing both extension and easy reconfiguration of the interconnection pattern. Experimental results using a prototype demonstrator based on the existing silicon SPIE600 chip, two spatial light modulators and a convolution setup relying on interchangeable Dammann gratings (in charge of the interconnection pattern, see Fig.5) enabled us to successfully demonstrate the cleaning noise operation using an optical mesh-interconnection kernel, and also motion detection in a sequence of gray level images using a larger interconnection kernel [17,18].
 

Fig.5 : a 4-f optical convolver and an electronic feed-back loop simulates SPA optical intra-chip interconnects. Our CGHs are 2-level phase Dammann gratings developed on photoresist.

Motion detection using the optically interconnected OSPP prototype

    Motion detection algorithms and devices provide an output consisting of a binary map indicating motion or absence of it at every pixel of a noise-corrupted gray-level image sequence. There is evidence that motion detection can be correctly achieved using a statistical regularization model based on spatio-temporal Markov Random Fields (MRF), an approach which lends naturally to the need of intensive stochastic computation. Motion detection is therefore an interesting example for our optoelectronic stochastic parallel processor.
    Figure 6 is a view of the complete setup. First binary ferroelectric liquid crystal SLM-A projects onto the chip the preprocessed image sequence at video-rate. For each image being projected, a whole simulated anneling procedure is started: control PC drives the current of the random number generator laser diode, while a secondary SLM (SLM-B) and a Dammann grating implement the PE's interconnection pattern through an electronic feed-back loop. This feed-back runs a hundred times, and the chip keeps updating in parallel. When "thermal equilibrium" at the current speckle laser power is achieved, the process resumes with the speckle laser power being decreased as required to implement the "temperature" variation for annealing. The processed data is recovered electronically from the SPIE600 array.
 

 


 

 
Fig. 6 : (a) optical architecture of the optically interconnected demonstrator.
(b) View of the complete optically interconnected setup.

    It is appropriate here to mention the difference between the reconstructed motion detection field and the mere subtraction of successive frames in a time sequence (the so called change detection field): a non-zero difference between two consecutive images may indicate motion, but it may as well be due to time variable noise or a change in illumination conditions; on the other hand, a null difference may be the result of subtracting consecutive pixel values at two different locations in the body of a uniformly lit moving object (see Fig.7).
 

Fig. 7 : Motion detection exemple on a synthetic sequence using the optically interconnected demonstrator

Hybrid CMOS/SEED-based prototype

    Overall performances of the optically interconnected prototype using SPIE600 turned out to be rather poor: while video rate was compatible with the components, non optimal operation of the Windows computer interface operating with the driving PC slowed the process to about 5 seconds per image when using the four nearest neighbor interconnecting hologram, and to about 24 seconds in the case of an height nearest neighbor interconnecting hologram. We elaborate then on the advantages of using a hybrid CMOS/SEED Smart Pixel Array to monolithically integrate photodetectors and modulators on the same chip, providing compact, high bandwidth intra-chip optoelectronic interconnects (see Fig.8). We have modeled the functioning of this monolithic processor, clearly showing a huge improvement of the system performance (the prototype is able to process more than ten thousands images per second) [18]. Equally important is the fact that the size of the whole system would be compatible with the standards of electronic packaging technologies.
 
 

Fig. 8 : architecture principles of the CMOS/SEED based OSPP.

Further research

    The OSPP represents one of the possible pathways for the introduction of novel optoelectronic devices in dedicated image processing systems; nevertheless, application of an OSPP to any connectionist-based signal processing system can be contemplated if only local (i.e. non shift-invariant) and real-time reconfigurable intra-chip interconnects were made somehow available within the SPA - an issue worth to be investigated.

Early stages of this work were supported by the European Commission under contract ERBCI1*CT93-0004.

Papers

...on optoelectronic cellular automata and optical-scale parallel computing :

    1.  J. Taboury, J.M. Wang, P. Chavel, F. Devos and P. Garda, "Optical cellular processor architecture. 1: Principles", Appl.Opt. Vol.9, pp.1643-1650, (1988).
    2.  J. Taboury, J.M. Wang, P. Chavel, F. Devos, "Optical cellular processor architecture. 2: Illustration and system considerations", Appl.Opt. Vol.28, pp.3138 (1989).
    3. I.Seyd-Darwish, P.Chavel, J.Taboury, F.Devos, R.Reynaud, T.Maurin, "Opto-electronic automata for lattice-gas", Optics in Complex Systems, SPIE 1319, pp.173, (1990).
    4. Ph. Lalanne et al., "Optoelectronic devices for Boltzmann machines and simulated annealing", Opt.Eng. 32, pp.1904-1914 (1993).
    5. G. Prémont, P.Lalanne, P.Chavel, P.Heremans, M.Kuijk, "Photothyristor-based stochastic elementary processor", Proc. from Opt.Comp.Int.Conf., Edinburgh, UK., August (1994).
    6. A. Dupret, E. Belhaire, J.C. Rodier, Ph. Lalanne, D. Prévost, P. Garda and P. Chavel, "An optoelectronic CMOS circuit implementing a simulated annealing algorithm ", IEEE Journal of Solid-State Circuits 31, pp.1046-1050, (1996).
    7. T.Grudzien, P.Koppa, P.Chavel, J.L.Oudar, R.Kuszelewicz, T.Rivera, "Optical adress recognition by optical bistables and vertical cavity surface emitting lasers", Optical Computing 1998, Brugge, Belgium, 17-20 June 1998, SPIE Proc.Vol.3490, P.Chavel, D.A.B.Miller, H.Thienpont, Ed., pp.397-400,  (1998).

...on optical random number generation & stochastic processing :

    8. Ph. Lalanne, E. Belhaire, J.C. Rodier, A. Dupret, P. Garda and P. Chavel, "Gaussian random number generation by differential detection of speckles", Optical Engineering, Vol. 34, No.6, p.1835-1837, June (1995).
    9. G. Prémont, Ph. Lalanne, P. Chavel, M. Kuijk and P. Heremans, "Generation of sigmoid probability functions by clipped differential speckle detection", Opt. Comm. 129, pp 347-352 (1996).

...on optoelectronic parallel processor for vision tasks :

    10. Ph. Lalanne, G. Prémont, D. Prévost, P. Chavel, "Stochastic Optoelectronic Retinae for Vision Tasks", Optical Computing`94, Edinburgh, August 1994, Proceedings, Institute of Physics Conference Series 139, 295-313 (1995).
    11. P.Chavel, Ph.Lalanne, "On parallel algorithms for optical image processors", Optical Computing 1994, Edinburgh, August 1994, B.S.Wherrett, Ed., pp.11-16, IoP Publishing, London, (1995).
    12. D. Prévost, Ph. Lalanne, J.C. Rodier and P. Chavel, "Video-rate simulated annealing for stochastic artificial retinas", Opt. Comm. Vol.132 pp.427-431 (1996).
    13. A.Cassinelli, P.Lalanne, P.Chavel, I.Glaser, "Demonstration of video-rate optoelectronic parallel processors for noise cleaning in binary images by simulated annealing", Optical Computing 1998, Brugge, Belgium, 17-20 June 1998, SPIE Proc.Vol.3490, P.Chavel, D.A.B.Miller, H.Thienpont, Ed., pp.163-166, (1998).
    14. Ph.Lalanne, G.Prémont, D.Prévost, P.Chavel, "Optoelectronic implementation of stochastic artificial retinas", Ann.Phys. Fr.24, No.3, (1999).
    15. A.Cassinelli, P.Chavel, "Video-Rate Optoelectronic Parallel Processors for Image Processing using Simulated Annealing", III. Euro Amer.Workshop on Optoelectronic Information Processing, pp.27, 31 May-2-June 1999, Colmar, France (1999).
    16. P. Chavel, A. Cassinelli and I. Glaser, "Optoelectronic cellular automata for video real time vision", Optics in Computing, OC'2000, Canada, Quebec18-23 juin 2000, SPIE 2000, pp.374-381, conférence invitée, (2000).
    17. P.Chavel, A.Cassinelli, I.Glaser, "Optoelectronic implementation of cellular automata for complex vision algorithms", ROMOPTO 2000, 6th Conf. on Optics, September 4-7, Bucharest, Romania, conf. invitée, (2000)
    18. A.Cassinelli, P.Chavel and M.P.Y.Desmulliez, "Dedicated Optoelectronic Stochastic Parallel Processor (OSPP) for real-time image processing: motion detection demonstration and design of an hybrid CMOS/SEED based prototype", in preparation.
    [Home]
Last updated 07/11/2000, acassin656@aol.com
Laboratoire Charles Fabry de l'Institut d'Optique, webmaster@iwww.iota.u-psud.fr