LeCun, Y. et al. Handwritten digit recognition with a back-propagation network. In Advances in Neural Information Processing Systems 2 (NIPS 1989) (ed. Touretzky, D. S.) 396–404 (1990).
Krizhevsky, A., Sutskever, I. & Hinton, G. E. ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems 25 (NIPS 2012) (eds Pereira, F. et al.) 1097–1105 (2012).
LeCun, Y., Bengio, Y. & Hinton, G. Deep learning. Nature 521, 436–444 (2015).
Miller, D. A. B. Waves, modes, communications, and optics: a tutorial. Adv. Opt. Photonics 11, 679–825 (2019).
Brunner, D., Soriano, M. C., Mirasso, C. R. & Fischer, I. Parallel photonic information processing at gigabyte per second data rates using transient states. Nat. Commun. 4, 1364 (2013).
Goodman, J. W., Leonberger, F. J., Kung, S.-Y. & Athale, R. A. Optical interconnections for VLSI systems. Proc. IEEE 72, 850–866 (1984). The first paper to provide a substantial analysis and reasons for the use of optics in interconnection (rather than for logic) in digital systems.
Miller, D. A. B. Rationale and challenges for optical interconnects to electronic chips. Proc. IEEE 88, 728–749 (2000).
Miller, D. A. B. Attojoule optoelectronics for low-energy information processing and communications. J. Lightwave Technol. 35, 346–396 (2017).
Miller, D. A. B. Are optical transistors the logical next step? Nat. Photon. 4, 3–5 (2010).
Athale, R. & Psaltis, D. Optical computing: past and future. Opt. Photon. News 27, 32–39 (2016).
Goodman, J. W. Introduction to Fourier Optics (Roberts and Co, 2005).
Liutkus, A. et al. Imaging with nature: compressive imaging using a multiply scattering medium. Sci. Rep. 4, 5552 (2014).
Saade, A. et al. Random projections through multiple optical scattering: approximating kernels at the speed of light. In 2016 IEEE Intl Conf. Acoustics, Speech and Signal Processing (ICASSP) 6215–6219 (IEEE, 2016).
Lin, X. et al. All-optical machine learning using diffractive deep neural networks. Science 361, 1004–1008 (2018). An optical implementation using multiple optimized layers for all-optical image classification.
Chang, J., Sitzmann, V., Dun, X., Heidrich, W. & Wetzstein, G. Hybrid optical–electronic convolutional neural networks with optimized diffractive optics for image classification. Sci. Rep. 8, 12324 (2018). An optical implementation of a single CNN layer demonstrated for hybrid optical–electronic image classification.
Rosenblatt, F. The Perceptron, A Perceiving and Recognizing Automaton Report no. 85-460-1 (Project Para, Cornell Aeronautical Laboratory, 1957).
Hebb, D. O. The Organization of Behavior (Wiley, 1949).
Widrow, B. & Hoff, M. E. Adaptive switching circuits. In 1960 IRE WESCON Convention Record 96–104 (Institute of Radio Engineers, 1960).
Hopfield, J. J. Neural networks and physical systems with emergent collective computational abilities. Proc. Natl Acad. Sci. USA 79, 2554–2558 (1982).
Carpenter, G. A. & Grossberg, S. A massively parallel architecture for a self-organizing neural pattern recognition machine. Comput. Vis. Graph. Image Process. 37, 54–115 (1987).
Kohonen, T. Self-organized formation of topologically correct feature maps. Biol. Cybern. 43, 59–69 (1982).
Rumelhart, D. E., Hinton, G. E. & Williams, R. J. Learning representations by back-propagating errors. Nature 323, 533–536 (1986).
Mead, C. Neuromorphic electronic systems. Proc. IEEE 78, 1629–1636 (1990).
Farhat, N. H., Psaltis, D., Prata, A. & Paek, E. Optical implementation of the Hopfield model. Appl. Opt. 24, 1469–1475 (1985). Optical implementation of content-addressable associative memory based on the Hopfield model for neural networks and on the addition of nonlinear iterative feedback to a vector–matrix multiplier.
Denz, C. Optical Neural Networks (Springer Science & Business Media, 2013).
Psaltis, D., Brady, D., Gu, X.-G. & Lin, S. Holography in artificial neural networks. Nature 343, 325–330 (1990). Introduction of nonlinear photorefractive crystals for optical computing.
Li, H.-Y. S., Qiao, Y. & Psaltis, D. Optical network for real-time face recognition. Appl. Opt. 32, 5026–5035 (1993).
Miller, D. A. B. Self-configuring universal linear optical component. Photon. Res. 1, 1–15 (2013). Proof that arbitrary linear operations such as singular value decompositions can be performed in optics—not just Fourier transforms and convolutions as in early optical computing.
Shen, Y. et al. Deep learning with coherent nanophotonic circuits. Nat. Photon. 11, 441 (2017). A silicon photonic neural network using meshes of MZIs for vowel recognition.
Fang, M. Y.-S., Manipatruni, S., Wierzynski, C., Khosrowshahi, A. & DeWeese, M. R. Design of optical neural networks with component imprecisions. Opt. Express 27, 14009–14029 (2019).
Wilkes, C. M. et al. 60 dB high-extinction auto-configured Mach–Zehnder interferometer. Opt. Lett. 41, 5318–5321 (2016).
Hughes, T. W., Minkov, M., Shi, Y. & Fan, S. Training of photonic neural networks through in situ backpropagation and gradient measurement. Optica 5, 864–871 (2018).
Feldmann, J., Youngblood, N., Wright, C. D., Bhaskaran, H. & Pernice, W. H. P. All-optical spiking neurosynaptic networks with self-learning capabilities. Nature 569, 208–214 (2019). A photonic circuit that exploits wavelength division multiplexing techniques for pattern recognition directly in the optical domain.
Tait, A. N. et al. Neuromorphic photonic networks using silicon photonic weight banks. Sci. Rep. 7, 7430 (2017).
Huang, C. et al. Giant enhancement in signal contrast using integrated all-optical nonlinear thresholder. In 2019 Optical Fiber Communications Conference and Exhibition (OFC) 415–417 (IEEE, 2019).
Nahmias, M. A., Shastri, B. J., Tait, A. N. & Prucnal, P. R. A leaky integrate-and-fire laser neuron for ultrafast cognitive computing. IEEE J. Sel. Top. Quantum Electron. 19, 1800212 (2013).
Amin, R. et al. ITO-based electro-absorption modulator for photonic neural activation function. APL Mater. 7, 081112 (2019).
Williamson, I. A. D. et al. Reprogrammable electro-optic nonlinear activation functions for optical neural networks. IEEE J. Sel. Top. Quantum Electron. 26, 7700412 (2020).
Miller, D. A. B. Novel analog self-electrooptic-effect devices. IEEE J. Quantum Electron. 29, 678–698 (1993).
Srinivasan, S. A. et al. High absorption contrast quantum confined stark effect in ultra-thin Ge/SiGe quantum well stacks grown on Si. IEEE J. Quantum Electron. 56, 5200207 (2020).
Ferreira de Lima, T., Shastri, B. J., Tait, A. N., Nahmias, M. A. & Prucnal, P. R. Progress in neuromorphic photonics. Nanophotonics 6, 577–599 (2017).
Nahmias, M. A. et al. Photonic multiply–accumulate operations for neural networks. IEEE J. Sel. Top. Quantum Electron. 26, 7701518 (2020). A review article on the state-of-the-art of photonic MACs along with detailed characterizations and comparisons of the performance of photonic and comparable electronic hardware.
Gupta, S., Agrawal, A., Gopalakrishnan, K. & Narayanan, P. Deep learning with limited numerical precision. In Proc. 32nd Intl Conf. Machine Learning (eds Bach, F. & Blei, D.) 1737–1746 (PMLR, 2015).
Hamerly, R., Bernstein, L., Sludds, A., Soljačić, M. & Englund, D. Large-scale optical neural networks based on photoelectric multiplication. Phys. Rev. X 9, 021032 (2019).
Lugt, A. V. Signal detection by complex spatial filtering. IEEE Trans. Inf. Theory 10, 139–145 (1964). The introduction of optical correlators.
Gregory, D. A. Real-time pattern recognition using a modified liquid crystal television in a coherent optical correlator. Appl. Opt. 25, 467–469 (1986).
Manzur, T., Zeller, J. & Serati, S. Optical-correlator-based target detection, recognition, classification, and tracking. Appl. Opt. 51, 4976–4983 (2012).
Javidi, B., Li, J. & Tang, Q. Optical implementation of neural networks for face recognition by the use of nonlinear joint transform correlators. Appl. Opt. 34, 3950–3962 (1995).
Koppal, S. J., Gkioulekas, I., Zickler, T. & Barrows, G. L. Wide-angle micro sensors for vision on a tight budget. In 2011 IEEE Conf. Computer Vision and Pattern Recognition (CVPR 2011) 361–368 (IEEE, 2011).
Hughes, T. W., Williamson, I. A. D., Minkov, M. & Fan, S. Wave physics as an analog recurrent neural network. Sci. Adv. 5, eaay6946 (2019).
Duarte, M. F. et al. Single-pixel imaging via compressive sampling. IEEE Signal Process. Mag. 25, 83–91 (2008).
Moretti, C. & Gigan, S. Readout of fluorescence functional signals through highly scattering tissue. Nat. Photonics 14, 361–364 (2020).
Rahmani, B., Loterie, D., Konstantinou, G., Psaltis, D. & Moser, C. Multimode optical fiber transmission with a deep learning network. Light Sci. Appl. 7, 69 (2018).
Caramazza, P., Moran, O., Murray-Smith, R. & Faccio, D. Transmission of natural scene images through a multimode fibre. Nat. Commun. 10, 2029 (2019).
Li, Y., Xue, Y. & Tian, L. Deep speckle correlation: a deep learning approach toward scalable imaging through scattering media. Optica 5, 1181–1190 (2018).
Horisaki, R., Takagi, R. & Tanida, J. Learning-based imaging through scattering media. Opt. Express 24, 13738–13743 (2016).
Ando, T., Horisaki, R. & Tanida, J. Speckle-learning-based object recognition through scattering media. Opt. Express 23, 33902–33910 (2015).
Mahoney, M. W. Randomized Algorithms for Matrices and Data (Now Publishers, 2011).
Dong, J., Rafayelyan, M., Krzakala, F. & Gigan, S. Optical reservoir computing using multiple light scattering for chaotic systems prediction. IEEE J. Sel. Top. Quantum Electron. 26, 7701012 (2019).
Gupta, S., Gribonval, R., Daudet, L. & Dokmanić, I. Don’t take it lightly: phasing optical random projections with unknown operators. In Advances in Neural Information Processing Systems 32 (NeurIPS 2019) (eds Wallach, H. et al.) 14855–14865 (2019).
Marshall, J. & Oberwinkler, J. The colourful world of the mantis shrimp. Nature 401, 873–874 (1999).
Thoen, H. T., How, M. J., Chiou, T.-H. & Marshall, J. A different form of color vision in mantis shrimp. Science 343, 411–413 (2014).
Wetzstein, G., Ihrke, I., Lanman, D. & Heidrich, W. Computational plenoptic imaging. Comput. Graph. Forum 30, 2397–2426 (2011).
Hinton, G. E. & Salakhutdinov, R. R. Reducing the dimensionality of data with neural networks. Science 313, 504–507 (2006).
Sitzmann, V. et al. End-to-end optimization of optics and image processing for achromatic extended depth of field and super-resolution imaging. ACM Trans. Graph. 37, 114 (2018). The first demonstration of end-to-end optimization of optics and image processing for a computational camera design with computer vision applications.
Chakrabarti, A. Learning sensor multiplexing design through back-propagation. In Advances in Neural Information Processing Systems 29 (NIPS 2016) (eds Lee, D. D. et al.) 3081–3089 (2016).
Martel, J. N. P., Muller, L. K., Carey, S., Dudek, P. & Wetzstein, G. Neural sensors: learning pixel exposures for HDR imaging and video compressive sensing with programmable sensors. IEEE Trans. Pattern Anal. Mach. Intell. 42, 1642–1653 (2020).
Horstmeyer, R., Chen, R. Y., Kappes, B. & Judkewitz, B. Convolutional neural networks that teach microscopes how to image. Preprint at https://arxiv.org/abs/1709.07223 (2017).
Marco, J. et al. DeepToF: off-the-shelf real-time correction of multipath interference in time-of-flight imaging. ACM Trans. Graph. 36, 219 (2017).
Su, S., Heide, F., Wetzstein, G. & Heidrich, W. Deep end-to-end time-of-flight imaging. In 2018 IEEE Conf. Computer Vision and Pattern Recognition (CVPR) 6383–6392 (IEEE, 2018).
Kellman, M., Bostan, E., Repina, N. & Waller, L. Physics-based learned design: optimized coded-illumination for quantitative phase imaging. IEEE Trans. Comput. Imaging 5, 344–353 (2019).
Sinha, A., Lee, J., Li, S. & Barbastathis, G. Lensless computational imaging through deep learning. Optica 4, 1117–1125 (2017).
Metzler, C. A., Ikoma, H., Peng, Y. & Wetzstein, G. Deep optics for single-shot high-dynamic-range imaging. In 2020 IEEE/CVF Conf. Computer Vision and Pattern Recognition (CVPR) 1372–1382 (IEEE, 2020).
Luo, Y. et al. Design of task-specific optical systems using broadband diffractive neural networks. Light Sci. Appl. 8, 112 (2019).
Haim, H., Elmalem, S., Giryes, R., Bronstein, A. M. & Marom, E. Depth estimation from a single image using deep learned phase coded mask. IEEE Trans. Comput. Imaging 4, 298–310 (2018).
Chang, J. & Wetzstein, G. Deep optics for monocular depth estimation and 3D object detection. In 2019 IEEE/CVF Intl Conf. Computer Vision (ICCV) 10192–10211 (IEEE, 2019).
Wu, Y., Boominathan, V., Chen, H., Sankaranarayanan, A. & Veeraraghavan, A. Phasecam3D—learning phase masks for passive single view depth estimation. In 2019 IEEE Intl Conf. Computational Photography (ICCP) 19–30 (IEEE, 2019).
Bertero, M. & Boccacci, P. Introduction to Inverse Problems in Imaging (CRC Press, 1998).
Barbastathis, G., Ozcan, A. & Situ, G. On the use of deep learning for computational imaging. Optica 6, 921–943 (2019).
Rivenson, Y. et al. Deep learning microscopy. Optica 4, 1437–1443 (2017).
Wu, Y. et al. Extended depth-of-field in holographic imaging using deep-learning-based autofocusing and phase recovery. Optica 5, 704–710 (2018).
Nehme, E. & Weiss, L. E., Michaeli, T. & Shechtman, Y. Deep-storm: super-resolution single-molecule microscopy by deep learning. Optica 5, 458–464 (2018).
Ouyang, W., Aristov, A., Lelek, M., Hao, X. & Zimmer, C. Deep learning massively accelerates super-resolution localization microscopy. Nat. Biotechnol. 36, 460–468 (2018).
Christiansen, E. M. et al. In silico labeling: predicting fluorescent labels in unlabeled images. Cell 173, 792–803 (2018).
Wu, Y. et al. Three-dimensional virtual refocusing of fluorescence microscopy images using deep learning. Nat. Methods 16, 1323–1331 (2019).
Wang, H. et al. Deep learning enables cross-modality super-resolution in fluorescence microscopy. Nat. Methods 16, 103–110 (2019).
Rivenson, Y., Zhang, Y., Günaydın, H., Teng, D. & Ozcan, A. Phase recovery and holographic image reconstruction using deep learning in neural networks. Light Sci. Appl. 7, 17141 (2018).
Boyd, N., Jonas, E., Babcock, H. & Recht, B. DeepLoco: Fast 3D localization microscopy using neural networks. Preprint at https://doi.org/10.1101/267096 (2018).
Weigert, M. et al. Content-aware image restoration: pushing the limits of fluorescence microscopy. Nat. Methods 15, 1090 (2018).
Nehme, E. et al. DeepSTORM3D: dense 3D localization microscopy and PSF design by deep learning. Nat. Methods 17, 734–740 (2020). An end-to-end optimization approach for point spread function engineering and neural-network-based locations for 3D fluorescence superresolution microscopy.
Liu, T. et al. Deep learning-based super-resolution in coherent imaging systems. Sci. Rep. 9, 3926 (2019).
Zhang, H. et al. High-throughput, high-resolution deep learning microscopy based on registration-free generative adversarial network. Biomed. Opt. Express 10, 1044–1063 (2019).
Escudero, M. C. et al. Digitally stained confocal microscopy through deep learning. In Proc. 2nd Intl Conf. Medical Imaging with Deep Learning (eds Cardoso, M. J. et al.) 121–129 (PMLR, 2019).
Rivenson, Y. et al. Deep learning enhanced mobile-phone microscopy. ACS Photonics 5, 2354–2364 (2018).
Goy, A., Arthur, K., Li, S. & Barbastathis, G. Low photon count phase retrieval using deep learning. Phys. Rev. Lett. 121, 243902 (2018).
Rivenson, Y. et al. Virtual histological staining of unlabelled tissue-autofluorescence images via deep learning. Nat. Biomed. Eng. 3, 466–477 (2019).
Wu, Y. et al. Bright-field holography: cross-modality deep learning enables snapshot 3D imaging with bright-field contrast using a single hologram. Light Sci. Appl. 8, 25 (2019).
Rivenson, Y. et al. PhaseStain: the digital staining of label-free quantitative phase microscopy images using deep learning. Light Sci. Appl. 8, 23 (2019).
Mengu, D., Luo, Y., Rivenson, Y. & Ozcan, A. Analysis of diffractive optical neural networks and their integration with electronic neural networks. IEEE J. Sel. Top. Quantum Electron. 26, 3700114 (2019).
Dagenais, M., Sharfin, W. F. & Seymour, R. J. Optical digital matrix multiplication apparatus. EU patent EP0330710A1 (1988).
The post Inference in artificial intelligence with deep optics and photonics – Nature.com appeared first on abangtech.
source https://abangtech.com/inference-in-artificial-intelligence-with-deep-optics-and-photonics-nature-com/
No comments:
Post a Comment