Ultrafast machine vision with 2D material neural network image sensors.

Nature
Lukas MennelThomas Mueller

Abstract

Machine vision technology has taken huge leaps in recent years, and is now becoming an integral part of various intelligent systems, including autonomous vehicles and robotics. Usually, visual information is captured by a frame-based camera, converted into a digital format and processed afterwards using a machine-learning algorithm such as an artificial neural network (ANN)1. The large amount of (mostly redundant) data passed through the entire signal chain, however, results in low frame rates and high power consumption. Various visual data preprocessing techniques have thus been developed2-7 to increase the efficiency of the subsequent signal processing in an ANN. Here we demonstrate that an image sensor can itself constitute an ANN that can simultaneously sense and process optical images without latency. Our device is based on a reconfigurable two-dimensional (2D) semiconductor8,9 photodiode10-12 array, and the synaptic weights of the network are stored in a continuously tunable photoresponsivity matrix. We demonstrate both supervised and unsupervised learning and train the sensor to classify and encode images that are optically projected onto the chip with a throughput of 20 million bins per second.

References

Jan 25, 1990·Nature·D PsaltisS Lin
Apr 29, 2006·Science·Ki-Hun JeongLuke P Lee
Mar 21, 2013·ACS Nano·Simone BertolazziAndras Kis
Mar 13, 2014·Nature Nanotechnology·Andreas PospischilThomas Mueller
Mar 13, 2014·Nature Nanotechnology·Jason S RossXiaodong Xu
Mar 13, 2014·Nature Nanotechnology·Britton W H BaugherPablo Jarillo-Herrero
May 29, 2015·Nature·Yann LeCunGeoffrey Hinton
Apr 12, 2017·Nature Communications·Stefan WachterThomas Mueller
May 23, 2017·Nature Nanotechnology·Patrick M SheridanWei D Lu
Jun 13, 2017·Nature Nanotechnology·Dong LiZengxing Zhang
Apr 25, 2018·Nature Communications·Mathieu MassicotteFrank H L Koppens
Jul 17, 2019·Nature Nanotechnology·Feichi ZhouYang Chai

❮ Previous
Next ❯

Citations

Jul 10, 2020·Lab on a Chip·Akihiro IsozakiKeisuke Goda
Mar 7, 2020·Nature·Yang Chai
Jul 11, 2020·Nature Nanotechnology·Chunsen LiuPeng Zhou
Dec 16, 2020·Scientific Reports·Molla Manjurul IslamTania Roy
Nov 19, 2020·Advanced Materials·Taimur AhmedSumeet Walia
Jan 30, 2021·Patterns·Aleix Boquet-PujadasNancy Guillén
Feb 20, 2021·Nature Communications·Hongwei TanSebastiaan van Dijken
Mar 7, 2021·Sensors·Martin PechJiří Bednář
Mar 21, 2021·Nature Communications·Qian-Bing ZhuDong-Ming Sun
Mar 28, 2021·Science Advances·Jingxi LiAydogan Ozcan
May 23, 2021·Advanced Science·Wen YangZhongming Wei
Jun 11, 2021·Light, Science & Applications·Chaoyue LiuDaoxin Dai
Jun 10, 2021·Advanced Materials·Lei YinJun He
Jun 18, 2021·Nature·C KasparW H P Pernice
Sep 1, 2021·ACS Nano·Seongin HongSunkook Kim

❮ Previous
Next ❯

Methods Mentioned

BETA
chip
scanning electron microscopy
electron-beam lithography

Software Mentioned

mathop

Related Concepts

Trending Feeds

COVID-19

Coronaviruses encompass a large family of viruses that cause the common cold as well as more serious diseases, such as the ongoing outbreak of coronavirus disease 2019 (COVID-19; formally known as 2019-nCoV). Coronaviruses can spread from animals to humans; symptoms include fever, cough, shortness of breath, and breathing difficulties; in more severe cases, infection can lead to death. This feed covers recent research on COVID-19.

Blastomycosis

Blastomycosis fungal infections spread through inhaling Blastomyces dermatitidis spores. Discover the latest research on blastomycosis fungal infections here.

Nuclear Pore Complex in ALS/FTD

Alterations in nucleocytoplasmic transport, controlled by the nuclear pore complex, may be involved in the pathomechanism underlying multiple neurodegenerative diseases including Amyotrophic Lateral Sclerosis and Frontotemporal Dementia. Here is the latest research on the nuclear pore complex in ALS and FTD.

Applications of Molecular Barcoding

The concept of molecular barcoding is that each original DNA or RNA molecule is attached to a unique sequence barcode. Sequence reads having different barcodes represent different original molecules, while sequence reads having the same barcode are results of PCR duplication from one original molecule. Discover the latest research on molecular barcoding here.

Chronic Fatigue Syndrome

Chronic fatigue syndrome is a disease characterized by unexplained disabling fatigue; the pathology of which is incompletely understood. Discover the latest research on chronic fatigue syndrome here.

Evolution of Pluripotency

Pluripotency refers to the ability of a cell to develop into three primary germ cell layers of the embryo. This feed focuses on the mechanisms that underlie the evolution of pluripotency. Here is the latest research.

Position Effect Variegation

Position Effect Variagation occurs when a gene is inactivated due to its positioning near heterochromatic regions within a chromosome. Discover the latest research on Position Effect Variagation here.

STING Receptor Agonists

Stimulator of IFN genes (STING) are a group of transmembrane proteins that are involved in the induction of type I interferon that is important in the innate immune response. The stimulation of STING has been an active area of research in the treatment of cancer and infectious diseases. Here is the latest research on STING receptor agonists.

Microbicide

Microbicides are products that can be applied to vaginal or rectal mucosal surfaces with the goal of preventing, or at least significantly reducing, the transmission of sexually transmitted infections. Here is the latest research on microbicides.

Related Papers

Computers and Biomedical Research, an International Journal
G D Tourassi, C E Floyd
Methods in Molecular Biology
Jinming ZouSung-Sau So
Indian Journal of Physiology and Pharmacology
Z H KhanS N Ragu Kumar
© 2021 Meta ULC. All rights reserved