研究人员Department of Connectomicsat the Max Planck Institute (MPI) for Brain Research study neural networks in the cerebral cortex in order to understand how the brain processes sensory experience to detect objects in the environment. Their work involves buildingconnectomes—maps of neuronal circuits that identify the individual connections between neurons.
与人工神经网络的“神经元”不同，生物神经元不会组织成一维层的整齐行。取而代之的是，它们被包装并连接在密集的3D空间填充网状网中，只能从用电子显微镜的纳米分辨率捕获的脑组织图像研究（图1）。
Figure 1. Dense reconstruction of approximately 500,000 cubic micrometers of mammalian cortical tissue yielding 2.7 m of neuronal cables, making up a connectome of about 400,000 synapses between 34,221 axons.
From Motta, Berning, Boergens, Staffler, Beining, Loomba, Hennig, Wissler, Helmstaedter.“Dense connectomic reconstruction in layer 4 of the somatosensory cortex.”Science。2019年10月24日。经AAAS的许可转载。1
Why Study Biological Neural Networks?
Convolutional neural networks (CNNs) were inspired by biological intelligence, with the feedforward connectivity of neurons and layers in a CNN resembling that of the visual cortex of humans and other animals. Increased computing power and the availability of massive amounts of data have improved the performance and accuracy of CNNs, but when compared with the human brain, they are remarkably inefficient, in both the energy they consume and the labels they require in training. A large-scale CNN classifier deployed in a cloud computing environment consumes several orders of magnitude more power than the human brain, and while a toddler can learn to classify objects after seeing just a few dozen examples, CNNs need millions of accurately labeled images. Research teams that rely on deep learning are beginning to bump up against these limitations. By analyzing connectomes to understand how evolution solved these challenges in biological neural networks, researchers may find clues for developing next-generation artificial neural networks.
3D electron micrographs reveal the bulbous cell bodies of individual neurons and the dense and tortuous network of thin neuronal cables connecting neurons. The single cable, oraxon，从每个神经元投射是一个非常薄的结构。轴突小于直径小于一微米，连接到相邻的神经元以及更遥远的神经元，例如相同皮质区域不同层的神经元，或者甚至在大脑的另一侧，几毫米远离的神经元。大脑皮层中的每个神经元可以从其自己的本地分支神经电缆树上接收数千个神经元的连接，或树突。这些个人连接点（synapses) between the axonal cable of one neuron and the dendritic cable of another are of a sub-micrometer scale.
在新兴连接组学领域的研究人员面临的挑战是开发技术以绘制在这一广泛量表上的神经元连接（图2）。Max Planck研究人员专注于连接组的致密重建，这需要在追踪神经元电缆和鉴定电子显微镜体积内突触的最高准确性。挑战是大规模的：大脑皮层中的一个立方毫米灰质的灰质包含分支神经元电缆和约10亿突触的公里。
Figure 2. Scales of neural connectivity in the cerebral cortex, from nanometer-scale synapses between individual neurons to millimeter-scale connection distances. Orange = axons; blue = dendrites.
Manual dense reconstruction of connectomes from electron microscopy data typically takes tens of thousands of work hours, even for smaller sample volumes containing around 1 million synapses. To automate the more labor-intensive parts of the reconstruction process, the Department of Connectomics developed FocusEM, a workflow that combines human annotation with automation powered by convolutional neural networks created in MATLAB®。The CNN models were trained and executed using parallel processing on a high-performance computing (HPC) cluster.
Focusem使得在体感皮层中重建0.9米的树突和约1.8米的轴突，识别近500,000个突触，只有4000个人工工作时间，比以前的效率高10至25倍。这项工作是publishedin the journalScience, where the researchers showed how accurate dense reconstructions at this scale can contribute to a detailed understanding of local brain circuitry.
Connectome Reconstruction Challenges
To image the sample, a block of brain tissue is extracted and stained with heavy metal compounds. The sample is transferred into an electron microscope equipped with a custom-built microtome. Tissue imaging alternates with tissue slicing, in which thin slices of 25–30 nm are obtained with the microtome’s diamond knife. Thousands of imaging and cutting alternations produce a 3D image dataset that is hundreds of gigabytes to terabytes in size (Figure 3).
Figure 3. Serial scanning electron microscopy for brain imaging. A probe of neuronal tissue gets imaged and subsequently cut with a custom-built microtome. Alternations of cutting and imaging result in a 3D image stack. Scale bar = 1µm.
为了绘制连接组，研究人员必须从每个神经元中绕过3D体积时从每个神经元中追踪轴突，以确定其与其他神经元的连接。
When two cable segments positioned near one another are detected, then researchers must closely analyze the image to determine whether they are part of the same axon, two separate pieces connected via a synapse, or unrelated segments.
Deep Learning for Large-Scale Reconstruction of Neuronal Circuits
The FocusEM workflow automates most of the time-consuming annotation and decision-making steps in the connectome reconstruction process. The workflow consists of three main stages:
- 基于图像处理算法和启发式方法的预处理步骤
- Image segmentation, based on image processing algorithms and deep learning
- lea形态学重建、基于机器rning combined with focused human queries
The preprocessing stage includes steps such as aligning individual 2D image slices within the 3D sample volume using a global least-squares solver, masking out easily identifiable structures such as blood vessels and nuclei, and correcting for image brightness.
图像分割阶段基于一个称为emem的工作流程，publishedby the Max Planck Department of Connectomics in the journalNeuronin 2015. SegEM uses a custom-built 3D CNN in combination with image segmentation algorithms such as watershed transforms. For the sample volume of 500,000 cubic micrometers in the current study, the SegEM stage produced 15 million distinct volume segments.
This morphological reconstruction stage relies on a set of machine learning classifiers that were custom-built and trained in MATLAB to support the FocusEM workflow:
- TheConnectEM分类器确定两个相邻体积段物理连接的可能性，如连续神经电缆的一部分。
- TheSynEMclassifier determines whether adjacent volume segments correspond to a neural connection, a synapse that occurs across a thin gap of nanometer scale (Figure 2); these can be identified via distinct image features such as clusters of synaptic vesicles.
- 四个TypeEMclassifiers categorize volume segments as belonging to an axon, a dendrite, a dendritic spine head (the location of a potential synapse), or a non-neuron cell type.
Focusem工作流程使用这些分类器来自动化密集重建过程中的许多步骤。受过训练的人类注释者专注于分类器的定向查询，以解决复杂情况，例如多个神经电缆之间的交叉点。
与手动重建方法相比，这种半自动化的工作流程的结果降低了工作时间超过十倍（图4）。FOCUSEM工作流程代码可从A下载GitLab repository。
图4.密集地重建神经组织的立方毫米所需的不同方法所需的工作时间。尽管手动方法是耗时且昂贵的，但Focusem允许在现实的时间范围和成本中密集重建更大的大脑体积。
高性能计算加速
In addition to minimizing the human work hours required to complete the dense reconstruction of a connectome, the Max Planck researchers sought to minimize the compute time required for the automated steps in the FocusEM workflow.
为了实现这一点,研究人员转向相比l computing. The Department of Connectomics accesses a compute cluster containing 2500 CPU cores and 32 GPUs via MATLAB Parallel Server™. The team used Parallel Computing Toolbox™ to help parallelize the image preprocessing algorithms and custom CNN classifiers. Aside from a global image registration step, most compute steps in the dense reconstruction workflow were data parallel because the classifiers could be run on different portions of the sample volume simultaneously.
“多功能性和速度是我们开发过程中的首要任务。从初始想法转变为高度平行的生产部署的能力而无需重写代码或重新思考数据结构对我们的团队至关重要。”
Moritz Helmstaedter, Director, Max Planck Institute for Brain Research, Department of Connectomics
500000立方微米的样本体积reconstruction, the FocusEM compute steps took approximately 100 hours of compute time. Compared with the 4000 human work hours required, the computational work was therefore not a bottleneck. Most of the FocusEM processing was computed on the CPUs, and they used about 20% of their local CPU capacity (384 cores). GPUs were used to accelerate training of the SegEM custom deep learning classifier used for image segmentation.
Analyzing the Connectome
Max Planck研究人员在完成了500,000立方微米的大脑皮层的第一次密集重建后，分析了所得的连接性和几何数据。他们的分析产生了对这个生存神经网络的局部特性的宝贵见解：
- Distinct classes of neurons (excitatoryandinhibitory) contacted their target cells with distinct innervation patterns, confirming the findings of previous experiments using only connectomic data.
- Geometry-based rules for how axons and dendrites fill cortical volumes do not explain the connectivity patterns observed, as has been proposed by some prior theoretical models.
- The measured distribution of synaptic sizes in the connectome can provide insights into learning processes that may have occurred in the brain.
With a sample volume more than 300 times larger than past dense cortical reconstructions, spanning about 7000 axons and about 400,000 synapses, this study provided a level of statistical power not previously available for addressing such questions of local brain circuitry.
Plans for Further Research
Having established the feasibility and scientific value of dense cortical reconstruction at the current scale of 500,000 cubic micrometers, the Max Planck research team is now working to obtain more types of brain samples, to allow comparisons between species and between different brain states, such as diseased and healthy states.
The Department of Connectomics has also begun to tackle the further challenge of reconstructing larger cortical volumes spanning multiple brain layers and containing longer-distance neural connections. They continue to improve automation techniques such as FocusEM to lower the costs of dense reconstructions. Work is under way to analyze a petabyte-sized dataset from a cubic millimeter sample volume, which matches the scale of functional units identified in past studies of brain function. The results achieved to date using parallelized MATLAB show they can complete the FocusEM compute steps for a petabyte-sized dataset on their local cluster without impeding the overall reconstruction effort.
1Readers may view, browse, and/or download material for temporary copying purposes only, provided these uses are for noncommercial personal purposes. Except as provided by law, this material may not be further reproduced, distributed, transmitted, modified, adapted, performed, displayed, published, or sold in whole or in part, without prior written permission from the publisher.
