SCI Publications

Contextual information plays an important role in solving vision problems such as image segmentation. However, extracting contextual information and using it in an effective way remains a difficult problem. To address this challenge, we propose a multi-resolution contextual framework, called cascaded hierarchical model (CHM), which learns contextual information in a hierarchical framework for image segmentation. At each level of the hierarchy, a classifier is trained based on downsampled input images and outputs of previous levels. Our model then incorporates the resulting multi-resolution contextual information into a classifier to segment the input image at original resolution. We repeat this procedure by cascading the hierarchical framework to improve the segmentation accuracy. Multiple classifiers are learned in the CHM; therefore, a fast and accurate classifier is required to make the training tractable. The classifier also needs to be robust against overfitting due to the large number of parameters learned during training. We introduce a novel classification scheme, called logistic disjunctive normal networks (LDNN), which consists of one adaptive layer of feature detectors implemented by logistic sigmoid functions followed by two fixed layers of logical units that compute conjunctions and disjunctions, respectively. We demonstrate that LDNN outperforms state-of-theart classifiers and can be used in the CHM to improve object segmentation performance.

There are many applications in MRI where it is desirable to have high spatial and high temporal resolution. This can be achieved by undersampling of k-space and requires special techniques for reconstruction. Even if undersampling artifacts are removed, sharpness of the edges can be a problem. We propose a new technique that uses the gradient from a reference image to improve the quality of the edges in the reconstructed image along with a spatio-temporal constraint to reduce aliasing artifacts and noise. The reference is created from undersampled dynamic data by combining several adjacent frames. The method was tested on undersampled radial DCE MRI data with little respiratory motion. The proposed method was compared to reconstruction using the spatio-temporal constrained reconstruction. Sharper edges and an increase in the contrast was observed by using the proposed method.

In the context of long-range digital neural circuit reconstruction, this paper investigates an approach for registering axons across histological serial sections. Tracing distinctly labeled axons over large distances allows neuroscientists to study very explicit relationships between the brain's complex interconnects and, for example, diseases or aberrant development. Large scale histological analysis requires, however, that the tissue be cut into sections. In immunohistochemical studies thin sections are easily distorted due to the cutting, preparation, and slide mounting processes. In this work we target the registration of thin serial sections containing axons. Sections are first traced to extract axon centerlines, and these traces are used to define registration landmarks where they intersect section boundaries. The trace data also provides distinguishing information regarding an axon's size and orientation within a section. We propose the use of these features when pairing axons across sections in addition to utilizing the spatial relationships among the landmarks. The global rotation and translation of an unregistered section are accounted for using a random sample consensus (RANSAC) based technique. An iterative nonrigid refinement process using B-spline warping is then used to reconnect axons and produce the sought after connectivity information.

Dynamic contrast-enhanced magnetic resonance imaging (MRI) is a technique used to study and track contrast kinetics in an area of interest in the body over time. Reconstruction of images with high contrast and sharp edges from undersampled data is a challenge. While good results have been reported using a radial acquisition and a spatiotemporal constrained reconstruction (STCR) method, we propose improvements from using spatially adaptive weighting and an additional edge-based constraint. The new method uses intensity gradients from a sliding window reference image to improve the sharpness of edges in the reconstructed image. The method was tested on eight radial cardiac perfusion data sets with 24 rays and compared to the STCR method. The reconstructions showed that the new method, termed edge-enhanced spatiotemporal constrained reconstruction, was able to reconstruct images with sharper edges, and there were a 36\%±13.7\% increase in contrast-to-noise ratio and a 24\%±11\% increase in contrast near the edges when compared to STCR. The novelty of this paper is the combination of spatially adaptive weighting for spatial total variation (TV) constraint along with a gradient matching term to improve the sharpness of edges. The edge map from a reference image allows the reconstruction to trade-off between TV and edge enhancement, depending on the spatially varying weighting provided by the edge map.

Keywords: MRI, Reconstruction, Edge enhanced, Compressed sensing, Regularization, Cardiac perfusion

Neuroscientists are developing new imaging techniques and generating large volumes of data in an effort to understand the complex structure of the nervous system. The complexity and size of this data makes human interpretation a labor-intensive task. To aid in the analysis, new segmentation techniques for identifying neurons in these feature rich datasets are required. This paper presents a method for neuron boundary detection and nonbranching process segmentation in electron microscopy images and visualizing them in three dimensions. It combines both automated segmentation techniques with a graphical user interface for correction of mistakes in the automated process. The automated process first uses machine learning and image processing techniques to identify neuron membranes that deliniate the cells in each two-dimensional section. To segment nonbranching processes, the cell regions in each two-dimensional section are connected in 3D using correlation of regions between sections. The combination of this method with a graphical user interface specially designed for this purpose, enables users to quickly segment cellular processes in large volumes.

Automated segmentation of electron microscopy (EM) images is a challenging problem. In this paper, we present a novel method that utilizes a hierarchical structure and boundary classification for 2D neuron segmentation. With a membrane detection probability map, a watershed merge tree is built for the representation of hierarchical region merging from the watershed algorithm. A boundary classifier is learned with non-local image features to predict each potential merge in the tree, upon which merge decisions are made with consistency constraints to acquire the final segmentation. Independent of classifiers and decision strategies, our approach proposes a general framework for efficient hierarchical segmentation with statistical learning. We demonstrate that our method leads to a substantial improvement in segmentation accuracy.

Automatic fingerprint identification systems (AFIS) have been studied extensively and are widely used for biometric identification. Given its importance, many well-engineered methods have been developed for the different stages that encompass those systems. The first stage of any such system is the segmentation of the actual fingerprint region from the background. This is typically achieved by classifying pixels, or blocks of pixels, based on a set of features. In this paper, we describe novel features for fingerprint segmentation that express the underlying manifold topology associated with image patches in a local neighborhood. It is shown that fingerprint patches seen in a high-dimensional space form a simple and highly regular circular manifold. The characterization of the manifold topology suggests a set of optimal features that characterize the local properties of the fingerprint. Thus, fingerprint segmentation can be formulated as a classification problem based on the deviation from the expected topology. This leads to features that are more robust to changes in contrast than mean, variance and coherence. The superior performance of the proposed features for fingerprint segmentation is shown in the eight datasets from the 2002 and 2004 Fingerprint Verification Competitions.

Keywords: Fingerprint segmentation, manifold characterization, feature extraction, dimensionality reduction

An automated system for differential white blood cell (WBC) counting based on morphology can make manual differential leukocyte counts faster and less tedious for pathologists and laboratory professionals. We present an automated system for isolation and classification of WBCs in manually prepared, Wright stained, peripheral blood smears from whole slide images (WSI). Methods: A simple, classification scheme using color information and morphology is proposed. The performance of the algorithm was evaluated by comparing our proposed method with a hematopathologist's visual classification. The isolation algorithm was applied to 1938 subimages of WBCs, 1804 of them were accurately isolated. Then, as the first step of a two-step classification process, WBCs were broadly classified into cells with segmented nuclei and cells with nonsegmented nuclei. The nucleus shape is one of the key factors in deciding how to classify WBCs. Ambiguities associated with connected nuclear lobes are resolved by detecting maximum curvature points and partitioning them using geometric rules. The second step is to define a set of features using the information from the cytoplasm and nuclear regions to classify WBCs using linear discriminant analysis. This two-step classification approach stratifies normal WBC types accurately from a whole slide image. Results: System evaluation is performed using a 10-fold cross-validation technique. Confusion matrix of the classifier is presented to evaluate the accuracy for each type of WBC detection. Experiments show that the two-step classification implemented achieves a 93.9\% overall accuracy in the five subtype classification. Conclusion: Our methodology achieves a semiautomated system for the detection and classification of normal WBCs from scanned WSI. Further studies will be focused on detecting and segmenting abnormal WBCs, comparison of 20x and 40x data, and expanding the applications for bone marrow aspirates.

Bone marrow evaluation is indicated when peripheral blood abnormalities are not explained by clinical, physical, or laboratory findings. In this paper, we propose a novel method for segmentation of haematopoietic cells in the bone marrow from scanned slide images. Segmentation of clumped cells is a challenging problem for this application. We first use color information and morphology to eliminate red blood cells and the background. Clumped haematopoietic cells are then segmented using circle detection and a splitting algorithm based on the detected circle centers. The Hough Transform is used for circle detection and to find the number and positions of circle centers in each region. The splitting algorithm is based on detecting the maximum curvature points, and partitioning them based on information obtained from the centers of the circles in each region. The performance of the segmentation algorithm for haematopoietic cells is evaluated by comparing our proposed method with a hematologist's visual segmentation in a set of 3748 cells.

Purpose: A connectome is a comprehensive description of synaptic connectivity for a neural domain. Our goal was to produce a connectome data set for the inner plexiform layer of the mammalian retina. This paper describes our first retinal connectome, validates the method, and provides key initial findings.

Methods: We acquired and assembled a 16.5 terabyte connectome data set RC1 for the rabbit retina at .2 nm resolution using automated transmission electron microscope imaging, automated mosaicking, and automated volume registration. RC1 represents a column of tissue 0.25 mm in diameter, spanning the inner nuclear, inner plexiform, and ganglion cell layers. To enhance ultrastructural tracing, we included molecular markers for 4-aminobutyrate (GABA), glutamate, glycine, taurine, glutamine, and the in vivo activity marker, 1-amino-4-guanidobutane. This enabled us to distinguish GABAergic and glycinergic amacrine cells; to identify ON bipolar cells coupled to glycinergic cells; and to discriminate different kinds of bipolar, amacrine, and ganglion cells based on their molecular signatures and activity. The data set was explored and annotated with Viking, our multiuser navigation tool. Annotations were exported to additional applications to render cells, visualize network graphs, and query the database.

Results: Exploration of RC1 showed that the 2 nm resolution readily recapitulated well known connections and revealed several new features of retinal organization: (1) The well known AII amacrine cell pathway displayed more complexity than previously reported, with no less than 17 distinct signaling modes, including ribbon synapse inputs from OFF bipolar cells, wide-field ON cone bipolar cells and rod bipolar cells, and extensive input from cone-pathway amacrine cells. (2) The axons of most cone bipolar cells formed a distinct signal integration compartment, with ON cone bipolar cell axonal synapses targeting diverse cell types. Both ON and OFF bipolar cells receive axonal veto synapses. (3) Chains of conventional synapses were very common, with intercalated glycinergic-GABAergic chains and very long chains associated with starburst amacrine cells. Glycinergic amacrine cells clearly play a major role in ON-OFF crossover inhibition. (4) Molecular and excitation mapping clearly segregates ultrastructurally defined bipolar cell groups into different response clusters. (5) Finally, low-resolution electron or optical imaging cannot reliably map synaptic connections by process geometry, as adjacency without synaptic contact is abundant in the retina. Only direct visualization of synapses and gap junctions suffices.

Conclusions: Connectome assembly and analysis using conventional transmission electron microscopy is now practical for network discovery. Our surveys of volume RC1 demonstrate that previously studied systems such as the AII amacrine cell network involve more network motifs than previously known. The AII network, primarily considered a scotopic pathway, clearly derives ribbon synapse input from photopic ON and OFF cone bipolar cell networks and extensive photopic GABAergic amacrine cell inputs. Further, bipolar cells show extensive inputs and outputs along their axons, similar to multistratified nonmammalian bipolar cells. Physiologic evidence of significant ON-OFF channel crossover is strongly supported by our anatomic data, showing alternating glycine-to-GABA paths. Long chains of amacrine cell networks likely arise from homocellular GABAergic synapses between starburst amacrine cells. Deeper analysis of RC1 offers the opportunity for more complete descriptions of specific networks.

Keywords: neuroscience, retina, vision, blindness, visus, crcns

Modern microscope automation permits the collection of vast amounts of continuous anatomical imagery in both two and three dimensions. These large data sets present significant challenges for data storage, access, viewing, annotation and analysis. The cost and overhead of collecting and storing the data can be extremely high. Large data sets quickly exceed an individual's capability for timely analysis and present challenges in efficiently applying transforms, if needed. Finally annotated anatomical data sets can represent a significant investment of resources and should be easily accessible to the scientific community. The Viking application was our solution created to view and annotate a 16.5 TB ultrastructural retinal connectome volume and we demonstrate its utility in reconstructing neural networks for a distinctive retinal amacrine cell class. Viking has several key features. (1) It works over the internet using HTTP and supports many concurrent users limited only by hardware. (2) It supports a multi-user, collaborative annotation strategy. (3) It cleanly demarcates viewing and analysis from data collection and hosting. (4) It is capable of applying transformations in real-time. (5) It has an easily extensible user interface, allowing addition of specialized modules without rewriting the viewer.

Keywords: Annotation, automated electron microscopy, citizen science, computational methods, connectome, networks, visualization

Neuroscientists are developing new imaging techniques and generating large volumes of data in an effort to understand the complex structure of the nervous system. To aid in the analysis, new segmentation techniques for identifying neurons in these feature rich datasets are required. However, the extremely anisotropic resolution of the data makes segmentation and tracking across slices difficult. This paper presents a complete method for segmenting neurons in electron microscopy images and visualizing them in three dimensions. First, we present an advanced method for identifying neuron membranes, necessary for whole neuron segmentation, using a machine learning approach. Next, neurons are segmented in each two-dimensional section and connected using correlation of regions between sections. These techniques, combined with a visual user interface, enable users to quickly segment whole neurons in large volumes.

In this paper, a new AdaBoost learning framework, called WNS-AdaBoost, is proposed for training discriminative models. The proposed approach significantly speeds up the learning process of adaptive boosting (AdaBoost) by reducing the number of data points. For this purpose, we introduce the weighted novelty selection (WNS) sampling strategy and combine it with AdaBoost to obtain an efficient and fast learning algorithm. WNS selects a representative subset of data thereby reducing the number of data points onto which AdaBoost is applied. In addition, WNS associates a weight with each selected data point such that the weighted subset approximates the distribution of all the training data. This ensures that AdaBoost can trained efficiently and with minimal loss of accuracy. The performance of WNS-AdaBoost is first demonstrated in a classification task. Then, WNS is employed in a probabilistic boosting-tree (PBT) structure for image segmentation. Results in these two applications show that the training time using WNS-AdaBoost is greatly reduced at the cost of only a few percent in accuracy.