SCIENTIFIC COMPUTING AND IMAGING INSTITUTE
at the University of Utah

Advances in three-dimensional electron microscopy (EM) have facilitated the collection of image stacks with a field-of-view that is large enough to cover a significant percentage of anatomical subdivisions at nano-resolution. When coupled with enhanced staining protocols, such techniques produce data that can be mined to establish the morphologies of all organelles across hundreds of whole cells in their in situ environments. Although instrument throughputs are approaching terabytes of data per day, image segmentation and analysis remain significant bottlenecks in achieving quantitative descriptions of whole cell organellomes. Here we describe computational workflows that achieve the automatic segmentation of organelles from regions of the central nervous system by applying supervised machine learning algorithms to slices of serial block-face scanning EM (SBEM) datasets. We also demonstrate that our workflows can be parallelized on supercomputing resources, resulting in a dramatic reduction of their run times. These methods significantly expedite the development of anatomical models at the subcellular scale and facilitate the study of how these models may be perturbed following pathological insults.

Bayesian frameworks are commonly used in tracking algorithms. An important example is the particle filter, where a stochastic motion model describes the evolution of the state, and the observation model relates the noisy measurements to the state. Particle filters have been used to track the lineage of cells. Propagating the shape model of the cell through the particle filter is beneficial for tracking. We approximate arbitrary shapes of cells with a novel implicit convex function. The importance sampling step of the particle filter is defined using the cost associated with fitting our implicit convex shape model to the observations. Our technique is capable of tracking the lineage of cells for nonmitotic stages. We validate our algorithm by tracking the lineage of retinal and lens cells in zebrafish embryos.

Altered renal function commonly affects patients with cirrhosis, a consequence of chronic liver disease. From lowdose contrast material-enhanced magnetic resonance (MR) renography, we can estimate the Glomerular Filtration Rate (GFR), an important parameter to assess renal function. Two-dimensional MR images are acquired every 2 seconds for approximately 5 minutes during free breathing, which results in a dynamic series of 140 images representing kidney filtration over time. This specific acquisition presents dynamic contrast changes but is also challenged by organ motion due to breathing. Rather than use conventional image registration techniques, we opted for an alternative method based on object detection. We developed a novel analysis framework available under a stand-alone toolkit to efficiently register dynamic kidney series, manually select regions of interest, visualize the concentration curves for these ROIs, and fit them into a model to obtain GFR values. This open-source cross-platform application is written in C++, using the Insight Segmentation and Registration Toolkit (ITK) library, and QT4 as a graphical user interface.

Early brain development of white matter is characterized by rapid organization and structuring. Magnetic Resonance diffusion tensor imaging (MR-DTI) provides the possibility of capturing these changes non-invasively by following individuals longitudinally in order to better understand departures from normal brain development in subjects at risk for mental illness [1]. Longitudinal imaging of individuals suggests the use of 4D (3D, time) image analysis and longitudinal statistical modeling [3].

The term prediction implies expected outcome in the future, often based on a model and statistical inference. Longitudinal imaging studies offer the possibility to model temporal change trajectories of anatomy across populations of subjects. In the spirit of subject-specific analysis, such normative models can then be used to compare data from new subjects to the norm and to study progression of disease or to predict outcome. This paper follows a statistical inference approach and presents a framework for prediction of future observations based on past measurements and population statistics. We describe prediction in the context of nonlinear mixed effects modeling (NLME) where the full reference population's statistics (estimated fixed effects, variance-covariance of random effects, variance of noise) is used along with the individual's available observations to predict its trajectory. The proposed methodology is generic in regard to application domains. Here, we demonstrate analysis of early infant brain maturation from longitudinal DTI with up to three time points. Growth as observed in DTI-derived scalar invariants is modeled with a parametric function, its parameters being input to NLME population modeling. Trajectories of new subject's data are estimated when using no observation, only the rst or the first two time points. Leave-one-out experiments result in statistics on differences between actual and predicted observations. We also simulate a clinical scenario of prediction on multiple categories, where trajectories predicted from multiple models are classified based on maximum likelihood criteria.

Lagrangian coherent structures are time-evolving surfaces that highlight areas in flow fields where neighboring advected particles diverge or converge. The detection and understanding of such structures is an important part of many applications such as in oceanography where there is a need to predict the dispersion of oil and other materials in the ocean. One of the most widely used tools for revealing Lagrangian coherent structures has been to calculate the finite-time Lyapunov exponents, whose maximal values appear as ridgelines to reveal Lagrangian coherent structures. In this paper we explore an alternative formulation of Lyapunov exponents for computing Lagrangian coherent structures.

We develop a novel supervised learning/classification method, called disjunctive normal random forest (DNRF). A DNRF is an ensemble of randomly trained disjunctive normal decision trees (DNDT). To construct a DNDT, we formulate each decision tree in the random forest as a disjunction of rules, which are conjunctions of Boolean functions. We then approximate this disjunction of conjunctions with a differentiable function and approach the learning process as a risk minimization problem that incorporates the classification error into a single global objective function. The minimization problem is solved using gradient descent. DNRFs are able to learn complex decision boundaries and achieve low generalization error. We present experimental results demonstrating the improved performance of DNDTs and DNRFs over conventional decision trees and random forests. We also show the superior performance of DNRFs over state-of-the-art classification methods on benchmark datasets.

Keywords: Random forest, Decision tree, Classifier, Supervised learning, Disjunctive normal form

Scene labeling is the problem of assigning an object label to each pixel. It unifies the image segmentation and object recognition problems. The importance of using contextual information in scene labeling frameworks has been widely realized in the field. We propose a contextual framework, called contextual hierarchical model (CHM), which learns contextual information in a hierarchical framework for scene labeling. At each level of the hierarchy, a classifier is trained based on downsampled input images and outputs of previous levels. Our model then incorporates the resulting multi-resolution contextual information into a classifier to segment the input image at original resolution. This training strategy allows for optimization of a joint posterior probability at multiple resolutions through the hierarchy. Contextual hierarchical model is purely based on the input image patches and does not make use of any fragments or shape examples. Hence, it is applicable to a variety of problems such as object segmentation and edge detection. We demonstrate that CHM outperforms state-of-the-art on Stanford background and Weizmann horse datasets. It also outperforms state-of-the-art edge detection methods on NYU depth dataset and achieves state-of-the-art on Berkeley segmentation dataset (BSDS 500).

Temporal modeling frameworks often operate on scalar variables by summarizing data at initial stages as statistical summaries of the underlying distributions. For instance, DTI analysis often employs summary statistics, like mean, for regions of interest and properties along fiber tracts for population studies and hypothesis testing. This reduction via discarding of variability information may introduce significant errors which propagate through the procedures. We propose a novel framework which uses distribution-valued variables to retain and utilize the local variability information. Classic linear regression is adapted to employ these variables for model estimation. The increased stability and reliability of our proposed method when compared with regression using single-valued statistical summaries, is demonstrated in a validation experiment with synthetic data. Our driving application is the modeling of age-related changes along DTI white matter tracts. Results are shown for the spatiotemporal population trajectory of genu tract estimated from 45 healthy infants and compared with a Krabbe's patient.

Keywords: linear regression, distribution-valued data, spatiotemporal growth trajectory, DTI, early neurodevelopment

We present a novel algorithm for computing hierarchical geodesic models (HGMs) for diffeomorphic longitudinal shape analysis. The proposed algorithm exploits the inherent parallelism arising out of the independence in the contributions of individual geodesics to the group geodesic. The previous serial implementation severely limits the use of HGMs to very small population sizes due to computation time and massive memory requirements. The conventional method makes it impossible to estimate the parameters of HGMs on large datasets due to limited memory available onboard current GPU computing devices. The proposed parallel algorithm easily scales to solve HGMs on a large collection of 3D images of several individuals. We demonstrate its effectiveness on longitudinal datasets of synthetically generated shapes and 3D magnetic resonance brain images (MRI).

Keywords: LDDMM, HGM, Vector Momentum, Diffeomorphisms, Longitudinal Analysis

Vector field simplification aims to reduce the complexity of the flow by removing features in order of their relevance and importance, to reveal prominent behavior and obtain a compact representation for interpretation. Most existing simplification techniques based on the topological skeleton successively remove pairs of critical points connected by separatrices, using distance or area-based relevance measures. These methods rely on the stable extraction of the topological skeleton, which can be difficult due to instability in numerical integration, especially when processing highly rotational flows. These geometric metrics do not consider the flow magnitude, an important physical property of the flow. In this paper, we propose a novel simplification scheme derived from the recently introduced topological notion of robustness, which provides a complementary view on flow structure compared to the traditional topological-skeleton-based approaches. Robustness enables the pruning of sets of critical points according to a quantitative measure of their stability, that is, the minimum amount of vector field perturbation required to remove them. This leads to a hierarchical simplification scheme that encodes flow magnitude in its perturbation metric. Our novel simplification algorithm is based on degree theory, has fewer boundary restrictions, and so can handle more general cases. Finally, we provide an implementation under the piecewise-linear setting and apply it to both synthetic and real-world datasets.

Keywords: vector field, topology-based techniques, flow visualization

A key challenge in the study of a time-varying vector fields is to resolve the correspondences between features in successive time steps and to analyze the dynamic behaviors of such features, so-called feature tracking. Commonly tracked features, such as volumes, areas, contours, boundaries, vortices, shock waves and critical points, represent interesting properties or structures of the data. Recently, the topological notion of robustness, a relative of persistent homology, has been introduced to quantify the stability of critical points. Intuitively, the robustness of a critical point is the minimum amount of perturbation necessary to cancel it. In this chapter, we offer a fresh interpretation of the notion of feature tracking, in particular, critical point tracking, through the lens of robustness.We infer correspondences between critical points based on their closeness in stability, measured by robustness, instead of just distance proximities within the domain. We prove formally that robustness helps us understand the sampling conditions under which we can resolve the correspondence problem based on region overlap techniques, and the uniqueness and uncertainty associated with such techniques. These conditions also give a theoretical basis for visualizing the piecewise linear realizations of critical point trajectories over time.

Recently, multi-scale notions of local homology (a variant of persistent homology) have been used to study the local structure of spaces around a given point from a point cloud sample. Current reconstruction guarantees rely on constructing embedded complexes which become diffcult to construct in higher dimensions. We show that the persistence diagrams used for estimating local homology can be approximated using families of Vietoris-Rips complexes, whose simpler construction are robust in any dimension. To the best of our knowledge, our results, for the first time make applications based on local homology, such as stratification learning, feasible in high dimensions.

Image boundaries are a fundamental component of many interactive digital photography techniques, enabling applications such as segmentation, panoramas, and seamless image composition. Interactions for image boundaries often rely on two complimentary but separate approaches: editing via painting or clicking constraints. In this work, we provide a novel, unified approach for interactive editing of pairwise image boundaries that combines the ease of painting with the direct control of constraints. Rather than a sequential coupling, this new formulation allows full use of both interactions simultaneously, giving users unprecedented flexibility for fast boundary editing. To enable this new approach, we provide technical advancements. In particular, we detail a reformulation of image boundaries as a problem of finding cycles, expanding and correcting limitations of the previous work. Our new formulation provides boundary solutions for painted regions with performance on par with state-of-the-art specialized, paint-only techniques. In addition, we provide instantaneous exploration of the boundary solution space with user constraints. Furthermore, we show how to increase performance and decrease memory consumption through novel strategies and/or optional approximations. Finally, we provide examples of common graphics applications impacted by our new approach.

Reconstruction of neural circuits at the microscopic scale of individual neurons and synapses, also known as connectomics, is an important challenge for neuroscience. While an important motivation of connectomics is providing anatomical ground truth for neural circuit models, the ability to decipher neural wiring maps at the individual cell level is also important in studies of many neurodegenerative diseases. Reconstruction of a neural circuit at the individual neuron level requires the use of electron microscopy images due to their extremely high resolution. Computational challenges include pixel-by-pixel annotation of these images into classes such as cell membrane, mitochondria and synaptic vesicles and the segmentation of individual neurons. State-of-the-art image analysis solutions are still far from the accuracy and robustness of human vision and biologists are still limited to studying small neural circuits using mostly manual analysis. In this chapter, we describe our image analysis pipeline that makes use of novel supervised machine learning techniques to tackle this problem.

Dual analysis uses statistics to describe both the dimensions and rows of a high-dimensional dataset. Researchers have integrated it into StratomeX, a Caleydo view for cancer subtype analysis. In addition, significant-difference plots show the elements of a candidate subtype that differ significantly from other subtypes, thus letting analysts characterize subtypes. Analysts can also investigate how data samples relate to their assigned subtype and other groups. This approach lets them create well-defined subtypes based on statistical properties. Three case studies demonstrate the approach's utility, showing how it reproduced findings from a published subtype characterization.

The details of the mechanical factors that modulate angiogenesis remain poorly understood. Previous in vitro studies of angiogenesis using microvessel fragments cultured within collagen constructs demonstrated that neovessel alignment can be induced via mechanical constraint of the boundaries (i.e., boundary conditions). The objective of this study was to investigate the role of mechanical boundary conditions in the regulation of angiogenic alignment and growth in an in vitro model of angiogenesis. Angiogenic microvessels within three-dimensional constructs were subjected to different boundary conditions, thus producing different stress and strain fields during growth. Neovessel outgrowth and orientation were quantified from confocal image data after 6 days. Vascularity and branching decreased as the amount of constraint imposed on the culture increased. In long-axis constrained hexahedral constructs, microvessels aligned parallel to the constrained axis. In contrast, constructs that were constrained along the short axis had random microvessel orientation. Finite element models were used to simulate the contraction of gels under the various boundary conditions and to predict the local strain field experienced by microvessels. Results from the experiments and simulations demonstrated that microvessels aligned perpendicular to directions of compressive strain. Alignment was due to anisotropic deformation of the matrix from cell-generated traction forces interacting with the mechanical boundary conditions. These findings demonstrate that boundary conditions and thus the effective stiffness of the matrix regulate angiogenesis. This study offers a potential explanation for the oriented vascular beds that occur in native tissues and provides the basis for improved control of tissue vascularization in both native tissues and tissue-engineered constructs.

Keywords: angiogenesis, deformation, image analysis, morphometry, orientation, strain

Modeling of early brain growth trajectories from longitudinal MRI will provide new insight into neurodevelopmental characteristics, timing and type of changes in neurological disorders from controls. In addition to an ongoing large-scale infant autism neuroimaging study 1, we recruited 4 infants with Down’s syndrome (DS) in order to evaluate newly developed methods for 4D segmentation from longitudinal infant MRI, and for temporal modeling of brain growth trajectories. Specifically to Down's, a comparison of patterns of full brain and lobar tissue growth may lead to better insight into the observed variability of cognitive development and neurological effects, and may help with development of disease-modifying therapeutic intervention.