Designed especially for neurobiologists, FluoRender is an interactive tool for multi-channel fluorescence microscopy data visualization and analysis.
Deep brain stimulation
BrainStimulator is a set of networks that are used in SCIRun to perform simulations of brain stimulation such as transcranial direct current stimulation (tDCS) and magnetic transcranial stimulation (TMS).
Developing software tools for science has always been a central vision of the SCI Institute.

Image Analysis

SCI's imaging work addresses fundamental questions in 2D and 3D image processing, including filtering, segmentation, surface reconstruction, and shape analysis. In low-level image processing, this effort has produce new nonparametric methods for modeling image statistics, which have resulted in better algorithms for denoising and reconstruction. Work with particle systems has led to new methods for visualizing and analyzing 3D surfaces. Our work in image processing also includes applications of advanced computing to 3D images, which has resulted in new parallel algorithms and real-time implementations on graphics processing units (GPUs). Application areas include medical image analysis, biological image processing, defense, environmental monitoring, and oil and gas.


Ross Whitaker


Sarang Joshi

Shape Statistics
Brain Atlasing

Tolga Tasdizen

Image Processing
Machine Learning

Chris Johnson

Diffusion Tensor Analysis

Shireen Elhabian

Image Analysis
Computer Vision

Funded Research Projects:

Publications in Image Analysis:

Small-molecule mimicry hunting strategy in the imperial cone snail, Conus imperialis
J. P. Torres, Z. Lin, M. Watkins, P. F. Salcedo, R. P. Baskin, S. Elhabian, H. Safavi-Hemami, D. Taylor, J. Tun, G. P. Concepcion, N. Saguil, A. A. Yanagihara, Y. Fang, J. R. McArthur, H. Tae, R. K. Finol-Urdaneta, B. D. Özpolat, B. M. Olivera, E. W. Schmidt. In Science Advances, Vol. 7, No. 11, American Association for the Advancement of Science, 2021.

Venomous animals hunt using bioactive peptides, but relatively little is known about venom small molecules and the resulting complex hunting behaviors. Here, we explored the specialized metabolites from the venom of the worm-hunting cone snail, Conus imperialis. Using the model polychaete worm Platynereis dumerilii, we demonstrate that C. imperialis venom contains small molecules that mimic natural polychaete mating pheromones, evoking the mating phenotype in worms. The specialized metabolites from different cone snails are species-specific and structurally diverse, suggesting that the cones may adopt many different prey-hunting strategies enabled by small molecules. Predators sometimes attract prey using the prey’s own pheromones, in a strategy known as aggressive mimicry. Instead, C. imperialis uses metabolically stable mimics of those pheromones, indicating that, in biological mimicry, even the molecules themselves may be disguised, providing a twist on fake news in chemical ecology.

Learning Deep Features for Shape Correspondence with Domain Invariance
Subtitled “arXiv preprint arXiv:2102.10493,” P. Agrawal, R. T. Whitaker, S. Y. Elhabian. 2021.

Correspondence-based shape models are key to various medical imaging applications that rely on a statistical analysis of anatomies. Such shape models are expected to represent consistent anatomical features across the population for population-specific shape statistics. Early approaches for correspondence placement rely on nearest neighbor search for simpler anatomies. Coordinate transformations for shape correspondence hold promise to address the increasing anatomical complexities. Nonetheless, due to the inherent shape-level geometric complexity and population-level shape variation, the coordinate-wise correspondence often does not translate to the anatomical correspondence. An alternative, group-wise approach for correspondence placement explicitly models the trade-off between geometric description and the population's statistical compactness. However, these models achieve limited success in resolving nonlinear shape correspondence. Recent works have addressed this limitation by adopting an application-specific notion of correspondence through lifting positional data to a higher dimensional feature space. However, they heavily rely on manual expertise to create domain-specific features and consistent landmarks. This paper proposes an automated feature learning approach, using deep convolutional neural networks to extract correspondence-friendly features from shape ensembles. Further, an unsupervised domain adaptation scheme is introduced to augment the pretrained geometric features with new anatomies. Results on anatomical datasets of human scapula, femur, and pelvis bones demonstrate that …

Image-Based Multiresolution Topology Optimization Using Deep Disjunctive Normal Shape Model
V. Keshavarzzadeh, M. Alirezaei, T. Tasdizen, R. M. Kirby. In Computer-Aided Design, Vol. 130, Elsevier, pp. 102947. 2021.

We present a machine learning framework for predicting the optimized structural topology design susing multiresolution data. Our approach primarily uses optimized designs from inexpensive coarse mesh finite element simulations for model training and generates high resolution images associated with simulation parameters that are not previously used. Our cost-efficient approach enables the designers to effectively search through possible candidate designs in situations where the design requirements rapidly change. The underlying neural network framework is based on a deep disjunctive normal shape model (DDNSM) which learns the mapping between the simulation parameters and segments of multi resolution images. Using this image-based analysis we provide a practical algorithm which enhances the predictability of the learning machine by determining a limited number of important parametric samples(i.e.samples of the simulation parameters)on which the high resolution training data is generated. We demonstrate our approach on benchmark compliance minimization problems including the 3D topology optimization where we show that the high-fidelity designs from the learning machine are close to optimal designs and can be used as effective initial guesses for the large-scale optimization problem.

Detection and segmentation in microscopy images,
N. Ramesh, T. Tasdizen. In Computer Vision for Microscopy Image Analysis, Academic Press, pp. 43-71. 2021.
DOI: 10.1016/B978-0-12-814972-0.00003-5

The plethora of heterogeneous data generated using modern microscopy imaging techniques eliminates the possibility of manual image analysis for biologists. Consequently, reliable and robust computerized techniques are critical to analyze microscopy data. Detection problems in microscopy images focuses on accurately identifying the objects of interest in an image that can be used to investigate hypotheses about developmental or pathological processes and can be indicative of prognosis in patients. Detection is also considered to be the preliminary step for solving subsequent problems, such as segmentation and tracking for various biological applications. Segmentation of the desired structures and regions in microscopy images require pixel-level labels to uniquely identify the individual structures and regions with contours for morphological and physiological analysis. Distributions of features extracted from the segmented regions can be used to compare normal versus disease or normal versus wild-type populations. Segmentation can be considered as a precursor for solving classification, reconstruction, and tracking problems in microscopy images. In this chapter, we discuss how the field of microscopic image analysis has progressed over the years, starting with traditional approaches and then followed by the study of learning algorithms. Because there is a lot of variability in microscopy data, it is essential to study learning algorithms that can adapt to these changes. We focus on deep learning approaches with convolutional neural networks (CNNs), as well as hierarchical methods for segmentation and detection in optical and electron microscopy images. Limitation of training data is one of the significant problems; hence, we explore solutions to learn better models with minimal user annotations.

Leveraging 31 Million Google Street View Images to Characterize Built Environments and Examine County Health Outcomes
Q. C Nguyen, J. M. Keralis, P. Dwivedi, A. E. Ng, M. Javanmardi, S. Khanna, Y. Huang, K. D. Brunisholz, A. Kumar, T. Tasdizen. In Public Health Reports, Vol. 136, No. 2, SAGE Publications, pp. 201-211. 2021.

Built environments can affect health, but data in many geographic areas are limited. We used a big data source to create national indicators of neighborhood quality and assess their associations with health.

We leveraged computer vision and Google Street View images accessed from December 15, 2017, through July 17, 2018, to detect features of the built environment (presence of a crosswalk, non–single-family home, single-lane roads, and visible utility wires) for 2916 US counties. We used multivariate linear regression models to determine associations between features of the built environment and county-level health outcomes (prevalence of adult obesity, prevalence of diabetes, physical inactivity, frequent physical and mental distress, poor or fair self-rated health, and premature death [in years of potential life lost]).
Compared with counties with the least number of crosswalks, counties with the most crosswalks were associated with decreases of 1.3%, 2.7%, and 1.3% of adult obesity, physical inactivity, and fair or poor self-rated health, respectively, and 477 fewer years of potential life lost before age 75 (per 100 000 population). The presence of non–single-family homes was associated with lower levels of all health outcomes except for premature death. The presence of single-lane roads was associated with an increase in physical inactivity, frequent physical distress, and fair or poor self-rated health. Visible utility wires were associated with increases in adult obesity, diabetes, physical and mental distress, and fair or poor self-rated health.
The use of computer vision and big data image sources makes possible national studies of the built environm

Lessons learned towards the immediate delivery of massive aerial imagery to farmers and crop consultants
A. A. Gooch, S. Petruzza, A. Gyulassy, G. Scorzelli, V. Pascucci, L. Rantham, W. Adcock, C. Coopmans. In Autonomous Air and Ground Sensing Systems for Agricultural Optimization and Phenotyping VI, Vol. 11747, International Society for Optics and Photonics, pp. 22 -- 34. 2021.
DOI: 10.1117/12.2587694

In this paper, we document lessons learned from using ViSOAR Ag Explorer™ in the fields of Arkansas and Utah in the 2018-2020 growing seasons. Our insights come from creating software with fast reading and writing of 2D aerial image mosaics for platform-agnostic collaborative analytics and visualization. We currently enable stitching in the field on a laptop without the need for an internet connection. The full resolution result is then available for instant streaming visualization and analytics via Python scripting. While our software, ViSOAR Ag Explorer™ removes the time and labor software bottleneck in processing large aerial surveys, enabling a cost-effective process to deliver actionable information to farmers, we learned valuable lessons with regard to the acquisition, storage, viewing, analysis, and planning stages of aerial data surveys. Additionally, with the ultimate goal of stitching thousands of images in minutes on board a UAV at the time of data capture, we performed preliminary tests for on-board, real-time stitching and analysis on USU AggieAir sUAS using lightweight computational resources. This system is able to create a 2D map while flying and allow interactive exploration of the full resolution data as soon as the platform has landed or has access to a network. This capability further speeds up the assessment process on the field and opens opportunities for new real-time photogrammetry applications. Flying and imaging over 1500-2000 acres per week provides up-to-date maps that give crop consultants a much broader scope of the field in general as well as providing a better view into planting and field preparation than could be observed from field level. Ultimately, our software and hardware could provide a much better understanding of weed presence and intensity or lack thereof.

Structural Connectome Atlas Construction in the Space of Riemannian Metrics
Subtitled “arXiv,” K. M. Campbell, H. Dai, Z. Su, M. Bauer, P. T. Fletcher, S. C. Joshi. 2021.

The structural connectome is often represented by fiber bundles generated from various types of tractography. We propose a method of analyzing connectomes by representing them as a Riemannian metric, thereby viewing them as points in an infinite-dimensional manifold. After equipping this space with a natural metric structure, the Ebin metric, weapply object-oriented statistical analysis to define an atlas as the Fŕechet mean of a population of Riemannian metrics. We demonstrate connectome registration and atlas formation using connectomes derived from diffusion tensors estimated from a subset of subjects from the Human Connectome Project.

Physics Informed Convex Artificial Neural Networks (PICANNs) for Optimal Transport based Density Estimation
Subtitled “arXiv,” A. Singh, M. Bauer, S. Joshi. 2021.

Optimal Mass Transport (OMT) is a well studied problem with a variety of applications in a diverse set of fields ranging from Physics to Computer Vision and in particular Statistics and Data Science. Since the original formulation of Monge in 1781 significant theoretical progress been made on the existence, uniqueness and properties of the optimal transport maps. The actual numerical computation of the transport maps, particularly in high dimensions, remains a challenging problem. By Brenier's theorem, the continuous OMT problem can be reduced to that of solving a non-linear PDE of Monge-Ampere type whose solution is a convex function. In this paper, building on recent developments of input convex neural networks and physics informed neural networks for solving PDE's, we propose a Deep Learning approach to solve the continuous OMT problem.

To demonstrate the versatility of our framework we focus on the ubiquitous density estimation and generative modeling tasks in statistics and machine learning. Finally as an example we show how our framework can be incorporated with an autoencoder to estimate an effective probabilistic generative model.

Determining uranium ore concentrates and their calcination products via image classification of multiple magnifications,
C. Ly, C. Vachet, I. Schwerdt, E. Abbott, A. Brenkmann, L.W. McDonald, T. Tasdizen. In Journal of Nuclear Materials, 2020.

Many tools, such as mass spectrometry, X-ray diffraction, X-ray fluorescence, ion chromatography, etc., are currently available to scientists investigating interdicted nuclear material. These tools provide an analysis of physical, chemical, or isotopic characteristics of the seized material to identify its origin. In this study, a novel technique that characterizes physical attributes is proposed to provide insight into the processing route of unknown uranium ore concentrates (UOCs) and their calcination products. In particular, this study focuses on the characteristics of the surface structure captured in scanning electron microscopy (SEM) images at different magnification levels. Twelve common commercial processing routes of UOCs and their calcination products are investigated. Multiple-input single-output (MISO) convolution neural networks (CNNs) are implemented to differentiate the processing routes. The proposed technique can determine the processing route of a given sample in under a second running on a graphics processing unit (GPU) with an accuracy of more than 95%. The accuracy and speed of this proposed technique enable nuclear scientists to provide the preliminary identification results of interdicted material in a short time period. Furthermore, this proposed technique uses a predetermined set of magnifications, which in turn eliminates the human bias in selecting the magnification during the image acquisition process.

Adversarial regression training for visualizing the progression of chronic obstructive pulmonary disease with chest x-rays,
R.B. Lanfredi, J.D. Schroeder, C. Vachet, T. Tasdizen. In Arxiv, In International Conference on Medical Image Computing and Computer-Assisted Intervention, 2019.

Knowledge of what spatial elements of medical images deep learning methods use as evidence is important for model interpretability, trustiness, and validation. There is a lack of such techniques for models in regression tasks. We propose a method, called visualization for regression with a generative adversarial network (VR-GAN), for formulating adversarial training specifically for datasets containing regression target values characterizing disease severity. We use a conditional generative adversarial network where the generator attempts to learn to shift the output of a regressor through creating disease effect maps that are added to the original images. Meanwhile, the regressor is trained to predict the original regression value for the modified images. A model trained with this technique learns to provide visualization for how the image would appear at different stages of the disease. We analyze our method in a dataset of chest x-rays associated with pulmonary function tests, used for diagnosing chronic obstructive pulmonary disease (COPD). For validation, we compute the difference of two registered x-rays of the same patient at different time points and correlate it to the generated disease effect map. The proposed method outperforms a technique based on classification and provides realistic-looking images, making modifications to images following what radiologists usually observe for this disease. Implementation code is available at

A Cooperative Autoencoder for Population-Based Regularization of CNN Image Registration
R. Bhalodia, S. Y. Elhabian, L. Kavan, R. T. Whitaker. In Medical Image Computing and Computer Assisted Intervention – MICCAI 2019, In Medical Image Computing and Computer Assisted Intervention -- MICCAI 2019, Springer International Publishing, pp. 391--400. 2019.

Spatial transformations are enablers in a variety of medical image analysis applications that entail aligning images to a common coordinate systems. Population analysis of such transformations is expected to capture the underlying image and shape variations, and hence these transformations are required to produce anatomically feasible correspondences. This is usually enforced through some smoothness-based generic metric or regularization of the deformation field. Alternatively, population-based regularization has been shown to produce anatomically accurate correspondences in cases where anatomically unaware (i.e., data independent) regularization fail. Recently, deep networks have been used to generate spatial transformations in an unsupervised manner, and, once trained, these networks are computationally faster and as accurate as conventional, optimization-based registration methods. However, the deformation fields produced by these networks require smoothness penalties, just as the conventional registration methods, and ignores population-level statistics of the transformations. Here, we propose a novel neural network architecture that simultaneously learns and uses the population-level statistics of the spatial transformations to regularize the neural networks for unsupervised image registration. This regularization is in the form of a bottleneck autoencoder, which learns and adapts to the population of transformations required to align input images by encoding the transformations to a low dimensional manifold. The proposed architecture produces deformation fields that describe the population-level features and associated correspondences in an anatomically relevant manner and are statistically compact relative to the state-of-the-art approaches while maintaining computational efficiency. We demonstrate the efficacy of the proposed architecture on synthetic data sets, as well as 2D and 3D medical data.

Which Two-dimensional Radiographic Measurements of Cam Femoroacetabular Impingement Best Describe the Three-dimensional Shape of the Proximal Femur?
P. R. Atkins, Y. Shin, P. Agrawal, S. Y. Elhabian, R. T. Whitaker, J. A. Weiss, S. K. Aoki, C. L. Peters, A. E. Anderson. In Clinical Orthopaedics and Related Research, Vol. 477, No. 1, 2019.


Many two-dimensional (2-D) radiographic views are used to help diagnose cam femoroacetabular impingement (FAI), but there is little consensus as to which view or combination of views is most effective at visualizing the magnitude and extent of the cam lesion (ie, severity). Previous studies have used a single image from a sequence of CT or MR images to serve as a reference standard with which to evaluate the ability of 2-D radiographic views and associated measurements to describe the severity of the cam lesion. However, single images from CT or MRI data may fail to capture the apex of the cam lesion. Thus, it may be more appropriate to use measurements of three-dimensional (3-D) surface reconstructions from CT or MRI data to serve as an anatomic reference standard when evaluating radiographic views and associated measurements used in the diagnosis of cam FAI.


The purpose of this study was to use digitally reconstructed radiographs and 3-D statistical shape modeling to (1) determine the correlation between 2-D radiographic measurements of cam FAI and 3-D metrics of proximal femoral shape; and 2) identify the combination of radiographic measurements from plain film projections that were most effective at predicting the 3-D shape of the proximal femur.


This study leveraged previously acquired CT images of the femur from a convenience sample of 37 patients (34 males; mean age, 27 years, range, 16-47 years; mean body mass index [BMI], 24.6 kg/m, range, 19.0-30.2 kg/m) diagnosed with cam FAI imaged between February 2005 and January 2016. Patients were diagnosed with cam FAI based on a culmination of clinical examinations, history of hip pain, and imaging findings. The control group consisted of 59 morphologically normal control participants (36 males; mean age, 29 years, range, 15-55 years; mean BMI, 24.4 kg/m, range, 16.3-38.6 kg/m) imaged between April 2008 and September 2014. Of these controls, 30 were cadaveric femurs and 29 were living participants. All controls were screened for evidence of femoral deformities using radiographs. In addition, living control participants had no history of hip pain or previous surgery to the hip or lower limbs. CT images were acquired for each participant and the surface of the proximal femur was segmented and reconstructed. Surfaces were input to our statistical shape modeling pipeline, which objectively calculated 3-D shape scores that described the overall shape of the entire proximal femur and of the region of the femur where the cam lesion is typically located. Digital reconstructions for eight plain film views (AP, Meyer lateral, 45° Dunn, modified 45° Dunn, frog-leg lateral, Espié frog-leg, 90° Dunn, and cross-table lateral) were generated from CT data. For each view, measurements of the α angle and head-neck offset were obtained by two researchers (intraobserver correlation coefficients of 0.80-0.94 for the α angle and 0.42-0.80 for the head-neck offset measurements). The relationships between radiographic measurements from each view and the 3-D shape scores (for the entire proximal femur and for the region specific to the cam lesion) were assessed with linear correlation. Additionally, partial least squares regression was used to determine which combination of views and measurements was the most effective at predicting 3-D shape scores.


Three-dimensional shape scores were most strongly correlated with α angle on the cross-table view when considering the entire proximal femur (r = -0.568; p < 0.001) and on the Meyer lateral view when considering the region of the cam lesion (r = -0.669; p < 0.001). Partial least squares regression demonstrated that measurements from the Meyer lateral and 90° Dunn radiographs produced the optimized regression model for predicting shape scores for the proximal femur (R = 0.405, root mean squared error of prediction [RMSEP] = 1.549) and the region of the cam lesion (R = 0.525, RMSEP = 1.150). Interestingly, views with larger differences in the α angle and head-neck offset between control and cam FAI groups did not have the strongest correlations with 3-D shape.


Considered together, radiographic measurements from the Meyer lateral and 90° Dunn views provided the most effective predictions of 3-D shape of the proximal femur and the region of the cam lesion as determined using shape modeling metrics.


Our results suggest that clinicians should consider using the Meyer lateral and 90° Dunn views to evaluate patients in whom cam FAI is suspected. However, the α angle and head-neck offset measurements from these and other plain film views could describe no more than half of the overall variation in the shape of the proximal femur and cam lesion. Thus, caution should be exercised when evaluating femoral head anatomy using the α angle and head-neck offset measurements from plain film radiographs. Given these findings, we believe there is merit in pursuing research that aims to develop the framework necessary to integrate statistical shape modeling into clinical evaluation, because this could aid in the diagnosis of cam FAI.

Identifying surface morphological characteristics to differentiate between mixtures of U3O8 synthesized from ammonium diuranate and uranyl peroxide,
S. T. Heffernan, N. Ly, B. J. Mower, C. Vachet, I. J. Schwerdt, T. Tasdizen, L. W. McDonald IV. In Radiochimica Acta, 2019.

In the present study, surface morphological differences of mixtures of triuranium octoxide (U3O8), synthesized from uranyl peroxide (UO4) and ammonium diuranate (ADU), were investigated. The purity of each sample was verified using powder X-ray diffractometry (p-XRD), and scanning electron microscopy (SEM) images were collected to identify unique morphological features. The U3O8 from ADU and UO4 was found to be unique. Qualitatively, both particles have similar features being primarily circular in shape. Using the morphological analysis of materials (MAMA) software, particle shape and size were quantified. UO4 was found to produce U3O8 particles three times the area of those produced from ADU. With the starting morphologies quantified, U3O8 samples from ADU and UO4 were physically mixed in known quantities. SEM images were collected of the mixed samples, and the MAMA software was used to quantify particle attributes. As U3O8 particles from ADU were unique from UO4, the composition of the mixtures could be quantified using SEM imaging coupled with particle analysis. This provides a novel means of quantifying processing histories of mixtures of uranium oxides. Machine learning was also used to help further quantify characteristics in the image database through direct classification and particle segmentation using deep learning techniques based on Convolutional Neural Networks (CNN). It demonstrates that these techniques can distinguish the mixtures with high accuracy as well as showing significant differences in morphology between the mixtures. Results from this study demonstrate the power of quantitative morphological analysis for determining the processing history of nuclear materials.

Quantifying Impurity Effects on the Surface Morphology of α-U3O8,
A. B. Hanson, R. N. Lee, C. Vachet, I. J. Schwerdt, T. Tasdizen, L. W. McDonald IV. In Analytical Chemistry, 2019.
DOI: doi:10.1021/acs.analchem.9b02013

The morphological effect of impurities on α-U3O8 has been investigated. This study provides the first evidence that the presence of impurities can alter nuclear material morphology, and these changes can be quantified to aid in revealing processing history. Four elements: Ca, Mg, V, and Zr were implemented in the uranyl peroxide synthesis route and studied individually within the α-U3O8. Six total replicates were synthesized, and replicates 1–3 were filtered and washed with Millipore water (18.2 MΩ) to remove any residual nitrates. Replicates 4–6 were filtered but not washed to determine the amount of impurities removed during washing. Inductively coupled plasma mass spectrometry (ICP-MS) was employed at key points during the synthesis to quantify incorporation of the impurity. Each sample was characterized using powder X-ray diffraction (p-XRD), high-resolution scanning electron microscopy (HRSEM), and SEM with energy dispersive X-ray spectroscopy (SEM-EDS). p-XRD was utilized to evaluate any crystallographic changes due to the impurities; HRSEM imagery was analyzed with Morphological Analysis for MAterials (MAMA) software and machine learning classification for quantification of the morphology; and SEM-EDS was utilized to locate the impurity within the α-U3O8. All samples were found to be quantifiably distinguishable, further demonstrating the utility of quantitative morphology as a signature for the processing history of nuclear material.

Image-based analysis and long-term clinical outcomes of deep brain stimulation for Tourette syndrome: a multisite study
K. A. Johnson, P. T. Fletcher, D. Servello, A. Bona, M. Porta, J. L. Ostrem, E. Bardinet, M. Welter, A. M. Lozano, J. C. Baldermann, J. Kuhn, D. Huys, T. Foltynie, M. Hariz, E. M. Joyce, L. Zrinzo, Z. Kefalopoulou, J. Zhang, F. Meng, C. Zhang, Z. Ling, X. Xu, X. Yu, A. YJM Smeets, L. Ackermans, V. Visser-Vandewalle, A. Y. Mogilner, M. H. Pourfar, L. Almeida, A. Gunduz, W. Hu, K. D. Foote, M. S. Okun, C. R. Butson. In Journal of Neurology, Neurosurgery & Psychiatry, BMJ Publishing Group, 2019.
DOI: 10.1136/jnnp-2019-320379

Deep brain stimulation (DBS) can be an effective therapy for tics and comorbidities in select cases of severe, treatment-refractory Tourette syndrome (TS). Clinical responses remain variable across patients, which may be attributed to differences in the location of the neuroanatomical regions being stimulated. We evaluated active contact locations and regions of stimulation across a large cohort of patients with TS in an effort to guide future targeting.

We collected retrospective clinical data and imaging from 13 international sites on 123 patients. We assessed the effects of DBS over time in 110 patients who were implanted in the centromedial (CM) thalamus (n=51), globus pallidus internus (GPi) (n=47), nucleus accumbens/anterior limb of the internal capsule (n=4) or a combination of targets (n=8). Contact locations (n=70 patients) and volumes of tissue activated (n=63 patients) were coregistered to create probabilistic stimulation atlases.
Tics and obsessive-compulsive behaviour (OCB) significantly improved over time (p<0.01), and there were no significant differences across brain targets (p>0.05). The median time was 13 months to reach a 40% improvement in tics, and there were no significant differences across targets (p=0.84), presence of OCB (p=0.09) or age at implantation (p=0.08). Active contacts were generally clustered near the target nuclei, with some variability that may reflect differences in targeting protocols, lead models and contact configurations. There were regions within and surrounding GPi and CM thalamus that improved tics for some patients but were ineffective for others. Regions within, superior or medial to GPi were associated with a greater improvement in OCB than regions inferior to GPi.
The results collectively indicate that DBS may improve tics and OCB, the effects may develop over several months, and stimulation locations relative to structural anatomy alone may not predict response. This study was the first to visualise and evaluate the regions of stimulation across a large cohort of patients with TS to generate new hypotheses about potential targets for improving tics and comorbidities.

A High-Resolution Head and Brain Computer Model for Forward and Inverse EEG Simulation
A. Warner, J. Tate, B. Burton,, C.R. Johnson. In bioRxiv, Cold Spring Harbor Laboratory, Feb, 2019.
DOI: 10.1101/552190

To conduct computational forward and inverse EEG studies of brain electrical activity, researchers must construct realistic head and brain computer models, which is both challenging and time consuming. The availability of realistic head models and corresponding imaging data is limited in terms of imaging modalities and patient diversity. In this paper, we describe a detailed head modeling pipeline and provide a high-resolution, multimodal, open-source, female head and brain model. The modeling pipeline specifically outlines image acquisition, preprocessing, registration, and segmentation; three-dimensional tetrahedral mesh generation; finite element EEG simulations; and visualization of the model and simulation results. The dataset includes both functional and structural images and EEG recordings from two high-resolution electrode configurations. The intermediate results and software components are also included in the dataset to facilitate modifications to the pipeline. This project will contribute to neuroscience research by providing a high-quality dataset that can be used for a variety of applications and a computational pipeline that may help researchers construct new head models more efficiently.

Clustering With Pairwise Relationships: A Generative Approach
Y.Y. Yu, S.Y. Elhabian, R.T. Whitaker. In CoRR, 2018.

Semi-supervised learning (SSL) has become important in current data analysis applications, where the amount of unlabeled data is growing exponentially and user input remains limited by logistics and expense. Constrained clustering, as a subclass of SSL, makes use of user input in the form of relationships between data points (e.g., pairs of data points belonging to the same class or different classes) and can remarkably improve the performance of unsupervised clustering in order to reflect user-defined knowledge of the relationships between particular data points. Existing algorithms incorporate such user input, heuristically, as either hard constraints or soft penalties, which are separate from any generative or statistical aspect of the clustering model; this results in formulations that are suboptimal and not sufficiently general. In this paper, we propose a principled, generative approach to probabilistically model, without ad hoc penalties, the joint distribution given by user-defined pairwise relations. The proposed model accounts for general underlying distributions without assuming a specific form and relies on expectation-maximization for model fitting. For distributions in a standard form, the proposed approach results in a closed-form solution for updated parameters.

Latent Space Non-Linear Statistics
L. Kuhnel, T. Fletcher, S. Joshi, S. Sommer. In CoRR, 2018.

Given data, deep generative models, such as variational autoencoders (VAE) and generative adversarial networks (GAN), train a lower dimensional latent representation of the data space. The linear Euclidean geometry of data space pulls back to a nonlinear Riemannian geometry on the latent space. The latent space thus provides a low-dimensional nonlinear representation of data and classical linear statistical techniques are no longer applicable. In this paper we show how statistics of data in their latent space representation can be performed using techniques from the field of nonlinear manifold statistics. Nonlinear manifold statistics provide generalizations of Euclidean statistical notions including means, principal component analysis, and maximum likelihood fits of parametric probability distributions. We develop new techniques for maximum likelihood inference in latent space, and adress the computational complexity of using geometric algorithms with high-dimensional data by training a separate neural network to approximate the Riemannian metric and cometric tensor capturing the shape of the learned data manifold.

Skeletal Shape Correspondence through Entropy
L. Tu, M. Styner, J. Vicory, S. Elhabian, R. Wang, J. Hong, B. Paniagua, J.C. Prieto, D. Yang, R. Whitaker, M. Pizer. In IEEE Transactions on Medical Imaging, Vol. 37, No. 1, IEEE, pp. 1--11. Jan, 2018.
DOI: 10.1109/tmi.2017.2755550

We present a novel approach for improving the shape statistics of medical image objects by generating correspondence of skeletal points. Each object's interior is modeled by an s-rep, i.e., by a sampled, folded, two-sided skeletal sheet with spoke vectors proceeding from the skeletal sheet to the boundary. The skeleton is divided into three parts: the up side, the down side, and the fold curve. The spokes on each part are treated separately and, using spoke interpolation, are shifted along that skeleton in each training sample so as to tighten the probability distribution on those spokes' geometric properties while sampling the object interior regularly. As with the surface/boundary-based correspondence method of Cates et al., entropy is used to measure both the probability distribution tightness and the sampling regularity, here of the spokes' geometric properties. Evaluation on synthetic and real world lateral ventricle and hippocampus data sets demonstrate improvement in the performance of statistics using the resulting probability distributions. This improvement is greater than that achieved by an entropy-based correspondence method on the boundary points.

F. Mesadi, E. Erdil, M. Cetin, T. Tasdizen
Image segmentation using disjunctive normal Bayesian shape, appearance models. In IEEE Transactions on Medical Imaging, Vol. 37, No. 1, IEEE, pp. 293--305. Jan, 2018.
DOI: 10.1109/tmi.2017.2756929

The use of appearance and shape priors in image segmentation is known to improve accuracy; however, existing techniques have several drawbacks. For instance, most active shape and appearance models require landmark points and assume unimodal shape and appearance distributions, and the level set representation does not support construction of local priors. In this paper, we present novel appearance and shape models for image segmentation based on a differentiable implicit parametric shape representation called a disjunctive normal shape model (DNSM). The DNSM is formed by the disjunction of polytopes, which themselves are formed by the conjunctions of half-spaces. The DNSM's parametric nature allows the use of powerful local prior statistics, and its implicit nature removes the need to use landmarks and easily handles topological changes. In a Bayesian inference framework, we model arbitrary shape and appearance distributions using nonparametric density estimations, at any local scale. The proposed local shape prior results in accurate segmentation even when very few training shapes are available, because the method generates a rich set of shape variations by locally combining training samples. We demonstrate the performance of the framework by applying it to both 2-D and 3-D data sets with emphasis on biomedical image segmentation applications.