Jingw en Yan , Hui Zhang, Lei Du, Eric Wernert, Andew J. Saykin, Li
Shen

• Imaging Genetics
• Sparse Canonical Correlation Analysis (SCCA)
• Computational Challenges and Methods
• Data Simulation
• Experimental Results

Genes Cells Systems
UCI, S. Potkin et al.
Behavior:
Disorders,
Complex interactions, phenomena, diseases.

Underlying Biological
Pathway and Mechanism

Biological Pathway
Candidate Gene/SNP
Sloan et al 2010
Risacher et al 2010
Genome-wide
Potkin et al 2009;
Saykin et al 2010
Risacher et al 2013
AV45 ROIs & APOE
Swaminathan et al 2012 PiB
ROIs & amyloid pathway
Potkin et al 2009 Mol Psych schizophrenia study
Ho et al 2010 FTO;
Reiman et al PNAS 2009
Chiang et al 2012 SNP/Gene networks & WM integrity
Shen et al 2010 ROIs;
Stein et al 2010 voxels

• Imaging Genetics
• Sparse Canonical Correlation Analysis (SCCA)
• Computational Challenges and Methods
• Data Simulation
• Experimental Results

X1
X2
X3
Y1
Y2
Y3
X1
X2
X3
W’X
Y1
Y2
Y3
X1
X2
X3
𝑹
Xu Yv
Y1
Y2
Y3
Xn Yn
Massive
Univariate
Analysis
Xn
Multivariate
Multiple
Regression
Yn Xn
Canonical
Correlation Analysis
Yn

• Sparse canonical correlation analysis (SCCA)
• R package: Penalized Multivariate Analysis (PMA) ( Witten, et al, 2009 ) max 𝒖,𝒗 𝒖 𝑻 𝑿 𝑻 𝒀𝒗 subject to 𝒖
𝑻
𝑿
𝑻
𝑿𝒖 = 1, 𝒗
𝑻
𝒀
𝑻
𝒀𝒗 = 1
𝑃
1 𝒖 ≤ 𝑐
1
, 𝑃
2 𝒗 ≤ 𝑐
2
• X, Y : imaging and genetics data respectively
• 𝑃
1 𝒖 , 𝑃
2 𝒗 : sparse penalties, mostly 𝐿
1 norm
• For simplicity, assuming 𝑿 𝑻 𝑿 = 𝑰 and 𝒀 𝑻 𝒀 = 𝑰
• Bi-convex and non differentiable problem
• Iterative solution

• Sparse canonical correlation analysis (SCCA)
• Problem max 𝒖,𝒗 𝒖 𝑻 𝑿 𝑻 𝒀𝒗 subject to 𝒖
𝑻 𝒖 = 1, 𝒗
𝑻 𝒗 = 1, 𝒖
1
≤ 𝑐
1
, 𝒗
1
≤ 𝑐
2
• Iterative solution
1. 𝒖 ← arg max 𝒖 𝒖 𝑻 𝑿 𝑻 subject to 𝒖 𝑻
𝒀𝒗, 𝒖 = 1, 𝒖
1
≤ 𝑐
1
2. 𝒗 ← arg max 𝒖 𝒖
𝑻
𝑿
𝑻 subject to 𝒗
𝑻
𝒀𝒗, 𝒗 = 1, 𝒗
1
≤ 𝑐
2
• 𝒖 ←
S(𝑿
S(𝑿 𝑻
𝑻
𝒀𝒗 , ∆)
𝒀𝒗 , ∆)
, 𝑺(𝑿 𝑻 𝒀𝒗 , ∆ ) is the soft thresholding
2 operator and ∆ ≥ 0 is chosen so that u
1
≤ c
1

• Imaging Genetics
• Sparse Canonical Correlation Analysis (SCCA)
• Computational Challenges and Methods
• Data Simulation
• Experimental Results

•
Example SCCA run at a small scale
•
Participants: 1000
•
Genotype: 3,200 SNPs
• Phenotype: 10,000 voxels
• Permutation: 10,000 permutation tests
•
Running time: more than 12,000 hours
•
Scale up
• Genotype (array): 6M SNPs
• Genotype (NGS): 40M variants
•
Phenotype: 200K voxels, imaging, cognitive and biomarker
•
Permutation: 10M permutation to reach p=10 -7
• Parameter tuning via cross-validation
• 10-fold cross-validation coupled with an 11-by-11 grid search
• SCCA runs: 10 × 11 × 11 = 1 , 210

• Intel Math Kernel Library ( MKL )
• accelerate application performance and reduce development time
• highly vectorized and threaded linear algebra, fast fourier transforms (FFT), vector math and statistics functions
• MKL has been optimized to utilize
• multiple processing cores
• wider vector units
• more varied architectures available in a high end system
•
MKL can provide parallelism transparently and speed up programs with supported math routines without changing code.
• Compiling R with MKL

• Xeon Phi SE10P Coprocessor
• 60 cores with 8GB GDDR5
• Intel x86 instruction set
• Usage of familiar programming models, software, and tools
• Pros
• The host system can offload computing workload partially to the
Xeon Phi
• Independently run a compatible program

• Texas Advanced Computing Center
Stampede cluster
• MKL + offload
• Each computing node
• Two Intel Xeon E5-2680 processors each with eight cores @2.7GHz.
• 32GB DDR3 memory
• The Xeon Phi SE10P Coprocessor has 61 cores with 8GB GDDR5
• The NVIDIA K20 GPUs on each node have 5GB of on-board
GDDR5
• Software
• CentOS 6.3.
• Stock R 3.01 package compiled with the Intel compilers (v.13) and built with MKL v.11.

• Imaging Genetics
• Sparse Canonical Correlation Analysis (SCCA)
• Computational Challenges and Methods
• Data Simulation
• Experimental Results

• FREGENE genome simulator
• Simulate sequence-like data over large genomic regions in large diploid populations
• Simulated data
• N=1,000 diploid individuals over 20,000 generations
1 0 Mb genome with the average mutation rate as 2 .
5 e-8
/site/generation
• 3,274 SNPs with minor allele frequency (MAF) greater than 0.05 included
• Four SNP data sets (i.e., g500, g1000, g2000, and g3274) by taking the first 500, 1,000, 2,000, and 3,274 SNPs from the entire data, respectively.

• Assumption
• Each image with multiple regions of interest (ROIs)
• Voxel within each ROI highly correlated
• Simulation
• Random positive definite non-overlapping group structured covariance matrix 𝑴
• Apply Cholesky decomposition to obtain the background imaging data
• Individual: N=1000, Size: 100x100
• We created three sets of phenotypic imaging data (i.e., p1000, p5000, and p10000), consisting of 1,000, 5,000 and 10,000 voxels respectively

• Imaging Genetics
• Sparse Canonical Correlation Analysis (SCCA)
• Computational Challenges and Methods
• Data Simulation
• Experimental Results

• R snowfall package (sfLapply) with MKL and offload model
Baseline
Parallel (MKL+ offload)

Correlation coefficient between the first pair of canonical components
• Accelerated SCCA implementations yielded the same results
• These correlation coefficients are close to the ground truth value of 1

•
Initial steps to accelerate the SCCA implementation for brain imaging genetics applications.
•
Parallelism achieved in system implementation level to accelerate linear algebra computation using math kernel library (MKL) and partial offloading computing workload.
•
The 2-fold speedup, although encouraging, is still insufficient to handle extremely large-scale neuroimaging genetics data
• millions of image voxels and millions of SNPs.
•
Future work
• Big data analytic strategies at the parallel computing model level
• Parallelization of multiplicative algorithms using MapReduce and CUDA.
•
Application to accelerate enhanced SCCA models as well as other bimultivariate statistical models for analyzing brain imaging genetics data.

This research was supported by
• NIH R01 LM011360
• NIH U01 AG024904
• NIH RC2 AG036535
• NIH R01 AG19771
• NIH P30 AG10133
• NSF IIS-1117335