1
Method S1. The detail procedures of the kernel regression
2
The pseudo-code of the kernel regression used in the present study is as follows. In the
3
pseudo-code, a normal (e.g., x), bold symbol (e.g., X), superscript symbol T and the symbol
4
I represents a vector, matrix, transpose operator and unit matrix, respectively.
5
6
1: Input: X, T, N and σ denote independent variables, predictors, the number of samples
7
and the parameter of radial basic function (RBF) kernel, respectively. We decided σ by
8
the leave-one-out cross-validation.
9
10
11
2: We calculate the Gramian matrix 𝛟 using RBF kernel by following equations:
𝛟←{𝜙(𝑥𝑖,𝐗)}, where 𝜙(𝑥i , 𝐗) = exp⁡(−(𝑥i − 𝐗)2 /𝜎).
3: 𝛼, 𝛽 ← random⁡numbers
12
Our aim is to obtain optimal weights coefficient 𝐰 of the Gramian matrix 𝛟. We
13
assumed the hyper-parameters 𝛼, which determines variance of weight coefficients, and
14
𝛽, which determines variance of noise. In order to obtain optimal 𝐰, 𝛼 and 𝛽, we
15
conduct an iterative optimization. Initially, we set random values to 𝛼 and 𝛽, and then
16
conducted the iterative optimization.
17
4: loop
18
5:
By using given 𝛼 and 𝛽, posterior distribution of w is given by the following
19
equation: 𝑝(𝐰, 𝐓) = 𝑁(𝐰|𝐌, 𝐒). We can calculate the mean M and variance S of
20
the distribution by the following equations.
21
𝐒 −𝟏 ← 𝛼𝐈 + 𝛽𝛟T 𝛟,
22
𝐌 ← 𝛽𝐒𝛟T 𝐓
1
23
6:
We update 𝛼 and 𝛽 under given posterior distribution of w by maximizing
24
marginal likelihood function 𝑝(𝐓|𝛼, 𝛽) = ∫ 𝑝(𝐓|𝐰, 𝛽)𝑝(𝐰|𝛼)d𝐰. First, we update
25
𝛼 by the following equations:
26
𝛼 ← 𝛾/𝐌T 𝐌, and
27
𝛾 ← ∑ 𝜆/⁡(𝛼 + 𝜆), where 𝜆 is eigen vector of 𝛽𝛟T 𝛟.
28
29
7:
Next, we update 𝛽 by the following equation.
𝛽 ← ∑(𝐓 − 𝛟𝐌)2 /(𝑁 − 𝛾)
30
11: end loop
31
12: We conduct the above loop 100 times and finally obtained the predictive variables by
32
calculating 𝛟𝐌.
33
34
2