MATEMATIQKI VESNIK originalni nauqni rad research paper 65, 4 (2013), 511–518 December 2013 RATE OF CONVERGENCE OF SOME NEURAL NETWORK OPERATORS TO THE UNIT-UNIVARIATE CASE, REVISITED George A. Anastassiou Abstract. This article deals with the determination of the rate of convergence to the unit of some neural network operators, namely, “the normalized bell and squashing type operators”. This is given through the modulus of continuity of the involved function or its derivative and that appears in the right-hand side of the associated Jackson type inequalities. 1. Introduction The Cardaliaguet-Euvrard operators were first introduced and studied extensively in [2], where the authors among many other things proved that these operators converge uniformly on compacta, to the unit over continuous and bounded functions. Our “normalized bell and squashing type operators” (1) and (18) were motivated and inspired by [2]. The work in [2] is qualitative where the used bellshaped function is general. However, our work, though greatly motivated by [2], is quantitative and the used bell-shaped and “squashing” functions are of compact support. We produce a series of inequalities giving close upper bounds to the errors in approximating the unit operator by the above neural network induced operators. All involved constants there are well determined. These are mainly pointwise estimates involving the first modulus of continuity of the engaged continuous function or some of its derivatives. This work is a continuation and simplification of our earlier work in [1]. 2. Convergence with rates of the normalized bell type neural network operators We need the following (see [2]). Definition 1. A function b : R → R is said to be bell-shaped if b belongs to L1 and its integral is nonzero, if it is nondecreasing on (−∞, a) and nonincreasing 2010 AMS Subject Classification: 41A17, 41A25, 41A30, 41A36 Keywords and phrases: Neural network approximation; bell and squashing functions; modulus of continuity. 511 512 G. A. Anastassiou on [a, +∞), where a belongs to R. In particular b (x) is a nonnegative number and at a b takes a global maximum; it is the center of the bell-shaped function. A bell-shaped function is said to be centered if its center is zero. The function b (x) may have jump discontinuities. In this work we consider only centered bell-shaped functions of compact support [−T, T ], T > 0. Example 2. (1) b (x) can be the characteristic function over [−1, 1]. (2) b (x) can be the hat function over [−1, 1], i.e., 1 + x, −1 ≤ x ≤ 0, b (x) = 1 − x, 0 < x ≤ 1 0, elsewhere. Here we consider functions f : R → R that are either continuous and bounded, or uniformly continuous. In this article we study the pointwise convergence with rates over the real line, to the unit operator, of the “normalized bell type neural network operators”, ¢¢ ¡ k ¢ ¡ 1−α ¡ Pn2 x − nk k=−n2 f n b n (1) (Hn (f )) (x) := ¡ ¡ ¢¢ , Pn2 1−α x − k k=−n2 b n n where 0 < α < 1 and x ∈ R, n ∈ N. The terms in the ratio of sums (1) can be nonzero iff ¯ ¯ ¯ µ ¶¯ ¯ 1−α ¯ ¯ k ¯¯ ¯n ¯x − k ¯ ≤ T x − ≤ T , i.e. ¯ ¯ ¯ n n ¯ n1−α iff nx − T nα ≤ k ≤ nx + T nα . (2) In order to have the desired order of numbers −n2 ≤ nx − T nα ≤ nx + T nα ≤ n2 , (3) it is sufficient enough to assume that n ≥ T + |x| . (4) When x ∈ [−T, T ] it is enough to assume n ≥ 2T which implies (3). Proposition 3. (see [1]) Let a ≤ b, a, b ∈ R. Let card (k) (≥ 0) be the maximum number of integers contained in [a, b]. Then max (0, (b − a) − 1) ≤ card (k) ≤ (b − a) + 1. Note 4. We would like to establish a lower bound on card (k) over the interval [nx − T nα , nx + T nα ]. From Proposition 3 we get that card (k) ≥ max (2T nα − 1, 0) . 513 Some neural network operators We obtain card (k) ≥ 1, if 1 2T nα − 1 ≥ 1 iff n ≥ T − α . So to have the desired order (3) and card (k) ≥ 1 over [nx − T nα , nx + T nα ], we need to consider ³ ´ 1 n ≥ max T + |x| , T − α . (5) Also notice that card (k) → +∞, as n → +∞. Denote by [·] the integral part of a number and by d·e its ceiling. Here comes our first main result. ³ ´ 1 Theorem 5. Let x ∈ R, T > 0 and n ∈ N such that n ≥ max T + |x| , T − α . Then ¶ µ T |(Hn (f )) (x) − f (x)| ≤ ω1 f, 1−α , (6) n where ω1 is the first modulus of continuity of f . Proof. Call [nx+T nα ] X V (x) := k=dnx−T nα e µ ¶¶ µ k b n1−α x − . n Clearly we obtain ∆ := |(Hn (f )) (x) − f (x)| ¯P ¯ ¡ ¢¢ ¡ ¢ ¡ ¯ [nx+T nα ] ¯ ¯ k=dnx−T nα e f nk b n1−α x − nk ¯ ¯ =¯ − f (x)¯¯ . V (x) ¯ ¯ The last comes by the compact support [−T, T ] of b and (2). Hence it holds ¯ ¯ ¢ µ ¯ [nx+T nα ] ¡ ¡ k ¢ µ ¶¶¯ X ¯ ¯ − f (x) f k n ¯ b n1−α x − ∆ = ¯¯ ¯ V (x) n ¯ ¯k=dnx−T nα e α ¯ ¯ ¡ ¢ µ µ ¶¶ [nx+T n ] X ω1 f, ¯ nk − x¯ k b n1−α x − . ≤ V (x) n α k=dnx−T n e Thus |(Hn (f )) (x) − f (x)| µ ≤ ω1 f, proving the claim. T n1−α ³ ¶ P[nx+T nα ] k=dnx−T nα e ¡ ¡ ¢¢´ b n1−α x − nk V (x) µ = ω1 f, T n1−α ¶ , 514 G. A. Anastassiou Our second main result follows. ³ ´ 1 Theorem 6. Let x ∈ R, T > 0 and n ∈ N such that n ≥ max T + |x| , T − α . Let f ∈ C N (R), N ∈ N, such that f (N ) is a uniformly continuous function or f (N ) is continuous and bounded. Then |(Hn (f )) (x) − f (x)| ≤ ¯ ¶ µX N ¯¯ (j) f (x)¯ T j nj(1−α) j! j=1 µ + ω1 f (N ) , T ¶ n1−α · TN . (7) N ! nN (1−α) Notice that as n → ∞ we have that R.H.S.(7)→ 0, therefore L.H.S.(7)→ 0, i.e., (7) gives us with rates the pointwise convergence of (Hn (f )) (x) → f (x), as n → +∞, x ∈ R. Proof. Note that b here is of compact support [−T, T ] and all assumptions are as earlier. By Taylor’s formula we have that µ ¶ NX µ ¶j Z k ³ −1 (j) ´ ¡ k − t¢N −1 n k f (x) k (N ) (N ) −x + dt. f = f (t) − f (x) n n j! n (N − 1)! x j=0 Call [nx+T nα ] X V (x) := k=dnx−T nα e µ µ ¶¶ k 1−α b n x− . n Hence ¡ ¢¢ ¢¢ ¡ ¢ ¡ µ ¶j ¡ 1−α ¡ N X f nk b n1−α x − nk b n x − nk f (j) (x) k = −x V (x) j! n V (x) j=0 ¡ ¡ ¢¢ Z k ³ ´ ¡ k − t¢N −1 n b n1−α x − nk (N ) (N ) + f (t) − f (x) n dt. V (x) (N − 1)! x Thus [nx+T nα ] X (Hn (f )) (x) − f (x) = k=dnx−T nα e = N X f (j) (x) j=1 j! f ¡ k ¢ ¡ 1−α ¡ ¢¢ x − nk n b n − f (x) V (x) [nx+T nα ] X k=dnx−T nα e µ ¢¢ ¶j ¡ 1−α ¡ k b n x− n k + R, −x n V (x) where [nx+T nα ] R := X k=dnx−T nα e ¡ ¡ ¢¢ Z k ³ ´ ¡ k − t¢N −1 n b n1−α x − nk (N ) (N ) f (t) − f (x) n dt. (8) V (x) (N − 1)! x 515 Some neural network operators So that |(Hn (f )) (x) − f (x)| ¯ N ¯¯ (j) X f (x)¯ ≤ j! j=1 [nx+T nα ] X k=dnx−T nα e And hence |(Hn (f )) (x) − f (x)| ≤ ¡ 1−α ¡ ¢¢ k b n x − T n + |R| . V (x) nj(1−α) j ¯¶ ¯ µX N T j ¯f (j) (x)¯ j=1 Next we estimate ¯ α ¡ ¡ ¢¢ ¯ [nx+T Xn ] b n1−α x − k Z nk ¯ n ¯ |R| = ¯ V (x) x ¯k=dnx−T nα e ¯ α ¡ ¡ ¢¢ Z k [nx+T n ] X b n1−α x − nk ¯¯ n ≤ ¯ ¯ x V (x) α k=dnx−T n e [nx+T nα ] ≤ X k=dnx−T nα e where + |R| . (9) ¯ ´ ¡ k − t¢N −1 ¯¯ dt¯¯ f (N ) (t) − f (N ) (x) n (N − 1)! ¯ ¯ ³ ´ ¡ k − t¢N −1 ¯ ¯ (N ) (N ) f (t) − f (x) n dt¯ ¯ (N − 1)! ³ ¡ ¡ ¢¢ b n1−α x − nk · γ ≤ (∗) , V (x) ¯Z k ¯ ¯ n | nk − t|N −1 ¯¯ ¯ (N ) (N ) γ := ¯ |f (t) − f (x) | dt¯ , ¯ x (N − 1)! ¯ and [nx+T nα ] (∗) ≤ X k=dnx−T nα e where nj(1−α) j! ¡ ¡ ¢¢ b n1−α x − nk · ϕ = ϕ, V (x) µ ϕ := ω1 f (N ) , (11) ¶ TN . n1−α n! nN (1−α) The last part of inequality (11) comes from the following: (i) Let x ≤ nk , then Z nk ¯ ¯ ¯¯ k − t¯¯N −1 ¯ (N ) ¯ (N ) γ= (t) − f (x)¯ n dt ¯f (N − 1)! x Z nk ³ ´ ¯¯ k − t¯¯N −1 (N ) ≤ ω1 f , |t − x| n dt (N − 1)! x ¢N −1 ¯ ¯¶ Z k ¡ k µ n ¯ k ¯¯ (N ) ¯ n −t ≤ ω1 f , ¯x − ¯ dt n (N − 1)! x ¢N µ ¶ ¡k µ ¶ T T TN (N ) n −x ≤ ω1 f , 1−α ≤ ω1 f (N ) , 1−α ; n N! n N ! nN (1−α) T (10) (12) 516 G. A. Anastassiou i.e., when x ≤ k n we get µ γ ≤ ω1 f (N ) , T n1−α ¶ TN . N ! nN (1−α) (ii) Let x ≥ nk , then ¯Z ¯ ¯ x¯ ¯ ¯¯t − k ¯¯N −1 ¯ ¯ ¯ (N ) ¯ ¯ (N ) n γ=¯ dt¯ (t) − f (x)¯ ¯f ¯ k ¯ (N − 1)! n ¡ ¢ N −1 Z x¯ ¯ t− k ¯ ¯ (N ) n (t) − f (N ) (x)¯ = dt ¯f k (N − 1)! n Z x ³ ´ ¡t − k ¢N −1 (N ) n ≤ ω1 f , |t − x| dt k (N − 1)! n ¢N −1 ¯ ¯¶ Z x ¡ µ ¯ t − nk k ¯¯ (N ) ¯ ≤ ω1 f , ¯x − ¯ dt k n (N − 1)! n ¢N ¯ ¯¶ ¡ µ µ ¶ ¯ k ¯¯ x − nk T TN (N ) ¯ = ω1 f , ¯x − ¯ ≤ ω1 f (N ) , 1−α . n N! n N ! nN (1−α) Thus in both cases we have µ ¶ T TN (N ) γ ≤ ω1 f , 1−α . n N ! nN (1−α) Consequently from (11), (12) and (14) we obtain µ ¶ T TN (N ) |R| ≤ ω1 f , 1−α . n N ! nN (1−α) Finally from (15) and (9) we conclude inequality (7). (13) (14) (15) Corollary 7. Let b (x) be a centered bell-shaped continuous function on R ∗ ∗ ∗ of compact ³ support [−T, ´ T ]. Let x ∈ [−T , T ], T > 0, and n ∈ N be such that 1 ∗ −α n ≥ max T + T , T , 0 < α < 1. Consider p ≥ 1. Then µ ¶ 1 1 T kHn (f ) − f kp,[−T ∗ ,T ∗ ] ≤ ω1 f, 1−α · 2 p · T ∗ p . (16) n From (16) we get the Lp convergence of Hn (f ) to f with rates. Corollary 8. Let b (x) be a centered bell-shaped continuous function on R ∗ ∗ ∗ of compact ³ support [−T, ´ T ]. Let x ∈ [−T , T ], T > 0, and n ∈ N be such that 1 n ≥ max T + T ∗ , T − α , 0 < α < 1. Consider p ≥ 1. Then kHn (f ) − f kp,[−T ∗ ,T ∗ ] ° ° ¶ 1 N ∗1 ¶ µ µX N T j · °f (j) ° 2p T T p T p,[−T ∗ ,T ∗ ] (N ) + ω f , , ≤ 1 1−α j(1−α) n n j! N ! nN (1−α) j=1 where N ≥ 1. (17) 517 Some neural network operators Here from (17) we get again the Lp convergence of Hn (f ) to f with rates. Proof. Inequality (17) now comes by integration of (7) and the properties of the Lp -norm. 2.1. The “normalized squashing type operators” and their convergence to the unit with rates We need Definition 9. Let the nonnegative function S : R → R, S has compact support [−T, T ], T > 0, and is nondecreasing there and it can be continuous only on either (−∞, T ] or [−T, T ]. S can have jump discontinuities. We call S the “squashing function” (see also [2]). Let f : R → R be either uniformly continuous or continuous and bounded. For x ∈ R we define the “normalized squashing type operator” ¡k¢ ¡ 1−α ¡ ¢¢ Pn2 · x − nk 2 f n ·S n (Kn (f )) (x) := k=−n , ¡ ¡ ¢¢ Pn2 1−α · x − k k=−n2 S n n ³ ´ 1 0 < α < 1 and n ∈ N : n ≥ max T + |x| , T − α . It is clear that P[nx+T nα ] k=dnx−T nα e (Kn (f )) (x) = f ¡k¢ n ¡ ¡ ¢¢ · S n1−α · x − nk W (x) where [nx+T nα ] X W (x) := k=dnx−T nα e , µ µ ¶¶ k S n1−α · x − . n Here we give the pointwise convergence with rates of (Kn f ) (x) → f (x), as n → +∞, x ∈ R. Theorem 10. Under the above terms and assumptions we obtain ¶ µ T |Kn (f ) (x) − f (x)| ≤ ω1 f, 1−α . n Proof. As in Theorem 5. ³ ´ 1 Theorem 11. Let x ∈ R, T > 0 and n ∈ N such that n ≥ max T + |x| , T − α . Let f ∈ C N (R), N ∈ N, such that f (N ) is a uniformly continuous function or f (N ) is continuous and bounded. Then ¯ ¶ µX µ ¶ N ¯¯ (j) f (x)¯ T j T TN (N ) |(Kn (f )) (x) − f (x)| ≤ + ω f , . · 1 1−α j(1−α) n j! n N ! nN (1−α) j=1 So we obtain the pointwise convergence of Kn (f ) to f with rates. 518 G. A. Anastassiou Proof. As similar to Theorem 6 is omitted. Note 12. The maps Hn , Kn are positive linear operators reproducing constants, in particular Hn (1) = Kn (1) = 1. REFERENCES [1] G.A. Anastassiou, Rate of convergence of some neural network operators to the unitunivariate case, J. Math. Anal. Appl. 212 (1997), 237–262. [w] P. Cardaliaguet, G. Euvrard, Approximation of a function and its derivative with a neural network, Neural Networks 5 (1992), 207–220. (received 23.12.2011; available online 10.09.2012) Department of Mathematical Sciences, University of Memphis, Memphis, TN 38152, U.S.A. E-mail: ganastss@memphis.edu