advertisement

Error Correcting Codes: Combinatorics, Algorithms and Applications (Fall 2007) Lecture 15: Gilbert-Varshamov Bound October 2, 2007 Lecturer: Atri Rudra Scribe: Thanh-Nhan Nguyen In the previous lectures, we have only seen upper bounds on the rate of a code (given a fixed relative distance). In this lecture, we study our first (and only) lower bound on rate of a code. 1 Gilbert-Varshamov Bound In today’s lecture we will prove the following result Theorem 1.1. Let q ≥ 2. For every 0 ≤ δ < 1 − 1q , and 0 < ε ≤ 1 − Hq (δ), there exists a code with rate R ≥ 1 − Hq (δ) − ε, and relative distance δ. The above result was proved for general codes by Gilbert (Section 1.1) and for linear codes by Varshamov (Section 1.2). Hence, the bound is called the Gilbert-Varshamov bound. 1.1 Gilbert Construction Gilbert proved Theorem 1.1 by the following greedy construction (where d = δn) (i) Start with the empty code: C ← ∅ (ii) If there exists a v ∈ [q]n such that ∆(v, c) ≥ d for every c ∈ C, add v to C. We claim that the above algorithm terminates and C has distance d. The latter is true by step (ii). For the former claim, note that, if we cannot add v at some point, we cannot add it later. The running time of this algorithm is q O(n) (as step (ii) in the worst-case could be repeated for every vector in [q]n , that is at most q n times). Further, we claim that after termination [ B(c, d − 1) ⊇ [q]n , c∈C because if not, then we can pick another another codeword. Therefore, [ | B(c, d − 1)| ≥ q n . c∈C It is obvious that X |B(c, d − 1)| ≥ | c∈C [ c∈C 1 B(c, d − 1)| (1) which by (1) implies that X V olq (c, d − 1) ≥ q n c∈C or X V olq (0, d − 1) ≥ q n . c∈C Thus, we have qn V olq (0, d − 1) qn ≥ nHq (δ) q = q n(1−Hq (δ)) , |C| ≥ (2) as desired. In the above, (2) follows from the fact that V olq (0, d − 1) ≤ V olq (0, δn) ≤ q nHq (δ) , (3) where the second inequality follows from the upper bound on the volume of a Hamming ball that we proved in an earlier lecture. It is worth noting that the code from Gilbert’s construction is not guaranteed to have any special structure. In particular, even storing the code can take exponential space. We have seen that linear codes have a much more succinct representation. Thus, a natural question is if one can show that linear codes achieve the the r ≥ 1 − Hq (δ) tradeoff that the Gilbert construction achieves. Varshamov showed that indeed this is the case and we look at his construction next. 1.2 Varshamov Construction Now we turn to the result due to Varshamov who showed that a random linear code, with high probability, lies on the GV bound. The Varshamov construction is yet another use of the probabilistic method. For a recap of the general idea behind the probabilistic method and other probability basics (such as the union bound), please refer to Lecture 4 from Spring 09. Pick a random linear code by picking a random k × n matrix G where each of kn entries is chosen uniformly and independently at random from Fq . We are done if we can show that For every m ∈ Fkq \ {0}, wt(mG) ≥ d. Fix m ∈ Fkq \ {0}. Note that for a random G, mG is a uniformly random vector from Fnq . Thus, we have V olq (0, d − 1) P r[wt(mG) < d] = qn q nHq (δ) ≤ (4) qn 2 where (4) follows from (3). Thus, by the union bound P r[∃m, wt(mG) < d] ≤ q k q −n(1−Hq (δ)) = q −ε·n , where the equality follows by choosing k = (1−Hq (δ)−ε)n. Since q −εn 1, by the probabilistic method, there exists a linear code C with relative distance δ and rate at least 1 − Hq (δ) − ε. In fact, the proof shows that most linear codes (with appropriate parameters) meet the Gilbert-Varshamov bound. 3