Beating the Union Bound by Geometric Techniques
advertisement
Beating the Union Bound by Geometric Techniques Raghu Meka (IAS & DIMACS) Union Bound Popularized Erdos the “When you haveby eliminated impossible, whatever remains, however improbable, must be the truth” Probabilistic Method 101 • Ramsey graphs – Erdos • Coding theory – Shannon • Metric embeddings – Johnson-Lindenstrauss • … Beating the Union Bound • Not always enough Lovasz Local Lemma: πΈ1 , πΈ2 , … , πΈπ , π dependent. Pr πΈπ < 1/4π, ⇒ Pr ∪ πΈπ < 1. • Constructive: Beck’91, …, Moser’09, … Beating the Union Bound I. Optimal, explicit π-nets for Gaussians • Kanter’s lemma, convex geometry II. Constructive Discrepancy Minimization • EdgeWalk: New LP rounding method Geometric techniques “Truly” constructive Outline I. Optimal, explicit π-nets for Gaussians • Kanter’s lemma, convex geometry II. Constructive Discrepancy Minimization • EdgeWalk: New LP rounding method Epsilon Nets • Discrete approximations • Applications: integration, comp. geometry, … Epsilon Nets for Gaussians Discrete approximations of Gaussian Explicit Even existence not clear! Nets in Gaussian space Thm: Explicit π-net of size (1/π)π • Optimal: Matching lower bound • Union bound: (π/π)π • Dadusch-Vempala’12: ((log π)/π)π π . First: Application to Gaussian Processes and Cover Times 10 Gaussian Processes (GPs) Multivariate Gaussian Distribution Supremum of Gaussian Processes (GPs) Given (ππ ) want to study • Supremum is natural: eg., balls and bins Supremum of Gaussian Processes (GPs) Given π£1 , … , π£π ∈ π π , want to study •• Covariance matrixlog π. Union bound: • More intuitive When is the supremum smaller? Why Gaussian Processes? Stochastic Processes Functional analysis Convex Geometry Machine Learning Many more! Cover times of Graphs Aldous-Fill 94: Compute cover time Fundamental graph parameter deterministically? Eg: πππ£ππ πΎπ = Θ(π log π) • KKLV00: π((log log π)2πππ£ππ ) approximation πΊπππ = Θ(π log 2 π) • Feige-Zeitouni’09: FPTAS for trees Cover Times and GPs Thm (Ding, Lee, Peres 10): O(1) det. poly. time approximation for cover time. Transfer to GPs Compute supremum of GP Computing the Supremum Question (Lee10, Ding11): PTAS for computing the supremum of GPs? • Ding, Lee, Peres 10: π(1) approximation • Can’t beat π(1): Talagrand’s majorizing measures Main Result Thm: PTAS for computing the supremum of Gaussian processes. Thm: PTAS for computing cover time of bounded degree graphs. Heart of PTAS: Epsilon net (Dimension reduction ala JL, use exp. size net) Construction of Net 19 Construction of π-net Simplest possible: univariate to multivariate π 1. How fine a net? Naïve: 1/π. big Union bound! 2. How a net? π Construction of π-net Simplest possible: univariate to multivariate π Lem: Granularity πΏ~π(π) enough. Key point that beats union bound π Construction of π-net This talk: Analyze ‘step-wise’ approximator Even out mass in interval [−πΏ, πΏ]. −4πΏ −3πΏ −2πΏ -πΏ πΎβ πΏ 2πΏ 3πΏ 4πΏ Construction of π-net Take univariate net and lift to multivariate π πΎ π −4πΏ −3πΏ −2πΏ -πΏ πΏ πΎβ 2πΏ 3πΏ Lem: Granularity πΏ~π(π) enough. 4πΏ Dimension Free Error Bounds Thm: For πΏ~ π 1.5 , π a norm, • Proof by “sandwiching” • Exploit convexity critically πΎ −4πΏ −3πΏ −2πΏ -πΏ πΏ πΎβ 2πΏ 3πΏ 4πΏ Analysis of Error Def: Sym. p, q. p βΌ π (less peaked), if ∀ sym. convex sets K, π πΎ ≤π πΎ . • Why interesting? For any norm, Analysis for Univarate Case Fact: πΎβ βΌ πΎ. Spreading away from origin! Proof: −4πΏ −3πΏ −2πΏ -πΏ πΏ 2πΏ 3πΏ 4πΏ Analysis for Univariate Case Def: Fact:πΎπ’πΎ=βΌscaled πΎπ’ . down πΎβ . π ←For πΎβ , πΏπ βͺ = π,1inward − π π,push πΎπ’ =compensates pdf of π. Proof: earlier spreading. πΎπ’ Push mass towards origin. Analysis for Univariate Case Combining upper and lower: πΎβ πΎ πΎπ’ Lifting to Multivariate Case π π πΎβ πΎ π πΎπ’ Kanter’s and unimodal, KeyLemma(77): for univariate: “peakedness” Dimension free! Lifting to Multivariate Case π π πΎβ πΎ Dimension free: key point that beats union bound! π πΎπ’ Summary of Net Construction 1. Granularity π 1.5 enough 2. Cut points outside π-ball Optimal π-net Outline I. Optimal, explicit π-nets for Gaussians • Kanter’s lemma, convex geometry II. Constructive Discrepancy Minimization • EdgeWalk: New LP rounding method Discrepancy • Subsets π1 , π2 , … , ππ ⊆ [π] • Color with 1 or -1 to minimize imbalance 1 2 3 4 5 1 * 1 1 * 3 * 1 1 * 1 1 1 1 1 1 1 1 * * * 1 1 0 1 * 1 * 1 1 Discrepancy Examples • Fundamental combinatorial concept Arithmetic Progressions Roth 64: Ω(π1/4 ) Matousek, Spencer 96: Θ(π1/4 ) Discrepancy Examples • Fundamental combinatorial concept Halfspaces Alexander 90: Matousek 95: Why Discrepancy? Complexity theory Communication Complexity Computational Geometry Pseudorandomness Many more! Spencer’s Six Sigma Theorem “Six standard deviations suffice” Spencer 85: System with n sets has discrepancy at most . • Central result in discrepancy theory. • Tight: Hadamard • Beats union bound: A Conjecture and a Disproof Spencer 85: System with n sets has discrepancy at most . • Non-constructive pigeon-hole proof Conjecture Spencer):get No Bansal 10:(Alon, Can efficiently efficient algorithm can find discrepancy . one. Six Sigma Theorem New elementary geometric proof of Spencer’s result Main: Can efficiently find a coloring with discrepancy π π . • Truly constructive EDGE-WALK: New LP rounding method • Algorithmic partial coloring lemma • Extends to other settings Outline of Algorithm 1. Partial coloring method 2. EDGE-WALK: geometric picture Partial Coloring Method • Beck 80: find partial assignment with < π/2 zeros 1 -1 01 01 -1 0-1 1 * 1 1 * * 1 1 * 1 1 1 1 1 1 * * * 1 1 1 * 1 * 1 Partial Coloring Method Input: Output: Lemma: Can do this in randomized time. Outline of Algorithm 1. Partial coloring Method 2. EDGE-WALK: Geometric picture Discrepancy: Geometric View • Subsets π1 , π2 , … , ππ ⊆ [π] • Color with 1 or -1 to minimize imbalance 1 2 3 4 5 1 * 1 * 1 * 1 1 * * 1 1 1 * 1 1 * 1 1 * * 1 1 1 1 3 1 1 0 1 1 -1 1 1 -1 3 1 1 0 1 Discrepancy: Geometric View • Vectors π£1 , π£2 , … , π£π ∈ 0,1 π . • Want 1 2 3 4 5 1 * 1 * 1 * 1 1 * * 1 1 1 * 1 1 * 1 1 * * 1 1 1 1 1 -1 1 1 -1 3 1 1 0 1 Discrepancy: Geometric View • Vectors π£1 , π£2 , … , π£π ∈ 0,1 π . • Want Gluskin 88: Polytopes, Kanter’s lemma, ... ! Goal: Find non-zero lattice point inside Edge-Walk Goal: Find point in Claim: Willnon-zero find goodlattice partial coloring. • Start at origin • Brownian motion till you hit a face • Brownian motion within the face Edge-Walk: Algorithm Gaussian random walk in subspaces • Subspace V, rate πΎ • Gaussian walk in V Standard normal in V: Orthonormal basis change Edge-Walk Algorithm Discretization issues: hitting faces • Might not hit face • Slack: face hit if close to it. Edge-Walk: Algorithm • Input: Vectors π£1 , π£2 , … , π£π . • Parameters: πΏ, Δ, πΎ βͺ πΏ , π = 1/πΎ 2 1. π0 = 0. For π‘ = 1, … , π. 2. ππππ‘ = Cube faces nearly hit by ππ‘ . π·ππ ππ‘ = Disc. faces nearly hit by ππ‘ . ππ‘ = Subspace orthogonal to ππππ‘ , π·ππ ππ‘ Edgewalk: Partial Coloring Lem: For with prob 0.1 and Edgewalk: Analysis Discrepancy faces much farther than cube’s Pr ππππ βππ‘π π πππ π. ππππ βͺ Pr[∝ππππ βππ‘π π2 ππ’ππ exp −100 . ′ π ] 1 Pr ππππ βππ‘π that πoften! ππ’ππ Hit cube more Key point beats ππππ ∝union exp bound −1 . Six Suffice 1. Edge-Walk: Algorithmic partial coloring 2. Recurse on unfixed variables Spencer’s Theorem Summary I. Optimal, explicit π-nets for Gaussians • Kanter’s lemma, convex geometry II. Constructive Discrepancy Minimization • EdgeWalk: New LP rounding method Geometric techniques Others: Invariance principle for polytopes (Harsha, Klivans, M.’10), … Open Problems • FPTAS for computing supremum? • Beck-Fiala conjecture 81? – Discrepancy π( π‘) for degree π‘. • Applications of Edgewalk rounding? Rothvoss’13: Improvements for bin-packing! Thank you Edgewalk Rounding Th: Given π£1 , … , π£π , thresholds π1 , π2 , … ππ , Can find π ∈ −1,1 1. 2. π with
