Contrast Preserving Decolorization Cewu Lu, Li Xu, Jiaya Jia, The Chinese University of Hong Kong Mono printers are still the majority • Fast • Economic • Environmental friendly Documents generally have color figures The printing problem The printing problem The printing problem The printing problem The printing problem HP printer The printing problem Our Result Decolorization Mapping Single Channel Applications Color Blindness Applications Color Blindness Decolorization could lose contrast Mapping( ) = Mapping( ) = Decolorization could lose contrast Mapping Pervious Work (Local methods) • Bala and Eschbach 2004 • Neumann et al. 2007 • Smith et al. 2008 Pervious Work (Local methods) Naive Mapping Result Color Contrast Pervious Work (Global methods) • Gooch et al. 2004 • Rasche et al. 2005 • Kim et al. 2009 Pervious Work (Global methods) Color feature preserving optimization g f (c ) mapping function Pervious Work (Global methods) g f (c ) In most global methods, color order is strictly satisfied Color order could be ambiguous Can you tell the order? Color order could be ambiguous brightness( ) < brightness ( ) YUV space Lightness( ) > Lightness ( ) LAB space Color order could be ambiguous The order of different colors cannot be defined uniquely by people B. Wong et al., Nature Methods, 2010 People with different culture and language background have different senses of brightness with respect to color. E. Ozgen et al., Current Directions in Psychological Science, 2004 K. Zhou et al., National Academy of Sciences, 2010 Color order could be ambiguous If we enforce the color order constraint, contrast loss could happen Input [Rasche et al. 2005] [Kim et al. 2009] Ours Our Contribution Relax the color order constraint Unambiguous color pairs Global Mapping Bimodal Contrast-Preserving Weak Color Order Polynomial Mapping The Framework • Objective Function Bimodal Contrast-Preserving Weak Color Order • Finite Multivariate Polynomial Mapping Function • Numerical Solution Bimodal Contrast-Preserving • Color pixel { x , y } , grayscale contrast color contrast (CIELab distance) g xy g x g y , xy • g xy follows a Gaussian distribution with mean G xy , 2 g xy xy exp 2 2 2 xy Bimodal Contrast-Preserving • Color pixel { x , y } , grayscale contrast color contrast (CIELab distance) g xy g x g y , xy • a Gaussian distribution with mean . g xy follows xy xy G xy , 2 g xy g xy xy exp 2 2 2 Bimodal Contrast-Preserving • Tradition methods (order preserving): m ax g N G sign ( x , y ) xy , 2 ( x , y ) N : neighborhood pixel set • Our bimodal contrast-preserving for ambiguous color pairs: m ax g ( x , y ) N G xy , 2 G xy , 2 Bimodal Contrast-Preserving m ax g G sign ( x , y ) xy , 2 ( x , y ) N sign(x,y) = 1 xy m ax g G ( x , y ) N , xy 2 G , xy 2 g xy xy xy g xy Bimodal Contrast-Preserving m ax g G sign ( x , y ) xy , 2 ( x , y ) N sign(x,y) = 1 g xy xy m ax g G ( x , y ) N , xy 2 G , xy 2 xy xy g xy Our Work • Objective Function Bimodal Contrast-Preserving Weak Color Order • Finite Multivariate Polynomial Mapping Function • Numerical Solution Weak Color Order • Unambiguous color pairs: rx ry & g x g y & b x b y or rx ry & g x g y & b x b y Weak Color Order • Unambiguous color pairs: rx ry & g x g y & b x b y or rx x,y 1.0 0.5 ry & g x g y & b x b y unam biguous color pair am biguous color pair • Our model thus becomes m ax g ( x , y ) N G xy , x,y 2 1 G x,y , xy 2 Our Work • Objective Function Bimodal Contrast-Preserving Weak Color Order • Finite Multivariate Polynomial Mapping Function • Numerical Solution Multivariate Polynomial Mapping Function m ax g ( x , y ) N x , y G xy , 2 1 x , y G xy , 2 Solve for grayscale image: g Variables (pixels): 400x250 = 100,000 Example Too many (easily produce unnatural structures) Multivariate Polynomial Mapping Function m ax g ( x , y ) N x , y G xy , 2 1 x , y G xy , 2 • Parametric global color-to-grayscale mapping Small Scale grayscale value f (color vector, ) Multivariate Polynomial Mapping Function • Parametric color-to-grayscale f ( c , ) m i i i m i is the i monomial basis of n , c { r , g , b } . th n span{ r 1 g 2 b 3 : d i = 0, 1, 2, ... d 1 d 2 d 3 n} d d d When n = 2, a grayscale is a linear combination of elements 2 2 2 { r , g , b , rg , gb , rb , r , g , b } Multivariate Polynomial Mapping Function • Parametric color-to-grayscale f ( c , ) m i i i Multivariate Polynomial Mapping Function • Parametric color-to-grayscale f ( c , ) m i i i r g b rg rb gb 2 2 2 r g b Multivariate Polynomial Mapping Function • Parametric color-to-grayscale f ( c , ) m i i i g b rg rb gb 2 2 2 r g b Multivariate Polynomial Mapping Function • Parametric color-to-grayscale f ( c , ) m i i i b rg rb gb 2 2 2 r g b Multivariate Polynomial Mapping Function • Parametric color-to-grayscale f ( c , ) m i i i rg rb gb 2 2 2 r g b Multivariate Polynomial Mapping Function • Parametric color-to-grayscale f ( c , ) m i i i r 2 rb gb 2 2 g b Multivariate Polynomial Mapping Function • Parametric color-to-grayscale f ( c , ) m i i i gb r 2 g 2 b 2 Multivariate Polynomial Mapping Function • Parametric color-to-grayscale f ( c , ) m i i i r 2 g 2 b 2 Multivariate Polynomial Mapping Function • Parametric color-to-grayscale f ( c , ) m i i i g 2 b 2 Multivariate Polynomial Mapping Function • Parametric color-to-grayscale f ( c , ) m i i i b 2 Multivariate Polynomial Mapping Function • Parametric color-to-grayscale f ( c , ) m i i i Multivariate Polynomial Mapping Function • Parametric color-to-grayscale f ( c , ) m i i i 0.1550 0.8835 0.3693 0.1817 0.4977 -1.7275 -0.4479 0.6417 0.6234 Multivariate Polynomial Mapping Function • Parametric color-to-grayscale f ( c , ) m i i i 0.1550 0.8835 0.3693 0.1817 0.4977 -1.7275 -0.4479 0.6417 0.6234 Our Model g xy g x g y f c x , f c y , m i ix m iy i • Objective function: m ax ( x , y ) N G xy , x,y 2 1 G x,y , xy 2 Numerical Solution m ax ( x , y ) N m in Define: E x , y G xy , 2 1 x , y G xy , ( x , y ) N In x , y G xy , ( x , y ) N 2 1 G In x , y G xy , x,y 2 2 , xy 2 1 x , y G xy , 2 Numerical Solution m in E E j E m i ( x , y ) N xi 0 m yi m xj m yj 1 2 x , y m xj m yj x , y 0 i x,y x, yG x, y , x, yG x, y , 2 2 1 x , y G , x,y 2 Numerical Solution Initialize : m i ( x , y ) N i xi m yi m xj m yj 2 x , y 1 m xj m yj x , y Numerical Solution m i ( x , y ) N xi m yi m xj m yj 2 x , y 1 m xj m yj x , y i obtain x,y x, yG x, y , x, yG x, y , 2 1 2 x,y G , x,y 2 Numerical Solution m i ( x , y ) N xi m yi m xj m yj 2 x , y 1 m xj m yj x , y i obtain obtain x , y x,y x, yG x, y , x, yG x, y , 2 1 2 x,y G , x,y 2 Numerical Solution m i ( x , y ) N xi m yi m xj m yj 2 x , y 1 m xj m yj x , y i obtain obtain x , y x,y x, yG x, y , x, yG x, y , 2 1 2 x,y G , x,y 2 Numerical Solution m i ( x , y ) N xi m yi m xj m yj 2 x , y 1 m xj m yj x , y i obtain obtain x , y x,y x, yG x, y , x, yG x, y , 2 1 2 x,y G , x,y 2 Numerical Solution m i ( x , y ) N xi m yi m xj m yj 2 x , y 1 m xj m yj x , y i obtain obtain x , y x,y x, yG x, y , x, yG x, y , 2 1 2 x,y G , x,y 2 Numerical Solution (Example) Iter 1 r 0.33 g b 0.33 0.33 2 rg rb gb r 0.00 0.00 0.00 0.00 g 2 b 2 0.00 0.00 Numerical Solution (Example) Iter 2 r 0.97 g b 0.91 0.38 rg rb -3.71 2.46 gb r 2 -4.01 -4.02 g 2 b 2 4.00 0.79 Numerical Solution (Example) Iter 3 r g b rg rb 1.14 -0.25 1.22 -1.55 -1.53 gb r 2 -3.51 -1.18 g 2 3.32 b 2 0.69 Numerical Solution (Example) Iter 4 r g b rg rb gb 1.33 -1.61 2.10 1.35 -0.36 -1.61 r 2 -1.69 g 2 b 2 1.70 0.29 Numerical Solution (Example) Iter 5 r g b rg 1.52 -2.25 2.46 2.69 2 rb gb r -1.38 -0.30 -1.95 g 2 0.79 b 2 -0.02 Numerical Solution (Example) Iter 13 r g b 1.98 -3.29 3.02 rg rb gb 5.94 -3.38 2.81 r 2 g 2 b 2 -2.91 -1.56 -0.96 Numerical Solution (Example) Iter 14 r g b 1.99 -3.31 3.03 rg rb gb 6.03 -3.42 2.89 r 2 g 2 b 2 -2.95 -1.62 -0.98 Numerical Solution (Example) Iter 15 r g b 2.00 -3.32 3.04 rg rb gb 6.10 -3.45 2.94 r 2 g 2 -2.98 -1.67 b 2 -1.00 Results Input Ours [Rasche et al. 2005] [Kim et al. 2009] Results Input Ours [Rasche et al. 2005] [Kim et al. 2009] Results Input Ours [Rasche et al. 2005] [Kim et al. 2009] Results Input Ours [Rasche et al. 2005] [Kim et al. 2009] Results (Quantitative Evaluation) • color contrast preserving ratio (CCPR) CCPR= # ( x , y ) | ( x , y ) , | g x g y | | | the set containing all neighboring pixel pairs with the original color difference . x,y Results (Quantitative Evaluation) 10 Our Results (Quantitative Evaluation) x,y 10 Results (Quantitative Evaluation) x,y x,y & g x,y 10 Results (Quantitative Evaluation) Number: 38740 Number: 24853 x,y x,y & g x,y 10 Results (Quantitative Evaluation) Number: 38740 CCPR= Number: 24853 24853 38740 64.2% Results (Quantitative Evaluation) Results (contrast boosting) substituting our grayscale image for the L channel in the Lab space Results (contrast boosting) substituting our grayscale image for the L channel in the Lab space Conclusion • A new color-to-grayscale method that can well maintain the color contrast. • Weak color constraint. • Polynomial Mapping Function for global mapping. The End Limitations • Color2gray is very subjective visual experience. Contrast enhancement may not be acceptable for everyone. • Compared to the naive color2grayscale mapping, our method is less efficient due to the extra operations. An arguable result Running Time • For a 600 × 600 color input, our Matlab implementation takes 0.8s • A C-language implementation can be 10 times faster at least.