Asymptotic Behavior of Linear Approximations of Pseudo-Boolean Functions Paper: jc11-4-2786; 2006/12/1 Asymptotic Behavior of Linear Approximations of Pseudo-Boolean Functions Guoli Ding , Robert F. Lax , Peter Chen , and Jianhua Chen Department of Mathematics, Louisiana State University, Baton Rouge, LA 70803, USA E-mail: ding, lax@math.lsu.edu Department of Computer Science, Louisiana State University, Baton Rouge, LA 70803, USA E-mail: chen, jianhua@csc.lsu.edu [Received March 25, 2006; accepted August 16, 2006] We study the problem of approximating pseudoBoolean functions by linear pseudo-Boolean functions. Pseudo-Boolean functions generalize ordinary Boolean functions by allowing the function values to be real numbers instead of just the 0-1 values. PseudoBoolean functions have been used by AI and theorem proving researchers for efficient constraint satisfaction solving. They can also be applied for modeling uncertainty. We investigate the possibility of efficiently computing a linear approximation of a pseudoBoolean function of arbitrary degree. We show some example cases in which a simple (efficiently computable) linear approximating function works just as well as the best linear approximating function, which may require an exponential amount of computation to obtain. We conjecture that for any pseudo-Boolean function of fixed degree k 1 where k is independent of the number of Boolean variables, the best linear approximating function works better than simply using the linear part of the target function. We also study the behavior of the expected best linear approximating function when the target pseudo-Boolean function to be approximated is random. Keywords: approximation, uncertainty, pseudo-Boolean functions 1. Introduction Pseudo-Boolean functions generalize ordinary Boolean functions by allowing the function values to be real numbers instead of just the 0-1 values. A pseudo-Boolean function f is a mapping from the set of all Boolean vectors of length n to the reals. Pseudo-Boolean functions have been used for solving Constraint Satisfaction Problems by the AI and theorem-proving research communities. For example, Dixon and Ginsberg [4] studied using pseudo-Boolean constraints for solving satisfiability problems and showed that pseudo-Boolean solvers tend to be more efficient compared with resolution-based satisfiability solver. Another motivation for studying pseudoBoolean functions from the AI point of view is that these Vol.11 No.4, 2007 functions provide a flexible framework for modeling uncertainty and yet they are fairly simple compared with other alternative approaches; e.g., fuzzy logic [14]. Fuzzy logic allows the variables x i to take values in the interval 0 1 and support quite complex fuzzy inferencing mechanisms using fuzzy rules, whereas pseudo-Boolean functions restrict the variables xi to take only Boolean (0 or 1) values. Uncertainty modeling by pseudo-Boolean functions is accomplished by allowing function values to be real numbers. In this paper, we study the problem of approximating arbitrary pseudo-Boolean functions by linear pseudoBoolean functions. The rationale for using approximations rather than the exact original functions consists in conceptual simplicity and computational tractability. Linear functions have a rather simple representation and intuitive appeal for human perception and understanding. Computationally, they are also much easier to handle compared with most other functions. There are many works in research on uncertainty in AI that utilize linear functions to model uncertainty. For example, in [11], linear belief functions are employed to describe the connections between returns of stocks with various factors in the financial market. A qualitative linear utility formulation is presented in [5] for modeling the usefulness of epistemic beliefs. Linear discriminant functions are also widely used in pattern classification sometimes even in situations in which the underlying patterns are non-linear. In [9], linear threshold functions are used for the natural language understanding task. Let n be a positive integer and B n be the set of all n-dimensional 0-1 vectors. A pseudo-Boolean function f x1 x2 xn is a mapping from Bn to the reals. A pseudo-Boolean function is often closely related to the probability distribution of Bernoulli random variables. For a simple example, if X1 X2 , and X3 are independent identically distributed Bernoulli random variables with PXi 1 23, then their joint probability distribution is described by the pseudo-Boolean function f x 1 x2 x3 1271 x 1 x2 x3 x1 x2 x1 x3 x2 x3 x1 x2 x3 in the sense that PX1 x1 X2 x2 X3 x3 f x1 x2 x3 . Pseudo-Boolean functions are also the main objects of study in the theory of cooperative games in economics (see [6]). Pseudo-Boolean functions also appear in the Journal of Advanced Computational Intelligence and Intelligent Informatics 1 Ding, G. et al. theory of evolutionary computation, where they are called fitness functions (see [8] and [15]). Jin [8] points out that evaluation of a fitness function in a real-world application is sometimes computationally very expensive, thus necessitating the approximation of the function by a low-degree function. Zhang and Rowe [15] study linear and quadratic approximations of pseudo-Boolean functions and compare these. For each variable x i , we define x̄i 1 xi . Then L x1 x̄1 x2 x̄2 xn x̄n is the set of literals. Clearly, every pseudo-Boolean function f x 1 x2 xn has a pseudo-Boolean expression ∑ αZ ∏ z where . . . . . . . . . . . . . (1) zZ Z L ∏ z is defined to be 1. z0/ As usual, we assume that αZ 0 if z z̄ Z for some z L. The degree of expression (1) is defined to be the largest Z such that α Z 0. Starting with expression (1), if we replace each x̄ i by 1 xi and then multiply out and collect terms, we obtain a representation of f of the form f x1 xn ∑ T N a T ∏ xi i T . . . . . (2) The expression (2) is called a multilinear polynomial. The following propositions are easy to prove (cf. [2]). Proposition 1.1 Every pseudo-Boolean function has a unique multilinear polynomial expression. Proposition 1.2 If a pseudo-Boolean function has a pseudo-Boolean expression of degree d, then its multilinear polynomial expression has degree at most d. Let N 1 2 3 n, x x1 x2 xn , and f x ∑ T N a T ∏ xi . . . . . . . . . (3) iT Let x be the linear function aT t 1 ∑ 2t ∑ T N i N aT ∑ 2t 1 T i xi . . . (4) where t T . It was proved in [6] that x is the best linear approximation of f x, in the sense that it minimizes ∑n xB f x l x2 over all linear functions l x. Even though an explicit expression for x is available, computing all its coefficients is a time-consuming task since each coefficient of x is a sum of exponentially many terms. In this paper, we discuss possibilities of substituting x with an efficiently computable function and still achieving the same quality of approximation. We will mainly focus on the asymptotic behavior of different approximations. Let Lx denote the linear part of f x. We first investigate the possibility of using the function Lx to approximate f x. We show some example functions which can 2 (or cannot) be approximated well by Lx. A conjecture is presented that suggests the approximation of f x by Lx will not be effective if f x has degree k 1 for k independent of n. We then study the problem of approximating by the expectation of x to a random f x with each coefficient a T being a random variable. We identify conditions under which such an approximation is (or is not) reasonable. 2. Relative Quality of the Best Linear Approximation In this section, we investigate the following natural question: Is there a good approximation ˆx of x? As mentioned earlier, we will take the asymptotic point of view. That is, we compare ˆx and x when n is sufficiently large. Clearly, we want ˆx to be linear. We also want ˆx to be efficiently computable. There are two natural choices for ˆx. If all coefficients of x have limits, as n approaches infinity, then this limit is a natural candidate for ˆx. However, this limit usually does not exist. In such a case, we propose the following. Let Lx be the linear function n a0/ ∑ ai xi i 1 which is the linear part of f x. If f x is a nonlinear function, we would like to compute ρ f ∑xBn f x x2 ∑xBn f x Lx2 which would tell us, when comparing with Lx, how much better the best linear approximation x is. In the following, we consider a few examples. We will make a conjecture based on these examples, which says that ρ f is small if f has bounded degree (with the bound being independent of n). This would imply that if f has bounded degree and n is large, then the linear part of f would not be nearly as good an approximation to f as the best linear approximation given by Eq. (4) and it would be worth performing the computations required in Eq. (4). 2.1. A Highly Nonlinear Function Let us consider the case when a T That is f x 1, for all T N. ∑ ∏ xi T N i T Then its best linear approximation is x n 3 2 n1 1 Journal of Advanced Computational Intelligence and Intelligent Informatics n 3 3 2 n ∑ xi . (5) i 1 Vol.11 No.4, 2007 Asymptotic Behavior of Linear Approximations of Pseudo-Boolean Functions Notice that all coefficients of x approach infinity, as n approaches infinity, thus x does not have a limit function. In such a case, we would like to compare x with Lx by computing ρ f . From Eq. (5) it is straightforward to verify that ∑ xBn f x x 2 5 1 9 10 n n 1 9 2.3. A Two-Term Function Let us consider f x x1 x2 xn xn1 xn2 x2n (6) Then its best linear approximation is and n ∑n ∑ xB n tions of unbounded degree with a small number of terms, as shown by the example from the next subsection. f x2 ∑ i 0 Since xBn f x n 2i 2 5n i x 2 ∑ xB ∑ xBn Lx2 f x f x2 it follows from Eqs. (6) and (7) that ρ f 1 o1. Therefore, we conclude that, asymptotically, Lx performs just as well as x. Since Lx is much easier to compute, this observation suggests that, in this example, we should use Lx, instead of x, to approximate this function f x. Let us consider the following quadratic function ∑ xi x j nn 1 1i j n n 1 1 2n xi 2n1 2n1 i∑1 . . . . . . (9) ∑ f x x 2 ∑ f x ˆ x 2 xB2n and x B2n 2 n1 ∑ x B2n 2 n1 1 f x n1 2n Lx2 1 1 n 2 which imply that ρ f 1 o1. Therefore, we can say that ˆx Lx 0 performs just as well as x. This example suggests that, in general, x can only perform well when the degree of f x is bounded, which can be considered as supporting evidence for our Conjecture above. 2.2. A Quadratic Function f x Clearly, all coefficients of x have limit zero. Thus this limit function, ˆx, is the same as Lx, and they both are the zero function. Now it follows from Eq. (9) that n x . . . . . (7) Then x 8 n 1 2 n ∑ xi . . . . (8) Clearly, x has no limit. Thus we consider ρ f . It follows from Eq. (8) that ∑n xB f x x f x Lx2 ∑ 2 ∑n xB n 12n5 nn and i 0 n6 2 n 2 i i i n2 12 4 3n 2nn 1 which imply that ρ f n22 1 o1. Therefore, x performs significantly better than Lx. Fix an integer k 1. Define ρ k n by ρk n sup ρ f , where the sup is over all pseudo-Boolean functions f : Bn R such that deg f k. We make the following conjecture. Conjecture. If k 1 is a fixed integer, then ρ k n o1. If this conjecture is true, it provides a justification for computing x, instead of simply using Lx. We point out that, if the conjecture is true, it is the best possible in the sense that the conjecture cannot be extended to funcVol.11 No.4, 2007 3. The Expected Linear Approximation i 1 We continue our study on the problem of finding a good approximation to x. In this section, we will consider it from a different viewpoint. A random polynomial is a polynomial whose coefficients are random variables. Random polynomials have applications in engineering and economics – see [1]. In this section, we consider a random multilinear polynomial; i.e., suppose in Eq. (2) that every a T aT ω is a random variable on a sample space Ω. A realization of a random multilinear polynomial is obtained when we evaluate the random variable coefficients at an element ω of the sample space Ω. Then we ask: (*) Instead of computing the best linear approximation x for each realization of f x, is it possible to find a single linear pseudo-Boolean function that depends only on the expected values of each a T and that serves as a good approximation for every realization of f x? 3.1. A Negative Answer In this subsection, we exhibit a scenario under which the answer to question (*) is negative. Suppose f x is a linear function. That is, a T 0, for all T N with T 1. In this case, it is clear that x Journal of Advanced Computational Intelligence and Intelligent Informatics 3 Ding, G. et al. f x. Moreover, the linear approximation obtained from the expected values of each a T is n ˆ x E a0/ ∑ E ai xi Let D D i 1 Obviously, ∑n xB x f x 2 0 Let us assume that the random variables a T , for all T 1, are independent. Let b0 a0/ E a0/ and bi ai E ai , for i 1 2 n. Then E b i 0 (i 0 1 2 n), E b20 V a0/ , and E b2i V ai (i 1 2 n). For each vector x B n , let us define X i N : xi 1. It follows that ˆ x 2 2 b0 ∑ bi iX 2 b0 ∑ b2i i X 2 ∑ bi b j E bi b j E bi E b j 0 n 2nV a0/ 2n1 ∑ V ai i 1 0 and β is finite, then The implication of this theorem is the following. If the variance of every a T is positive and finite, then we can say that, on average, to approximate f x, its best linear approximation x is not too much better than x, the best linear approximation of f x. For example, if all the aT are independent random variables with expected value 1 and positive variance, then the linear pseudo-Boolean function from Eq. (5) would be a good approximation for every realization of f x if n is large. We prove the theorem by proving a sequence of three lemmas, using notation as above. ∑ ∑ λX2T V E D is bounded below by a positive number. This example shows that the answer to (*) is negative in general. That is, we have to compute x for each f x because the difference between ∑ f x x2 xB ˆ x f x x 2 where f x i 1 f x ∑ xBn if T X; if T X Proof. It is routine to verify that n aT 2t V a0/ ∑ V ai ∑n T NX N which can be arbitrarily large, provided and E t 1 1 2t λX T t 1 2T X Thus x 2 f x Lemma 3.1 where the last sum is taken over all indices i j in X 0. Notice that δ xB E D E D 5 n n 5 O1 E D 6 5 xBn f x ∑n x 2 f x Theorem. If α But what can we say about δ E ∑ f x ˆx2 ? ∑ xBn ∑ aT t 1 aT ∑ 2t T N ∑ aT ∑ aT λX T T X x Bn 2 x T X , under any measure, can be ar- T N bitrarily large. ∑ iX aT t 1 2t N ∑ T aT ∑ 2t 1 T i aT T X t 1 N2 ∑ T Therefore, 3.2. A Positive Answer In this subsection, we assume that E f x V aT β for all T N. Again, we assume that a S and aT are independent if S T . Let aT E aT , for all T N. Let f x ∑ aT ∏ xi α T N 4 a t 1 ∑ T 2t ∑ T N iN a ∑ 2t T 1 T i ∑ λX2T E ∑ λX2T V aT ∑ ∑ S T N 2 λX S λX T aS aT ∑ λX2T T N T N xi a2T 2 T N Then its best linear approximation is iT 2 T N x x ∑ S T N λX2T V aT f x aT 2 λX S λX T aS aT x 2 which proves the lemma. Lemma 3.2 E D α 3n 1 Journal of Advanced Computational Intelligence and Intelligent Informatics o1. Vol.11 No.4, 2007 Asymptotic Behavior of Linear Approximations of Pseudo-Boolean Functions Proof. By Lemma 3.1, E D ∑ ∑ It is straightforward to verify that λX2T V aT T NX N α ∑ ∑ T NX N λX2T ∑ λX2T X T t 1 2t 1 2nt 1 t 1 2t 2nt 1 t 1 2t 2 2 2 t 1 2t 1 2nt ∑ ∑ 0/ R N T S T 2nt ∑ 1 t 1 2s 2t t t 1 ∑ t s s 0 2 2nt t 1 2s 2t n t ∑ t 0 3n 2t 1 t 2nt Lemma 3.3 α 2β n5 5 n 5 2 n n3 3 2t 1 t n 3 2 f x 81 T ∑ µX T aT 2 5 2 x t x ∑ T T ¼ E d x 2 µX S s 1 β n 2T X . Then 2t ∑ aS ∑ µX T aT T N ∑ aS ∑ µX T aT S X T N T ∑ µX2 T aT 2 T N µX T µX T aT aT ¼ N ¼ x 2 f x x 2 aS ∑ µX2 T V ∑ µX SV T N 2s2s 1 n ∑ s n x x 0 n 5 2 x ∑ a T 12s x s1 s 2s s 0 n5 5 On the other hand, ∑ ∑ µX2 T ∑ ∑ µX2 T T NX N n ∑ n t 5 2 t ∑ i 0 t i t 1 2i 2t 2 2nt n5 5 The result follows from the last three equations immediately. D 2α S X It follows that ∑ µX S aS aS µX T aS a For each S X, we have n E D 2 S X t 0 n5 5 Then S X Vol.11 No.4, 2007 2 ¼ Notice that f x x2 can be expressed in a similar way (with each aT replaced by a T ). Let 4 and f x µX T a X NT N n n2 26n 81 9 Proof. Let µX T S X T N S T 2n3t 1 t 2 o1 3n 1 ∑ 2n2t 1 t which proves the lemma. ∑ T N aS aS S S¼ X X NS X ∑ 2 ∑ ∑ µX S 2 1t 12t λX2T 2 2 Therefore, ∑ ∑ d x f x 2n2t 1 t 2n3t 1 t 2 T NX N n ∑ a2S 2 1 2s 2t S T 2 2nt ∑ aS 2 S X S X nt ∑ aS T N ∑ λX2T 2 2 T . Then, for X N nt 2 S X ∑ λX2T X T For any T X N, let S X T and R X any fixed T N, x f x 5 2 n5 5 Proof of the Theorem. The theorem follows from the last two lemmas immediately. 4. Discussion Pseudo-Boolean functions have been studied extensively by the Operations Research community for optimizations. In recent years the AI research community, in particular, the theorem-proving researchers have investigated the use of pseudo-Boolean functions and pseudo- Journal of Advanced Computational Intelligence and Intelligent Informatics 5 Ding, G. et al. Boolean optimizations for various AI applications. In [13], pseudo-Boolean functions are used for modeling a job-shop scheduling problem. Lower bounding method has been developed in [12] for pseudo-Boolean optimization problems. The existing AI literature indicates the usefulness of pseudo-Boolean functions for AI-related optimization and constraint solving. We take the perspective that pseudo-Boolean functions can be used for modeling uncertainty, in addition to being a model for constraints. The function value f x for a pseudo-Boolean function f on a point x can be seen as modeling the degree of certainty (positive or negative) of x belonging to the concept described by the function f . Consider the following terrorist profiling problem. Suppose we know (from existing records) that there are five basic attributes relevant for recognizing instances of terrorists/non-terrorists: x: having traveled to certain sensitive regions; y: being from some specific ethnic groups; z: being an old person; u: having attended a pilot school; v: having past criminal record. Assume that we also know that the most relevant composite attributes for terrorists profiling are: x y z xu xv. Assume that we have also seen three specific known instances of terrorist together with their ratings (how dangerous they are) given by some source (say, CIA). We model the terrorists profiles by a pseudo-Boolean function of the form f x y z u v ax by cz dxu exv Here V = x y z u v is the set of Boolean variables and each of the coefficients a b c d e takes value in 1 1. Suppose the known instances of terrorists are given by: f 11101 05 f 11010 18 and f 10011 20. It is intuitive to see from the examples that attributes xu, x, xv are quite important to make an instance’s rating highly positive (more likely to be a terrorist) - the instance vector 10011 with x xu xv 1 and y z 0 has the highest rating among the three instances. On the other hand, the attribute z seems to reduce an instance’s rating: the vector 11101 (with z 1) has much smaller rating than the other two instances (with z 0. In [3], we described a method for learning the “best” pseudo-Boolean function that is compatible with a limited set of training data. This method involves considering a polyhedron whose points correspond to vectors of coefficients of pseudo-Boolean functions that are compatible with the data. The best function then corresponds to the centroid of this polyhedron. (This is similar to the Bayes point used in the theory of Support Vector Machines [7].) Using this learning method here, we obtain the pseudo-Boolean function f 077x 015y 077z 088xu 035xv. Note that this pseudo-Boolean function is not a linear one. The best approximating linear function is given 6 by 088 0354 077 088 0352x 015y 077z 0882u 0352v. That is, 03075 1385x 015y 077z 044u 0175v Applying the resulting to the 3 known instance vectors, we obtain 11101 06325, 11010 16675, and 10011 16925. On the other hand, if we apply the function Lx, the linear part of f , L 077x 015y 077z, the results are much worse: L11101 015, L11010 092, and L10011 077. This example illustrates that the best linear approximation of a quadratic function is generally better than using the linear part of the original function, which is related to our Conjecture in section 2. 5. Conclusions Pseudo-Boolean functions are useful for AI applications such as constraint solving and uncertainty modeling. Approximating a pseudo-Boolean function of arbitrary degree by a linear pseudo-Boolean function is often desired for simplicity and computational efficiency. Using the linear part of the target function as an approximation would produce similar asymptotic effect as the best linear approximation function does for some cases and save computations, but in general this approach may not be valid. We also considered random multilinear polynomials and found conditions that tell when the best linear approximation to the multilinear polynomial given by the expected values of the coefficient random variables may be used to approximate every realization of the random multilinear polynomial. Acknowledgements This research was partially supported by NSF grant ITR0326387 and AFOSR grants FA9550-05-1-0454, F49620-03-10238, F49620-03-1-0239 and F49620-03-1-0241. We thank the referees for helpful comments. References: [1] A. T. Bharucha-Reid and M. Sambandham, “Random Polynomials,” Academic Press, 1986. [2] E. Boros and P. L. Hammer, “Pseudo-Boolean Optimization,” Discrete Appl. Math., 123 (1-3), pp. 155-225, 2002. [3] G. Ding, J. Chen, R. Lax, and P. Chen, “Efficient learning of pseudo-boolean functions from limited training data,” Foundations of Intelligent Systems: 15th International Symposium, ISMIS 2005, Lect. Notes in Comp. Sci. 3488 (Springer, 2005), pp. 323-331. [4] H. E. Dixon and M. L. Ginsberg, “Inference methods for a pseudoBoolean satisfiability solver,” Proc. of American Association of AI Conference, 2002. [5] P. H. Giang and P. P. Shenoy, “A Qualitative Linear Utility Theory for Spohn’s Theory of Epistemic Beliefs,” Proceedings of Int. Conference on Uncertainty in AI, San Francisco, CA, pp. 220-229, 2000. [6] P. L. Hammer and R. Holzman, “Approximations of PseudoBoolean Functions; Applications to Game Theory,” ZOR – Methods and Models of Operations Research, 36, pp. 3-21, 1992. [7] R. Herbrich and T. Graepel, “Bayes Point Machines,” Journal of Machine Learning Research, 1, pp. 245-279, 2001. [8] Y. Jin, “A Comprehensive Survey of Fitness Approximation in Evolutionary Computation,” Soft Computing, 9, pp. 3-12, 2005. Journal of Advanced Computational Intelligence and Intelligent Informatics Vol.11 No.4, 2007 Asymptotic Behavior of Linear Approximations of Pseudo-Boolean Functions [9] R. Khardon, D. Roth, and L. G. Valiant, “Relational Learning for NLP using Linear Threshold Elements,” Proceedings of Int. Joint Conference on AI (IJCAI’99), Aug., 1999. [10] R. Lax, G. Ding, P. Chen, and J. Chen, “Approximating PseudoBoolean Functions on Non-uniform Domains,” Proceedings of IJCAI-05, pp. 1754-1755, 2005. [11] L. Liu, C. Shenoy, and P. P. Shenoy, “A Linear Belief Function Approach to Portfolio Evaluation,” Proceedings of Int. Conference on Uncertainty in AI, San Francisco, CA, pp. 370-377, 2003. [12] V. M. Manquinho and J. Marques-Silva, “Integration of Lower Bound Estimates in Pseudo-Boolean Optimization,” Proc. of IEEE Int. Conference on Tools with AI, 2004. [13] S. Prestwich and C. Quirke, “Boolean and Pseudo-Boolean Models for Scheduling,” Proc. of Second International Workshop on Modeling and Reformulating Constraint Satisfaction Problems, 2003. [14] L. A. Zadeh and J. Kacprzyk (Eds.), “Fuzzy Logic for the Management of Uncertainty,” John Wiley & Sons, 1992. [15] H. Zhang and J. E. Rowe, “Best Approximations of Fitness Functions of Binary Strings,” Natural Computing, 3, pp. 113-124, 2004. Name: Peter P. Chen Affiliation: Department of Computer Science, Louisiana State University Address: Baton Rouge, Louisiana, 70803, USA Brief Biographical History: Dr. Peter Chen, a Distinguished Chair Professor at LSU (Baton Rouge), has taught at MIT, UCLA, and Harvard. Main Works: ¯ Prof. Chen is internationally known for his work on the Entity-Relationship (ER) model. He has received many awards including ACM/AAAI Allen Newell Award, IEEE Harry Goode Award, Pan Wen-Yuen Outstanding Research Award, DAMA International Achievement Award, and Stevens Software Method Innovation Award. He has served as a keynote speaker at approximately 30 international conferences. Name: Guoli Ding Membership in Learned Societies: Affiliation: Department of Mathematics, Louisiana State University ¯ Fellow of IEEE, ACM, AAAS ¯ Member of European Academy of Sciences ¯ Invited Expert in XML Schema and XLink WGs of W3C ¯ Member of U.S. NSF/CISE advisory committee and U.S. Air Force Scientific Advisory Board ¯ Listed in Who’s Who in America and Who’s Who in the World Address: Baton Rouge, Louisiana, 70803, USA Brief Biographical History: 1991 Received Ph.D. from Rutgers University 1991- Assistant Professor, Mathematics Department at Louisiana State University ????- Associate Professor, Mathematics Department at Louisiana State University ????- Professor, Mathematics Department at Louisiana State University Name: Jianhua Chen Affiliation: Main Works: Department of Computer Science, Louisiana State University ¯ Prof. Ding has published many research articles in the fields of graph theory and matroid theory. Address: Baton Rouge, Louisiana, 70803, USA Name: Brief Biographical History: Robert F. Lax 1988 Received Ph.D. in Computer Science from Jilin University, Chang Chun, China 1988- Visiting Assistant Professor, the Computer Science Department of Louisiana State University ????- Assistant Professor, the Computer Science Department of Louisiana State University currently Associate Professor, the Computer Science Department of Louisiana State University Affiliation: Department of Mathematics, Louisiana State University Main Works: ¯ Dr. Chen’s research interests include machine learning and data mining, Web mining, fuzzy Logic and Fuzzy Clustering, Knowledge Representation and Reasoning. Address: Baton Rouge, Louisiana, 70803, USA Brief Biographical History: 1973 Received Ph.D. from MIT 1973- Faculty member, Department of Mathematics, Louisiana State University Main Works: ¯ Prof. Lax has published research articles in the fields of algebraic curves and algebraic coding theory. Vol.11 No.4, 2007 Journal of Advanced Computational Intelligence and Intelligent Informatics 7