Product Form Feature Selection for Numerical Definition-Based Design
Chih-Chieh Yang
Department of Multimedia and Entertainment Science, Southern Taiwan University,
Tainan County, Taiwan 71005
Hung-Yuan Chen
Department of Visual Communication Design, Southern Taiwan University,
Tainan County, Taiwan 71005
Abstract
For developing appealing products, it is important for designers to pin point
critical product form features (PFFs) that influence consumers’ affective responses
(CARs) toward a product design. Manual inspection of critical form features based on
expert opinions has not proved to meet with the acceptance of consumers. In this
paper, a product form feature selection model based on CARs and a numerical
definition-based systematic approach (NSDA) is proposed. First, NSDA was used to
generate an explicit numerical definition of product form design. Next, CARs were
described using single adjectives as affective dimensions. The evaluation data of
consumers can be gathered by a semantic differential (SD) experiment. The prediction
models of CARs were constructed using support vector regression (SVR) and
multiple linear regression (MLR). Two feature selection methods, namely SVR with
support vector machine recursive feature elimination (SVM-RFE) and MLR with the
stepwise procedure, were used to select critical form features. A case study of knife
design is given to demonstrate the experimental results of the two feature selection
methods. The results of our experiment show that the feature ranking obtained from
SVM-RFE is very helpful to determine the importance of the form features. It is also
possible to select a subset of features with a given predictive performance of the SVR
model. On the other hand, the results of MLR with the stepwise procedure provide
several useful statistics to analyze the form features. The effects of the form features
for inducing specific CAR of knife design can also be interpreted by the standardized
regression coefficients. Since only a small data set of knife design was adopted in this
study, a further study using different kinds of product samples is worthy of ongoing
investigation.
Keywords: Feature selection; support vector regression; support vector machine
recursive feature elimination (SVM-RFE); Multiple linear regression; Kansei
engineering; Product form design.
1
1. Introduction
A product’s appearance is one of the most important factors affecting a
consumer’s purchasing decision. Many systematic product design studies have been
carried out to get a better insight into consumers’ subjective perceptions. The most
notable research is Kansei engineering (KE) (Jindo et al., 1995). The basic
assumption of KE studies is that there exists a cause-and-effect relationship between
product form features (PFFs) and consumers’ affective responses (CARs) (Han and
Hong, 2003). Therefore, a prediction model can be constructed from collected product
samples and the CAR data. Using these models, the relationship between PFFs and
CARs can be analyzed and a specially designed product form for specific target
consumer groups can be produced more objectively and efficiently. The construction
of the prediction model can be regarded as a regression problem taking PFFs of
product samples as input data and the CAR data as output values. Various methods
can be used to build the prediction model including multiple linear regressions (MLR)
(Chen and Chang, 2009), quantification theory type I (QT1) (Lai et al., 2006), partial
least squares regression (PLSR) (Han and Kim, 2003), neural networks (NN) (Hsiao
and Huang, 2002) and support vector regression (SVR) (Yang and Shieh, 2010). Of
these methods, SVR’s remarkable performance makes it the first choice in a number
of real-world applications but it is seldom adapted to the product design field.
In the study of Yang and Shieh (2010), SVR was introduced to develop a model
that predicts CARs for product form design with a very satisfactory predictive
performance. However, the PFFs of product samples (mobile phone design) were
analyzed by a morphological analysis (Jones, 1992), which results in a product form
representation with a mixed of continuous and discrete design attributes. This kind of
product form encoding is actually an abstract and simplified representation of the
form design and not directly linked to the form features of a product. A convincing
remedy, proposed by Chang and Chen (2008), to this problem is to employ a
numerical definition-based systematic approach (NSDA) to generate an explicit
numerical definition for product form design. Using NSDA, it is beneficial that not
only the importances of the form features obtained from feature selection methods can
be directly interpreted in the domain of the form features, but also the integration with
hybrid Kansei engineering system, such as the automatic product design systems
proposed by Hsiao and Tsai (2005) and Tsai and Chou (2007), to generate new
product form design according to the specified affective responses is quite
straightforward.
Although being an important research topic for KE, the problem of product form
2
feature selection according to CARs has not been intensively investigated. CARs are
often influenced by a wide variety of form features. The number of form features
could be many and might be highly correlated to each other. The relative importance
of each form feature is hard to identify so the selection of the critical form features
that please the consumer is a difficult task. When designing new products, critical
design features are often arrived at based on the opinions of expert (such as product
designers) or focus groups. However, the selection of features based on expert opinion
has its drawbacks such as a lack of objectivity and expert availability (Han and Kim,
2003). Moreover, the designers often need to combine or re-organize form features.
Therefore, if the relative importance and the ranking of the form features can be
analyzed, it can facilitate the decision making of designers. Since the feature selection
methods are connected with the prediction model of CAR, not only the critical form
feature can be selected but also their influence to produce specific CARs can be
extracted. Only a few attempts have been made to overcome these above-mentioned
shortcomings in the product form feature selection process. Han and Kim (2003) used
several statistical methods for screening critical design features including principal
component regression (PCR), cluster analysis, and PLSR. In Han and Yang (2004), a
genetic algorithm-based partial least squares method (GA-based PLS) was applied to
screen design variables.
In the present study, a product form feature selection model is constructed based
on CARs and NSDA in the context of KE. CARs were described using single
adjectives and NSDA was used to define the product form design. In order to gather
the evaluation data of consumers, a semantic differential (SD) experiment was
conducted using questionnaire investigation. The prediction models of CARs were
constructed using both SVR and MLR. Two feature selection methods, including SVR
with support vector machine recursive feature elimination (SVM-RFE) and MLR with
the stepwise procedure, were used to pin point critical form features. A case study of
knife design is given to demonstrate the analysis result. The reminder of the paper is
organized as follows: Section 2 reviews the problem of product form feature selection
and introduces the two feature selection methods, namely SVR with SVM-RFE and
MLR with the stepwise procedure. Section 3 presents the proposed model for product
form feature selection. Section 4 describes the experimental results and analyses of
the two feature selection methods. Section 5 offers some brief conclusions.
2. Background review
2.1. The problem of product form feature selection
3
Although there are only sparse literatures existing for product form feature
selection, the problem of feature selection can be found in many other fields besides
that of product design. The crux of the problem is how to find the subset of PFFs with
the least possible generalization errors and to select the smallest possible subset with a
high predictive capability. Different approaches have been proposed for solving the
feature selection problem including rough sets (Wakaki et al., 2004), rough-fuzzy
(Jensen, 2005), neuro-fuzzy (Pal et al., 1996), and support vector machine (SVM)
(Liu and Zheng, 2006). Another crucial issue in solving the PFFS problem is how to
deal with the nonlinear correlations between PFFs and CARs effectively (Park and
Han, 2004). The most widely adapted techniques such as MLR and QT1 do not
handle nonlinear relationships very well. In contrast, SVM and SVR are known for
their elegance in solving nonlinear problems by applying the kernel trick, which
automatically maps a feature space nonlinearly.
Based on whether or not feature selection is performed independently of the
learning algorithm that constructs the prediction model, feature selection methods can
be grouped into two categories: the filter method and the wrapper method (Kohavi
and John, 1997). The wrapper method is model-dependent. Based on the predictive
performance, the method evaluates directly the goodness of the selected feature’s
subset, which should intuitively yield a better performance. Many reported
experimental results also favored the wrapper method (Juang and Katagiri, 1992;
Wakaki et al., 2004). A well-known wrapper method, SVM-RFE, which was proposed
to select a relevant set of features for cancer classification problem (Guyon et al.,
2002). SVM-RFE, as a sequential backward feature elimination method, has been
successfully applied for product form feature selection based on a multiclass fuzzy
SVM classification model (Shieh and Yang, 2008b). In this study, SVM-RFE was
integrated into the SVR model to identify critical form features and obtain the feature
ranking. However, from a product designer’s point of view, the predictive
performance of the constructed model or which subset of the form features was
selected may not be main concern. On the contrary, it is more appealing for a design
practitioner to get better insight into the relationships of the form features and to
examine the effects of the features for inducing specific affective responses. Therefore,
MLR, as a linear model, is also adapted for product form feature selection in this
study. MLR equipped with a couple of well-developed analyzing statistics yields a
simple but intuitive manner to study the form features.
2.2. Feature selection using SVR with SVM-RFE
For a regression problem, a set D of l data samples, each represented are
4
given as ( xi , yi ) where xi  Rn is the input feature vector, yi  R is the desired
output value. In a standard SVR with a linear  -insensitive loss function the
objective is to find a function y  f ( x) , where  is a precision parameter
representing the radius of the tube located around f ( x) . Then, the SVR problem can
be regarded as the solution to
minimize
l
1 2
w  C (i  i* )
2
i 1

 yi  w   ( xi )  bi    i , i  0

subject to  w   ( xi )  bi  yi    i* , i*  0
i  1,..., l

(1)
where  ( x) is the high dimensional feature space that is nonlinearly mapped from
the input space R n . The parameter C  0 is a regularized constant determining the
trade-off between the empirical error and the regularization term.  i is the upper
training error ( i* is lower), subject to the  -insensitive tube y  (w   ( x)  b)   .
SVR fits f ( x) to the data by minimizing the empirical error (  i and  i* ) and the
regularization term
1 2
w . The regularization term is a measure for the model’s
2
complexity, and can be interpreted as a measure for the flatness of the regression
hyperplane in feature space. By introducing the Lagrange multipliers  , * , a kernel
function K and exploiting the optimality constraints, the constrained optimization
problem in Eq. (1) can be solved and the values of multipliers  and  * can be
obtained. For detailed derivations the reader is referred to Vapnik (1995). The weight
vector w can be expressed as follows:
l
l
w   (i  i* )( j   *j ) K ( xi , x j )
(2)
i 1 j 1
An efficient wrapper approach called SVM-RFE was used to conduct product
form feature selection in this study. The selection criterion of SVM-RFE was
developed according to Optimal Brain Damage (OBD) which has proved to be better
than earlier methods (Rakotomamonjy, 2003). OBD was first proposed by LeCun et al.
(1990) and uses the change of the cost function as the feature selection criterion,
which is defined as the second order term in the Taylor series of the cost function:
1 2 L
(3)
( D f )2
f 2
2 ( )
in which L is the cost function of any learning machine, and  is the weight of
cf 
features. OBD uses c f to approximate the change in the cost function caused by
5
removing a given feature f by expanding the cost function in the Taylor series.
Therefore, for SVR, the measure of OBD can be considered as the removed feature
that has the least influence on the weight vector norm w
2
in Eq. (1). The ranking
criterion can be written as
c f  w  w(  f )
2

1
2
l
l
 (
i 1 j 1
l
l
i
 (
i 1 j 1
2
  i* )( j   *j )K ( xi , x j ) 
( f )
i
(4)
  i*(  f ) )( (j  f )   *(j  f ) ) K (  f ) ( xi , x j )
where  and  * is the corresponding solution of Eq. (1), the notation  f means
that the feature f has been removed, and K is the kernel function calculated using
xi and x j . SVM-RFE starts with all the features. At each step, feature weights are
obtained by comparing the training samples with the existing features. Then, the
feature with the minimum c f is removed. This procedure continues until all features
are ranked according to the removed order.
2.3. Feature selection using MLR with the stepwise procedure
MLR is a traditional statistical method for modeling the linear relationship
between the input vector (independent variables) xi  R n and a desired output value
(dependent variable) yi  R . The regression equation of MLR constructed with l
data samples is
yi  b0  b1 xi ,1  ...  bn xi ,n  ei , i  1,..., l
(5)
where b0 is the regression constant, b1 ,..., bn are the partial regression coefficients
corresponding to the n input variables, and ei is the error term. The coefficients of
MLR are often estimated using ordinary least squares (OLS) procedure such that the
l
sum of the squared errors (  ei ) is minimized. The coefficient of determination, R2 ,
i 1
indicates the percentage of the variation in yi explained by the independent variables
xi . The value of R2 (ranged from 0 to 1) determines the goodness of fit for the
6
regression model. The coefficients b1 ,..., bn in Eq. (5), which are also denoted as
unstandardized regression coefficients, can be used to construct the regression
equation directly. However, the standardized regression coefficients 1 ,..., n , instead
of unstandardized regression coefficients, are more suitable for analyzing the
importance of variables. Each standardized regression coefficient represents the
change in response per standard deviation change in one output value y . Since the
input variables are often highly correlated with one another, the variance inflation
factor (VIF) can be used to assess the problem of multicollinearity.
For the task of product form feature selection in this study, MLR is combined
with the stepwise procedure to select significant independent variables based on the
statistic of F-test. In the stepwise procedure a variable that added into the model in the
earlier stage of selection may be deleted at the later stages. At each step, the
independent variable not in the model which has the smallest probability of F is added,
and the probability is sufficiently small (say 0.05). On the other hand, the variables
already included in the model are removed if the probability of F is sufficiently large
(say 0.1). In fact, the inclusion and exclusion of variables in the stepwise procedure is
the same as forward selection and backward elimination.
3. The proposed product form feature selection model
This study aims to construct a product form feature selection model based on
CARs and NSDA. First, NSDA is used to generate an explicit numerical definition of
product form design. Next, CARs are described using single adjectives as affective
dimensions. The representative affective dimensions are extracted using factor
analysis. In order to gather the evaluation data of consumers, an SD experiment is
conducted using questionnaire investigation. The SVR prediction models according to
the representative adjectives can be constructed using the form features of NSDA as
input data and the evaluated SD scores as output. Optimal training parameters of the
SVR model are determined using a grid search with leave-one-out cross validation
(LOOCV). In order to streamline the selection of critical product form features,
SVM-RFE is capable to identify the smallest feature subset with a given predictive
performance. Finally, MLR with the stepwise procedure is adopted to analyze the
importance of the form features and to interpret their effects to the designated CARs.
3.1. Preparing numerical definition-based product form samples
A numerical definition-based form representation for generic knife form, which
was previously proposed by Chang and Chen (2008), was adopted in this study. The
7
development of this form representation was based on an extensive investigation of
commercially-available knives. As shown in Fig. 1, the main structure of the knife
form was defined using three NURBS curves and 15 control points, namely the blade
profile (P1~P7), the lower edge of the handle (P8~P12), and the shape of the
tail-section (P13~P15). These NURBS curves were used to construct a extrude
surface with a fixed height for the knife blade. Note that control point p1 , located at
the intersection of the two blade curves, is specified as the origin (0, 0, 0) . As for the
modeling of the knife handle, it can be achieved by manipulating six adjustable design
elements, namely a centric curve to define the contour of the lower edge of the handle,
the width-to-height proportion, the proportional scale of the sectional area, the
rounded radius parameter, the sectional inclined angle, and a tail arc curve. In addition,
five sectional contours were uniformly distributed along the axis of the knife handle
and construct a loft surface. The resulting 39 form features for the numerical
definition of the knife form are listed in Table 1. A total of 43 knife model were
created using 3D software by three experienced product designers. The corresponding
form features for each 3D model were recorded. In order to reduce the perceived
differences caused by the materials and textures of the 3D model, the blade was
assigned with a gray material and the handle a black material. All models were
rendered under the same lighting and camera settings.
< Insert Fig. 1 about here >
< Insert Table 1 about here >
3.2. Extracting representative affective dimensions
In product design research, adjectives are often used to describe CARs. Prior to
conducting the adjective evaluation experiment, it is important to reduce the number
of adjectives and extract the most representative ones to allow consumers to describe
their subjective perceptions. By doing so, not only the phenomena of relevancy,
dependency, redundancy, cause/effect, and similarity existing among adjectives can be
reduced, but the experimental results of questionnaire investigation can also be
improved with better mental consistency for the subjects. 31 candidate adjectives,
which can be used to describe the affective responses toward knife design, were
collected in a previous study (Chen and Chang, 2009). These adjectives were used as
initial affective dimensions for consumers to evaluate a small amount of
representative knife samples in an SD experiment. The evaluated SD data of the initial
adjectives were then analyzed using factor analysis. The extracted three factors
8
account for 37.5%, 32.9% and 21.5% of explained-variance, respectively. For the task
to select representative adjectives, a simple and intuitive way is to pick up the
adjectives with the largest absolute factor loadings in each factor. Three extracted
adjectives including elegant, modern, and individualized were used in the following
adjective evaluation experiment. For detailed reports to extract these adjectives for
knife design the authors refer to the article of Chen and Chang (2009).
3.3. Questionnaire investigation for adjective evaluation
In order to collect the CAR data for knife design, 56 subjects (31 male and 25
female) were asked to evaluate each knife using a 9-point Likert scale using three
representative adjectives. The rendered images of the knife samples were presented to
the subjects in the format of questionnaires. In order to collect the evaluation data in a
more effective way, a user-friendly interface for the questionnaire was provided (see
Fig. 2). The images of the knife samples are displayed in the interface. The
presentation order of the knife samples was randomized to avoid any systematic
effects. After each section of the experiment, the evaluation data of each subject can
be recorded directly, thus simplifying the data post-processing time. The adjective
data was normalized to the range [0, 1]. All the subjects’ evaluation scores toward
each knife were averaged to reach a final utility score. These scores were applied as
the output values in the SVR and MLR models.
< Insert Fig. 2 about here >
3.4. Constructing the SVR prediction model
SVR was used to construct a prediction model based on the form features of the
knife samples and the average CAR ratings obtained from all the respondents. Since
SVR can only deal with one output value at a time, each prediction model need to be
constructed according to different adjectives. The training scheme of a single
prediction model based on SVR is depicted in Fig. 3. First, the knife samples consist
of a series of input vectors are fed into the training model. These input vectors are
mapped into feature space by a map function  . The mapping  is usually
nonlinear and unknown. Instead of calculating  , the kernel function K is used to
compute the inner product of two vectors xi and x j in the feature space  ( xi ) and
 ( x j ) , that is, K ( xi , x j )   ( xi )   ( x j ) . SVR is also known as its elegance in solving
9
the nonlinear problem with the kernel functions that automatically do a nonlinear
mapping to a feature space. Any function satisfying Mercer’s condition can be used as
the kernel function (Vapnik, 1995). Of the commonly used kernel functions, the
Gaussian kernel is favored for many applications due to its good features (Wang et al.,
2003); and thus was adapted as follows:
Gaussian kernel: K ( xi , x j )  exp( xi  x j
2
/ 2 2 ) ,
where  is the spread width parameter determining the influence of squared
distance between xi and x j to the kernel value. Using Gaussian kernel, dot
products are computed with the output values of the training samples under the map
 . The weights in the SVR represent the knowledge acquired from the knife samples.
Finally, the dot products are added up using the weights in terms of the Lagrange
multipliers,  i and  i* . This, plus a constant term b yields the final predictive
output value as follows:
l
y  f ( x)   ( i   i* ) K ( x, xi )  b .
i 1
The following procedure was used to train the knife samples:
(1) Transform data to the format of SVR;
(2) Normalize the form features linearly to the range [0, 1];
(3) Conduct a grid search with LOOCV to optimize the best parameter C ,  and
the spread width  of Gaussian kernel;
(4) Use the best parameter set to train the whole training set.
<Insert Fig. 3 about here>
3.5. Selecting critical form features using SVM-RFE
The SVM-RFE process was conducted based on the optimal SVR model using
the parameters obtained from a grid search, thus enabling critical product form
features according to each affective dimension to be selected. The relative importance
of the form features can be identified by analyzing the weight distribution in each
iterative step of the SVM-RFE process. The ranking criterion c f of SVR was
computing using Eq. (4). Since the SVM-RFE is a wrapper approach, the SVR model
was retrained and the weight values of features were updated during each iterative
step. Consequently, product designers can not only obtain the information about the
10
feature ranking but also that on the relative importance of the form features in each
iterative step of the SVM-RFE process. The complete SVM-RFE procedure for
product form feature selection, modified from Shieh and Yang (2008b), is described
as follows:
(1) Start with an empty ranked features list R  [] and the selected feature list
F  [1,..., n] ;
(2) Repeat until all features are ranked:
(a) Train SVR with all the training samples, with all features in F ;
(b) Compute the ranking criterion c f of SVR for features in F using Eq. (4);
(c) Find the feature with the smallest ranking criterion: a  arg min f c f ;
(d) Add the feature a into the ranked feature list R : R  [a, R] ;
(e) Remove the feature a from the selected feature list F : F  F  [a] ;
(3) Output: Ranked feature list R .
3.6. Analyzing the importance of form features using MLR
4. Experimental results
4.1. Results of form feature selection using SVR with SVM-RFE
4.1.1. Analysis of feature ranking
Table 2 shows the results of the feature ranking obtained by SVM-RFE. For the
sake of clarity, only the top 12 selected form features for the three adjectives are listed.
Also, if a feature from v1~ v30 is selected, the feature name is assigned with an
additional subscript x or y to indicate the direction of the NURBS control points
explicitly. As shown in Table 2(a), for the adjective “elegant”, there are 9 features
belong to the knife handle except for the features v8 y (6th), v14 y (11th) and
v12 y (12th). Another characteristic for the results of “elegant” is that the dominant
dimension of the NURBS control points is the y axis. According to the top 5
selected features a design suggestion for producing an “elegant” knife can be made. A
designer can first manipulate the features v18 y (3rd), v 20 y (1st) and v 26 y (4th)
to determine the main form of the knife handle; then adjust the features v31 (2nd)
and v32 (5th) to refine the knife form by tuning the inclined angle of the fore- and
tail- sections. The feature ranking of the adjective “modern” is shown in Table 2(b).
11
The top 6 selected features include v6 y (5th), v7 x (1st), v9 x (3rd), and v14 y
(4th) on the control points of the knife blade, plus v 26 y (6th) and the inclined angle
v32 (2nd) of the knife handle. Note that the two features v 26 y and v32 are
important for both the adjectives “elegant” and “modern”. The feature ranking of the
adjective “individualized” is shown in Table 2(c). The obtained feature ranking using
SVM-RFE is very helpful to product designers when they are determining the
importance of the form features while designing a knife.
< Insert Table 2 about here >
4.1.2. Predictive performance of selected feature subset
As described in the previous section, the feature ranking can be obtained from
the process of SVM-RFE. Since the number of form features is large, it is vital to
select the more important features according to the predictive performance of the SVR
model. During the elimination steps, each feature is screened out one at a time
according to the criterion c f . Therefore, the features which have less influence on the
prediction results will be eliminated in the earlier steps than those which have more
influence. Fig. 4 shows the predictive performance, measured in the root mean
squared error (RMSE), of the selected feature subset for the three adjectives. In the
beginning of the process before removing any feature, the SVR model performs best
with the lowest RMSE for the three adjectives. During the following steps, the
tendency of the RMSE increased gradually until a certain critical step, for example,
the 34th step of the adjective “elegant” (drawn in red line). Product designers can
select the smallest feature subset with a given predictive performance. If the allowable
RMSE for the adjective “elegant” is 0.08, the top 5 features can be selected without
deducting the performance significantly. It also seems to be a suitable threshold of the
RMSE using the value of 0.08 for the adjective “modern” (drawn in green line).
According to this threshold, the top 6 feature can be selected. As for the adjective
“individualized” (drawn in blue line), the threshold is difficult to determine since the
increase of the RMSE is not obvious.
< Insert Fig. 4 about here >
4.2. Results of form feature selection using MLR with the stepwise procedure
4.2.1. Selected features by the stepwise procedure
12
The results of the selected form features using MLR with the stepwise procedure
are shown in Table 3. The selection criterion of the form features is determined by the
probability of F in the stepwise procedure. For the adjective “elegant”, 11 features are
selected including v13x , v19 x , v 20 y , v23x , v25x , v27 x , v31 , v33 , v34 ,
v35 , and v38 . It can be observed that the features selected by MLR are very
different from the top 12 features selected by SVM-RFE (see Table 2[a]). More
specifically, there are only two selected features are identical ( v 20 y and v31 ). The
results for the other two adjective, modern and individualized, also exhibit the same
phenomenon. This difference is mainly due to the linear or nonlinear nature of these
two methods. MLR, as a linear model, does not have good predictive performance.
The coefficients of determination ( R2 ) for the three adjectives are 0.758, 0.685, and
0.759, respectively. Another reason for the unsatisfactory predictive performance of
MLR is that the number of knife samples (43) is not sufficient enough for a relative
larger number of features (39), that is, suffered from the problem of “curse of
dimensionality”. Nevertheless, the unstandardized regression coefficients still can be
used to construct the regression equation, as a simplified model neglecting the
underlying nonlinear relationships between the form features. The problem of
multicollinearity is not found in the models of the three adjectives since the VIF
values of all the selected features are less than 10, following to the suggestion of Hair
et al. (1995).
< Insert Table 3 about here >
4.2.2. Effects of form features for inducing affective responses
The effects of the form features for inducing specific CAR of knife design can be
interpreted by the standardized regression coefficients shown in Table 3. Recall that
each standardized coefficient represents the change of a form feature in response per
standard deviation change in the value of CAR. Moreover, a positive value indicates
the change of the feature and the CAR value are in the same direction, vice versa. For
the adjective “elegant”, for instance, the selected feature v13x , which represents the
x -axis coordinate of the NURBS control point P7 , has a positive value 0.379 for the
standardized coefficient. To explain the effects of the features in a graphical manner,
as shown in Fig. 5, if P7 on the curve of the blade edge was translated along the
x -axis for 1 standard deviation distance, it can produce a more “elegant” feeling in a
magnitude of 0.379. Similarly, a designer can consider the following suggestions:
translating the control point P10 ( v 20 y ) along the y -axis; rotating the fore-section
13
( v31 ) in a clockwise direction; reducing the proportional scale of the 2nd section
shape ( v34 ); reducing the width-to-height proportion ( v38 ). By interpreting the
standardized regression coefficients of MLR, not only the critical form features can be
identified but it also provides direct suggestions on how to produce a specific
affective response. However, since this analysis is based on a linear model, the joint
effects for adjusting these features at the same time need to be further examined.
< Insert Fig. 5 about here >
5. Conclusions
In the product design field, selecting critical form features and analyzing their
effects for inducing designated affective responses are very helpful to produce
successful products. However, there are only sparse literatures existing for this
problem. In this study, a product form feature selection model based on CARs and
NSDA is proposed. NSDA was employed to generate an explicit numerical definition
for product form design and CARs were described using single adjectives. The
adjective evaluation can be gathered by the SD experiment. Two feature selection
methods, namely SVR with SVM-RFE and MLR with the stepwise procedure, were
used to analyze the form features using knife design as an example. The SVM-RFE
process, as a state-of-the-art machine learning approach, is capable to obtain the form
feature ranking and select a subset of features with a given performance of the SVR
model. MLR combined with the stepwise procedure, as a traditional statistical method,
can be used to select significant form features based on the F-test. Further analysis
can be made using the statistics such as the regression coefficients and the VIF values.
The standardized regression coefficients also provide a useful means how to produce
a specific affective response. The proposed method for product form feature selection
provides product designers a potential tool for systematically studying the form
features and their effects on affective responses.
References
Chang, Y. M. and Chen, H. Y. (2008). Application of novel numerical definition-based
systematic approach (NDSA) to the design of knife forms. Journal of the
Chinese Institute of Industrial Engineers, 25(2), 148-161.
Chen, H. Y. and Chang, Y. M. (2009). Extraction of product form features critical to
determining consumers' perceptions of product image using a numerical
definition-based systematic approach. International Journal of Industrial
14
Ergonomics, 39(1), 133-145.
Guyon, I., Weston, J., Barnhill, S. and Bapnik, V. (2002). Gene selection for cancer
classification using support vector machines. Machine Learning, 46(1-3),
389-422.
Hair, J., Anderson, R., Tatham, R. and Black, W. (1995). Multivariate data with
readings. US America: Prentice Hall Inc.
Han, S. H. and Hong, S. W. (2003). A systematic approach for coupling user
satisfaction with product design. Ergonomics, 46(13/14), 1441-1461.
Han, S. H. and Kim, J. (2003). A comparison of screening methods: selecting
important design variables for modeling product usability. International
Journal of Industrial Ergonomics, 32, 189-198.
Han, S. H. and Yang, H. (2004). Screening important design variables for building a
usability model. International Journal of Industrial Ergonomics, 33, 159-171.
Hsiao, S. W. and Huang, H. C. (2002). A neural network based approach for product
form design. Design studies, 23, 67-84.
Hsiao, S. W. and Tsai, H. C. (2005). Applying a hybrid approach based on fuzzy
neural network and genetic algorithm to product form design. International
Journal of Industrial Ergonomics, 35, 411-428.
Jensen, R. (2005). Combining rough and fuzzy sets for feature selection. School of
Informatics, University of Edinburgh. Doctor of Philosophy.
Jindo, T., Hirasago, K. and Nagamachi, M. (1995). Development of a design support
system for office chairs using 3-D graphics. International Journal of Industrial
Ergonomics, 15, 49-62.
Jones, J. C. (1992). Design Methods. New York, Van Nostrand Reinhold.
Juang, B. H. and Katagiri, S. (1992). Discriminative learning for minimum error
classification. IEEE Transactions of Signal Process, 40(12), 3043-3054.
Kohavi, R. and John, G. (1997). Wrapper for feature subset selection. Artificial
Intelligence, 97(1-2), 273-324.
Lai, H. H., Lin, Y. C., Yeh, C. H. and Wei, C. H. (2006). User-oriented design for the
optimal combination on product design. International Journal of Production
Economics, 100, 253-267.
LeCun, Y., Denker, J., Solla, S., Howard, R. and Jackel, L. D. (1990). Optimal brain
damages. Advances in Neural Information Processing Systems. D. S.
Touretzky. CA: Mateo, Morgan Kaufmann. 2: 598-605.
Liu, Y. and Zheng, Y. F. (2006). FS_SFS: A novel feature selection method for support
vector machines. Pattern Recognition, 39, 1333-1345.
Pal, S. K., Basak, J. and De, R. K. (1996). Feature selection: a neuro-fuzzy approach.
IEEE International Conference on Neural Networks, Washington, DC.
15
Park, J. and Han, S. H. (2004). A fuzzy rule-based approach to modeling affective
user satisfaction towards office chair design. International Journal of Industrial
Ergonomics, 34, 31-47.
Rakotomamonjy, A. (2003). Variable selection using SVM-based criteria. Journal of
Machine Learning Research, 3, 1357-1370.
Shieh, M. D. and Yang, C. C. (2008b). Multiclass SVM-RFE for product form feature
selection. Expert Systems with Applications, 35(1-2), 531-541.
Tsai, H. C. and Chou, J. R. (2007). Automatic design support and image evaluation of
two-colored products using color association and color harmony scales and
genetic algorithm. Computer-Aided Design, 39(9), 818-828.
Vapnik, V. N. (1995). The nature of statistical learning theory. New York, Springer.
Wakaki, T., Itakura, H. and Tamura, M. (2004). Rough set-aided feature selection for
automatic web-page classification. Proceedings of the IEEE/WIC/ACM
International Conference on Web Intelligence.
Wang, W., Xu, Z., Lu, W. and Zhang, X. (2003). Determination of the spread
parameter in the Gaussian kernel for classification and regression.
Neurocomputing, 55, 643-663.
Yang, C. C. and Shieh, M. D. (2010). A support vector regression based prediction
model of affective responses for product form design. Computers & Industrial
Engineering, doi:10.1016/j.cie.2010.07.019.
16
Fig. 1. The numerical definition of generic knife form.
Origin (0,0)
P2 (v3x, v4y)
P3 (v5x, v6y)
P4 (v7x, v8y)
P9 (v17x, v18y)
P1 (v1x, v2y)
P11 (v21x, v22y)
p8 (v15x, v16y)
P13 (v25x, v26y)
P5 (v9x, v10y)
P10 (v19x, v20y)
P7 (v13x, v14y)
P6 (v11x, v12y)
P12 (v23x, v24y)
P14 (v27x, v28y)
P15 (v29x, v30y)
The fore-section shape (v33)
and its inclined angle (v31)
3rd section shape
(v35)
2nd section shape
(v34)
The tail-section shape (v37)
and its inclined angle (v32)
4th section shape
(v36)
Proportion of width and height (v38); Rounded size (v39)
17
Fig. 2. An interface of questionnaire for adjective evaluation.
18
Fig. 3. Training scheme of the SVR model.
Input
vectors
Mapped
vectors
x1
 ( x1 )
x2
 ( x2 )
Kernel
function
Weights
K ( x, x1 )
K ( x, x2 )
1  1*
 2   2*
y
...
...
...
xl
 ( xl )
K ( x, xl )
19
 l   l*
predictive
output
Fig. 4. Performance of feature subset (RMSE) of the three adjectives for SVM-RFE: y1 (elegant), y 2 (modern), and y3 (individualized).
20
Fig. 5. Effects of the form features in creating an “elegant” feeling.
Reducing the width-to-height proportion (v38)
creates a more “elegant” feeling.
Reducing the proportional scale of the 2nd section
(v34) creates a more “elegant” feeling.
Curve of blade back
Translating control point P10 (v20y)
in the upward direction creates a
more “elegant” feeling.
(0, 0)
Rotating the fore-section (v31) in a clockwise
direction produces a more “elegant” feeling.
Curve of blade edge
Translating control point P7 (v13x) in the rightward direction to reduce the radian of the arc
curve of the blade edge (as shown by dotted line) produces a more “elegant” feeling.
21
Table 1. The list of form features for the numerical definition of the knife form.
Knife form decomposed descriptions
Feature specifications
Features
Curve of blade back
( x, y ) coordinate of 4 NURBS control points (P1~P4)
v1 ~ v8
Curve of blade edge
( x, y ) coordinate of 3 NURBS control points (P5~P7)
v9 ~ v14
Centric curve of knife handle
( x, y ) coordinate of 5 NURBS control points (P8~P12)
v15 ~ v24
Tail arc curve of knife handle
( x, y ) coordinate of 3 NURBS control points (P13~P15)
v25 ~ v30
Knife blade
Knife handle
Fore-section inclination of knife handle
Inclined angle (60°/90°/120°)
v31
Tail-section inclination of knife handle
Inclined angle (60°/90°/120°)
v32
Fore-section shape
Proportional scale (1/0.75/0.5)
v33
2nd section shape
Sectional
variation 3rd section shape
of knife
handle
4th section shape
Proportional scale (1/0.75/0.5)
v34
Proportional scale (1/0.75/0.5)
v35
Proportional scale (1/0.75/0.5)
v36
Tail-section shape
Proportional scale (1/0.75/0.5)
v37
Proportion of all sections
Width to height proportion (1/0.67/0.5)
v38
Rounded radius of all sections
Rounded radius (1mm/3mm/5mm)
v39
22
Table 2. Top 12 of feature ranking of the adjectives obtained by SVM-RFE: (a) elegant (b) modern and (c) individualized.
Elegant
(a)
Rank
1
2
3
4
5
6
7
8
9
10
11
12
(b)
Modern
(c)
Individualized
Feature
v 20 y
*b/h
Rank
Feature
*b/h
Rank
h
1
b
1
v31
v18 y
v 26 y
h
h
h
h
b
h
h
2
3
4
5
6
7
8
v7 x
v32
v9 x
v14 y
v6 y
v 26 y
h
b
b
b
h
h
h
2
3
4
5
6
7
8
h
h
b
b
9
10
11
12
h
h
h
h
9
10
11
12
v32
v8 y
v 28 y
v 24 y
v30 y
v 22 y
v14 y
v12 y
v29x
v39
v25x
v30 y
v33
v36
*b/h denotes that the form feature belongs to a knife blade or handle (b: blade; h: handle).
23
Feature
v30
v14 y
v36
v38
v37
v23x
v2 y
v1x
v17 x
v34
v35
v33
*b/h
h
b
h
h
h
h
b
b
h
h
h
h
Table 3. Regression coefficients of selected features obtained from MLR with the
stepwise procedure.
Form
features
Elegant
(Constant)
v13x
v19 x
v 20 y
v23x
v25x
v27 x
v31
v33
v34
v35
v38
Unstandardized
coefficients
B
Std. Error
4.380
0.034
0.640
0.033
0.102
0.009
0.489
0.011
-7.840
-0.077
0.041
Standardized
coefficients
Beta
－
Sig.
Collinearity
statistics
VIF
－
0.379
0.131
0.275
0.000
0.001
0.020
0.007
1.361
1.285
1.168
3.796
0.040
0.021
-0.226
-0.195
0.188
0.047
0.062
0.056
1.529
1.298
1.154
-0.009
1.714
-2.157
1.043
-2.941
0.005
0.536
0.556
0.537
0.536
-0.171
0.298
-0.382
0.180
-0.501
0.085
0.003
0.001
0.016
0.000
1.192
1.108
1.241
1.104
1.068
4.235
0.042
0.039
0.749
-0.747
-10.043
0.175
-1.918
0.122
0.010
0.017
0.590
0.423
4.436
0.047
0.659
－
－
0.439
0.264
0.142
-0.193
-0.268
0.416
-0.315
0.000
0.000
0.024
0.013
0.087
0.030
0.001
0.006
1.251
1.307
1.316
1.249
1.469
1.291
1.226
1.998
-2.376
0.642
0.651
0.316
-0.375
0.004
0.001
1.081
1.108
5.732
-0.008
-0.044
0.052
0.004
0.020
－
0.000
0.028
0.035
－
R 2  0.758
Modern
(Constant)
v13x
v14 y
v19 x
v21x
v23x
v29x
v34
v37
v38
R 2  0.685
Individualized
(Constant)
v3x
v9 x
-0.211
-0.243
24
1.705
1.454
v17 x
v19 x
v 20 y
v21x
v 24 y
v25x
v 28 y
v33
v34
v36
v38
4.549
1.829
0.289
0.039
1.630
-0.620
-0.021
0.666
-0.089
0.218
0.087
-0.790
0.492
0.475
0.428
0.262
0.007
0.206
0.027
0.034
0.023
0.301
0.282
0.269
0.287
-0.259
-0.356
0.378
-1.497
1.122
1.803
-0.280
0.178
0.168
0.149
0.025
0.007
0.003
0.003
0.000
0.001
0.014
0.032
0.059
0.047
1.446
1.776
1.638
2.488
1.747
1.566
1.367
1.247
1.096
1.193
R 2  0.759
25