Realistic Skin Movement for Character Animation Xiaosong Yang ,Jian J Zhang National Centre for Computer Animation The Media School, Bournemouth University Poole, Dorset, United Kingdom E-mail:xyang; jzhang @bournemouth.ac.uk Abstract— Skin movement modeling is a crucial element for realistic animation of both human and creature characters. In this paper, we are primarily concerned with a special skin movement effect, skin sliding. Although physical properties, such as skin elasticity, are usually regarded important in physical simulation, it often leads to an increasing computational load. A complete physical solution will also limit the animator’s ability to fine-tune the visual effects. In this paper we present a physically based skin sliding modeling method borrowing an idea from structural mechanics called bar networks. The advantage of using a bar network system in simulating the physical properties of skin surfaces is that only a set of sparse linear equations need to be solved, making the method much faster than a simulation based technique. As only one additional step is added to the animation pipeline to restore the skin elastic property, this algorithm is compatible with all existing skin deformation method such as smooth skinning, muscle base skin deformation, cluster and Free Form Deformations, and it therefore does not disturb the existing production pipeline.(Abstract) Keywords-Character animation, skin deformation I. INTRODUCTION Realistic skin movement has been paid a great deal of research efforts over the recent years. It is crucial for convincing character animation. In this paper, we concentrate mainly on the modeling of the skin sliding phenomenon, which, comparing with other skinning related deformation effects, has received relatively little attention in computer animation. Human and animal skin exhibits complex physical characteristics, making it more dynamic than kinematic. Skin becomes an extremity to the multiple layers of underlying anatomical structures (bones, muscles, fat). The movement of skin during body motion is highly dependent on the above said underlying anatomical structures thereby making it essential to delve into the physical aspects of the skin-sliding phenomenon. This is where physics based simulation approaches come into play. Incorporating physical properties of anatomic structures can potentially improve realism. Physics can be used at the muscle level, bones, fat or elastic skin layer. Some work on skin sliding has been carried out in this category using spring-mass systems to link the skin surface with the underlying anatomical structure. Spring networks usually do not consider the elasticity of the skin itself. One possible way to simulate that would be to replace the edges of the skin mesh also with springs. Though this would general a good effect of skin slide, the large number c 978-1-4244-6349-7/10/$26.00 2010 IEEE of springs on the skin model would increase the computation significantly. And more importantly, as with all physical simulations, user control becomes an issue. It takes a great effort to tune the simulation to what the animator desires and if the simulation engine requires tweaking, that adds an additional overhead. Skin sliding can be essentially thought of as a threedimensional problem with two-dimensional attributes. The skin slides over a surface (muscle, bone). With relation to the surface, the movement of the skin is parallel to the surface, with little or no perpendicular motion to the surface. In this paper we present a novel method to model skin slide. This method is largely motivated by the work presented in [1]. Bar-nets are useful in form finding as shown in [2] which uses a bar net driven system to perform skinning. By considering the skin as a sheet and reducing it to a 2D problem, it becomes very easy to take into consideration the physical properties of skin elasticity. In addition, our method can be easily integrated into the animation pipeline without any change to the traditional methods of skinning, giving the animator the freedom to control and design actively during the skinning phase. Attacking skin sliding as a 2D problem makes our method very efficient and fast. Please note we use the word ‘simulation’ throughout the paper, by which we really mean ‘physically-inspired modeling’. Our method does not rely on the use of accurate physical properties of the skin. The parameters of the barnets, can be adjusted to achieve the effect of different skin elasticity. This is left primarily to the user. This paper is structured as follows: After a brief introduction of the related work and our contribution, we present our five-step skin-sliding algorithm in detail. Then we provide experimental examples and analysis of the computation complexity. In the end, we discuss the limitations of our method and the possible improvements in future. II. RELATED WORK Skin deformation, or skinning, has time and again proven to be an indefatigable part of character animation. In the current scenario of computer animation, where realism is paramount, efficient and visually believable techniques of skin deformation are essential. An intuitive attempt to deform a character was involving a skeleton into skin deformation. This approach treats the skin as a shell that moves by an explicit function of the skeleton. Vertices of the skin are deformed by a weighted combination of the joint transformations of the character’s V4-203 skeleton[3-6]. Collectively, such methods are known as the Smooth Skinning. The joint-based system is popular owing to its interactivity and use of minimal animation data. There are basically two main approaches to modeling skin deformations, namely, physics-based approaches (also known as anatomy-based approach [7-8]) and example-based skinning [9-11]. Physics-based methods are based on the anatomy, elastic mechanics, or biomechanics of skin deformation originating from the movements of muscles and tendons. They are physically meaningful and have a capacity to generate more realistic simulation. In addition, physicsbased approaches usually use numerical calculations, which often incur heavy computation expenses. Skin sliding can be achieved by anchoring the skin using anchor springs and modeling restoring forces through energy functions when displacement of the skin occurs. Due to the heavy computation expense involved, this method is mainly used in high-quality film visual effects where anatomical accuracy is a must for believable computer generated characters. The example-based approach forms a suitable alternative where computational expenses are to be minimized. An artist models certain key poses of the characters. New poses are interpolated from these key poses. However because each example skin shape was modeling separately, it is impossible to realize a smooth sliding effect using example based method. Because of the heavy computation involved in the simulation of skin sliding effect, no commercial animation software provides this function until 2009. Maya 2009 in its muscle package gives the animator an option to paint the sliding weight in the muscle simulation. However what it does is only to push out the skin surface wherever there is collision between muscle and skin surface mesh. Some unpleasant effects such as distortion in the local areas may be generated with no consideration of the skin’s elastic property. Our method can work together with this “push-out” function to fix the problem and generate natural looking skin sliding effects above the muscles and bones. Some examples are given in the Experiment section. A. Our Contribution Our fast skin-sliding method is developed by adapting an idea from structural mechanics. Structural mechanics employs bar-net, which is basically a network of nodes whose connections consist of rigid bars. The shape of the network depends on the structural and material properties and also on the forces acting on it. In order for the skinning problem to be converted to the 2D realm from the 3D realm, we develop an unwrapping technique, which can flatten the skin using the network shape finding technique [1]. With this enhanced unwrapping method, we map a skin patch to a 2D plane. The process is made faster due to the fact that only a set of sparse linear equations need to be solved, once. To speed up sliding simulation, we present a 2D image based look-up table to do the point-in-triangle test. For each vertex in the sliding area, a small set of triangles is selected to perform the test and compute the barycentric coordinate. This reduces the computation complexity to O(n). V4-204 Our skin sliding simulation is compatible with any kinds of skin deformation method, irrespective of the type of skinning methods (smooth skinning, muscle base skin deformation or physical based skinning method). Only an extra step is added to the deformed shape to restore the elasticity of the skin surface. And since the overall shape remains the same even after the skin sliding, the animator’s creative work is well protected. III. SKIN SLIDING THROUGH STRUCTURAL MECHANICS In this section, we give a brief overview of the structural mechanics based skin sliding simulation. The entire skin surface of the character needs not be considered for skin sliding computation – only the area with muscle deformation is required. So, initially, the animator needs to mark the area of the skin surface (Figure 1) to be used in the skin sliding operation. At the binding pose, that is, during the pre-animation phase we implement a bar-net based unwrapping method to map the 3D mesh to 2D square. This method is applied to each skin patch undergoing sliding. The 2D bar net is created as a pre-processing step to encode the physical properties of the original skin surface. The animator can then use various skin deformation methods including smooth skinning, wire deformer, cluster of freeform deformation to animate the character. Using the same unwrap method, the deformed skin surface in each key frame is mapped to the same 2D square. By comparing these two 2D Bar-nets (before and after skin deformation), the skin sliding can be generated on the deformed skin surfaces. The workflow of this skin-sliding method is shown in Figure 2. Our sliding methodology works on the key frames of the skin deformation rather than the deformation method itself. This helps in keeping the sliding technique to be independent of the skin deformation thereby making it compatible with all existing skinning methods. Figure 1. Select face patches on the skin surface to simulate skin sliding. Figure 2. Work flow of the skin sliding algorithm. 2010 2nd International Conference on Computer Engineering and Technology [Volume 4] r r r x = D −1 ( p x − D f x f ), r r r y = D −1 ( p y − D f y f ), r r r z = D −1 ( p z − D f z f ), (1) D = C tQC and Df = C tQCf Where , Q is the diagonal matrix (qii), where qii is the force density of the bar i. r r r px , py , pz (a) (b) Figure 3. Unwrapping of skin patch. (a) Original skin mesh of the front part of the face with only one boundary loop. (b) Unwrapping of the mesh. Please see last page of this document for AN EXAMPLE of a 2-COLUMN Figure. A. Bar Network First, A bar-net or force density mechanical network [1] is a structure commonly used in structural engineering. Its shape depends on the structural and material properties and the forces acting upon it. A bar-network connects ns points, in three-dimensional space with straight-line segments, called bars. These points on the net are known as nodes. The nodes can be either fixed or free. Fixed nodes will not have their positions changed regardless of whether they are subject to external forces or not. Free nodes can be moved to balance the acting forces on the net. Each bar connects two nodes. These bars can be stretched and squashed resulting in the repositioning of the end nodes, but they remain topologically static that is they retain their bar shape. Upon external forces acting on free nodes, the bar network may achieve different shapes. The final shape represents the rest shape of the network and is the result of the balance of all external and internal forces. The network described above is in fact a graph with links connecting pairs of nodes. Assuming there are ns nodes (n free nodes and nf fixed nodes) and m bars in the network. A matrix Cs(m*ns) called the branch-node matrix can be formed, which represents in a tabular form the graph of the network. For each row which represents a bar, the elements on the column i and j which are the two linked nodes indices, will be set 1 and -1, all others will be set to 0. This branchnode matrix can be further subdivided into two sub-matrices, C and Cf, by grouping the free-node columns and fixed-node columns of the original matrix respectively. These matrices are used in computing the rest shape of a bar-net. In our case, the effect of stiffness of a network can be approximated by the quantity of force-length ratios of all the bars. Some researchers call this quantity the force density. We will use this parameter to simulate the elasticity of the skin surface. An added advantage is that it is also able to solve the form finding problem with a set of sparse linear equations. Equation (1) shows how to compute the coordinates of the free nodes at the equilibrium state: [Volume 4] are the external force vectors. Therefore, with the given interconnection, the force density vector, the external forces and the coordinates of the fixed nodes, the positioning of the free nodes is determined by the equilibrium of the forces. B. Finding the Mapping between the 3D Mesh and a 2D Square The crucial part of our method is to find a mapping between a 3D mesh to the 2D square. Before we get into the detail of our unwrapping method, we need to build up the correspondence between these two meshes. At the binding pose, the animation needs to mark out where on the skin surface he/she expected to get the sliding effect by selecting patches on the mesh. Sliding won’t across the boundary of each patch, thus each patch’s sliding can be handled separately. The topology of a patch might be very complicated. But it is always possible to construct a mapping from a 3D patch to a simple square on the 2D plane. This means there is always a cutting method which can unwrap the patch into a one piece on a 2D plane (notice distortion is permitted). The first thing needing to do is to find out the complexity of each patch, i.e. the number of boundary loops. Normally there will be at least one boundary from the patch. Find the longest boundary Bi which will be mapped to the 2D square’s four edges. All other holes or boundaries will be mapped onto the inside area of the 2D square. The reason for choosing the longest boundary is to minimize the distortion during the unwrapping. On the boundary Bi we can automatically choose four points which are evenly distributed along the edge loops. These four points will be mapped to the four corners of the 2D square. All vertices on the boundary loop will be mapped to the four edges of the square. C. Unwrapping from 3D to 2D There are many different unwrapping methods to map a 3D mesh to a 2D plane for the purpose of texture UV definition. Based on the target they are pursuing, they can be categorized into area, angle and edge-length preserving methods [12]. In our skin sliding simulation, the unwrapping is merely used to find the correspondence between the original mesh and the deformed mesh. Hence for that reason, the quality of the pelting is not of high significance. And precisely for that reason, we had to devise a fast and stable unwrapping method as opposed to a high quality pelting method, which maps the two skin patches (before and after skin deformation) in the exact same manner. If the selected skin patch has more than one boundary loops, all the vertices on other boundaries (except Bi) will set 2010 2nd International Conference on Computer Engineering and Technology V4-205 to free at this first unwrapping step. The vertices on Bi are fixed during the bar-net deformation while leaving all the vertices inside to freely settle inside the square. From the topology linkage information of the bars and the nodes within the network, we can define C and Cf. In order to keep the topology information of the 3D skin patch and keep the consistency between this mapping and mapping after deformation, we define the force density qi of edge i according to its original length Len(ei). Len(ei ) (2) q =1− i Max ( Len(e j ), j = 1..m) where m is the number of edges involved in this patch. The external force vectors, px = py = pz = 0, so all the free nodes will stay on the 2D plane and kept inside the 2D square. The final equilibrium state of this bar-net will be the 2D mapping of the 3D skin patch. Fig. 3a shows an example of the unwrapping of front part of a human face which has only one boundary edge loop, Fig. 3b shows an example of unwrapping of a mesh with five boundaries, the neck boundary was used to be mapped to the edges of the 2D square. The reasons that we choose bar network to make the unwrapping are: (1) Bar network based unwrapping will not create fold which must be guaranteed to find the correspondence between the two unwrapped meshes. For each free node, the forces acting on it should be balanced. If a node is folded, all edges will be lying on one side, so does the accumulated force loads. It is definitely not an equilibrium state. (2) The spring based pelting solution described in [13] makes use of springs, requiring numerical solution to solve the force equations. Our bar-net based solution needs to solve only a set of sparse linear equations and hence is faster by a great magnitude. D. Skin Deformation Skin deformation is crucial for skin sliding to work. As stated earlier, it is the outcome of the deformation that is significant and not the technique used to achieve the same. Regardless of the type of skin deformation used (FFD, smooth skinning, muscle based skinning or physical simulation), the inputs to our skin sliding method are the original mesh and the deformed mesh. E. Mapping the deformed skin to 2D After skin deformation using the traditional methods, we are left with a deformed skin patch, which corresponds, to the original patch prior to deformation. Topologically the skin patch remains the same, with only the positions of the vertices modified to conform to the new deformed shape. Using the same mapping method, the deformed 3D skin patch can be mapped to the 2D plane. The bar-node connection information is still the same with the first bind pose mapping, however we need to change the force density between the nodes based on the new lengths of the edges. And if the sliding patch has more than one boundary, all inside boundary vertices need to be set to fix and keep their position in the first binding pose mapping on the 2D plane. This is to make sure no sliding happened across boundaries such as hole like eyes which will change the appearance of V4-206 animator’s modeling. By solving a set of sparse linear equations, the new bar-net can be settled in no time. We choose the Indexed Storage of Sparse Matrics to save the memory from the big matrix. And conjugate gradient method was used to iteratively get the solution. Since in the sliding simulation, the displacement of vertices are normally very small, we use the positions of the bar network at the bind pose as the initial guess, so only a very few iterations are needed to achieve the desired convergence tolerance. Normally less than 15 iterations are enough for our experiments. Figure 4. Two Bar-Nets ((a) is the unwrapping result for the original skin patch, (b) is the deformed skin patch). F. Generating the 3D Skin Slide Effect from the 2D Bar Network Suppose the original 3D mesh is M1, the deformed 3D mesh is M2, the Bar-net from the original mesh M1 is B1, and the Bar-net from the deformed mesh M2 is B2. In order to generate the 3D mesh after the skin slide process, two steps need to be performed, namely, computing the barycentric coordinates of the mapped vertex in the 2D plane and interpolation of the 3D vertex in the deformed mesh M2 using the barycentric coordinates to get the new position of the vertex. It has to be noted that if the bar-nets are not triangle meshes, then the polygons should be triangulated firstly. To find the corresponding triangle Tj in B2 for each vertex ui in B1, we have to perform a point-in-triangle test. Since it is time consuming to process all the triangles for the test, we present an image based look-up table to speed up the test. To facilitate this process, the 2D bar network B2 is rendered as a 1024*1024 color image I, where the color of each triangle is set to the triangle id in B2. For efficiency, if the number of triangles in the patch is very small, we can even use indexed image or grey image. The rendering of triangle meshes is already highly optimized in current graphics hardware allowing the creation of the 2D image lookup table at a quick rate. To find the triangles where ui lies, we can use the 2D coordinate (x,y) of ui to find the closest pixel and its neighbors which will totally cover ui. From these selected pixels, we can get the indexes of the triangles covering the point ui. In practice, the number of triangles involved depends on the linkage of the original bind pose mesh M1. The maximum number of the triangles though, is the maximum edges linked to one vertex in the mesh M1. In our experiment, for most vertices, only 1-2 triangles are picked to do the test. Though there are many ways to do the point-in-triangle test, we opted for the area based test, since we require the barycentric coordinates that can be easily calculated from the 2010 2nd International Conference on Computer Engineering and Technology [Volume 4] area of the triangles formed by the point and the vertices of the triangle. Area(ΔABC) < Area(ΔPBC) + Area(ΔAPC) + Area(ΔABP) If the above condition is satisfied, the point is definitely outside the triangle. If it is equal to the area of triangle ABC, then P is inside and the triangle is the correct one. Following through, the computation of the barycentric coordinate of P in the triangle ΔABC is ( a, b, c ) = ( Area (ΔPBC ) Area ( ΔAPC ) Area ( ΔABP ) , , ) Area (ΔABC ) Area ( ΔABC ) Area (ΔABC ) For each vertex P in the deformed mesh M2, its new position P’ is calculated by the following interpolation: P' = a × VA + b × VB + c × VC (4) VA, VB and VC are the coordinates in the mesh M2 for the vertex A, B and C. Using the 3D coordinates from M2, and the mesh configuration from M1, we successfully combine the deformed skin patch and original patch together. Thus we have developed an intuitive mechanism by which fast skin sliding is achieved by understanding the phenomena of sliding in relation to computer animated characters, rather than purely simulating it. IV. EXPERIMENTAL RESULTS In Appendix A, we demonstrate three applications of the proposed skin sliding method, the finger and elbow skin deformation and the facial animation. On a computer with an AMD Athlon 64x2 Dual Core Processor 4200+, 2.19GHz, 2G RAM, and NVidia GeForce 8500 GT, the absolute computation time required for the mapping and the creation of the lookup table is given below. It shows that our method gives a real-time performance for interactive control. TABLE II. Elbow Facial Animation needed. Following this only a simple linear interpolation was required to essentially compute the new slid mesh. In short we reduce the complexity of the whole algorithm to O(n), where n is the number of vertices involved in the skin sliding simulation. V. An efficient skin sliding technique has been developed for realistic animation of virtual human and animal characters. By considering the bind pose skin surface as the elastic target, we are able to simulate skin elasticity with good results. With our method, the animator can still use their familiar deformation method to achieve the result they expected. Our method basically adds an additional step after the deformation to restore the skin elasticity in order to achieve the skin sliding effect. In summary, our skin-slide method delivers the following advantages when compared to other methods: • With the bar-net form finding method, only sparse linear equations need to be solved, leading to fast computations. • It offers near real-time performance for artist interactivity. • Natural integration into standard industry animation pipeline is straightforward, as the sliding process is independent of skin deformation techniques. • It preserves the modeled skin shape during animation. The properties of the bar-net naturally prevent unwanted creases, wrinkles and folding from happening. We have implemented this technique into a prototype program in the form of a plug-in to the Autodesk Maya software. ACKNOWLEDGMENT COMPUTATION TIME OF THE EXPERIEMENTS Nod es Bars 51 316 90 593 Binding Pose Unwrapping (milliseconds)* Iteration time 15 3 44 53 Unwrapping during animation (milliseconds)* Iteration time 6 0 14 8 Index table creation (milliseconds)* 1 2 This research is in part supported by the GreatWestern Research Fellowship grant and EPSRC grant EP/F030355/1. We would also like to acknowledge FaceGen Modeller (http://www.facegen.com/modeller.htm), which was used to create the original facial models and animations. (* indicates the computation time of three of the most important steps in the skin sliding method) The algorithm is fast as it involves only some simple computations in the skin sliding simulation. The animator only needs to mark out some areas to define the skin patches involved in sliding simulation before animation. A pelting computation is then carried out on each patch. This requires the solution of a set of sparse linear equations, the dimension of which is the same as the number of free nodes (the vertices in this patch). During the animation, the bar network based unwrap computation is required again for the deformed patches, and also a rendering of the deformed twodimensional bar-net. However since the initial guess of the solution is very close to the answer, the unwrap of the deformed mesh is much faster than the pre-processing unwrapping of the bind pose mesh. Only very few iterations are needed to obtain the rest shape of the new bar network. For each vertex, normally one point-inside-triangle test was [Volume 4] CONCLUSIONS REFERENCES [1] [2] [3] [4] [5] [6] H.J. Schek, The force density method for form finding and computation of general networks, Computer Methods in Applied Mechanics and Engineering, vol. 3, 1974, pp. 115–134. J.J. Zhang, , X. Yang, , and Y. Zhao, Bar-net driven skinning for character animation, Computer Animation and Virtual Worlds, vol. 18, 2007, pp. 437-446. N. Magnenat-Thalmann, R. Laperrire, and D. Thalmann, Jointdependent local deformations for hand animation and object grasping, In Proceedings on Graphics interface, vol. 88, 1988, pp. 26-33. X.C. WANG and C. PHILLIPS, Multi-weight enveloping: leastsquares approximation techniques for skin animation, In SCA’02: Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation, 2002, pp. 129–138. D.L. James and C.D. Twigg, Skinning mesh animations, ACM Transactions on Graphics, vol. 24, 2005. L. Kavan, S. Collins, J. Žára, and C. O'Sullivan, Skinning with dual quaternions, In Proceedings of the 2007 Symposium on interactive 3D Graphics and Games (Seattle, Washington, 2010 2nd International Conference on Computer Engineering and Technology V4-207 April 30 - May 02, 2007). I3D '07. ACM, New York, NY, pp.39-46. [7] R. Turner and D. Thalmann, The Elastic surface layer model for animated character construction, In Proceedings of CG International, Lausanne, Switzerland, 1993, pp.399–412. [8] F. Scheepers, R.E. Parent, W.E. Carlson and S.F.May, AnatomyBased Modeling of the Human Musculature, proceedings of SIGGRAPH 97, Computer Graphics Proceedings, Annual Conference Series, August 1997, pp.163-172 [9] O. Weber, O. Sorkine, Y. Lipman and C. Gotsman, Context-aware skeletal shape deformation, Computer Graphics Forum, vol. 26, 2007, pp. 265-274. [10] X.C. Wang, and C. Phillips, Multi-weight enveloping: least squares approximation techniques for skin animation, In SCA’02: Proceedings of the 2002 ACM SIGGRAPH/Eurographics symposium on Computer animation, 2002, pp. 129-138. [11] J.P. Lewis, and M..Cordner, and N. Fong, Pose Space Deformation: a unified approach to shape interpolation and skeleton driven deformation, In SIGGRAPH 2000: Proceedings of the 27th annual conference on Computer graphics and interactive techniques, 2000, pp. 165-172. [12] M.S. Floater and K. Hormann, Surface Parameterization: a Tutorial and Survey, in Advances in Multiresolution for Geometric Modelling, N. A. Dodgson, M. S. Floater, and M. A. Sabin (eds.), SpringerVerlag, Heidelberg, 2005, pp. 157-186. [13] D. Piponi, and G. Borshukov, 2000. Seamless texture mapping of subdivision surfaces by model pelting and texture blending. In Proceedings of the 27th Annual Conference on Computer Graphics and interactive Techniques International Conference on Computer Graphics and Interactive Techniques. ACM Press/Addison-Wesley Publishing Co., New York, NY, 471-478. APPENDIX A. EXPERIMENT RESULT (i) (ii) (iii) Figure 5. (i) Shows the selection of area (faces) where skin sliding is to be performed. (ii) Skin sliding activated. The green mesh represents the slid skin. (iii) Shows the close-up of the skin sliding in action. a. Figure 6. V4-208 b. c. (d) (a) The bind pose mesh shown in green. (b) The area selected for the skin sliding to affect. (c) Face animated to the final pose. (d) Green mesh showing the effect of skin slide between the eyes. The effect is evident when compared with the original blue mesh underneath.. 2010 2nd International Conference on Computer Engineering and Technology [Volume 4]