Thursday, June 24, 2010

3D Avatar - animated model

Final Year Project Report - Chapter 10

10. Virtual 3D hand Modeling

10.1 3D Avatar– the Signing Human
To facilitate the communication from normal person to the Deaf/hearing impaired 3D avatar is used. It facilitates the vision based interpreting of sign language to the Deaf/hearing impaired. It also helps as a rapid [1] way of learning sign language and verbal language.
10.2              Bones and Joints of a Human Hand
The skeleton of a real hand is shown in Figure 10.1. There are total of 27 boned which are joined together and form the hand. The 21 Degree of Freedom [DOF] of the bone joints of the hand are controlled by 29 mussels [2] that enable us to grasp and hold objects properly.
A description...
Figure 10.1 Bones of the right hand (dorsal view) [4]

The constrains of the DOF points are utmost important , because it forms the dynamic behavior of the hand. As shown in Table 10.1, most joints have native angles of rotation.
Joint
Range X
Range Y
Range Z
CMC 1
-180 ~ 20
-180~ 0
-90 ~ 120
CMC 4
-10 ~ 0
0
0~5
CMC 5
-20 ~ 0
0
0 ~ 10
MCP 1
-90 ~ 0
0
0
MCP 2-5
-90 ~ 30
0
-20 ~ 20
PIP 2-5
-120 ~ 0
0
0
DIP 2-5
-80~0
0
0
IP(thumb)
-90 ~ 20
0
0
Table 10.1 DOF of the Joints in Degree (°) [3].
The human hands are complex articulated structures with multiple degrees of freedom. This makes the modeling and animation of high quality flexible virtual hands extremely difficult especially for real-time interactive applications. On this project we employ virtual hands for real-time Sign Language visualization for which they are of the utmost importance [5]. Figure 10.2 shows the skeleton of the 3D avatar used on this project which contains 16 bones including palm and 2 bones for arms with total of 18 bones.


C:\Users\chand\Desktop\HandSketch.jpg
Figure 10.2 Skeleton of the Project’s 3D Model
10.3              Modeling and Skinning
The 3D Object was modeled using 3D Studio Max (3dsMax9) as shown in Figure 10.3. It is easy to use, innovative and professional software for the modeling of 3-Dimensional characters [6] which provides a variety of virtual characters to be easily modeled by manipulating parameters through its graphical interface. One of the goals of modeling is to develop an anatomically correct model that has only the necessary number of vertexes and is optimized for animation.

D:\Education\Final Year Project\Jpgs\Desktop\3.JPG
Figure 10.3 Modeling on 3DsMax9
When modeling the 3dMax Objects, it is very important to name different parts of the models with relative names, so that it can be uniquely partitioned and identified on java. Figure 10.4 listed names does help to uniquely identify different parts of the full model on java 3D.
D:\Education\Final Year Project\Jpgs\Desktop\5.JPG
Figure 10.4 Named 3D objects on 3dsMax

After modeling, the mesh was exported as a wave front <.obj> file object format.
10.4. Java 3D
Java 3D is a scene graph-based 3D application programming interface (API) for the Java platform. It runs on top of either OpenGL or Direct3D. Since version 1.2, Java 3D has been developed under the Java Community Process [7] and freely available Open Source API
10.4.1. Java3D Initiations for Animation
There are few essential coding that required for java3D initiating with.
·       Simple Universe
private SimpleUniverse universe;
private GraphicsConfiguration config;
config = SimpleUniverse.getPreferredConfiguration();
·       Creating Java3D canvas
private Canvas3D canvas;
canvas = new Canvas3D(config);
·       Bounding Sphere
private Bounds influenceRegion;
·       Camera View Point
ViewingPlatform viewPlatform = universe.getViewingPlatform();
·       Setting Background
TextureLoader textureLoader =
new TextureLoader(bgPic, canvas);
Background background = new Background(textureLoader.getImage());
background.setApplicationBounds(influenceRegion);
·       Lighting
Color3f lightColor = new Color3f(Color.WHITE);
Vector3f lightDirection = new Vector3f(0.0f, 20f, -25f);
DirectionalLight light = new
DirectionalLight(lightColor,lightDirection);
light.setInfluencingBounds(influenceRegion);
10.4.2. Adding 3D Model to Canvas
The Wave Front ‘.obj’ file is loaded into java scene and Material for the object is added.3D model is scaled as needed.
TransformGroup model = getHuman(scene, influenceRegion);
Transform3D scaleModel = new Transform3D();
scaleModel.setScale(.8);

Color brown = new Color(255, 128, 64);
Appearance brownAppearance = getAppearance(brown);
namemap.get("p11").setAppearance(brownAppearance);

Separate parts of the 3D model are referenced in java as follows:
arm2t = new TransformGroup();
arm2t.addChild(namemap.get("arm2"));
10.4.3. Setting basic transformations to particles

Every particle of the loaded model need to place and rotate to make a meaningful model.Java3D provides X, Y, Z axis for making required adjustments as shown on Figure 10.5.

C:\Users\chand\Desktop\axis.jpg Figure 10.5 Java3d axis
One of the most difficult and essential part of the work is to transform and align the model particles on relevant reference places. There are two main points to set correctly for proper operation of the model.
·       Pivot point
o      Point of a particle where its rotation takes place around
·       Particle location
o      Position of the particle relates to its parent particle
If a particle misaligned, then the model will have deformations on when it’s real-time animating as shown in Figure 10.6.The extra sphere on the figure shows the current pivot point of the ‘arm1’ particle. To move the pivot point the elbow joint and place the particle on correct point use a Transform3Dobject and apply the matrix transformation.
C:\Users\chand\Desktop\pivotpointBad.jpg
Figure 10.6 Misplaced pivot point of ‘arm1’
modifyPartPivot(arm2x, arm2t, arm2, new Vector3f(0.218f, 0.347f, 0.340f), new Vector3f(-0.241f, -0.326f, -0.333f));
The 1st Vector3f provides the pivot point adjustment and the other with particle position adjustment.
Separated parts are joined in Java, using as parent and child nodes of TransformGroupObject type.
modifyPart(arm1, palm);

Finally all the particles are added to the root object, which is then rendered (compiled) and added to the Universe. Figure 10.7 shows the 3D model on Java3D jFrame.
root.addChild(background);
root.addChild(light);
human.addChild(body);
root.compile();
universe.addBranchGraph(root);

Figure 10.7 3D Avatar Model on Java3D
Finally, a set of public methods (Figure 10.8) are written to support external integration of the module with other modules. Rotation angle of each 18 joints on x, y, z, axis are set from external parameters at real-time.
temp.rotX((float) Math.toRadians((rotation - jointRotX)));
temp.rotY((float) Math.toRadians((rotation - jointRotY)));
temp.rotZ((float) Math.toRadians((rotation - jointRotZ)));
Figure 10.8 Public methods
Figure 10.9 through Figure 10.11 shows the rotation of hand on x, y, z, axis.
C:\Users\chand\Desktop\rotX.jpg
Figure 10.9 Rotation on X- axis
C:\Users\chand\Desktop\rotY.jpg
Figure 10.10 Rotation on Y- axis

C:\Users\chand\Desktop\rotZ.jpg
Figure 10.11 Rotation on Z- axis
Figure 10.12 shows a combinational transformation of 3 axis rotation.
C:\Users\chand\Desktop\rotXYZ.jpg
Figure 10.12 Combinational rotations on 3 axes
The values of the above rotation angles are saved at configuration/training of the system and saved at the database connected to the system. A simplified diagram of the overall system is shown on Figure 10.13.
Figure 10.13 System Diagram
References
[1]                       Science News ;Animated 3-D Boosts Deaf Education;”Andy” The Avatar Interprets By Signing
<http://www.sciencedaily.com/releases/2001/03/010307071110.htm>
[2]                       Book: Human hand function
Lynette A. Jones, Susan J. Lederman
Oxford University Press US, 2006
[3]                       High Quality Flexible H-Anim Hands for Sign Language Visualization Desmond E. van Wyk, James Connan Department of Computer Science University of the Western Cape, Private Bag X17 Bellville, 7535, South Africa
[4]  The Visual Dictionary. Volume 3: Human body, Bones of The Hand (dorsal view), http://www.infovisual.info/03/027 en.html.
[5]  High Quality Flexible H-Anim Hands for Sign
Language Visualization
Desmond E. van Wyk, James Connan
Department of Computer Science
University of the Western Cape, Private Bag X17 Bellville, 7535, South Africa
[6]                       http://en.wikipedia.org/wiki/3D_studio_max
[7]                       http://www.stmuc.com/moray/


Abbreviations
Degree of Freedom [DOF]
Application Programming interface [API]
Department of Electrical and Information Engineering

2 comments:

  1. total code of the main class files is on this link

    https://docs.google.com/Doc?docid=0AdXMw15inQl6ZGZ2a3g0NWNfNzBnM3E3cDRncw&hl=en

    ReplyDelete
  2. Total Netbeans project is available here:

    http://www.4shared.com/file/riM4GkxJ/avatar.html

    but you needs a password to unzip the file which is given free if you request me through my email:
    chandpriyankara@engineering.com

    ReplyDelete