Mert Utku
Fazıl Doruk İnanç
Meryem Selin Akkız
Sarp Açıkgöz
1. Intro
This report consists of the required data and information that we collected in order to accomplish our exoskeleton finger project. The explanation of important terms such as Modular Section and
Identification for Control Model and Mental State Inference Model is made and mathematical representation is examined.
2. Force Assisted Hand and Finger Device
The core aspect of this research is to put use of robotic treatment that can positively influence hand recovery. This thesis consisted of information about an exoskeleton mechanism with the aim of providing grasping assistance and rehabilitation for hand muscle neourons that have lost their functionalty due to a stroke. The design of this device was in relation with the analysis of four mechanism that has been concluded by other research teams. These mechanisms are Pneumatic
Cylinders, Artificial Rubber Muscles, Linkage Mechanism and Cable-Driven Mechanism. In this design, for the sake of being affordable and simple, the main focus is on assisting two joints in the hand. One being the MP Joint and the other being the PIP (PIJ) Joint.
MP JOINT The finger bones are called phalanges. The metacarpophalangeal joint (MP joint), or knuckle, is where the finger bones meet the hand bones.
At the MP joints, the fingers can move in multiple directions. They can bend, straighten, spread apart and move together. MP joints are important for both pinching and gripping. [1]
PIP (PIJ) JOINT The ligaments surrounding this joint are taut during flexion creating an effective gripping mechanism. As a result, the PIP joint has a crucial role to play in the 'power grip', which along with the 'precision grip' enabled the human race to develop the ability to manufacture and employ tools. [2]
Figure http://www.assh.org/handconsult/Cases-A-
Z/Condition-
Information/ArticleID/50826/Finger-
Laceration#prettyPhoto
1.1

3. Kinematic Model of Hand and Concept of Design
The design consists of index finger and thumb and since both have a different flexion and extension motion, researchers had to produce different mechanisms for each of them. They decided that the index finger will mimic the other three fingers in the same arrangement.
This design was done in virtual ways by CAD Software. The skeleton of a hand is represented as a serial link chain (Free Body with Joints and Degrees of Freedom). Meaning that the palm of the hand is the base frame and the fingertip is the end. In the designed model there are 6 degrees of freedom.
3.1 Measurement
Image processing based measurement is used to measure the relationship between MP joint angles and PIP joint angle either in the situation of extension or flexion condition.
3.2 Experiment
Eight color markers are subjected to a functional hand according to the positions of the figure 1.1. with all joints marked with an individual color, they analyze the flexion and extension motion of each joint by estimating the position via a 3D image position processor. (Optical Motion
Measurement System). The reason of this analysis is to find the relation angle between the joints while performing a specific movement. [This is how they collected the data]
Calculations are done in relation with Vectors. So basically, two joints act as a vector and the angle between them is estimated with dot product which is fairly simple and as follows;
4. Construction and Components
At the early stage of the development, the combination of air cylinders and slotted link cam mechanism is used in the systems.
Where θ is a joint angle, which is the position is located between two vectors,
A and B.
The joint angles can be estimated via the equation (3).
Air cylinder for thumb finger is made of SMC (CJ1B4) where the single motion of the air cylinder moves the cam along with the follower to assist the MP joint rotation motion.

In addition, we used TAIYO Tekko
(7Z-2SR) air cylinder for the mechanism of the index and middle finger because of the triangular link will push by the cam and make the rotational motion to assist PIP joint. [3]
When the user wants to use the device, the user needs to move a small finger because there is a bending sensor.
If bending sensor is bended and make changes to the electrical resistance. The bending angle becomes greater than threshold and the system activated and started to grasp the object.
In others words, when the user starts moving the small finger. The bending angle is measured as an electrical voltage. The voltage is measured by Analog/Digital converter. The control voltage is calculated and transmitted the output via Digital/Analog converter.
Electric Pressure regulator (E/P regulator) will activate the air compressor and supply air to the air cylinders to assist hand.
5. Kinematic analysis of the finger exoskeleton
To begin with, literally exoskeleton is the external skeleton which protects the animals from outside threats. This term is not limited just for the animals and the human beings articially produced exoskeletons and used this for various purposes throughout the history. The exoskeletons are divided into two categories which are active and passive exoskeletons. The passive exoskeletons’ movements are based on human musculoskeletal system and it is used for protection of the humans. The active exoskeleton is used for increasing the movement abilities. The active exoskeleton is capable of enhancing the movement capabilities of the full human body and only a selected part of it.

6. Materials and methods
The main design criterion in this study is to develop finger movement abilities close to natural by applying the lowest possible forces on a finger joint. The internal joint forces are transmitted by the exoskeleton by constructing a design which is composed of four link system mounted on a human palm. The finger exoskeleton system used in this experiment composed of three link subsystems and one link which is supporting the finger. The d, c, e parts of the system are link subsystems and k is the link that supports the finger. The whole system is driven by two actuators which are S1 and S2. S1 controls the movement of the subsystem and S2 position is related to S1 position . In addition, S2 changes the position of the k-link. The finger is connected to the k-link and moves around the finger joint center of rotation. There is an important design requirement in this system. The system controls the change of position of the S2 actuator in such a way that a straight line between A, R and S points is obtained.
Figure 1 Physical representation of the system
There is also the geometrical representation of the system which is shown in figure 2. İn the figure the d,c,e and k are the lengths of the main system links. The S1 and S2 actuators’ lenghts are x1 and x2 respectively. There is also the related angles shown in the model. α1, α2, α3 and αk are the

angles of the e,d , c and k-links respectively.S1 actuator angle is α7. The system basic dimensions are a1, b1 a and b. The s2 actuator is placed between points E and R.
Figure 2 Geometric representation of the Ssytem
After the representations of the figure the report includes the formulas used for calculating the related angles.
α1=arcsin((a+s*sin α2-c*sin α3)/e)
α2=arcsin((α1+x1*sin α7)/(d-m))
α3=arctg((d*sin α2+a)/(d*cos α2+b))-arccos[((d*sin α2+a) 2
+(d*cos α2+b)
2
)
2
+c
2
-e
2
/(2*((d*sin
α2+a) 2 +(d*cos α2+b) 2
)*c]
α7=arccos[(b1-(d-m)*cos α2)/x1]
For the calculation of the αk angle for the k-link the y-coordinates of the A and R are used
αk= arccos[(Y a
-Y b
)/p).

Then the angular velocities are calculated in the report by taking the time derivatives of the angles.
After that the distance between points A and S is calculated .
7. Results
The solution of all the equations related with the experiment was found using Mathlab simulink software. Then the detailed information about specified values , such as actuator length, actuator velocity, angles and angular velocities are given in the report. It is important because giving detailed information about the specified values gives the chance to understand the system better.
After that the mathlab simulink software was used with specific values in order to check the system behavior. Simulation lasted for 0.20 s and a full range of the angle of the k-link was found to be 1.745 [rad].
These experiment parameters are used in the real life model for left palm of a 33 yaer old man.
Then various time based diagrams are plotted in the report like the length of S1 and S2 actuators compared with the αk angle.

The other time-based diagrams plotted in the report are v1 and v2 velocities compared with ω k angular velocity, angles in the system versus time and angular velocities versus time.
8. Discussion
Numerical results that is found by using mathlab simulink software is compared with the real life model. A video capturing method is used in order to make a comparison between the real life model and the Mathlab simulation. The differences in the simulation and real life model is presented via tables . The results are not much different from each other and the Simulink model could succesfully present the real life model.
9. A Finger Exoskeleton Robot for Finger Movement Rehabilition
In this article before they start to design and simulation of the finger exoskeleton robot for finger movement rehabilitation, they analyse the systems that they could use in finger exoskeleton.
These mechanisms are underactuated mechanisms and tendon-driven mechanism. Underactuated mechanisms has fewer driving sources than the number of degree of freedom and generally tendonactuated or linkage mechanisms are used as underactuated mechanisms. Tendon-driven mechanics are simple and absorbs shock but joints in the finger loaded more because of friction. In the hand exoskeletons electric actuators, pneumatic actuators and smart material actuators could be used as actuators. [4]
Finger bending motion is studied and analysed for design of the finger exoskeleton robot. After that, 30N-mm input torqued motor is selected as actuator. However, moderate stroked patients more torqued motor should be used. [4]
In the design, they use a slider and N-shaped linkage. The prototype is formed by three sliders, five links, ten bolts and one motor.

Tzu-Heng Hsu, Yen-Cheng Chiang, Wei-Tun Chan, & Shih-Jui Chen. (2017). A Finger Exoskeleton Robot for Finger Movement
Rehabilitation. Inventions, Vol 2, Iss 3, p 12 (2017) , (3), 12. https://doi.org/10.3390/inventions2030012
As the motor rotates, the blue crank moves the gray coupler forwards or backwards. The gray coupler pushes and pulls the yellow slider arm, making it move along the slot. When the yellow slider moves, this cause the green link, or N-shaped linkage, to rotate, which in turn causes the yellow and outer red sliders to move together. The N-shaped linkage continues to push and pull the outer red slider, causing it to move along the slot. The outer red slider connects to the human dinger and causes it to bend. [5]
They made finite element analysis for their design and set 30N-mm input torque because of motor selection. Mesh element was 0.05mm averagely. PLA (Poly Lactic Acid) is used as material for
3D printing. PLA is biodegradable and biocompatible but mechanical strength is low so they use maximum stress fracture criterion in design. In the analyse they saw maximum normal stress is about 4,68 MPa and PLA ‘s tensile strength is 20 MPa as seen in datasheet. This analyse shows that PLA is suitable for this project.
[5]
Tzu-Heng Hsu, Yen-Cheng Chiang, Wei-Tun Chan, & Shih-Jui Chen. (2017). A Finger Exoskeleton Robot for Finger Movement
Rehabilitation. Inventions, Vol 2, Iss 3, p 12 (2017) , (3), 12. https://doi.org/10.3390/inventions2030012

They use speech commands to control the stepper motor. Arduino was used for taking the commands and a driving circuit was used to control the stepper motor.
28BYJ-48 stepper motor is used because of size and torque factors. Arduino converts speech to text and compared with command that stored in system to control the finger exoskeleton robot.
In the analyses of the motion there’s considerable error in x direction between 4s and 8s and this error is occurred because of friction force of 3D printed parts or the tolerance of 3Dprinter.
Using CNC machined aluminium parts can reduce this error. [6]
In healthy people, torque in joints is around 0.1-0.3 N-m [2,3] but they use 0.003 N-m input torqued motor. So, their motor is suitable only for slight stroke patients. For more moderate stroke patients 12 Volt 35BYJ-46 motor which has 78 N-mm input torque could be used.
[6]
10. The Mirror Neurons
Mirror neurons are part of the human mirror system and it is attributed various functions. There are three major views about the decoding of the mirror neurons. First of all L1 view asserts that the mirror neurons decode the motor parameters of an event. For instance, an observer codes the trajectory of the hand when the demonstrator directs his hand to the coffee cup. Secondly, L2 view is also about the motor parameters but it decodes the schema of the motor plan. For instance, it decodes the act of putting an object in a cup without giving much importance to the trajectory.
Thirdly, L3 view tells that the mirror neurons decode the intentions of the observed actions. For example, the person observes his friend who is grasping the cup in a pinching motion can interpret that his friend will drink tea from that cup. There is an experiment which focuses on the mirror neuron’s intention decoding acts and experimenter observes that the macaque monkey does not respond to non-goal directed actions. Indeed, their mirror neuron firings are limited to goal directed actions. For instance, in one case the macaque monkeys are trained to use inverse pliers for picking up peanuts. They found out that the mirror neuron activity does not correlate with the hand movement when the monkey uses inverse pliers. Instead, they found that mirror neuron activity increases with the motion of the tip of the pliers. This finding supports the intention coding view because in the experiment monkey aims to pick up the peanuts and the mirror neuron activity reaches higher levels in the motion of the tips of the pliers. Therefore, the hand movement is not that important for the mirror neurons. There is another function attributed to mirror neurons which

is the prediction role of them. The MSI (mental state inference) model attributes this function and they conducted an experiment in order to prove their hypothesis. They placed an hammer in a table and a person grasps this hammer in the experiment. The observer can predict what the demonstrator can do with the hammer. The alignment of the hand and thumb can give information to the observer about what will demonstrator do with the hammer. For instance the holding of the hammer or prying a nail with the hammer requires different alignments of the hand and thumb and it leads to the understanding of demonstrator’s intention by the observer.
11. Mental State Inference Model
The mental state inference model is related with the motor component of the mirror neurons and it deals with the motor control role of the mirror neurons. In this model a predictive role is assigned to mirror neurons. When an observer sees an action F5 region mirror neurons are used to create motor imagery or mental simulation of the movement for mental state inference. Secondly, this model asserts that mirror neurons can understand the intentions of the others by analysing the kinematics of goal directed actions. For instance when a person moves his/her hand to grasp a hammer he will have a intention to grasp it and the alignment of his hand will be different if he just grasps the hammer for putting it in another place or prying a nail. The model states that an observer can infer the intention of the demonstrator with analysing his hand’s alignment when he moves his hand to grasp the hammer. Additionally, The MSI model suggests that a forward prediction is carried out in the observer’s dynamic related brain regions and if his guess does not match with the observed action then he will change his guess to the observed action in a mental simulation loop.
Fig9. The schematic representation of the MSI model

12. MOSAIC Model
The MOSAIC (Modular Section and Identification for Control) model was first introduced for the function of motor control. The aim of the motor control is to produce motor commands at a specific time t with a desired state. The relationship between the output, time and desired state is represented with various functions (Fig11). In the model, there are multiple controller-predictor pairs that contributes to the overall control. Controller that have a better predicting state, has less prediction error and becomes more influential in the overall control. Prediction errors that come from the Predictors are computed with the normalization and forms the responsibility signal.
Another function of the MOSAIC model is based on the imitation and action recognition. In this function, the realization of imitation has three stages. First one is the conversion of the visual data of the actor’s movement to have a desirable input format to the motor system of the imitator. This step is important for the visual processing system to understand the observed action and make a connection between it and the desired state. Second stage is forming the motor command required to accomplish the trajectory of the observed action. An output is taken from the controllers in order to have inputs for predictors paired with controllers. These inputs are not used for the real action generation. Outputs taken from the predictors creates prediction errors by comparing the actors actual action and observer’s predictions. Responsibility signals that come from normalization, represents the continuous observed action and can be used immediately or stored for later use in the model. These stages can be seen in Fig10.
These two functions of MOSAIC representation provide to have an output of predictors that can be resembled to mirror neuron activity. There is also a model called MSI, where the mirror neurons may implement a motor-sensory forward model. However, the main difference between these models is MSI deals with kinematics and MOSAIC focuses on the dynamics. The output of the forward model required by MSI is in visual like coordinates, on the other hand in the MOSAIC model the output is more closely related to the actual variables of the controlled limb. We thought that, this difference can be explained with an example; during the reaching process of hand to a hammer, the coordinates of the hand and hammer and the difference between them, is related with the MSI model. However, the velocity and the joint angles of the hand is related with the MOSAIC model. Joint angles are important for understanding the intention of the actor, for instance he can nail by using the hammer of just move it.

Fig10.
Fig1.2.
Fig11.
13. PsychoPy
In the psychopy we uploaded the video of a finger which does pinching. After that we need to give trigger from serial or parallel port. Parallel port is more accurate than serial port so we need to write a code to give trigger from parallel port. Before write a code for giving trigger, we used the tool of Parallel port in I/O part of
PsychoPy but we are not sure to use that tool. For this reason, we tried to write a code and it can be seen in the Appendix.

Fig11.Tool of Parallel port Fig12. Tool of parallel port and the movie file. that gives Trigger between 1ht and 0 th frame.
14. Conlusion
With these collected data, we obtained some results and made predictions that are useful for the future of our project. From our previous research, we saw that the best way to give a trigger is the usage of both the visual and aural stimulus. We filmed a video that has the same finger motion that we want to accomplish with this project. this video is also used for giving frame in PsychoPy. The aural stimulus would have better effects if the sound would familiar to the person. So, we can use the clicking sound of a mouse or the ring sound of a hotel bell. However, there is a problem on giving frame in PsychoPy. In the serial port we couldn’t give the frame, then we tried using a USB but the computer didn’t identify it. In order to solve this problem, we will consult to our advisor. In the design of the finger in SolidWorks we mainly focused on the part where the exoskeleton gets attached to the finger. There are two possible joint placed that we can attach it. For both of them there is a half circular part for the finger to pass through. In order to make it suitable for every finger size, we are planning to attach a hook and pile for the other half of the circle. Exoskeleton will be formed with a 3D printer and from our researched the appropriate material was seen to be PLA.

15. APENDIX
Code: import serial from psychopy import visual,monitors mon = monitors.Monitor('SonyG55') mon.setDistance(114) win = visual.Window(size=[1024,768], monitor=mon) mov = visual.MovieStim3(win, 'exo.mp4', size=(320, 240),
flipVert=False, flipHoriz=False, loop=False) print(320, 240) print(3.67) mov.draw() nFramesOn = 5 port=serial.Serial('COM1',9600) for frameN in range(200):
if frameN==nFramesOn:
port.write(1) port.close()
In this code [7] , we have some errors and also, we have problem to import the video with code and counting frames inside the video.

SolidWorks Design

Referances:
[1] http://www.assh.org/handcare/hand-arm-conditions/MP-Joint-Arthritis
[2] https://www.kenhub.com/en/library/anatomy/proximal-interphalangeal-joints-of-the-hand
[3] Mohd Nor Azmi Bin Ab Patar, 1Takashi Komeda, 2Jamaluddin Mahmud
1Research Organization for Advance Engineering, Shibaura Institute of Technology,Japan
2Centre of Excellence for Humanoid Robotics and Bio-Sensing, Universiti Teknologi MARA,
Malaysia
[4] Tzu-Heng Hsu, Yen-Cheng Chiang, Wei-Tun Chan, & Shih-Jui Chen. (2017). A Finger
Exoskeleton Robot for Finger Movement Rehabilitation. Inventions, Vol 2, Iss 3, p 12 (2017), (3),
12. https://doi.org/10.3390/inventions2030012
[5] Wang, J.; Li, J.; Zhang, Y.; Wan, S. Design of an exoskeleton for index finger rehabilitation.
In Proceedings of the IEEE EMBS conference, Minneapolis, MN, USA, 2–6 September 2009. [6]
Kawasaki, H.; Kimura, H.; Nishimoto, Y.; Hayashi, H.; Sakaeda, H. Hand rehabilitation support system based on self-motion control, with a clinical case report. In Proceedings of the Automation
Congress, Budapest, Hungary, 24–26 July 2006.
[7] http://www.psychopy.org/api/serial.html