Literature Search Report Team M&M
Project Topic: Refreshable Braille Display
Members: SokKimEng Chham, Kyle Fleming, Nikita Gupta, Carolina Trigueros Torres
Figure 1: Example of Learning Process [1b]
Summary of Project
The goal of our senior design project is to create an interactive way to encourage both blind and able­sighted peoples to learn the braille script. This project would serve both the handicapped and those who live and work with those handicapped every single day, and could truly prove to be a vital step in the process of learning to cope with a disability. The idea behind our braille teaching device is to automate the process of learning the language, allowing the student to escape the endless stacks of paper and tedious manual devices used in teaching the script. The machine would work similarly to the graphic shown above, figure 1. In some way the user would input a word, or be given a word at random, and would then be given the chance to feel that word with his/her hand in braille. This mode of teaching reaches the student in three different senses, sight, hearing and feel, and thus activates his/her brain more (as explained in detail with sources later). It is also just plain more entertaining for the student, and will hopefully encourage them to stick with the lessons instead of quitting. Lastly, games and puzzles can be added as the software becomes more complex. While extremely complex games and software is out of the scope of a mechanical engineering project, if this product were to be developed for the market, this would be a sure upgrade. As of now our idea is to use a series of gears attached to wheels which spin and form the braille letters. Each wheel is controlled by its own servo motor, and the motors are controlled by a microcomputer such as an arduino or raspberry pi. Input and output can be done through something as simple as an LED screen or as complex as an iPhone or tablet. Voice recognition software will allow for the user to communicate with the device, responding with answers when prompted with a random word or ready to hear commands when the user is setting up a game. Since we are considering using wheels that spin and are partially uncovered, we will need to analyze the safety of our device and make sure fingers and objects don’t get caught within the mechanism. We will also need to determine how robust our device is and what can be done to increase its robustness. For example, if liquid is spilled on top of it will electrocution be an issue, or will it just cease to function?
We also have a number of other ideas for how we could design the mechanical components of this design. Other possible options are a design similar to a rack and pinion design as shown in the internet section of this report, different wheel designs, or using fan belts to drive the wheels. Unless otherwise advised, our plan will be to try multiple inexpensive options, and determine empirically which design functions best. 1. Internet Search
We hope to use speech recognition software to interact with the Braille learner. With speech both the sighted and blind user can learn braille from the device. The user will say out loud what she/he has read from the tactile display. The speech recognition software will analyze and correct the user. At another level, we can also include an input keyboard device. Whatever is inputted can be repeated by the speech software and displayed on the tactile display to be corrected or read. There are many speech recognition software available as open source code. [4a]
Also of importance in developing a teaching/learning device is Multimodal Interaction (MMI). Compared to unimodal systems this provides a more natural experience to the user by engaging more senses. [5a] We might first focus on unimodal interaction and if we have time, then add more complexity to our system.
Figure 2: Braille Buddy [7a]
An internet search of different methods for refreshable tactile displays turned up a concept design called “Braille Buddy.” It uses electroactive polymers for its tactile display. It also has a six key input, one for each dot, so that the user can also input braille. However, no prototype has been made or designed. It seems to be a purely conceptually device at the moment. [7a]
A common way of creating refreshable tactile displays is using pins that move up and down to form the dots of the braille letter. Different mechanisms or material properties are used to raise and lower the dots. Many use the piezo effect, where crystals expand when a voltage is applied to control the pins. [10a] Another method uses solenoids. [11a]
Figure 2: Perpendicular Rotary Wheel Shape [1a]
In our early concept design, the wheel on which the dots that form the characters is shaped like in Figure 1. Each wheel will be an octagon with each side of the wheel containing a column of three dots. Two wheels will make up up one character. This orientation is preferred so that all the wheels will be on one shaft and there is space for gears and other controlling parts to be placed between the characters. In this orientation the wheel turn perpendicular to the direction in which Braille is read.
Figure 3: Horizontal Rotary Wheel Shape [1a]
An alternate way to orient the wheels would be that the wheels turn in the same direction that Braille is read. Each character would then be composed of three wheels that contain two dots each. The wheels would be shaped like a square. The advantage in this orientation is that the wheel is smaller and we need to keep track of only four positions. More wheels also require more motors. However, in this orientation it would be relatively easy to add more characters if desired. [1a] Since each individual wheel will be moving individually, we need a way to control the motion of each wheel. One method is to glue gears onto the wheels and the gear is connected to a motor through a shaft/gear system. Another method is what toy helicopters use for counter rotating blades or coaxial rotors. This would eliminate the need to fit gears in between the wheels. [2a] Figure 4: Rack and Pinion Method [3a]
An alternative to wheels is the rack and pinion method. The dots are printed on a thin rigid bar and the rack is on the bottom on the bar. The gears that control the motion and consequently the characters that are shown are placed underneath the bar. The advantage to this system is that since the gears are underneath, the spacing is not as tight. Also there is ample space in this system to stagger the motors and gears. [3a] Another alternative to wheels would be a belt track like they have on date stamps. With a belt, the rotating shape would not have to be a circle allowing more freedom with the design.
Figure 5: Rotating belt on Date Stamp [6a]
2. Journal Search
Figure 6: Learning Format [1b]
We want to build a system that is similar in concept to Araki, M. et al. [1b] We can use their system as a reference for the format of the interaction between the user and device. Their tactile display however used pins that were pushed up to create the dots. The paper focuses less on the tactile display and more on the user­device interaction format specifically Multimodal Interaction (MMI). [1b]
T. Nitta et al. developed Galatea­toolkit, an open­source, license­free software that can be used to develop “unique life­like agents that can communicate with users via spoken language.” [4b] There is also an open­source speech recognition engine Julius available. [6b]
While researching possible ideas to create a braille teaching machine, we found that a refreshable braille cell is a very common design. [2b, 3b, 5b]. A refreshable braille display is an electro­mechanical device that displays braille characters, usually by independently raising pins through holes in a flat surface. There are many ways to do this. For example, piezoelectric and magnetic technologies have been developed as commercial pin actuators for refreshable Braille displays. [2b]. Piezoelectric bimorphs reeds are mounted as a stair stepped stack of cantilevers, each with a braille pin resting on its free end. Each reed lifts the pin above the surface to be read. As described in [3b], it is also possible to use magnetic latches to pull/push the pin up and down by running current through coil wound around the pins. We thought about using magnets but one of the major drawbacks with this method is cost and difficulty in assembly. It would be very expensive to get magnets as small as would be needed, and using them in such close proximity also poses a problem. We found that the most common method used is the piezoelectric bimorph; however, this method is very expensive, and therefore, not readily available to everyone. [7b] We decided to try to build a device that is not only more simplistic in design but also low­cost. 3. Patent Search
Through our patent search we came across many alternatives to our rotary idea. One common method was the use of electromechanical braille cells. One way to do this is to have a processor that converts a speech or digital message into a braille message that can be read in a separate display. The braille characters are formed by pins [1c, 7c]. A second way of doing this was applying an operating potential that would cause bending in a piezoelectric reed. This will result in the braille indicia protruding through the reading surface [2c, 8c]. Another way of controlling the pins would be using bimetallic latches [4c]. Other devices have implemented a combination of bending and the use of springs [5c, 6c]. The pins can also be controlled by an actuator. An alternative to pins could be an electroactive polymer actuator. Each dot in the braille character could be made of this special polymer that deforms, causing a bump, when current passes through it. When there is no current the polymer remains flat [3c]. The dots can also be represented by tiny magnetic balls which attach to a magnetic recording medium. The medium is a rotating wheel, and as it rotates the magnetic balls facing downwards are released, while other balls are attached to form the desired character[9c]. 4. Dead Ends
Some dead ends we stumbled upon included a portable braille display that attached to a notetaker and a plastic sheet whose material is displaced into braille characters protruding from surface among others. [1b, 2b]. Some interacted with the user by reacting to nerve impulses in the brain, which is beyond the scope of this project. [4d] We decided that there wasn’t a lot of relevant information to gather from these sources. 5. References
a. Internet Search/ Miscellaneous [1a] MaxiAids. Advertisement. MaxiAids.com. N.p., n.d. Web. 10 Oct. 2013. <http://www.maxiaids.com/products/5322/Pocket­Braille­Cube­Learning­Device.html>.
[2a] Helicopter Model in Columbia University’s Mudd Building’s MechE lab.
[3a] Cheng, Andrew, Megan Davies, Maxwell Davis, Ritwik Ghosh, and Kuhan Puspanathan.BrailleDisplay_Team15. BrailleDisplay_Team15. N.p., n.d. Web. 10 Oct. 2013. <https://sites.google.com/site/brailledisplayteam15/home>.
[4a] "List of Speech Recognition Software." Wikipedia. Wikimedia Foundation, 10 Apr. 2013. Web. 10 Oct. 2013. <http://en.wikipedia.org/wiki/List_of_speech_recognition_software>.
[5a] Ali, Abdo El. "Multimodal Interaction: An Introduction." Lecture. Multimodal Interaction: An Introduction. Multimodal Interaction: An Introduction. Web. 10 Oct. 2013. <http://www.slideshare.net/Abd0/multimodal­interaction­an­introduction>.
[6a] Offistamp® Traditional Date Stamp, 5MM. Digital image. Staples.com. N.p., n.d. Web. 11 Oct. 2013. <http://www.staples.com/Offistamp­Traditional­Date­Stamp­5MM/product_920280>.
[7a] Sheri, Yasaman. "My Best Buddy Braille." Yanko Design: Form Beyond Function. N.p., n.d. Web. 15 Oct. 2013. <http://www.yankodesign.com/2010/09/01/my­best­buddy­braille/>.
[8a] Hollister, Sean. "Braille Buddy Concept Keyboard Teaches Six­dot Typing Alt." Rev. of Yanko Design. Engadget n.d.: n. pag. Engadget. Web. 15 Oct. 2013. <http://www.engadget.com/2010/09/05/braille­buddy­concept­keyboard­teaches­six­dot­typin
g/>.
[9a] Braille Buddy ­ Tool for Braille Education. Perf. Yasaman Sheri. Vimeo. N.p., n.d. Web. 15 Oct. 2013. <http://vimeo.com/25527635>.
[10a] "Refreshable Braille Display." Wikipedia. Wikimedia Foundation, 10 Nov. 2013. Web. 15 Oct. 2013. <http://en.wikipedia.org/wiki/Refreshable_braille_display>.
[11a] "Braille Display Page." Braille Display Page. N.p., n.d. Web. 15 Oct. 2013. <http://www.deafblind.com/display.html>.
b. Journal Search
[1b] Araki, Masahiro, Kana Shibahara, and Yuko Mizukami. "Spoken Dialogue System for Learning Braille." Computer Software and Applications Conference (COMPSAC)(2011): 152­56. IEEE Xplore. Web. 10 Oct. 2013.
[2b] Lee, J. S., and Lucyszyn, S., 2005, "A Micromachined Refreshable Braille Cell." Journal of Microelectromechanical Systems, 14.4, pp. 673­82.
[3b] Haga, Y., Makishi, W., Iwami, K., Totsu, K., Nakamura, K., and Esashi, M., 13 April 2005, “Dynamic Braille display using SMA coil actuator and magnetic latch.” Sensors and Actuators A: Physical, 119.2, pp. 316­322
[4b] Nitta, T., Et Al. "Activities of Interactive Speech Technology Consortium (ISTC) Targeting Open Software Development for MMI Systems." Robot and Human Interactive Communication, 2004. ROMAN 2004. 13th IEEE International Workshop on(2004): 165­70. IEEE Xplore. Web. 10 Oct. 2013.
[5b] Roberts, J., Slattery, O., and Kardos, D., 2000, "49.2: Rotating­Wheel Braille Display for Continuous Refreshable Braille." SID Symposium Digest of Technical Papers, 31.1, pp. 1130­133
[6b] Lee, Akinobu, and Kawahara Tatsuya. "Recent Development of Open­source Speech Recognition Engine Julius." Asia­Pacific Signal and Information Processing Association 2009 Annual Summit and Conference (2009): 131­37. Engineering Village. Web. 10 Oct. 2013.
[7b] Bogda, G., Vishoot, B., Grider, C., and Schroeder D., Spring 2011, "Design and Testing of a Low­Cost Refreshable Braille Display Actuation Technology." Journal of Engineering Technology, 28.1, pp.26­32.
[8b] Bar­Cohen, Y., Electroactive Polymer (EAP) Actuators as Artificial Muscles: Reality, Potential, and Challenges. Bellingham: SPIE ­ The International Society for Optical Engineering, 2004.
c. Patent Search
[1c] Bucchieri; Vittorio, 2013, “Apparatus and method for presenting and controllably scrolling Braille text.” U.S. Patent, 8382480.
[2c] James F. Tetzlaff, 1979, “Electromechanical Braille Cell.” U.S. Patent, 4283178.
[3c] Peichun Yang, 2004, “Electroactive polymer actuator braille cell and braille display.” U.S. Patent, 20040175676. [4c] Rose; Leonard, 1981, “Braille Display Reader.” US Patent, 4266936.
[5c] Frans Jan Tieman, 2001, “Braille cell and display comprising a plurality of braille cells, as well as a method for controlling a plurality of pins of a braille cell.” U.S. Patent, 6217338.
[6c] Edward James Tapscott, “Automated Braille Inserter.” U.S. Patent, 20100003646.
[7c] John Roberts; Oliver Slattery; David Kardos, 2004, “Refreshable braille reader.” U.S. Patent, 6776619. [8c] Patrick Murphy; Todd Conard; Waldemar Tunkis; Michael Goldenberg, 2012, “Braille Display Device and Method of Constructing Same.” U.S. Patent, 20120214139.
[9c] Jean J. Eltget, 1985, “Apparatus for converting electric signals representing data into a group of raised dots constituting the image of said data.” U.S. Patent, 4500293.
d. Dead Ends
[1d] https://www.google.com/patents/CA2494291C?cl=en&dq=braille&hl=en&sa=X&ei=WQpLUvPBJN
G24APc14H4BA&sqi=2&pjf=1&ved=0CD4Q6AEwAQ
[2d]
https://www.google.com/patents/CA2379707A1?cl=en&dq=braille&hl=en&sa=X&ei=WQpLUvPBJ
NG24APc14H4BA&sqi=2&pjf=1&ved=0CEwQ6AEwAw
[3d]
https://www.google.com/patents/US20130164717?dq=braille&hl=en&sa=X&ei=WQpLUvPBJNG24
APc14H4BA&sqi=2&pjf=1&ved=0CGgQ6AEwBw
[4d]
http://www.kip.uni­heidelberg.de/cms/vision/projects/recent_projects/tactile_vision_substitution_system/
the_ptd/
[5d]
Danieli GA. Analytical Description of Meshing of Constant Pressure Angle Teeth Profiles on a Variable Radius Gear and its Applications. J. Mech. Des..2000;122(1):123­129. doi:10.1115/1.533551.
A Closed­Loop Neurorobotic System for Investigating Braille­Reading Finger Kinematics
Jeremie Pinoteau, Luca Leonardo Bologna, Jesus Alberto Garrido,and Angelo Arleo
http://www.anc.upmc.fr/biblioNca/pdf/Pinoteau2012a.pdf