ACADEMIA
Virtual reality world teaches deaf children math skills
Purdue University is using technology employed in the films "King Kong" and "Lord of the Rings" to create colorful characters in a virtual reality world who teach deaf children math. Computer graphics technology students are working in Purdue's Envision Center for Data Perceptualization with high-tech cameras and "cybergloves" that can translate body and hand motions into digital images. The resulting cartoon rabbit, robot and pig use sign language in fun, interactive environments projected on the walls and floor of a "cave" of screens that surrounds the deaf students. They wear lightweight stereoscopic glasses so the virtual reality images appear three-dimensional. A device monitors the student's head position so the environment is consistently redrawn to match the user's perspective. A wrist tracker and telemetric "pinch gloves" monitor the student's hand and finger movements allowing interaction with the virtual environment and prompting responses from the characters. 
The virtual reality program is designed to provide early elementary school age students with disabilities with a number of active, individualized learning conditions: • The ability to control their environment. • The ability to engage in learning activities at their own pace. • The ability to repeat activities as needed. • The ability to see or feel items or processes in concrete terms. • The ability to practice daily living tasks in a safe and barrier-free environment. • Motivation to succeed. For example, in a virtual candy store environment the student communicates to the storekeeper in sign language, some of which is specific to mathematics. Pinch gloves allow students to count candies and to add and subtract by putting candy on or off the counter. The task can be repeated over and over at the student's own pace while providing consistent and understandable feedback. "Learning is the development of one experience into a new experience," said Ronnie Wilbur, professor and chair of Purdue's audiology, speech sciences and linguistics department who has served as a consultant to the College of Technology during the project. "Immersive learning environments such as this are more effective than traditional computer software." The project's supervising professor, Nicoletta Adamo-Villani, said virtual reality helps break down some of the barriers deaf children experience. "Environments are more stimulating when students are able to interact with the subject and travel through the scenes," said Adamo-Villani, assistant professor of computer graphics. "Hands-on experiences equate to a better understanding of mathematical concepts in real-world situations." Adamo-Villani said research shows that humans process visual information 60,000 times faster than textual information and that an eight-week virtual reality program can improve student math skill scores by 16 percent. She said that learning enthusiasm remains even after the novelty of virtual reality fades. "Our mission is to be a worldwide leader in virtual learning environment development," said Laura Arns, the associate director of the Envision Center who is helping develop this virtual reality application. "We want to advance industry-leading concepts, software and services while fostering an environment of empowerment, creativity and commitment." The program is being designed to overcome the barriers deaf children experience in learning math skills. Those barriers include: • Significant delay in reading comprehension. • Parents' inability to convey math concepts in sign language. • Difficulty taking advantage of supplemental learning opportunities, such as television shows and dinner-table conversation. Adamo-Villani hopes by reducing the impact of these barriers, the program can help increase the number of deaf students who go on to study science, technology, engineering and mathematics — fields in which they are statistically underrepresented. Background research for the application revealed that deaf students historically have had difficulty gaining entry into higher education that leads to these careers. "As a leading educator in these fields, Purdue is dedicated to increasing access for deaf students," Adamo-Villani said. Because research shows that children prefer bright colors, student programmers are working to ensure that the program is as colorful as possible. Also, certain colors increase alpha waves directly linked to awareness and improve students' attention span and elicit emotional responses. Specific examples in the Purdue program include shelves lined with jarfuls of hard candies and Chef Pig's icing-smeared apron. The programmers continue the time-consuming task of calibrating and cleaning up data, tasks necessary to make the characters' movements as smooth and lifelike as possible. "Fluid, non-mechanical motion is fundamental to learning sign language effectively," said Edward Carpenter, the graduate research assistant working directly with the five undergraduate programmers who have dubbed themselves "Dented Can, LLC." "That's why we have invested our efforts in developing natural gestures that are appealing to children." Undergraduate David D. Jones never expected to be mastering complex skills such as environment modeling, character modeling and rigging, motion-capture data application and programming for interaction in 3-D space so early in his academic career. He said he is excited that his work will be utilized for years to come to help others learn — a prospect that he said far outshines any class project grade he will receive. "I truly hope that this program plants some of the seeds necessary to develop good math skills in those children," said Jones, who continues his computer graphics training as a graduate student at Purdue fall semester. "Who knows, maybe someday they'll be as fortunate as I was and get to apply those skills to something interesting and worthwhile." Gifted children attending academic camps at Purdue came to the Envision Center this summer to test the system and provide feedback. In the fall, a portable version of the virtual reality application will be taken to Indianapolis and introduced to classrooms at the Indiana School for the Deaf. "Three years of inventive, collaborative work have created a tool I am eager to see used to open new horizons for deaf students," Wilbur said. Adamo-Villani credits the university's investment in information technology and its emphasis on internal and external collaboration for making the new instructional tool possible. She and her students from the Department of Computer Graphics Technology have been working with the Envision Center for Data Perceptualization; Information Technology at Purdue; the College of Liberal Arts' Department of Speech, Language and Hearing Sciences; and the Indiana School for the Deaf to develop the virtual reality program.
The virtual reality program is designed to provide early elementary school age students with disabilities with a number of active, individualized learning conditions: • The ability to control their environment. • The ability to engage in learning activities at their own pace. • The ability to repeat activities as needed. • The ability to see or feel items or processes in concrete terms. • The ability to practice daily living tasks in a safe and barrier-free environment. • Motivation to succeed. For example, in a virtual candy store environment the student communicates to the storekeeper in sign language, some of which is specific to mathematics. Pinch gloves allow students to count candies and to add and subtract by putting candy on or off the counter. The task can be repeated over and over at the student's own pace while providing consistent and understandable feedback. "Learning is the development of one experience into a new experience," said Ronnie Wilbur, professor and chair of Purdue's audiology, speech sciences and linguistics department who has served as a consultant to the College of Technology during the project. "Immersive learning environments such as this are more effective than traditional computer software." The project's supervising professor, Nicoletta Adamo-Villani, said virtual reality helps break down some of the barriers deaf children experience. "Environments are more stimulating when students are able to interact with the subject and travel through the scenes," said Adamo-Villani, assistant professor of computer graphics. "Hands-on experiences equate to a better understanding of mathematical concepts in real-world situations." Adamo-Villani said research shows that humans process visual information 60,000 times faster than textual information and that an eight-week virtual reality program can improve student math skill scores by 16 percent. She said that learning enthusiasm remains even after the novelty of virtual reality fades. "Our mission is to be a worldwide leader in virtual learning environment development," said Laura Arns, the associate director of the Envision Center who is helping develop this virtual reality application. "We want to advance industry-leading concepts, software and services while fostering an environment of empowerment, creativity and commitment." The program is being designed to overcome the barriers deaf children experience in learning math skills. Those barriers include: • Significant delay in reading comprehension. • Parents' inability to convey math concepts in sign language. • Difficulty taking advantage of supplemental learning opportunities, such as television shows and dinner-table conversation. Adamo-Villani hopes by reducing the impact of these barriers, the program can help increase the number of deaf students who go on to study science, technology, engineering and mathematics — fields in which they are statistically underrepresented. Background research for the application revealed that deaf students historically have had difficulty gaining entry into higher education that leads to these careers. "As a leading educator in these fields, Purdue is dedicated to increasing access for deaf students," Adamo-Villani said. Because research shows that children prefer bright colors, student programmers are working to ensure that the program is as colorful as possible. Also, certain colors increase alpha waves directly linked to awareness and improve students' attention span and elicit emotional responses. Specific examples in the Purdue program include shelves lined with jarfuls of hard candies and Chef Pig's icing-smeared apron. The programmers continue the time-consuming task of calibrating and cleaning up data, tasks necessary to make the characters' movements as smooth and lifelike as possible. "Fluid, non-mechanical motion is fundamental to learning sign language effectively," said Edward Carpenter, the graduate research assistant working directly with the five undergraduate programmers who have dubbed themselves "Dented Can, LLC." "That's why we have invested our efforts in developing natural gestures that are appealing to children." Undergraduate David D. Jones never expected to be mastering complex skills such as environment modeling, character modeling and rigging, motion-capture data application and programming for interaction in 3-D space so early in his academic career. He said he is excited that his work will be utilized for years to come to help others learn — a prospect that he said far outshines any class project grade he will receive. "I truly hope that this program plants some of the seeds necessary to develop good math skills in those children," said Jones, who continues his computer graphics training as a graduate student at Purdue fall semester. "Who knows, maybe someday they'll be as fortunate as I was and get to apply those skills to something interesting and worthwhile." Gifted children attending academic camps at Purdue came to the Envision Center this summer to test the system and provide feedback. In the fall, a portable version of the virtual reality application will be taken to Indianapolis and introduced to classrooms at the Indiana School for the Deaf. "Three years of inventive, collaborative work have created a tool I am eager to see used to open new horizons for deaf students," Wilbur said. Adamo-Villani credits the university's investment in information technology and its emphasis on internal and external collaboration for making the new instructional tool possible. She and her students from the Department of Computer Graphics Technology have been working with the Envision Center for Data Perceptualization; Information Technology at Purdue; the College of Liberal Arts' Department of Speech, Language and Hearing Sciences; and the Indiana School for the Deaf to develop the virtual reality program.