GranpaDream: Unity3D + Vuforia Engine AR app
• Integrated Media Design Course Project.
• #Unity3D #Vuforia Engine
“GrandpaDream” which is an augmented reality (AR) app helps elderly people play the piano even if they have never done so before. “GrandpaDream” utilizes a mobile phone or iPad camera, combined with the Vuforia Engine in Unity3D to recognize where, in real-time, to superimpose a 3D ball or dot on a piano key in order to play a melody. The user in the video played an old Taiwanese song from the 1970s, which was not only nostalgic for him but allowed him to have a new experience he had never had before, even at old age. This is a kind of technical creations that could benefit the elderly’s physical health.
DESIGN
During my four years at the Taipei National University of the Arts, I attended the Integrated Media Design class where students were asked to divide into groups to conduct a survey of elderly people about their regrets that came from not realizing a dream. Then each group had to collect information from the interview and identify key ideas to make a design that meets the elderly’s needs by using Unity3D combined with AR technology.
At first, we interviewed one of my classmates’ grandfather and heard him talk about his life experience. From his words, we knew that he had worked in YAMAHA and done some woodworking to make pianos for around ten years before his retirement. He let his children learn piano for the love of it although he had never learned it before because he came from a needy family in the 1960s. During that time, children had to work long hours to raise themselves instead of taking art classes. Until now, he has not received any training to play piano.
Figure 1. The diagram shows the key ideas we have selected from the interview.
After the interview, we discussed what we were going to design and used simple materials such as paper, markers and tape to create a prototype of the AR app. The idea was to make a live instruction app that could instruct people to play a melody in a short time without any understanding of music theory.
Figure 2. The animation of the prototype performs how “GrandpaDream” AR app would work.
Then I uploaded the actual piano keyboard image which I wanted for the AR app recognize in the real world to the Vuforia Developer Portal website to test whether this picture was augmentable or not. The piano picture shown in Figure 3. was rated four stars which meant that this perspective of the piano was easy to recognize.
Figure 3. Press “Play” button on the top right corner of the page to download GrandpaDream AR app. Then focus the mobile phone’s camera on this image and adjust perspective view until a blue dot appears on the screen of the smart phone as shown in Figure 4.
