A Portfolio by Theodore Teo
A Portfolio by Theodore Teo
In current times, there is usually difficulty with motivating young children with speech delays/disorders into practicing their speech due to their short attention span. In the perspective of the team, there is a need for more engagement and incentives to motivate these children to practice their speech in their own time, especially at the comfort of their own home. From my own perspective, besides engagement of the potential solution, there also needs to be constant self motivation in order for these children to effectively practice their speech at a constant pace without it feeling repetitive.
Through user interviews and research, it was found that children often struggle to keep up with their speech therapy practice at home. Children with speech-language disorders have difficulty sourcing high quality applications that assist with their speech therapy, and gamification has been reported to be impactful on 'health related interventions' in a positive way (D.Theodoros, 2012). It was also reported that children will be required to be constantly motivated and monitored by an adult to practice their speech, and that gamification can assist greatly with that(B. Ahmed, 2017).
The ideal concept of Talk to the Book is an interactive storybook device that allows children between 3 to 7 years old undergoing speech therapy to practice their speech at home. The device supports the child in their practise of pronouncing particular sounds and difficult words, and is meant to be used in conjunction with the child’s speech therapist. In order to combat the challenge of having to self motivate, Talk to the Book can hold diverse stories (and different characters) that emphasise on different sounds to engage the child and pinpoint the specific kind of sounds they need to practice in their speech, and to ensure their experience in practicing speech remains fresh and new in their mundane life.
Image done by Julian Sunde
The book has a rotating handle that the child can open up like a jack-in-the-box. It opens up in a 90 degree angle where the child can pull a background from a slot and insert it. The magnets from the backgrounds will help clip the background in place to prevent it from falling while the children are roleplaying
The backgrounds are shaped in a way where the top layer is in front while the bottom layer is considered the ground. This allows a more immersive experience for the child and simulate the feeling of a stage play
The backgrounds have different materials to simulate the child’s thinking and encourage tactile feedback and learning
The book has a speaker in it where a voice will tell the story and prompt the child to speak. The story is chosen by the child’s speech therapist according to the child’s needs, and made sure to have specific sounds included many times
The book has a microphone so when the speaker prompts the child to speak, the microphone will register and pass the speech into API to check if the speech is correct
The book has RFID readers to register the characters that the child puts on when prompted
The book would be compact and smaller, so the child could bring it around and practice their speech anytime
The book would automatically store or send to the speech therapist, the recordings and speech related data from the child each time the child practices
Video by Oscar Dunstan
The actual product was displayed in UQ Great Court on the 27th May. Majority of our setup has to do with invoking the atmosphere of a warm home with children in it. We were given a spot under the tree, and we used props requested from UQ Life, such as a sofa, coffee table and plants, to build a feeling of warmth and comfort. A small teepee tent was also borrowed from a kind user, and was set up along with soft toys and a koala chair to further emphasise that this exhibit is intended for children.
Image by Julian Sunde
The actual product is a simulation of the ideal concept, missing a few features that were not entirely important to the overall experience, such as sending data to the speech therapist. It contains RFID readers and characters glued to RFID cards to register them. The prompts and voice were simulated using Unity3D's interface. The book also has two strings connected to it that hold the lid of the box up to 90 degrees, as well as a velvet pocket that is held together by velcro to store the characters.
Reception from the audience was overwhelmingly positive.
Majority of the audience stuck to interacting with the first scene rather than continuing onto the second scene, which is expected, given how slow the voice was and the noise pollution around our exhibit. Our exhibit was close to 2 other exhibits that required music and sound for their products to function.
The device is made out of acrylic sheets that are glued together and spray painted with dark blue paint. RFID readers, Arduino, Neopixel LED lights, Unity3D and Google Speech-to-Text API make up the bulk of the technical implementation. I was not involved with this part as my focus is more on the design of the backgrounds and visual materials.
Most of the demonstration is done through Wizard of Oz testing due to the amount of bugs encountered in the APIs and time constraints.
Clear acrylic sheets were spray painted a deep blue and fixed with hinges. I helped Oscar sandpaper the edges as they were very sharp and not child-friendly.
2 different types of RFID readers were added in the product. One was a card based RFID reader (12 MHz RFID sensor from SparkFun) and the other is a smaller, circular shaped based RFID reader. When the correct card is tapped onto the RFID reader after the prompt, the story will then move on.
Neopixels LED strips connect to the Arduino to enhance auditory feedback – whenever the child does a correct action, i.e says the correct word or puts the correct character on the correct spot, the LED strips will light up a different colour; it could be green, yellow, orange, purple, pink or blue. We are careful not to put red into the LED strips as red is universally recognised as a negative colour.
Our technical team also used Google’s Speech-to-Text API to detect speech from users that were spoken from a built in microphone, since the API has accurate transcriptions and can easily recognise the words spoken compared to other speech APIs, therefore saving time for us to work on getting the other technologies to work.
Our technical team used Unity3D as it helps power and coordinate the LED lights, speech-to-text API and Arduino into a single controllable function. It also allowed us to do a controllable Wizard of Oz simulation through the software interface. The user's speech gets taken into Unity, pass it to Google API where it was transcribed to text, then passed to Arduino and Neopixel LED lights where the device supposedly recognises if the speech is accurate or not.
The product holds 4 completed backgrounds that were glued on car-grade cardboard by me and Oscar to support the backing, and various materials were used to craft the tactile points of the backgrounds that the child can interact with during the story.
I focused largely on making the backgrounds, and made sure that each material gives off a tactile feeling that is accurate to the prompts of the story itself, be it sense of touch or the shape of an object.
I made sketches and notes for the construction of each background. They help inform the decisions of the backgrounds. Team discussions were held to assist with sourcing materials or generate better ideas.
The backgrounds were painted by me using iPad and Procreate. I measured the canvas from the size of the acrylic sheets to ensure that the dimensions and RFID reader placements are accurate. I took serious consideration with the perspective of the backgrounds, and the size was measured to ensure the perspective looks right in a 90 degree angle.
I used white thread to make the spider web. The string was shaped like a web to stimulate a pop up feeling of the shape of a cartoon-ish spider web. Hemp-like thread was used as it is strong even if it's not accurate to real spider silk, so that it is durable enough to reduce damage from children playing with it.
I used raffia wool from Kmart for the hay due to its coarse texture and likeliness of hay. It even smells like hay. PVA glue is extensively applied to ensure the child does not easily tear off the wool.
I glued sheets of foam together with PVA glue and sides of the foam were cut at an angle to look more like rocks. In this case, the softness of the texture matters less and the surface area is more important. If the material had been hard, it would take longer to mould into a desired shape and it might damage the background if it were folded to fit into the box.
I used soft plastic from school folders for the material of the lake; it has a sheem and a clear surface that reflects light much like water. I also used sheets of foam to make the lily pads and the lily, and to enhance the child's experience. Since the child needs to place Simon the swan on the water and the lake itself is painted to be more realistically large, the lily pads and lily serve both as decoration and a deterrent for the child to place the swan in the exact spot of the RFID reader.
Initially, moss bought from Kmart was used for the bush; it looked like a proper bush and the colour was accurate to the original bush. However, the moss stains and has a very unpleasant slimy texture, and therefore had to be discarded. Squirmies and craft paper borrowed from Team Snap, Crackle, Pop! were then used for the texture - it still works well despite the appearance of it as the squirmies are soft, which helps the story since Sofie the koala needed to land on something that's explicitly soft.
Packaging foam, when painted, had a very rough, brownie-like texture, which was used to make lumps of dug up soil. It also helps that the texture is soft, much like regular soil. Another form of foam sheet was cut into tiny pieces and painted to be sprinkled around and enhance the look of the soil as the packaging foam looked more like poop than soil. The rope itself enhances the story by adding another form of optional interaction where Sophie helps the snake up through the use of the rope. Though outside of required speech/tactile interaction, this can bring a little more play to the child's experience with the book.
My team and I have held many discussions to make sure we stay on track on the theme of 'Future Mundane' and 'Enriched Play'. There were certain design decisions that I would like to bring to light:
There were initially divided opinions, as some team members preferred children as our target audience and others preferred adults.
For me personally, I wanted to focus more on adults, since there were more resources and research about speech therapy dedicated to children than adults, I felt that adults also deserve to have better incentivization to practice their speech due to their busy schedule.
We were then advised by Lorna that adults are less likely to try tactile interactions, and for adults who are doing speech therapy, they are more likely than children to see the bigger picture and feel like they have to do it to get better at their speech. Because of that, it would be easy for children compared to adults to incentivize them to practice their speech.
I was finally convinced then, to focus on children, and it turned out to be a great decision in my opinion, as there were a lot more creative ideas explored with incentivization when it comes to children compared to if we focused on adults.
My team and I discussed about whether the storybook is best done in a home setting or in a clinical setting.
All of us decided that the storybook works most effectively as a home practice. Through interviews with users in the field, it was reported that speech therapists usually see their patients once a week, and that home practices are optimal for child’s progress. One benefit of doing it in a clinical setting is that the therapist can monitor the child’s progress while the child is engaged in the book, but the child might not be comfortable with practicing with someone other than their parent(s), and data storage is always an option for home practice for therapists to analyse the child’s progress. Most importantly, we wanted to make a device that makes it more enjoyable for the child to learn at their own pace and not necessarily in a set amount of time in a confined space not of their choosing.
Our team had some divided opinions on screens vs no screens as well. In our initial concept, we planned to have the device be a digital screen that recognises the object. However, all of us had a unanimous vote to abandon that idea as we wanted to increase tactile interaction with children. Blue light from digital screens have been reported to affect brain development in children due to melatonin suppression that can affect sleep (A. Rehman, 2021). In keeping the spirit of learning through play, we wanted to make sure the child has as much of a holistic and healthy learning experience as possible.
Our team thought of making the concept an interactive storybook, since kids love stories. Previously, the team was considering adding high scores as an incentive. I suggested that the incentive of kids pronouncing certain words, moving onto the next part of the story would be more than enough due to their curiosity. This would also be a good way of showing the child’s progress.
Based on the design decisions my team and I made, we got to work on progressive prototypes. My contributions are detailed below.
I made the paper prototype with paper and pencil to user test with a child. I wrote the story in a night due to the time constraints we had with the user's parent's schedule, and it differs from the final story. The prototype consists of character cutouts, 2 backgrounds, and text cutouts, and I tested the child by reading the story out loud and prompting them occasionally.
After the user testing, my team and I made the mid-fidelity prototype for the presentation using a shoebox. For my part, I worked mostly on the physical demonstration of the prototype. I drew the backgrounds according to the dimensions given to me. I also drew characters to put onto the background, according to the size of the RFID cards. Admittedly my passion for art made me overlook that the backgrounds were supposed to already have a more 3D space, and I painted the backgrounds to be completely 2D instead. Since this is a mid-fi prototype however, it still demonstrates the purpose well.
Initially we included text in the story in this prototype, but after feedback, but we took it out and replaced it with colourful LED lights to emphasise incentives more. We felt that text and readability is not as needed to practice speech in general.
We also did a demonstration of our concept in the video below using the mid-fidelity prototype as well as showed the technical functionalities behind the prototype.
The bulk of our research and interviews are focused on speech therapists and researchers who are familiar with speech pathology. In terms of testing prototypes, I managed to test the paper prototype on a child and interview their parent with regards to the design of the prototype and experiences of bonding with the child.
The person I have interviewed is a friend of mine who is currently going through their placement at UQ, and is a student studying speech pathology, and the interview was through a Zoom call. Their interview informed our research greatly when it came to the design process. My team and I came up with general questions that relate to their experiences working with kids with speech issues and some of the practices they’ve gone through, and dove into reasons why kids go through speech therapy in the first place. I took great care to make sure that the questions are not leading and are focused on speech therapy. Some key takeaways from the interview are:
These findings led our team and I to conclude that the device will be primarily used as home practice, having a voice read out the story, figuring out our target audience, and take into consideration the child's mental health.
We had some trouble with finding kindergartens and children who are willing to let us user test; luckily I was able to get a hold of a friend who happens to have a 5 year old child. I made the paper prototype due to the limited window of availability that our user has and did the paper prototype on the child. The focus of the user testing is more on the needs and attention of the child as well as the interactions with the parent and child. Key takeaways from both the user testing and interviews are:
All in all, despite the child not having any speech issues, I felt it was good to test with them to access their attention span and to gauge what the child wants as well as their motivations.
In order to inform more people about our design process and product, we made exhibit materials. For my part, I made the brochure using Affinity Publisher, and designed the stickers made from the characters in the mid-fi prototype using Procreate. I also assisted with the shirt design and made the logo using Procreate.
Exhibit stickers (also from mid-fidelity prototype)
Exhibit shirt design
Chaotic Control Logo
My team has set up 4 success criteria and we met 3 of the success criteria of getting the voices, RFIDs and LEDs to work. There were a few kinks and features that my team and I did not have the time to do, such as sending data to the speech therapist to evaluate the child's progress (which is one of the success criteria) and adding a more playful way to open the box by adding a handle for the child to turn the box open. However, I felt that we demonstrated our actual product well with the success criteria we had, and I am confident that given more time we would be able to fulfill all 4 success criteria and making it into a more well rounded product.
In relating to the overall themes of Skilled Play and Future Mundane, children with developmental delays/disorders who are required to practice speech therapy in their mundane life can look forward to doing so through tactile storytelling in the future, engaging them more than speech therapy exercises given in the present. This encourages them to practice speech therapy more effectively, therefore enriching their own learning through play and be more motivated to do so as a potential future.
I feel that our team largely has met desired outcomes, with the exception of setting up a system where data can be sent to the speech therapist in terms of both design and functionality. However, we did not have much opportunity to test our final prototype due to lack of access to target audience and time constraints. Despite that, in terms of user experience, many exhibit patrons expressed that they were impressed by the design and our considerations in terms of motivating the target users to practice speech therapy, and more such as bonding with their parents. If given the choice, I would still definitely do more user testing for the final prototype to ensure that our product design is of higher quality.
In terms of human values, I felt that our team has taken incentivization and children's mental health into serious consideration by preventing the device from telling the child they are wrong while asking them to repeat, and because of that I feel that the product has met the criteria for the human values generated based on research. A small concern I have however, is privacy issues in terms of microphone. If I had more time I would definitely take that into consideration and do more user testing on the final prototype.
In relation to other work in the domain, I felt that our product was more well researched compared to other work that are less user friendly and less targeted towards children, as we dedicated a lot more time to researching our design decisions and interviews with professionals in the field making sure we checked all the boxes in terms of users' needs.
I’m Theodore Teo, and am a final year UX student at UQ. My technical experience involves game development in Unity3D, web development, VR and AR. I'm also well versed in visual design, user experience design and web design. I enjoy storyboarding and making comics in my free time, as well as learning new skills outside of my field. I have a passion for design research and social work as well.
Check out the Brochure
