While students at Austin Center for Design (AC4D), classmate Kylie Jack and I co-founded Kites & Ladders, which developed products to amplify the voices of people on the autism spectrum. Working primarily with nonverbal autistic individuals and their families, we were designing tools to support independence, communication, meaningful relationships, and participation in broader society.
I conducted the initial research and synthesis and developed an early version of the Visual Voice product system concept along with a storyboard. While we worked closely throughout the prototyping phase, Kylie focused on developing the hardware prototypes while my emphasis was on testing, visual design, and prototyping a tablet app.
Classmate Eli Robinson and I conducted the initial Contextual Inquiries and Participatory Interviews that seeded Kites & Ladders in fall 2012. We identified key insights, which I used to create a storyboard, a theory of change, and the initial Visual Voice concept to support creative self-expression and communication for nonverbal autistic individuals. Kylie and I worked together to prototype and test the Visual Voice system (which included a wearable biofeedback device, a camera app, and a tablet app), before sharpening our focus on prototyping the wearable device.
Our research centered on how people with special needs use technology in school and beyond. We conducted contextual inquiries to understand the landscape and learn about the scope of “adaptive technology.” From there, we narrowed our focus to autism and conducted participatory interviews with moms of nonverbal individuals on the spectrum.
During contextual inquiries in schools, we observed students using a range of technologies: calculators, computers, specialized communication devices, and more. We were intrigued by the use of an iPad between an autistic student and his aide and focused subsequent research on adaptive technology and autism spectrum disorders.
We conducted participatory interviews with mothers of non-verbal autistic children. Beginning with a journaling activity, we discussed their experiences with technology and moved into hands-on activities that utilized tangible stimuli such as pictures and words to help the participant tell stories and articulate a more ideal future state.
Jacob: A Portrait of Autism
On one of our research outings, we met Jacob. Jacob is like a lot of other 12 year olds, except he doesn’t speak in words that other people understand. He has been diagnosed on the autism spectrum and has Down’s Syndrome. When he was younger, his mom felt like she didn’t really know him—until he started using a camera. Jacob began taking pictures. Lots of them. He started showing them to family members to communicate how he saw the world.
In our interview with Jacob’s mom, she recounted a trip to the store where Jacob flipped his camera into video mode and placed it on the conveyor belt. When the clerk picked up the camera with a puzzled expression, Jacob burst out laughing.
At this moment, Jacob’s mom realized he had a sense of humor.
We transcribed and externalized our research data onto the wall. Then, using a mix of affinity diagrams and visual diagrams and models, we synthesized our research into insights.
Several key insights emerged highlighting how technology enables and motivates connection and self-expression, especially for students with communication challenges.
However, a student’s adoption of a new technology is unpredictable and depends on physical abilities and emotional needs. Parents and educators often prefer mainstream devices that are flexible and don’t differentiate the student with special needs from peers. Plus, others will use the device if the autistic individual will not.
Technology enables users to externalize inner thoughts and creativity.
For a population that is isolated, technology promotes and nurtures social relationships.
Barriers to Entry
Small obstacles and repeated disappointments feel insurmountable in a situation that is already challenging.
Mainstream and wearable devices can support people with special needs without differentiating them from their peers.
Theory of Change
We developed a theory of change that identified how people felt during diagnosis and day-to-day management of autism, the desired future state, and characteristics of products that could help achieve the ideal state.
To guide our product ideation process, we drew from our research and insights to outline Design Pillars. We used these statements to evaluate product and system concepts, and we also returned to the Design Pillars throughout the prototyping process to ensure we were staying true to the empathy-driven research from earlier phases in the design process.
Visual Voice System
We created a storyboard walking through the experience of an autistic individual using the Visual Voice system. Key moments from the story were extracted to illustrate key Visual Voice products.
Wristband: biofeedback device alerts the wearer to emotional triggers and opportunities to share
Camera app: application syncs with wristband data and encourages the user to snap photos from his or her perspective
Photo sharing app: playful app encourages photo editing and sharing and rewards positive social interaction
We created and tested several prototypes for the Harmony wristband. In addition to sketching possible interface designs, we prototyped two wristbands using an Arduino and an iPhone, consulted with an engineer who was bootstrapping open-source biofeedback hardware, and 3-D printed form factors of a design concept.
Visual Voice Application
A boy tried an analog version of the app concept. After taking photos with an iPhone, we printed them on a portable photo printer. He then decorated the photos using markers and stickers. His mom and aide saw a glimpse into his world--his favorite colors and objects, as well as the playful characters he imagines peeking out behind fences.
We tested both hardware prototypes with autistic individuals and their caregivers. To gather additional baseline biometric data, we also conducted tests and observations with both autistic and non-autistic individuals using a commercial fitness tracker.
On the Visual Voice Application side, we held several analog tests where autistic participants were able to modify and share photos manually, mimicking the self-expression and connection components of the system. A cognitive walk-through with designers helped us gain insights into how to improve the interactive app prototype.
Kites & Ladders, LLC
We launched Kites & Ladders, LLC, to recognize and value the experiences of people with autism and to support communication of their unique points of view, which ultimately leads toward independence. Applying user-centered design practices, Kites & Ladders centered on hardware and apps that combine biofeedback and visual communication systems to help autistic people become aware of and express their feelings and worldview to others.
People on the autism spectrum often struggle to communicate when feeling intense emotions or stress. The Harmony wristband uses sensors and biofeedback so wearers can anticipate and express changes in emotional state. Advance notice helps them better understand and respond to sensory and environmental triggers.
Autistic individuals and caretakers continue to express a desire for products like the Harmony wristband and Visual Voice system. However, the cost and technical challenges related to hardware development, the proliferation of wearables, and the relatively small market size led us to shelve the project after completing our studies at AC4D.