Back to Resources

Strategy

Sonitunes (Digital Lines App): University AT Project

This app addresses the issue of how to make digital lines accessible to students who are visually impaired.

SHARE

• Share on Twitter
• Share on Facebook
• Share on LinkedIn
• Share on Pinterest

As we transition to digital education materials, developers are facing new types of accessibility challenges. Traditionally, graphics have been produced tactually; how can students with visual impairments access these graphics in a digital format? There are many pieces that have to come together before new standards are created. Let’s take a look at the process.

Process 

1. Identify the issue
2. Come up with a creative, innovative solution
3. Field test solution with students, TVIs and educators
4. Improve the solution (Repeat steps 1 – 4 as many times as necessary)
5. Pair new solution with traditional tactile training and bridge between the two formats
6. Train TVIs on new solution and the bridge between tactile and digital formats
7. Train students using all formats (tacile, bridge, and digital formats) and then apply the new tech skills into mainstream educational tasks
8. Build a systematic curriculum that grows the new tech skill along with the mainstream educational skill

Example

In theory this sounds good: identify an issue and solve it. Talking with mainstream software developers who are creating new types of technology, I learned that it can often be a 10 year process from the time a robust project – developing cutting edge tech – is started to a full public release. And once launched, the software continues to evolve and be updated regularly.

Let’s take a look at a common task for preschool/early elementary students: Following and creating ‘lines’. Take a look at traditional worksheets for young students. Students are asked to draw a line between two matching items. The line might be from a bird to a nest, three apples and the number 3, or the letter A and an apple. Traditionally, matching activities were put into braille/tactile format for students who are visually impaired. There are literally hundreds of apps that include the same activity; however, these apps,like most preschool and early elementary apps, are currently not accessible for students who rely on a screen reader.

Note: Drawing a physical line teaches students with vision more than just matching – it also provides practice with eye-hand coordination and emerging writing skills. Digital line drawing will also teach additional skills.

1. Identify the issue

Create an accessible digital line.

2. Come up with a creative, innovative solution

Current available options: There are several Android based apps (Feelif and Vital) that are using haptics (vibrations) and sonification (sounds) to enable a student to follow the visual line that has been tagged. iPads do not have the vibration feature; however, there are several ways to make an accessible ‘line’ on an iOS device. In Pages, you can create an accessible bar chart; touch anywhere on the bar to hear the bar labels. (Learn more about accessible bar charts here.) SAS Graphics Accelerator uses sonification to make charts and graphs accessible. (Learn more about the Accelerator here.) For decades, students who are visually impaired have been using sonification for higher math with talking graphing calculators; however, this type of sonification has not been yet been made available for activities appropriate for young students and young students have not been systematically taught how to use sonification. Just like children learn to walk before they run, it makes sense to mirror traditional skills and sequence such as drawing a line before graphing a parabola.

Solution: Create a proof of concept app that enables a young student using a screen reader to locate, follow and create digital lines using sonification. Vertical lines will use pitch (dragging up the line increases the pitch; dragging down the line decreases the pitch), horizontal lines will use tempo (speed of sound increases to the right and decreases to the left) and diagonal lines use both pitch and tempo. Each color will be associated with a different sound. 

The app will include a tutorial, activities to practice creating a line, then applying the skills to create a line between two matching items. Additional academic skills will be incorporated, such as the terms right/left, horizontal/vertical/diagonal, short/long, etc.

Computer Science Team at the University of North Carolina

In a recent post, Hidden Resources: Collaborative AT Projects with Universities, we learned about teaming with university computer science and/or biomedical engineering departments to create accessible educational tools. The Sonified Lines app project was presented to the Serious Games class at the University of North Carolina, spring semester, 2019. A dedicated three-person student team took on the project with support from their professor. This passionate and talented group of students began researching how to create a sonified line while running VoiceOver and began fleshing out an app that would teach the desired skills. 

Hmm, this all sounds good in theory, but will a sonified line really work for students who are visually impaired? Take a look at the demonstration videos! The first video introduces the Sonitunes app and demonstrates how to find and follow a digital line that has been sonified.

The third video demonstrates drawing a line to match two items.

Next Steps

Let us know what you think of using Sonification and of Sonitunes! Do you have ideas on how to improve the app? Additional concepts that can be taught? Additional ideas for teaching sonification?

The semester is over and the three-person team is moving on. One team member (who graduated in May 2019) is volunteering to work on Sonitunes this summer, before starting his new job. Sonitunes is being shared as a proof of concept with interested app developers. Sonitunes may be pitched to another team in the fall as well. Please stay tuned to find out more about Sonitunes!

Thank you!

A big thanks to the dedicated Sonitunes team (Amit Enand, Christopher Burras, and Dasol Lee), their supporting computer science professor (Dr. Diane Pozefsky),and the University of North Carolina’s computer science department for strongly supporting accessibility-related projects!

Related Post

Accessible Number Line App: University AT Project

By Diane Brauner

Event

Multimedia accessibility: The multimodal toolbox approach

Guide

Explaining accommodations to substitute teachers

Guide

Creating Tactile Graphic Images Part 3: Tips for Embossing