Remember when. . . ‘automatic image descriptions’ using rudimentary AI was first available on smart phones? Everyone had a good laugh when those image descriptions described a teen as a ten-year old or a 30-year old woman as a 45-year old person. How things have changed! Fast forward to March 2024 and see what JAWS can do with Picture Smart AI.
Since 2019, JAWS has had the Picture Smart feature – the ability to choose a picture and submit it to be analyzed. Picture Smart announced a caption and a string of words that give basic information about the picture, such as what is in the picture (people, human face, dog), the environment (outside, sky, tree,) and possibly what the objects in the picture are doing (sitting, swimming, reading). Keep in mind that AI uses data to learn and adapt – AI teaches itself and is always improving!
JAWS has just announced Picture Smart AI – which is image descriptions on steroids! The Short Description is typically a short paragraph with concise information. The Long Description has in-depth information and can be multiple paragraphs.
Students immediately think about how they can use Picture Smart AI to view and post photos on social media platforms. As an educator diving deeper into digital math accessibility, my first thought is how can students harness AI-generated image descriptions for educational purposes? The possibilities are endless – and just like AI – the educational potential will continue to evolve!
Math and science classes use charts and graphs to condense large amounts of information into easily understood formats. While text is easily accessed with a screen reader, charts and graphs tend to be more challenging. Can students use Picture Smart AI to instantly view an online or digital graph?
Let’s look at the Favorite Fruit, Pie Chart and the example descriptions from the current Picture Smart and new Picture Smart AI. (Example from Freedom Scientific’s What’s New in JAWS page)
Caption is chart, pie chart. These tags describe the photo: circle, diagram, screenshot, text. This tag probably describes the photo: font. This tag vaguely describes the photo: design.
The image is a pie chart that shows the favorite fruits of a group of people. There are four categories: melon, apple, mango, and orange. The melon category is blue and takes up 19% of the pie. The apple category is green and takes up 19% of the pie. The mango category is yellow and takes up 25% of the pie. The orange category is orange and takes up 37% of the pie.
The image is a pie chart on a white background with a title at the top that reads “Favorite Fruit.” The chart is divided into four segments, each representing a different fruit with a corresponding percentage, indicating the preference of a group of people.
The largest segment is orange, representing oranges, and takes up 37% of the chart. The second largest is yellow, indicating mangoes, accounting for 25%. Two other fruits, melon and apple, each take up 19% of the chart; melon is depicted in blue, and apple in green.
Overall, the chart provides a visual representation of the preferred fruits with oranges being the most popular, followed by mangoes, and then melon and apple with equal preference.
The pie chart example is fairly straight forward. Let’s take a look at the Timeline example. In this example, the Short Description contains all the pertinent information. The long description also includes details on the layout. By default, JAWS will read the short description. The viewer can choose to have the Long Description read too.
Caption is timeline. These tags describe the photo: display, font, screenshot, text. These tags probably describe the photo: diagram, number.
The image is a timeline that spans from September 2024 to May 2025. It is divided into 5 sections, each with a different task and deadline. The tasks are as follows:
– September 2024: Synergize scalable e-commerce
– November 2024: Disseminate standardized metrics
– January 2025: Coordinate e-business applications
– March 2025: Foster holistically superior methodologies
– May 2025: Deploy strategic networks with compelling e-business needs
The image you provided is of a slide with a title “TIMELINE” at the top center. Below the title is a horizontal timeline graphic that spans from September 2024 to May 2025. The timeline is divided into five sections, each with a designated month and year, and an associated objective:
1. “SEP 2024” with the objective “Synergize scalable e-commerce”
2. “NOV 2024” with the objective “Disseminate standardized metrics”
3. “JAN 2025” with the objective “Coordinate e-business applications”
4. “MAR 2025” with the objective “Foster holistically superior methodologies”
5. “MAY 2025” with the objective “Deploy strategic networks with compelling e-business needs”
Each section is represented by an arrow pointing to the right that starts from the month and extends to the right with the objective contained in the body of the arrow. The colors of the image are mostly shades of blue and grey, with the timeline graphic against a lighter background. At the bottom right corner, there’s a page number or slide number indicated by “11,” and just above it, the year “2024” is written as a note. The overall aesthetic is professional and likely part of a business presentation.
“Introducing Picture Smart AI YouTube” video by Freedom Scientific
Want to try Picture Smart AI? It is currently available as an Early Adopter program feature. TSVIs and students, please try Picture Smart AI for educational purposes and provide feedback using the “Send Feedback” form that is available in the Early Adopter program dialog box.
When submitting an image, Picture Smart AI displays a short description by default. To display both a short and longer description, select the More Results link at the bottom of the results window. You can also add the SHIFT key to a Picture Smart command to immediately display both short and long descriptions. For example, INSERT+SPACEBAR, P, SHIFT+F.
Note: AI technology can sometimes make mistakes. Known as AI Hallucinations, these can include describing visual elements that do not exist or making assumptions about what may be happening that are not accurate. Do not rely solely on AI generated descriptions and consider double checking information especially if used for professional purposes.
Picture Smart AI is also available as part of Fusion.
The following enhancements are available with both Picture Smart AI and the current Picture Smart.
To view additional examples of Picture Smart and Picture Smart AI descriptions, go to Freedom Scientific’s What’s New in JAWS page and scroll down to Enhancements in JAWS 2024.2403.3 (March 2024).
By Diane Brauner
Back to Paths to Technology’s Home page