Skip to content using Screen Reader
What it would Take
Home | Back to How To List

Designing Games for Screen Readers

Different Screen Readers Behaving Differently


As noted in Screen Readers and Games, there are at least six major screen readers used by gamers.

Though they behave similarly when reading block text, spread sheet cells, and button face text, they behave differently with regards to more advanced controls and navigation.

These differences may make your game playable with one screen reader but not with others.

For example, the Smugglers 4 game we refer to here can be played with Supernova but not with JAWS because of the different ways those screen readers handle virtual cursors.

The design issues to take from this are:
  • Games with more basic controls and game objects are better candidates for screen reader accessibilty
  • Test your games against multiple screen readers, depending on your markets

Screen Reader Hello World


Using any language: C++, Java, Basic (caveat: see Coding Languages below) write a HelloWorld-ish program that displays:
  • A single button which has text on its button face
  • A single block of a few sentences of text
Ensure that both of your controls can receive focus and that the focus is set to the button when the program first executes.

Turn on a screen reader and run your program. The screen reader should at a minimum speak the Window title. It may also:
  • Tell you the program has started
  • Speak the button text
Tab between the two controls. The screen reader should speak their text.

Speaking Non-Displayed Text


Screen readers normally speak only what is displayed on the screen. However, there are ways to get a screen reader to speak text you can't see.

This can be useful for speaking:
  • Controls or game artifacts that have images but no text displayed
  • Additional information for blind gamers only
  • Text that is organized specifically for audio presentation

Hidden Text


One method for implementing hidden text is to display the text in the same color as its background. This method has been used on Web pages for years.

The trick is not to let the hidden text deform your display.

Talk Directly to the Screen Reader


It is possible for a program to send speech directly to some screen readers without that text being visible on the screen.

Programs written in Java using the Swing GUI toolkit can send text directly to JAWS, without displaying that text. Our games do this. So does Lone Wolf from GMA Games.

JAWS also has an extensive and well supported scripting language that can be used to speak directly to that screen reader.

The following Java snippet shows how we write text to the Accessibility Context of a JLabel.

Some of the subtleties are:
  • We break long strings into chunks that the JAWS buffer can handle.
  • We do this in a thread separate from the main Swing event thread, to avoid blocking the UI.
  • We put that JLabel on a panel that always is visible. Else this doesn't work.
  • The text being spoken is not the text displayed on the otherwise innocuous JLabel. It's the text we're writing to that JLabel's Accessibility Context.
/////////////////////////////////////////////////////////////////////////////
public void speakJAWS(final String strText){

 if(strText.length() < 256){
  m_jlblJAWSDummy.getAccessibleContext().setAccessibleName(" ");
  m_jlblJAWSDummy.getAccessibleContext().setAccessibleName(strText);
  m_jlblJAWSDummy.repaint();
 }
 else{
   final String[] strLines = 
    KTextHelper.getParsedStrings(strText, 250, " .,");
   Thread worker = new Thread(){
    public void run(){

     for(int i=0; i < strLines.length; i++){

      try{
       Thread.sleep(500L);
      }catch(InterruptedException ie){}

     m_jlblJAWSDummy.getAccessibleContext().setAccessibleName(" ");
     m_jlblJAWSDummy.getAccessibleContext().setAccessibleName(strLines[i]);
     m_jlblJAWSDummy.repaint();

     }
    }
   };
   worker.start();
  }
 }
However, Java isn't supported by all major screen readers; and even C++ would likely have to be coded to each individual screen reader. So directly speaking to screen readers is likely not a good option.

Best to use displayed text.

Linearization


People who are sighted scan a screen for context. Our attention jumps around the display. It's very much a direct-access process.

People who are blind are used to getting their input from a screen reader in a linear fashion. Audio is linear.

Some implications of this for your design are:
  • Put the important stuff first
  • Verboseness is bad
  • Enable the gamer to interrupt an utterance

Memorization


People who are blind operate their computers by memorization. They memorize:
  • Key assignments
  • Traversal order
  • Audio clues as to context
When you're designing the audio portion of your game, particularly the controls, ask yourself, "How easy is this to memorize?"

Making Buttons Speak


Screen readers can't speak a button if it has no text.

Put text on your button faces and the screen readers we mention here should speak each button as it gains focus.

However, radio buttons need a speaking group identifier. See Labels below.

Toggle buttons and check boxes should cause screen readers to speak their selection status. Best to test this against your specific target screen readers.

Retro Fitting Buttons


Niels Bauer Games was in the process of coding Smugglers 4 to have buttons without any text. A beta tester who is blind noted that this made the game unplayable.

Niels Bauer added a Blind Gamer option that put text on top of those buttons. Problem solved.

See the Smugglers 4 screen below with the blind option turned on. Compare it to the Smugglers 4 screen further down this page with the blind option turned off.

Smugglers 4 screen with blind option turned on

The major design implication is that for some games simply adding (perhaps optional) text to buttons, and possibly other controls, is sufficient to make that game playable with a screen reader.

Making Other Controls Speak


Lists, drop downs, text entry fields, spinners, sliders, and more esoteric controls may or may not work with specific screen readers.

Lists and drop downs shouldn't be a problem. The screen reader shold speak each item as it gains focus. However, see Labels below.

Text entry fields are normally identified as such by screen readers.

Spinners and sliders are spoken differently by different screen readers.

For custom controls, your best bet is to make some text artifact visible and test this against your target screen readers.

Focus Traversal and Speaking


If you're just mousing about, it's easy to totally overlook traversal order, or even leave some controls inaccessible by keystroke. When you first try to navigate by keystroke and sound, it can get embarassing very fast.

For your screen reader accessible games, it is absolutely vital that:
  • All controls be reachable by keystroke
  • The focus traversal order meet user expectations
Typically a gamer who is blind will expect the Tab key to make the major jumps and the left and right Arrow keys to make the lesser jumps.

You could also use the up and down Arrow keys and the Page up and down keys for traversal.

Western gamers expect focus traversal to be from left to right, top to bottom. Remember, your sighted gamers will see this.

The important design element here is to implement a complete, consistent, and familiar means of traversing your controls.

Making Labels and Group Boxes Speak


Your game probably includes labels. They may be fancier than the plain text used in business apps, but they're still little bits of text that guide your player. Often they are associated with a control, such as a list.

Your game probably includes group boxes, titled or plain.

Gamers who are sighted can visually link your labels and group boxes with the controls or other artifacts as you intend. For screen reader accessible games, you may have to do this explicitly.

One way to at least have your labels speak via screen readers is to have them receive focus.

Group boxes, even with titles, can not receive focus in most coding languages. And different screen readers may or may not speak titled group boxes even if they do receive focus. Possible solutions include:
  • Add a separate label adjacent to the group box
  • Simply deny that information to gamers using screen readers

Screen Readers and Hotkeys


Screen readers make extensive use of hotkeys for navigation and speaking text.

Do not take away screen reader hotkeys!
  • Don't use the hotkeys for your games
  • Don't turn off the hotkeys

Some hotkeys are generally common among screen readers.

Function JAWS Supernova
Say all Insert+DownArrow ?
Say next line DownArrow NumPad6 or F8
Say next Word Insert+NumPad5 Numpad5 or F7
Say next Character RightArrow NumPad4 or F6
Interrupt speech Ctrl LeftCtrl
Repeat speech Insert ?

Hotkeys for a single vendor may differ between desktop and laptop computers. This is true for JAWS.

Supernova offers not only its own hotkey set, but emulates the JAWS, Window-Eyes, and ZoomText hotkeys as well.

Most screen readers enable the user to reconfigure their hotkeys. For some single show-stopper, a gamer might reconfigure their screen reader's hotkey in order to play your game. Better for you to just code to avoid conflicts.

You can get full lists of hotkeys and their options from the above vendors at their Web sites. Typically, hotkey assignments do not change when a screen reader vendor issues a new release.

Using Hotkeys


In addition to buttons, you can also use hotkeys to trigger actions. Jim Kitchen and Ian Humphreys both use this technique effectively in their games. (See Examples.)

Hotkeys and Gamer Expectations


Consistency is especially important to the gamer who is blind because they can't visually detect anomalies in your user interface.

In our games, the F1 key always displays Help, the F2 key displays blind-specific help, and all of our other hotkey assignments are consistent across all of our games.

The design element to remember for gamers who are blind is that they are going to memorize your hotkey assignments. You make it easier for them if you use a consistent and memorizable assignment schema.

Hotkeys vs. Traversal


For gamers who are blind, sometimes it's better to trigger a control directly via hotkeys. Sometimes it's better to traverse into that control and trigger it via the Enter key.

For gamers who are blind, good candidates for hotkeys are controls that are used individually including:
  • Status controls, such as Time Elapsed, Score, Level
  • Immediate need controls, such as Fire Lasers
  • Game-level controls, such as Help, Main Menu, Options
Game Objects are often good candidates for traversal, for example:
  • Individual words in a word game
  • Individual planets on a space travel screen
In our Scrambled Sayings screen below, the F4 key speaks the Score, the F5 key speaks the Time, the F6 key speaks the game Level.

The Tab key starts on the first word, then traverses the game controls on the right, then returns to the first word.

Scrambled Sayings game screen

Making Block Text Speak


By block text we mean descriptive or dialog paragraphs.

Screen readers enable the user to speak a block of text in its entirety, or by line, word, or character. This is the easiest thing for screen readers to do, and generally costs you nothing.

If your block text is indeed text. If it is a bitmap rendered image of text, your gamer's screen reader won't be able to read it.

Bolding, italics, and font sizes are not normally heard in a screen reader. So any information you intend to convey with these visual artifiacts will be unavailable to the gamer who is blind.

Headings


Users who are blind make a major use of headings to organize the data they receive.

Screen readers signify headings when they read Web pages, and sometimes in word processing text.

It's unlikely that a screen reader is going to identify a heading as such in your game. However, you could use strings followed by colons, as in "Topic:" to indicate headers.

The following text annotations to indicate structure are familiar to users who are blind:
  • Three asterisks
  • Three dashes
  • Slashes
  • Vertical Bars

Punctuation


Punctuation is important to screen readers.
  • End sentences with the proper punctuation
  • Use punctuation to indicate short pauses in an utterance
  • Use punctuation to indicate structure, as in using a colon at the end of a heading
  • Punctuation is not necessary on button faces or other controls

Sound Gaps


Sometimes an utterance is clearer when it contains brief pauses. Think of how you speak to empahasize things or focus attention on something.

You can use the same technique in your game by:
  • Breaking a piece of text into separate sentences
  • Using punctuation
The design point here is that you want your text to sound clearly as well as read clearly.

Making Help Text Speak


Help text is a special case of block text. Structure, especially for FAQ type help, is especially important to users who are blind.

One technique we've found especially useful is to user HTML for help text.

That's because a screen reader can grab the headers and organize the information.

Making Help Systems Speak


Help systems such as Microsoft's or the JavaHelp system may bring their own challenges to screem readers.

We don't use either technology in our games and so don't address them here.

Making Game Objects Speak


Games often have canvas play areas with game objects located on them. For example, see the Smugglers 4 screen below.

Smugglers 4 game with blind option turned off

Some canvases are rendered simply as bitmap images, especially if double buffering is used for performance reasons.

Other canvases might have individual game objects on them.

Some screen readers implement a "cursor" or "virtual cursor" specifically to address this kind of issue. However, only Supernova seems able to make this work for games.

The design implications are to make each game object:
  • Actually an object, not just a part of the canvas bitmap
  • Have text, either visible or hidden, that a screen reader can speak
That being said, games that emphasize game objects on canvas play areas, especially if those game objects are numerous and moving, generally are not good candidates for blind accessibility.

This is probably your most important design decision with respect to screen reader accessibility:

Can this game be made blind accessible without changing the fundamental nature of the game?

Tables and Headers


If your game includes tables, make the column headers screen reader accessible.

For a gamer who is blind, not making your headers speak is like giving someone only one half of a sports game score - pretty useless.

Normally, a screen reader will treat column headers the same as data rows, either speaking individual cells or speaking all cells in a row.

Don't use just images for table headers.

In the Smugglers 4 Trade Screen below, when you use the Arrow keys to move the JAWS cursor over the topmost data row, JAWS speaks the entire row. That is, you hear "Supplies dollar one hundred eleven one hundred eight zero."

When you UpArrow to the column headers, JAWS speaks "Name Price Stock You."

Linked Controls


Sometimes a game will have controls or game objects linked. That is, an action on one control will have an effect on another part of the display.

An example is the table in the Trade Screen dialog box from Smugglers 4 below.

When a sighted gamer mouses a cell in the table, they can see values associated with each planet change on the Space Screen in the play area canvas behind that dialog box.

A gamer who is blind can't see those values. Fortunately, they are not essential to the game.

Trade Screen on top of main play area canvas

The design element to glean from this example is, make your linked objects speak.

Making Keystrokes Speak


People who are blind expect their screen readers to speak each keystroke.

Don't code your game such that it prevents screen readers from doing this.

Tooltips and Screen Readers


A screen reader's virtual cursor might trigger tooltips, because the Arrow pointer is moving about per the gamer's Up, Down, Left, and Right Arrow keys.

However, the screen reader won't read the tooltip's text.

Furthermore, if you use balloon tooltips, particularly if they persist, then the balloon may prevent the screen reader from reading stuff you really want your gamer to hear.

If you must use tooltips, the the best practices for screen reader accessibility are:
  • Don't put anything essential in your tooltips
  • Don't use balloon tooltips

Speech Speed


Sighted people generally read at a rate of about 150 words per minute. People who are blind read at up to 300 words per minute, or faster.

Screen readers can be set to speak at these higher rates. Screen reader users generally use this feature.

An interesting design implication is that if you have a gamer who is sighted playing with a gamer who is blind, the blind gamer may be able play considerably faster than the sighted gamer; especially if the game has large blocks of text.

Interrupting Speech


Our colleague, Dark, an expert in audio games says:

"...being able to interrupt long descriptions, ---- especially of controls, is very much recommended, since there's nothing worse than having to sit through a tutorial explanation of what control X is or what the game is about when your playing for the tenth time."

Screen readers let the user interrupt utterances. For example, JAWS uses the Ctrl key to do this.

Skipping Speeches


When your gamer is traversing quickly through a set of controls or game objects, it is convenient to not hear each item.

In some games, this is just an aesthetic or user experience issue. In twitch games or game situations such as a battle, this can make a difference as to whether the game is playable at all.

Screen readers generally include this feature.

Third Party Software


If you're using a game engine or other third party software in your game, then that third party software might not work with screen readers.

Simply something to be aware of.

Using Non-Speach Sounds to Maintain Context


Sometimes non-speech sounds can augment screen reading. They're shorter than speech and add variety to your user interface.

For example, in linear collections or in grids, it's useful to tell the gamer who is blind when they've reached the end of that collection or the edge of that grid.

We use the Perkins Bell, the sound a Perkins School for the Blind brailler makes, to indicate the end of a collection or the right edge of a grid.

We use other sounds to indicate the start of a collection when traversing backwards, or the other edges of a grid.

Consider non-speech sounds in your design.

Screen Readers and Coding Languages


The language your game is coded in does matter to screen readers.

Generally screen readers can read text written by programs written in C++, C, C#, and Basic.

This includes button faces and other controls, as well as block text.

However, the design issues discussed here still apply.

Screen Readers and Flash


Screen readers have problems reading some programs written in Flash.

These are associated with a screen reader interacting with the flash controls themselves, especially if those controls use Flash's advanced graphics display features.

We've heard that JAWS has particular problems with those and that Supernova's Flash compatibility mode is not always reliable.

That being said, Adobe has made considerable efforts to make Flash accessible to screen readers. See:

www.adobe.com/accessibility/products/flash/
http://webaim.org/techniques/flash/

Screen Readers and Java


Not all screen readers can read text written by programs written in Java.

There are two primary Java GUI toolkits, Swing and SWT. Swing requires installation of Oracle's free Java Access Bridge in order for text it writes to be readable by screen readers. And those screen readers must have code that accesses the Java Access Bridge.

Our tests indicate that Window-Eyes does not work with programs written in Java using Swing. It's been suggested that Java programs written using SWT are readable by Window-Eyes, but we haven't tested this ourselves.

As of January 2011, our tests suggest that JAWS does not work with any programs written in Java using Swing and running on any 64-bit Windows system.

This appears to be due to JAWS inability to work with the Java Access Bridge in a 64-bit Windows environment. It does not imply that JAWS does not work with other software in that environment; for instance, programs written in Java using SWT.

Supernova, NVDA, ZoomText, and VoiceOver appear to have no problems with programs written in Java.

The implication for your game design is that Java probably is not the best choice for a screen reader accessible game.

System Interactions


Some screen readers may cause the Windows cursor to flicker rather than do its normal blink.

Some screen readers, such as Window-Eyes, require the Windows cursor in order for their own virtual cursor to work. Your game may have screens that totally suppress the Windows cursor.

Screen Readers and Braille


Some screen readers such as JAWS, Supernova, Window-Eyes, and VoiceOver, can output to Braille devices.

These are used by some people who are blind and people who are both blind and deaf.

We don't address the design implications of Braille output here.

Screen Reader Artifacts


Some screen readers will tell you when a program is starting. Window-Eyes does; JAWS does not.

Some screen readers will tell you their configuration status each time they read a screen. This can be distracting.

For example, when reading text, JAWS may say "Text dash read only edit insert F1 help."

It's generally possible to turn this off via the screen reader's controls.

Windows Title Bars


Most screen readers will always speak the Windows title bar.

Though they may add artifacts such as the word "Frame" or "Form one", or similar speech.

Sound Crud


Screen readers often speak little bits of text they get from Windows or your app.

For example a button gets focus and the screen reader speaks something about "image" and a number.

A screen reader may speak spontaneously. My copy of Window-Eyes randomly mutters "I am starting. Zero." while my PC is running idle with the focus in Windows, not an app.

People who regularly use a screen reader are used to this behavior. There's little you can do about it.

Scripting


Some screen readers implement their own scripting language.

Many screen reader users are familiar with using these scripts.

For some screen readers, notably JAWS, there is an extensive ecosystem of script coders.

This could be used to integrate your game with a screen reader. Of course, such integration would be limited to just that specific screen reader.

Screen Resolutions


People who are blind often have small screens. Resolutions of 800 by 600 are still common in this market. Netbooks with smaller resolutions also are not uncommon.

Where this impacts screen readers is that if your game has a minimum resolution of 1024 by 768, a screen reader may not be able to see your text.

Background Music and Sounds


Background music makes it hard to hear screen readers.
You should make it possible for your gamer to turn off background music.

However, incidental sounds may be vital to gamers who are blind. Don't make the mistake of turning off incidental sounds with the background music.

Final Words


Games that are not good candidates for screen reader accessibility include:
  • Games with many moving objects
  • Games with strong visual elements
  • Games with very little text
  • Games that emphasize canvas play areas
Better candidates are:
  • Story games
  • Strategy games
  • Word games
  • Card games
  • Some puzzle games
  • Time management games
However, even for a game with a canvas play area, the Smugglers 4 example demonstrates how a simple change can make that game screen reader accessible.

For more on screen readers and games, go to our Top 25 Web Sites for Gamers who are Blind page, updated annually.

John Bannick
Chief Technical Officer
7-128 Software

jbannick@7128.com