Hi all,

For folks who have joined us recently: welcome to The Inside Story of NVDA series where a seasoned NVDA developer offers a backstage pass to inner workings of NVDA screen reader. Because we are talking about software internals, these posts can become geeky quickly, so please hold on.

This is the last part of a three-part series on NVDA objects. So far, we talked about object names and roles and anatomy of NVDA objects. This post will talk about API and overlay classes and some last minute remarks on NVDA objects. I might stretch this a little bit and talk about one or two behavior classes, and I can go into progress bar output and suggestions announcements in a future post (talking about all sorts of NVDA behavior classes such as table navigation, dialogs, progress bars, and others will take an entire post).

To recap: an NVDA object is an abstract (idealized) representation of a single GUI control. An NVDA object by itself will not provide useful information to you, nor offer a way to help you interact with a given control effectively. This is where it relies on more concrete implementations such as accessibility API objects to provide the actual mechanism behind things such as object navigation, focus and foreground announcement, among others. While talking about the anatomy of an NVDA object, I did show an example of what I mean by “abstract class” – how NVDA can obtain object location in various ways (three were shown: base NVDA object, IAccessible, and UIA), and I ended with class inheritance tree for NVDA objects. If you want to learn more about things I just talked about, click the “anatomy” link above (in some ways, you need to read the first two parts of NVDA objects posts because what I’m about to describe builds on top of them).

Recall that programmers can represent the world in terms of classes and objects, and people define inheritance to make the world a more interesting place. Simply put, inheritance is where a custom object or a class can derive power from a base (parent) class, with the custom class being able to edit or replace parts of the parent class for various reasons. For example, a base class can define a “name” method, which can be customized by a class that inherits from the base class to offer additional details. For a quick example, see object location example from the previous Inside Story post.

NVDA is no exception. If only the base NVDA object (NVDAObjects.NVDAObject) was offered, NVDA would not be flexible. No custom commands for apps, no way to obtain label and role for a custom control or two, no automatic reading of dialog text, no browse mode… you name it because only one way of doing it will be offered. But remember that NVDA object is a class, and Python, offering object-oriented programming as a paradigm, will let you customize how you can obtain control label and roles. And this is where API, behavior, and overlay classes come in (I am including behavior class here since add-ons do come with overlay classes derived from behaviors such as progress bars).

API, behavior, and overlay classes can be thought of as layers on top of the base NVDA object. The first layer is windows and behaviors. In Microsoft Windows, controls have what’s called a “window handle,” used to distinguish one control from another (you may see documentation saying “hwnd” to refer to window handle). To house window handle and other properties, a window class (NVDAObjects.window.Window) class is defined on top of the base NVDA object. This is necessary because without a window handle, NVDA:

• Cannot distinguish GUI controls
• Cannot create nor interact with accessibility API’s and resulting API classes such as IAccessible

In addition to the base window object, NVDA comes with a collection of window objects for things such as working with Excel. These objects do not necessarily use accessibility API’s for various tasks – the ones that require accessibility API’s are grouped under respective API directories. Window objects mostly rely on properties such as window handle and class name to inform NVDA as to what kind of control it is dealing with.

Another class (or a collection of classes) immediately on top of the base NVDA object is behaviors. Behavior classes include table cell representation (and consequently offering table navigation commands), progress bar, dialog, editable text, among others. These are designed to let NVDA behave in certain ways – for example, playing progress bar beeps. I can go into behavior classes in a future post (upon request); for now, we will focus on other classes below.

Now the second layer: API classes. These classes represent various API’s and controls powered by them, including MSAA/IAccessible (NvDAObjects.IAccessible.IAccessible), Java Access Bridge (NVDAObjects.JAB.JAB), and UI Automation (NVDAObjects.UIA.UIA). As I noted above, these require a window handle to function properly, so they inherit from the base window class. The job of these API classes is to supplement the base NVDA object by offering a concrete implementation of things such as name, role, description, and other properties and methods (remember that the base NVDA object will say “not implemented” for many of its properties and methods as it is an abstract class) by consulting the actual accessibility API features. Once defined, API classes offer a way for NVDA to deal with controls powered by that API, as well as offering common services for use by overlay classes (described below). For example, how NVDA obtains labels (names) for controls differ between IAccessible and UIA, and any UIA class can use UIA ways of looking up control labels whenever needed without defining their own ways (because the base UIA NVDA object does the heavy lifting; you can override how labels are “defined” in an overlay class if required). API classes are not meant to be used directly in most situations – in most cases, one would inherit from these classes as appropriate depending on the type of NVDA object the control is seen as (for example, if dealing with an IAccessible object, one would create a class that inherits from the base IAccessible class); the only time these API classes are used is if you need to edit NVDA source code to add capabilities to these classes that can be used by inherited, or overlay classes.

The next (and final) layer is overlay classes both built into NVDA and defined by many NVDA add-ons, many deriving from a concrete accessibility API class. As the name suggests, an overlay class is an “custom overlay” – designed to help NVDA in a specific situation. For example, an overlay class named “SysListView32” (deriving from IAccessible) offers table navigation commands in places such as parts of Task Manager, and a class named “SuggestionsList” (coming from UIA) announces suggestion count in apps such as Windows 10/11 Settings app. What makes NVDA very flexible is that add-ons can define their own overlay classes – a good example is how NVDA can work with various Mozilla apps and controls after installing Mozilla Apps Enhancements add-on.

The technical way of saying all this is that an overlay class is a customized or an extended API class, offering custom ways of doing certain things. All NVDA cares about is that it can do its job of announcing labels, reacting to events, and doing all sorts of things without knowing what’s going on under the hood, and that’s the user experience side of NVDA objects. At the code level, overlay classes can extend or replace services offered by the base class whether it is an API class, a behavior class, or in at least one add-on, base NVDA object. In other words, overlay classes can shape how NVDA comes across to users, more so if add-ons define overlay classes of their own.

To demonstrate the power of overlay classes in shaping NVDA’s behavior and user experience, consider StationPlaylist add-on and how NVDA offers table navigation commands in tracks list. As part of this add-on, an app module for StationPlaylist Studio (splstudio) is defined, and inside this app module is an overlay class that offers an abstract view of a track item. The track item class (appModules.splstudio.SPLTrackItem) derives from a long chain of IAccessible classes:

• SysListView32 list item (NVDAObjects.IAccessible.syslistview32.ListItem), offering table navigation commands for certain lists powered by:
• Fake table row and navigation (NVDAObjects.behaviors.RowWithFakeNavigation), defining table navigation commands for rows that are not really tables (as far as NVDA could see at first glance), but that’s not all:
• Base IAccessible/MSAA services (NVDAObjects.IAccessible.IAccessible)
• Window (NVDAObjects.window.Window)
• Base NVDA object (NVDAObjects.NVDAObject)

Ultimately, table navigation commands in SPL’s tracks list come from SysListView32.ListItem. But the track item class inside SPL app module overrides certain methods for obtaining column content, changing the way NVDA will obtain cell contents. This is all handled by a single overlay class coming from an add-on (now you see the power of overlay classes; a word to add-on authors: your code will affect how users think about NVDA to some degree). Or, to put it simply, what overlay classes do is change NVDA’s behavior to suit the situation at hand, and this is all possible thanks to abstraction, inheritance, and polymorphism.

As a final thing, how does NVDA know about “special situations?” This is done when constructing an NVDA object to represent the control you are on. First, NVDA will determine which API class needs to be used. Then NVDA will find out if it is an object it knows about (built-in overlay classes by calling findOverlayClasses for the chose nAPI), followed by asking the app module representing the app the control is part of, then iterating through loaded global plugins to see if they know about the object (for the app module and global plugins, chooseNVDAObjectOverlayClasses is called). NVDA can also ask the app module if the object must be customized further such as using a different role (event_NVDAObject_init). If the resulting object turns out to be an overlay class (not an API class), then it can be considered a “special situation,” otherwise services provided by the accessibility API’s will be used directly because the resulting NVDA object is a pure API class (no customizations).

This concludes the three-part series on NVDA objects – what they are, how they work, what’s inside, and how it is customized. The key takeaway from this series is that NVDA goes great lengths to offer a common user experience across accessibility API’s, with NVDA thinking about special situations all the time. In some ways, knowing about NVDA objects, its anatomy, and the purpose of overlay classes is the minimum knowledge required to create effective add-ons, especially when designing app modules.

Hope this helps. Questions and comments are appreciated.

Cheers,

Joseph