Re: Screen Refresh


 
Edited

Hi,

Here's why:

Suppose a screen reader went great lengths to obtain content for whatever page you are browsing. Now suppose that, somehow, the screen reader forgets what it was looking at when you switch screens (windows/apps). If the screen reader switches back to the web browser you were using, it doesn't remember what the page "looked" like, so it goes great lengths again to obtain document content.

Now suppose that you are browsing multiple websites, perhaps using tabs to do so. Because the screen reader does not remember content you were looking at whenever screens (or tabs) are changed, the screen reader must figure out where it is, whatever it is looking at, and ask the web browser for browse mode content. Suppose this happens so quickly that you notice the screen reader is slow to respond, and a bug is filed about it.

The task performed by a screen reader to obtain browse mode content is, as a Mozilla engineer put it, "swallowing the document". That is, in order for NVDA to obtain information from a web document and convert it into a suitable representation, it needs to ask the web browser for raw document content. This takes time depending on how responsive the web browser is to requests from other apps (including NVDA) and the length of the document. It goes something like this:

  1. User (to web browser): I want to browse to a particular website.
  2. Browser (to user, after a pause): okay, here is the document you have requested.
  3. NVDA (to web browser): but wait, the user is blind, so I need a way to present the document in a suitable way.
  4. Web browser (to NVDA): what do you want me to do for you?
  5. NVDA (to web browser): I'm going to send a messenger (no joke), and you are supposed to send the raw document (complete with HTML and other code) to them.
  6. Web browser (depending on the browser the user is using): okay, I will wait for you (old browser)/hey, I gave you all the tools you need (accessibility API's for instance), so why are you asking me to give you raw content (new browser)?
  7. NVDA messenger (to web browser): here I am, give me the raw document content please.
  8. Web browser (to NVDA messenger, after gathering the raw document content in a suitable form): here it is.
  9. NVDA messenger (to NVDA): okay, I did what you've asked.
  10. NVDA (to NVDA messenger): well done, I will tell my user about it.
  11. NVDA (to user): Loading document...
  12. NVDA (to user, after the document is formatted): here it is (then starts reading from the top if the user says so).
  13. NVDA (to self): phew, that was a long trip. I don't want to go through this again. Wait, I have a notebook which I can use to jot down document content.

The bottlenecks are the messenger and subsequent document formatting steps.

Technical: the "messenger" in this interaction is a piece of NVDA called Remote Helper, a DLL called upon to inject itself into other processes (programs) for various tasks. An important task this DLL does is ask web browsers for raw document content, which is then passed onto NVDA so it can convert it into a browse mode representation. Newer web browsers are concerned about code injection, so they do their best to provide the web document in a documented way, and one such effort is IAccessible2. In theory, using an API to access web document should eliminate the need for document refreshing (NVDA+F5 in browse mode) whenever content changes, which is different than screen refreshing in JAWS (JAWS key+Escape).

Hope this answers a lot of questions.

Cheers,

Joseph

Join nvda@nvda.groups.io to automatically receive all group messages.