1. Messages
  2. Will NVDA eventually use AI for better GUI recognition?

Re: Will NVDA eventually use AI for better GUI recognition?

Luke Robinett <blindgroupsluke@...>
 

I’m good at coming up with great ideas but that doesn’t necessarily mean I have any idea how the heck to make them happen LOL. All joking at my own expense aside, I would think this kind of information already exists in some API somewhere? AWS perhaps? Like I say, apples voiceover screen reader is already doing this so the tech is definitely out there somewhere.

On Jan 12, 2021, at 4:22 PM, Pranav Lal <pranav.lal@gmail.com> wrote:

Hi all,

As others have mentioned, the problem is the model. We need someone to compile data on what user interfaces look like, probably at a control level and then determine how to interact with it.

Suppose there is a non-standard edit box, NVDA would need to know that there is an edit box and then would also need to interact with it. It may not get the relevant events from it so it is hard to say how good such a feature would be but this is very possible to do. Does anyone have a database of pictures of windows controls?


Pranav
Join nvda@nvda.groups.io to automatically receive all group messages.