Hello,

On Mon, 28 Sep 2020 10:03:12 -0700
"Luke Robinett" <blindgroupsluke@gmail.com> wrote:

With that in mind, it seems like this could be evolved even further. Is the technology available such that NVDA could try to not only recognize text from the app but ascertain the structure of the user interface, perhaps organizing things into form elements like buttons, text fields etc and then presenting them in the same element browser we use for web pages?

Have you heard about Sibiac (
http://www.azslow.com/index.php?topic=372.0)?
I'm personally not a musician, so I haven't tested, but some people
reports great success with it and it more or less does what you've asked
for.

Barring that, perhaps a

more immediately reachable upgrade would be different actions we can
take on items in the OCR result. For example, instead of just the option
to hit enter to invoke something, what about a Context menu that lets
you invoke it or simply position the mouse cursor where that element was
found?

It is possible already. To position your mouse on the recognized item you
just need to press NVDA+numpad Slash to route the mouse to the review
cursor position. After that you can perform any mouse operation.

--
Regards
Lukasz