Re: OCR: Interacting with items from recognized UI

Kara Goldfinch

Hi Luke,

You can move the mouse to where the virtual focus is by pressing NVDA+numpad slash, then you can press numpad slash to click it.

I've also run into the  same issue as you where enter sometimes doesn't work, so here's a couple of things I also try.

After moving the mouse to the thing you want to  click, press shift+numpad slash twice with a short pause in-between. This locks and unlocks the left  mouse button. This could help if the app expects the button to be held down longer than usual.

Another thing to bear in mind is that sometimes the text that OCR recognises might just be the control's label, and the actual control could be above, below or to the side of it. I've noticed this in Omnissphere for example. The only way round this is to randomly click around the label with golden cursor, possibly listening for a different colour/brightness level depending on how you have it set up. I usually get tired of this pretty quickly, however.

I hope this helps any.


On 10/01/2021 01:02, Luke Robinett wrote:



For inaccessible UIs I’ll often run NVDA OCR on the UI and then press enter or space bar on the UI controls I find in the recognized text such as tabs, menus and buttons. I find this only works in some programs and other times seems to be ignored. I believe the issue in some cases is that some UIs only respond when an actual mouse is clicked on a control. Is there a way to have the mouse cursor move to the UI element corresponding with its label in the OCR viewer and produce a click on that element? If not, is this something I could submit on GitHub as a feature request? It would make the already helpful OCR feature even more useful.





Join to automatically receive all group messages.