Topics

NVDA integration in video games


abdul muhamin
 

Hi. Now a days, video games developers are trying to integrate NVDA support in video games like skullgirls and the emulator called retroarch, I’ve seen a file in both the mentioned game/emulator called NVDA controller client.dll, I think its related to the speech during the menues. So my question is, can we also integrate the NVDA to any software or game? I’m sure it requires advance knowledge in programing languages, what do you think?

 

Sent from Mail for Windows 10

 


Sascha Cowley
 

First, this topic does not belong on the main list; it should be on the chat or development sublists.
Second, the NVDA controler client is a programmatic way of developers integrating their software with NVDA. For example, it allows them to send speech or Braille to NVDA to be output. As for weather we can "integrate the NVDA to any software or game", that depends what you mean. There is no technical reason why any proficient developer would be unable to so integrate their Windows program with NVDA. At the end-user side, the answer is not really: integration of this type requires modification of the program itself, while integration via scripting requires some sort of events that NVDA can hook into, which many programs simply do not provide.


Luke Robinett
 

Is there a practical reason for so heavily policing the types of questions people ask here or is it more of a control thing? I mean, dude’s question was NVDA related. Why be such a cop about it?

On Oct 28, 2020, at 6:52 PM, Sascha Cowley via groups.io <sascha.camille@...> wrote:

First, this topic does not belong on the main list; it should be on the chat or development sublists.
Second, the NVDA controler client is a programmatic way of developers integrating their software with NVDA. For example, it allows them to send speech or Braille to NVDA to be output. As for weather we can "integrate the NVDA to any software or game", that depends what you mean. There is no technical reason why any proficient developer would be unable to so integrate their Windows program with NVDA. At the end-user side, the answer is not really: integration of this type requires modification of the program itself, while integration via scripting requires some sort of events that NVDA can hook into, which many programs simply do not provide.


Josh Kennedy
 

if NVDA had real time recognition like voiceover does on iOS 14, then we would have access to a lot more applications including video games. Voiceover uses a Neural Engine and artificial intelligence to do screen recognition, image recognition and description, and real time live OCR and it also recognizes controls based on pixels on the screen and matrices. Making many applications including video games very accessible and applications which did not used to be accessible or now suddenly accessible with voiceover artificial intelligent Neural Engine recognition. it would be excellent if NVDA could implement this feature very soon. I believe that it directly access of the GPU, and based on pixels on the screen and live screen or some kind of image scanning, it recognizes various inaccessible controls directly even if they are just drawing on the screen with pixels, then it turns those controls into accessible controls that voiceover can interpret. If anybody I could do this it would be excellent.


Andy B.
 

I heard that screen recognition wasn’t that great. If NVDA took this route, maybe things would work a little better, but there are also many people out there with computers not able to handle such a thing. If this new voiceover feature isn’t available on an iPhone 8+ because it is too weak, I don’t want to know what it would take on a laptop or computer. In any case, talking flight monitor does implement NVDA controller, etc into it’s code base. Check it out at www.talkingflightmonitor.net. I am one of the devs.

 

 

From: nvda@nvda.groups.io <nvda@nvda.groups.io> On Behalf Of Josh Kennedy
Sent: Thursday, October 29, 2020 12:12 PM
To: nvda@nvda.groups.io
Subject: Re: [nvda] NVDA integration in video games

 

if NVDA had real time recognition like voiceover does on iOS 14, then we would have access to a lot more applications including video games. Voiceover uses a Neural Engine and artificial intelligence to do screen recognition, image recognition and description, and real time live OCR and it also recognizes controls based on pixels on the screen and matrices. Making many applications including video games very accessible and applications which did not used to be accessible or now suddenly accessible with voiceover artificial intelligent Neural Engine recognition. it would be excellent if NVDA could implement this feature very soon. I believe that it directly access of the GPU, and based on pixels on the screen and live screen or some kind of image scanning, it recognizes various inaccessible controls directly even if they are just drawing on the screen with pixels, then it turns those controls into accessible controls that voiceover can interpret. If anybody I could do this it would be excellent.


Gene
 

What do you mean by this sentence?
If this new voiceover feature isn’t available on an iPhone 8+ because it is too weak, I don’t want to know what it would take on a laptop or computer.
Smart phones aren't nearly as powerful as computers.

Gene

-----Original Message-----
From: Andy B.
Sent: Thursday, October 29, 2020 2:10 PM
To: nvda@nvda.groups.io
Subject: Re: [nvda] NVDA integration in video games



I heard that screen recognition wasn’t that great. If NVDA took this route, maybe things would work a little better, but there are also many people out there with computers not able to handle such a thing. If this new voiceover feature isn’t available on an iPhone 8+ because it is too weak, I don’t want to know what it would take on a laptop or computer. In any case, talking flight monitor does implement NVDA controller, etc into it’s code base. Check it out at www.talkingflightmonitor.net. I am one of the devs.







From: nvda@nvda.groups.io <nvda@nvda.groups.io> On Behalf Of Josh Kennedy
Sent: Thursday, October 29, 2020 12:12 PM
To: nvda@nvda.groups.io
Subject: Re: [nvda] NVDA integration in video games





if NVDA had real time recognition like voiceover does on iOS 14, then we would have access to a lot more applications including video games. Voiceover uses a Neural Engine and artificial intelligence to do screen recognition, image recognition and description, and real time live OCR and it also recognizes controls based on pixels on the screen and matrices. Making many applications including video games very accessible and applications which did not used to be accessible or now suddenly accessible with voiceover artificial intelligent Neural Engine recognition. it would be excellent if NVDA could implement this feature very soon. I believe that it directly access of the GPU, and based on pixels on the screen and live screen or some kind of image scanning, it recognizes various inaccessible controls directly even if they are just drawing on the screen with pixels, then it turns those controls into accessible controls that voiceover can interpret. If anybody I could do this it would be excellent.


Luke Robinett
 

I do a lot of music production work and much of that software is very graphical and inaccessible out of the box. People have created some really cool NVDA add-ons in that space that utilize screen recognition and some sort of learning algorithms to give access to user interfaces that are otherwise completely unrecognizable by NVDA. For an example of such a project, check out Sibiac. I think this shows the potential for NVDA to be augmented to do what you described, giving access to graphical interfaces that normally don’t provide hooks for screen readers.

On Oct 29, 2020, at 12:15 PM, Gene <gsasner@...> wrote:

What do you mean by this sentence?
If this new voiceover feature isn’t available on an iPhone 8+ because it is too weak, I don’t want to know what it would take on a laptop or computer.
Smart phones aren't nearly as powerful as computers.

Gene
-----Original Message----- From: Andy B.
Sent: Thursday, October 29, 2020 2:10 PM
To: nvda@nvda.groups.io
Subject: Re: [nvda] NVDA integration in video games



I heard that screen recognition wasn’t that great. If NVDA took this route, maybe things would work a little better, but there are also many people out there with computers not able to handle such a thing. If this new voiceover feature isn’t available on an iPhone 8+ because it is too weak, I don’t want to know what it would take on a laptop or computer. In any case, talking flight monitor does implement NVDA controller, etc into it’s code base. Check it out at www.talkingflightmonitor.net. I am one of the devs.







From: nvda@nvda.groups.io <nvda@nvda.groups.io> On Behalf Of Josh Kennedy
Sent: Thursday, October 29, 2020 12:12 PM
To: nvda@nvda.groups.io
Subject: Re: [nvda] NVDA integration in video games





if NVDA had real time recognition like voiceover does on iOS 14, then we would have access to a lot more applications including video games. Voiceover uses a Neural Engine and artificial intelligence to do screen recognition, image recognition and description, and real time live OCR and it also recognizes controls based on pixels on the screen and matrices. Making many applications including video games very accessible and applications which did not used to be accessible or now suddenly accessible with voiceover artificial intelligent Neural Engine recognition. it would be excellent if NVDA could implement this feature very soon. I believe that it directly access of the GPU, and based on pixels on the screen and live screen or some kind of image scanning, it recognizes various inaccessible controls directly even if they are just drawing on the screen with pixels, then it turns those controls into accessible controls that voiceover can interpret. If anybody I could do this it would be excellent.











 

This is indeed an interesting idea. From my tests, with NVDA's current OCR function, I was able to get NVDA to read texts in games like Sims 3, albeit not 100% reliable or accurate.
Imagine if NVDA is able to recognize in-game texts and interact with game controls in realtime...that would be a monumental step forward, though that would likely require the use of AI technology and a reasonably powerful computer. Even the new voice recognition feature in VoiceOver is limited to newer iPhones.


Josh Kennedy
 

for those people who heard the voiceover recognition is not that great, you are wrong. I tried it, and it is just completely amazing! I now have access to apps which were completely completely in accessible before. It's amazing.


Matthew Whitaker
 

How does it work for VoiceOver? Sorry if this is off topic… 



On Oct 30, 2020, at 10:59 AM, Josh Kennedy <joshknnd1982@...> wrote:

for those people who heard the voiceover recognition is not that great, you are wrong. I tried it, and it is just completely amazing! I now have access to apps which were completely completely in accessible before. It's amazing.


Matthew Whitaker
 

Also what about NVDA as well? 



On Oct 30, 2020, at 10:59 AM, Josh Kennedy <joshknnd1982@...> wrote:

for those people who heard the voiceover recognition is not that great, you are wrong. I tried it, and it is just completely amazing! I now have access to apps which were completely completely in accessible before. It's amazing.


Chelsea
 

I'd like to know what screen reader the new Sony PlayStation is going to use.

Chelsea

On 10/30/20, Matthew Whitaker <matthew.whitaker2011@...> wrote:
Also what about NVDA as well?



On Oct 30, 2020, at 10:59 AM, Josh Kennedy <joshknnd1982@...>
wrote:

for those people who heard the voiceover recognition is not that great,
you are wrong. I tried it, and it is just completely amazing! I now have
access to apps which were completely completely in accessible before. It's
amazing.






abdul muhamin
 

Sony has there own screen reader, probably they will update it on ps5

 

Sent from Mail for Windows 10

 

From: Chelsea
Sent: Friday, October 30, 2020 10:07 PM
To: nvda@nvda.groups.io
Subject: Re: [nvda] NVDA integration in video games

 

I'd like to know what screen reader the new Sony PlayStation is going to use.

 

Chelsea

 

On 10/30/20, Matthew Whitaker <matthew.whitaker2011@...> wrote:

> Also what about NVDA as well?

>> On Oct 30, 2020, at 10:59 AM, Josh Kennedy <joshknnd1982@...>

>> wrote:

>> 

>> for those people who heard the voiceover recognition is not that great,

>> you are wrong. I tried it, and it is just completely amazing! I now have

>> access to apps which were completely completely in accessible before. It's

>> amazing.

>> 

>

 

 

 

 

 


abdul muhamin
 

Have anyone tried NVDA with retro arch and skulgirls?

 

Sent from Mail for Windows 10

 

From: Chelsea
Sent: Friday, October 30, 2020 10:07 PM
To: nvda@nvda.groups.io
Subject: Re: [nvda] NVDA integration in video games

 

I'd like to know what screen reader the new Sony PlayStation is going to use.

 

Chelsea

 

On 10/30/20, Matthew Whitaker <matthew.whitaker2011@...> wrote:

> Also what about NVDA as well?

>> On Oct 30, 2020, at 10:59 AM, Josh Kennedy <joshknnd1982@...>

>> wrote:

>> 

>> for those people who heard the voiceover recognition is not that great,

>> you are wrong. I tried it, and it is just completely amazing! I now have

>> access to apps which were completely completely in accessible before. It's

>> amazing.

>> 

>

 

 

 

 

 


Sarah k Alawami
 

Yep, works grate with retro arch. And scull girls is wonderful, terrable acting, but still.

--

Sarah Alawami, owner of TFFP. . For more info go to our website.

Check out my adventures with a shadow machine.

to subscribe to the feed click here and you can also follow us on twitter

Our discord is where you will know when we go live on twitch. Feel free to give the channel a follow and see what is up there.

For stream archives, products you can buy and more visit my main lbry page and my tffp lbry page You will also be able to buy some of my products and eBooks there.

Finally, to become a patron and help support the podcast go here

On 30 Oct 2020, at 14:46, abdul muhamin wrote:

Have anyone tried NVDA with retro arch and skulgirls?

 

Sent from Mail for Windows 10

 

From: Chelsea
Sent: Friday, October 30, 2020 10:07 PM
To: nvda@nvda.groups.io
Subject: Re: [nvda] NVDA integration in video games

 

I'd like to know what screen reader the new Sony PlayStation is going to use.

 

Chelsea

 

On 10/30/20, Matthew Whitaker <matthew.whitaker2011@...> wrote:

> Also what about NVDA as well?

>> On Oct 30, 2020, at 10:59 AM, Josh Kennedy <joshknnd1982@...>

>> wrote:

>> 

>> for those people who heard the voiceover recognition is not that great,

>> you are wrong. I tried it, and it is just completely amazing! I now have

>> access to apps which were completely completely in accessible before. It's

>> amazing.

>> 

>

 

 

 

 

 


Matthew Whitaker
 

How does NVDA work with that program? Is there a tutorial on it?



On Oct 30, 2020, at 5:49 PM, Sarah k Alawami <marrie12@...> wrote:

Yep, works grate with retro arch. And scull girls is wonderful, terrable acting, but still.

--

Sarah Alawami, owner of TFFP. . For more info go to our website.

Check out my adventures with a shadow machine.

to subscribe to the feed click here and you can also follow us on twitter

Our discord is where you will know when we go live on twitch. Feel free to give the channel a follow and see what is up there.

For stream archives, products you can buy and more visit my main lbry page and my tffp lbry page You will also be able to buy some of my products and eBooks there.

Finally, to become a patron and help support the podcast go here

On 30 Oct 2020, at 14:46, abdul muhamin wrote:

Have anyone tried NVDA with retro arch and skulgirls?

 

Sent from Mail for Windows 10

 

From: Chelsea
Sent: Friday, October 30, 2020 10:07 PM
To: nvda@nvda.groups.io
Subject: Re: [nvda] NVDA integration in video games

 

I'd like to know what screen reader the new Sony PlayStation is going to use.

 

Chelsea

 

On 10/30/20, Matthew Whitaker <matthew.whitaker2011@...> wrote:
> Also what about NVDA as well?
>> On Oct 30, 2020, at 10:59 AM, Josh Kennedy <joshknnd1982@...>
>> wrote:
>> 
>> for those people who heard the voiceover recognition is not that great,
>> you are wrong. I tried it, and it is just completely amazing! I now have
>> access to apps which were completely completely in accessible before. It's
>> amazing.
>> 
>

 

 

 

 

 



abdul muhamin
 

I wish other game developers consider NVDA in there games, specially NetherRealm studios!

 

Sent from Mail for Windows 10

 

From: Sarah k Alawami
Sent: Saturday, October 31, 2020 2:49 AM
To: nvda@nvda.groups.io
Subject: Re: [nvda] NVDA integration in video games

 

Yep, works grate with retro arch. And scull girls is wonderful, terrable acting, but still.

--

Sarah Alawami, owner of TFFP. . For more info go to our website.

Check out my adventures with a shadow machine.

to subscribe to the feed click here and you can also follow us on twitter

Our discord is where you will know when we go live on twitch. Feel free to give the channel a follow and see what is up there.

For stream archives, products you can buy and more visit my main lbry page and my tffp lbry page You will also be able to buy some of my products and eBooks there.

Finally, to become a patron and help support the podcast go here

On 30 Oct 2020, at 14:46, abdul muhamin wrote:

Have anyone tried NVDA with retro arch and skulgirls?

 

Sent from Mail for Windows 10

 

From: Chelsea
Sent: Friday, October 30, 2020 10:07 PM
To: nvda@nvda.groups.io
Subject: Re: [nvda] NVDA integration in video games

 

I'd like to know what screen reader the new Sony PlayStation is going to use.

 

Chelsea

 

On 10/30/20, Matthew Whitaker <matthew.whitaker2011@...> wrote:

> Also what about NVDA as well?

>> On Oct 30, 2020, at 10:59 AM, Josh Kennedy <joshknnd1982@...>

>> wrote:

>> 

>> for those people who heard the voiceover recognition is not that great,

>> you are wrong. I tried it, and it is just completely amazing! I now have

>> access to apps which were completely completely in accessible before. It's

>> amazing.

>> 

>

 

 

 

 

 

 


abdul muhamin
 

No need for tutorials, just install and enjoy

 

Sent from Mail for Windows 10

 

From: Matthew Whitaker
Sent: Saturday, October 31, 2020 2:50 AM
To: nvda@nvda.groups.io
Subject: Re: [nvda] NVDA integration in video games

 

How does NVDA work with that program? Is there a tutorial on it?

 

 



On Oct 30, 2020, at 5:49 PM, Sarah k Alawami <marrie12@...> wrote:

 

Yep, works grate with retro arch. And scull girls is wonderful, terrable acting, but still.

--

Sarah Alawami, owner of TFFP. . For more info go to our website.

Check out my adventures with a shadow machine.

to subscribe to the feed click here and you can also follow us on twitter

Our discord is where you will know when we go live on twitch. Feel free to give the channel a follow and see what is up there.

For stream archives, products you can buy and more visit my main lbry page and my tffp lbry page You will also be able to buy some of my products and eBooks there.

Finally, to become a patron and help support the podcast go here

On 30 Oct 2020, at 14:46, abdul muhamin wrote:

Have anyone tried NVDA with retro arch and skulgirls?

 

Sent from Mail for Windows 10

 

From: Chelsea
Sent: Friday, October 30, 2020 10:07 PM
To: nvda@nvda.groups.io
Subject: Re: [nvda] NVDA integration in video games

 

I'd like to know what screen reader the new Sony PlayStation is going to use.

 

Chelsea

 

On 10/30/20, Matthew Whitaker <matthew.whitaker2011@...> wrote:

> Also what about NVDA as well?

>> On Oct 30, 2020, at 10:59 AM, Josh Kennedy <joshknnd1982@...>

>> wrote:

>> 

>> for those people who heard the voiceover recognition is not that great,

>> you are wrong. I tried it, and it is just completely amazing! I now have

>> access to apps which were completely completely in accessible before. It's

>> amazing.

>> 

>

 

 

 

 

 

 

 


Matthew Whitaker
 

Uh ok… how and where can you install everything and how does it work with NVDA?



On Oct 30, 2020, at 5:53 PM, abdul muhamin <abdulmuhamin600@...> wrote:

No need for tutorials, just install and enjoy
 
Sent from Mail for Windows 10
 
From: Matthew Whitaker
Sent: Saturday, October 31, 2020 2:50 AM
To: nvda@nvda.groups.io
Subject: Re: [nvda] NVDA integration in video games
 
How does NVDA work with that program? Is there a tutorial on it?
 
 


On Oct 30, 2020, at 5:49 PM, Sarah k Alawami <marrie12@...> wrote:
 
Yep, works grate with retro arch. And scull girls is wonderful, terrable acting, but still.
--
Sarah Alawami, owner of TFFP. . For more info go to our website.
to subscribe to the feed click here and you can also follow us on twitter
Our discord is where you will know when we go live on twitch. Feel free to give the channel a follow and see what is up there.
For stream archives, products you can buy and more visit my main lbry page and my tffp lbry page You will also be able to buy some of my products and eBooks there.
Finally, to become a patron and help support the podcast go here
On 30 Oct 2020, at 14:46, abdul muhamin wrote:
Have anyone tried NVDA with retro arch and skulgirls?
 
Sent from Mail for Windows 10
 
From: Chelsea
Sent: Friday, October 30, 2020 10:07 PM
To: nvda@nvda.groups.io
Subject: Re: [nvda] NVDA integration in video games
 
I'd like to know what screen reader the new Sony PlayStation is going to use.
 
Chelsea
 
On 10/30/20, Matthew Whitaker <matthew.whitaker2011@...> wrote:
> Also what about NVDA as well?
>> On Oct 30, 2020, at 10:59 AM, Josh Kennedy <joshknnd1982@...>
>> wrote:
>> 
>> for those people who heard the voiceover recognition is not that great,
>> you are wrong. I tried it, and it is just completely amazing! I now have
>> access to apps which were completely completely in accessible before. It's
>> amazing.
>> 
> 
 
 
 
 
 
 

 


abdul muhamin
 

For skullgirls, you have to buy it from steam, and for retroarch. Download here

From: Matthew Whitaker
Sent: Saturday, October 31, 2020 2:54 AM
To: nvda@nvda.groups.io
Subject: Re: [nvda] NVDA integration in video games

 

Uh ok… how and where can you install everything and how does it work with NVDA?

 

 



On Oct 30, 2020, at 5:53 PM, abdul muhamin <abdulmuhamin600@...> wrote:

 

No need for tutorials, just install and enjoy

 

Sent from Mail for Windows 10

 

From: Matthew Whitaker
Sent: Saturday, October 31, 2020 2:50 AM
To: nvda@nvda.groups.io
Subject: Re: [nvda] NVDA integration in video games

 

How does NVDA work with that program? Is there a tutorial on it?

 

 




On Oct 30, 2020, at 5:49 PM, Sarah k Alawami <marrie12@...> wrote:

 

Yep, works grate with retro arch. And scull girls is wonderful, terrable acting, but still.

--

Sarah Alawami, owner of TFFP. . For more info go to our website.

to subscribe to the feed click here and you can also follow us on twitter

Our discord is where you will know when we go live on twitch. Feel free to give the channel a follow and see what is up there.

For stream archives, products you can buy and more visit my main lbry page and my tffp lbry page You will also be able to buy some of my products and eBooks there.

Finally, to become a patron and help support the podcast go here

On 30 Oct 2020, at 14:46, abdul muhamin wrote:

Have anyone tried NVDA with retro arch and skulgirls?

 

Sent from Mail for Windows 10

 

From: Chelsea
Sent: Friday, October 30, 2020 10:07 PM
To: nvda@nvda.groups.io
Subject: Re: [nvda] NVDA integration in video games

 

I'd like to know what screen reader the new Sony PlayStation is going to use.

 

Chelsea

 

On 10/30/20, Matthew Whitaker <matthew.whitaker2011@...> wrote:

> Also what about NVDA as well?

>> On Oct 30, 2020, at 10:59 AM, Josh Kennedy <joshknnd1982@...>

>> wrote:

>> 

>> for those people who heard the voiceover recognition is not that great,

>> you are wrong. I tried it, and it is just completely amazing! I now have

>> access to apps which were completely completely in accessible before. It's

>> amazing.

>> 

>