NVDA integration in video games
|
abdul muhamin
Hi. Now a days, video games developers are trying to integrate NVDA support in video games like skullgirls and the emulator called retroarch, I’ve seen a file in both the mentioned game/emulator called NVDA controller client.dll, I think its related to the speech during the menues. So my question is, can we also integrate the NVDA to any software or game? I’m sure it requires advance knowledge in programing languages, what do you think?
Sent from Mail for Windows 10
|
|
|
|
Sascha Cowley
First, this topic does not belong on the main list; it should be on the chat or development sublists.
Second, the NVDA controler client is a programmatic way of developers integrating their software with NVDA. For example, it allows them to send speech or Braille to NVDA to be output. As for weather we can "integrate the NVDA to any software or game", that depends what you mean. There is no technical reason why any proficient developer would be unable to so integrate their Windows program with NVDA. At the end-user side, the answer is not really: integration of this type requires modification of the program itself, while integration via scripting requires some sort of events that NVDA can hook into, which many programs simply do not provide.
|
|
|
|
Luke Robinett
Is there a practical reason for so heavily policing the types of questions people ask here or is it more of a control thing? I mean, dude’s question was NVDA related. Why be such a cop about it?
On Oct 28, 2020, at 6:52 PM, Sascha Cowley via groups.io <sascha.camille@...> wrote:
|
|
|
|
Josh Kennedy
if NVDA had real time recognition like voiceover does on iOS 14, then we would have access to a lot more applications including video games. Voiceover uses a Neural Engine and artificial intelligence to do screen recognition, image recognition and description, and real time live OCR and it also recognizes controls based on pixels on the screen and matrices. Making many applications including video games very accessible and applications which did not used to be accessible or now suddenly accessible with voiceover artificial intelligent Neural Engine recognition. it would be excellent if NVDA could implement this feature very soon. I believe that it directly access of the GPU, and based on pixels on the screen and live screen or some kind of image scanning, it recognizes various inaccessible controls directly even if they are just drawing on the screen with pixels, then it turns those controls into accessible controls that voiceover can interpret. If anybody I could do this it would be excellent.
|
|
|
|
Andy B.
I heard that screen recognition wasn’t that great. If NVDA took this route, maybe things would work a little better, but there are also many people out there with computers not able to handle such a thing. If this new voiceover feature isn’t available on an iPhone 8+ because it is too weak, I don’t want to know what it would take on a laptop or computer. In any case, talking flight monitor does implement NVDA controller, etc into it’s code base. Check it out at www.talkingflightmonitor.net. I am one of the devs.
From: nvda@nvda.groups.io <nvda@nvda.groups.io> On Behalf Of Josh Kennedy
Sent: Thursday, October 29, 2020 12:12 PM To: nvda@nvda.groups.io Subject: Re: [nvda] NVDA integration in video games
if NVDA had real time recognition like voiceover does on iOS 14, then we would have access to a lot more applications including video games. Voiceover uses a Neural Engine and artificial intelligence to do screen recognition, image recognition and description, and real time live OCR and it also recognizes controls based on pixels on the screen and matrices. Making many applications including video games very accessible and applications which did not used to be accessible or now suddenly accessible with voiceover artificial intelligent Neural Engine recognition. it would be excellent if NVDA could implement this feature very soon. I believe that it directly access of the GPU, and based on pixels on the screen and live screen or some kind of image scanning, it recognizes various inaccessible controls directly even if they are just drawing on the screen with pixels, then it turns those controls into accessible controls that voiceover can interpret. If anybody I could do this it would be excellent.
|
|
|
|
Gene
What do you mean by this sentence?
toggle quoted messageShow quoted text
If this new voiceover feature isn’t available on an iPhone 8+ because it is too weak, I don’t want to know what it would take on a laptop or computer. Smart phones aren't nearly as powerful as computers. Gene
-----Original Message-----
From: Andy B. Sent: Thursday, October 29, 2020 2:10 PM To: nvda@nvda.groups.io Subject: Re: [nvda] NVDA integration in video games I heard that screen recognition wasn’t that great. If NVDA took this route, maybe things would work a little better, but there are also many people out there with computers not able to handle such a thing. If this new voiceover feature isn’t available on an iPhone 8+ because it is too weak, I don’t want to know what it would take on a laptop or computer. In any case, talking flight monitor does implement NVDA controller, etc into it’s code base. Check it out at www.talkingflightmonitor.net. I am one of the devs. From: nvda@nvda.groups.io <nvda@nvda.groups.io> On Behalf Of Josh Kennedy Sent: Thursday, October 29, 2020 12:12 PM To: nvda@nvda.groups.io Subject: Re: [nvda] NVDA integration in video games if NVDA had real time recognition like voiceover does on iOS 14, then we would have access to a lot more applications including video games. Voiceover uses a Neural Engine and artificial intelligence to do screen recognition, image recognition and description, and real time live OCR and it also recognizes controls based on pixels on the screen and matrices. Making many applications including video games very accessible and applications which did not used to be accessible or now suddenly accessible with voiceover artificial intelligent Neural Engine recognition. it would be excellent if NVDA could implement this feature very soon. I believe that it directly access of the GPU, and based on pixels on the screen and live screen or some kind of image scanning, it recognizes various inaccessible controls directly even if they are just drawing on the screen with pixels, then it turns those controls into accessible controls that voiceover can interpret. If anybody I could do this it would be excellent.
|
|
|
|
Luke Robinett
I do a lot of music production work and much of that software is very graphical and inaccessible out of the box. People have created some really cool NVDA add-ons in that space that utilize screen recognition and some sort of learning algorithms to give access to user interfaces that are otherwise completely unrecognizable by NVDA. For an example of such a project, check out Sibiac. I think this shows the potential for NVDA to be augmented to do what you described, giving access to graphical interfaces that normally don’t provide hooks for screen readers.
toggle quoted messageShow quoted text
On Oct 29, 2020, at 12:15 PM, Gene <gsasner@gmail.com> wrote:
|
|
|
|
This is indeed an interesting idea. From my tests, with NVDA's current OCR function, I was able to get NVDA to read texts in games like Sims 3, albeit not 100% reliable or accurate.
Imagine if NVDA is able to recognize in-game texts and interact with game controls in realtime...that would be a monumental step forward, though that would likely require the use of AI technology and a reasonably powerful computer. Even the new voice recognition feature in VoiceOver is limited to newer iPhones.
|
|
|
|
Josh Kennedy
for those people who heard the voiceover recognition is not that great, you are wrong. I tried it, and it is just completely amazing! I now have access to apps which were completely completely in accessible before. It's amazing.
|
|
|
|
Matthew Whitaker
How does it work for VoiceOver? Sorry if this is off topic…
toggle quoted messageShow quoted text
|
|
|
|
Matthew Whitaker
Also what about NVDA as well?
toggle quoted messageShow quoted text
|
|
|
|
Chelsea <Lady.arwen15@...>
I'd like to know what screen reader the new Sony PlayStation is going to use.
toggle quoted messageShow quoted text
Chelsea
On 10/30/20, Matthew Whitaker <matthew.whitaker2011@gmail.com> wrote:
Also what about NVDA as well?On Oct 30, 2020, at 10:59 AM, Josh Kennedy <joshknnd1982@gmail.com>
|
|
|
|
abdul muhamin
toggle quoted messageShow quoted text
From: Chelsea
Sent: Friday, October 30, 2020 10:07 PM To: nvda@nvda.groups.io Subject: Re: [nvda] NVDA integration in video games
I'd like to know what screen reader the new Sony PlayStation is going to use.
Chelsea
On 10/30/20, Matthew Whitaker <matthew.whitaker2011@...> wrote: > Also what about NVDA as well? > > > >> On Oct 30, 2020, at 10:59 AM, Josh Kennedy <joshknnd1982@...> >> wrote: >> >> for those people who heard the voiceover recognition is not that great, >> you are wrong. I tried it, and it is just completely amazing! I now have >> access to apps which were completely completely in accessible before. It's >> amazing. >> > > > > > > >
|
|
|
|
abdul muhamin
toggle quoted messageShow quoted text
From: Chelsea
Sent: Friday, October 30, 2020 10:07 PM To: nvda@nvda.groups.io Subject: Re: [nvda] NVDA integration in video games
I'd like to know what screen reader the new Sony PlayStation is going to use.
Chelsea
On 10/30/20, Matthew Whitaker <matthew.whitaker2011@...> wrote: > Also what about NVDA as well? > > > >> On Oct 30, 2020, at 10:59 AM, Josh Kennedy <joshknnd1982@...> >> wrote: >> >> for those people who heard the voiceover recognition is not that great, >> you are wrong. I tried it, and it is just completely amazing! I now have >> access to apps which were completely completely in accessible before. It's >> amazing. >> > > > > > > >
|
|
|
|
Yep, works grate with retro arch. And scull girls is wonderful, terrable acting, but still. -- Sarah Alawami, owner of TFFP. . For more info go to our website. Check out my adventures with a shadow machine. to subscribe to the feed click here and you can also follow us on twitter Our discord is where you will know when we go live on twitch. Feel free to give the channel a follow and see what is up there. For stream archives, products you can buy and more visit my main lbry page and my tffp lbry page You will also be able to buy some of my products and eBooks there. Finally, to become a patron and help support the podcast go here
On 30 Oct 2020, at 14:46, abdul muhamin wrote:
|
|
|
|
Matthew Whitaker
How does NVDA work with that program? Is there a tutorial on it?
toggle quoted messageShow quoted text
|
|
|
|
abdul muhamin
I wish other game developers consider NVDA in there games, specially NetherRealm studios!
Sent from Mail for Windows 10
From: Sarah k Alawami
Sent: Saturday, October 31, 2020 2:49 AM To: nvda@nvda.groups.io Subject: Re: [nvda] NVDA integration in video games
Yep, works grate with retro arch. And scull girls is wonderful, terrable acting, but still. -- Sarah Alawami, owner of TFFP. . For more info go to our website. Check out my adventures with a shadow machine. to subscribe to the feed click here and you can also follow us on twitter Our discord is where you will know when we go live on twitch. Feel free to give the channel a follow and see what is up there. For stream archives, products you can buy and more visit my main lbry page and my tffp lbry page You will also be able to buy some of my products and eBooks there. Finally, to become a patron and help support the podcast go here On 30 Oct 2020, at 14:46, abdul muhamin wrote:
|
|
|
|
abdul muhamin
toggle quoted messageShow quoted text
From: Matthew Whitaker
Sent: Saturday, October 31, 2020 2:50 AM To: nvda@nvda.groups.io Subject: Re: [nvda] NVDA integration in video games
How does NVDA work with that program? Is there a tutorial on it?
|
|
|
|
Matthew Whitaker
Uh ok… how and where can you install everything and how does it work with NVDA?
toggle quoted messageShow quoted text
|
|
|
|
abdul muhamin
For skullgirls, you have to buy it from steam, and for retroarch. Download here
From: Matthew Whitaker
Sent: Saturday, October 31, 2020 2:54 AM To: nvda@nvda.groups.io Subject: Re: [nvda] NVDA integration in video games
Uh ok… how and where can you install everything and how does it work with NVDA?
|
|
|