The Content of the Add-ons Store (was: Re: [nvda] remote)


Luke Davis
 

Brian Vogel wrote:

-Then I certainly hope that the intent is to:1. Populate the Add-On store with all possible speed.
That does seem to be the goal of opening (mandating) the store in the middle of an API breaking release. To get as many maintainers as possible uploading their add-ons, and to the store instead of the old setup.
A bit inconvenient, but it does make sense for widest rapid adoption.

2. Have both officially vetted and unofficial add-ons.
Brian, I continue to disagree with the use of this phrase "officially vetted" or derivatives. IMO it implies a level of oversight and review which does not exist. The remainder of this message explains why I think that.

Currently, I see no difference between "official" and "unofficial", because NV Access and the add-ons community, make no determination about the quality of any add-on. There is no "vetting", because there is no review baked into the current system. Maybe some vetting is coming, but for now, it's not there.

Contrary to what has been repeatedly stated, even requiring translatability, as Rui said, is also, not there.

To me, "officially vetted", means that someone at NV Access, or at the top of the add-ons community, went over the code even moderately, or ran code related automated tools on it, to decide whether the code meets some standard. There is nothing even remotely like that happening any more.

From https://github.com/nvaccess/addon-datastore/blob/master/README.md#readme
"
About security

Ensuring that an add-on is safe to run is a difficult challenge that isn't addressed here. However, the metadata for a new submission (add-on release) can be confirmed to match its manifest description. Additionally, add-on file integrity can be enforced via a SHA256 checksum. The checksum allows NVDA to ensure that add-on releases are immutable.

Human review process / code audit

* NV Access doesn't require a manual review of the add-on (code or user experience) itself before the add-on submission.
* You are welcome to review code / UX of add-ons and provide that feedback directly to add-on authors.
* The SHA256 checksum of the .nvda-addon prevents undetected changes.
* User reviews/rating of add-ons are currently out of scope.
"

The only thing that is automatically enforced now, is the outer container of the add-on: the packaging, and the format of the manifest. Even the distribution is not given requirements other than "fill in this form to distribute", though GitHub is recommended, and of course it must have a GPL 2 compatible license.

A few technical submission requirements does not equal vetting, as I understand the word vetting. If you think otherwise, I would encourage you to look at Firefox's technical requirements for add-on submission.

They have review processes. We don't. But even without the review processes, there are still basic packaging requirements that must be met to distribute an add-on and expect it to work with the software. That does not equal vetting until it reaches the review process.

{vet}
V 2: examine carefully; "Someone should vet this report before it goes out"
From WordNet (r) 3.0 (2006)

At the very least, implies human or machine intentionality to find errors, bad behavior, or a failure to meet standards.

Luke


Cyrille
 

Hello

I can only agree with Luke.
On the community website, there used to be a basic review process. But it has been withdrawn months ago (maybe even more than one year). More recently (until February 2023), Joseph and Noelia were just doing a check on behalf of NVAccess to check that the download link is working and that the fields of the manifest are correct, e.g. do not put last tested version to 2099.1.0)
Now that add-ons are submitted to the add-on store the check on the download link and the manifest are done automatically. As Luke indicated, these checks are just checks to confirm that the add-on is technically compatible with the store, nothing more.

Since the submission process in the add-on store should be lighter than the former process for the community website, NVAccess expects that add-ons that were not submitted on the community website be submitted on the add-on store. So that more add-on become "official", that is, listed in the add-on store. "Official" means nothing else more.

Regarding translatability, there has never been a mandatory requirement for the add-ons to be translatable, even if making strings translatable was encouraged. And actually it seems that most (if not all) add-ons on the community website have their strings translatable. But again, it was not a requirement for an add-on to be accepted on the website.

Cheers,

Cyrille





2. Have both officially vetted and unofficial add-ons.


On Fri, Mar 31, 2023 at 07:30 AM, Luke Davis wrote:
Brian Vogel wrote:

-Then I certainly hope that the intent is to:1. Populate the Add-On store with all possible speed.
That does seem to be the goal of opening (mandating) the store in the middle of
an API breaking release. To get as many maintainers as possible uploading their
add-ons, and to the store instead of the old setup.
A bit inconvenient, but it does make sense for widest rapid adoption.

2. Have both officially vetted and unofficial add-ons.
Brian, I continue to disagree with the use of this phrase "officially vetted" or
derivatives. IMO it implies a level of oversight and review which does not
exist. The remainder of this message explains why I think that.

Currently, I see no difference between "official" and "unofficial", because NV
Access and the add-ons community, make no determination about the quality of any
add-on. There is no "vetting", because there is no review baked into the current
system. Maybe some vetting is coming, but for now, it's not there.

Contrary to what has been repeatedly stated, even requiring translatability,
as Rui said, is also, not there.

To me, "officially vetted", means that someone at NV Access, or at the top of
the add-ons community, went over the code even moderately, or ran code related
automated tools on it, to decide whether the code meets some standard. There is
nothing even remotely like that happening any more.

From https://github.com/nvaccess/addon-datastore/blob/master/README.md#readme
"
About security

Ensuring that an add-on is safe to run is a difficult challenge that isn't addressed here. However, the metadata for a new submission (add-on release) can be confirmed to match its manifest description. Additionally, add-on file integrity can be enforced via a SHA256 checksum. The checksum allows NVDA to ensure that add-on releases are immutable.

Human review process / code audit

* NV Access doesn't require a manual review of the add-on (code or user
experience) itself before the add-on submission.
* You are welcome to review code / UX of add-ons and provide that feedback
directly to add-on authors.
* The SHA256 checksum of the .nvda-addon prevents undetected changes.
* User reviews/rating of add-ons are currently out of scope.
"

The only thing that is automatically enforced now, is the outer container of the
add-on: the packaging, and the format of the manifest. Even the distribution is
not given requirements other than "fill in this form to distribute", though
GitHub is recommended, and of course it must have a GPL 2 compatible license.

A few technical submission requirements does not equal vetting, as I understand
the word vetting. If you think otherwise, I would encourage you to look at
Firefox's technical requirements for add-on submission.

They have review processes. We don't. But even without the review processes,
there are still basic packaging requirements that must be met to distribute an
add-on and expect it to work with the software. That does not equal vetting
until it reaches the review process.

{vet}
V 2: examine carefully; "Someone should vet this report before it goes out"
From WordNet (r) 3.0 (2006)

At the very least, implies human or machine intentionality to find errors,
bad behavior, or a failure to meet standards.

Luke


 

Hi,

The idea of translating add-ons in version 1.0 could be traced to the fact that it is the translations team and the workflow that actually powers community add-ons website. The infrastructure of the community add-ons website consists of:

  1. Add-on readme files where the documentation comes from.
  2. Ikiwiki for managing the website content and making it available in a variety of languages.
  3. Translations workflow for not only managing documentation and translation, but also for managing actual localization data for add-ons (nvda.po files, for example).
  4. The work of humans and machines responsible for promoting and directing people to the website, discussing new add-on releases, adding features and resolving bugs, translating add-on discussions into various languages, and until a few weeks ago, reviewing add-on submissions to make sure manifests were okay and sometimes code works as expected (yes, mostly I and Noelia did the review work, and in my case, I reviewed code for add-ons hosted on community add-ons website to make sure they were ready for upcoming NVDA releases; I use Windows Subsystem for Linux for this, specifically using tools such as Grep, Flake8, Mypy, and other tools to check add-on code; I have a portable alpha NVDA with all add-ons registered on community add-ons installed for code review purposes). Most of the human work (some would say labor) were done by volunteers, and I am one such volunteer (I wrote add-ons since 2013 and am one of the founders of NVDA add-ons forum in 2013). You could say that I am one of the ovlunteers assisting with "vetting" (notice the quotes) add-ons until recently, but as I wrote a few hours ago, the human review process in the past could be seen as restrictive.

A few more things, mostly philosophical things:

  1. Translations as a requirement for add-on 1.0: while I understand the reasoning for the push for translations in add-on 1.0, I strongly disagree with it. The reason we create add-ons is to improve support for an application, create drivers for speech synthesizers and braille displays, and add interesting and sometimes much needed features that will work anywhere. I would say that, at least for app modules, add-ons are workarounds (that's me bluntly describing essentially what an app module really is). If add-ons provide solutions or a milestone toward one or several solutions, then the first order of business is to get it to work effectively. Only after the solution works should we consider translating - of course the best scenario is for the author to be considerate and make messages translatable from the beginning, but we need to understand that not all add-on authors are multilingual speakers or have translation and localization experience, or perhaps not familiar with internationalization much (localization and internationalization are two different things); translation, localization, and internationalization are specialized tasks, requiring a good understanding of cultural and local idioms and shows willingness to discuss what could be technical terms in a way that's understandable across languages, customs, norms, and assumptions. If an add-on author is willing to learn basics of this, that's good. But asking (or in some cases, demanding) that an add-on be made translatable in version 1.0 is asking add-on authors to learn specialized tasks when they may not have time to do so - remember, most add-on authors are volunteers with little to no training on how localization process works. Should we authors show consideration by learning how to make our messages translatable from the beginning? At least if folks are willing, but let us not demand it as a requirement for version 1.0 (this is why when I produced add-ons, I made sure that my add-ons were doing what they are supposed to do in 1.0 before opening up for translations; of course I made sure messages were translatable from the start).
  2. The word "vetting": the subtext is that we are dividing the add-ons ecosystem into two classes: ones approved by an entity (say, NV Access) and deserve to be listed on community add-ons website, and ones that were not deserving of being listed on the website (at least that's one way for folks to understand the word "vetting"). The overall idea of add-on store is to remove that distinction - as I noted above, human review process could be seen as restrictive, thereby enforcing the class divisions, and that's what made some folks shake their heads and said, "we are going to push for a change." I do acknowledge my role in making the process restrictive to some members of the community (and I apologize for it), and I hope that the new add-on store in the works can be seen as an evidence of the change being worked on to better cater to the needs of add-ons community. In some respects, even though the add-on store mechanism is largely based on Windows Package Manager (winget), there are influences from various communities, particularly the Spanish community. The vision I see for NV Access add-on store is a place where add-ons and the people behind them are showcased and celebrated regardless of where they come from, and this is why I did say somewhere that anyone with a GitHub account can submit add-ons or ask someone to do so. As I said before, we are in a transitoinal period, and I hope the add-on store can serve as a conversation starter on how we are defining the term "vetted" and even perpetuating and the effect it has to individuals and throughout the NVDA community. I'm not asking that we stop using the term "vetting" at this moment, but I'm asking that we think more about how we got here and what we can do to make things better because the add-on store is, can, and should be a community effort.
  3. Are we ready to let go of add-ons: anyone writing add-ons (or using add-ons) should remind themselves that we cannot be responsible for add-ons forever - we sometimes need to let go of what we wrote. If it wasn't for the burnout I experienced last year (after asking folks throughout add-ons community about compatibility with NVDA 2022.1), I would have thought I could maintain add-ons for a long time, but that is not to be the case. I let go of most add-ons I developed over the years because I wanted and needed to move on. There were two other reasons for letting go of add-ons: I am tired and had enough and wanted a break from NVDA community, and I wanted to step aside so others can keep the movement toward equal access to technology going. I know what I just wrote could be shocking (or no surprise for some) as I seem to be enthusiastic about NVDA and add-ons; but I knew for some time that my enthusiasm toward NVDA is not as strong as say, five years ago. And the burnout I experienced last year (folks who read my message around this time last year may remember what I'm talking about) reaffirmed the idea that I was on the right track toward exiting the NVDA community - back then, I took compatibility issues with add-ons as a personal issue and felt anxious whenever I heard add-ons not working with 2022.1, including add-ons not developed by me. But the burnout taught me that I was wrong, or to say it bluntly, I was an idiot - taking a community issue too personally, and that I was a perfectionist which can cause problems. Therefore, shortly after experiencing burnout in May 2022, I made up my mind that I will let go of most of my add-ons to reduce community workload and toward the eventual end: saying goodbye to a community I called home for the last eleven years, perhaps coming back with a different role a while later. Ultimately, what I'm saying is this: whenever you write or use add-ons, you must be willing to let add-ons go.

I know what I wrote might be hard for some to process, but I'm sharing these with you to show that I am a human and the raw emotions I'm experiencing as an NVDA code contributor, add-on author, and a user of NVDA screen reader. All that's left for me to do (at least planned) is submit two NVDA pull requests, maintain two add-ons, and make the way for NV Access and the entire NVDA community to keep the project moving forward. As for the last part, the least I can do is offer a vision of NV Access add-on store (at least what I believe the visoin is) so we can be on the same page.

Hope this helps.

Cheers,

Joseph


David Ingram
 

Hi Joseph, i want to thank you for your explanation that you just Gave.  The question that i have is when do you think that the nvda add-onstore will be ready for prime time?  I ask this question because i've set nvda to the nvda add-on store and i'm loolooking forward to seeing the new improvements.

-----Original Message-----
From: <nvda@nvda.groups.io>
Sent: Mar 31, 2023 3:47 AM
To: <nvda@nvda.groups.io>
Subject: Re: The Content of the Add-ons Store (was: Re: [nvda] remote)

 

Hi,

The idea of translating add-ons in version 1.0 could be traced to the fact that it is the translations team and the workflow that actually powers community add-ons website. The infrastructure of the community add-ons website consists of:

  1. Add-on readme files where the documentation comes from.
  2. Ikiwiki for managing the website content and making it available in a variety of languages.
  3. Translations workflow for not only managing documentation and translation, but also for managing actual localization data for add-ons (nvda.po files, for example).
  4. The work of humans and machines responsible for promoting and directing people to the website, discussing new add-on releases, adding features and resolving bugs, translating add-on discussions into various languages, and until a few weeks ago, reviewing add-on submissions to make sure manifests were okay and sometimes code works as expected (yes, mostly I and Noelia did the review work, and in my case, I reviewed code for add-ons hosted on community add-ons website to make sure they were ready for upcoming NVDA releases; I use Windows Subsystem for Linux for this, specifically using tools such as Grep, Flake8, Mypy, and other tools to check add-on code; I have a portable alpha NVDA with all add-ons registered on community add-ons installed for code review purposes). Most of the human work (some would say labor) were done by volunteers, and I am one such volunteer (I wrote add-ons since 2013 and am one of the founders of NVDA add-ons forum in 2013). You could say that I am one of the ovlunteers assisting with "vetting" (notice the quotes) add-ons until recently, but as I wrote a few hours ago, the human review process in the past could be seen as restrictive.

A few more things, mostly philosophical things:

  1. Translations as a requirement for add-on 1.0: while I understand the reasoning for the push for translations in add-on 1.0, I strongly disagree with it. The reason we create add-ons is to improve support for an application, create drivers for speech synthesizers and braille displays, and add interesting and sometimes much needed features that will work anywhere. I would say that, at least for app modules, add-ons are workarounds (that's me bluntly describing essentially what an app module really is). If add-ons provide solutions or a milestone toward one or several solutions, then the first order of business is to get it to work effectively. Only after the solution works should we consider translating - of course the best scenario is for the author to be considerate and make messages translatable from the beginning, but we need to understand that not all add-on authors are multilingual speakers or have translation and localization experience, or perhaps not familiar with internationalization much (localization and internationalization are two different things); translation, localization, and internationalization are specialized tasks, requiring a good understanding of cultural and local idioms and shows willingness to discuss what could be technical terms in a way that's understandable across languages, customs, norms, and assumptions. If an add-on author is willing to learn basics of this, that's good. But asking (or in some cases, demanding) that an add-on be made translatable in version 1.0 is asking add-on authors to learn specialized tasks when they may not have time to do so - remember, most add-on authors are volunteers with little to no training on how localization process works. Should we authors show consideration by learning how to make our messages translatable from the beginning? At least if folks are willing, but let us not demand it as a requirement for version 1.0 (this is why when I produced add-ons, I made sure that my add-ons were doing what they are supposed to do in 1.0 before opening up for translations; of course I made sure messages were translatable from the start).
  2. The word "vetting": the subtext is that we are dividing the add-ons ecosystem into two classes: ones approved by an entity (say, NV Access) and deserve to be listed on community add-ons website, and ones that were not deserving of being listed on the website (at least that's one way for folks to understand the word "vetting"). The overall idea of add-on store is to remove that distinction - as I noted above, human review process could be seen as restrictive, thereby enforcing the class divisions, and that's what made some folks shake their heads and said, "we are going to push for a change." I do acknowledge my role in making the process restrictive to some members of the community (and I apologize for it), and I hope that the new add-on store in the works can be seen as an evidence of the change being worked on to better cater to the needs of add-ons community. In some respects, even though the add-on store mechanism is largely based on Windows Package Manager (winget), there are influences from various communities, particularly the Spanish community. The vision I see for NV Access add-on store is a place where add-ons and the people behind them are showcased and celebrated regardless of where they come from, and this is why I did say somewhere that anyone with a GitHub account can submit add-ons or ask someone to do so. As I said before, we are in a transitoinal period, and I hope the add-on store can serve as a conversation starter on how we are defining the term "vetted" and even perpetuating and the effect it has to individuals and throughout the NVDA community. I'm not asking that we stop using the term "vetting" at this moment, but I'm asking that we think more about how we got here and what we can do to make things better because the add-on store is, can, and should be a community effort.
  3. Are we ready to let go of add-ons: anyone writing add-ons (or using add-ons) should remind themselves that we cannot be responsible for add-ons forever - we sometimes need to let go of what we wrote. If it wasn't for the burnout I experienced last year (after asking folks throughout add-ons community about compatibility with NVDA 2022.1), I would have thought I could maintain add-ons for a long time, but that is not to be the case. I let go of most add-ons I developed over the years because I wanted and needed to move on. There were two other reasons for letting go of add-ons: I am tired and had enough and wanted a break from NVDA community, and I wanted to step aside so others can keep the movement toward equal access to technology going. I know what I just wrote could be shocking (or no surprise for some) as I seem to be enthusiastic about NVDA and add-ons; but I knew for some time that my enthusiasm toward NVDA is not as strong as say, five years ago. And the burnout I experienced last year (folks who read my message around this time last year may remember what I'm talking about) reaffirmed the idea that I was on the right track toward exiting the NVDA community - back then, I took compatibility issues with add-ons as a personal issue and felt anxious whenever I heard add-ons not working with 2022.1, including add-ons not developed by me. But the burnout taught me that I was wrong, or to say it bluntly, I was an idiot - taking a community issue too personally, and that I was a perfectionist which can cause problems. Therefore, shortly after experiencing burnout in May 2022, I made up my mind that I will let go of most of my add-ons to reduce community workload and toward the eventual end: saying goodbye to a community I called home for the last eleven years, perhaps coming back with a different role a while later. Ultimately, what I'm saying is this: whenever you write or use add-ons, you must be willing to let add-ons go.

I know what I wrote might be hard for some to process, but I'm sharing these with you to show that I am a human and the raw emotions I'm experiencing as an NVDA code contributor, add-on author, and a user of NVDA screen reader. All that's left for me to do (at least planned) is submit two NVDA pull requests, maintain two add-ons, and make the way for NV Access and the entire NVDA community to keep the project moving forward. As for the last part, the least I can do is offer a vision of NV Access add-on store (at least what I believe the visoin is) so we can be on the same page.

Hope this helps.

Cheers,

Joseph

 


 

Hi,

That I don't know as things are still being developed.

Cheers,

Joseph


 

On Fri, Mar 31, 2023 at 01:30 AM, Luke Davis wrote:
At the very least, implies human or machine intentionality to find errors, bad behavior, or a failure to meet standards.
-
Luke,

We're going to have to agree to disagree here.  There was a vetting process, and there still is a vetting process.  Things are reviewed, period, end of sentence.

And, to be clear, I was taking much more about what has been the case up until now.  It is impossible to deny that any add-on that's currently on the Community Add-Ons page was not thoroughly reviewed, vetted, gone over, whatever you want to call it.

But even the add-ons store process is a vetting, it's just a different vetting and a less restrictive one.  If an add-on ends up in the add-on store, then it becomes "official" for lack of a better term.  That doesn't mean that it's heavily reviewed, or looked at at all by humans, but that it has made it through the process to get there.  There is a process, and I have no idea what you'd like to call that process if not vetting.

And with this, I'm done.  As this has arrived at the point where we're arguing about what constitutes vetting, or what vetting means.  Any requirement prior to acceptance, of any kind, is vetting.
--

Brian Virginia, USA Windows 11 Pro, 64-Bit, Version 22H2, Build 22621; Office 2016, Version 16.0.15726.20188, 32-bit

It is much easier to be critical than to be correct.

       ~ Benjamin Disraeli, 1804-1881