Re: A few thoughts: Web Aim survey, quantity versus quality, feeling burnt out and tutorials


Brian's Mail list account
 

Yeah well I'm surveyed and consultation out. it seems these days there are such consultations done all over the place about everything from babies nappies to screenreaders and I've yet to see any of them bear any fruit in the way things are done because I feel that everything these days is driven by vested interests and cost.
Brian

bglists@...
Sent via blueyonder.
Please address personal email to:-
briang1@..., putting 'Brian Gaff'
in the display name field.

----- Original Message -----
From: "Mário Navarro" <mario.gnv@...>
To: <nvda@nvda.groups.io>
Sent: Wednesday, January 24, 2018 3:50 PM
Subject: Re: [nvda] A few thoughts: Web Aim survey, quantity versus quality, feeling burnt out and tutorials



absolutely right.

thank you good friend.


long life to Joseph Lee.

cheers.



Às 17:25 de 23/12/2017, Joseph Lee escreveu:

Dear NVDA community,

As I read messages on recent discussions, I realized just how much
enthusiasm and concern people have over NVDA and its future. At the
same time, it became clear to me that I and other developers and
community elders need a day off and just listen to you all, as
listening allows us to think about what others are saying and plan
things accordingly.

But first, a humble opinion about surveys and other points:

First, when calls for the seventh Web AIM survey went out, I told
people to not just do it to “increase” market share. I specifically
told screen reader companies to not coerce users to do it, but let
people take it out of their own willingness. This advice was to avoid
a fiasco that happened with Web AIM 6 where AI Squared (now part of
VFO) staff told Window-Eyes users to fill out the survey in mass
numbers, which became a small controversy within the screen reading
world, and to me, making Web AIM results no longer credible.

As some folks pointed out, Web AIM numbers depend on how many people
fill it out and where they come from (and this is true of any surveys
where word of mouth drives participation). The results also depend on
demographics and other factors such as choices given, how the
questions are worded, and overall objective. If one or more data
points seems to be dominant, they can be either skewed or outliers,
with the more extreme cases being termed “outliers” and they affect
how the results are explicated (interpreted). Even skewed data, such
as what I can perceive from some surveys including recent Web AIM
iterations can affect statistical calculations to a point where it
raises genuine questions about bias, credibility, and others (after
all, success of statistics, particularly inferential statistics,
depends on a representative sample or a close equivalent that allows
researchers to approximate the real world, which is prone to errors if
not done correctly such as misinterpretation, bad outliers, only some
groups participating, not looking at things more deeply and what not).

One important thing to note is that Web AIM is a representative
survey, thus the result in front of me could reflect reality. However,
due to recent controversy, possible type I and II errors
(false-positive and false-negative, respectively) and because of
outliers and skewed data and participation, it does not truly reflect
actual data, which is a point some folks here are trying to say and I
concur with. My explication of Web AIM 7 is that, in some parts of the
world, JAWS for Windows is more popular. However, given the fact that
not all geographical regions are represented, I’d counter by saying
that this is not a true representative sample that includes every
continent, and if it did, the story would be different and will
reflect reality a bit better (not a lot because there are other ways
of skewing data such as filling it out on behalf of an organization,
robotic fillers and so on). Coupled with the fact that Web AIM went
through a major controversy recently that damaged its credibility
somewhat, I would dare not trust Web AIM results again.

This leads to my second point: quantity versus quality. If NV Access
went straight for quantity alone, they could have implemented all
possible feature requests in hopes of boosting market share. The
reality in front of us says otherwise: not all feature requests are
here. Numerous factors contribute to this problem:

* Lack of leading developers: in 2017, a long-time NVDA developer
started working for another organization, and NV Access has been
looking for his replacement ever since. Even if the replacement is
found, it’ll take several months for him or her to become used to
this community, learn about accessibility and how to interact with
members, and earn our trust (it took Reef Turner a year to fully
earn our trust). Folks can counter this by saying that there are
countless contributors out there, but ultimately what gets into
NVDA depends on pull requests and review time from NV Access.
* Attitudes about open-source software from organizations: as some
folks pointed out, there are prevailing attitudes about
open-source in organizations that makes it a bit harder for NVDA
to land on their computers, which allows developers to assess true
needs of organizations through user feedback. Without valuable
feedback from organizations (a quality one at that), we won’t see
huger progress in NVDA development.
* Outside attitudes about the NVDA community: from the inside, NVDA
community is seen as a tight nit of enthusiasts who strives to
make NVDA better every day. On the outside, however, we have a
mixed bag of reputations, from admiration to honorable mentions to
disdain. Every organization have these mixed reputations,
especially more so for a community powered by technology such as
Linux kernel developers, web browser vendors and web standards
organizations, and even screen reader community. Not only we need
to show that we are united inside, we need to showcase unity
outside of this community.
* Inside matters just as outside: public relations outside of NVDA
community is important, but unity within an organization is just
as important as public organizational face (I’ll address
developer’s point of view below). What makes NVDA stand out is our
unity despite coming from different circumstances and backgrounds.

Most of these point to quality, not quantity alone. In summary,
quantity is important, but quality is just as important as how many
people download NVDA 2017.4 between Christmas and New Year.

Lastly, in regards to organization internals, I’d like to address
something I really wanted to say for the past few weeks: sometimes, I
felt burnt out. My initial response to your enthusiasm over my audio
tutorials was that I’ll ask for justifications for producing an
updated version, seeing that there are countless free videos and
tutorials out there. This was partly because I truly felt burnt out
with academics, speech and debate competitions and what not
(especially after a debate regarding a potential feature held not long
ago), at one point telling myself that I’ll retire from the NVDA
community sooner than later and feeling as though I carried important
burdens on my shoulders. But you didn’t see that justification post;
instead, I posted links to where you can download the 2018 version of
my audio tutorial series. In effect, I’ve given up my Christmas
holidays for this community, knowing that I needed a time to listen to
you all and do something about it. All this was possible because of a
simple act of listening and thinking about what the community means to
me and what my work means to everyone. I’m committed to finishing
Welcome to NVDA 2018 series before NVDA 2018.1 ships, with several
addenda coming after that, all because of support from this community
and outsiders. And I promise again: The Welcome to NVDA 2018 series
was, and will remain, free for all. All I ask of you in return is
donate to a cause that makes equal access to technology possible,
especially during this holiday season and beyond.

I’m sure for many of you, my musings above are a bit hard to digest.
Now you know why I don’t trust Web AIM survey results, quality is just
as important as quantity, and read a confession from a community
leader on his inner feelings. But there are two more things you need
to know, something all of us needs to think about:

Community leaders won’t stay with you forever. In early 2017, I sensed
that a long-time NVDA developer would leave this community for
something better. Only I and others didn’t know until summer that it
would be Jamie moving onto Mozilla Foundation.

I also felt, back in early 2017, that my active time with the NVDA
community is slowly drawing to a close. I don’t know when it’ll
happen, but I’ve been laying foundations for the next generation of
developers and enthusiasts to take the lead. This is one of the
reasons for setting up the devlearning subgroup, because I felt it is
time for me and other leaders to teach NVDA internals and other
concepts to the next group of community leaders and developers so they
can bring NVDA to the next level and do more amazing things than I and
others did (in my case, for the past five years).

Lastly, I sense a time when this community will face a sharp divide to
a point where people will start questioning the merits of this
community. I only told a select few earlier because it wasn’t right
for me to disclose it early and for them to prepare a plan. The screen
shade debate is, in fact, a sort of a preview of what is to come. One
of the fundamental questions you will face at that time will be
whether you still have your first love for NVDA, and whether you still
have your original reasons for joining this community. The survival of
this community at that time will depend on your ability to unite to
face a difficult situation, even if that means facing possible splits.
One thing you should NOT do at that time: ignoring new users and
outside critics, because they are influential opinion leaders and are
key stakeholders in NVDA’s future. One thing you SHOULD do though:
listen to others and think critically.

Hope this makes sense.

Merry (early) Christmas,

Joseph

--
A acção pode nem sempre ser felicidade, mas não há felicidade sem acção...

Join nvda@nvda.groups.io to automatically receive all group messages.