Our discussion yesterday and playing with a touch pad today reminded me
of this question. I've tried moving systematically with a physical
mouse and using a mouse pad but even if I move in the same or similar
ways, the results don't seem to follow a pattern and make sense. Why,
for example, if I move the mouse down repeatedly on the desktop, don't I
move down the list Because I gather that if I'm in the wrong place when
I move, nothing will happen or I'll be moved to the wrong place, I've
tried moving down with my finger on different parts of the mouse pad,
from left to right. ?

Gene

Assuming you are on a windows 10 or 11 machine, and talking about the items on the desktop-

It’s because of the analogy that is used for sighted people- a desktop is literally a desktop.  The items on the desktop can are like literal tools or plates or cups on a table.  When you dump things on a desktop, they can be any position.  On a windows desktop, the items on it are designed to be able to be dragged into any position and grouped in any way a sighted person finds reasonable.  

Because it is a file system, there is an order to it that screen readers can take advantage of- and most sighted people take advantage of it to ‘clean up the desktop’ and place their icons in a reasonable grid ordering.

There is a reason that iPhones are designed to force the user to lay out their app icons in a grid- it just makes so much more sense that way and you can’t do silly stuff like drag your program icon on top of another program icon. 

 On the off chance that you are talking about right-clicking on something and opening menus:

 It’s because where the menu appears on the screen is based on two things: how big the rectangular area of the menu is, and where it appears relative to the sides of the screen.

You can imagine it by taking an index card and holding up to your touchscreen.  The list is written on the index card.  The index card has to stay entirely on the screen of the laptop. 

So you can feel the edges of and the corners of the index card, and feel approximately where the menu has to be positioned to to stay entirely on the screen.

The most complicated, position-changing menus are the sorts of menus that come up when you right-click on something.  The rule is that the corner of the menu appears where you right-clicked- but WHICH corner is decided by how close the position you clicked is to the edge of the screen and how big the menu is.  

If the menu can fit down and to the right, then where you clicked will be the upper left hand corner.  You are right on the corner of the menu. So you’ll need to move to the right- about half of one finger’s width on a touchscreen, and I don’t know exactly how much with the mouse but if it speaks an item from the menu you’re on the menu and then you can go straight down to go through the list in order.

If your menu can fit to the right, but not down, your click will be the lower left corner of the menu- like the lower left of your index card. Your pointer is right on the edge of the menu- so you will need to move the mouse to the right by a little- if you move it about half the width of one finger, that’s usually enough to stay within the area.  

If your menu can fit down, but not to the right, then your menu will appear below the mouse.  The mouse will no longer be on a corner- instead, it will be on the top edge of the index card, and the menu will be below it with the right edge jammed up to the edge of the screen- so you will need to move the mouse down and you will be moving from the top to the bottom.

If your menu can’t fit down or to the right, then the mouse pointer will be on the bottom edge and the menu will appear above it- so you will need to move the mouse up and you will be moving from the bottom to the top.  

Menu Lists that are controlled by actual clickable buttons are generally easier, because they generally appear either right above the button you clicked if it’s near the bottom of the screen ( such as the start menu) or right below the button (such as the file tab on a standard program’s menu bar). Then you can confidently steer straight up or straight down.

If you have submenus, the same ‘fit on the screen, and don’t cover up the menus’ rules apply.  You can simulate this by taking two index cards, positioning the first one fully on the screen, and then figure out where you can position the second one so that it both touches the place you ‘clicked’ on the first menu, does not cover up the first menu, and stays fully visible on the screen.

So those are the rules.  It’s way easier, when dealing with menus without sight, to take advantage of the fact that pressing the ‘down’ button always puts the focus on the first item of the menu. 

Also, the start menu is a three-panel split menu and is kind of a shitshow that follows it’s own rules and needs both tabbing AND arrow keys to get through.

Also, the menus vary in size- so instead of index cards as an analogy, what you have are arbitrary pieces of paper that can be any size but only have to follow the ‘stay in the area of the screen and follow the positioning rules for submenus.

In other words, unless the system is designed to be laid out in a coherent, consistent grid, like an iPhone, where your icons on a desktop are physically positioned on the screen depends on where they were dragged or if they were ‘cleaned up’ and put in a grid order recently.   And where the menus appear during clicks depends on how big they are and their proximity to the edges of the screen.

-Tara

On Sun, Nov 20, 2022 at 4:04 AM Gene <gsasner@...> wrote:

Our discussion yesterday and playing with a touch pad today reminded me of this question.  I've tried moving systematically with a physical mouse and using a mouse pad but even if I move in the same or similar ways, the results don't seem to follow a pattern and make sense.  Why, for example, if I move the mouse down repeatedly on the desktop, don't I move down the list  Because I gather that if I'm in the wrong place when I move, nothing will happen or I'll be moved to the wrong place, I've tried moving down with my finger on different parts of the mouse pad, from left to right.  ?

Gene

On Sun, Nov 20, 2022 at 12:04 PM, Gene wrote:

When I move through the desktop, it appears that things are organized in columns.  Is that how they appear to a sighted user?  If not, I don't understand how they appear in any rational order.

-
While they can be, they very often aren't.  I tend to keep my desktop in "clusters" where related things, e.g., all web browsers, are in one cluster, all audio players are in another, etc.

What many do not understand is that the Desktop is just a folder, like any other, that has a "very special relationship" with Windows in terms of how it is displayed as the desktop on screen.  If you open the Desktop folder in File Explorer, it us subject to precisely the same rules as any other folder as far as sort order you've chosen goes.  When displayed as the desktop itself, anything can be anywhere.

When a screen reader is traversing the desktop, it relies on File Explorer order, hence the reason that first letter nav works like it does.  But two icons with the same first letter on the displayed desktop need not be anywhere near to each other.

A touchscreen on a laptop is, effectively, no different than the touchscreen on a phone, and if you're using a screen reader with mouse tracking on, you can demonstrate what I've said above my gliding your finger over the desktop in some orderly fashion.  If you've been putting shortcuts on the desktop over a long period of time, and no one (typically sighted) has rearranged them they will likely be in a sort of row/column layout starting at the upper left, but they will not be in anything like alphabetical order.  And if you've deleted shortcuts over time, that doesn't cause the remaining ones to move, there is just a hole where one you've deleted was, and that hole (or the first open hole) is what gets filled when you next create one.

I hate touchscreens because I so often point to the screen to show someone something (a sighted someone, obviously) and end up accidentally activating things I do not intend to.  But as far as someone who cannot see getting an understanding of how the desktop is physically laid out, if they're accustomed to using a smartphone then that's the fastest way to learn about it.  The mouse pointer always follows your finger.  And if the mouse pointer happens to be near the upper right, and you touch the lower left, it jumps to where your finger has touched.
--

Brian - Windows 10, 64-Bit, Version 22H2, Build 19045

There are two novels that can change a bookish fourteen-year-old's life: The Lord of the Rings and Atlas Shrugged.  One is a childish fantasy that often engenders a lifelong obsession with its unbelievable heroes, leading to an emotionally stunted, socially crippled adulthood, unable to deal with the real world. The other, of course, involves orcs.

    ~ John Rogers

On Sun, Nov 20, 2022 at 10:48 AM, Tara Roys wrote:

The touchpad is NOT a miniature version of screen.

-
Yep, and that's critical to remember. 

I bring this up only because that's in direct contrast to an actual touchscreen itself, where if you touch it the mouse pointer does move, immediately, to where you've touched on the screen.

A touchpad/mousepad and a touchscreen behave very differently.
 
--

Brian - Windows 10, 64-Bit, Version 22H2, Build 19045

There are two novels that can change a bookish fourteen-year-old's life: The Lord of the Rings and Atlas Shrugged.  One is a childish fantasy that often engenders a lifelong obsession with its unbelievable heroes, leading to an emotionally stunted, socially crippled adulthood, unable to deal with the real world. The other, of course, involves orcs.

    ~ John Rogers

On Sun, Nov 20, 2022 at 12:04 PM, Gene wrote:

If not, I don't understand how they appear in any rational order.

-
Gene,

I want to comment on this separately.

"Rational order" is, when it comes to visual attention, whatever works for you.  Everyone has different systems of organization that they prefer, and they tend to set up their desktops accordingly.

One small example, for me, is that I always want my Recycle Bin at the lower right corner of the screen, regardless of what else is on the screen or where.

As Joseph Lee has pointed out, repeatedly, sighted folks take in the entire screen as a gestalt and since it's generally the person using the machine that has organized the screen to their liking, what's logical/rational to one person is a hopeless mess to another.  

It's very much akin to the "kitchen organization" wars that get used in comedies so frequently.  Everyone thinks that their own system is "best" and "the only one."  But none of them are exactly the same.

Visual attention has characteristics that are sui generis and that cannot really be adequately explained to someone who has never been able to see in a way that they can posess a true "gut feel" for how it works.  It's no different than being unable to explain what "blue" is or what the Mona Lisa really looks like, in detail, like you experience it if you can see.  Some things are, truly, ineffable.
--

Brian - Windows 10, 64-Bit, Version 22H2, Build 19045

There are two novels that can change a bookish fourteen-year-old's life: The Lord of the Rings and Atlas Shrugged.  One is a childish fantasy that often engenders a lifelong obsession with its unbelievable heroes, leading to an emotionally stunted, socially crippled adulthood, unable to deal with the real world. The other, of course, involves orcs.

    ~ John Rogers

On Sun, Nov 20, 2022 at 12:39 PM, Gene wrote:

I thought you were saying in your original message that they [a mousepad/touchpad and touchscreen] behave the same but I just saw your additional comment. 

-
Nope, as you've seen.

The easiest analogy I can give is that a touchpad/trackpad/mousepad on a laptop is kinda like a regular mouse and plain old mousepad, but inverted.

With a regular mouse on a mousepad, you move the mouse itself around and it moves the mouse pointer from wherever it happens to be relative to how you're moving the mouse itself.  The initial position of the mouse pointer is either "wherever you last left it" or wherever it lands when a new program opens full screen.  It does not automatically re-center or do anything else, it just begins moving from where it is.

Now, when thinking of a trackpad on a laptop, think of the mouse clinging to the bottom of that trackpad, and your finger being the equivalent of your hand on a regular mouse.  When you start moving your finger it simply starts moving the mouse pointer from wherever it happens to be sitting at that point.  The mouse pointer does not move, at all, just from the touch of a fingertip at ANY place on the trackpad, you have to begin moving your finger after it's down to get the mouse pointer to move.  It's like your finger is moving that "clinging mouse" from the upper side.  The mouse itself (NOT the pointer) is instantly positioned under your magic finger, but the mouse pointer does not move until or unless your finger does.  It's the same as picking up an optical mouse off the table and moving it through the air, then placing it back down.  This doesn't result in mouse pointer movement, either, the optical port has to have a way of detecting movement by pattern changes directly beneath it, and it doesn't have that in mid-air.

That's the huge difference between a conventional mouse or a trackpad, and a touchscreen.  On a touchscreen (any touchscreen) the mouse pointer (or its equivalent) is immediately shifted to where you've touched initially, then moves from there.  Yes, there can be certain exceptions when a screen reader like TalkBack or VoiceOver is involved when it comes to activating a button, but you have to actually get to that button using your finger, first, then you're allowed to activate by double-tapping (most commonly) anywhere on the screen.  Otherwise the targeting involved, particularly for keyboard buttons, would be well-nigh impossible for many who cannot see them to achieve.
--

Brian - Windows 10, 64-Bit, Version 22H2, Build 19045

There are two novels that can change a bookish fourteen-year-old's life: The Lord of the Rings and Atlas Shrugged.  One is a childish fantasy that often engenders a lifelong obsession with its unbelievable heroes, leading to an emotionally stunted, socially crippled adulthood, unable to deal with the real world. The other, of course, involves orcs.

    ~ John Rogers

On Sun, Nov 20, 2022 at 12:45 PM, Gene wrote:

I sometimes think about how differently blind people use computers as an example of why sighted people have such a hard time imagining doing things without sight.  We move a cursor through a document and hear where we are, whereas a sighted person sees the screen and can jump to where he/she wants to work in a document by moving the mouse there and clicking to place the cursor there. 

-
Yep.

And there's even a difference in sensory filtering on both sides.  When I began dealing with screen readers, the verbosity drove me insane.  I simply could not, initially, filter any of it out as "background noise."  I was constantly hitting CTRL or ESC to shut the darned thing up so I could think.  I still do that, but not nearly to the extent that I used to have to.  I'm now accustomed to much of the "endless chatter" and can ignore it knowing what I'm about to do next and almost pretending it's not there.  It's the auditory equivalent of visual filtering out of irrelevant information, which occurs automatically and beneath the level of consciousness.

Blind people who've always used screen readers and other speaking devices develop the auditory equivalent of "visual garbage filtering" for certain speech just a sighted people do for all sorts of visual stimuli.  One could never, ever, do something like driving if you were attending, literally, to everything that is within your field of vision at once.  There's just too much and if you were doing that all the time (driving or not) your brain would burn up.  Right now, as I type this, I intentionally focused away for a few seconds.  I've got all of the furniture in my den, the trees outside in the bright sun with remaining leaves fluttering in the wind, my partner seated to my right working on his laptop, and more surrounding me.  Yet, when I'm typing, and reading my screen as I type, my visual attention is focused on the screen, with nothing but that little hamster on the wheel in my brain on "emergency monitoring" duty for all the rest of it.  Provided something does not occur that should redirect my visual attention elsewhere, it's almost as though none of it is there.

I have always marveled at the blind individuals I know who are incredibly good at document formatting.  Almost all (not absolutely all, but a great deal) of document formatting has to do with visuals.  If you've never been able to see, the "visuals" you're dealing with are entirely abstractions, and you have absolutely no idea of what they look like, because you cannot know that if you've never been able to see.  Yet, somehow, there are document formatting whiz kids who have never been able to see.  How they structure that, in a non-visual way, that translates to a visually correct/satisfying arrangement will always remain a complete mystery to me.  And that's because I can see and dealing with the visual by vision is so much easier than any other way.  We're all inherently lazy in that whatever is the easiest way to get the result that we want or is needed is generally what we will do.  And when it comes to sensory input, things defined largely or entirely by a given one means using that given one with all others being ignored.
--

Brian - Windows 10, 64-Bit, Version 22H2, Build 19045

There are two novels that can change a bookish fourteen-year-old's life: The Lord of the Rings and Atlas Shrugged.  One is a childish fantasy that often engenders a lifelong obsession with its unbelievable heroes, leading to an emotionally stunted, socially crippled adulthood, unable to deal with the real world. The other, of course, involves orcs.

    ~ John Rogers

It may be that I'm so used to doing it that I'm not sure what you filter out.  Is an example moving to something on the desktop and, after hearing what the item is, more or less ignoring information like one of thirty?

While I hear the information if I just let the screen-reader read, if I'm thinking about something else, I ignore it.

Gene

On 11/20/2022 12:09 PM, Brian Vogel wrote:

On Sun, Nov 20, 2022 at 12:45 PM, Gene wrote:

I sometimes think about how differently blind people use computers as an example of why sighted people have such a hard time imagining doing things without sight.  We move a cursor through a document and hear where we are, whereas a sighted person sees the screen and can jump to where he/she wants to work in a document by moving the mouse there and clicking to place the cursor there. 

-
Yep.

And there's even a difference in sensory filtering on both sides.  When I began dealing with screen readers, the verbosity drove me insane.  I simply could not, initially, filter any of it out as "background noise."  I was constantly hitting CTRL or ESC to shut the darned thing up so I could think.  I still do that, but not nearly to the extent that I used to have to.  I'm now accustomed to much of the "endless chatter" and can ignore it knowing what I'm about to do next and almost pretending it's not there.  It's the auditory equivalent of visual filtering out of irrelevant information, which occurs automatically and beneath the level of consciousness.

Blind people who've always used screen readers and other speaking devices develop the auditory equivalent of "visual garbage filtering" for certain speech just a sighted people do for all sorts of visual stimuli.  One could never, ever, do something like driving if you were attending, literally, to everything that is within your field of vision at once.  There's just too much and if you were doing that all the time (driving or not) your brain would burn up.  Right now, as I type this, I intentionally focused away for a few seconds.  I've got all of the furniture in my den, the trees outside in the bright sun with remaining leaves fluttering in the wind, my partner seated to my right working on his laptop, and more surrounding me.  Yet, when I'm typing, and reading my screen as I type, my visual attention is focused on the screen, with nothing but that little hamster on the wheel in my brain on "emergency monitoring" duty for all the rest of it.  Provided something does not occur that should redirect my visual attention elsewhere, it's almost as though none of it is there.

I have always marveled at the blind individuals I know who are incredibly good at document formatting.  Almost all (not absolutely all, but a great deal) of document formatting has to do with visuals.  If you've never been able to see, the "visuals" you're dealing with are entirely abstractions, and you have absolutely no idea of what they look like, because you cannot know that if you've never been able to see.  Yet, somehow, there are document formatting whiz kids who have never been able to see.  How they structure that, in a non-visual way, that translates to a visually correct/satisfying arrangement will always remain a complete mystery to me.  And that's because I can see and dealing with the visual by vision is so much easier than any other way.  We're all inherently lazy in that whatever is the easiest way to get the result that we want or is needed is generally what we will do.  And when it comes to sensory input, things defined largely or entirely by a given one means using that given one with all others being ignored.
--

Brian - Windows 10, 64-Bit, Version 22H2, Build 19045

There are two novels that can change a bookish fourteen-year-old's life: The Lord of the Rings and Atlas Shrugged.  One is a childish fantasy that often engenders a lifelong obsession with its unbelievable heroes, leading to an emotionally stunted, socially crippled adulthood, unable to deal with the real world. The other, of course, involves orcs.

    ~ John Rogers

On Sun, Nov 20, 2022 at 01:03 PM, Gene wrote:

I wasn't sure before what the difference was between a mouse pad and a touch screen.

-
The following is not meant as hair-splitting, because I've fallen prey to it, too.  But the term "mouse pad" as two words, or even one, should be avoided if the reference is to a trackpad/touchpad.

I say that because actual pads meant for use under conventional mice are incredibly common, and are always referred to as mouse pads in that context.  So, it can be confusing.

If someone is in the middle of a conversation regarding regular mice and laptop trackpads/touchpads, then mouse pad or mousepad would be clear.  But were I to have stumbled upon what you wrote above independent of this conversation, I would be sorely confused, as a mouse pad (literal) and touchscreen are not related in any way.

Just figured you'd like to know.  I've got to exercise more self-discipline on that front myself.
 
--

Brian - Windows 10, 64-Bit, Version 22H2, Build 19045

There are two novels that can change a bookish fourteen-year-old's life: The Lord of the Rings and Atlas Shrugged.  One is a childish fantasy that often engenders a lifelong obsession with its unbelievable heroes, leading to an emotionally stunted, socially crippled adulthood, unable to deal with the real world. The other, of course, involves orcs.

    ~ John Rogers

On Sun, Nov 20, 2022 at 12:45 PM, Gene wrote:

I sometimes think about how differently blind people use computers as an example of why sighted people have such a hard time imagining doing things without sight.  We move a cursor through a document and hear where we are, whereas a sighted person sees the screen and can jump to where he/she wants to work in a document by moving the mouse there and clicking to place the cursor there. 

-
Yep.

And there's even a difference in sensory filtering on both sides.  When I began dealing with screen readers, the verbosity drove me insane.  I simply could not, initially, filter any of it out as "background noise."  I was constantly hitting CTRL or ESC to shut the darned thing up so I could think.  I still do that, but not nearly to the extent that I used to have to.  I'm now accustomed to much of the "endless chatter" and can ignore it knowing what I'm about to do next and almost pretending it's not there.  It's the auditory equivalent of visual filtering out of irrelevant information, which occurs automatically and beneath the level of consciousness.

Blind people who've always used screen readers and other speaking devices develop the auditory equivalent of "visual garbage filtering" for certain speech just a sighted people do for all sorts of visual stimuli.  One could never, ever, do something like driving if you were attending, literally, to everything that is within your field of vision at once.  There's just too much and if you were doing that all the time (driving or not) your brain would burn up.  Right now, as I type this, I intentionally focused away for a few seconds.  I've got all of the furniture in my den, the trees outside in the bright sun with remaining leaves fluttering in the wind, my partner seated to my right working on his laptop, and more surrounding me.  Yet, when I'm typing, and reading my screen as I type, my visual attention is focused on the screen, with nothing but that little hamster on the wheel in my brain on "emergency monitoring" duty for all the rest of it.  Provided something does not occur that should redirect my visual attention elsewhere, it's almost as though none of it is there.

I have always marveled at the blind individuals I know who are incredibly good at document formatting.  Almost all (not absolutely all, but a great deal) of document formatting has to do with visuals.  If you've never been able to see, the "visuals" you're dealing with are entirely abstractions, and you have absolutely no idea of what they look like, because you cannot know that if you've never been able to see.  Yet, somehow, there are document formatting whiz kids who have never been able to see.  How they structure that, in a non-visual way, that translates to a visually correct/satisfying arrangement will always remain a complete mystery to me.  And that's because I can see and dealing with the visual by vision is so much easier than any other way.  We're all inherently lazy in that whatever is the easiest way to get the result that we want or is needed is generally what we will do.  And when it comes to sensory input, things defined largely or entirely by a given one means using that given one with all others being ignored.
--

Brian - Windows 10, 64-Bit, Version 22H2, Build 19045

There are two novels that can change a bookish fourteen-year-old's life: The Lord of the Rings and Atlas Shrugged.  One is a childish fantasy that often engenders a lifelong obsession with its unbelievable heroes, leading to an emotionally stunted, socially crippled adulthood, unable to deal with the real world. The other, of course, involves orcs.

    ~ John Rogers