In playing a bit more, once I open any site that has audio/video content, in order to access the control to change what's blocked (or not blocked) I have to throw focus to the address box, Shift+Tab once, then right arrow to get to that button. Hitting enter then gives a dropdown related to that site and its specific permissions, and the autoplay selection is the first in the permissions list.
It appears that all adding/changing of sites is done via this method, while the Autoplay Settings dialog is mostly focused on removing individual sites or all sites for which you've set non-default handling.
Brian - Windows 10 Pro, 64-Bit, Version 2004, Build 19041
It’s hard waking up and realizing it’s not always black and white.
~ Kelley Boorn