I think usability splits into at least four somewhat overlapping parts: visual interface, documentation, installation, and making modification no harder than it has to be.
The visual interface may be the most specialized of the bunch--people's eyes and minds vary a lot, so a "good" user interface is, at best, one that's good for a lot of the target audience.
Interaction with users: I've heard that one way of improving user interfaces is to torture programmers by making them watch from behind one way glass or otherwise remotely while people try to use their programs.
After having accepted that usability is a separate skill, I've wondered if it's possible to outsource it, whether by recruiting usability mavens into the open source community or by having some of it done for pay by compahies which get the money back by offering conveniant packages. Eric's told me that both are being tried.
Random usability thing: the little black triangle in Firefox. I'd been using Firefox for months, and enduring having my search window stuck on Ebay for most of them before I noticed the little black triangle which gives pull-down menus. How many of you have noticed your little black triangles? There are two more by the forward and backward arrows.
I think the little black triangle could be improved by making it a little black downward pointing arrow, but it really is a fight between making all the handy features accessible and not cluttering the screen up too much.
And I've finally gotten a handle on adding new bookmarks--afaik, it isn't documented. Arguably, it's my fault for using the slow bookmark dragging process instead of noticing the Move icon right there at the top of the manage bookmarks screen. If there's any way to put a new bookmark where I want it as a simple part of adding it, I haven't discovered it yet.
It also took me a while to realize that checking "open in sidebar" does not improve my life in any way whatsoever.
On the other hand, it's not my fault that the slot for the url is called "location".