This is not to say that marketable work by the industry's standards can't also be brilliant. But the publishing industry is not a meritocracy. By its nature, art is individual. A decade ago, what the mass majority of readers could read at all was selected by a very small number of people. If their tastes didn't match up with the tastes of the reader... then three people lost out: publisher (who made no money), reader (who had to settle for something they didn't care for or not read at all)... and writer (who wasn't bought). It also means there's a "house style" throughout the entire industry, one so pervasive most of us don't even realize that we're all writing books with similar structures because those are the books that sell to generations of readers who have been taught this is what "good books" look like.
But good stories come in more forms than the ones we're entrained to accept as "well-written." Every currently acceptable style was once avante garde. The novel itself is only a few centuries old. A lot of the artificial distinctions created by a paper society no longer need apply: novellas aren't "bad" art because they've been considered unsellable by our existing business model. Books don't have to be written in trilogies or issued in chunks. "It's the way we've always done it" is not a valid argument unless the alternatives are unworkable... and I should know. I'm a pretty old-fashioned sort and I get that argument all the time.
And I agree with every word-- I'm especially fond of the point that everything we read now was avante-garde (or at least an experiment that worked) at some time.
I'm fascinated by the way fanfic writers and readers have developed a complex system of indexing and recommendation from the bottom up. I was tempted to start this post by saying that no one reads the slushpile for fun, but fanfic readers do, or at least they read it for free.
The limitation I can see is that, as far as I can tell, it's optimized for finding the fiction which scratches itches which the readers already know they have.
There's no system yet for finding stories that readers don't know yet that they want, and those are the stories haikujaguar is especially advocating for.
I'm not sure what such a system would look like-- it isn't likely to be automated, and I don't think it can be unless we've at least got AI which can understand natural language and human beings very well. I don't think we're very close to AI at that level.
It would probably be possible to have automated systems to identify the worst conventional prose, but they'd write off James Joyce and not be able to recognize fiction written in dialects-- and dialects are definitely part of the range I hope would be tremendously expanded by decentralized publishing. In any case, this is just snipping off one type of badness. It couldn't begin to recognize the smoothly written story which just doesn't quite satisfy, or is a good enough example of its type but nothing special.
I can't imagine a program (at anything near the current level of knowledge) which can manage one of the hard problems-- identifying fiction which is likely to induce a trance, even for the general public, let alone optimizing trance induction for individual readers. 
And trance induction is just the beginning-- there's plenty more to stories that people really love.
Amazon's and pandora's recommendation systems have limits-- I've heard that amazon's is really uninspired for sf , and pandora's isn't much good if you're familiar with a musical field.
OK, if it can't be automated, how can people do it?
We don't necessarily want those programs.....if they can identify trance-induction, they can probably produce it, too. I recommend Leiber's The Silver Eggheads, a very funny sf satire about the future of publishing. The artificially generated fiction is read once and forgotten immediately, and called wordwooze. Only robots write their own fiction.
I don't use amazon recommendations for fiction much, but I've found they're very good for books on T'ai Chi and Feldenkrais.