Sunday 24 September 2017

Fairness and the SPFBO

No System is Perfect

Kenneth Arrow won a Nobel prize for proving that no voting system is perfect. Each has its strengths but also, under certain conditions, will seem deeply unfair to someone. All democracies are flawed.

This observation extends to many other systems. Importantly for us it extends to selecting the "best" book from a field of 300.

It is possible to see a flaw and suggest a fix. But this will still leave the system flawed - just in a new way that may seem better to you and worse to someone else.

A natural consequence of this truth is that any such system will be subject to valid criticism. This is something that the system has to live with.

In addition to the unavoidable flaws a system may be corrupt. Flaws cannot be avoided but corruption can. A system that allows room for corruption (unfairness) will attract accusations of foul play even if none is actually happening. Hence it is important to have rules that allow no room for it.

For the SPFBO it is better that we select a good book by a process that is not only fair but seen to be fair, than to select the best book by a process that has room for unfairness in it (even if none is actually present).

Requiring each blog to choose the best book from their batch is a flaw since the blog may feel that a better book exists in the remaining 270 and has not made it to the final. It is a flaw that can only be fixed by introducing other flaws.

One seriously considered fix was the "Senlin Net" whereby a blog with a very strong 2nd choice (like last year with Pornokitsch and Senlin Ascends) could offer that book to other blogs to consider as their finalist instead of one from the batch they were assigned.

This is a reasonable idea whose only drawbacks are that it involves more work for the blogs (in the limit we could ask all blogs to read all books, resulting in a LOT more work but a better result - the line has to be drawn somewhere though) and that it may feel harsh to the "best" book from the batch that gets bumped to make room.

Other suggestions for the Senlin Net involve additional finalists. But this increases the workload for bloggers while diluting the cachet of being a finalist.

The Senlin Net was never entirely ruled out.

Importantly the Senlin Net suggestion always involved a push rather than a pull. I.e. one blogger with no connection to the author pushes the title out for consideration.

All bloggers are asked at the start of the SPFBO to identify any books in their batch by authors who they know. Those titles are then swapped with ones from other blogs. This is naturally to ensure fairness. It's not because anyone is saying they don't trust the blog to be impartial given the relationship with the author - it is to stop anyone being able to say that.

A possibility that has been suggested is that a Senlin Net be employed based on a pull. I.e. that the blog could select any entry over the best from their own batch.

The reason this isn't tenable is that we then have to consider the mechanism by which this book came to the blog's attention. It would be a system that favoured:

i) Authors with a high profile.
ii) Authors with pockets deep enough to send out physical review copies.
iii) Authors with whom the blog has some relationship, anything from best friend to casual social media acquaintance.

This is not to say that any of these things would happen or are even likely to happen, but that someone would then be able to say that.

In a contest where 97% of all entries are cut at the first stage it is natural for some to look for fault. The SPFBO must admit to its unavoidable flaws and to allow no space for accusations of unfairness.

A Senlin Net based on pull would allow the accusation that the blog that did it was always going to choose their friend. Valid or not, such an accusation would taint the contest and lessen the value of "finalist". The taint would spread to the winner and to past and future winners.

So, in order to avoid such issues, there will be no Senlin Net based on pull.

Whether there will be one based on push is still a matter for consideration and will only need to be considered should a blogger have a second choice they were very upset not to be able to put through, and another blogger is unhappy that the best from their batch is not good enough.

No comments:

Post a Comment