For most of recorded human history, philosophers have debated what free will is and isn't. Recent work by social psychologists suggests that free will is actually an emotonally-driven moral accountability system justifying punishments and rewards, and is not based entirely on our subjective experience of the world.
I'll move on to that in a second, but first let's acknowledge that the idea of free will has never been particularly well-defined outside of a moral accountability context. As MIT philosopher Richard Holton explains in this charming video, the idea as it is commonly used already conflates determinism, predictability, and foreknowledge into a single gooey mess:
I thought about this when I ran across Azim Shariff and Kathleen Vohs' article about the social implications of free will in this month's Scientific American: what do we really mean when we say "free will," and what kind of social function does it serve?
Shariff and Vohs are no strangers to that question. Shariff co-wrote an important study earlier this year suggesting that people believe in free will because they think of other people as having it, not because they have it themselves, and that they become more likely to rely on the idea when they want to see other people punished for something. Vohs was principal author of an equally significant study that looked at the other side of the question, finding that when people didn't believe in free will they were more likely to cheat.
What this tells me is that free will is quite often an emotional tool used to justify retribution—punishment justified for its own sake, even if it doesn't do anything to address the damage done by the crime, reduce the likelihood that the offender will commit similar crimes in the future, or deter others from following suit. The villain in an old Western deserves to die because he's a terrible human being who did terrible things according to his own free will, and this makes him someone who fully deserves to suffer in kind. The hero does better, and this makes any suffering he endures unjust.
What Shariff and Vohs have found is that at least a significant percentage of the population relies on this kind of understanding of free will when they make moral decisions and assess the moral decisions of others. Their conclusion is supported by a 2010 study finding that a majority of participants, drawn from four different communities, believe that "(a) our universe is indeterministic and (b) moral responsibility is not compatible with determinism."
But if free will helps us fear retribution and inflict retribution on others, it impedes our understanding of justice in other ways. The concept of free will ignores the role that personal history, mental illness, unequal access to resources, trauma, and other factors may play in our decisionmaking processes—which gives us less incentive to make sure that people are able to live the sorts of lives that allow them to avoid self-destructive, and other-destructive, behavior. "It is," the 19th-century American political philosopher Frederick Douglass wrote, "easier to build strong children than repair broken men." Easier, yes—but it doesn't satisfy our need for retribution, and that need seems to be inextricably bound with the concept of free will.