Deinonychus antirrhopus

Thanks for the kind comments.

I agree with you that there is still value in the market price. But if there is anything to draw from Manski's argument is that the market price isn't so tightly related to actual probability as we might otherwise think.

I think Alex at MarginalRevolution said it better than I could in his "My take" section.

I should point out that Manski has proposed another mechanism to this ... subjective probability surveys. Ask people directly what they believe the *probability* of an event is as opposed to just whether they "expect" it or not.

In practice, market-based preference revelatory mechanisms do have value. Market participation is more credible because you are putting something on the line (although you could perhaps creatively emulate this with a survey); and as you noted, they seem to work, give or take, largely because they do move up or down as you'd expect the "real probability" to move. Horse racing, sports gambling, etc., all shows that these sorts of markets can do ok in practice ... but they aren't perfect. Longshots tend to be over-valued for one example.

Quick edit comment: Further, as the price of the contract increases the interval containing the mean probability belief also increases.

I wasn't sure what you meant by that. The values of the upper and lower bounds increase, but the actual distance between the bounds varies non-monotonically. If the price is zero, the bounds are zero and zero. If the price is .5, the distance between bounds is maximized (.25 to .75). If the price is 1, the distance is again zero with both bounds being equal to 1.

This is intuitive. At a price of .5, you have a very large range (.5) for each and every participant in the market (sellers are between zero and .5, buyers are between .5 and 1). As the price goes up, more participants are in the smaller range and fewer are in the larger, so the aggregate effect is that you can be more certain about where the overall mean lies. The extreme is where everyone is at 1. The reverse happens as prices approach zero.

BTW, I've enjoyed reading your blog for awhile. Keep up the good work.

Victor,

Thanks for the comment on the interval. I see how one could read that as meaning the size of the interval increases, when I meant, as you note, the bounds of the interval both increase in value. I'll add that to the update to clear up any possible confusion.

Actually, Robin Hanson had some good ideas (PDF) about how to craft markets around poll-like media. I recall that the Policy Analysis Market, which was mislabled as a "terror market", would have this form. Participants would correct the current guess. Based on scoring rules, at the time that the question (eg, election results are issued) is determined, they could earn or lose money. They could also sell out and reduce their investment early.

You're confusing yourself with probability-for. It's not a feeling that people have, at least if you want to avoid unnecessary paradoxes.

Take the differing probabilities as coming from differing information. B knows that two heads have already fallen, C knows only about one of them. B and C have differing probabilities of three heads in a row, but it comes from what they know already differing.

The mean probability does not represent anything at all about the ``real'' probability. It's an average you'd have a hard time saying what it might signify, as if you had averaged the number of apples and the number of oranges available in Florida. What is it? Fruit? Not exactly. It depends if you want an apple or an orange.

The market clearing price depends on how much money is in the hands of differing willing players with differing information.

IF you assume that everybody has access to the same information, and that's all the information there is, then you get a probability from the price. That is, you can't bet your own money with different odds and make money, on the average.

You're confusing yourself with probability-for. It's not a feeling that people have, at least if you want to avoid unnecessary paradoxes.

I strongly urge you to check out subjective probability. Subjective probability theory is pretty much "a feeling that people have". It is based on their knowledge and is a completely valid concept of probability.

The mean probability does not represent anything at all about the ``real'' probability. It's an average you'd have a hard time saying what it might signify, as if you had averaged the number of apples and the number of oranges available in Florida. What is it? Fruit? Not exactly. It depends if you want an apple or an orange.

Not quite. It is a statistic for the unknown parameter, in this case the probability that event X occurs.

The market clearing price depends on how much money is in the hands of differing willing players with differing information.

This part is also covered by Manksi when he looks at the effect of budgets that are no longer independent of people probability assessments.

A statistic for an unknown parameter isn't anything. It's like car commercials measuring ``comfort.'' They run some tests, add up the numbers, and come up with comfort. There is no comfort independent of the statistic.

You can't create a word that way. A word needs a use, and that in turn governs how the word changes, what ambiguities it takes on, and what tricks it can play.

A statistic for an unknown parameter defines no word.

You can be puzzled by the resulting paradox (``it _must_ mean something''), or not, I guess. I don't see why it's surprising. Words develop on their own or not at all.

There is a perfectly good and rigorous analysis of all this in terms of partial information; and you can model subjective belief, as far as its effects go, as partial information (just let each subject have his own Baysian priors, reflecting his personal dot-connection propensities); and you get out a rigorous demonstration that the mean probability is meaningless.

A statistic for an unknown parameter isn't anything.

All parameters (at least for the interesting things) are unknown. If we knew what all the parameters were there would be no reason for hte existence of statistics. Irrespective of one's philosophical base in viewing statistics, all parameters are treated as being unknowns (well the subjective view point doesn't hold with this, but the parameter has no set value though, so it is close enough).

There is a perfectly good and rigorous analysis of all this in terms of partial information; and you can model subjective belief, as far as its effects go, as partial information (just let each subject have his own Baysian priors, reflecting his personal dot-connection propensities); and you get out a rigorous demonstration that the mean probability is meaningless.

Funny that you would invoke Bayesianism. A Bayesian would not have a problem looking at these prices as probabilities. I know because I'm a Bayesian.

There's two kinds of Baysians; ones that count favorable and unfavorable states from among a fundamental uniform set, and those who take any old numbers and normalize them to total one.

The guy with the uniform fundamental set can actually prove things involving probability.

Well, using my own "fuzzy math" way of reasoning--I've never had the opportunity to learn calculus--I must say that I've always had a pretty good instinctive grasp of statistics, and it's long struck me that the value of these predictive markets is not in giving specific probabilities. I don't think that if Bush is trading at 7 that means he's 70% likely to win. I just think it means he's likely to win.

Take a look some time at Strategypage's prediction market, which you can see by clicking right here.

Their market gets far less attention than the others but it's fairly popular and it's often startling what questions they'll ask. Some of them are quite silly but sometimes they predict events that you wouldn't think a market could possibly predict. For example, they predicted (correctly) that John Kerry would choose John Edwards as his Veep, and on the question of about a dozen other candidates for the Veep slot, correctly predicted that none of them would be it.

They claim (based on their numbers, someone smarter than me would have to double-check) is that their market is correct over the lifetime of contract 84% of the time, and at contract close 94% of the time. Their market is I believe considered to have made the correct prediction whether the result was 51-49 or 99-1.

Thus, what seems instinctively right to me is not to say that if Kerry is trading at 30, that means he's something like 30% likely to win. What it means is that whether Kerry is trading at 49 or he's trading at 2, Kerry is 85-95% likely to lose.

But that's at market close (presumably, the day before the election).

It would also tell me that the price on any given day is not particularly meaningful, because the trade floor is fickle, people play price games, they become influenced by sudden events and become more uncertain (or more certain). If the prices are constantly jousting and going back and forth, that indicates THE MARKET is uncertain. Not the question, but the market for it. If trading is hugely overbalanced, however, the market's not uncertain and there's less likely to be a big shift before close.

In other words, I think you guys may be making a classic mistake: you're thinking that the variable is measuring something it's not. It's not measuring the candidate's odds of winning. It's measuring the certainty of the traders while the contract is still open. Once the contract closes, the market's prediction will have been made--and the market, if it's correct, should be correct highly often--94% in the case of Strategypage, maybe higher in bigger markets like ICS or the Iowa market.

Am I making my point? Otherwise these predictive markets really wouldn't be that impressive. If at final closing a question closed at $55 (out of a hundred), then you'd think the market is predicting a slightly-better-than 50% likelihood. It's not. It's predicting that the contract closing at 55 will in fact be what happens--not 55% or 60% or whatever, it's just stating flat out THIS WILL HAPPEN. And it will be right in the overhwelming majority of cases.

The prices don't match probability of occurrence. They match occurrences that the market is FIRMLY CERTAIN of, and the markets are RARELY wrong.

Am I making sense?

Since I typed a lot, let me summarize. Using Strategypage's market (since it's the only one I know which claims a specific success rate), they claim a market accuracy of 94% at close.

This means that if the contract on Bush winning closes at 51, Bush is 94% likely to win.

It means that if the contract on Bush winning closes at 49, Bush is 94% likely to lose.

If the contract on Bush closes at 85, Bush is still 94% likely to win.

If the contract on bush closes at 12, Bush is 94% likely to lose.

And the rest is either just noise or a measure of the confidence level of the investors, NOT the confidence level of the prediction itself.

Although I'm certain there is some analysis that could be done to say what the relationship is between investor confidence and how often the market mis-predicts, I would be willing to bet rather heavily that it's not a 1:1 ratio of probability to share price.