Our Man doesn't believe in polls. Sure, they exist and all, but so do weather stories - they are there to fill the void between the adverts in the paper on slow news days. Case in point: The Nikkei in a hardhitting report (er, regurgitated news release) has UNCOVERED THE TRUTH - Japanese worry alot. In fact more than any other nation, apparently. In fact, in fact, in fact, everyone is scared shitless about the current global and/or domestic situations and/or the living conditions of their families.
Er, well folk were seriously miffed about the "fiscal deficit," "safety of the food supply," "unemployment" and "global warming," all categories in which Japan's percentage exceeded the global average. "Concern about macroeconomic matters was notable," screams the Nikkei.
Meanwhile, the Japanese anxiety rates were below the global averages for such matters as "the war in Afghanistan," "food prices" and "gasoline prices."
Ehhhhhhhhhhhhhh?????????????
So is Japan quaking or not? Our Man guesses not. Later in the article we learn the survey was "conducted from spring through summer, targeting 11 countries. This is the first time Japan was included in the poll. The figures for Japan are based on valid responses from 500 people age 20 or older by JWT, a major New York-based ad agency and group firm of U.K. advertising giant WPP plc."
Ah-huhh. So, it's dated, over-simplified and from a tiny control group, probably done on the cheap by folk in a call centre somewhere where there ain't no better jobs. Oh, and written by non-journos. Type in the words "Ann Mack, director of trendspotting at JWT" (Anxious Annie is the PR bod who generated this nonsense) and "anxiety" into Google, and you get 70,800 links.
Seems the boss makes a good living worrying about anxiety.
Wanna stop worrying? Stop reading crap.
5 comments:
This is a silly post.
500 is a perfectly fine (some could argue quite large) sample size, as long as the research team made sure that the group it selected was a random sample (you don't mention which company JWT used to conduct the survey but I imagine that they used a company that adheres to general social research standards). It also doesn't matter much who asks the actual questions (you seem to imply that it was done by incompetents who could not get better jobs); as long as the call team (you imply that it was a phone survey) can read it's all good because they HAVE to stick to the script in front of them.
I don't understand what you mean by "oversimplified."
And polling done in spring and summer does not mean it's dated. There is usually a considerable lag time between when a poll was conducted and when the results are made public. For example, the 2008 Census was just released a couple of months ago. In fact, I would argue that the longer the lag time, the more reliable the results should be, since we can be reasonably assured that the research team did a diligent job analyzing the results.
It looks like a fairly low-quality poll to me too.
500 respondents is sufficient for many polls. But, there's no word on how the poll was conducted (phone, personal interviews, email - what?), how many didn't respond and how (or if) they controlled for that shortfall, how they controlled for variability in the sample population compared to the true population and so on. I'm not saying they didn't do this, but there's not a single word about it in an otherwise decently lengthy article, which is always a danger sign.
Also, they don't give the actual questions asked and the given alternatives or answer format (the details can greatly affect the responses). There's no mention of expected error rates, and they don't even mention how the "anxiety rate" is calculated.
And never mind that the questions inevitably are language- and culture dependent (does "I'm a bit worried" for an american really imply the same level or kind of anxiety as "ちょっと心配" for a japanese?), so the only meaningful comparison you can do is across time for the same country in any case. Which means the the only results for japan with any meaning would be the ranking of what worries people most.
And then the conclusions drawn from the poll are at best not supported by the shown data, and at worst made up from whole cloth.
Where in the data does: "JWT concluded that the anxiety felt by Japanese was deeply rooted in long-term concerns, not just the current economic crisis. " come from?
And: "Among the problems weighing heavily on Japanese people's minds are the demographic changes due to the falling birthrate and aging population, the economic downturn, prolonged unemployment and Japan's diminishing influence in Asia." - what questions did they ask about those issues and what were the results? They don't say.
Since those kind of things are country-specific it can't have been part of the standard questionnaire. If they aren't completely making it up they probably asked free-form questions as well - but then, 500 people is suddenly not a lot. You would want to know how many actually stated those reasons to the interviewer, what other reasons people stated and if they were prodded or reminded about those issues during the previous interview. But they don't say.
Another crap poll press release designed to drum up paying customers for the advertising agency.
500 is not an adequate sample size for 120 million and believe me, many many polls in this country (especially those done by certain major newspapers) are done not to get raw objective data, but to get data to support a certain political position - I've seen some of the questions asked and they are not neutral questions. I see these polls every week where I work and I wouldn't trust them any farther than I could throw Konishki, their methodology is very suspect.
Anon, may Our Man refer you to the learned commenter Janne, though he is 90 percent wrong about pizza, Our Man wouldn't like to sample his portions.
Rev.: I heard that. BTW, think Our Man will have to lift one of your funny re-cut films, they are tres good, though haven't tried them out on any focus groups.
The margin of error in this comment is + or - 245.
Good reverend, the needed sample size is not really dependent on the total population size (if it were, you couldn't sample a stationary analog signal). It is dependent on some statistical properties of the underlying population, though, but if anything, a larger population size lowers the minimum needed sample size somewhat, rather than increasing it (the variability of a larger population is lower).
Sample size does depend on what kind of questions you ask though; simple binary questions ("Is Ourmani terribly wrong about kebab-pizza?") need a smaller sample than multifaceted or open-ended questions. Likewise, questions with a fairly even distribution of answers ("are you male of female?") need smaller sample size than questions with lopsided distribution ("Sour herring is yummy - yes or no?") to achieve the same degree of certainty.
So by itself, 500 respondents are not necessarily insufficient, whether you're sampling the population of Iceland or Japan. I do agree it sounds insufficient for the kind of open-ended questions they were asking - and not saying anything about the confidence intervals or the methods used actually tells us all we need to know about how much to trust this piece of fluff.
Post a Comment