It’s been a year since we started the Development Impact blog, and I thought I would use the one year anniversary to focus on one of the classic papers in impact evaluation. This paper (gated version here, ungated version here) is by Gordon Smith and Jill Pell and appeared in the BMJ back in 2003.
The paper is a systematic review of RCTs to “To determine whether parachutes are effective in preventing major trauma related to gravitational challenge.” The motivation is clear. After all, as Smith and Pell point out, despite the fact that parachutes are widely used, they have been associated with adverse effects due to failure and iatrogenic injury (for those unfamiliar with iatrogenic injury, this is a concept which mostly applies to macroeconomists as is clear from the Wikipedia definition here). Moreover, the utility of parachutes can also be questioned since studies of free fall do not show 100% mortality. Indeed, survival has been documented in free falls from heights as high as 33.000 feet.
In addition, as Smith and Pell point out, observational studies of parachute use are likely to be rife with selection bias issues. They put it aptly: “The relevance to parachute use is that individuals jumping from aircraft without the help of a parachute are likely to have a high prevalence of pre-existing psychiatric morbidity. Individuals who use parachutes are likely to have less psychiatric morbidity and may also differ in key demographic factors, such as income and cigarette use.”
What’s more, the institutional settings for parachute adoption are plausibly biased as well. Smith and Pell cite physicians’ reliance on technology and the military-industrial complex as two examples which make it less likely that a rigorous, independent study will be done.
So, what do they do to shed some light on this issue? They do a literature search in line with QUOROM (quality of reporting of met-analyses) guidelines using a wide range of relevant databases as well as broader internet searches looking for studies with a randomized control group and involved jumps from greater than 100 meters and an intervention that in some way approximated a parachute.
The result: not one study. So clearly this is an area for either faith-based policy or some rigorous trials. However, as Smith and Pell point out, it may be hard to recruit subjects for such a study. They do offer a good alternative: “…we feel assured that those who advocate evidence based medicine and criticize use of interventions that lack an evidence base will not hesitate to demonstrate their commitment by volunteering for a double blind, randomised, placebo controlled, crossover trial.”
So this study clearly indicates a need for further research. As a frequent traveler, it also helped me realize why airlines put life jackets under seats instead of parachutes – a quick internet search revealed at least one (semi) RCT involving life jackets versus none for parachutes. Fortunately the possibilities for an RCT on parachutes are looking up – Andreas Moser alerts us to the fact that Ryan Air at least allows one to take parachutes as carry on. So now we just need volunteers planning on boarding one of these planes to agree to carry on board what may or may not be a parachute, and a way to get around the “did anyone else give you something to carry on board?” question…
Hopefully, next year, if we are still writing, we can focus on this on our April 1st anniversary.
Join the Conversation