There is (and has been) a battle raging in New York State over the sale of wine in grocery stores. In his Table Hopping blog, Times Union writer Steve Barnes linked to an article that supposedly espouses some positive effects of selling wine in grocery stores. The study in question was conducted by the American Association of Wine Economists, so there could be a possibility of bias. Just maybe. In today’s post, we’re going to go through a little crash course (no pun intended) in research methodology to show why you should be wary of this study’s conclusions.
So first off, the goal of any research is to uncover how certain aspects of world work. What are the relationships between certain phenomena? In the case of this study, titled “Regulating the Availability of Beer, Wine, and Spirits in Grocery Stores: Beverage-Specific Effects on Prices, Consumption and Traffic Fatalities” it is looking at the relationship between the availability of alcoholic beverages and their prices, consumption and traffic fatalities. The abstract makes clear that the researchers (from Cornell nonetheless) believe their study demonstrates that higher shares of wine availability are correlated with lower traffic fatalities. As Steve Barnes’ post indicates, the implied effect of having wine in grocery stores means less traffic deaths. So imagine you’re a state considering whether or not to sell wine in grocery stores. You hear about this study, or maybe even read its abstract, and you see that the availability of wine in grocery stores may not increase social problems, but ACTUALLY DECREASE them? Selling wine in grocery stores is associated with lower traffic fatalities? Well sign us up!
It’s easy to read a lot of headlines about studies and be mislead. Most news outlets don’t actually conduct any due diligence on the studies they cite and report on. Usually they just read the abstract or conclusion section and post that as if it’s the gospel. But the key to any study is not its results, but how it got its results. And so here’s where we take a look at how the American Association of Wine Economists came to conclude that if you sell wine in grocery stores, you can decrease the amount of traffic fatalities.
Since I don’t want this post to be incredibly long, I’ll focus solely on the study’s traffic related conclusion. I think the best way to go about looking at this study’s methods is to brainstorm about how you might go about studying the relationship between wine availability in grocery stores and traffic fatalities. Here we break down our independent and dependent variables. Independent variables are the phenomena that have an effect on our dependent or outcome variables. We can manipulate the independent variables to see how they would change the outcome. Our variables depend on our research question.
Research Question: What are the effects of wine availability in grocery stores (an alcohol policy) on traffic fatalities (a social problem)?
Independent Variable: To find our independent variable, we need to find some way to measure wine availability in grocery stores. Now, these policies are state determined and since the policy in question for New York State is at the state level, we should use state-level data. So our state level variable would be whether a state has wine available in grocery stores or not. It’s a binary variable, yes or no. My beef with the AAWE study is not with its independent variables. They use state level data on wine availability.
Dependent Variable: Now here is where it gets tricky. We want to measure traffic fatalities by state, since we’re using state level independent variables. But we simply don’t want to use any traffic fatalities, but rather, ALCOHOL RELATED traffic fatalities. Why? Because it doesn’t make sense to think that the availability of wine in grocery stores would have any effect on non-alcohol traffic deaths right? People don’t drive to the grocery store, buy wine and then crash into other people and kill them because they bought a bottle of wine. They would crash into other people and kill them perhaps if they had been drinking the wine, therefore making it an alcohol-related death. So are dependent variable would be a measure of alcohol related traffic fatalities by state. This data is available through NHTSA. In fact, the AAWE study DOES get its data from the NHTSA but they don’t use alcohol-related fatality data in their study. That’s the first reason why their study isn’t convincing. Who cares about the effect of wine in grocery stores on all traffic deaths? It doesn’t make any sense to think that there would/should be an effect.
BUT the study says there is an effect? So doesn’t that mean something? If the sales of wine in grocery stores has an effect on all traffic fatalities, then isn’t that still a good thing? Well, it would be, but only if we could say that the wine availability was the reason traffic fatalities decreased. So what we’re talking about here, are control variables.
What are control variables? In the criminal justice field, we use the following example to explain them. Homicides go up as ice cream sales go up. Does that mean ice cream sales cause murders? Of course not. It’s simply that ice cream sales go up in the summer and there are more homicides in the summer. So the original relationship was said to be spurious, as in there was something else accounting for the relationship, a third variable. Or maybe more. The purpose of a control variable is to put these “third variables” into the model that might have an effect on our dependent variable and control for their effect. If we control for all other possible influences on the dependent variable, then we can say our independent variable did have an effect on the outcome.
The AAWE study does insert some control variables into the study, but not the variables necessary to convince me that they’re really isolated the effect of wine availability. For example, what could have an effect on traffic fatalities? Well, I can think of a few things. The number of cars on the road, the number of young drivers, the number of older drivers, weather conditions, road quality, road lighting, speed limits, you get the picture. Did the AAWE study control of any of this? The closest they come is a variable measuring how many miles a vehicle has traveled per driver. That gets us an imperfect (to say the least) measure of vehicle use but not much else. So in that respect, the AAWE fails to control for spuriousness as there are MANY other explanations for low traffic fatality rates.
So, in conclusion, what did we learn? The AAWE study does a poor job of measuring and operationalizing its variables. It doesn’t control for enough other factors so we can’t be sure that they are really isolating the effect of wine availability in grocery stores. Finally, even if they did isolate an effect, their conclusions are hard to believe because they didn’t measure the social problem of real concern, which is alcohol related traffic fatalities. One might argue that alcohol related fatalities are included in overall fatalities, which is true, but measuring the total fatalities includes too many other types of fatalities to make any meaningful contribution. It would be like analyzing the effect of gun laws on all types of crime. Sure, there would be measurements of gun crimes aggregated into the dependent variable, but it doesn’t make sense because it would include non-gun related crimes such as money laundering, drunk driving, child pornography possession, crimes that had nothing to do with guns. There’s too much other stuff clogging up the data.
My opinion on wine in grocery stores remains the same. Allow wine to be sold in grocery stores. However, the AAWE study did nothing to influence my opinion and frankly, anyone advocating for wine in grocery stores that uses this study in their argument, is doing a disservice to their cause. If you want to read the study, you can find it in PDF format here.
There’s a lot of pressure to have fun on New Year’s. It’s similar to the pressure one feels on their birthday. You know everyone is going to ask you, “What did you do for New Years?” I mean, everyone alive does SOMETHING on New Year’s. If you don’t drink and have no friends, you’re in bed. A few years ago, I was at the movies. But this year I started seeing a lot of advertisements on Steve Barnes’ Table Hopping Blog about New Year’s parties. Now, I know exactly what I don’t want on New Year’s because I’m pretty confident with who I am.
Wolff’s Biergarten was doing another Beefsteak event, which would be amazing if it wasn’t New Year’s eve. There is a place and time for pigging out on all you can eat filet, but ringing in the New Year didn’t seem like it. Having gone to inaugural beefsteak in the fall, I felt it was a little too soon and the $85 price tag was out of my range.
There were 6 of us total who bought tickets and because the place is relatively small (which isn’t a bad thing at all) we wanted to make sure we got a table together. A very accommodating bartender told my friend that we should get there at 8 to make sure we got a table. Show up at 8 instead of 9 and get in an extra hour of free drinking? Sure! So we did, grabbed a nice table on the first floor (so we wouldn’t have to trip down the stairs) grabbed a deck of cards and started to play BS while sipping from the pub’s awesome glasses.
One of the big reasons why I wanted to come here was that they don’t offer anything like Miller Lite or Bud Light on their beer menu. So that automatically removes most people who don’t know anything about beer. It also removes the crowd that wants to get drunk on 30 light beers on New Year’s. My night consisted of drinking Newcastle, Carlsberg, Fuller’s London Pride, Belhaven Twisted Thistle, and a few others I can’t remember. I definitely got my $50 worth of beer. While we were initially worried about the bar being super crowded, it turns out that the servers would simply come over to our table and ask to refill our beers. You can’t beat that. It was everything being an American should be, getting drunk and fat while not even leaving your seat.
Dinner was a choice of prime rib (which looked like a decent sized cut), cornish game hen, and stuffed mushrooms. I went for the game hen because well, when do I cook a game hen? It was amazingly (sorry Louie C.K.) moist, and to enjoy it I just broke off the wings and ate with my fingers. It came with asparagus and roasted potatoes. Great stuff. Dessert was self serve and consisted of brownies, fudge creations, red velvet cake, something in a flute that was impossible to eat because they didn’t have Friendly’s style long spoons and some really good tart-like sweet things (that was really descriptive eh?). The one thing I wish they had was a basket of bread because it would’ve helped add to my carb diet for the evening.
The servers were friendly and really attentive. The rest of the crowd was a lot of fun, everyone joined in to sing along with the classics on the radio and I didn’t see any fist fights or puking. The only awkward part was the actual countdown to midnight because well, there are no TVs inside the place and everyone’s phone was running slightly differently. But we got to drink our champagne and spend another hour and a half drinking before we headed off into the night. Walking home, we walked past Pearl Street, past a bunch of drunk girls wearing no clothes and a groups of guys that looked like they were really pissed off that they didn’t get laid (thanks Louie C.K.) and I was REALLY GLAD that I had gone to the Olde English for New Year’s. I’ll definitely be back next year.