Categories
Development economics methodology methods Research Vignettes World Bank

Vignettes as method… both an input and output tool

Vignettes can be an invaluable tool for generating, controlling for, interpreting and contextualising research data in a variety of fields. They can be both input and output methods of research. Finally, they can also frame, and be framed by, the research raising important questions about comparability, use, and ability to measure culturally relative norms. This post discusses the use of vignettes, when they might be appropriate, some relevant literature, and matters to be aware of.

Gourlay et al define vignettes as “short stories about a hypothetical person, traditionally used within research (qualitative or quantitative) on sensitive topics”. They can be used to gain insight into people’s beliefs, and can be a way of discussing sensitive or personal topics freely by projecting these on to a third person. Vignettes can be constructed through text or images, and Hughes and Huby set out a guide to constructing and interpreting vignettes as a methodology in the social sciences.

Vignettes can both generate or refine data. As the GIF above shows, they can be an input tool, generating and shaping the data gathered. Similarly, they can be used as an output tool, contextualising and illustrating research findings. Kandemir and Budd note that, given the debate ongoing about the precise definition and use of vignettes, simulations, real-life stories, anecdotes, or simply a narrative form of presenting research findings have all been referred to as vignettes.

In development economics, vignettes are not commonly deployed, but can respond to a specific methodological problem. For example, the measurement of intangibles such as happiness or subjective wellbeing or satisfaction can suffer from cultural relativity. Concepts, words, and their ultimate meaning can all vary by country or culture, making comparative studies in their area particularly tricky to undertake. However, in an era when the value of measurements like GDP have been questioned as inadequate, being able to measure satisfaction, happiness and similarly intangible attitudes is especially important for those working in development.

Reviewing the literature on what constitutes happiness between cultures, Uchida et al find that there are a variety of factors that determine an individual’s likelihood of declaring themselves happy, and that these requirements are culture-specific. What constitutes a good life in Japan includes emotional support from others, while personal achievement and self-esteem feature more prominently in the United States.

There are also cultural differences in the way we describe our happiness. Research by Minkov cited here notes that responses from Middle Eastern respondents tends towards extremes of happiness and unhappiness, while Asian and Western respondents tend towards moderation. Minkov and Bond have also examined the genetics of happiness as well as the impact of local climate, finding that certain genetic traits have a far greater impact on perceived happiness than factors such as recent economic growth or the rule of law.

King et al examined the incomparability of survey results when measuring subjective wellbeing and explored the use of vignettes as a way of measuring the incomparability. They find that “[b]ecause the actual (but not necessarily reported) levels of the vignettes are invariant over respondents, variability in vignette answers reveals incomparability. Our corrections require either simple recoded or a statistical model designed to save survey administration costs. With analysis, simulations, and cross-national surveys, we show how response incomparability can drastically mislead survey researchers and how our approach can alleviate this problem”.

One technique to measure and make sense of the variables in to anchor survey results to vignettes. This method is used by Angelini et al in their cross-country comparison of subjective wellbeing in 10 European countries. Standardised vignettes on life stories were included in each of the ten national surveys, and the vignette responses were included through econometrically rescaling self-responses according to how the respondent rated the standardised vignettes. This reduces disparities in life satisfaction across nations, and changes the ranking across counties from the original unadjusted cross-country data.

While the anchoring vignettes method has the potential to improve welfare comparisons and reporting, it relies on certain assumptions like vignette equivalence. This means that a particular vignette is capable of being interpreted and have the same cultural ramifications across the populations surveyed – for example, is a large family a blessing or a burden? There are ways of testing vignette equivalence, but this also usually requires some degree of interpretations and relativity.

Jed Friedman remarks that “[v]ignette anchoring can indeed improve the inter-comparability of different samples so where researchers have the opportunity to add meaningful vignettes to a planned survey then they should do so. But the assumption of vignette equivalence is not guaranteed, especially when comparing dramatically different populations in terms of culture and custom. If we have any doubt about vignette equivalence there may be no alternative to more focused mixed-methods research into the interpretation of [subjective wellbeing] concepts specific to the populations studied”.

While vignettes anchored to subjective wellbeing analyses can measure the incomparability of data, there are limitations to their use that also need clear explanation. The problems with language, interpretation, translation, and cultural norms remain, and there is the possibility that use of a vignette as a methodological tool to control for variability and comparability is still simply testing cultural frames. As noted, vignettes can both be used as a tool to output (explain, contextualise and understand) data, and as a tool to input (gather, frame, collect) data. Kandemir and Budd note that, to be of value, the vignette must be specific enough to guide the respondent, but not too specific as to nudge the respondent into an “expected” answer. In short, vignettes can both frame, and be framed by, the cultural norms in which they are constructed and deployed, and any researcher must be mindful of, and reflexive about, the use existing frames within vignettes when these are used both as research method inputs and outputs.

Categories
Development economics PhD Procreate Research Uncategorized Vignettes Visualising ESL

Making research relevant: Meet Ann, Pol, and Lil

One of the biggest challenges for anyone working on conceptual issues is making their research relevant to everyone else. After all, the first question any researcher should be asking themselves is “who cares”? This is not the passive-aggressive, somewhat depressing question that it could be, but rather a positive nudge to any researcher to bear in mind why you’re doing the research, and who is going to benefit from it.

My research is primarily conceptual, but has three main audiences: academics engaged in sociolegal research, policy makers working in international development, and a lay audience seeking more innovative responses to the financial crisis. Using designerly approaches to make the research tangible and visible, this post puts a face to each of these groups. Let’s meet each in turn.

Introducing Academic Ann…

Academic Ann

Ann works at a university as a law lecturer. She is researching the importance of the legal system for any country wanting to attract foreign investment. She is planning a trip to Sri Lanka – her target country for research – and is going to talk to government ministers, business people and investors, and local communities around investment zones. But how should she frame her research? How can she hold the interests and views of such a wide range of people in one frame at the same time?

Introducing Policy Polly

Polly has been working at an international development institution for several years now. Her job requires her to use existing research to make policy recommendations for foreign governments, international agencies and charities. She wanted to work in development to reduce poverty, but has become disappointed by the lack of tangible impact her work has, and has been wondering whether there is an alternative approach to understand the causes and ways of addressing poverty.

Introducing Lay Lillian

Lay Lillian

Lillian is not an academic. Nor does she work in development or policy. In fact, she is a retired dinner lady and pillar of her local community. She doesn’t know much about “the law” or “the economy”, but she cares about her community, and knows that something isn’t working properly. The economy crashed in 2008, and a decade of austerity was rolled out. She was told that there was “no more money” and that budgets across the country needed to be cut. But while her community saw centres close and support disappear, she noticed that the rich continued to get richer. So, she began reading about the crisis, and noticed that there were a lot of people arguing that we need to “do” economics and law differently. Lillian is an interested bystander, and wants to know more.

Why the characters? Personae can be a useful tool for exploring research concepts. We can see the relevance of concepts and frames for our own lives. They can make the conceptual visible and tangible.

Each of these will be developed in future posts, and will each bump up against the limitations of the ways we currently do, talk, and think about legal and economic phenomena. Each of them will then try an ESL lens using embeddedness and then moving beyond embeddedness. Through their eyes, we can explore the benefits and drawbacks of reframing in different contexts and for different audiences.

In the meantime, these characters are by me, using ProCreate on an iPad. They are rough first drafts, and the characters will be developed along with their stories. Copyright 2020.

Categories
Development Research World Bank

World Development Reports (and another blog post)

(Originally posted 5th October 2016 on my previous site):

These are annual Reports published by the World Bank into development triumphs and lessons. Each World Development Report (WDR) usually has a focus, and the 2017 WDR’s title is “Law and Governance”. Being just a little excited about this, I wrote a blog post that was published on the SLSA website, exploring the extent to which the World Bank was likely to open itself up to different approaches. You can read it┬áhere.

Overall, I’m not holding my breath for a radical realignment of World Bank policy, and even though the rewards could be immeasurable, I’m certainly not holding out for Economic Sociology of Law-based approaches to feature. But even just considering the role of law and governance in enabling (or hindering) development is a step forward. The World Bank is beginning to recognize that there is more to life than just economic equations.

There has also been a move towards experimentalism in development, which builds on behavioural economics and other slight expansions in policy over the past few years. While this is to be welcomed, there is a long way to go.