Clare's research blog
This mini animation, with sound, asks what the reintegration of the economic, legal and social might look like and sound like.
This is relevant for each and every interaction that occurs – there are always legal and economic aspects to every interaction, but we tend to forget that talking about the legal and economic in isolation from the social is a metaphor, or fiction.
How could frames or lenses like ESL help us to remember the social basis for any legal or economic aspects of interactions?
Vignettes can be an invaluable tool for generating, controlling for, interpreting and contextualising research data in a variety of fields. They can be both input and output methods of research. Finally, they can also frame, and be framed by, the research raising important questions about comparability, use, and ability to measure culturally relative norms. This post discusses the use of vignettes, when they might be appropriate, some relevant literature, and matters to be aware of.
Gourlay et al define vignettes as “short stories about a hypothetical person, traditionally used within research (qualitative or quantitative) on sensitive topics”. They can be used to gain insight into people’s beliefs, and can be a way of discussing sensitive or personal topics freely by projecting these on to a third person. Vignettes can be constructed through text or images, and Hughes and Huby set out a guide to constructing and interpreting vignettes as a methodology in the social sciences.
Vignettes can both generate or refine data. As the GIF above shows, they can be an input tool, generating and shaping the data gathered. Similarly, they can be used as an output tool, contextualising and illustrating research findings. Kandemir and Budd note that, given the debate ongoing about the precise definition and use of vignettes, simulations, real-life stories, anecdotes, or simply a narrative form of presenting research findings have all been referred to as vignettes.
In development economics, vignettes are not commonly deployed, but can respond to a specific methodological problem. For example, the measurement of intangibles such as happiness or subjective wellbeing or satisfaction can suffer from cultural relativity. Concepts, words, and their ultimate meaning can all vary by country or culture, making comparative studies in their area particularly tricky to undertake. However, in an era when the value of measurements like GDP have been questioned as inadequate, being able to measure satisfaction, happiness and similarly intangible attitudes is especially important for those working in development.
Reviewing the literature on what constitutes happiness between cultures, Uchida et al find that there are a variety of factors that determine an individual’s likelihood of declaring themselves happy, and that these requirements are culture-specific. What constitutes a good life in Japan includes emotional support from others, while personal achievement and self-esteem feature more prominently in the United States.
There are also cultural differences in the way we describe our happiness. Research by Minkov cited here notes that responses from Middle Eastern respondents tends towards extremes of happiness and unhappiness, while Asian and Western respondents tend towards moderation. Minkov and Bond have also examined the genetics of happiness as well as the impact of local climate, finding that certain genetic traits have a far greater impact on perceived happiness than factors such as recent economic growth or the rule of law.
King et al examined the incomparability of survey results when measuring subjective wellbeing and explored the use of vignettes as a way of measuring the incomparability. They find that “[b]ecause the actual (but not necessarily reported) levels of the vignettes are invariant over respondents, variability in vignette answers reveals incomparability. Our corrections require either simple recoded or a statistical model designed to save survey administration costs. With analysis, simulations, and cross-national surveys, we show how response incomparability can drastically mislead survey researchers and how our approach can alleviate this problem”.
One technique to measure and make sense of the variables in to anchor survey results to vignettes. This method is used by Angelini et al in their cross-country comparison of subjective wellbeing in 10 European countries. Standardised vignettes on life stories were included in each of the ten national surveys, and the vignette responses were included through econometrically rescaling self-responses according to how the respondent rated the standardised vignettes. This reduces disparities in life satisfaction across nations, and changes the ranking across counties from the original unadjusted cross-country data.
While the anchoring vignettes method has the potential to improve welfare comparisons and reporting, it relies on certain assumptions like vignette equivalence. This means that a particular vignette is capable of being interpreted and have the same cultural ramifications across the populations surveyed – for example, is a large family a blessing or a burden? There are ways of testing vignette equivalence, but this also usually requires some degree of interpretations and relativity.
Jed Friedman remarks that “[v]ignette anchoring can indeed improve the inter-comparability of different samples so where researchers have the opportunity to add meaningful vignettes to a planned survey then they should do so. But the assumption of vignette equivalence is not guaranteed, especially when comparing dramatically different populations in terms of culture and custom. If we have any doubt about vignette equivalence there may be no alternative to more focused mixed-methods research into the interpretation of [subjective wellbeing] concepts specific to the populations studied”.
While vignettes anchored to subjective wellbeing analyses can measure the incomparability of data, there are limitations to their use that also need clear explanation. The problems with language, interpretation, translation, and cultural norms remain, and there is the possibility that use of a vignette as a methodological tool to control for variability and comparability is still simply testing cultural frames. As noted, vignettes can both be used as a tool to output (explain, contextualise and understand) data, and as a tool to input (gather, frame, collect) data. Kandemir and Budd note that, to be of value, the vignette must be specific enough to guide the respondent, but not too specific as to nudge the respondent into an “expected” answer. In short, vignettes can both frame, and be framed by, the cultural norms in which they are constructed and deployed, and any researcher must be mindful of, and reflexive about, the use existing frames within vignettes when these are used both as research method inputs and outputs.
One of the biggest challenges for anyone working on conceptual issues is making their research relevant to everyone else. After all, the first question any researcher should be asking themselves is “who cares”? This is not the passive-aggressive, somewhat depressing question that it could be, but rather a positive nudge to any researcher to bear in mind why you’re doing the research, and who is going to benefit from it.
My research is primarily conceptual, but has three main audiences: academics engaged in sociolegal research, policy makers working in international development, and a lay audience seeking more innovative responses to the financial crisis. Using designerly approaches to make the research tangible and visible, this post puts a face to each of these groups. Let’s meet each in turn.
Ann works at a university as a law lecturer. She is researching the importance of the legal system for any country wanting to attract foreign investment. She is planning a trip to Sri Lanka – her target country for research – and is going to talk to government ministers, business people and investors, and local communities around investment zones. But how should she frame her research? How can she hold the interests and views of such a wide range of people in one frame at the same time?
Polly has been working at an international development institution for several years now. Her job requires her to use existing research to make policy recommendations for foreign governments, international agencies and charities. She wanted to work in development to reduce poverty, but has become disappointed by the lack of tangible impact her work has, and has been wondering whether there is an alternative approach to understand the causes and ways of addressing poverty.
Lillian is not an academic. Nor does she work in development or policy. In fact, she is a retired dinner lady and pillar of her local community. She doesn’t know much about “the law” or “the economy”, but she cares about her community, and knows that something isn’t working properly. The economy crashed in 2008, and a decade of austerity was rolled out. She was told that there was “no more money” and that budgets across the country needed to be cut. But while her community saw centres close and support disappear, she noticed that the rich continued to get richer. So, she began reading about the crisis, and noticed that there were a lot of people arguing that we need to “do” economics and law differently. Lillian is an interested bystander, and wants to know more.
Why the characters? Personae can be a useful tool for exploring research concepts. We can see the relevance of concepts and frames for our own lives. They can make the conceptual visible and tangible.
Each of these will be developed in future posts, and will each bump up against the limitations of the ways we currently do, talk, and think about legal and economic phenomena. Each of them will then try an ESL lens using embeddedness and then moving beyond embeddedness. Through their eyes, we can explore the benefits and drawbacks of reframing in different contexts and for different audiences.
In the meantime, these characters are by me, using ProCreate on an iPad. They are rough first drafts, and the characters will be developed along with their stories. Copyright 2020.
We do seem to be talking more about economics – what it should do and look like. But there is still a whiff of revolution about calls for the discipline to be more evidence-based and, well, scientific. This article, by Philip Aldrick in the Times yesterday, argues for more careful scientific approaches, and this is worth noting. Of course, in the natural sciences, this would be taken as read. Drugs need to be extensively trialled before they are sold and used to treat disease in humans. But for some reason, in the social sciences, theory and ideology have the ability to shape policy just as much as evidence.
Aldrick’s piece cites two studies launched by Nesta, a UK Innovation think tank, roughly seven years ago. The first was a retrospective review of the effectiveness of business clusters; do small businesses do better when they are closely located and can share location and labour advantages? The second was a randomised controlled trial on whether tax relief for small creative companies worked. The results of the studies were not their most important findings however.
For the sake of finishing a story, the first study proved relatively inconclusive, and could not find any clear correlation between clusters and growth. The second study found that tax and financial incentives were helpful in the short term for small creative businesses, but after 12 months any advantage had faded.
So, what was the main impact? The reason these two studies are remarkable are for their illustration of research methods. While retrospective reviews – generally the majority of most empirical work in the social sciences – can only look for correlation, randomised controlled trials (RCTs) can go deeper, further, and can identify causative factors. In other words, we can target specific factors and identify why things happen. This is important because it means we can be more scientific about what works, how it works, and why. And this means we can begin to base policy on evidence rather than theory. RCTs also offer a way of measuring the extent of policy impacts. By having a test group and a control group, we can gauge the extent to which a policy really makes a difference. And that means we can evaluate whether a policy is financially and economically viable. So, RCTs offer a way of seeing not only what works, but how much.
Why is this news? Similar to other recent posts on here, there is increasing discussion of economics and how the discipline can be improved in the mainstream media. Aldrick’s argument is that economics – both the research and the formulation of policy – can and should be more scientific in its approach. And to this end he calls for more RCTs and longer term studies testing causation before policy is enacted. The government has launched the Business Basics Fund with Nesta to carry out trials investigating, among other things, productivity. UK productivity lags behind that of other countries, attributed generally to poor management practices. But how can management practices be altered to improve productivity?
Questions like this lend themselves readily to RCTs where different techniques can be trialled in comparison with a control group. Nevertheless, there are questions of macroeconomics that are not suitable for trials. We cannot test interest rates or tariffs, for example, against control groups. And this remains a problem for the larger questions tackled in macroeconomics, where theory remains a significant influencer of policy.
Calls for greater use of careful empirical data in shaping economic, legal and social science policy is not new though. Economic sociology, economic sociology of law, and sociolegal approaches have long stressed the need for analysis and understanding to be based firmly in the real world, on real data, and about real people. Increasing access to big data and AI could enhance this. As Aldrick states, “Economics is a social science. Why not make it more scientific?”
Clare's research blog 