Monday, July 24, 2017

Nudging Marshrutka Safety

[Note: Dustin Gilbreath is a Policy Analyst at CRRC-Georgia. This article was originally published on Eurasianet. ]

Auto safety is a perennial issue across Eurasia, as the generally poor condition of highways and byways, the proliferation of haphazardly maintained vehicles and a proclivity for reckless driving mean that death is a constant part of life on the road.

Among the most hazardous forms of public transportation are the ubiquitous communal minibuses known as marshrutkas, which ferry passengers around and between cities and towns. Marshrutkas are a mixture of taxi and public bus, and, for passengers, make up in convenience what they lack in comfort. But there is also a significant risk involved in using marshrutkas because many are old and in need of repair, and they are operated by overworked, stressed-out and distracted drivers.

But there is good news for marshrutka users: an experimental project conducted in Georgia suggests that a cost-efficient monitoring program could significantly increase rider safety.

The monitoring format, developed by the Caucasus Research Resource Centers-Georgia, underwent a month-long test, starting last September 20. CRRC recruited paid observers to ride on marshrutkas and monitor the driving behavior of operators. Monitors tracked marshrutka trips in three phases. In the first, they recorded whether a group of drivers engaged in dangerous driving behaviors, including passing in places where it was illegal, and distracted driving behaviors like smoking and talking on a cell phone. This contingent formed the trial’s control group.

For the second phase, monitors followed a different group of drivers, who were told in advance that they were being observed and that if they were judged the safest driver in the survey, they would receive a fuel voucher. The drivers in the second group were also told that an anonymous monitor would at some point in the near future come back to observe their road behaviors.

The last step involved monitors returning to both the control and treatment groups unannounced to observe driving.

By comparing the results of the first phase to the second, it was possible to determine the effect of overt observation on driving patterns. Evaluating participants in the second phase to the same minibuses in the third provided insight into whether those who knew they were monitored, and were told they would be monitored again, maintained safer driving practices. Lastly, by comparing the first-phase drivers to their third-phase performances, it was possible to test for the consistency of driving behaviors by the control group.

The results of the trial indicated that a small, anonymous monitoring program could be effective in improving driver practices. While in the first round of observation, 96 percent of drivers engaged in some form of dangerous driving, among those in the second group, who were told in advance that they were being monitored, the number was 70 percent. And while 79 percent of drivers made illegal passes in first phase, 43 percent from the second group engaged in such behavior.

In the third phase, carried out several weeks after the first two phases, drivers from the second group still engaged in 14 percent fewer dangerous driving behaviors than those from the first group who had not been told in advance that they were being observed.

CRRC-Georgia’s project involved the monitoring of 360 inter-city minibus trips in a randomized control trial (RCT). RCTs are considered the gold standard in social science because they provide firm evidence of cause and effect through randomly giving a treatment to some individuals, and not others and then comparing outcomes.

While the marshrutka safety experiment raises hopes that a monitoring program could encourage changed behaviors, the evidence is not definitive. The experiment did not take weather into account, a factor that can potentially alter driving patterns. It also could not measure precisely for the possibility of contamination of the source pool, i.e. drivers who had been observed later talking about the project with colleagues, and encouraging other drivers to be more careful behind the wheel.

While the experiment was carried out in Georgia, there is no particular reason to suspect that such a policy would not work in other areas and contexts.

Monitoring projects could be conducted either by non-governmental organizations or by municipal government agencies. A government-run project would likely stand a better chance of improving safety, as drivers could be fined for hazardous driving, as well as rewarded for safe driving. This policy would likely have a greater impact since social scientists have repeatedly shown that individuals strive to avoid losses much more intensively than they seek out gains. 

[Note: Dustin Gilbreath is a Policy Analyst at CRRC-Georgia. This article was originally published on Eurasianet. The full report this article is based on is available here: The data and replication code for the analysis in this article is available here: The research presented in this article was funded through the East-West Management Institute’s (EWMI) Advancing CSO Capacities and Engaging Society for Sustainability (ACCESS) project, funded by the United States Agency for International Development (USAID). The content of this article is the sole responsibility of the author and does not necessarily reflects the views of USAID, the United States Government, or EWMI.]

Monday, July 17, 2017

Some in Georgia fear visa liberalization will lead to more refugees

Visa liberalization with the EU Schengen zone countries has been a much celebrated milestone for Georgia. But with new opportunities for Georgia to move closer to Europe come new opportunities for anti-European sentiment. CRRC data show that some people in Georgia fear that visa liberalization could increase the number of refugees coming to Georgia. To complicate the issues further, Austrian Minister of Foreign Affairs Sebastian Kurz made a comment suggesting that there was a need to build refugee camps outside the EU, in countries like Georgia. Such statements could play on fears of an increase in refugees and foreigners more generally in Georgia.

In looking at data from the most recent CRRC/NDI survey, conducted in April 2017, approximately half of the population of Georgia believe that as a result of visa liberalization more refugees will come to Georgia, with a sizable share of the population (20%) responding they don’t know whether this will or will not be the case. While it is true that the visa liberalization process required the Government of Georgia to harmonize their laws in accordance with EU legislation, including making laws on the process of accepting refugees and asylum seekers clearer, the truth is that these laws will not necessarily increase the number of refugees and asylum seekers in Georgia. CRRC/NDI data, however, suggest that at least part of the population is misinformed about the issue.


In contrast, when asked about potential threats of visa liberalization more broadly in an earlier, CRRC/CIPDD survey conducted in January/February 2017, far fewer people mentioned refugees, or foreigners more generally, entering Georgia. According to the findings of CRRC/CIPDD survey, 27% said visa liberalization would have no negative consequences and 15% said they didn’t know. Slightly over half (58%) named some negative consequences of the visa liberalization. Rather small shares, though, named the threat of more foreigners (5%), refugees (4%) and terrorists (9%) coming to Georgia.

Note: This was an open-ended question for which each respondent could provide up to two answers.

While the answers to these two survey questions cannot be compared, they give us some understanding of the fears of the population of Georgia about visa liberalization regarding the possible influx of refugees. Although a small share of the population was worried about refugees entering the country even before visa liberalization came into force and saw that possibility as a potential threat, a much bigger threat was associated with Georgian citizens leaving the country.

To explore the CRRC/NDI data presented in this blog post, visit our online data analysis tool. Keep an eye out for the CIPDD dataset in the near future as well.

Monday, July 10, 2017

Visa liberalization: Expectations in Georgia

In March, 2017, after nearly five years of negotiations, a visa liberalization agreement with the Schengen zone countries came into force for Georgian citizens. Even though political elites generally perceive this achievement as a step forward for Georgia, the public’s attitudes and expectations about visa liberalization are not solely positive. Using CRRC/NDI April 2017 survey data, this blog post presents some assessments of the EU-Georgia visa liberalization.

Nine in ten people in Georgia report having heard about visa liberalization with the Schengen zone countries for Georgian citizens, however, not everyone feels they have enough information about the rules of visa free travel. Importantly, roughly 4 in 10 people disagree with the view that visa free travel will benefit them or people like them.


Note: For these two questions, the sample was split equally: half of the respondents were asked the question “Do you agree or disagree that visa free travel will benefit people like you?”, while the other half was asked the question “Do you agree or disagree that visa free travel will benefit you?” 

A number of specific statements about visa liberalization were also assessed during the survey. Overall, attitudes are rather mixed. There is a widespread belief that visa liberalization will not have any negative consequences for the Georgian economy. Approximately 2/3 of the population think it will increase emigration from Georgia. Probably most importantly, 78% think that ordinary people will not be able to afford traveling in the EU, even though visas are not required.


To conclude, expectations of the visa liberalization are not uniformly positive in Georgia. To explore the CRRC/NDI survey findings, visit CRRC’s Online Data Analysis portal.

Monday, July 03, 2017

Municipal Transparency Ratings

In 2014, CRRC-Georgia requested information from Georgian municipalities through the Ministry of Regional Development and Infrastructure (MRDI). Only 37 of 63 municipalities responded to our request. 31 of these provided some information. Only 17 provided complete information in response to our request. Based on this experience, CRRC-Georgia decided to rate the municipalities’ responses. While a score of 0 means that the municipality didn't respond at all, municipalities received 1 point for at least responding to the letter. Municipalities which received a score of 2 provided us with some of the information requested, while municipalities that scored 3 provided all requested information.

Months later, we repeated our endeavor. However, this time, freedom of information requests were submitted directly to the municipalities. This time around, we asked for data in an Excel file. Almost all the municipalities responded, however, the quality of responses varied. 44 municipalities sent Excel files and 39 contained the requested information. An analogous rating system was used to rate the responses to this round of freedom of information requests.

Based on these two rounds of our unintentional rating of municipal transparency, we created an index of municipal FOI transparency, summing up the two scores. Zero corresponds to no response for both rounds of requests while six reflects response with full data provided.

Monday, June 26, 2017

CRRC’s Fifth Annual Methodological Conference: In Search of Methodological Innovation

CRRC’s fifth annual Methodological Conference took place on June 23 and 24, 2017 in Tbilisi. This year the conference’s focus was on policy analysis in the South Caucasus, and the search for methodological innovation. Over 50 participants representing institutions in the United States, United Kingdom, Georgia, Azerbaijan, Armenia, Russia, and Canada attended.

Alexis Diamond of the Keck Graduate Institute (KGI), San Francisco gave an opening address for the conference titled: Deliberate ignorance: The dangers of knowing too much too soon. The talk covered a wide range of issues in evaluation, however, emphasis was placed on the importance of honest evaluation.
The first day of the conference had four sessions, with papers on a wide variety of subjects from the geographies of polarization and inequality in Tbilisi to a field experiment on marshrutka safety and a machine learning approach to profiling tax awareness in Armenia. 
The opening slide of David Sichinava’s presentation on Spatial Patterns of Emerging Inequalities in Tbilisi, Georgia.
The second day of the conference was dedicated to workshops. Alexey Levinson of the Levada Center, Moscow lead a workshop on open-ended group discussions and Aaron Erlich of McGill University discussed the fundamentals of multiple imputation. The conference also included workshops on case studies in public health, web surveys, and synthetic controls.
Aaron Erlich discussing why and when to use multiple imputation.
For more information, the full conference program can be accessed here.

Monday, June 19, 2017

Back to the USSR? How poverty makes people nostalgic for the Soviet Union

A recent CRRC/NDI survey asked whether the dissolution of the Soviet Union was a good or bad thing for Georgia. People’s responses were split almost evenly: 48% reported that the dissolution was a good thing, whereas 42% said it was a bad thing for the country. Such a close split raised questions in the media about why people took one view or another.

While it is tempting to explain assessments of a past event, such as the dissolution of the Soviet Union, using people’s attitudes towards foreign policy issues, this blog post only looks at respondents’ socio-demographic and economic characteristics and some reported behaviors that could potentially shape their attitudes. Specifically, we look at the impact of gender, age, education, ability to speak English and Russian, frequency of internet use, settlement type and the number of durable goods a household possesses, out of the ten durables the survey asked about: a refrigerator, color TV, smartphone, tablet computer, car, air conditioner, automatic washing machine, personal computer, hot water, and central heating. We interpret the number of durables owned as a measure of the households’ economic status. Surely, this measure is not perfect and gives us only partial information about the household’s economic conditions. However, this is the best available measure from this particular survey, provided that many people do not like reporting their income or expenditures, or do not provide accurate information on these.

The chart below shows the results of a logistic regression model which predicts the odds of a respondent saying that the dissolution of the Soviet Union was a good thing for Georgia. The dots on the chart indicate point estimates for each independent variable, and the lines show 95% confidence intervals. If a line does not cross the vertical red line, we are 95% confident that the variable has an impact on the dependent variable, i.e. the belief that the dissolution of the Soviet Union was a good thing for Georgia. The further a horizontal line from the vertical red line, the larger the effect of the variable.

The model shows that gender and the ability to speak either English or Russian do not influence people’s assessments of the dissolution of the Soviet Union. As one might expect, age has a significant, negative impact: the older a person is, the lower is the probability that s/he will express a positive attitude towards the dissolution of the USSR. Education, frequency of internet use and possession of durables have the opposite impact: people with tertiary education are more likely to assess the dissolution positively than people with less than tertiary education. Likewise, people, who use the internet at least once a week assess the dissolution more positively than people who use the internet less often or never. Also, the more durables a household owns, the higher the probability of assessing the dissolution of the Soviet Union as a positive event for Georgia.

As expected, settlement type also matters: the chart shows the effect of living in Tbilisi, other large towns, predominantly Georgian-speaking rural settlements and ethnic minority settlements, which are compared to small towns – the reference category. Large towns include six cities with more than 40 thousand people, whereas smaller urban settlements are grouped into a small town category. We define an ethnic minority settlement as a location in which 40% or more of the inhabitants are ethnic minorities. Normally, these are towns and villages with large Armenian or Azerbaijani populations in the Kvemo Kartli, Samtskhe-Javakheti and Kakheti regions.

While residents of large towns and rural settlements have similar opinions about the dissolution of the Soviet Union as residents of small towns, Tbilisi residents are more likely to assess the dissolution positively. In contrast, those living in minority settlements tend to assess the dissolution negatively.
Based on the above, we conclude that age, education, frequency of internet use, possession of durables and settlement type influence an individual’s assessment of the dissolution of the Soviet Union. As a next step, we tested whether age, education, frequency of internet use, and possession of durables influence individuals’ attitudes differently in different settlement types.

The analysis shows that the impact of age, education and internet usage does not vary by settlement type. However, we observe a very different picture in the case of household possessions: possessing more durables increases the probability of positive assessment of the dissolution of the USSR in all settlement types except for (non-minority) villages and small towns. Its impact is largest, however, for residents of ethnic minority settlements. If an individual living in such a settlement has no durables, his or her probability of assessing the dissolution of the Soviet Union positively is below 20%. However, as the number of durables in the household increases, the probability of a positive assessment increases nearly linearly, and exceeds 60% when the household owns all ten items asked about on the survey.

Hence, we conclude that age, education, settlement type, and economic conditions significantly influence people’s assessments of the dissolution of the Soviet Union. The impact of a household’s economic situation is largest in ethnic minority settlements. Therefore, economic deprivation, arguably caused by and interrelated with a number of other factors, seems to be the most important driver of negative assessments of the dissolution, rather than minority status per se.  

To have a closer look at the CRRC/NDI data, visit CRRC’s Online Data Analysis tool.

Monday, June 12, 2017

Most households in Georgia report limiting food consumption, despite economic growth

According to the World Bank, GDP in Georgia increased from USD 10.1 billion to USD 13.9 billion between 2009 and 2015. Despite this growth, according to CRRC’s Caucasus Barometer survey (CB), the share of those who reported not having enough money to buy food on at least a weekly basis did not decrease between 2011 and 2015. This blog post shows how this finding differs by settlement type and reported household income.

As the chart below shows, the general picture did not change between 2011 and 2015. Only about one third of the population claims it never happened during the 12 month prior to the survey that they did not have enough money to buy food they or their family needed. More than one third report encountering such difficulties periodically and about a quarter monthly or more often.

Note: Answer options “Every day” and “Every week” have been combined for the charts in this blog post.

Taking into account the margin of error, the share of people who reported not having enough money to buy food every week or more often is approximately the same in different settlement types. Importantly, the most common response in Tbilisi and the second most common response in other urban settlements is “Never”. This answer is, however, reported by less than half of the population of these settlement types.

Note: Answer options “Don’t know” and “Refuse to answer” were excluded from the analysis.
A lack of money for food logically suggests a low income. The chart below shows that the higher the reported household income, the higher the share of the population reporting never being in a situation when they did not have enough money for food. According to CB 2015, 61% of the population reported their household income was less than USD 250 the month prior to the survey.

CB data show that a large share of households in Georgia have financial difficulties supporting their families’ primary needs and a majority struggle with not having enough money for food at least some of the time.

To have a closer look at the Caucasus Barometer data, visit CRRC’s Online Data Analysis tool.