What was the primary objective of this study?
Our aim in conducting this study was to provide real-time, actionable intelligence to help policymakers make informed decisions during the crisis, both to make iterative improvements on entitlements and to imagine, where relevant, other means of meeting their needs. This strategic intention guided the design of our research methodology.
So far, we have presented iterative findings directly to policymakers on a weekly basis, including a range of central and state government stakeholders engaged in entitlement schemes and COVID-19 planning.
Who were the key stakeholders involved in conducting this study?
- Omidyar Network India and Rohini Nilekani Philanthropies provided funding support for this study
- Dalberg led the design of the core survey, data analysis and dissemination of results
- Kantar Public India (specialist unit of Kantar IMRB) executed the data collection and provided recruiting support for the qualitative interviews.
What did the study investigate?
This study aimed to understand three key issues:
- The extent of the financial impact of the crisis on low-income households
- The extent to which families have been able to access and use entitlements
- The extent to which entitlements have helped families tide over the crisis
What was the scale of the study?
This study was amongst the largest such efforts underway at the time in India. It covered 47,000 low-income households across 15 Indian states over two rounds (link to question 7). Round 1 of the survey covered ten states (Assam, Bihar, Kerala, Madhya Pradesh, Maharashtra, Odisha, Rajasthan, Telangana, Uttar Pradesh, and West Bengal). In Round 2, five more states (Gujarat, Haryana, Jharkhand, Karnataka, and Punjab) were added.
In addition, we conducted 100+ human centered design or in-depth qualitative interviews with respondents from low-income households and NGOs in these 15 states to supplement survey results with a nuanced understanding of people’s lived experiences on the ground.
How were the 15 states selected for the study?
Although we did not aim to be nationally representative while selecting states for our study, we sought to cover the vast majority of low-income populations across the country. Other factors that influenced state selection were regional diversity, as well as existing and potential networks with policymakers.
Which schemes did the study cover?
We studied six schemes under which entitlements in the form of top-ups or advance payments were announced during the Covid-19 crisis, including: PDS rations, PM KISAN, PM Ujjwala Yojana, MGNREGS, Jan Dhan Yojana transfers to women-owned accounts, and Social Pensions.
We focused on these national schemes because they have a broad reach, can be compared across the country, and because they represent both cash and in-kind transfers. While many individual states have developed their own state-level disbursal mechanisms, we did not capture these in this study in detail.
When was the fieldwork conducted?
We tracked select entitlements over two survey rounds in April and May. Round 1 of data collection covered 19,000 households across 10 states between 3-16 April 2020, and Round 2 covered 28,000 households across 15 states between 5 May 2020 to 3 June 2020.
What sampling methodology was used in the quantitative survey?
Our sampling methodology was designed to capture perspectives of low-income populations amidst the height of the initial Covid-19 crisis.
The sampling frame was created using a database developed prior to the COVID-19 crisis by Kantar Public, a division of the social research and consulting firm Kantar. This database includes 500,000 phone numbers collected over the past five years, and its demographic characteristics align broadly with the 2011 Census.
The sample was drawn randomly from this database, by filtering for two criteria:
- Respondents who self-reported as belonging to BPL families. We included all families who self-reported as BPL, because we wanted to also include families who fall below the poverty line but do not have BPL/AAY cards. We did this in order to capture the experiences of families who need support, but do not meet official eligibility criteria for various government schemes. After interviews were conducted, we validated self-reported BPL status by applying two filters: i) possession of a BPL or AAY ration card or registration on BPL/AAY list; or ii) self-reported monthly household income of INR 30,000 or lower. Families that fulfilled either of these criteria were retained in the sample. Therefore we refer to our sample as low-income and not BPL
- Respondents with phone numbers collected or updated within the past 3 years
Randomized sample selection was conducted within a framework of quotas designed to ensure proportionate coverage of rural and urban populations, men and women, and broad geographic distribution across districts within each state.
Finally, we applied population weights when conducting analyses on the total sample across all 15 states, to appropriately reflect the contribution of each state across the pool of 15 states surveyed. The weights are based on the number of households registered for PHH / AAY ration cards in the NFSA database, since our study focuses on low-income populations.
Learn more about our sampling methodology here.
What were the key takeaways from this study?
- Most families said entitlements were helpful during the crucial stretch of the crisis
- PDS performed well for the vast majority of households, especially for grains. Distribution of pulses also improving steadily over time, although half of all HH were yet to receive them as of June 3
- While cash largely flowed, we identified a clear opportunity to improve cash transfer coverage, close the gaps in delivery, and make withdrawal easier
- Still, the financial impacts of the crisis were immediate, deep and likely long lasting. Families will rapidly need greater and sustained support
Are there any biases or limitations in your survey methodology?
As with any survey, there are some inherent biases and limitations in our data which we feel are important to call out.
- Sample characteristics: We triangulated our sample demographics against other nationally representative various existing public data sources that are nationally representative. Our sample broadly represents the key characteristics of the populations our study focused on in the real world, such as rural-urban split, and social category distribution, with some variations in income distribution and occupational breakdown. These differences are to be expected, given the other limitations and our express focus on low income populations.
Phone survey biases: Telephonic surveys in general, not just our own, have inherent limitations:
- They exclude a number of populations, such as those who don’t own a phone, those not able to charge their phone (such as those on the move), those who lack the money to top up their phone credit, and those without network coverage
- The underlying datasets most phone numbers are drawn from are often not fully randomized
Inbuilt biases in interview-based surveys:
- 9% of respondents declined to take the survey once they were briefed about its purpose or taken through the consent script, posing a risk of self-selection bias due to non-response
- Interview-based surveys rely on self-reporting and are limited by what respondents are willing and able to answer
- Limitations in the framing, sequence, translation, communication, and interpretation of questions also exist
Limitations per our focus areas:
- By design, our study covers low-income households only. Many households above the poverty line have also experienced extreme economic shocks and may also need financial relief or other assistance - they are not captured by this survey
- Our study does not cover all types of government support. We focus on six national schemes only, because they have a very broad reach, can be compared across the country, and because both cash and in-kind transfers are represented
In our view, these limitations are acceptable in light of our objective to rapidly capture perspectives on the ground of low-income people across the country as the COVID-19 crisis unfolded. Further details on biases are available here.
What measures were taken to maintain ethical standards throughout the study?
Our work was broadly grounded in three key principles to ensure high ethical standards:
- Seek consent and protect anonymity
Before each interview, our enumerators read out a comprehensive consent script, and the survey was continued only if the respondent provided their informed consent. Respondents were free to terminate the survey at any stage, without needing to provide any justification. We also anonymized the data to protect individual respondents’ identities.
- Ask no more than necessary
Our respondents’ time is extremely valuable. We ensured that every question asked in the survey added high value to the research without causing any physical, mental or emotional harm to the respondent. Our goal was to derive maximum actionable insights by asking the most critical questions in a short timeframe. Therefore, our survey was restricted strictly to less than 20 minutes, and avoided themes that would compromise on our respondents’ well-being in an already challenging environment
- Provide appropriate support to distressed respondents
This is a period of high distress, especially for vulnerable segments of the population. We realized that our respondents may reach out to us during or after we have interviewed them. We adopted mechanisms to ensure that field teams were equipped to respond to distressed respondents in an extremely sensitive way. These included region-specific COVID-19 helplines, rigorous enumerator training, and follow-ups with distressed respondents where relevant.
- Seek consent and protect anonymity
What are your plans for these findings going forward?
It is our hope that our findings, which are amongst the most comprehensive datasets on the status of relief entitlements announced by the government, can help policy stakeholders make decisions to support the poor as they weather the ongoing crisis.
Looking ahead, we will continue to share our findings widely among policymakers, research organisations and civil society organisations, in service of two main objectives: supporting policymakers in decision making during the crisis and recovery, and making data available widely to researchers to further policy-making and practice on the topic. We also plan to share the underlying raw dataset by early August. We will be commencing three additional studies shortly to understand the gendered impact of the crisis, the impact on early childhood development, and the impact on children's education in India.
We are also exploring opportunities to further investigate broader social impacts of the crisis on livelihoods, income shifts, jobs recovery among other topics. We welcome your ideas and potential collaboration opportunities on these topics. See our contact details below. (link to contact section)
How can I cite your data?
Recommended citation: Swetha Totapally, Petra Sonderegger, Priti Rao, Gaurav Gupta, Efficacy of government entitlements for low-income families during Covid-19. Dalberg, 2020.
We’re excited at the prospect of our data supporting the efforts of policymakers and researchers, and would love to engage with your work.
How can I get in touch with your team?
We’d love to hear from you.
Write to us at firstname.lastname@example.org