Study Design

Site selection

The policy interventions are based on a quasi-experimental design in five counties in the Ningxia Hui Autonomous region (hereafter Ningxia). Ningxia is located in the northwest of China bordering Gansu, Shaanxi and Inner Mongolia. It has a population of 6.3 million, a territorial area of 51,954.40 sq.km, and is composed of 22 counties – about half of which lie in mountainous regions while the rest are in plains or valleys. With an annual per capita consumption expenditure was 16,674 and 5,908 Yuan amongst urban and rural residents respectively in 2012, it is one of the poorest provinces in China.

With the provincial government, we selected two mountainous counties with no recent pilot health sector reform projects as our treatment sites (T). We then matched three other counties to act as comparison sites (C). These counties are also located in mountainous areas and have similarly low levels of income and poor access to health care. They serve as comparison sites in the sense that they did not experience the specific interventions of our project but did experience the same province-wide policy changes as the two intervention counties. Our choice of counties means that the results are most generalizable to rural areas in China with similarly low levels of development.

Randomization

Embedded in our design was a matched-pair cluster randomized experiment using the township health centre and its catchment area as the unit of randomization. In our two treatment counties, half of the 28 towns and their primary health care facilities (township health centres and village health posts) were randomly selected to receive both supply and demand side interventions (the provider payment intervention introducing capitation plus pay-for-performance at village and township health centres and the modified NCMS benefit package), while the other half received only the demand side improved NCMS package. Towns were paired before randomisation in such a way as to ensure matches were as similar as possible on the basis of a Mahalanobis distance measure derived from data on a set of baseline characteristics. In each pair, one town was randomly assigned to receive the provider payment intervention.

Data collection

In both treatment and comparison counties, towns were stratified by income and a weighted sampling scheme was used to randomly select villages and households. In treatment counties, 40 percent of villages in each town were sampled, from which 33 households in each village were randomly selected. In comparison counties, 20 percent of villages were sampled, yielding 20 households per village. A household survey sample size of 7,000 was targeted to yield approximately 28,000 to 30,000 individuals. Townships health centres and village clinics for the facility surveys were selected within this sampling frame so that 66 township health centres and 267 village clinics were targeted. For the management information system, the sampling frame included the universe of all towns and villages in Haiyuan and Yanchi. Figure 1 provides a diagrammatic presentation of the survey design.

Fig 1. Description of Survey Design

Household survey

Three longitudinal household surveys were fielded in all five counties in 2009, 2011, and 2012, with a fourth one planned for 2015. They collected information on insurance status, individual health status, health care utilization, patient satisfaction of health care visit, perceived health care quality at each facility level, health expenditure, household consumption expenditure, ownership of durable goods, and various demographic characteristics.

Questionnaires available here:

Village health post survey

Two village health post surveys were fielded in 2009 and 2011, with a third one planned for 2015, providing information on characteristics of the village doctor including training, availability and quantity of different types of public health and medical services provided, and sources of revenue and expenses. In addition, separate village doctor knowledge surveys were also conducted in 2009 and 2011 assessing doctors’ ability to diagnose and treat conditions such as hypertension, stroke, ischemic cerebrovascular disease, tuberculosis, and childhood diarrhea. The survey consisted of both multiple choice questions and brief case descriptions of conditions that village doctors are expected to be able to treat.

Questionnaires available here:

Township health centre survey

A township health centre survey was fielded in 2011 and another one planned for 2015, providing information on the type and organization of the health care facility, human resources, medical departments established, equipment available, the quantity and quality of services provided, staff and facility incentives and revenue and expenses.

Questionnaires available here:

Management information system

An electronic management information system was set up at village and township health centres that collected data on every outpatient visit for all NCMS enrolees within Haiyuan and Yanchi including, patient characteristics such as age, gender, residence, and NCMS id, patient diagnoses, drugs prescribed, tests and examinations performed, out-of-pocket expenditure, and total health expenditure.