Introduction

Community-based management of acute malnutrition (CMAM) was initially piloted in four districts of Sierra Leone in 2007. Since then, the program had been expanded to provide CMAM services in various health centres in other districts. By 2010, all districts in the country had been implementing CMAM. Given the national scale of the CMAM program in Sierra Leone, a wide area survey method was needed to assess the coverage achieved by the program. UNICEF and the Sierra Leone Ministry of Health and Sanitation (MoHS) with technical support from Valid International conducted SLEAC or Simplified Lot Quality Assurance Sampling Evaluation of Access and Coverage and SQUEAC or Semi-Quantitative Evaluation of Access and Coverage from the 20th of February to the 25th of April 2011. This report describes the process and presents the findings of the survey.

 

Methods
SLEAC Sampling Design

SLEAC was used as the wide-area survey method to classify coverage at the district level. The district was selected as the unit of classification because service delivery was managed and implemented at the district level.

The primary sampling units (PSUs) used in the SLEAC surveys were census enumeration areas (EAs). In rural districts, EAs were individual villages and hamlets. In urban and peri-urban districts, EAs were city-blocks. In rural districts, lists of potential PSUs were sorted by chiefdom. In urban and peri-urban districts, lists of potential PSUs were sorted by electoral ward (sections). The structure of the district-level samples are shown in Figure 1.

A target sample size of $ \ n = 40 \ $ current SAM cases was used in the rural and urban districts. This is the standard SLEAC sample size for large populations. A lower target sample size was used (i.e. $ \ n = 33 \ $) in the single peri-urban district because this district had a much lower population than the other districts.

The number of PSUs to sample ($ n_{PSU} $) needed to reach the target sample size in each district was calculated using estimates of average EA population and SAM prevalence using the following formula:

 
$$ n_{PSU} = \frac {\text {target sample size (n)}} {\text {average population}_{\text {all ages}} \times \frac {\text {percentage of population}_{\text {6-59 months}}}{100} \times \frac {\text {SAM prevalence}}{100}} $$

 
Average EA population was estimated as:

 
$$ \text {Average EA population} = \frac {\text {District population}}{\text {Total number of EAs}} $$

 
using data from the Sierra Leone 2004 Population and Housing Census.

The percentage of the population aged between 6 and 59 months was estimated as 17.7%. This is a national average taken from the Sierra Leone 2004 Population and Housing Census. This estimate is used by Sierra Leone government departments, UNOs, and NGOs.

SAM prevalences were taken from reports of SMART surveys of prevalence in each district that had been undertaken in the lean period of the previous year. The prevalence of SAM using MUAC and oedema was used since this matched program admission criteria.

The Sierra Leone Central Statistics Bureau provided information on the total district populations and total number of EAs in each district. The Sierra Leone Central Statistics Bureau also provided lists of enumeration areas for the Western Area districts and large-scale maps of the EAs that were selected for sampling.

 

Figure 1: Structure of samples in urban and rural districts
sampleStructureSLEAC

 
PSUs were selected using the following systematic sampling procedure:

Step 1: The lists of EAs were sorted by chiefdom for rural districts and by section for urban and peri-urban districts.

Step 2: A sampling interval was calculated using the following formula:

 
$$ \text {Sampling interval} = \frac {\text {Number of EAs in district}}{n_{PSU}} $$

 
Step 3: A random starting PSU from the top of the list was selected using a random number within the sampling interval. The random number was generated by coin-tossing.

The PSUs selected by this procedure were sampled using a case-finding method tailored to the particular district:

  • In rural districts, a district-specific case-finding question was developed from the base case- finding question:

    Where can we find children who are sick, thin, have swollen legs or feet, or have recently been sick and have not recovered fully, or are attending a feeding program?

    This question was adapted and improved using information collected from traditional birth attendants, female elders, traditional health practitioners, cares of children in the program, and other key informants to include local terms (in all local languages) and local aetiological beliefs regarding wasting and oedema. This question was used as part of an active and adaptive case finding method.

  • In urban and peri-urban districts, house-to-house and door-to-door case-finding was used. This was done because it was felt that active and adaptive case-finding would not work well in these districts. Sampling was aided by the use of large-scale maps provided by the Sierra Leone Central Statistics Bureau (see Figure 2).

 

Figure 2: Large-scale maps for urban and per-urban districts
urbanMapsSLEAC

 
After all PSUs in a district have been sampled, the survey team met at the district headquarters for data collation and analysis. The simplified Lot Quality Assurance Sampling (LQAS) classification technique was applied to the collated data. Coverage standards:

  • Low coverage: Below 20%.
  • Moderate coverage: Between 20% and 50%.
  • High coverage: Above 50%

were decided centrally by MoHS and UNICEF staff before the start of the surveys. These standards were used to create decision rules using the rule-of-thumb formulae:

 
$$ \begin{align*}
d_{1} &= \left \lfloor \ {n \times p_{1}} \ \right \rfloor = \left \lfloor \ {n \times {\frac {20}{100}}} \ \right \rfloor = \left \lfloor \ \frac{n}{5} \ \right \rfloor \\[0.5cm]
&\text {and} \\[0.5cm]
d_{2} &= \left \lfloor \ {n \times p_{2}} \ \right \rfloor = \left \lfloor \ {n \times {\frac {50}{100}}} \ \right \rfloor = \left \lfloor \ \frac{n}{2} \ \right \rfloor \\[0.5cm]
&\text {where} \\[0.5cm]
n &= \text {sample size achieved by the survey} \\
p_{1} &= \text {lower coverage threshold} \\
p_{2} &= \text {upper coverage threshold}
\end{align*} $$

 
In order to classify coverage in each district, two coverage estimators were used. Point coverage provides a snapshot of program performance and places a strong emphasis on the coverage and timeliness of case-finding and recruitment. Point coverage is defined as:

 
$$ \text {Point coverage} = \frac {\text {Number of current cases attending program} \ (c)}{\text {Number of current cases} \ (n)} $$

 
Period coverage, on the other hand, includes recovering cases into the estimation of coverage. These are cases who should be in the program because they haven’t met discharge criteria yet. Period coverage is expressed as:

 
$$ \text {Period coverage} = \frac {\text {Number of current and recovering cases attending program} \ (c)}{\text {Number of current and recovering cases} \ (n)} $$

 
Using both point and period coverage estimators, coverage in each district was classified by comparing the decision rules against the value of $ \ c \ $ based on the algorithm presented in Figure 3.

 

Figure 3: Coverage classification algorithm
sleacAlgorithm

 
A short questionnaire, similar to that shown in Annex 2, asking about barriers to coverage was administered to carers of non-coverage cases found. This data was tabulated from the questionnaires using a tally-sheet and presented as a Pareto chart.

 

SLEAC implementation process

The process as described above was completed in eight weeks staffed by fifteen mid-level health management staff and a principal surveyor provided by Valid International. Three survey teams with five members each were used. The teams were divided into two sub-teams. A survey team was headed by a “captain” who was in charge of managing the sub-teams, organising travel and survey logistics, and co-ordinating survey activities with the principal surveyor.

Each district was divided into three segments. Segmentation was informed by logistics with each segment being served by a road (when possible).

Each survey team was assigned to one of the three segment and provided with:

  • A list of PSUs (sorted my chiefdom) to sample.
  • A list or the locations of CMAM program sites.
  • A list of the names and home villages of chiefs and chief’s assistants for each PSU

Each survey team started case-finding in the farthest PSU and then moved to the next-farthest PSU for case-finding and so-on. At the end of each day, the survey teams lodged in health centres, local guesthouses, or in villagers’ homes. They restarted case-finding on the following day. This continued until all the PSUs had been sampled. The surveys teams came together at the district headquarters for data collation and analysis and results shared with district-level health management staff.

Upon completion, the survey team was able to:

  • Classify coverage in each district
  • Map coverage by district for the whole country
  • List barriers to coverage ranked by their relative importance

 

Using SQUEAC to investigate in detail barriers to service uptake and access

SQUEAC was used for more focused and in-depth investigation of factors to coverage in one of the districts that was classified as having low coverage. It would have been ideal to do a SQUEAC assessment for a district with low coverage and a district with moderate coverage to allow for comparison (as shown in Figure 4). However, due to time constraints, it was deemed more important to assess factors to low coverage in one district.

 

Figure 4: Algorithm for conducted SQUEAC after a SLEAC survey
SLEAC and SQUEAC 2

 
The SQUEAC assessment was started by using the information on barriers to coverage gained from the SLEAC survey in the Western Area rural district as leads for further investigation. Also, survey team members were asked to share their observations and findings from visiting the various villages in the district and put forward different hypotheses of how these barriers or factors impact on coverage. The survey team then adapted and developed investigation tools and strategies to gain more insight on the barriers identified and to test the hypotheses generated. These strategies include question guides to ask specific key informants and collection and analysis of specific data from patient records.

The survey team applied this investigation plan and adapted it based on new leads or information gained during the process. A mind map with factors/barriers initially identified served as the sub-topics to which supporting data and information gathered during the investigation were organised around. The mind map facilitated the triangulation of information and also helped in drawing links between the various factors/barriers identified and investigated resulting in the creation of a concept map.

The SQUEAC investigation was done in four days. Due to time constraints, the Western Area rural district was chosen for the investigation because it was small and the various villages and health centres can be visited easily. Only 5 out of the 11 health centres (and their corresponding catchment areas) implementing CMAM were assessed during this process. However, the survey team believed that the findings from these health centres were most probably reflective of what was happening in the others not assessed.

 

Results and Findings
SLEAC: Classification and mapping of coverage by district

For point coverage, only 3 out of the 14 districts surveyed achieved moderate coverage classification. These were the districts of Kenema, Pujehun and Bo. The remaining 11 districts only reached a low coverage classification. None of the districts got a high coverage classification. The point coverage classification was homogenuously low across all the districts. The national point coverage classification was low.

For period coverage, half of the districts reached moderate coverage (Bombali, Kono, Kenema, Bonthe, Pujehun, Bo, Western Area Rural) while the other half reached low coverage classification. As was with point coverage, none of the districts got a high coverage classification. The period coverage classification was patchy or heterogeneously moderate across all the districts. The national period coverage classification is moderate.

Table 1 and Table 2 summarises the classification of point and period coverage (respectively) by district. Figure 5 and Figure 6 present the same results as a map of per-district coverage (point and period respectively).

 

Table 1: Point coverage classification by district

District SAM cases in program (c) SAM cases found (n) Decision rule 1 Is $ c > d_{1} $ ? Decision rule 2 Is $ c > d_{2} $ ? Classification
Bombali 4 30 6 No 15 No Low
Koindadugu 0 32 6 No 16 No Low
Kambia 0 28 5 No 14 No Low
Port Loko 2 30 6 No 15 No Low
Tonkolili 1 28 5 No 14 No Low
Kono 2 16 3 No 8 No Low
Kailahun 4 34 6 No 17 No Low
Kenema 8 34 6 Yes 17 No Moderate
Bonthe 7 41 8 No 20 No Low
Pujehun 6 27 5 Yes 11 No Moderate
Bo 6 22 4 Yes 11 No Moderate
Moyamba 6 40 8 No 20 No Low
Western Rural 6 46 9 No 23 No Low
Western Urban 2 20 4 No 10 No Low
Total 54 428 85 No 214 No Low

 

Figure 5: Map of point coverage classification
Case-finding Effectiveness of CMAM programmes in Sierra Leone

 

Table 2: Period coverage classification by district

District SAM cases in program (c) SAM cases found (n) Decision rule 1 Is $ c > d_{1} $ ? Decision rule 2 Is $ c > d_{2} $ ? Classification
Bombali 10 36 7 Yes 18 No Moderate
Koindadugu 6 38 7 Yes 19 No Low
Kambia 0 28 5 No 14 No Low
Port Loko 2 30 6 No 15 No Low
Tonkolili 6 33 6 No 16 No Low
Kono 5 19 3 Yes 9 No Moderate
Kailahun 7 37 7 No 18 No Low
Kenema 12 38 7 Yes 19 No Moderate
Bonthe 9 43 8 Yes 21 No Moderate
Pujehun 7 28 5 Yes 14 No Moderate
Bo 6 22 4 Yes 11 No Moderate
Moyamba 6 40 8 No 20 No Low
Western Rural 11 51 10 Yes 25 No Moderate
Western Urban 2 20 4 No 10 No Low
Total 97 471 94 Yes 235 No Moderate

 

Figure 6: Map of period coverage classification
Treatment coverage of CMAM programmes in Sierra Leone

 

SLEAC: Barriers to service uptake and access

Tabulated data from questionnaires administered to cases not covered by the program revealed that the most critical barrier to coverage is the lack of knowledge of the program. This accounts for about 68% of the reasons why these cases are not covered by the program.

Table 3 lists the various barriers to coverage ranked by their relative importance and Figure 7 presents the same data through a Pareto chart.

 

Table 3: Barriers to coverage

Theme Specific reason Number of responses
Lack of knowledge about the program (281)
Carer not aware of the program 265
Carer waiting for health staff to admit child to program 11
Carer doesn’t think program can help child 1
Carer doesn’t know the CMAM program site 1
Carer doesn’t know health centre provided RUTF 1
Carer thought they had to pay for service 1
Wrong information on program 1
Lack of knowledge about SAM (49)
Child not recognised as malnourished 47
Carer expected child to recover with medicines 1
Carer expected child to recover over time 1
No RUTF 19
Distance (12)
Too far 10
RUTF too little to justify journey 2
Time and opportunity costs (12)
No time / too busy 8
No one else to take care of other children 4
Discharged then relapsed 12
Other reasons (9)
Mother is sick 2
Husband refused 1
Husband doesn’t help in taking care of child 1
Carer shy / ashamed about coming to the health centre 1
Child not improving 1
Carer cannot carry more than one child 1
Child gets allergies with RUTF 1
Wrong discharge 1
Rejection (8)
Rejection 7
Carer thought child will not be admitted 1
Defaulted 7

 

Figure 7: Pareto chart of barriers to coverage
sleacBarriersPareto