Comparison of fine-scale malaria strata derived from population survey data collected using RDTs, microscopy and qPCR in South-Eastern Tanzania

Abstract

Background

Malaria-endemic countries are increasingly adopting data-driven risk stratification, often at district or higher regional levels, to guide their intervention strategies. The data typically comes from population-level surveys collected by rapid diagnostic tests (RDTs), which unfortunately perform poorly in low transmission settings. Here, a high-resolution survey of Plasmodium falciparum prevalence rate (PfPR) was conducted in two Tanzanian districts using rapid diagnostic tests (RDTs), microscopy, and quantitative polymerase chain reaction (qPCR) assays, enabling the comparison of fine-scale strata derived from these different diagnostic methods.

Methods

A cross-sectional survey was conducted in 35 villages in Ulanga and Kilombero districts, south-eastern Tanzania between 2022 and 2023. A total of 7,628 individuals were screened using RDTs (SD-BIOLINE) and microscopy, with two thirds of the samples further analysed by qPCR. The data was used to categorize each district and village as having very low (PfPR < 1%), low (1%≤PfPR < 5%), moderate (5%≤PfPR < 30%), or high (PfPR ≥ 30%) parasite prevalence. A generalized linear mixed model was used to analyse infection risk factors. Other metrics, including positive predictive value (PPV), sensitivity, specificity, parasite densities, and Kappa statistics were computed for RDTs or microscopy and compared to qPCR as reference.

Results

Significant fine-scale variations in malaria risk were observed within and between the districts, with village prevalence ranging from 0% to > 50%. Prevalence varied by testing method: Kilombero was low risk by RDTs (PfPR = 3%) and microscopy (PfPR = 2%) but moderate by qPCR (PfPR = 9%); Ulanga was high risk by RDTs (PfPR = 39%) and qPCR (PfPR = 54%) but moderate by microscopy (PfPR = 26%). RDTs and microscopy classified majority of the 35 villages as very low to low risk (18–21 villages). In contrast, qPCR classified most villages as moderate to high risk (29 villages). Using qPCR as the reference, PPV for RDTs and microscopy ranged from as low as < 20% in very low transmission villages to > 80% in moderate and high transmission villages. Sensitivity was 62% for RDTs and 41% for microscopy; specificity was 93% and 96%, respectively. Kappa values were 0.7 for RDTs and 0.5 for microscopy. School-age children (5–15 years) had higher malaria prevalence and parasite densities than adults (P < 0.001). High-prevalence villages also had higher parasite densities (Spearman r = 0.77, P < 0.001 for qPCR; r = 0.55, P = 0.003 for microscopy).

Conclusion

This study highlights significant fine-scale variability in malaria burden within and between the study districts and emphasizes the variable performance of the testing methods when stratifying risk at local scales. While RDTs and microscopy were effective in high-transmission areas, they performed poorly in low-transmission settings; and classified most villages as very low or low risk. In contrast, qPCR classified most villages as moderate or high risk. The findings emphasize that, where precise mapping and effective targeting of malaria are required in localized settings, tests must be both operationally feasible and highly sensitive. Furthermore, when planning microstratification efforts to guide local control measures, it is crucial to carefully consider both the strengths and limitations of the available data and the testing methods employed.

Background

Precise mapping of malaria prevalence is crucial for the eventual elimination of the disease from different localities. In line with World Health Organization (WHO) guidelines, National Malaria Control Programmes (NMCPs) in Africa are increasingly adopting data-driven stratification of malaria burden, in most cases at either district or higher regional levels [1,2,3]. These stratifications involve assessing risk levels and burden in geographical areas at the subnational level (e.g. zones, regions, and districts) [2, 4, 5], and can include fine-scale mapping (down to wards and villages levels) as countries progress towards elimination [6,7,8]. The data for such stratification may come from health facilities, active malaria screening during population surveys, or proxy data sources such as antenatal care clinic visits [6, 9, 10].

When developing country-level malaria strategies, the prevalence of malaria, representing the proportion of confirmed positive cases of Plasmodium falciparum (or other Plasmodium sp.) among all individuals tested [11, 12], can be classified into various transmission categories. The WHO has previously used the following cutoff points for malaria endemicities: below 1% as very low, 1–10% as low, 10–35% as moderate, and above 35% as high burden malaria stratum [12]. Different NMCPs may adapt these criteria with slight adjustments based on local epidemiological insights. For instance, some countries, including Tanzania and Kenya, have used the parasite prevalence data to categorize their geographic zones as either very low risk (PfPR < 1%), low risk (1% ≤ PfPR < 5%), moderate risk (5% ≤ PfPR < 30%), or high risk strata (PfPR ≥ 30%) [2, 13]. Another measure that can be used for generating these strata is the annual parasite incidence (API), which is the number of diagnostically confirmed malaria cases per 1000 individuals per year and is usually obtained from health facilities data [12, 14]. API estimates are simpler to generate because they rely on facility-level data, but do not account for sub-clinical malaria infections, which can also contribute to transmission and impede malaria elimination effort [15].

National malaria programs usually rely on different actively and passively collected data to measure malaria burden and monitor the effectiveness of control measures [16,17,18]. For instance, Tanzania employs multiple platforms, including the District Health Information software (DHIS2) populated with data from routine health facility visits, the Malaria Indicator Surveys (MIS) and Tanzania Demographic and Health Surveys (TDHS), which are done every 4–5 years through household surveys, and the school malaria parasite surveillance (SMPS) targeting kids aged 5–16 years [2, 19,20,21]. A common feature of these established systems is that most rely primarily on rapid diagnostic tests (RDTs) and microscopy [13, 22, 23], though samples are sometime also preserved for PCR assays.

Microscopy, long used in malaria diagnosis, can quantify parasite loads and identify different Plasmodium species, which are essential for precise treatment choices [24, 25]. However, its effectiveness depends significantly on the skill and experience of the microscopist, making it unreliable in some contexts, and it can miss a substantial number of true infections due to sub-optimal accuracy [26,27,28]. In contrast, RDTs offer a consistent and user-friendly option, enabling quick, on-site diagnosis without specialized skills or equipment. RDTs have become widely used in both point-of-care settings and population surveys due to their operational simplicity and cost-effectiveness [29,30,31,32,33]. While the technique enhances access to diagnostics, especially in remote areas, RDTs have lower sensitivity for detecting low-level infections, such as those with < 100 parasites/µL of blood, and cannot quantify parasite density [34, 35]. Additionally, current RDTs may detect antigens for over three weeks post-treatment especially those targeting histidine-rich proteins 2 (HPR2), leading to poor specificity and potential overestimation of malaria cases in high transmission areas [30, 33, 34, 36].

In contrast, polymerase chain reaction (PCR) assays are known for their high sensitivity and specificity [37]. While conventional PCR assays typically provide qualitative information on malaria infections, quantitative PCR (qPCR) can offer additional quantitative measures of malaria parasite density [38, 39]. Unfortunately, the widespread use of PCR assays for population surveys is hindered by cost constraints and the need for specialized expertise and infrastructure for implementation [37, 40, 41].

The increased focus on evidence-based strategies in malaria control also includes a transition from broad subnational stratifications to more granular, fine-scale approaches [6, 10]. However, although current methods like RDTs and microscopy are favored for their operational simplicity, their effectiveness in detailed risk stratification, which are critical for targeting both clinical and sub-clinical infections for malaria elimination, remains poorly understood. Some authors have also suggested that RDTs may have vastly reduced performance in settings where the malaria burden has been significantly reduced [42]. This calls for a rigorous evaluation and comparison of these methods against highly sensitive techniques such as qPCR to refine malaria stratification approaches for malaria elimination. Indeed, available evidence, including data from Kenya and Tanzania, suggest that PCR assays are generally better at pin-pointing main malaria hotspots in communities than RDTs and microscopy [43, 44]. The study from Tanzania further showed that in subsequent treatment campaigns relying on RDT-based screening, ~ 45% of infections remain untreated, even if treatment is offered to all members of households with an infected individual [44]. In the Kenyan study, the authors went further to suggest that since detection of hotspots depends on the sensitivity of diagnostic tools, health authorities working in malaria elimination settings should consider using PCR to guide detection of the residual hotspots, as this provides greatest opportunities to find asymptomatic individuals and sub-patent parasite reservoirs in the communities [43].

All these studies clearly show that while sub-national stratification may be the most effective approach to decide on how to allocate resources, the type of data used for such epidemiological profiling matters significantly; especially when the stratification is done at local-sub-district levels. In places like southeastern Tanzania, which has experienced decades of sustained malaria interventions and progress, and where robust entomological surveillance already exists [45], addition of detailed parasite prevalence data from population-level surveys is required to enable more precise, fine-scale stratifications at both district and sub-district levels.

The aim of this study was, therefore, to generate a high-resolution population-level survey map of P. falciparum prevalence in two districts in south-eastern Tanzania and to compare the fine-scale malaria strata obtained using data from different test methods, namely RDTs, microscopy, and qPCR. Additionally, the study evaluated the performance of RDTs and microscopy relative to qPCR in a range of transmission settings from high to very low. This study also sought to provide detailed population survey data on malaria burden to complement the ongoing entomological surveys in the study area.

Methods

Study site

The study was conducted in Morogoro region, in south-eastern Tanzania (Fig. 1), in the two districts of Kilombero (population: ~583,000; 8.2414°S, 36.3349°E; elevation: ~270 m) and Ulanga (population: ~233,000; 8.9889°S, 36.6133°E; elevation: ~800 m). The average malaria prevalence in the Morogoro region has previously been estimated to exceed 10%, with P. falciparum as the dominant malaria species [19, 20, 46]. The main economic activities for residents include rice farming, sugarcane farming and maize farming, though the area also has other food crops and large commercial tree plantations (teak). The known annual rainfall range is 1200–1400 mm in the lower-lying plains of Kilombero district, and 1400–2100 mm in the higher areas in Ulanga district [47]. Approximately 90% of the rainfall occurs during the wet seasons between December to April, with dry seasons typically lasting from June through September [47]. The annual mean daily temperature is around 27 °C in the lowlands and approximately 23 °C in the highlands. Relative humidity averages from 75% in the lowlands to 80% in the highlands.

Fig. 1

Study villages in Kilombero and Ulanga districts, south-eastern Tanzania

Study design, procedures and survey tools

The cross-sectional surveys were conducted once per village, and the entire surveillance spanned two consecutive years, from 2022 in Ulanga to 2023 in Kilombero, covering the months of April to September each year. Villages were randomly selected from each district, and sample sizes per village were proportionately determined based on the population of each village using Cochran’s formula adjusted for finite populations [48,49,50]. The sample size aimed to achieve a 95% confidence interval with a precision of 5% for each specific village prevalence estimate see supplementary Table 3.

The expected prevalence varied depending on the village and was derived from previous surveys and health centers within each village. These earlier unpublished surveys had covered a much smaller subset of the areas and yielded prevalence rates ranging from as low as 1% to as high as 45%. For villages without these population surveys, health facility data from the given village or from neighboring villages were utilized. Based on the estimated sample sizes per village, representative households were determined, assuming an average household size of three individuals. Households were selected by randomizing the names of all households obtained from the respective village administrations. This selection ensured equal representation of households from each sub-village, thereby covering all parts of the village. The selected households were visited and recruited if they consented.

The screening criteria included individuals aged 5–60 years who had not taken malaria medications in the preceding two weeks. This precaution aimed to prevent potential overestimation by RDTs, as they may detect residual traces post-treatment [51]. Individuals needing special medical attention, such as pregnant women, were excluded from the study. All eligible individuals in selected households were allowed to participate. Each participant who underwent malaria screening was assigned a unique identification number that was also linked to their corresponding household ID. On-site finger-prick blood samples were collected for three diagnostic tests: (1) RDTs, (2) creating thick and thin blood smears, and (3) collecting 3–5 dried blood spots (samples) on Whatman 903™ protein saver cards. Subsequently, these samples were transported to the reference laboratory for microscopy and qPCR analysis.

Ethical considerations, survey team, and trainings

Permission to conduct this study was obtained from the Ifakara Health Institute Review Board (Ref: IHI/IRB/No: 1/2021) and the National Institute for Medical Research-NIMR (NIMR/HQ/R.8a/Vol. 1X/3735). Additionally, approvals were obtained from regional, district, ward, and respective selected village authorities before commencing the surveys, given the screening was done at centralized location in each village. Written informed consent was obtained from individual adult participants (and parents or guardians of those aged below 18) on the day before the actual testing. The study team consisted of 11 members, including three molecular laboratory technologists, four licensed medical laboratory microscopists, two licensed clinical officers, and two social scientists. Prior to the survey commencement, a five-day training session was conducted at the Ifakara Health Institute laboratory. This training covered explanations of the study protocols, pilot implementations, procedures for protecting human participants, quality assurance and training on data collection tools.

Tests using malaria rapid diagnostic tests (RDTs)

A small blood drop obtained through a finger prick was collected onto the RDTs (SD Bioline Ag Pf/Pan), following the manufacturer’s instructions. The buffer solution was applied according to standard RDTs procedures and left on the bench surface for up to 20 min. The type of RDT used were capable of detecting P. falciparum infections by targeting histidine-rich protein-2, which react on the Pf-line. Additionally, they could detect Plasmodium malariae, Plasmodium vivax and Plasmodium ovale by targeting glycolytic lactate dehydrogenase, expressed by the Pan-line on RDTs [35, 52]. The RDT results were recorded on a paper form, and any individuals who tested positive for malaria were promptly treated with Artemether Lumefantrine (ALu), following Tanzania’s national malaria treatment guidelines [53].

Tests using microscopy

Thick and thin blood smears were created in the field, stained with 10% Giemsa for 15 min then examined for the presence of malaria parasites under oil immersion at 100X magnification [49, 54, 55]. Two experienced microscopists independently read the slides, and discrepancies between them were resolved by a third, more experienced microscopist. They read the thick smear first, and if an infection was detected, the thin smear was read to identify parasite species. The presence of both asexual and sexual malaria parasite stages discriminating P. falciparum, P. malariae, and P. ovale was recorded. Asexual stage parasites were counted per 200 white blood cells and assuming 8000 WBC/µL [56]. The mean count of malaria parasite by microscopy between the two readers was calculated and confirmed by the third reader.

Tests using real-time qPCR assays

A representative sample of approximately two thirds of all samples was randomly selected from each village and screened further by quantitative polymerase chain reaction (qPCR) i.e. 4905 samples out of the total 7628 samples. Out of the five spots on the Whatman protein saver card, three were punched using a handheld 6 mm slot hole puncher. These punched spots were then used for DNA extraction with the Quick-DNA™ Miniprep Plus Kit (Zymo Research, USA) [57], and eluted with 50 µL of elution buffer, stored at −20 °C for further detection and quantification of P. falciparum infections using probe-level allele-specific quantification (PlasQ)-multiplex qPCR assays protocols [39, 58, 59]. The detection and quantification of P. falciparum parasites were performed using the Bio-Rad CFX96 real-time PCR system (Bio-Rad Laboratories, USA) [58] and analyzed with Bio-Rad CFX maestro software. The qPCR reaction, PlasQ primers and probes mix, are summarized in supplementary online Tables 1 & 2. DNA amplification processes included: activation at 95 °C for 1 min, denaturation at 95 °C for 15 s, and annealing and elongation at 57 °C for 45 s for 45 cycles, followed by melting [58].

The qPCR assays were run with positive controls (samples with confirmed P. falciparum) and a non-template control (samples with no P. falciparum as negative control). For absolute parasite quantification, the WHO international standard for P. falciparum nucleic acid amplification techniques were used (WHO reference from NIBSC#04/176) [39]. The standard was reconstituted following the manufacturer’s instructions and serially diluted in the range of 100,000 parasites/µL to 0.01 and analysed in triplicates.

During the qPCR assay, the prepared standards were run together with unknown samples, and at the end of the assay, the standard curve and samples were normalized and analyzed with Bio-Rad CFX maestro software. The obtained normalized Ct values of the samples and the linear regression equation derived from the standard curve were used to calculate the parasites density of the unknown samples, expressed as parasites per microlitre (parasites/µL) of blood.

Malaria stratifications (PfPR) categories

Malaria stratifications generally rely on predefined PfPR categories, with NMCPs adopting WHO definitions. In this study, which focuses on fine-scale stratifications at the village level (the lowest administrative boundaries), PfPR categories predefined in a study conducted in mainland Tanzania were adapted [2, 6]. These categories stratify malaria risk at the council level, which is also considered fine scale, as it is below the district level [6]. The arbitrary risk categories used are: very low risk (PfPR < 1%), low risk (1% ≤ PfPR < 5%), moderate risk (5% ≤ PfPR < 30%), and high risk (PfPR ≥ 30%). In this study, the strata were defined as fine-scale because they were performed at the village level using data derived from village prevalence estimates.

Data analysis

All results from RDTs, microscopy, and qPCR were entered into the Open Data Kit (ODK) system [60], and subsequently downloaded as an excel file for further cleaning. The datasets for RDTs, microscopy, and qPCR results were merged based on the participant’s ID using the Pandas Python package [61]. Generalized linear mixed models (GLMMs) with a binomial distribution were utilized to evaluate the relationship between malaria infection risk and the predictors age and gender. These models were implemented using the R statistical software, where random effects for both Village and House ID were incorporated to address the hierarchical structure and intra-cluster correlations within the dataset.

Additionally, to evaluate the performance of RDTs and microscopy in fine-scale malaria stratifications compared to qPCR, their agreement was tested using Kappa statistic [62], and the resulting Kappa values interpreted as follows: κ < 0.20 as poor agreement, 0.21–0.40 as fair, 0.41–0.60 as moderate, 0.61–0.80 as substantially good and 0.81–1 as almost perfect agreement [63]. In addition, the positive predictive value (PPV) for RDTs and microscopy was computed, using qPCR results as the reference, per village, as (proportion of positive test results that are actually true positives, estimated as PPV = True Positives/(True Positives + False Positives)). Fine-scale stratification by villages was performed using data from qPCR, RDT, and microscopy to generate prevalence maps with QGIS software version 3.26, enabling visualization of malaria prevalence across the study area. To further analyse this data, Inverse Distance Weighting (IDW) interpolation techniques were employed. IDW estimates values at unsampled locations by weighting observed data points inversely to their distance, creating a smooth, continuous surface [64]. This method was applied to the malaria prevalence data from RDTs, microscopy, and qPCR for each village, producing continuous surfaces that visually depict spatial variations in malaria risk across the study area.

The geometric mean of parasite density, estimated by microscopy and qPCR, was calculated for each village. These densities were also statistically compared across different gender and age groups within each village. The non-parametric Mann-Whitney statistics were used to compare the parasite densities between two categorical groups, while Kruskal-Wallis statistical tests were used to compare more than two categorical groups [65, 66]. For example, differences in parasite densities between age groups were tested using Kruskal-Wallis statistics, and if statistically significant, the Mann-Whitney statistics were applied for pairwise statistical significance tests. All analyses comparing parasite densities excluded the negative cases and focused solely on investigating parasite density distribution among malaria-positive patients within each respective village. Lastly, to test for statistical correlations between parasite prevalence and parasite densities estimated by both qPCR and Microscopy, non-parametric Spearman’s rank correlation tests were employed [67]. Additionally, a logistic regression model was used to evaluate the probability of detecting malaria infections (positive or negative) with both RDTs and microscopy at varying parasite densities estimated by qPCR.

Results

Baseline study population

This survey covered 35 villages across Ulanga and Kilombero districts. A total of 7628 participants (> 5 years) were recruited upon consent and tested for malaria using RDTs and microscopy. The number of participants tested per village ranged from over 132 to 449. Additionally, 64.3% of these participants (4905) were also tested using qPCR (see Fig. 2). Males comprised 38% of the study population, while females made up 62%. Among the participants, 35% were school-aged children (5–15 years), and 65% were aged 16 years and above (Table 1).

Fig. 2

Schematic representation of the study sampling procedures

Table 1 Baseline characteristics of the study populations

Malaria prevalence by RDTs, microscopy and qPCR

In the Ulanga district, malaria transmission was found to be high by both qPCR and RDTs, with P. falciparum prevalence rates of 53.89% [95% CI 52.06–55.72] and 38.35% [95% CI 36.92–39.79], respectively. However, microscopy categorized it as moderate, with a prevalence rate of 26.07% [95% CI 24.77–27.36] (Table 2). Within this moderate to high transmission strata in Ulanga, males had a significantly higher prevalence of malaria compared to females. The odds ratios of malaria infection in males compared to females were estimated as 1.6 [95% CI 1.4–1.8] (P < 0.001) by RDTs, 1.4 [95% CI 1.2–1.6] (P < 0.001) by microscopy, and 1.5 [95% CI 1.2–1.7] (P < 0.001) by qPCR (Table 2). All tests - RDTs, microscopy, and qPCR - indicated that school-age children (5–15 years) had a significantly higher prevalence of malaria infections than the other age groups, (P < 0.001), refer Table 2.

Table 2 Malaria prevalence in Ulanga district by sex and age groups, as measured using RDTs, microscopy and qPCR

In Ifakara council, within the Kilombero district, both RDTs and microscopy categorized the area as a low risk stratum, with observed prevalence rates of 2.68 [95% CI 2.12–3.24] and 1.84 [95% CI 1.37–2.30], respectively (Table 3). However, qPCR classified Kilombero district as a moderate risk stratum with a prevalence rate of 8.77 [95% CI 7.55–9.99] (Table 3). Notably, there were no statistically significant differences in malaria prevalence between males and females in this low to moderate transmission setting, as indicated by both RDTs (Odds ratios 1.2% [95% CI 0.8, 2.], P = 0.361) and microscopy (Odds ratio 1.33% [95% CI 0.7, 2.0], P = 0.521), as well as qPCR (Odds ratios 1.2% [95% CI [0.7–1.4], P = 0.941). Additionally, school-age children (5–15 years) exhibited a significantly higher risk of malaria infections compared to those 16 years old and above, as demonstrated by both RDTs and microscopy (P < 0.001). However, qPCR demonstrated no significant difference between the two groups (P < 0.124) (refer to Table 3).

Table 3 Malaria prevalence in the Kilombero district by sex and age groups, as measured using RDTs, microscopy and qPCR

Micro-stratification of malaria risk using data collected by qPCR, RDTs, and microscopy

Significant variability in malaria infections was observed at the individual village level, with prevalence rates ranging from 0% to over 50% across the study area (Fig. 3; Table 4). Additionally, the method used to test for malaria significantly impacted the risk categorization of villages. Among the 35 villages surveyed, qPCR data indicated that only one village (1.2% of all villages) had very low malaria prevalence (PfPR < 1%). In contrast, RDTs identified 12 villages (34.3% of all villages) and microscopy identified 11 villages (31.4% of all villages) as having very low prevalence. For moderate transmission, qPCR, RDTs, and microscopy categorized 15, 9, and 8 villages, respectively. For high transmission, qPCR identified 14 villages, RDTs identified 8, and microscopy identified 6. Notably, qPCR detected more malaria infections than RDTs and microscopy, resulting in many villages being classified into higher transmission categories. For actual value of prevalence per village refer to supplementary Tables 3 & 4.

Fig. 3

Fine-scale malaria mapping of 35 surveyed villages in the Ulanga and Kilombero districts using qPCR, RDTs, and microscopy data is shown in the top panel. The bottom panel indicates malaria risk generated by interpolating prevalence data obtained for each surveyed village by qPCR, RDTs, and microscopy. Categories defined based on calculated prevalence rates as either very low risk (PfPR < 1%), low risk (1% ≤ PfPR < 5%), moderate risk (5% ≤ PfPR < 30%), or high risk (PfPR ≥ 30%) (total number of villages = 35)

Table 4 Number of villages categorized into different risk strata based on the P. falciparum prevalence rate (PfPR) data from qPCR, RDTs and microscopy

Overall, using qPCR data, over 80% of the villages were classified as moderate to high risk, significantly higher than the 48% classified by RDTs and 40% by microscopy. Conversely, while only 17% of the villages were classifiable as having low or very low malaria risk based on qPCR data, as high as 51% and 60% of the villages were classified into these same categories based on RDT and microscopy data (Table 4; Fig. 4).

Fig. 4

Percentage of villages categorized by different testing methods as either very low risk (PfPR < 1%), low risk (1% ≤ PfPR < 5%), moderate risk (5% ≤ PfPR < 30%), or high risk (PfPR ≥ 30%) (Total number of villages = 35)

Comparison of the performance of RDTs and microscopy relative to qPCR

In this comparative analysis, only samples tested by all three methods—PCR, RDTs, and microscopy—were included (n = 4905). Among these, qPCR identified 1712 (34.9%) as positive, whereas RDTs and microscopy classified 1289 (26.3%) and 843 (17.2%) positives, respectively (Table 5; Fig. 5).

Table 5  Proportion of malaria positive samples missed by RDTs and microscopy when qPCR is used as the reference

Fig. 5

The Venn diagram illustrates positive samples detected exclusively by a specific tool while the other two missed them (qPCR only: 594 positive, RDT only: 171 positive, Microscopy only: 84 positive). Additionally, it shows intersections indicating positive detection by two tools when one detects negative (qPCR & RDT: 415 positive; qPCR & Microscopy: 56 positive; RDT & Microscopy: 56 positive). It also indicates intersections where all tools detect positive samples (qPCR, RDT, & Microscopy: 647 positive)

Both RDT and microscopy missed several infections otherwise identified by qPCR. This category of false negatives included cases where qPCR identified a sample as positive, but microscopy identified it as negative, cases classified as positive by qPCR but negative by RDTs, and cases where RDTs indicated positive results while microscopy indicated negative result. Out of the 1712 positives detected by qPCR, RDTs missed 650 (37.97%) and microscopy missed 1009 (58.9%) (Table 5). Additionally, when comparing microscopy to RDTs, microscopy failed to detect 45.46% (586/1289) of malaria infections detected by RDTs. RDTs correctly identified 1062 (62.03%) samples as true positives, while microscopy identified 703 (41.06%) as true positives (Table 6). Furthermore, RDTs misclassified 227 (7.10%) samples as false positives, while microscopy misclassified 140 (4.38%) (Table 6). More importantly, 56 samples were classified as positive by both RDTs and microscopy but were missed by qPCR (Fig. 5).

Table 6 Evaluation metrics for assessing the performance of RDTs and microscopy relative to qPCR during the fine-scale stratification of malaria risk in Ulanga and Kilombero districts, southeastern Tanzania

Positive predictive values (PPVs), sensitivity, specificity, and agreement of RDTs and microscopy when compared to qPCR

Considering qPCR as the benchmark, the sensitivity (the proportion of actual positives which were correctly identified as such) of RDTs was 62.0% [95% CI 60.0–64.2], while that of microscopy was 41.0% [95% CI 38.8–43.4]. The specificity (proportion of actual negatives which were correctly identified as such) was 92.9% [95% CI 92.00–93.7] for RDTs and 95.6% [95% CI 94.9–96.3] for microscopy (Table 6). Overall, the positive predictive value (PPV), i.e. the probability that individuals with a positive test result actually have true infection, was 82.4% [95% CI 80.3–84.4] for RDTs and 83.4% [95% CI 80.8–86.0] for microscopy (Table 6). Importantly however, the PPV for both RDTs and microscopy varied with malaria endemicity, generally increasing with prevalence, ranging from less than 20% in very low transmission areas to over 80% in high transmission areas (Fig. 6).

Fig. 6

Estimates of the positive predictive values (PPVs) of RDTs and microscopy at different malaria endemicities across the study villages, defined based on either qPCR-derived strata (A) or RDT-derived strata (B). Panel C illustrates the trend in sensitivity of both RDTs and microscopy across age groups. The shaded area represents the 95% confidence interval. Panel D displays the detection probability of both RDTs and microscopy relative to parasite density estimated by qPCR

When considering the micro-strata generated using qPCR data, the PPV of RDTs and microscopy started at 0% in very low risk strata and gradually increased to > 80% as villages shifted towards high risk strata (Fig. 6A). However, when referring to the strata generated using RDTs data (Fig. 6B), the PPV of both RDTs and microscopy started at 20% in very low risk strata and gradually increased to > 80% in high-risk strata. The agreement between RDTs and qPCR was good (Kappa value = 0.68 [95% CI 0.6–0.8]), while the agreement between microscopy and qPCR showed fair agreement (Kappa value = 0.5 [95% CI 0.4–0.6]), (Table 6).

The sensitivity of both RDTs and microscopy varied by age, where RDTs sensitivity was higher for school-aged children (> 80%) and dropped to 75% and 60% for 16–20 years and > 20 years, respectively (Fig. 6C). A similar trend of sensitivity was observed for microscopy (Fig. 6C), indicating that RDTs and microscopy perform better in detecting malaria in school-aged children compared to adults.

The relationship between parasite density and the malaria detection probability by RDTs and microscopy was also examined. In this analysis, the probability of RDTs detecting positive malaria infections was maximized reaching 1 at 100 parasites/µL, where the logistic regression (logit (p)) model saturated (Fig. 6D). At this density in contrast, the probability of microscopy to detect malaria infections was only 0.85% (Fig. 6D), suggesting higher sensitivity of RDTs vs. microscopy.

Parasites density estimates and their correlations with Plasmodium prevalence

Further, asexual parasite densities estimated by both microscopy and qPCR were investigated and compared across different sex and age groups using Mann-Whitney statistics for two categories and the Kruskal-Wallis statistical test for more than two categories. Overall, PCR was capable of detecting approximately 100 fold lower parasite densities compared to microscopy. The geometric mean asexual parasite density estimated by microscopy was 2206.4 parasites/µL (95% CI 1976.7–2462.8), while that estimated by PCR was 27.07 parasites/µL (95% CI 23.23–31.54) (Fig. 7A and B).

Fig. 7

Geometric mean parasite densities per age group estimated by A qPCR and B microscopy. C (density estimated by qPCR) and D (density estimated by microscopy) show the correlation between parasite density and prevalence estimates by qPCR. E (density estimated by qPCR) and F (density estimated by microscopy) show the correlation between parasite density and prevalence estimates by RDTs

The asexual parasite density of infected individuals significantly differed between males and females as estimated by qPCR (P < 0.001), with males harboring a higher parasite density compared to females, though similar trend was observed by microscopy, this sex difference was not statistically detectable by microscopy (P = 0.11). Importantly, the geometric mean parasite density estimated by both microscopy and qPCR per village demonstrated a significant positive correlation with the parasite prevalence of the respective village. Thus, villages with high malaria prevalence also had high malaria parasite densities compared to villages with lower prevalence (Fig. 7C-F). Considering qPCR-estimated geometric mean parasite densities, the Spearman rank correlation score was 0.77 (P < 0.001) and 0.76 (P < 0.001) when the malaria prevalence of the villages was estimated by qPCR and RDTs, respectively (Fig. 7C and E). On the other hand, the Spearman rank correlation for the parasite density estimated by microscopy was 0.55 (P < 0.003) and 0.48 (P < 0.012) for qPCR and RDTs estimated prevalence of the villages, respectively (Fig. 7D and F).

When parasite density by age groups were analyzed, both microscopy and qPCR revealed a significant difference in estimated malaria parasite densities between age groups based on Kruskal-Wallis statistics (P < 0.001) (Fig. 7A and B). Pairwise tests by Mann-Whitney statistics revealed that school-aged children (5–15 years old) harbored a higher parasite density than those 16 years old and above (P < 0.001) as indicated by both microscopy and qPCR (Fig. 7A and B).

Discussion

In malaria-endemic countries, data-driven risk stratification is increasingly used at district or higher regional levels to guide intervention strategies and optimize resource allocation. Additionally, the geographical variations in levels of endemicities and the shift towards elimination in some settings necessitates finer resolution for optimal resource allocation [6, 10, 68]. In most settings in Africa, the data used for epidemiological stratification of malaria typically comes from rapid diagnostic tests (RDTs) or microscopy-based testing, which despite wide-scale availability and low operational costs, often perform poorly in low transmission settings [43, 69, 70]. While direct comparisons of these diagnostic tools for fine-scale stratification are currently limited, selecting the most appropriate data sources and testing methods is crucial, as different methods can yield significantly different results depending on endemicity, particularly in elimination settings. Even without alternative testing methods, data users and decision-makers need to understand the limitations of their selected approaches, especially the weaknesses of current dominant data sources like RDTs or microscopy. In this study, a high-resolution survey of P. falciparum malaria was conducted in two Tanzanian districts, comparing fine-scale strata obtained using RDTs, microscopy, and qPCR assays.

The study showed significant variability in malaria risk at a fine scale. Within less than 150 km, malaria prevalence estimates ranged from 0% to over 50% across contiguous villages in an area broadly classified as moderate risk (~ 17% PfPR) by recent government stratification [46]. Such fine-scale variability is not uncommon and has been observed in several other settings [71]. In one study in Madagascar, there was a tenfold difference of malaria prevalence within a radius of less than 50 km [72]. For precise micro-stratifications, this study emphasizes the importance of carefully selecting diagnostic tools, especially for local malaria elimination efforts. The findings of this study indicate that RDTs and microscopy have poor positive predictive values, which can be even less than 20% in villages with very low and low transmission as the proportion of truly infected individuals is very small compared to non-infected persons. There were also significant discrepancies in the resulting micro-strata depending on the test method used. For instance, among the 35 surveyed villages, RDTs and microscopy classified 12 and 11 as very low and 6 and 10 as low risk strata, respectively, while qPCR identified only 1 village as very low and 5 as low transmission. This means RDTs and microscopy classified majority of the villages as very low to low risk while qPCR classified most villages as moderate to high risk (Table 4).

Clear demarcation of areas with very low to low risk versus those with moderate to high risk is essential, particularly in the push towards elimination. As countries increasingly adopt data-driven decision-making for malaria control, there is a risk of improper resource allocation or premature withdrawal of effective interventions from localities erroneously deemed as nearing elimination. Local authorities need to decide which data to use for local-level micro-stratification and whether RDTs, commonly used for broader-scale sub-national stratification, suffice for fine-scale local decision-making. Previous evidence has shown that hotspots identified by RDTs are less stable than those identified by microscopy and PCR [43]. Hotspots of febrile malaria infections are also generally unstable and variable over geographical spaces, while hotspots of asymptomatic cases tend to be more permanent and can be more practically targeted for transmission control [73]. This current study also found significant positive correlations between malaria parasite densities and malaria prevalence in southeastern Tanzania, emphasizing the need to incorporate tests that depict sub-microscopic infections into malaria stratification and decision making to better target the hotspots. As reported in other studies, villages classified as low-transmission areas in this study also had lower geometric mean parasite densities compared to those with higher transmission rates.

The findings of this study, benchmarked against qPCR, reveal limited detection capabilities of RDTs and microscopy in overall fine-scale stratifications, especially in low transmission settings. Previous studies have emphasized the usefulness of routine hospital data for micro-stratifications [8, 10, 74, 75]. However, evidence indicates that both microscopy and RDTs are less effective in identifying stable febrile malaria hotspots, except for asymptomatic hotspots, which are reliably identified by microscopy [73], however still not stable when transmission is low [76]. This research underscores the importance of identifying subclinical infections using sensitive tools to advance malaria elimination, particularly through fine-scale population surveys. Additionally, there is evidence suggesting potential benefits from integrating hospital and school-age children survey data or even antenatal care centers [2, 9]. Nevertheless, these approaches heavily rely on rapid diagnostic tests (RDTs) as the primary tool for malaria detection. While the World Health Organization (WHO) recommends monitoring RDT performance alongside microscopy, this study is particularly relevant as have directly compared the fine-scale stratification capabilities of RDTs, microscopy, and qPCR at a fine scale.

This study demonstrated overall good agreement between RDTs and qPCR, while microscopy showed fair agreement. However, RDTs missed over 38% of malaria infections, particularly among adults over twenty years old, who were found to harbor lower parasite densities compared to those under twenty years old. However, RDTs remain useful in testing fever-positive malaria cases in hospitals and are widely employed in population surveys due to their cost-effectiveness and ease of implementation [32]. As evidenced in this study, carefully re-consideration of using RDTs for finer-scale mapping and intervention planning at sub-district level should be a priority. Similarly, microscopy missed > 50% of the malaria infections detected by qPCR, which is consistent with previous studies, including a meta-analysis of 42 studies, which showed that microscopy misses over 50% of malaria infections [26, 77]. Operational challenges, such as the level of expertise required for accurate detection and the need for electricity and precise sample handling procedures, contribute to these limitations. Interestingly, microscopy underestimated malaria risk by classifying more villages as low strata compared to qPCR. Nonetheless, microscopy still plays a crucial role when used in conjunction with tools like RDTs, providing valuable information about malaria parasite densities [78,79,80]. Here, parasite densities estimated by microscopy were 100 times higher than those measured by qPCR, consistent with similar trends observed in previous studies [58]. The findings of this study also indicate that the false-negative rate of microscopy decreases with increasing parasite density, a pattern observed in other studies too [78].

The analysis also revealed variations in parasite densities across different age groups, with school-age children (5–15 years old) exhibiting higher parasite densities compared to individuals aged 16 and above. Notably, our study identified a reduced sensitivity of both RDTs and microscopy among adults aged over 16 years, consistent with findings from prior studies conducted before 2015 in various regions [81,82,83,84,85,86,87,88]. It is possible that this pattern is driven by age-related differences in malaria parasite prevalence, as observed in Table 3, and may be confounded by unequal sampling of the age distribution. Furthermore, the findings suggest that this trend may be attributed to the lower parasite density estimates observed in adults within the study (Fig. 7A and B). Significantly, this research provides valuable insights, highlighting the potential implications of these trends, particularly in fine-scale mapping scenarios, where RDTs and microscopy may underestimate burden at very low and low transmission strata, with qPCR serving as the reference standard in this study.

When selecting a tool for a stratification exercise, it is crucial to consider several key operational factors. First, to assess whether transmission levels are sufficiently low to require a high-sensitivity tool capable of differentiating between locations with the lowest prevalence. This was exemplified in much of this study area, especially in the northern zones where qPCR was clearly more sensitive than RDTs and microscopy. Second, evaluate the logistical and cost implications associated with using each tool for testing individuals. Finally, consider the ethical requirements and the ability to provide immediate results and treatment when necessary. Ultimately, when aiming to achieve more precise fine-scale mapping of malaria infections to facilitate more accurate resource allocation, the choice of testing tools should be based on the balance between sensitivity and operational feasibility.

Sensitive molecular tools like qPCR are available, but qPCR has operational challenges, including the need for well-designed infrastructure, high costs, and expertise, and it is not portable for remote areas. Efforts are underway to develop portable qPCR technologies, but cost and expertise remain significant barriers. To address these gaps, NMCPs should develop innovative plans, which might include: (a) establishing centralized facilities for receiving and processing qPCR samples and conducting such surveys infrequently, such as every 3 years; (b) partnering with local research organizations to support high-accuracy evaluations using nucleic acid-based tests; (c) exploring alternative methods for micro-stratification, such as geo-spatial modeling that integrates information such as land use, elevations, and other environmental factors, and potentially combining facility and population survey data.

A related point to emphasize is the overall need for highly sensitive, cost-effective, and potentially reagent-free tools that align with the economic context of malaria-endemic settings. Recent innovations such as high/ultra-sensitive RDTs, the saliva-based tests [89] or the use of Infrared spectroscopy (IR) and machine learning (ML) [79, 90] have shown promise in detecting malaria infections at sensitivities equivalent to PCR, but further research are needed before these technologies can be routinely deployed. Such reagent-free assays like the IR-AI based approaches would be particularly transformative for scaling up effective micro-stratification of malaria risk in Africa.

It is important to recognize that prevalence rates obtained from different tests are not directly comparable; for example, a 5% prevalence detected by RDTs does not equate to a 5% rate detected by PCR. In some cases, ranking prevalence may be more critical than the exact rates. Additionally, variations in sample sizes should be considered, especially in fine-scale stratifications, as confidence intervals (CIs) may cause overlap across strata. Therefore, it is important to assess stratifications based on prevalence CIs rather than relying solely on point estimates. In this study, despite some villages spanning two or three strata based on their CIs, RDTs and microscopy still classified more villages as very low to low risk, while qPCR frequently classified them as moderate to high (see Supplementary Table 4).

More importantly, the need for more detailed data becomes crucial when stratification occurs at local scales, such as comparing wards or villages within districts, as done in this paper, rather than at the national level, where regions or districts are compared. While most malaria stratifications are currently conducted at national and sub-national levels using RDTs and sometimes microscopy, this study highlights that the choice of test methods can influence decision-making and overall control strategies, especially in finer-scale stratifications. Determining appropriate public health decisions was beyond the scope of this study and may vary based on the scale of stratification. However, this study emphasizes that decision-making should account for the strengths and limitations of the available data when planning stratifications. Additionally, countries may establish locally relevant thresholds for deciding which interventions to implement or withdraw.

This study also raised some important new questions. For example, it is interesting to observe that areas with low transmission also have persistently low parasite densities compared to those in higher transmission settings, which are usually reported to acquire immunity and become protective. Although this phenomenon of low parasite density in low transmission areas was not explored in detail in this study, it could suggest residual immunity among individuals due to recent declines in transmission or potential migration of participant [91]. Studies have demonstrated that in low transmission areas, highly virulent parasites are more exposed to facilitate malaria transmission by mosquitoes compared to low virulent ones [92]. Consequently, high virulent parasites are detected and treated, leading to their removal from the population [92]. This leaves behind low virulent parasites that are less exposed and maintain low densities, becoming asymptomatic, undetectable, and untreated [93]. This phenomenon may contribute to long-term parasite transmission strategies, highlighting the importance of using highly sensitive tools for screening [58, 92, 94].

In interpreting the findings of this study, several limitations should be considered. The IDW technique used here is primarily intended for visualizing general trends in malaria risk rather than providing precise prevalence estimates or specific risk levels for each village (both sampled and unsampled). Unlike the exact data shown in the top panel of Fig. 3, IDW interpolation is subject to smoothing effects, particularly in areas with fewer data points, which may reduce the accuracy of the maps. Villages without data may contribute to differences, such as the appearance of low-risk areas in the southern regions, where actual values could range from moderate to high. Nonetheless, this approach effectively visualizes likely malaria risk patterns in unsampled areas by using known data from sampled villages, based on the results of different screening methods. Furthermore, seasonal variations are expected to influence village-level prevalence estimates and, consequently, the distribution of malaria risk. However, seasonality does not affect the primary objective of this study, which is to evaluate the performance of qPCR, RDTs, and microscopy for fine-scale stratification.

This study also did not account for all factors that may contribute to the broader heterogeneity of malaria infections in southeastern Tanzania. It is crucial for future studies to also investigate how different categorizations of malaria prevalence-based strata impact the agreement between diagnostic tools. This will help determine the most appropriate categories for decision-making and resource allocation. Future investigations should also delve into potential environmental, geographical, immunological or genetic diversity of the parasite influences underlying this variability. Additionally, the biological significance of missed infections by both RDTs and microscopy was not explored. Consequently, the study did not estimate the transmission burden associated with these undetected positive samples, nor assess the parasite densities necessary to sustain transmission in the population.

Conclusion

As countries progress towards malaria elimination, fine-scale mapping of malaria risk becomes increasingly important. This study highlights significant variability in village-level malaria risk within and between districts in southeastern Tanzania, an area where the scale-up of effective interventions has led to substantial progress, yet cases persist despite high intervention coverage. Secondly, the study underscores the variable performance of different testing methods in stratifying risk. While RDTs and microscopy, the primary test methods used in low-income endemic settings and the main sources of data for ongoing epidemiological stratification efforts, were effective in high-transmission areas, they performed poorly in low-transmission settings, often classifying most villages as very low or low risk. In contrast, qPCR classified most villages as moderate or high risk. These findings demonstrate the importance of using appropriate testing methods for data-driven, fine-scale risk stratification to enhance targeted interventions aimed at reducing and eliminating malaria. The study underscores the need for proper choices of malaria testing approaches that are both operationally feasible and sufficiently sensitive to enable precise mapping and effective targeting of malaria in local contexts. More importantly, public health authorities must recognize the strengths and limitations of their available data when planning local stratification or making decisions. While innovation for more effective strategies is ongoing, sensitive molecular tools like qPCR, despite their operational challenges, will be crucial for accurate malaria risk mapping and intervention planning, especially in settings with significantly reduced risk. Going forward, developing new tools that balance operational costs and sensitivity, particularly in low transmission settings, will be essential for effective malaria control and eventual elimination.