Limited information is available on the usefulness of the PIMA™ analyser in predicting antiretroviral treatment eligibility and outcome in a primary healthcare clinic setting in disadvantaged communities in KwaZulu-Natal, South Africa.
The study was conducted under the eThekwini Health Unit, Durban, KwaZulu-Natal. Comparison of the enumeration of CD4+ T-cells in 268 patients using the PIMA™ analyser and the predicate National Health Laboratory Services (NHLS) was undertaken during January to July 2013. Bland-Altman analysis to calculate bias and limits of agreement, precision and levels of clinical misclassification at various CD4+ T-cell count thresholds was performed.
There was high precision of the PIMA™ control bead cartridges with low and normal CD4+ T-cell counts using three different PIMA™ analysers (%CV < 5). Under World Health Organization (WHO) guidelines (≤ 500 cells/mm3), the sensitivity of the PIMA™ analyser was 94%, specificity 78% and positive predictive value (PPV) 95%. There were 24 (9%) misclassifications, of which 13 were false-negative in whom the mean bias was 149 CD4+ T-cells/mm3. Most (87%) patients returned for their CD4 test result but only 67% (110/164) of those eligible (≤ 350 cells/mm3) were initiated on antiretroviral therapy (ART) with a time to treatment of 49 days (interquartile range [IQR], 42–64 days).
There was adequate agreement between PIMA™ analyser and predicate NHLS CD4+ T-cell count enumeration (≤ 500 cells/mm3) in adult HIV-positive individuals. The high PPV, sensitivity and acceptable specificity of the PIMA™ analyser technology lend it as a reliable tool in predicting eligibility and rapid linkage to care in ART programmes.
Poor rates of linkage to care for those with low CD4+ T-cell counts, eligible for antiretroviral therapy (ART), have been reported in several African cohort studies.
Several attrition steps exist in the continuum of care pathway: patients lost to care between testing HIV-positive and going for a CD4 test
The Alere PIMA™ POC has been evaluated against the ‘gold standard’ flow cytometry platforms, for example, Beckman Coulter using panleucogating (PLG)
This study assessed the accuracy, sensitivity and specificity of the Alere PIMA™ POC analyser in CD4+ T-cell count enumeration compared to the predicate South African National Health Laboratory Services (NHLS) flow cytometry test (Beckman Coulter) and its potential operational role as a predictor of ART eligibility in a primary healthcare clinic (PHC) in Durban, South Africa.
The study occurred at Lancers Road PHC, a facility under the eThekwini Health Unit, situated in the centre of the convergence of the taxi rank in the city centre of Durban. This PHC offers HIV Counselling and Testing (HCT) – approximately 900 per month to walk-in patients who receive pre- and post-test counselling and CD4 testing for the staging of HIV-1-infected disease to determine eligibility for ART. Patients are advised to return after 7 days for their CD4 results. As per the SA HIV and AIDS guidelines
Routine CD4+ T-cell enumeration is conducted at the NHLS one day after the blood draw via Beckman Coulter flow cytometry using PLG methodology, the standard of care in this setting as described previously.
Quality control and routine PIMA™ analyser maintenance were performed daily as per manufacturer’s guidelines: one control has low CD4+ T-cell counts (115 cells/mm3 – 235 cells/mm3) and the other has normal CD4+ T-cell counts (719 cells/mm3 – 1355 cells/mm3). Daily quality control was conducted on all 3 analysers for the first 10 measurements when a new cartridge was used and over a period of 165 days (23 January – 25 March 2014). Accuracy and precision of the NHLS PLG testing was established in the NHLS laboratories by daily monitoring of instrument stability (Flow check TM, Beckman Coulter Miami, FL) and system performance verification using normal (394 cells/mm3 – 754 cells/mm3) and low (62 cells/mm3 – 206 cells/mm3) Immunotrol™ controls (Beckman Coulter, Miami, FL). The Addington NHLS laboratory participates in the NHLS proficiency testing panels and is accredited by the South African National Accreditation System.
Comparisons of CD4+ T-cell enumeration was undertaken between flow cytometry instruments (PIMA™ POC analysers and the predicate NHLS) on 268 blood samples. Due to transport logistics, the NHLS laboratory performs testing the day after the blood draw. Therefore, a subset of 100 blood samples were tested by the PIMA™ analyser, FACS Calibur and the NHLS to ensure that differences observed between the PIMA™ analyser versus NHLS were not due to CD4 testing performed on the next day in the NHLS laboratory. CD4+ T-cell enumeration using the FACS Calibur reference method
Prediction of the benefits of the PIMA™ POC CD4 testing in terms of ART eligibility and decision making was undertaken. Additionally, an assessment was undertaken to determine whether HIV-infected individuals return for their CD4+ test result and how many are lost to follow-up between ART eligibility and initiation.
The protocol was approved by the Biomedical Research Ethics Committee, University of KwaZulu-Natal (BE 212/11) and the eThekwini Research Ethics committee (28 November 2011). Written informed consent was obtained from patients > 18 years of age enrolled in the study.
It was determined that a sample size of 254 HIV-positive patients would be required to detect a difference of 15 cells/mm3 between the results of the PIMA™ POC analyser and the conventional test with 95% probability and 80% power assuming the standard deviation of difference in means is 85. In order to allow for potential problems with samples, the sample size was increased by 14 patients giving a sample size of 268.
Pairwise comparison of the PIMA™ analysers was conducted using
The percentage similarity (% SIM) model, Bland-Altman (BA) plots, limits of agreement (LOA) and Lin’s concordance correlation coefficient were used to assess agreement between PIMA™ analysers, FACS Calibur and NHLS.
To assess the diagnostic accuracy of CD4+ T-cell counts by the PIMA™ POC analysers in identifying ART eligibility, sensitivity, specificity, false-negative (FN) and false-positive (FP) rates, positive predictive value (PPV) and negative predictive value (NPV) were computed for the ART initiation thresholds of ≤ 200 cells/mm3, ≤ 350 cells/mm3 and ≤ 500 cells/mm3 CD4+ T-cells. All analyses were performed using STATA (Statacorp, College Station, TX, USA) statistical version 13.
There was high reproducibility and instrument precision (%CVs < 5%) within PIMA™ analysers 1, 2, 3 of the control cartridges over a replicate set of 10 bead analyses and over time (
System performance verification using normal (394 cells/ mm3 – 754 cells/mm3) and low (62 cells/mm3 – 206 cells/mm3) Immunotrol controls for the NHLS PLG testing was < 6%.
The majority (218/268) of HIV-1-positive individuals undergoing CD4+ T-cell count testing were women of whom 25% were 25–29 years, whereas the majority of the men were older than 30 years (
Characteristics of HIV-1-positive individuals undergoing CD4+ T-cell count enumeration.
Patient characteristics | Female |
Male |
Total |
||||||
---|---|---|---|---|---|---|---|---|---|
% | Range | % | Range | % | Range | ||||
Median age (IQR), years | 32 | - | 26–37 | 33 | - | 30–40 | 32 | - | 27–38 |
18–24 | 39 | 17.90 | - | 4 | 8 | - | 43 | 16.00 | - |
25–29 | 54 | 24.77 | - | 8 | 16 | - | 62 | 23.10 | - |
30–34 | 46 | 21.10 | - | 18 | 36 | - | 64 | 23.90 | - |
35–39 | 44 | 20.20 | - | 7 | 14 | - | 51 | 19.00 | - |
> 40 | 35 | 16.10 | - | 13 | 26 | - | 48 | 17.90 | - |
Median (IQR) NHLS CD4 count cells/mm3 | 292 | - | 184–453 | 254 | - | 151–387 | 286 | - | 176.5–444.5 |
Number (%) NHLS ≤ 350 cells/mm3 | 130 | 60 | - | 34 | 68 | - | 164 | 61.19 | - |
Number (%) NHLS ≤ 500 cells/mm3 | 176 | 81 | - | 42 | 82 | - | 218 | 81 | - |
Median (IQR) PIMA™ CD4 count cells/mm3 | 328 | - | 204–451 | 308 | - | 179–419 | 322 | - | 204–449 |
Number (%) PIMA™ ≤ 350 cells/mm3 | 114 | 52.30 | - | 31 | 62 | - | 145 | 54.10 | - |
Number (%) PIMA™ ≤ 500 cells/mm3 | 173 | 79 | - | 42 | 84 | - | 215 | 80 | - |
IQR, interquartile range; NHLS, National Health Laboratory Services.
Female,
Male,
Total,
In a subset of 100 samples, the highest agreement was observed between PIMA™ analysers and FACS Calibur as evidenced by smaller mean bias of 7.52 and narrower BA limits of agreement from -111 to 126 and a correlation of 0.97 (
Bland–Altman comparison of PIMA™ analysers versus National Health Laboratory Services versus FACS Calibur.
Measure of agreement | PIMA analysers – NHLS |
PIMA analysers – FACS Calibur |
FACS Calibur – NHLS |
---|---|---|---|
Mean bias (± 1 s.d.) | -12.78 ± 106.63 | 7.52 ± 59.26 | -20.3 ± 97.97 |
95% CI bias | -33.94–8.38 | -4.24–19.28 | -39.74 – -0.86 |
BA 95% LOA | -226.04–200.48 | -111.01–126.05 | -216.23–175.63 |
% Similarity to predicate (%SIM Mean ± s.d.) | 101.3 ± 15 | 103.1 ± 12.7 | 98.7 ± 11.9 |
NHLS, National Health Laboratory Services; BA, Bland-Altman; LOA, limits of agreement.
An overall correlation of 0.91 in CD4+ T-cell counts between the PIMA™ analysers and NHLS was observed (
Comparison of CD4+ T-cell counts obtained by the PIMA™ analysers and the National Health Laboratory Services in whole-blood samples.
Bland–Altman plot PIMA™ point-of-care analyser – National Health Laboratory Services versus the average of PIMA™ point-of-care analyser and National Health Laboratory Services.
Comparison of PIMA™ analysers versus National Health Laboratory Services as categorised by CD4+ T-cell counts.
Measure of agreement | ≤ 350 cells/mm3 |
351 cells/mm – 500 cells/mm3 |
≤ 500 cells/mm3 |
> 500 cells/mm3 |
All CD4+ T-cell counts |
---|---|---|---|---|---|
Median bias | 21 | 13 | 21 | -23 | 18 |
BA bias (± 1 s.d.) | 32.9 ± 61.0 | 21.5 ± 90.8 | 30.1 ± 69.4 | -36.1±150.04 | 17.5 ±93.8 |
95% CI bias | 23.4–42.3 | -3.5–46.6 | 20.8–39.4 | -78.3–6.1 | 6.2–28.8 |
BA 95% LOA | -89.2 –154.9 | -160.0–203.0 | -108.7–168.9 | -336.2–263.9 | -170.0–205.0 |
% Similarity to predicate (% SIM mean ± s.d.) | 107.4 ± 15.2 | 102.7 ± 10.5 | 106.2 ± 14.35 | 97.97 ± 10 | 106 ± 15.5 |
%SIM CV | 14.20% | 10.20% | 13.50% | 10.20% | 14.60% |
BA, Bland-Altman; LOA, limits of agreement.
Under previous SA ART guidelines of ≤ 200 cells/mm3 and ≤ 350 cells/mm3, the PIMA™ POC analysers displayed a sensitivity and specificity of 73.5%/98.4% and 83.5%/92.3%, respectively (
Performance of PIMA™ analysers compared to National Health Laboratory Services at different CD4+ T-cell thresholds.
CD4+ T-cells/mm3 | Sensitivity (%) | Specificity (%) | Number misclassified | Number misclassified (%) | Correctly classified (%) | FP Rate | FN Rate | Negative predictive value (%) | Positive predictive value (%) |
---|---|---|---|---|---|---|---|---|---|
≤ 200 | 73.50 | 98.40 | 25 | 9.3 | 90.7 | 3/25 | 22/25 | 85.20 | 95.30 |
≤ 350 | 83.50 | 92.30 | 35 | 13.0 | 87.0 | 8/35 | 27/35 | 78.10 | 94.50 |
≤ 500 | 94.00 | 78.40 | 24 | 9.0 | 91.0 | 11/24 | 13/24 | 75.50 | 94.90 |
Note: PIMA™ point-of-care [POC] testing for CD4 counts in predicting antiretroviral initiation in HIV-infected individuals in KwaZulu-Natal, Durban, South Africa.
FP, false-positive; FN, false-negative.
As the study was conducted during 2013, linkage to care data is presented according to the NHLS laboratory CD4 test result of ≤ 350 cells/mm3,
Comparison of CD4+ T-cell counts with respect to antiretroviral therapy eligibility by PIMA™ point-of-care analyser versus National Health Laboratory Services in those HIV-1-infected patients who returned and did not return for their results.
The median time to ART initiation from date of CD4 test was: 49 days (IQR 42–64 regardless of CD4+ T-cell count; 36–63 days in those with ≤ 200 cells/mm3).
Conventional flow cytometry to determine CD4 counts usually requires that samples be sent to a central laboratory, which may be off-site. Although the turn-around time for a CD4 test result by the NHLS is 24–72 hours, HIV-1-infected patients are counselled to return to the PHC within 1 week for receipt of these results. POC technologies can reduce these delays resulting in rapid linkage to care. This study demonstrated a high PPV and sensitivity and acceptable specificity in predicting ART eligibility (≤ 500 cells/mm3) using the PIMA™ POC analyser as compared to the NHLS CD4 test.
The majority of HIV-1-positive individuals undergoing CD4 testing were women, of whom 25% were 25–29 years old, whereas the majority of men were older than 30 years of age. There were no significant differences in the median CD4+ T-cell count in men versus women performed by the NHLS versus the PIMA™ POC analyser, although the median count was higher in the latter. Overall, according to NHLS versus PIMA™ POC, 81% versus 80% of individuals were eligible for ART initiation (≤ 500 cells/mm3), of whom 82% versus 84% were men and 81% versus 79% were women, respectively.
There was high reproducibility in all three PIMA™ POC analysers using normal and low beads with coefficient of variation < 5% over time (10 and 165 days). The PIMA™ POC analyser slightly overestimates NHLS flow cytometry in CD4+ T-cell enumeration in this study, which corroborates most studies using capillary or venous blood.
The adequate correlation between the PIMA™ POC analyser and FACS Calibur (0.97) corroborates similar findings in another study.
The overall sensitivity of the PIMA™ POC CD4 test in HIV-1-infected adults and pregnant mothers to determine their eligibility for ART has been reported at 96.3% in individuals with a CD4+ T-cell count of ≤ 250 cells/mm3,
From the operational perspective in the use of the PIMA™ POC analyser, similar to other studies using venous or capillary blood,
Under the standard SA HIV and AIDS guidelines operating at the time of the study, we observed that the median time for patients to return for their CD4 results was 8 days and 7 days in those with ≤ 200 cells/mm3, with a median of 49 days regardless of CD4+ T-cell count from CD4 testing to ART initiation. The use of the PIMA™ POC analysers could facilitate the fast tracking of patients with CD4+ T-cell count ≤ 200 cells/mm3 onto ART within 7 days. In this study, the provision of immediate CD4 test results to patients would have prevented the 35/268 not having access to their results (through them not returning), in whom over half (57%) were eligible for ART.
The high rates (61%) of ‘walk–in’ patients found in this study who were eligible (≤ 350 cells/mm3) for immediate ART at the time of the HIV test, half of whom had CD4+ T-cell counts ≤ 248 cells/mm3, and the time lapse to ART initiation undergirds the urgent need for the use of the rapid PIMA™ POC technology. At a threshold of ≤ 500 cell/mm3, 75% of patients had a median CD4+ T-cell count of 444 cells/mm3 at the time of the HIV test. A recent study reported that providing same-day POC CD4 testing that is not rapid has no benefit in health outcomes.
In summary, the overall agreement between PIMA™ POC analyser and NHLS CD4+ T-cell count enumeration in adult HIV-1-positive individuals was acceptable with clinically insignificant mean bias. Together with high PPV and sensitivity and acceptable specificity, the PIMA™ POC CD4 test has the potential role for CD4+ T-cell enumeration in PHC settings and lends itself to be an excellent facilitator in rapid linkage to care in ART programmes, particularly that it has been demonstrated in simulated cohort models of HIV-1-infected adults and pregnant women, to result in not only better clinical outcomes but also to cost savings in the long term.
We acknowledge the input of the Thembi Ngubane and Thabisile Maluleka, the phlebotomists on the study, and are indebted to the participants themselves for being part of this project. We thank especially the Lancers Road clinic staff for their co-operation and the eThekwini Municipality Health Unit for permission to conduct the study in their facility. The Medical Research Council HIV Prevention Research Unit for infrastructural support for the study.
The authors declare that they have no financial or personal relationships which may have inappropriately influenced them in writing this article.
P.K. and A.C. designed the study and provided overall supervision for the study. S.N. recruited patients and obtained informed consents. E.S. collected data from study subjects and N.N. provided clinical oversight. M.S. and S.R. conducted laboratory analyses. T.R. conducted the statistical analyses and M.S., P.K. and A.C. undertook data interpretation. All authors contributed to the writing of the manuscript and approved the final version.