Clinical test performance

October 7, 2009 by Giuseppe Cardillo · 2 Comments
Filed under: Biotecnology, Genetic

Rating: 0 (from 0 votes)

Rating: 9.0/10 (2 votes cast)

Clinical test performance

This function was written to compute some benchmarks for a clinical test, but it can be used in all dichotomous test.

It was selected for the publication into two books:

S. Harisha – Biotechnology procedures and experiments handbook – Infinity Science Press – Hingham, MA, USA – ISBN 978-1-934015-11-7

A.J. Nair – Introduction to Biotechnology and genetic engineering – Infinity Science Press – Hingham, MA, USA – ISBN 978-1-934015-16-2

For a clinical test, you have two conditions (healthy and sick) and a dichotomous test (positive or negative). For an ideal, perfect test, it is positive for all sick and for none healthy subjects and negative for all healthy and none sick. But we live in a real world and so, the situation can be summarized by a 2×2 matrix (call it X)

	Sick (D+)	Healthy (D-)	Row sum
Test Positive (T+)	True Positive	False Positive	R1
Test Negative (T-)	False Negative	True Negative	R2
Column sum	C1	C2	total = N

We define:

cs=sum(X);
rs=sum(X,2);
N=sum(X(:));
d=diag(X);

Prevalence of disease

The prevalence of disease is defined: D+ over N = (True Positive + False Negative)/N = C1/N

1	pr=cs(1)/N;

Sensitivity and Specificity

The Sensitivity is the probability that the test is positive on sick subjects: P(T+|D+) = True Positive over C1 = TP/C1
The Specificity is the probability that the test is negative on healthy subjects: P(T-|D-) = True Negative over C2 = TN/C2
In Matlab both parameters are obtained with only one instruction:

1	SS=d./cs';

Of course, the false proportions will be

1	fp=(1-SS);

The 95% confidence interval critical values for Sensitivity and Specificity are computed in this way:

1	acv=1.96sqrt(SS.fp./cs');

Of course, the critical interval lower bound cannot be less than 0 and the upper bound cant be greater than 1 and so:

CI=max(0,sensivity-aci) – min(1,sensivity+aci)

Youden’s index (J)

Youden’s J statistics (also called Youden’s index) is a single statistic that captures the performance of a diagnostic test.
The use of such a single index is “not generally to be recommended”. It is equal to the risk difference for a dichotomous test and it defined as:
J = Sensitivity + Specificity – 1

1	J=sum(SS)-1;

A perfect test has J=1

Accuracy and Mis-classification rate

The Accuracy (or Power) is the probability that the test correctly classifies the subjects and is defined: Acc=(TP+TN)/N

1	acc=trace(X)/N;

The Mis-classification rate is the complement to 1:

1	mcr=1-acc;

Positive and Negative predictivity

The Positive predictivity is the probability that the subjects is sick when the test is positive: P(D+|T+) = True Positive over R1 = TP/R1
The Negative predictivity is the probability that the subjects is Healthy when the test is negative: P(D-|T-) = True Negative over R2 = TN/R2
In Matlab both parameters are obtained with only one instruction:

1	PNp=d./rs;

The 95% confidence interval critical values for positive and negative predictivity are computed in this way:

1	PNpcv=1.96sqrt(PNp.(1-PNp)./rs);

As for sensitivity and specificity, the critical interval lower bound cannot be less than 0 and the upper bound cant be greater than 1.

Positive and Negative Likelihood Ratio

When we decide to order a diagnostic test, we want to know which test (or tests) will best help us rule-in or rule-out disease in our patient. In the language of clinical epidemiology, we take our initial assessment of the likelihood of disease (“pre-test probability”), do a test to help us shift our suspicion one way or the other, and then determine a final assessment of the likelihood of disease (“post-test probability”).

Likelihood ratios tell us how much we should shift our suspicion for a particular test result. Because tests can be positive or negative, there are at least two likelihood ratios for each test. The “positive likelihood ratio” (LR+) tells us how much to increase the probability of disease if the test is positive, while the “negative likelihood ratio” (LR-) tells us how much to decrease it if the test is negative.

The LR+ and LR- are defined in terms of sensitivity and specificity: LR+ = sensitivity / (1-specificity); LR- = (1-sensitivity) / specificity.
Considering the previous defined arrays SS and fp

1 2	PLR=SS(1)/fp(2); NLR=fp(1)/SS(2);

Error and Diagnostic Odds Ratio

Error Odds Ratio indicates if the probability of being wrongly classified is highest in the diseased or in the non-diseased group.
If the error odds is higher than one the probability is highest in the diseased group (and the specificity of the test is better than the sensitivity),
if the value is lower than one the probability of an incorrect classification is highest in the non-diseased group (and the sensitivity of the test is better than the specificity).
EOR is defined as (Sensitivity/(1-Sensitivity))/(Specificity/(1-Specificity));

Diagnostic Odds Ratio is often used as a measure of the discriminative power of the test. Has the value one if the test does not discriminate between diseased and not diseased.
Very high values above one means that a test discriminates well. Values lower than one mean that there is something wrong in the application of the test.
DOR is defined as (Sensitivity/(1-Sensitivity))/((1-Specificity)/Specificity);

1 2	EOR=(SS(1)/fp(1))/(SS(2)/fp(2)); DOR=(SS(1)/fp(1))/(fp(2)/SS(2));

Discriminant Power

The discriminant power for a test, also termed the test effectiveness, is a measure of how well a test distinguishes between affected and unaffected persons. It is the sum of logs of Sensivity and Specificity over own false proportion, scaled by the standard deviation of the logistic normal distribution curve (square root of 3 divided by π). Test effectiveness is interpreted as the standardized distance between the means for both populations.

1	dpwr=(realsqrt(3)/pi)*sum(log(SS./fp));

A test with a discriminant value of 1 is not effective in discriminating between affected and unaffected individuals.
A test with a discriminant value of 3 is effective in discriminating between affected and unaffected individuals.

Test Bias

A test which shows provable and systematic differences in the results of people based on group membership.
For example, a test might be considered biased if members of one particular gender or race consistently and systematic have statistically different results from the rest of the testing population.
It is defined as (T+)/(D+)=(TP+FP)/(TP+FN)

A perfect test has a TB=1;
if TB<1 the test underestimates the disease because there are more affected than positive test;
if TB>1 the test overestimates the disease because there are more positive test than affected

1	TB=rs(1)/cs(1);

If any problems occurs in execution, or if you found a bug, have a suggestion or question just contact me at:
giuseppe dot cardillo-edta at poste dot it

You can visit my homepage http://home.tele2.it/cardillo

My profile on XING http://www.xing.com/go/invita/13675097

My profile on LinkedIN http://it.linkedin.com/in/giuseppecardillo