Re: accuracy on a matrix

[Jaroslav Hajek]

On Tue, Jun 2, 2009 at 9:49 PM, Carlo Rossi <address@hidden> wrote:
>
>> > Hello,
>> > it isn't  obvious because implementing it (but with
>> Matlab in this two way:
>> >
>> > classification = knnclassify(TEST, TRAIN, GROUP, 1);
>> > [C, order] = confusionmat(TARGET, classification);
>> > cp = classperf(TARGET, Kclassification);
>> > acc1 =
>> (cp.Sensitivity*cp.Prevalence)cp.Specificity*(1-cp.Prevalence)
>> > acc2 = sum(diag( C )) / sum( C(:) )
>> >
>> > According to here I should return the same accuracy:
>> > http://en.wikipedia.org/wiki/Accuracy_and_precision
>> >
>> > But they are diffent! So for this reason I asked If
>>  I were using the right formula. Does anyone have
>> experience with this stuff?
>> > I need to understand why the are different
>> > thanks,
>> >
>>
>> The acc1 formula above is only valid for binary
>> classification and
>> requires splitting the class labels into positive and
>> negative. See
>> the documentation of classperf, "PositiveValue" and
>> "NegativeValue".
>
> sorry you finished into my antispam! Then, only acc1 formula is right for 
> only binary or even the acc2. Actually I need one that can work on multiple 
> classes...
> thanksm
>

Accuracy is defined as the ratio of (correctly classified cases) /
(all cases), which is naturally extended to multiple classes and given
by the simple formula acc2.
However, given the definitions of Sensitivity, Prevalence and
Specificity, I don't see a similarly natural way to extend them to
multiple classes. Matlab apparently does neither; instead it
partitions the class labels into two groups and then applies the
binary formulas.


-- 
RNDr. Jaroslav Hajek
computing expert & GNU Octave developer
Aeronautical Research and Test Institute (VZLU)
Prague, Czech Republic
url: www.highegg.matfyz.cz