ICBA competition - CSU results

The CSU Face Identification Evaluation System (http://www.cs.colostate.edu/evalfacerec/index.html) was tested  on the BANCA database using protocol MC to provide the baseline error measurements. This software package provides the implementation of  standard face recognition algorithms. It includes standardized image pre-processing software written in ANSI C, four distinct face recognition algorithms and analysis software to study algorithm performance.

The algorithms included are:

Principle Component Analysis (PCA)
Combined Principle Component Analysis  and Linear Discriminant Analysis (PCA+LDA)
Bayesian Intrapersonal/Extrapersonal Classifier (BIC) using maximum a posteriori (MAP) and  maximum likelihood (ML)
Elastic Bunch Graph Matching algorithm (EBGM)

PCA performs a projection of the feature vectors onto a basis and then it computes a distance between pair of images using choice of distance measures. These measure include CityBlock (L1), Euclidean (L2), Correlation, Covariance, Mahalanobis L1, Mahalanobis L2 amd Mahalanobis Cosine. PCA+LDA uses Fisher's Linear Discriminants on PCA-projected feature vectors. This method  offers the same distance measures as for the PCA and one additional distance measure (LDASoft). The BIC algorithm projects the feature vector onto extrapersonal and intrapersonal subspaces and computes the probability that each feature vector came from one or the other subspace. EBGM algorithm uses face graphs formed out of Gabor jets extracted from landmarks on the face. Face graphs serve a low dimensional representation of image data. When the face graphs of a probe image and template are determined, a similairity between the face graphs is returned on the output. Three different method for measuring similarity of bunchgraphs were tested: FGMagnitude, FGNarrowingLocalSearch and FGPredictiveStep. For more details about the algorithms and distance measures please refer to the system user's guide:
http://www.cs.colostate.edu/evalfacerec/algorithms/version5/faceIdUsersGuide.pdf

For verification setup on the BANCA database  the disimilarity score that the software  produced was used for a pair of images (all measures are normalized to become disimilarities). With regard to parameter setting we used the  default values offered by the system.  All algorithms were evaluated using a global thresholding (GT) and PCA+LDA and BIC also using a client-specific thresholding (CST). It should also  be noted that EBGM needed a special image pre-normalization, which actually slightly differed from the normalization required in the competition. However as the resolutions were similar we believe that the results are still valuable for comparison.  Selected results are presented in  tables below.

Global threshold (GT)

 Client-specific threshold (CST)

  Group 1 Group 2
  FAR FRR WER FAR FRR WER
R=0.1 44.23 6.41 9.848 79.42 1.282 8.386
R=1 24.42 13.33 18.88 17.69 12.82 15.26
R=10 3.654 45.13 7.424 0 62.31 5.664

BIC:MAP+GT

  Group 1 Group 2
  FAR FRR WER FAR FRR WER
R=0.1 40.19 6.923 9.948 62.88 1.282 6.882
R=1 25.38 12.05 18.72 17.12 10 13.56
R=10 2.885 47.44 6.935 0 63.08 5.734

BIC:ML+GT

  Group 1 Group 2
  FAR FRR WER FAR FRR WER
R=0.1 35.58 5.641 8.362 38.27 0.5128 3.945
R=1 20.77 14.36 17.56 2.308 25.38 13.85
R=10 0.7692 45.13 4.802 0.1923 44.36 4.207

EBGM:FGMagnitude+GT

  Group 1 Group 2
  FAR FRR WER FAR FRR WER
R=0.1 48.65 5.385 9.318 84.04 0 7.64
R=1 13.85 18.72 16.28 9.808 12.05 10.93
R=10 1.346 40.26 4.883 0.5769 43.08 4.441

EBGM:FGNarrowingLocalSearch+GT

  Group 1 Group 2
  FAR FRR WER FAR FRR WER
R=0.1 74.62 3.077 9.58 92.88 0 8.444
R=1 15.19 25.9 20.54 4.231 33.33 18.78
R=10 0.9615 50.51 5.466 1.538 47.18 5.688

EBGM:FGPredictiveStep+GT

  Group 1 Group 2
  FAR FRR WER FAR FRR WER
R=0.1 90.58 0.7692 8.934 81.35 0.7692 8.094
R=1 26.92 24.62 25.77 27.12 21.03 24.07
R=10 1.538 70.77 7.832 1.154 65.9 7.04

PCA+LDA:CityBlock (L1)+GT

  Group 1 Group 2
  FAR FRR WER FAR FRR WER
R=0.1 32.5 10.77 12.74 85.58 1.026 8.712
R=1 13.46 18.97 16.22 7.308 11.28 9.295
R=10 1.923 42.31 5.594 0.5769 42.82 4.417

PCA+LDA:Correlation+GT

  Group 1 Group 2
  FAR FRR WER FAR FRR WER
R=0.1 32.88 10.77 12.78 85.38 1.282 8.928
R=1 13.46 19.23 16.35 9.615 8.718 9.167
R=10 1.923 42.82 5.641 0.5769 42.82 4.417

PCA+LDA:Covariance+GT 

  Group 1 Group 2
  FAR FRR WER FAR FRR WER
R=0.1 87.12 1.282 9.085 85.96 0 7.815
R=1 26.35 24.1 25.22 10 37.69 23.85
R=10 1.538 68.72 7.646 0.7692 68.21 6.9

PCA+LDA:Euclidean (L2)+GT

  Group 1 Group 2
  FAR FRR WER FAR FRR WER
R=0.1 84.62 1.538 9.091 85 0 7.727
R=1 26.92 19.49 23.21 12.31 31.54 21.92
R=10 1.346 65.13 7.145 0.3846 67.95 6.527

PCA+LDA:LDASoft+GT

  Group 1 Group 2
  FAR FRR WER FAR FRR WER
R=0.1 35.19 9.744 12.06 92.12 0 8.374
R=1 10.19 24.62 17.4 6.923 14.1 10.51
R=10 4.808 32.82 7.354 0 51.54 4.685

PCA+LDA:Mahanalobis Cosine+GT

  Group 1 Group 2
  FAR FRR WER FAR FRR WER
R=0.1 97.88 0.2564 9.132 89.23 1.282 9.277
R=1 26.73 29.49 28.11 6.346 51.79 29.07
R=10 1.731 73.33 8.24 0 84.36 7.669

PCA+LDA:Mahanalobis L1+GT

  Group 1 Group 2
  FAR FRR WER FAR FRR WER
R=0.1 100 0 9.091 97.31 0.2564 9.079
R=1 28.46 29.23 28.85 27.88 27.18 27.53
R=10 1.923 74.1 8.485 0 77.69 7.063

PCA+LDA:Mahanalobis L2+GT

  Group 1 Group 2
  FAR FRR WER FAR FRR WER
R=0.1 80.58 2.051 9.19 75.19 1.026 7.768
R=1 24.04 19.49 21.76 18.85 18.72 18.78
R=10 3.462 54.1 8.065 0 73.33 6.667

PCA:CityBlock (L1)+GT 

  Group 1 Group 2
  FAR FRR WER FAR FRR WER
R=0.1 56.35 3.59 8.386 79.62 2.564 9.569
R=1 15.77 27.69 21.73 2.692 46.15 24.42
R=10 2.308 49.23 6.573 0.5769 57.95 5.793

PCA:Correlation+GT

  Group 1 Group 2
  FAR FRR WER FAR FRR WER
R=0.1 56.35 3.59 8.386 79.62 2.564 9.569
R=1 15.77 27.69 21.73 2.692 46.15 24.42
R=10 2.308 49.23 6.573 0.5769 57.95 5.793

PCA:Covariance+GT

  Group 1 Group 2
  FAR FRR WER FAR FRR WER
R=0.1 46.73 13.08 16.14 87.5 1.282 9.12
R=1 14.62 25.64 20.13 6.731 26.15 16.44
R=10 5.769 37.69 8.671 0 69.23 6.294

PCA:Euclidean (L2)+GT

  Group 1 Group 2
  FAR FRR WER FAR FRR WER
R=0.1 30.58 13.08 14.67 82.12 0 7.465
R=1 13.27 20 16.63 1.923 17.18 9.551
R=10 2.308 35.9 5.361 0 50.77 4.615

PCA:Mahanalobis Cosine+GT

  Group 1 Group 2
  FAR FRR WER FAR FRR WER
R=0.1 100 0.5128 9.557 98.08 0 8.916
R=1 34.81 23.85 29.33 25.58 30.77 28.17
R=10 4.038 67.44 9.802 0.1923 78.97 7.354

PCA: Mahanalobis L1+GT

  Group 1 Group 2
  FAR FRR WER FAR FRR WER
R=0.1 100 0.2564 9.324 97.5 0 8.864
R=1 20.38 41.03 30.71 25.38 33.08 29.23
R=10 1.154 81.79 8.485 0 96.15 8.741

PCA:Mahanalobis L2+GT



  Group 1 Group 2
  FAR FRR WER FAR FRR WER
R=0.1 16.73 11.54 12.01 17.12 5.385 6.451
R=1 6.731 26.92 16.83 5.962 18.46 12.21
R=10 1.923 43.85 5.734 1.923 38.21 5.221

BIC:MAP+CST

  Group 1 Group 2
  FAR FRR WER FAR FRR WER
R=0.1 19.23 11.03 11.77 16.73 6.154 7.115
R=1 6.346 24.62 15.48 7.692 15.13 11.41
R=10 3.077 40.77 6.503 2.308 32.56 5.058

BIC:ML+CST
 
  Group 1 Group 2
  FAR FRR WER FAR FRR WER
R=0.1 55.58 5.128 9.714 56.15 3.333 8.135
R=1 11.54 34.36 22.95 8.846 28.72 18.78
R=10 2.885 51.54 7.308 1.538 52.56 6.177

PCA+LDA:CityBlock (L1)+CST

  Group 1 Group 2
  FAR FRR WER FAR FRR WER
R=0.1 23.85 9.231 10.56 25.58 3.077 5.122
R=1 5.962 14.62 10.29 4.423 5.641 5.032
R=10 2.5 26.41 4.674 1.154 19.23 2.797

PCA+LDA:Correlation+CST

  Group 1 Group 2
  FAR FRR WER FAR FRR WER
R=0.1 27.88 8.462 10.23 31.92 2.051 4.767
R=1 5.962 14.62 10.29 4.615 5.385 5
R=10 2.5 27.69 4.79 0.7692 19.23 2.448

PCA+LDA:Covariance+CST

  Group 1 Group 2
  FAR FRR WER FAR FRR WER
R=0.1 55 3.59 8.263 51.73 4.615 8.899
R=1 10.58 33.85 22.21 8.077 29.74 18.91
R=10 2.308 55.13 7.11 1.731 54.62 6.538

PCA+LDA:Euclidean (L2)+CST

  Group 1 Group 2
  FAR FRR WER FAR FRR WER
R=0.1 53.85 4.103 8.625 50 4.615 8.741
R=1 10 31.28 20.64 8.654 27.18 17.92
R=10 2.308 50.77 6.713 2.115 49.74 6.445

PCA+LDA:LDASoft+CST

  Group 1 Group 2
  FAR FRR WER FAR FRR WER
R=0.1 37.88 8.718 11.37 41.92 2.564 6.142
R=1 9.423 20 14.71 6.538 15.13 10.83
R=10 2.885 32.56 5.583 0.3846 33.08 3.357

PCA+LDA:Mahalanobis Cosine+CST

  Group 1 Group 2
  FAR FRR WER FAR FRR WER
R=0.1 67.5 4.615 10.33 65 2.821 8.473
R=1 14.23 34.36 24.29 12.31 32.05 22.18
R=10 2.5 64.87 8.17 1.154 61.28 6.62

PCA+LDA:Mahalanobis L1+CST

  Group 1 Group 2
  FAR FRR WER FAR FRR WER
R=0.1 68.46 5.897 11.59 67.12 4.615 10.3
R=1 16.92 37.18 27.05 12.31 35.64 23.97
R=10 2.692 69.74 8.788 0.5769 65.13 6.445

PCA+LDA:Mahanalobis L2+CST