WRIGHTET
AL.:ROBUSTFACERECOGNITIONVIASPARSEREPRESENTATION225
Fig.14.ROCcurvesforoutlierrejection.Verticalaxis:truepositiverate.Horizontalaxis:falsepositiverate.ThesolidredcurveisgeneratedbySRCwithoutliersrejectedbasedon(15).TheSCI-basedvalidationandSRCclassificationtogetherperformalmostperfectlyforupto30percentocclusion.(a)Noocclusion.(b)
Tenpercentocclusion.(c)Thirtypercent.(d)Fiftypercent.
Fig.15.Robusttrainingsetdesign.(a)Varyingillumination.Topleft:foursubsetsofExtendedYaleB,containingincreasinglyextremelightingconditions.Bottomleft:estimatedneighborlinessofthepolytopeconvðÆBÞforeachsubset.(b)Varyingexpression.Topright:fourfacialexpressionsintheARdatabase.Bottomright:estimatedneighborlinessofconvðÆBÞwhentakingthetrainingsetfromdifferentpairsofexpressions.
sincetheoccludingobjectcouldpotentiallybequitesimilartooneoftheothertrainingclasses.However,iftheocclusionisrandomanduncorrelatedwiththetrainingimages,asinSection4.3,theaveragebehaviormayalsobeofinterest.Infact,thesetwoconcerns,sufficiencyandrobustness,arecomplementary.Fig.15ashowstheestimatedneighbor-linessforthefoursubsetsoftheExtendedYaleBdatabase.Noticethatthehighestneighborliness,%1,330,isachievedwithSubset4,themostextremelightingconditions.Fig.15bshowsthebreakdownpointforsubsetsoftheARdatabasewithdifferentfacialexpressions.Thedatasetcontainsfourfacialexpressions,Neutral,Happy,Angry,andScream,picturedinFig.15b.Wegeneratetrainingsetsfromallpairsofexpressionsandcomputetheneighborlinessofeachofthecorrespondingpolytopes.ThemostrobusttrainingsetsareachievedbytheNeutralþHappyandHappyþScreamcombinations,whiletheleastrobustnesscomesfromNeutralþAngry.NoticethattheNeutralandAngryimagesarequitesimilarinappearance,while(forexample)HappyandScreamareverydissimilar.
Thus,bothforvaryinglighting(Fig.15a)andexpression(Fig.15b),trainingsetswithwidervariationintheimagesallowgreaterrobustnesstoocclusion.Designingatrainingsetthatallowsrecognitionunderwidelyvaryingconditionsdoesnothinderouralgorithm;infact,ithelpsit.However,thetrainingsetshouldnotcontaintoomanysimilarimages,asintheNeutral+AngryexampleinFig.15b.Inthelanguageofsignalrepresentation,thetrainingimagesshouldformanincoherentdictionary[9].
offeaturesused(inourfacerecognitionexample,approxi-mately100aresufficienttomakethedifferencenegligible).Moreover,occlusionandcorruptioncanbehandleduniformlyandrobustlywithinthesameclassificationframework.Onecanachieveastrikingrecognitionperfor-manceforseverelyoccludedorcorruptedimagesbyasimplealgorithmwithnospecialengineering.
Anintriguingquestionforfutureworkiswhetherthisframeworkcanbeusefulforobjectdetection,inadditiontorecognition.Theusefulnessofsparsityindetectionhasbeennoticedintheworkin[61]andmorerecentlyexploredin[62].Webelievethatthefullpotentialofsparsityinrobustobjectdetectionandrecognitiontogetherisyettobeuncovered.Fromapracticalstandpoint,itwouldalsobeusefultoextendthealgorithmtolessconstrainedcondi-tions,especiallyvariationsinobjectpose.Robustnesstoocclusionallowsthealgorithmtotoleratesmallposevariationormisalignment.Furthermore,inthesupplemen-taryappendix,whichcanbefoundontheComputerSocietyDigitalLibraryathttp://www.77cn.com.cn/10.1109/TPAMI.2008.79,wediscussouralgorithm’sabilitytoadapttononlineartrainingdistributions.However,thenumberoftrainingsamplesrequiredtodirectlyrepresentthedistributionoffaceimagesundervaryingposemaybeprohibitivelylarge.Extrapolationinpose,e.g.,usingonlyfrontaltrainingimages,willrequireintegratingfeaturematchingtechniquesornonlineardeformationmodelsintothecomputationofthesparserepresentationofthetestimage.Doingso,inaprincipledmanner,itremainsanimportantdirectionforfuturework.
5CONCLUSIONS
AND
DISCUSSIONS
Inthispaper,wehavecontendedboththeoreticallyandexperimentallythatexploitingsparsityiscriticalforthehigh-performanceclassificationofhigh-dimensionaldatasuchasfaceimages.Withsparsityproperlyharnessed,thechoiceoffeaturesbecomeslessimportantthanthenumber
ACKNOWLEDGMENTS
TheauthorswouldliketothankDr.HarryShum,Dr.XiaoouTangandmanyothersattheMicrosoftResearch,Asia,forhelpfulandinformativediscussionsonfacerecognition,duringtheirvisitthereinFall2006.Theyalso