A 201.4 GOPS real-time multi-object recognitionprocessor is presented with a three-stage pipelined architecture.Visual perception based multi-object recognition algorithm isapplied to give multiple attentions to multiple objects in the inputimage. For human-like multi-object perception, a neural perceptionengine is proposed with biologically inspired neural networksand fuzzy logic circ
34IEEEJOURNALOFSOLID-STATECIRCUITS,VOL.45,NO.1,JANUARY
2010
Fig.2.Three-stagemulti-objectrecognitionalgorithm.
dedicatedvectordistancecalculationunitsareemployedintheobjectdecisionstage.
Overall,theproposedalgorithmemploysgrid-basedROIpro-cessingthatdividestheinputimageintoanumberoftwo-di-mensional(2-D)grid-tilesandperformstheprocessingbasedonthem.Itenables ne-grainedROIextractionofmultipleob-jectsandreducestheeffectiveprocessingareaofinputimages.Toevaluatetheproposedalgorithm,weperformexperimentswith50of ceobjectsoutofColumbiaobjectimagelibrary(Coil-100)[10].Itisappliedto2400sampleimagesthatincluderandomobjectsinnaturalbackgroundscenes,witha16384-entrydatabasemadebytheSIFT.Asaresult,overallrecog-nitionratebytheproposedalgorithmismeasuredas92%.ForevaluationsoftheROIdetectionbyvisualperception,truepos-itiveratethatrepresentstheratioofcorrectlydetectedregionoutofgroundtruthROIandfalsepositiveratethatrepresentstheratioofincorrectlydetectedregionoutofnotinterestedre-gion[11]aremeasuredas70%and5%,respectively.Thevisualperceptionbarelyaffectstheoverallrecognitionratewhilere-ducingtheprocessingareaoftheimagesto32.8%onaverage.
III.SYSTEMARCHITECTURE
Fig.3showstheoverallblockdiagramoftheproposedpro-cessor.Itconsistsof21IPblocks:aneuralperceptionengine(NPE),aSPUtask/powermanager(STM),16SIMDprocessorunits(SPUs),adecisionprocessor(DP),andtwoexternalmemoryinterfaces.TheNPEisresponsibleforthe rstvisualperceptionstage.ItextractstheROIgrid-tilesforeachobjectandsendsthemto16SPUsfordetailedimageprocessing.The16SPUs,whosepowerdomainisseparatedintofour,areresponsiblefortheseconddescriptorgenerationstage.TheyextractobjectfeaturesoutoftheselectedROIsandconvertthemtodescriptorvectors.Thedescriptorvectorsoutofthe16SPUsaregatheredattheDP.TheDPacceleratesthevectormatchingprocessofdescriptorvectorsforthethirdobjectdecisionstage.TheSTMisspeciallydevisedtodistributethetasksoftheROIgrid-tilesfromtheNPEtothe16SPUsandtomanagethem.Italsocontrolsthepipelinestagesoftheoverallprocessorandmanagesfourpowerdomainsof16SPUs.Theoverall21IPblocksareinterconnectedthroughtheproposedmulti-castingNoC.
Toincreaseparallelismandhardwareutilizationofthepro-posedprocessor,theproposedthreestagesareexecutedinthepipelineinframelevelasshowninFig.4.ThepipelineddataareROIgrid-tilesanddescriptorvectorsbetweenthe rst–secondstageandsecond–thirdstage,respectively.Unliketheexecu-tiontimeofthe rstvisualperceptionstageisconstantduetoits xedcomputationamount,theexecutiontimeofthesecondde-scriptorgenerationandthirdobjectdecisionarevaryingwiththenumberofROIgrid-tilesanddescriptorvectors.Inordertobal-ancetheexecutiontimesofthreestages,theSTMestimatestheworkloadofthefollowingdescriptorvectorandobjectdecisionstagebasedonthenumberofextractedROIgrid-tilesandde-scriptorvectors,respectively,andcontrolstheirexecutiontimesusingtwopipelinetimebalancingschemes.
Tocontroltheexecutiontimeofthedescriptorgenerationstage,theSTMperformsworkload-awaretaskscheduling(WATS)thatdiffersthenumberofschedulingSPUsaccordingtothestage’sinputworkload.Fig.5(a)showsthe owchartoftheWATS.First,theSTMmeasuresthenumberofROIgrid-tilesfromtheNPEandclassi esittooneofNwork-loadlevelsdividedbyN-1thresholdvalues.Andthen,theSTMdeterminesthenumberofoperatingSPUsaccordingtotheclassi edworkloadlevel.SinceitallocatestheSPUsin