A 201.4 GOPS real-time multi-object recognitionprocessor is presented with a three-stage pipelined architecture.Visual perception based multi-object recognition algorithm isapplied to give multiple attentions to multiple objects in the inputimage. For human-like multi-object perception, a neural perceptionengine is proposed with biologically inspired neural networksand fuzzy logic circ
KIMetal.:A201.4GOPS496mWREAL-TIMEMULTI-OBJECTRECOGNITIONPROCESSORWITHBIO-INSPIREDNEURALPERCEPTIONENGINE
37
Fig.7.Detailedvisualperceptionalgorithm.
staticfeaturessuchasintensity,color,andorientationandgen-eratethesaliencymapthatcombinestheextractedfeaturemapsthroughrepeatednormalizations.TheODEisproposedtoper-formthe nalROIclassi cationforeachobjectusingthegener-atedsaliencymap.TheRISCcontrollertakesaroleincontrol-lingthethreededicatedenginesandperformingsoftwareori-entedoperationsbetweenthededicatedoperationsoftheen-gines.A24KBmemoryisusedforstoringoriginalimagesanddatacommunicationamongthethreeenginesbysharinginter-mediateprocessingdata.Afterthe nalROIclassi cation,theNPEtransfersinformationoftheobtainedROIgrid-tilestotheSTMthroughaFIFOqueue.
Fig.7showsthedetailedvisualperceptionalgorithmoper-atedbytheNPE,whichbroadlyconsistofsaliencymapgen-erationandROIclassi cation.ThesaliencymapgenerationismainlybasedonItti’ssaliencybasedvisualattention[8]andac-celeratedbytheVAE.First,theRGBchannelsofVGAsized
60pixelsandaninten-inputimagearedown-sizedto
80
sityfeaturemapandtwocolorfeaturemapsaregeneratedbyper-pixel lteringoperations.Fourorientationfeaturemaps,forthedirectionof0,45,90,and135,aregeneratedfromthein-tensityfeaturemapwiththeGabor ltering.Aftergeneratingmulti-scaleGaussianpyramidimagesforeachof7maps,eachimageistransformedbyacenter-surroundmechanismtoen-hancethepartsoftheimagethatdifferfromtheirsurround-ings.Finally,thesaliencymapisgeneratedbyrepeatedcom-binationofnormalizedfeaturemaps.Themotionvectormap,generatedbytheME,isalsocombinedinthisstep.Amongtheseprocesses,computationallyintensiveimage lteringop-erationssuchasGabor,Gaussian,andcenter-surround lteringareacceleratedbythehardwareacceleratorVAE.Thenormal-izationprocesses,whichincludeirregularoperationsandcanbeperformedindifferentways,areperformedbysoftwarebytheRISCcontroller.Aftersaliencymapgeneration,ROIclassi ca-tionisperformedbytheODE.First,the10mostsalientpointsareselectedastheseedpointsoutofthesaliencymap.Then,fromthemostsalientseedpoint,theROIofanobjectgrowsfromneighborpixelsoftheseedthroughrepeatedhomogeneityclassi cations.Fortheclassi cationofeachpixel,anintensity,saliency,andlocationareusedforhomogeneityevaluation.Thesimilaritiesbetweentheseedandtargetpixelaremeasuredforabovethreemetrics,andbasedonthesummatedresult,the nalclassi cationthatthetargetpixelisdeterminedtobejoinedtotheROIornotisdetermined.Incasethattheotherseedpointsareincludedbythegrownregion,theyareinhibitedfromtheseedpointsinthenextROIclassi cation.Afterrepeatingclas-si cationprocessesfor10seedpoints,theROIofeachobjectinpixelunitisquantizedtothesmallsizedgrid-tileunit.
InthedesignoftheVAEandODE,biologicallyinspiredcel-lularneuralnetworksandneuro-fuzzyclassi erareemployedforfastfeatureextractionandrobustclassi cation,respectively.IntheVAE,2-Dcellularneuralnetworksareusedtorapidlyex-tractvariousfeaturesfromtheinputimageusingitsregionalandcollectiveprocessing[7].Fig.8showsoverallblockdi-agram,circuits,andmeasuredwaveformsoftheODE.Item-ploysGaussianfuzzymembershipandsingle-layerneuralnet-