I am deeply indebted to Dr. Michael Berry, my major advisor, for his kind guidance and support. I also thank Dr. Susan Dumais, director of the Information Sciences Research Group at Bellcore, for her technical advice. In addition, she graciously allowed us
Abstract
Astheamountofelectronicinformationincreases,traditionallexical(orBoolean)http://www.77cn.com.cnrge,heterogeneouscol-lectionswillbedif culttosearchsincethesheervolumeofunrankeddocumentsreturnedinresponsetoaquerywilloverwhelmtheuser.Vector-spaceapproachestoinformationretrieval,ontheotherhand,allowtheusertosearchforconceptsratherthanspeci cwordsandranktheresultsofthesearchaccordingtotheirrelativesim-ilaritytothequery.Onevector-spaceapproach,LatentSemanticIndexing(LSI),hasachievedupto30%betterretrievalperformancethanlexicalsearchingtechniquesbyemployingareduced-rankmodeloftheterm-documentspace.However,theoriginalimplementationofLSIlackedtheexecutionef ciencyrequiredtomakeLSIusefulforlargedatasets.AnewimplementationofLSI,LSI++,seekstomakeLSIef cient,extensible,portable,andmaintainable.TheLSI++ApplicationProgrammingInterface(API)allowsapplicationstoimmediatelyuseLSIwithoutknowingtheimplementationdetailsoftheunderlyingsystem.LSI++supportsbothserialanddistributedsearchingoflargedatasets,providingthesameprogramminginterfaceregardlessoftheimple-mentationactuallyexecuting.Inaddition,aWorld-WideWebinterfacewascreatedtoallowsimple,intuitivesearchingofdocumentcollectionsusingLSI++.Timingre-sultsindicatetheserialimplementationofLSI++searchesupto6timesfasterthantheoriginalimplementationofLSI,whiletheparallelimplementationsearchesnearly180timesfasteronlargedocumentcollections.
iii