I am deeply indebted to Dr. Michael Berry, my major advisor, for his kind guidance and support. I also thank Dr. Susan Dumais, director of the Information Sciences Research Group at Bellcore, for her technical advice. In addition, she graciously allowed us
Traditionallexical(orBoolean)retrievaltechniques,whilesometimesvaluabletoex-pertstrainedtosearchcollectionsfromaspeci cdiscipline,oftenreturntoomuchinformationtotheuser.Othertimes,becausethetermsusedinthequerydifferfromthetermsusedinthedocument,valuableinformationisneverfoundinthedocumentcollection.
LatentSemanticIndexing(LSI)[DDF90],avector-spaceapproachtoconceptualinformationretrieval,isusefulinsituationswheretraditionallexicalinformationre-trievalapproachesfail.LSIestimatesthesemanticcontentofthedocumentsinacollectionandusesthatestimatetorankthedocumentsinorderofdecreasingrele-vancetoauser’squery.Sincethesearchisbasedontheconceptscontainedinthedocumentsratherthanthedocument’sconstituentterms,LSIcanretrievedocumentsrelatedtoauser’squeryevenwhenthequeryandthedocumentsdonotshareanycommonterms.Also,sinceLSIranksthedocumentsaccordingtotheirrelevancetotheuser’squery,thesystemhelpstheuserdecidewhichinformationmaybemorespeci ctotheuser’sinterests.
AlthoughLSIiscapableofachievingsigni cantretrievalperformancegainsoverstandardlexicalretrievaltechniques(see[Dum91]),thecomplexityoftheLSImodeloftencausesitsexecutionef ciencytolagfarbehindtheexecutionef ciencyofthesimpler,Booleanmodels,especiallyonlargedatasets.BycarefullyexaminingtheLSImodelandnotingthevariousoptimizationsthatcanbeappliedtoitsunderlyingimplementation,though,boththeretrievalbene tsoftheLSImodelandanexecu-tionef ciencynearthatoftheBooleanretrievaltechniquescanbeattained.Here,anef cient,extensible,maintainable,andportableimplementationoftheLSImodelispresented,andasimpleuserinterface,createdwiththenewimplementationoftheLSImodel,http://www.77cn.com.cningboththenewimplementationoftheLSImodelanditscorrespondinguserinterface,userscanquicklysearchlargedatasetswithoutunderstandinganydetailsoftheLSImodelorimplementation.
2