RESEARCH|RESEARCHARTICLES
thesestructuressupport;fractalstructures,suchasriversandtrees,wherecomplexityarisesfromhighlyiteratedbutsimplegenerativeprocesses;andevenabstractknowledge,suchasnaturalnumber,naturallanguagesemantics,andintuitivephysicaltheories(17,46–48).
Capturinghowpeoplelearnalltheseconceptsatthelevelwereachedwithhandwrittencharac-tersisalong-termgoal.Inthenearterm,applyingourapproachtoothertypesofsymbolicconceptsmaybeparticularlypromising.Humanculturesproducemanysuchsymbolsystems,includinggestures,dancemoves,andthewordsofspokenandsignedlanguages.Aswithcharacters,theseconceptscanbelearnedtosomeextentfromoneorafewexamples,evenbeforethesymbolicmeaningisclear:Considerseeinga“thumbsup,”“fistbump,”or“highfive”orhearingthenames“BoutrosBoutros-Ghali,”“KofiAnnan,”or“BanKi-moon”forthefirsttime.Fromthislimitedexperience,peoplecantypicallyrecognizenewexamplesandevenproducearecognizablesem-blanceoftheconceptthemselves.TheBPLprin-ciplesofcompositionality,causality,andlearningtolearnmayhelptoexplainhow.
ToillustratehowBPLappliesinthedomainofspeech,programsforspokenwordscouldbeconstructedbycomposingphonemes(subparts)systematicallytoformsyllables(parts),whichcomposefurthertoformmorphemesandentirewords.Givenanabstractsyllable-phonemeparseofaword,realisticspeechtokenscanbegener-atedfromacausalmodelthatcapturesaspectsofspeechmotorarticulation.ThesepartandsubpartAlphabet of charactersi)New machine-generated characters in each alphabetii)Human or Machine?iii)121212i)Human or Machine?12ii)1212121212iii)componentsaresharedacrossthewordsinalanguage,enablingchildrentoacquirethemthroughalong-termlearning-to-learnprocess.Wehavealreadyfoundthataprototypemodelexploitingcompositionalityandlearningtolearn,butnotcausality,isabletocapturesomeaspectsofthehumanabilitytolearnandgeneralizenewspokenwords(e.g.,EnglishspeakerslearningwordsinJapanese)(49).Furtherprogressmaycomefromadoptingarichercausalmodelofspeechgeneration,inthespiritofclassic“analysis-by-synthesis”proposalsforspeechperceptionandlanguagecomprehension(20,50).
Althoughourworkfocusedonadultlearners,itraisesnaturaldevelopmentalquestions.Ifchil-drenlearningtowriteacquireaninductivebiassimilartowhatBPLconstructs,themodelcouldhelpexplainwhychildrenfindsomecharactersdifficultandwhichteachingproceduresaremosteffective(51).Comparingchildren’sparsingandgeneralizationbehavioratdifferentstagesoflearningandBPLmodelsgivenvaryingback-groundexperiencecouldbetterevaluatethemodel’slearning-to-learnmechanismsandsuggestimprove-ments.Bytestingourclassificationtasksoninfantswhocategorizevisuallybeforetheybegindrawingorscribbling(52),wecanaskwhetherchildrenlearntoperceivecharactersmorecausallyandcomposi-tionallybasedontheirownproto-writingexperi-ence.CausalrepresentationsareprewiredinourcurrentBPLmodels,buttheycouldconceivablybeconstructedthroughlearningtolearnatanevendeeperlevelofmodelhierarchy(53).
Last,wehopethatourworkmayshedlightontheneuralrepresentationsofconceptsandthedevelopmentofmoreneurallygroundedlearningmodels.Supplementingfeedforwardvisualpro-cessing(54),previousbehavioralstudiesandourresultssuggestthatpeoplelearnnewhandwrittencharactersinpartbyinferringabstractmotorprograms(55),arepresentationgroundedinpro-ductionyetactiveinpurelyperceptualtasks,independentofspecificmotorarticulatorsandpotentiallydrivenbyactivityinpremotorcortex(56–58).Couldwedecoderepresentationsstruc-turallysimilartothoseinBPLfrombrainimagingofpremotorcortex(orotheraction-orientedregions)inhumansperceivingandclassifyingnewchar-actersforthefirsttime?Recentlarge-scalebrainmodels(59)anddeeprecurrentneuralnetworks(60–62)havealsofocusedoncharacterrecogni-tionandproductiontasks—buttypicallylearningfromlargetrainingsampleswithmanyexamplesofeachconcept.Weseetheone-shotlearningcapacitiesstudiedhereasachallengefortheseneuralmodels:oneweexpecttheymightrisetobyincorporatingtheprinciplesofcomposition-ality,causality,andlearningtolearnthatBPLinstantiates.
REFERENCESANDNOTES
Fig.7.Generatingnewconcepts.(A)Humansandmachinesweregivenanovelalphabet(i)andaskedtoproducenewcharactersforthatalphabet.Newmachine-generatedcharactersareshownin(ii).Humanandmachineproductionscanbecomparedin(iii).Thefour-charactergridsineachpairthatweregeneratedbythemachineare(byrow)1,1,2;1,2.(B)Humansandmachinesproducednewcharacterswithoutareferencealphabet.Thegridsthatweregeneratedbyamachineare2;1;1;2.SCIENCEsciencemag.org
1.B.Landau,L.B.Smith,S.S.Jones,Cogn.Dev.3,299–321(1988).2.E.M.Markman,CategorizationandNaminginChildren(MIT
Press,Cambridge,MA,1989).
3.F.Xu,J.B.Tenenbaum,Psychol.Rev.114,245–272(2007).4.S.Geman,E.Bienenstock,R.Doursat,NeuralComput.4,1–58
(1992).
5.Y.LeCun,L.Bottou,Y.Bengio,P.Haffner,Proc.IEEE86,
2278–2324(1998).
11DECEMBER2015?VOL350ISSUE6266
1337
RESEARCH|RESEARCHARTICLES
6.G.E.Hintonetal.,IEEESignalProcess.Mag.29,82–97(2012).7.A.Krizhevsky,I.Sutskever,G.E.Hinton,Adv.NeuralInf.
Process.Syst.25,1097–1105(2012).
8.Y.LeCun,Y.Bengio,G.Hinton,Nature521,436–444(2015).9.V.Mnihetal.,Nature518,529–533(2015).
10.J.Feldman,J.Math.Psychol.41,145–170(1997).11.I.Biederman,Psychol.Rev.94,115–147(1987).12.T.B.Ward,Cognit.Psychol.27,1–40(1994).
13.A.Jern,C.Kemp,Cognit.Psychol.66,85–125(2013).14.L.G.Valiant,Commun.ACM27,1134–1142(1984).
15.D.McAllester,inProceedingsofthe11thAnnualConferenceon
ComputationalLearningTheory(COLT),Madison,WI,24to26July1998(AssociationforComputingMachinery,NewYork,1998),pp.230–234.
16.V.N.Vapnik,IEEETrans.NeuralNetw.10,988–999(1999).17.N.D.Goodman,J.B.Tenenbaum,T.Gerstenberg,Concepts:
NewDirections,E.Margolis,S.Laurence,Eds.(MITPress,Cambridge,MA,2015).
18.Z.Ghahramani,Nature521,452–459(2015).
19.P.H.Winston,ThePsychologyofComputerVision,P.H.Winston,
Ed.(McGraw-Hill,NewYork,1975).
20.T.G.Bever,D.Poeppel,Biolinguistics4,174(2010).21.L.B.Smith,S.S.Jones,B.Landau,L.Gershkoff-Stowe,
L.Samuelson,Psychol.Sci.13,13–19(2002).
22.R.L.Goldstone,inPerceptualOrganizationinVision:Behavioral
andNeuralPerspectives,R.Kimchi,M.Behrmann,C.Olson,Eds.(LawrenceErlbaum,City,NJ,2003),pp.233–278.23.H.F.Harlow,Psychol.Rev.56,51–65(1949).
24.D.A.Braun,C.Mehring,D.M.Wolpert,Behav.BrainRes.206,
157–165(2010).
25.C.Kemp,A.Perfors,J.B.Tenenbaum,Dev.Sci.10,307–321
(2007).
26.R.Salakhutdinov,J.Tenenbaum,A.Torralba,inJMLRWorkshop
andConferenceProceedings,vol.27,UnsupervisedandTransferLearningWorkshop,I.Guyon,G.Dror,V.Lemaire,G.Taylor,D.Silver,Eds.(Microtome,Brookline,MA,2012),pp.195–206.27.J.Baxter,J.Artif.Intell.Res.12,149(2000).
28.Y.LeCunetal.,NeuralComput.1,541–551(1989).
29.R.Salakhutdinov,J.B.Tenenbaum,A.Torralba,IEEETrans.
PatternAnal.Mach.Intell.35,1958–1971(2013).
30.V.K.Mansinghka,T.D.Kulkarni,Y.N.Perov,J.B.Tenenbaum,
Adv.NeuralInf.Process.Syst.26,1520–1528(2013).
31.K.Liu,Y.S.Huang,C.Y.Suen,IEEETrans.PatternAnal.Mach.
Intell.21,1095–1100(1999).
32.M.-P.Dubuisson,A.K.Jain,inProceedingsofthe12thIAPR
InternationalConferenceonPatternRecognition,Vol.1,ConferenceA:ComputerVisionandImageProcessing,Jerusalem,Israel,9to13October1994(IEEE,NewYork,1994),pp.566–568.
33.G.Koch,R.S.Zemel,R.Salakhutdinov,paperpresentedatICML
DeepLearningWorkshop,Lille,France,10and11July2015.34.M.Revow,C.K.I.Williams,G.E.Hinton,IEEETrans.Pattern
Anal.Mach.Intell.18,592–606(1996).
35.G.E.Hinton,V.Nair,Adv.NeuralInf.Process.Syst.18,515–522
(2006).
36.S.-C.Zhu,D.Mumford,FoundationsTrendsComput.Graphics
Vision2,259–362(2006).
37.P.Liang,M.I.Jordan,D.Klein,inProceedingsofthe27th
InternationalConferenceonMachineLearning,Haifa,Israel,21to25June2010(InternationalMachineLearningSociety,Princeton,NJ,2010),pp.639–646.
38.P.F.Felzenszwalb,R.B.Girshick,D.McAllester,D.Ramanan,
IEEETrans.PatternAnal.Mach.Intell.32,1627–1645(2010).39.I.Hwang,A.Stuhlmüller,N.D.Goodman,http://arxiv.org/abs/
1110.5667(2011).
40.E.Dechter,J.Malmaud,R.P.Adams,J.B.Tenenbaum,in
Proceedingsofthe23rdInternationalJointConferenceonArtificialIntelligence,F.Rossi,Ed.,Beijing,China,3to9August2013(AAAIPress/InternationalJointConferencesonArtificialIntelligence,MenloPark,CA,2013),pp.1302–1309.
41.J.Rule,E.Dechter,J.B.Tenenbaum,inProceedingsofthe37th
AnnualConferenceoftheCognitiveScienceSociety,D.C.Noelleetal.,Eds.,Pasadena,CA,22to25July2015(CognitiveScienceSociety,Austin,TX,2015),pp.2051–2056.42.L.W.Barsalou,Mem.Cognit.11,211–227(1983).
43.J.J.Williams,T.Lombrozo,Cogn.Sci.34,776–806(2010).44.A.B.Markman,V.S.Makin,J.Exp.Psychol.Gen.127,331–354
(1998).
45.D.N.Osherson,E.E.Smith,Cognition9,35–58(1981).
46.S.T.Piantadosi,J.B.Tenenbaum,N.D.Goodman,Cognition
123,199–217(2012).
47.G.A.Miller,P.N.Johnson-Laird,LanguageandPerception
(Belknap,Cambridge,MA,1976).
48.T.D.Ullman,A.Stuhlmuller,N.Goodman,J.B.Tenenbaum,
inProceedingsofthe36thAnnualConferenceoftheCognitiveScienceSociety,QuebecCity,Canada,23to
26July2014(CognitiveScienceSociety,Austin,TX,2014),pp.1640–1645.
49.B.M.Lake,C.-y.Lee,J.R.Glass,J.B.Tenenbaum,in
Proceedingsofthe36thAnnualConferenceoftheCognitiveScienceSociety,QuebecCity,Canada,23to26July2014(CognitiveScienceSociety,Austin,TX,2014),pp.803–808.50.U.Neisser,CognitivePsychology(Appleton-Century-Crofts,
NewYork,1966).
51.R.Treiman,B.Kessler,HowChildrenLearntoWriteWords
(OxfordUniv.Press,NewYork,2014).
52.A.L.Ferry,S.J.Hespos,S.R.Waxman,ChildDev.81,472–479
(2010).
53.N.D.Goodman,T.D.Ullman,J.B.Tenenbaum,Psychol.Rev.
118,110–119(2011).
54.S.Dehaene,ReadingintheBrain(Penguin,NewYork,2009).55.M.K.Babcock,J.J.Freyd,Am.J.Psychol.101,111–130
(1988).
56.M.Longcamp,J.L.Anton,M.Roth,J.L.Velay,Neuroimage19,
1492–1500(2003).
57.K.H.James,I.Gauthier,Neuropsychologia44,2937–2949
(2006).
58.K.H.James,I.Gauthier,J.Exp.Psychol.Gen.138,416–431
(2009).
59.C.Eliasmithetal.,Science338,1202–1205(2012).60.A.Graves,http://arxiv.org/abs/1308.0850(2014).
61.K.Gregor,I.Danihelka,A.Graves,D.J.Rezende,D.Wierstra,
inProceedingsoftheInternationalConferenceonMachineLearning(ICML),Lille,France,6to11July2015(InternationalMachineLearningSociety,Princeton,NJ,2015),pp.1462–1471.
62.J.Chungetal.,Adv.NeuralInf.Process.Syst.28,(2015).
ACKNOWLEDGMENTS
ThisworkwassupportedbyaNSFGraduateResearchFellowshiptoB.M.L.;theCenterforBrains,Minds,andMachinesfundedbyNSFScienceandTechnologyCenterawardCCF-1231216;ArmyResearchOfficeandOfficeofNavalResearchcontractsW911NF-08-1-0242,W911NF-13-1-2012,andN000141310333;theNaturalSciencesandEngineeringResearchCouncilofCanada;theCanadianInstituteforAdvancedResearch;andtheMoore-SloanDataScienceEnvironmentatNYU.WethankJ.McClelland,T.PoggioandL.SchulzformanyvaluablecontributionsandN.Kanwisherforhelpinguselucidatethe
threekeyprinciples.WearegratefultoJ.GrossandtheOmniglot.comencyclopediaofwritingsystemsforhelpingtomakethisdatasetpossible.OnlinearchivesareavailableforvisualTuringtests(http://github.com/brendenlake/visual-turing-tests),Omniglotdataset(http://github.com/brendenlake/omniglot),andBPLsourcecode(http://github.com/brendenlake/BPL).
SUPPLEMENTARYMATERIALS
www.sciencemag.org/content/350/6266/1332/suppl/DC1MaterialsandMethodsSupplementaryTextFigs.S1toS11
References(63–80)
1June2015;accepted15October201510.1126/science.aab3050PHYSICALCHEMISTRYSpectroscopiccharacterizationofisomerizationtransitionstatesJoshuaH.Baraban,1*P.BryanChangala,1?GeorgCh.Mellau,2JohnF.Stanton,3AnthonyJ.Merer,4,5RobertW.Field1?Transitionstatetheoryiscentraltoourunderstandingofchemicalreactiondynamics.Wedemonstrateamethodforextractingtransitionstateenergiesandpropertiesfromacharacteristicpatternfoundinfrequency-domainspectraofisomerizingsystems.Thispattern—adipinthespacingsofcertainbarrier-proximalvibrationallevels—canbeunderstoodusingtheconceptofeffectivefrequency,weff.Themethodisappliedtothecis-transconformationalchangeintheS1stateofC2H2andthebond-breakingHCN-HNCisomerization.Inbothcases,thebarrierheightsderivedfromspectroscopicdataagreeextremelywellwithpreviousabinitiocalculations.Wealsoshowthatitispossibletodistinguishbetweenvibrationalmodesthatareactivelyinvolvedintheisomerizationprocessandthosethatarepassivebystanders.hecentralconceptofthetransitionstateinchemicalkineticsisfamiliartoallstudentsofchemistry.SinceitsinceptionbyArrhe-nius(1)andlaterdevelopmentintoafulltheorybyEyring,Wigner,Polanyi,andEvans(2–5),theideathatthethermalratedependsprimarilyonthehighestpointalongthelowest-DepartmentofChemistry,MassachusettsInstituteofTechnology,Cambridge,MA02139,USA.2Physikalisch-ChemischesInstitut,Justus-Liebig-Universit?tGiessen,D-35392Giessen,Germany.3DepartmentofChemistryandBiochemistry,UniversityofTexas,Austin,TX78712,USA.4DepartmentofChemistry,UniversityofBritishColumbia,Vancouver,BCV6T1Z1,Canada.5InstituteofAtomicandMolecularSciences,AcademiaSinica,Taipei10617,Taiwan.
*Presentaddress:DepartmentofChemistryandBiochemistry,UniversityofColorado,Boulder,CO80309,USA.?Presentaddress:JILA,NationalInstituteofStandardsandTechnology,and
DepartmentofPhysics,UniversityofColorado,Boulder,CO80309,USA.?Correspondingauthor.E-mail:rwfield@mit.edu
1Tenergypathfromreactantstoproductshasre-mainedessentiallyunchanged.Mostofchemicaldynamicsisnowfirmlybasedonthisideaofthetransitionstate,notwithstandingtheemergenceofunconventionalreactionssuchasroaming(6,7),whereaphotodissociatedatomwandersbeforeabstractingfromtheparentfragment.Despitetheclearimportanceofthetransitionstatetothefieldofchemistry,directexperimentalstudiesofthetransitionstateanditspropertiesarescarce(8).Here,wereporttheobservationofavibrationalpattern,adipinthetrendofquantumlevelspac-ings,whichoccursattheenergyofthesaddlepoint.Thisphenomenonisexpectedtoprovideagenerallyapplicableandaccuratemethodforcharacterizingtransitionstates.Onlyasubsetofvibrationalstatesexhibitadip;thesestatescontainexcitationalongthereactioncoordinateandarebarrier-proximal,meaningthattheyaremore
sciencemag.orgSCIENCE
1338
11DECEMBER2015?VOL350ISSUE6266
Human-level concept learning through probabilistic programinduction
Brenden M. Lake et al.Science 350, 1332 (2015);
DOI: 10.1126/science.aab3050
This copy is for your personal, non-commercial use only.
Permission to republish or repurpose articles or portions of articles can be obtained byfollowing the guidelines here.
The following resources related to this article are available online at
www.sciencemag.org (this information is current as of December 10, 2015 ):
Updated information and services, including high-resolution figures, can be found in the onlineversion of this article at:
http://www.sciencemag.org/content/350/6266/1332.full.html
Supporting Online Material can be found at:
http://www.sciencemag.org/content/suppl/2015/12/09/350.6266.1332.DC1.html
A list of selected additional articles on the Science Web sites related to this article can befound at:
http://www.sciencemag.org/content/350/6266/1332.full.html#related This article cites 50 articles, 2 of which can be accessed free:
http://www.sciencemag.org/content/350/6266/1332.full.html#ref-list-1 This article appears in the following subject collections:Psychology
http://www.sciencemag.org/cgi/collection/psychology
Science (print ISSN 0036-8075; online ISSN 1095-9203) is published weekly, except the last week in December, by theAmerican Association for the Advancement of Science, 1200 New York Avenue NW, Washington, DC 20005. Copyright2015 by the American Association for the Advancement of Science; all rights reserved. The title Science is aregistered trademark of AAAS.
Downloaded from www.sciencemag.org on December 11, 2015If you wish to distribute this article to others, you can order high-quality copies for yourcolleagues, clients, or customers by clicking here.