Downloading note: if your browser has problems in correctly handling the gzipped postscript files, try downloading the file and saving it (click right button on link, and select "Save Link as".) Then uncompress the file using gzip or winzip ( on a windows system, the compressed file may have a strange name, but you should still be able to work with it using winzip.) If you have any problems downloading or printing papers, send email to me tplate at acm.org
Tony Plate (2004) Slides for presentation on HRRs at AAAI 2004 Symposium on Compositional Connectionism (2-up PDF, 15 pages, 56k)
Tony Plate (2003) "Holographic Reduced Representations", CSLI Lecture Notes Number 150, CSLI Publications, Stanford, CA, ISBN 1575864304. Available on Amazon.com in paperback for US$25.
Contents:
The back-cover blurb:
While neuroscientists garner success in identifying brain regions and in analyzing individual neurons, ground is still being broken at the intermediate scale of understanding how neurons combine to encode information. This book proposes a method of representing information in a computer that would be suited for modeling the brain's methods of processing information.In stark contrast to traditional computing where "every bit counts," the method proposed by Plate distributes information over large numbers of components, which are mathematically modeled by high-dimensional vectors. No single unit or even a small number of them means anything in particular. The meaningful entity is the total pattern over all the units. Superficially, the patterns appear random.
Holographic Reduced Representations (HRRs) are introduced here to model how the brain could distribute each piece of information among thousands of neurons. It had been previously thought that the semantic structure of natural language sentences cannot be encoded practically in a distributed representation but HRRs can overcome problems of earlier proposals. This work has implications for psychology, neuroscience, linguistics, and computer science and artificial intelligence.
From the preface:
This book is a based on my PhD thesis. It has been significantly rewritten and updated for publishing as a book. The major changes include:
- New related work described in chapter 2
- A new section surveying techniques for learning in distributed representations
- Complete rewriting of the chapter on analogy processing, including:
- redesigned experiments
- discussion of the relationship between HRRs and kernel methods for computing similarity
- clear exposition of how a HRR-based model can perform structure- sensitive analogy retrieval in a single-stage (cf 2-stage MAC/FAC model of human performance on analogy retrieval)
- Many references to new work have been added
- Subject and author indices added
Tony Plate (2002) "Convolution-based memory models", Macmillian Encyclopedia of Cognitive Science, ed Lynn Nadel. Draft version (PDF, 9 pages, 73k)
Tony Plate (2002) "Distributed Representations", Macmillian Encyclopedia of Cognitive Science, ed Lynn Nadel. Draft version (PDF, 15 pages, 111k)
Vince Darley, Alexander Outkin, Tony Plate, and Frank Gao (2001) "Learning, Evolution and Tick Size Effects in a Simulation of the Nasdaq Stock Market". In Proceedings of the Joint 5th World Multiconference on Systemics, Cybernetics and Informatics (SCI 2001) and the 7th International Conference on Information Systems Analysis and Synthesis (ISAS 2001), Orlando, USA. PDF (266K, 6 pages)
Alexander Outkin, V. Darley, T. Plate, F. Gao, (2000) "Sixteenths or Pennies? Observations from a Simulation of the Nasdaq Stock Market". Presented at the Conference on Computational Intelligence for Financial Engineering, New York. PDF (112K, 18 pages)
Tony Plate (2000) "Randomly connected sigma-pi neurons can form associative memories". Network: Computation in neural systems. Vol. 11, No. 4, pp321-332. PDF (270k).
Tony Plate (2000) "Analogy retrieval and processing with distributed vector representations". Expert Systems: The International Journal of Knowledge Engineering and Neural Networks, Special Issue on Connectionist Symbol Processing, Vol. 17, No. 1, pp29-40. PDF (70k).
Tony Plate, Joel Bert, John Grace and Pierre Band (2000) "Visualization of feed-forward neural networks." Neural Computation, Vol. 12, No. 6, pp1337-1354. PDF (1000k).
Tony Plate (1999) "Accuracy versus interpretability in flexible modeling: implementing a tradeoff using Gaussian process models." Behaviourmetrika, special issue on Analysis of knowledge representations in neural network (NN) models. Vol. 26, No. 1, pp29-50. Pre-publication version available: gzipped postscript (475k).
1998
Tony Plate, Joel Bert, John Grace and Pierre Band "Visualizing the function computed by a feedforward neural network." Technical report CS-TR-98-5, Victoria University of Wellington, Computer Science. 14 pages, contains 2 color figures. gzipped postscript (508K). In several of the figures, color is very important. Here is a file containing just the pages with the color figures (send this to color printer): gzipped postscript (330K).
Tony Plate (1998). "Structured operations with distributed vector representations" In Keith Holyoak, Dedre Gentner, and Boicho Kokinov, Advances in Analogy Research: Integration of Theory and Data from the Cognitive, Computational, and Neural Sciences. NBU Series in Cognitive Science. New Bugarian University, Sofia. gzipped postscript (167K). This was a presentation at the Analogy'98 workshop, in Sofia, Bulgaria, Jul 16-20, 1998.
Tony Plate (1998). "Analogy retrieval and processing with distributed represenations" Technical report CS-TR-98-4, Victoria University of Wellington, Computer Science. 16 pages. Longer version of Sofia workshop paper. Tech Report 98-4 (270K).
Tony Plate "Chunks, bindings, STAR and Holographic Reduced Representations." Behavioral and Brain Sciences. Commentary on "Processing Capacity Defined by Relational Complexity: Implications for Comparative, Developmental, and Cognitive Psychology", by Halford, Wilson and Phillips. gzipped postscript (70k)
Tony Plate (1998) "Accuracy versus interpretability in flexible modeling: implementing a tradeoff using Gaussian process models." In Tom Downs and Marcus Frean and Marcus Gallagher (eds), Proceedings of the Ninth Australian Conference on Neural Networks (ACNN'98). University of Queeensland. gzipped postscript (112K)
1997
Tony Plate (1998), "Controlling the hyper-parameter search in MacKay's Bayesian neural network framework", in Neural Networks: Tricks of the Trade, eds Genevieve Orr, Klaus-Robert Müller and Rich Caruana, Springer Verlag, Lecture Notes in Computer Science 1524, pp93-112.
Tony Plate (1998), "Avoiding roundoff error in backpropagating derivatives", in Neural Networks: Tricks of the Trade, eds Genevieve Orr, Klaus-Robert Müller and Rich Caruana, Springer Verlag, Lecture Notes in Computer Science 1524, pp227-233.
Tony Plate (1997), "A common framework for distributed representation schemes for compositional structure", in Connectionist Systems for Knowledge Representation and Deduction, Frederic Maire and Ross Hayward and Joachim Diederich, editors, pp15-34, Queensland University of Technology, gzipped postscript (102K)
Tony Plate (1997), "Structure Matching and Transformation with Distributed Representations", in Connectionist-Symbolic Integration, R. Sun and F. Alexandre (eds), Lawrence Erlbaum Associates. gzipped postscript (75K)
Tony Plate, Joel Bert, John Grace and Pierre Band (1997) "A comparison between neural networks and other statistical techniques for modeling the relationship between tobacco and alcohol and cancer", in Advances in Neural Information Processing 9 (NIPS*96), M.C. Mozer, M.I. Jordan and T. Petsche, Eds, pp967-973, MIT Press. gzipped postscript (57K)
Tony Plate, Joel Bert, John Grace and Pierre Band (1997), "Visualizing the function computed by a feedforward neural network." In Proceedings of The Fourth International Conference on Neural Information Processing (ICONIP'97), Springer Verlag. gzipped postscript (172K)
1995
Tony A. Plate (1995), "Holographic Reduced Representations", IEEE Transactions on Neural Networks, vol. 6, no. 3, pp623-641, 1995, 21 pages: gzipped postscript (149K).
Tony A. Plate (1995), "Estimating analogical similarity by vector dot-products of Holographic Reduced Representations", Draft, August 1995, 44 pages: gzipped postscript (124K) gzipped 2up-postscript (128K).
Tony A. Plate (1995), "Networks which learn to store variable-length sequences in a fixed set of unit activations", Unpublished manuscript, March 1995, 19 pages: gzipped postscript (69K).
1994
Tony A. Plate (1994), "Distributed Representations and Nested Compositional Structure", PhD Thesis, Department of Computer Science, University of Toronto, 1994, 216 pages: gzipped postscript (565K) gzipped 2up-postscript (570K).
Tony A. Plate (1994), "Estimating structural similarity by vector dot products of Holographic Reduced Representations", In J. D. Cowan, G. Tesauro, and J. Alspector, editors, Advances in Neural Information Processing Systems 6 (NIPS*93), pp1109-1116, Morgan Kaufmann, San Mateo, CA, 1994, 8 pages: gzipped postscript (29K).
1993
Tony A. Plate, (1993) "Holographic recurrent networks", In C. L. Giles, S.J. Hanson, and J. D. Cowan, editors, Advances in Neural Information Processing Systems 5 (NIPS*92), pp34-41, Morgan Kaufmann, San Mateo, CA, 1993, 8 pages: gzipped postscript (42K).
1992
Tony A. Plate (1991), "Holographic Reduced Representations: Convolution algebra for compositional distributed representations." In P. Mehra and B.~W. Wah, editors. Artificial neural networks: concepts and theory. IEEE Computer Society Press Tutorial, 1992, 6 pages. (Note: this is a reprint of IJCAI91 paper.) gzipped postscript (54K).
Tony A. Plate (1991), "Holographic Reduced Representations: Convolution algebra for compositional distributed representations", In John Mylopoulos and Ray Reiter, editors, Proceedings of the 12th International Joint Conference on Artificial Intelligence (IJCAI), pp30-35, Morgan Kaufmann, San Mateo, CA, 1991, 6 pages. gzipped postscript (54K) PDF (216K).
1990
Yorick A. Wilks, Dan C. Fass, Cheng ming Guo, James E. McDonald, Tony Plate, and Brian M. Slator (1990), "Providing machine tractable dictionary tools", Machine Translation, 5: pp99-154.
James E. McDonald, Tony A. Plate, and Roger W. Schvaneveldt (1990), "Using pathfinder to extract semantic information from text", In Roger W. Schvaneveldt, editor, "Pathfinder Associative Networks", Ablex, New Jersey.
vsa, R package for HRR calculations and support for adding other Vector Symbolic Architecture schemes. ("R" is an open-source implementation of the S language for statistical analysis)
vcalc1.0, for a fast HRR calculator, with various examples.
tap.printf, Java package implementing printf-like functionality. Two ways of getting it:
ploteff: Splus methods for visualizing non-linear functions (e.g., neural networks, Gaussian process models, and bagged trees) through plots of "additive effects". This software implements the visualization techniques described in the 1999 Neural Computation paper and in CS-TR-98-5 and used in the Behaviormetrika paper. Available as Unix shar file (64 Kbytes) OR ploteff.zip (18 Kbytes). Also available at The Statlib Archives.