Page Contents:

Various Publications on Vector Symbolic Architectures

My Publications

Software

Publications

Downloading note: if your browser has problems in correctly handling the gzipped postscript files, try downloading the file and saving it (click right button on link, and select "Save Link as".) Then uncompress the file using gzip or winzip ( on a windows system, the compressed file may have a strange name, but you should still be able to work with it using winzip.) If you have any problems downloading or printing papers, send email to metplate at acm.org

2004

Tony Plate (2004) Slides for presentation on HRRs at AAAI 2004 Symposium on Compositional Connectionism (2-up PDF, 15 pages, 56k)

2003

Tony Plate (2003) "Holographic Reduced Representations", CSLI Lecture Notes Number 150, CSLI Publications, Stanford, CA, ISBN 1575864304. Available on Amazon.com in paperback for US$25.

Contents:

- Introduction 1
- Review of connectionist and distributed memory models 25
- Holographic Reduced Representation 93
- HRRs in the frequency domain 145
- Using convolution-based storage in systems that learn 153
- Estimating analogical similarity 175
- Discussion 221

The back-cover blurb:

While neuroscientists garner success in identifying brain regions and in analyzing individual neurons, ground is still being broken at the intermediate scale of understanding how neurons combine to encode information. This book proposes a method of representing information in a computer that would be suited for modeling the brain's methods of processing information.In stark contrast to traditional computing where "every bit counts," the method proposed by Plate distributes information over large numbers of components, which are mathematically modeled by high-dimensional vectors. No single unit or even a small number of them means anything in particular. The meaningful entity is the total pattern over all the units. Superficially, the patterns appear random.

Holographic Reduced Representations (HRRs) are introduced here to model how the brain could distribute each piece of information among thousands of neurons. It had been previously thought that the semantic structure of natural language sentences cannot be encoded practically in a distributed representation but HRRs can overcome problems of earlier proposals. This work has implications for psychology, neuroscience, linguistics, and computer science and artificial intelligence.

From the preface:

This book is a based on my PhD thesis. It has been significantly rewritten and updated for publishing as a book. The major changes include:

- New related work described in chapter 2
- A new section surveying techniques for learning in distributed representations
- Complete rewriting of the chapter on analogy processing, including:

- redesigned experiments
- discussion of the relationship between HRRs and kernel methods for computing similarity
- clear exposition of how a HRR-based model can perform structure- sensitive analogy retrieval in a single-stage (cf 2-stage MAC/FAC model of human performance on analogy retrieval)
- Many references to new work have been added
- Subject and author indices added

2002

Tony Plate (2002) "Convolution-based memory models", Macmillian Encyclopedia of Cognitive Science, ed Lynn Nadel. Draft version (PDF, 9 pages, 73k)

Tony Plate (2002) "Distributed Representations", Macmillian Encyclopedia of Cognitive Science, ed Lynn Nadel. Draft version (PDF, 15 pages, 111k)

2001

Vince Darley, Alexander Outkin, Tony Plate, and Frank Gao (2001) "Learning, Evolution and Tick Size Effects in a Simulation of the Nasdaq Stock Market". In Proceedings of the Joint 5th World Multiconference on Systemics, Cybernetics and Informatics (SCI 2001) and the 7th International Conference on Information Systems Analysis and Synthesis (ISAS 2001), Orlando, USA. PDF (266K, 6 pages)

2000

Alexander Outkin, V. Darley, T. Plate, F. Gao, (2000) "Sixteenths or Pennies? Observations from a Simulation of the Nasdaq Stock Market". Presented at the Conference on Computational Intelligence for Financial Engineering, New York. PDF (112K, 18 pages)

Tony Plate (2000) "Randomly connected sigma-pi
neurons can form associative memories".
*Network: Computation in neural systems.*
Vol. 11, No. 4, pp321-332.
PDF (270k).

Tony Plate (2000) "Analogy retrieval and processing with
distributed vector representations". *Expert Systems: The
International Journal of Knowledge Engineering and Neural
Networks*, Special Issue on Connectionist Symbol
Processing, Vol. 17, No. 1, pp29-40.
PDF (70k).

Tony Plate, Joel Bert, John Grace and Pierre Band (2000) "Visualization of feed-forward neural networks." Neural Computation, Vol. 12, No. 6, pp1337-1354. PDF (1000k).

1999

Tony Plate (1999) "Accuracy versus interpretability in flexible modeling:
implementing a tradeoff using Gaussian process models." *Behaviourmetrika,
special issue on Analysis of knowledge representations in neural network
(NN) models*. Vol. 26, No. 1, pp29-50. Pre-publication version available:
gzipped postscript (475k).

1998

Tony Plate, Joel Bert, John Grace and Pierre Band "Visualizing the function computed by a feedforward neural network." Technical report CS-TR-98-5, Victoria University of Wellington, Computer Science. 14 pages, contains 2 color figures. gzipped postscript (508K). In several of the figures, color is very important. Here is a file containing just the pages with the color figures (send this to color printer): gzipped postscript (330K).

Tony Plate (1998). "Structured operations with distributed vector representations"
In Keith Holyoak, Dedre Gentner, and Boicho Kokinov, *Advances in Analogy
Research: Integration of Theory and Data from the Cognitive, Computational,
and Neural Sciences.* NBU Series in Cognitive Science. New Bugarian
University, Sofia. gzipped postscript (167K).
This was a presentation at the Analogy'98
workshop, in Sofia, Bulgaria, Jul 16-20, 1998.

Tony Plate (1998). "Analogy retrieval and processing with distributed represenations" Technical report CS-TR-98-4, Victoria University of Wellington, Computer Science. 16 pages. Longer version of Sofia workshop paper. Tech Report 98-4 (270K).

Tony Plate "Chunks, bindings, STAR and Holographic Reduced Representations." Behavioral and Brain Sciences. Commentary on "Processing Capacity Defined by Relational Complexity: Implications for Comparative, Developmental, and Cognitive Psychology", by Halford, Wilson and Phillips. gzipped postscript (70k)

Tony Plate (1998) "Accuracy versus interpretability in flexible modeling:
implementing a tradeoff using Gaussian process models." In Tom Downs and
Marcus Frean and Marcus Gallagher (eds), *Proceedings of the Ninth Australian
Conference on Neural Networks (ACNN'98). University of Queeensland.*
gzipped postscript (112K)

1997

Tony Plate (1998), "Controlling the hyper-parameter search in MacKay's Bayesian neural network framework", in Neural Networks: Tricks of the Trade, eds Genevieve Orr, Klaus-Robert Müller and Rich Caruana, Springer Verlag, Lecture Notes in Computer Science 1524, pp93-112.

Tony Plate (1998), "Avoiding roundoff error in backpropagating derivatives", in Neural Networks: Tricks of the Trade, eds Genevieve Orr, Klaus-Robert Müller and Rich Caruana, Springer Verlag, Lecture Notes in Computer Science 1524, pp227-233.

Tony Plate (1997), "A common framework for distributed
representation schemes for compositional structure", in
*Connectionist Systems for Knowledge Representation and
Deduction*, Frederic Maire and Ross Hayward and Joachim
Diederich, editors, pp15-34, Queensland University of
Technology, gzipped
postscript (102K)

Tony Plate (1997), "Structure Matching and Transformation
with Distributed Representations", in
*Connectionist-Symbolic Integration*, R. Sun and
F. Alexandre (eds), Lawrence Erlbaum Associates. gzipped postscript
(75K)

Tony Plate, Joel Bert, John Grace and Pierre Band (1997)
"A comparison between neural networks and other statistical
techniques for modeling the relationship between tobacco and
alcohol and cancer", in *Advances in Neural Information
Processing 9 (NIPS*96)*, M.C. Mozer, M.I. Jordan and
T. Petsche, Eds, pp967-973, MIT Press. gzipped postscript (57K)

Tony Plate, Joel Bert, John Grace and Pierre Band (1997),
"Visualizing the function computed by a feedforward neural
network." In *Proceedings of The Fourth International
Conference on Neural Information Processing (ICONIP'97)*,
Springer Verlag. gzipped
postscript (172K)

1995

Tony A. Plate (1995), "Holographic Reduced Representations", IEEE Transactions on Neural Networks, vol. 6, no. 3, pp623-641, 1995, 21 pages: gzipped postscript (149K).

Tony A. Plate (1995), "Estimating analogical similarity by vector dot-products of Holographic Reduced Representations", Draft, August 1995, 44 pages: gzipped postscript (124K) gzipped 2up-postscript (128K).

Tony A. Plate (1995), "Networks which learn to store variable-length sequences in a fixed set of unit activations", Unpublished manuscript, March 1995, 19 pages: gzipped postscript (69K).

1994

Tony A. Plate (1994), "Distributed Representations and Nested Compositional Structure", PhD Thesis, Department of Computer Science, University of Toronto, 1994, 216 pages: gzipped postscript (565K) gzipped 2up-postscript (570K).

Tony A. Plate (1994), "Estimating structural similarity
by vector dot products of Holographic Reduced
Representations", In J. D. Cowan, G. Tesauro, and
J. Alspector, editors, *Advances in Neural Information
Processing Systems 6 (NIPS*93)*, pp1109-1116, Morgan
Kaufmann, San Mateo, CA, 1994, 8 pages: gzipped postscript
(29K).

1993

Tony A. Plate, (1993) "Holographic recurrent networks",
In C. L. Giles, S.J. Hanson, and J. D. Cowan, editors,
*Advances in Neural Information Processing Systems 5
(NIPS*92)*, pp34-41, Morgan Kaufmann, San Mateo, CA,
1993, 8 pages: gzipped
postscript (42K).

1992

Tony A. Plate (1991), "Holographic Reduced Representations: Convolution
algebra for compositional distributed representations." In P. Mehra and
B.~W. Wah, editors. *Artificial neural networks: concepts and theory*.
IEEE Computer Society Press Tutorial, 1992, 6 pages. (Note: this is a reprint
of IJCAI91 paper.) gzipped postscript
(54K).

1991

Tony A. Plate (1991), "Holographic Reduced Representations: Convolution
algebra for compositional distributed representations", In John Mylopoulos
and Ray Reiter, editors, *Proceedings of the 12th International Joint
Conference on Artificial Intelligence* (IJCAI), pp30-35, Morgan Kaufmann,
San Mateo, CA, 1991, 6 pages. gzipped
postscript (54K) PDF (216K).

1990

Yorick A. Wilks, Dan C. Fass, Cheng ming Guo, James E. McDonald, Tony Plate, and Brian M. Slator (1990), "Providing machine tractable dictionary tools", Machine Translation, 5: pp99-154.

James E. McDonald, Tony A. Plate, and Roger W. Schvaneveldt (1990), "Using pathfinder to extract semantic information from text", In Roger W. Schvaneveldt, editor, "Pathfinder Associative Networks", Ablex, New Jersey.

Software

*vsa*, R package for HRR calculations and support for adding other Vector Symbolic Architecture schemes. ("R" is an open-source implementation of the S language for statistical analysis)

- R package (updated 2009): vsa_0.1.tar.gz

*vcalc1.0*, for a fast HRR calculator, with various examples.

- Unix C sources (updated 2009): compressed tar archive

*tap.printf*, Java package implementing
printf-like functionality. Two ways of getting it:

- Unix compressed tar archive tapprintf_tar.gz (71Kb)
- Windows zip archive: tapprintf.zip (118Kb).

*ploteff: *Splus methods for visualizing non-linear functions (e.g.,
neural networks, Gaussian process models, and bagged trees) through plots
of "additive effects". This software implements the visualization techniques
described in the 1999 *Neural Computation* paper and in CS-TR-98-5
and used in the *Behaviormetrika* paper. Available as Unix
shar file (64 Kbytes) OR ploteff.zip
(18 Kbytes). Also available at The
Statlib Archives.

Tony Plate (

Last modified 2000-11-16 21:49:04 tap