Claude Shannon in his classic 1949 paper, Communication theory of secrecy systems, (available in pdf from here) introduced the concepts of confusion and diffusion. To this day, these are the guiding principles for the design of cryptographic algorithms.

Roughly speaking, confusion obscures the relationship between the plaintext and ciphertext, while diffusion spreads the plaintext statistics through the ciphertext. A one-time pad relies entirely on confusion while a simple substitution cipher is another (weak) example of a confusion-only cryptosystem. A double transposition is the classic example of a diffusion-only cryptosystem.

Note that confusion alone is, apparently, "enough", since the one-time pad is provably secure. But diffusion alone is, perhaps, not enough, at least using relatively small blocks. A stream cipher is simply a weaker version of a one-time pad and hence stream ciphers employ only confusion.

Modern block ciphers employ both confusion and diffusion. The codebook aspects of such systems provide confusion analogous to---though on a much grander scale---a simple substitution. Well-designed block ciphers spread any local statistics throughout the block, thus employing the principle of diffusion.

Where do public-key cryptosystems fit in? That is a very good question.