By a communication system we will mean a system of the type indicated schematically in Fig. 1. It consists of essentially five parts:
- An information source which produces a message or sequence of messages (letters, functions of time and space, etc.) to be communicated to the receiving terminal.
- A transmitter which operates on the message in some way (change, encode, sample, compress, quantize, interleave, construct, etc.) to produce a signal suitable for transmission.
- The channel is merely the medium used to transmit the signal from transmitter to receiver (a pair of wires, a coaxial cable, a band of radio frequencies, a beam of light, etc.).
- The receiver ordinarily performs the inverse operation of that done by the transmitter, reconstructing the message from the signal.
- The destination is the person (or thing) for whom the message is intended.
A Mathematical Theory of Communication
ReplyDeleteby C. E. Shannon
http://cm.bell-labs.com/cm/ms/what/shannonday/shannon1948.pdf
(Reprinted with corrections from The Bell System Technical Journal, Vol. 27, pp. 379–423, 623–656, July, October, 1948.)
In 1948, Shannon wrote the following:
ReplyDeleteIf the number of messages in the set is finite then this number or any monotonic function of this number can be regarded as a measure of the information produced when one message is chosen from the set, all choices being equally likely.
In the case of a discrete source of information we were able to determine a definite rate of generating information, namely the entropy of the underlying stochastic process. With a continuous source the situation is considerably more involved. In the first place a continuously variable quantity can assume an infinite number of values and requires, therefore, an infinite number of binary digits for exact specification. This means that to transmit the output of a continuous source with exact recovery at the receiving point requires, in general, a channel of infinite capacity (in bits per second). Since, ordinarily, channels have a certain amount of noise, and therefore a finite capacity, exact transmission is impossible.
The Information
ReplyDeleteby James Gleick
http://www.kushima.org/is/?p=1921
http://www.kushima.org/is/?p=1553