### Available:*

Library | Item Barcode | Call Number | Material Type | Item Category 1 | Status |
---|---|---|---|---|---|

Searching... | 30000010226229 | Q360 J68 2000 | Open Access Book | Book | Searching... |

Searching... | 30000010226230 | Q360 J68 2000 | Open Access Book | Book | Searching... |

### On Order

### Summary

### Summary

As this Preface is being written, the twentieth century is coming to an end. Historians may perhaps come to refer to it as the century of information, just as its predecessor is associated with the process of industrialisation. Successive technological developments such as the telephone, radio, television, computers and the Internet have had profound effects on the way we live. We can see pic- tures of the surface of Mars or the early shape of the Universe. The contents of a whole shelf-load of library books can be compressed onto an almost weight- less piece of plastic. Billions of people can watch the same football match, or can keep in instant touch with friends around the world without leaving home. In short, massive amounts of information can now be stored, transmitted and processed, with surprising speed, accuracy and economy. Of course, these developments do not happen without some theoretical ba- sis, and as is so often the case, much of this is provided by mathematics. Many of the first mathematical advances in this area were made in the mid-twentieth century by engineers, often relying on intuition and experience rather than a deep theoretical knowledge to lead them to their discoveries. Soon the math- ematicians, delighted to see new applications for their subject, joined in and developed the engineers' practical examples into wide-ranging theories, com- plete with definitions, theorems and proofs.

### Reviews 1

### Choice Review

This attractive book by G. Jones (Univ. of Southampton, UK) and J. Jones (Open Univ., UK) presents two closely related subjects usually taught separately. Information theory, the invention of Claude Shannon, is fundamental to the modern communications industry. So is coding theory, which treats how information is transformed before and after transmission. Topics from information theory discussed here are the fundamental concepts of communication channel, information, and entropy. The most important result is Shannon's fundamental theorem on channel capacity. (The proof is in an appendix.) Most of these ideas are discussed in the context of the binary symmetric channel only. Topics from coding theory here revolve around the problem of correcting transmission errors. Hamming distance is introduced, as well as various bounds on code size, and the Hamming and Golay linear codes. There is no treatment of data compression or cryptography, the two other major branches of coding theory. Index of symbols; solutions to all problems. Information and Coding Theory is written in a clear but informal style, an excellent book for upper-division undergraduates through faculty as well as readers with the necessary background in linear algebra and probability theory. M. Henle; Oberlin College

### Table of Contents

Preface | p. v |

Notes to the Reader | p. xiii |

1. Source Coding | p. 1 |

1.1 Definitions and Examples | p. 1 |

1.2 Uniquely Decodable Codes | p. 4 |

1.3 Instantaneous Codes | p. 9 |

1.4 Constructing Instantaneous Codes | p. 11 |

1.5 Kraft's Inequality | p. 13 |

1.6 McMillan's Inequality | p. 14 |

1.7 Comments on Kraft's and McMillan's Inequalities | p. 16 |

1.8 Supplementary Exercises | p. 17 |

2. Optimal Codes | p. 19 |

2.1 Optimality | p. 19 |

2.2 Binary Huffman Codes | p. 22 |

2.3 Average Word-length of Huffman Codes | p. 26 |

2.4 Optimality of Binary Huffman Codes | p. 27 |

2.5 r-ary Huffman Codes | p. 28 |

2.6 Extensions of Sources | p. 30 |

2.7 Supplementary Exercises | p. 32 |

3. Entropy | p. 35 |

3.1 Information and Entropy | p. 35 |

3.2 Properties of the Entropy Function | p. 40 |

3.3 Entropy and Average Word-length | p. 42 |

3.4 Shannon-Fano Coding | p. 45 |

3.5 Entropy of Extensions and Products | p. 47 |

3.6 Shannon's First Theorem | p. 48 |

3.7 An Example of Shannon's First Theorem | p. 49 |

3.8 Supplementary Exercises | p. 51 |

4. Information Channels | p. 55 |

4.1 Notation and Definitions | p. 55 |

4.2 The Binary Symmetric Channel | p. 60 |

4.3 System Entropies | p. 62 |

4.4 System Entropies for the Binary Symmetric Channel | p. 64 |

4.5 Extension of Shannon's First Theorem to Information Channels | p. 67 |

4.6 Mutual Information | p. 70 |

4.7 Mutual Information for the Binary Symmetric Channel | p. 72 |

4.8 Channel Capacity | p. 73 |

4.9 Supplementary Exercises | p. 76 |

5. Using an Unreliable Channel | p. 79 |

5.1 Decision Rules | p. 79 |

5.2 An Example of Improved Reliability | p. 82 |

5.3 Hamming Distance | p. 85 |

5.4 Statement and Outline Proof of Shannon's Theorem | p. 88 |

5.5 The Converse of Shannon's Theorem | p. 90 |

5.6 Comments on Shannon's Theorem | p. 93 |

5.7 Supplementary Exercises | p. 94 |

6. Error-correcting Codes | p. 97 |

6.1 Introductory Concepts | p. 97 |

6.2 Examples of Codes | p. 100 |

6.3 Minimum Distance | p. 104 |

6.4 Hamming's Sphere-packing Bound | p. 107 |

6.5 The Gilbert-Varshamov Bound | p. 111 |

6.6 Hadamard Matrices and Codes | p. 114 |

6.7 Supplementary Exercises | p. 118 |

7. Linear Codes | p. 121 |

7.1 Matrix Description of Linear Codes | p. 121 |

7.2 Equivalence of Linear Codes | p. 127 |

7.3 Minimum Distance of Linear Codes | p. 131 |

7.4 The Hamming Codes | p. 133 |

7.5 The Golay Codes | p. 136 |

7.6 The Standard Array | p. 141 |

7.7 Syndrome Decoding | p. 143 |

7.8 Supplementary Exercises | p. 146 |

Suggestions for Further Reading | p. 149 |

Appendix A. Proof of the Sardinas-Patterson Theorem | p. 153 |

Appendix B. The Law of Large Numbers | p. 157 |

Appendix C. Proof of Shannon's Fundamental Theorem | p. 159 |

Solutions to Exercises | p. 165 |

Bibliography | p. 191 |

Index of Symbols and Abbreviations | p. 195 |

Index | p. 201 |