Joint source/channel coding for discrete memoryless channels: Lessons to learn

Abstract

The design of optimal joint source/channel coding and decoding is examined for discrete memoryless channels with end-to-end distortion as the criterion for reliable communication. Joint source/channel encoders which map sequences of source symbols directly to sequences of channel symbols without any intermediate "bit" representation of source are considered. Optimum joint source/channel decoder that minimizes end-to-end distortion for a given encoder mapping is derived. The encoder mapping can be many to one, in the sense that many source sequences can be mapped to one sequence of channel symbols. To begin with, as an exercise, random coding bound on end-to-end distortion is derived for a general Maximum A Posteriori (MAP) decoder which has some estimate on the apriori probabilities of source symbols. It is shown that, the KL distance of the actual apriori probabilities with the estimated ones plays an important role. Then, a random coding bound on end-to-end distortion is derived with our optimal minimum distortion decoder mentioned above for the case when all source symbols are equally likely. It is shown that the performance increase with minimum distortion decoding as opposed to MAP (same as Maximum Likelihood (ML) decoding in this case when all source symbols are equally likely) is characterized by the faster decay of end-to-end distortion with respect to channel use.