USRE34965E - Inter-frame predictive encoding system with encoded and transmitted prediction error - Google Patents
- ️Tue Jun 13 1995
.Iadd.This application is a reissue of Ser. No. 07/465,747, now U.S. Pat. No. 4,985,768. .Iaddend.
This invention relates to an inter-frame predictive encoding system for a video signal.
In a general video signal, display information of a frame tends to correlate with display information of a subsequent frame and a preceding frame. In the transmission of a digitized video signal, such a correlation between successive frames is used in reducing a digital data transmission rate.
Inter-frame predictive encoding uses the correlation between successive frames to reduce the digital data transmission rate. The inter-frame predictive encoding is generally effective in handling a video signal representing moving pictures.
As will be explained later, a prior-art inter-frame predictive encoding system has some problems.
SUMMARY OF THE INVENTIONIt is an object of this invention to provide an improved inter-frame predictive encoding system.
According to a first aspect of this invention, an inter-frame predictive encoding system comprises means for setting reference frames separated at equal intervals, the reference frames being selected from successively inputted frames of a video signal; means for encoding each of the reference frames; means for generating a prediction signal for a dependent frame between the reference frames on the basis of a signal of the reference frames which precedes and follows the dependent frame respectively; means for predicting a signal of the dependent frame on the basis of the prediction signal corresponding thereto; and means for encoding an error of said predicting of the signal of the dependent frame.
According to a second aspect of this invention, an inter-frame predictive encoding system comprises means for separating successively-inputted frames of a video signal into reference frames and dependent frames, the reference frames being spaced at predetermined intervals, the dependent frames residing between the reference frames; means for encoding display information of the reference frames; means for predicting display information of each dependent frame on the basis of display information of reference frames which precede and follow said dependent frame respectively; means for generating an error between the predicted display information and corresponding actual display information of said dependent frame; and means for encoding said error.
According to a third aspect of this invention, an inter-frame predictive encoding system comprises means for performing motion-compensated inter-frame prediction using motion vectors between successively inputted frames of a video signal; the improvement comprising means for setting specified frames separated at equal intervals, the specified frames being selected from successively inputted frames of a video signal; means for detecting motion vectors between the specified frames; means for detecting motion vectors between adjacent non-specified frames between the specified frames as differences with respect to said motion vectors between the specified frames.
BRIEF DESCRIPTION OF THE DRAWINGFIG. 1 is a block diagram of a conventional inter-frame predictive encoding system.
FIG. 2 is a block diagram of a conventional decoding system.
FIG. 3 is a diagram of a stream of successive frames in an inter-frame predictive encoding system and a decoding system according to a first embodiment of this invention.
FIG. 4 is a block diagram of the inter-frame predictive encoding system according to the first embodiment of this invention.
FIG. 5 is a diagram showing the relation between prediction values and successive frames in the encoding system of FIG. 4.
FIG. 6 is a block diagram of a decoding system according to the first embodiment of this invention.
FIG. 7 is a block diagram of an inter-frame predictive encoding system according to a second embodiment of this invention.
FIG. 8 is a block diagram of an inter-frame predictive encoding system according to a third embodiment of this invention.
FIG. 9 is a block diagram of the motion vector detector of FIG. 8.
FIG. 10 is a block diagram of the vector difference detector of FIG. 8.
FIG. 11 is a block diagram of a circuit for generating a control signal fed to the switches of FIG. 4.
FIG. 12 is a timing diagram of signals generated in the circuit of FIG. 11.
DESCRIPTION OF THE PRIOR ARTFIG. 1 shows a conventional inter-frame predictive encoding system. With reference to FIG. 1, a digitized video signal is fed to a
subtracter2 via an
input terminal1. The
subtracter2 subtracts a prediction digital signal from the digitized video signal to generate a prediction error (difference) signal. The prediction signal will be explained later.
The prediction error signal outputted from the
substracter2 is subjected to an orthogonal transform by an
orthogonal transform device3. The orthogonal transform increases the efficiency of encoding. The output signal from the
orthogonal transform device3 is quantized by a
quantizer4. The output data from the
quantizer4 is encoded by an encoder S into variable-length codes such as Huffman codes. The codes outputted from the encoder S are transmitted via an
output terminal6.
The output data from the
quantizer4 is subjected to an inverse quantization by an
inverse quantizer7 so that it is converted into a representative (setting of a representative). The representative outputted from the
inverse quantizer7 is subjected to an inverse orthogonal transform by an
inverse transform device8. The output signal from the
inverse transform device8 corresponds to a prediction error (difference) which is generated through a decoding process in a decoding system explained later. An
adder9 adds the prediction error and a one-frame preceding prediction signal, generating a digitized video signal which corresponds to a digitized video signal generated by a decoding process in the decoding system. A
frame memory10 delays the output signal from the
adder9 by a time corresponding to a period of one frame. The output signal from the
frame memory10 is passed through a spatial low pass filter (a spatial LPF) 11, being converted into the prediction signal. The prediction signal is fed from the
spatial LPF11 to the
subtracter2 and the
adder9 via a
switch12. The
spatial LPF11 multiples the output data from the
adder9 by a coefficient which varies with a spatial frequency. The
spatial LPF11 ensures that a quantization error remains in the prediction error at a reduced rate. The
spatial LPF11 is effective and advantageous since a larger quantity of quantization errors are present in a high-frequency range and an inter-frame correlation is weakened by noises in the high-frequency range.
The
switch12 periodically couples and uncouples the
spatial LPF11 to and from the
subtracter2 and the
adder9 to periodically reset the inter-frame prediction. The period of the resetting the inter-frame prediction is generally set so as to correspond to 30 to 100 frames. When the inter-frame prediction is being reset, the prediction signal is fixed and essentially intra-frame coding is performed. The resetting prevents calculation errors from accumulating up to an unacceptable level. The calculation error would result from error codes generated in a transmission line and mismatching between orthogonal converters of a transmitter side and a receiver side in recursive-type inter-frame predictive en coding such as shown in FIG. 1. A shorter period of the resetting enables calculating errors from less accumulating but decreases the efficiency of the encoding.
FIG. 2 shows a conventional decoding system designed for the combination with the encoding system of FIG. 1. With reference to FIG. 2, variable-length digital data is fed to a
decoder22 via an
input terminal21. The
decoder22 converts the variable-length data into original fixed-length data. The output data from the
decoder22 is subjected to an inverse quantization by an
inverse quantizer23 so that it is converted into a representative (setting of a representative). The representative outputted from the
inverse quantizer23 is subjected to an inverse orthogonal transform by an
inverse transform device24. The output signal from the
inverse transform device24 corresponds to a prediction error (difference). An
adder25 adds the prediction error and a one-frame preceding prediction signal, generating a digitized video signal. A
frame memory27 delays the output signal from the
adder25 by a time corresponding to a period of one frame. The output signal from the
frame memory27 is passed through a spatial low pass filter (a spatial LPF) 28, being converted into the prediction signal. The prediction signal is fed from the
spatial LPF28 to the
adder25. The
spatial LPF28 is similar to the
spatial LPF11 of the encoding system.
The conventional encoding system of FIG. 1 and the conventional decoding system of FIG. 2 have problems as follows. In the case where these conventional systems are used for recording and reproducing data into a storage medium such as an information recording disk or an information recording tape, a short period of resetting the inter-frame prediction is necessary to decode data from arbitrary storage locations of the recording medium during a random access or a search. Especially, a visual search requires a very short period of resetting the inter-frame prediction since the decoding is performed at intervals of several frames. The short period of resetting the inter-frame prediction decreases the efficiency of encoding.
In the case of reverse play back from the recording medium, the predictive decoding is impossible since the direction of a time axis is opposite to the direction of a time axis which occurs during the encoding and recording.
Since the inter-frame prediction uses the preceding frame but does not use the subsequent frame, the accuracy and efficiency of the prediction tends to be inadequate. When a picture represented by data changes greatly or when a scene represented by data changes, the inter-frame prediction tends to be unreliable.
Since it is necessary for the encoding system to also perform the decoding process, the structure of the encoding system tends to be complicated.
DESCRIPTION OF THE FIRST PREFERRED EMBODIMENTIn an inter-frame predictive encoding system and an inter-frame predictive decoding system according to a first embodiment of this invention, as shown in FIG. 3, frames are separated into independent frames (denoted by hatched squares) and dependent frames (white squares). The independent frames are also referred to as the reference frames. On a time axis, the independent frames are separated at predetermined equal intervals corresponding to two or more frames. A predetermined number of dependent frames are present between two adjacent independent frames. Display information in an independent frame is encoded independent of other frames. Display information in a dependent frame is encoded by prediction using the preceding and subsequent independent frames.
FIG. 4 shows an inter-frame predictive encoding system according to a first embodiment of this invention. With reference to FIG. 4, a digitized video signal is fed to a
movable contact37c of a
switch37 via an
input terminal1. The
movable contact37c of the
switch37 periodically changes between a first position and a second position at a predetermined period. When the
movable contact37c of the
switch37 assumes the first position, it contacts with a first
fixed contact37a of the
switch37 and separates from a second
fixed contact37b of the
switch37. When the
movable contact37c of the
switch37 assumes the second position, it contacts with the second
fixed contact37b of the
switch37 and separates from the first
fixed contact37a of the
switch37. As will be made clear hereinafter, when the
movable contact37c of the
switch37 contacts with the first
fixed contact37a of the
switch37, the frame represented by the current input video signal is defined as an independent frame. Otherwise, the frame represented by the current input video signal is defined as a dependent frame.
When the
movable contact37c of the
switch37 contacts with the first
fixed contact37a of the
switch37, the input video signal is transmitted through the
switch37 to a first
fixed contact38a of a
switch38. A
movable contact38c of the
switch38 is connected to an input terminal of an
orthogonal transform device3. The
movable contact38c of the
switch38 periodically contacts with and separates from the first
fixed contact38a and a second
fixed contact38b of the
switch38 in a manner and at a timing similar to those of the
switch37. Specifically, when the
movable contact37c of the
switch37 contacts with the first
fixed contact37a of the
switch37, the
movable contact38c of the
switch38 contacts with the first
fixed contact38a of the
switch38 so that the input video signal is further transmitted through the
switch38 to the
orthogonal transform device3.
When the
movable contact37c of the
switch37 contacts with the second
fixed contact37b of the
switch switch37 to a
multi-frame memory31. The
multi-frame memory31 delays the input video signal by a time corresponding to a predetermined number of frames. Specifically, the delay time of the
multi-frame memory31 corresponds to (N-1) frames in the case where one independent frame occurs per N successive frames and the letter N denotes a predetermined natural number equal to two or greater. The output video signal from the
multi-frame memory31 is fed to a
subtracter2. The
subtracter2 substracts a prediction signal from the video signal to generate a prediction error (difference) signal. The prediction signal will be explained later. The prediction error signal is outputted from the
subtracter2 to the second
fixed contact38b of the
switch38. When the
movable contact37c of the
switch37 contacts with the second
fixed contact37b of the
switch37, the
movable contact38c of the
switch38 contacts with the second
fixed contact38b of the
switch38 so that the prediction error signal is transmitted through the
switch38 to the
orthogonal transform device3.
During one-frame periods separated by equal intervals corresponding to a predetermined number of frames, that is, during periods corresponding to independent frames, the
movable contact37c of the
switch37 contacts with the first
fixed contact37a of the
switch37 while the
movable contact38c of the
switch38 contacts with the first
fixed contact38a of the
switch38. During other periods, that is, during periods corresponding to dependent frames, the
movable contact37c of the
switch37 contacts with the second
fixed contact37b of the
switch37 while the
movable contact38c of the
switch38 contacts with the second
fixed contact38b of the
switch38.
The independent-frame video signal or the dependent-frame prediction error signal outputted from the
subtracter2 is subjected to an orthogonal transform by the
orthogonal transform device3. The orthogonal transform increases the efficiency of encoding. The output signal from the
orthogonal transform device3 is quantized by a
quantizer4. The output data from the
quantizer4 is encoded by an
encoder5 into variable-length codes such as Huffman codes. The codes outputted from the
encoder5 are transmitted via an
output terminal6.
The output data from the
quantizer4 is applied to a first
fixed contact39a of a
switch39. A second fixed
contact39b of the
switch39 has no connection with other circuits. The
movable contact39c of the
switch39 is connected to an input terminal of an
inverse quantizer7. The
movable contact39c of the
switch39 periodically contacts with and separates from the first
fixed contact39a and a second
fixed contact39b of the
switch39 in a manner and at a timing similar to those of the
switch37. Specifically, when the output signal from the
quantizer4 represents an independent frame, the
movable contact39c of the
switch39 contacts with the first
fixed contact39a of the
switch39 so that the output signal from the
quantizer4 is transmitted to the
inverse quantizer7. When the output signal from the
quantizer4 represents a dependent frame, the
movable contact39c of the
switch39 contacts with the second
fixed contact39b of the
switch39 so that the transmission of the output signal from the
quantizer4 to the
inverse quantizer7 is interrupted. In this way, only the output signal from the
quantizer4 which represents an independent frame is transmitted to the
inverse quantizer7.
The independent-frame output signal from the
quantizer4 is subjected to an inverse quantization by the
inverse quantizer7 so that it is convened into a representative (setting of a representative). The representative outputted from the
inverse quantizer7 is subjected to an inverse orthogonal transform by an
inverse transform device8. The output signal from the
inverse transform device8 corresponds to a reproduced signal of an independent frame. The output signal from the
inverse transform device8, that is, the reproduced signal of an independent frame, is written into a
frame memory32.
A first fixed
contact40a of a
switch40 is connected to the output terminal of the
frame memory32. A second fixed
contact40b of the
switch40 has no connection with other circuits. A
movable contact40c of the
switch40 is connected to an input terminal of a
frame memory33. The
movable contact40c of the
switch40 periodically contacts with and separates from the first
fixed contact40a and the second
fixed contact40b of the
switch40 in a manner and at a timing similar to those of the
switch37. Specifically, when the output signal from the
inverse transform device8 which represents the current independent frame is written into the
frame memory32, the
movable contact40c of the
switch40 connects with the first
fixed contact40a of the
switch40 so that the reproduced signal of the preceding independent frame is transferred from the
frame memory32 to the
frame memory33 via the
switch40. In this way, the reproduced signal of the current independent frame and the reproduced signal of the preceding independent frame are prepared in the
frame memory32 and the
frame memory33 respectively.
The reproduced signal of the current independent frame and the reproduced signal of the preceding independent frame remain stored in the
frame memory32 and the
frame memory33 respectively until the
frame memory32 is fed with the reproduced signal of the subsequent independent frame from the
inverse transform device8. The reproduced signal of the current independent frame and the reproduced signal of the preceding independent frame are repeatedly outputted from the
frame memory32 and the
frame memory33 to
multipliers34 and 35 respectively. Specifically, the number of times of outputting the reproduced signal of the current independent frame and the reproduced signal of the preceding independent frame is equal to N-1.
The
multiplier34 multiplies the reproduced signal of the current independent frame by a weight coefficient ct and outputs the resultant to an
adder36. The
multiplier35 multiplies the reproduced signal of the preceding independent frame by a weight coefficient (1-α) and outputs the resultant to the
adder36. The
adder36 adds the resultants of the multiplications, generating a prediction signal fed to the
subtracter2.
The weight coefficients a and (1-α) are determined in accordance with the time relation between the dependent frame inputted into the
subtracter2 and the independent frames related to the prediction signal inputted into the
subtracter2. For example, a linear prediction is used in the determination of the weight coefficients a and (1-α). Specifically, the weight coefficient α is given by the following equation.
α=(m-mp)/N
where the character m denotes the order number (1, 2, 3, . . . ) of the frame being a coded object; the character mp denotes the order number (0, N, 2N, . . . ) of the previous independent frame; the number m is larger than the number mp; and the character N denotes a predetermined natural number equal to or larger than 2.
FIG. 5 shows the relation between the frame number and the level of the prediction signal (the prediction value) in the case where the number N equals 4. In the generation of the prediction signal, a larger weight is offered to the independent frame closer to the objective dependent frame while a smaller weight is offered to the other independent frame. As understood from FIG. 5, in the case where display information of dependent frames linearly varies from the display information of the preceding independent frame to the display information of the current independent frame, very accurate prediction values are presented.
In the encoding system of FIG. 4, the operation of the
switches37 and 38 determines independent frames which are separated in a time axis at equal intervals corresponding to a predetermined number of frames. In addition, dependent frames are defined between independent frames. The inter-frame correlation between coded data is cut at each independent frame. Therefore, a visual search is enabled by decoding only data of independent frames or by performing a random access in unit corresponding to independent frames.
In the encoding system of FIG. 4, the prediction signal is generated by adding the data of the two adjacent independent frames with variable weighting parameters. Specifically, the weight coefficients α and (1-α) used in the generation of the prediction signal are determined in accordance with the time relation between the dependent frame inputted into the
subtracter2 and the independent frames related to the prediction signal inputted into the
subtracter2. Therefore, the prediction can well follow the variation of display information between successive frames, and an S/N (a signal to noise ratio) of the prediction signal can be high.
In the encoding system of FIG. 4, encoded data are arranged symmetrically on a time axis so that a reverse reproduction of data from a recording medium can be realized.
The switches 37-40 are changed in response to a switch control signal. FIG. 11 shows a circuit for generating the switch control signal. As shown in FIG. 11, the switch control circuit includes a
frame sync separator501 which separates a frame sync signal from the input video signal. As shown in FIG. 12, the frame sync signal has a train of pulses. The pulses of the frame sync signal outputted from the
frame sync separator501 are counted by a
counter502. Each time four successive pulses of the frame sync signal are counted by the
counter502, the
counter502 outputs a pulse as shown in FIG. 12. The duration of each output pulse from the
counter502 agrees with the one-frame period. The sequentially-outputted pulses from the
counter502 compose the switch control signals fed to the switches 37-40.
FIG. 6 shows an inter-frame predictive decoding system according to the first embodiment of this invention. With reference to FIG. 6, variable-length digital data is fed to a
decoder22 via an
input terminal21. The
decoder22 converts the variable-length data into original fixed-length data. The output data from the
decoder22 is subjected to an inverse quantization by an
inverse quantizer23 so that it is converted into a representative (setting of a representative). The representative outputted from the
inverse quantizer23 is subjected to an reverse orthogonal transform by an
inverse transform device24. For independent frames, the output signal from the
inverse transform device24 corresponds to a reproduced video signal. For dependent frames, the output signal from the
inverse transform device24 corresponds to a prediction error signal.
The output signal from the
inverse transform device24 is applied to a
movable contact47c of a
switch47. A first fixed
contact47a of the
switch47 is connected to an input terminal of a
frame memory42. A second fixed
contact47b of the
switch47 is connected to an
adder41. The
movable contact47c of the
switch47 periodically changes between a first position and a second position at a predetermined period. When the
movable contact47c of the
switch47 assumes the first position, it contacts with the first
fixed contact47a of the
switch47 and separates from the second
fixed contact47b of the
switch47. When the
movable contact47c of the
switch47 assumes the second position, it contacts with the second
fixed contact47b of the
switch47 and separates from the first
fixed contact47a of the
switch47. When the output signal from the
inverse transform device24 represents an independent frame, the
movable contact47c of the
switch47 contacts with the first
fixed contact47a of the
switch47 so that the independent-frame signal is fed to and written into the
frame memory42. When the output signal from the
inverse transform device24 represents a dependent frame, the
movable contact47c of the
switch47 contacts with the second
fixed contact47b of the
switch47 so that the dependent-frame signal is fed to the
adder41.
The
adder41 adds the dependent-frame signal and a prediction signal, reproducing a digitized video signal of a dependent frame. The prediction signal will be explained later. The
adder41 outputs the reproduced video signal of a dependent frame to a second fixed contact 48b of a switch 48. A first fixed
contact48a of the switch 48 is connected to a first
fixed contact49a of a
switch49 which will be explained later. A
movable contact48c of the switch 48 is connected to an
output terminal26. The
movable contact48c of the switch 48 periodically contacts with and separates from the first
fixed contact48a and the second fixed contact 48b of the switch 48 in a manner and at a timing similar to those of the
switch47. Specifically, when the
adder41 outputs the reproduced video signal of a dependent frame, the
movable contact48c of the switch 48 contacts with the second fixed contact 48b of the switch 48 so that the dependent-frame video signal is transmitted from the
adder41 to the
output terminal26 via the switch 48.
As described previously, the output signal from the
inverse transform device24 which agrees with the reproduced signal of an independent frame is written into the
frame memory42. A movable contact 49c of a
switch49 is connected to the output terminal of the
frame memory42. A first fixed
contact49a of the
switch49 is connected to an input terminal of a
frame memory43. A second fixed
contact49b of the
switch49 has no connection with other circuits. The movable contact 49c of the
switch49 periodically contacts with and separates from the first
fixed contact49a and the second
fixed contact49b of the
switch49 in a manner and at a timing similar to those of the
switch47. Specifically, when the output signal from the
inverse transform device24 represents the current independent frame and is thus written into the
frame memory42, the movable contact 49c of the
switch49 connects with the first
fixed contact49a of the
switch49 so that the reproduced signal of the preceding independent frame is transferred from the
frame memory42 to the
frame memory43 via the
switch49. At the same time, the
movable contact48c of the switch 48 connects with the first
fixed contact48a of the switch 48 so that the reproduced signal of the preceding independent frame is transferred from the
frame memory42 to the
output terminal26 via the
switches48 and 49. As understood from the previous description, the reproduced signal of the current independent frame and the reproduced signal of the preceding independent frame are prepared in the
frame memory42 and the
frame memory43 respectively.
The reproduced signal of the current independent frame and the reproduced signal of the preceding independent frame remain stored in the
frame memory42 and the
frame memory43 respectively until the
frame memory42 is fed with the reproduced signal of the subsequent independent frame from the
inverse transform device24. The reproduced signal of the current independent frame and the reproduced signal of the preceding independent frame are repeatedly outputted from the
frame memory42 and the
frame memory43 to
multipliers44 and 45 respectively. Specifically, the number of times of outputting the reproduced signal of the current independent frame and the reproduced signal of the preceding independent frame is equal to N-1.
The multiplier 44 multiplies the reproduced signal of the current independent frame by a weight coefficient a and outputs the resultant to an adder 445. The
multiplier45 multiplies the reproduced signal of the current independent frame by a weight coefficient (1-α) and outputs the resultant to the adder 445. The
adder46 adds the resultants of the multiplications, generating a prediction signal fed to the
adder41. The weight coefficients α and (1-α) are determined similarly to the determination of the weight coefficients in the encoding system of FIG. 4.
The switches 47-49 are changed in response to a control signal generated by a circuit similar to the circuit of FIG. 11.
During the processing of the input video signal by the encoding system of FIG. 4, independent frames are advanced relative to dependent frames. To compensate the advance of the independent frames, the reproduced video signal of an independent signal is outputted from the
frame memory42 to the
output terminal26 when the prediction process related to dependent frames between two independent frames is completed. Accordingly, the
frame memory42 also functions to perform the time correction.
FIG. 7 shows an inter-frame predictive encoding system according to a second embodiment of this invention which is similar to the embodiment of FIG. 4 except for designs explained later.
In the encoding system of FIG. 7, the
inverse quantizer7, the
inverse transform device8, and the switch 39 (see FIG. 4) are omitted while the input terminal of the
frame memory32 is connected to the first
fixed contact37a of the
switch37.
When the input video signal of an independent frame is fed to the
orthogonal transform device3 via the
switches37 and 38, the independent-frame signal is also fed to the
frame memory32 and is written thereinto. Accordingly, the input video signal of independent frames is directly used in the generation of a prediction signal fed to the
subtracter2. It should be noted that, in the encoding system of FIG. 4, the resultant of the processing of the input video signal of an independent frame by the
devices3, 4, 7, and 8 is used in the generation of a prediction signal.
The encoding system of FIG. 7 dispenses with a decoding process. In addition, the encoding system of FIG. 7 has advantages similar to those of the encoding system of FIG. 4.
DESCRIPTION OF THE THIRD PREFERRED EMBODIMENTIn an inter-frame predictive encoding system according to a third embodiment of this invention, frames are separated into independent frames and dependent frames as in the embodiment of FIGS. 3-6.
FIG. 8 shows an inter-frame predictive encoding system according to a third embodiment of this invention. With reference to FIG. 8, a digitized video signal is fed to a movable contact 142c of a
switch142 via an
input terminal101. The movable contact 142c of the
switch142 periodically changes between a first position and a second position at a predetermined period. When the movable contact 142c of the
switch142 assumes the first position, it contacts with a first fixed
contact142a of the
switch142 and separates from a second
fixed contact142b of the
switch142. When the movable contact 142c of the
switch142 assumes the second position, it contacts with the second
fixed contact142b of the
switch142 and separates from the first fixed
contact142a of the
switch142. As will be made clear hereinafter, when the movable contact 142c of the
switch142 contacts with the first fixed
contact142a of the
switch142, the frame represented by the current input video signal is defined as an independent frame. Otherwise, the frame represented by the current input video signal is defined as a dependent frame.
When the movable contact 142c of the
switch142 contacts with the first fixed
contact142a of the
switch142, the input video signal is transmitted through the
switch142 to a first fixed
contact143a of a
switch143. A movable contact 143c of the
switch143 is connected to an input terminal of an
orthogonal transform device103. The movable contact 143c of the
switch143 periodically contacts with and separates from the first fixed
contact143a and a second fixed contact 143b of the
switch143 in a manner and at a timing similar to those of the
switch142. Specifically, when the movable contact 142c of the
switch142 contacts with the first fixed
contact142a of the
switch142, the movable contact 143c of the
switch143 contacts with the first fixed
contact143a of the
switch143 so that the input video signal is further transmitted through the
switch143 to the
orthogonal transform device103.
When the movable contact 142c of the
switch142 contacts with the second
fixed contact142b of the
switch142, the input video signal is transmitted through the
switch142 to a
multi-frame memory131. The
multi-frame memory131 delays the input video signal by a time corresponding to a predetermined number of frames. Specifically, the delay time of the
multi-frame memory131 corresponds to (N-1) frames in the case where one independent frame occurs per N successive frames and the letter N denotes a predetermined natural number equal to two or greater. The output video signal from the
multi-frame memory1;31 is fed to a
subtracter102. The
subtracter102 subtracts a prediction signal from the video signal to generate a prediction error (difference) signal. The prediction signal will be explained later. The prediction error signal is outputted from the
subtracter102 to the second fixed contact 143b of the
switch143. When the movable contact 142c of the
switch142 contacts with the second
fixed contact142b of the
switch142, the movable contact 143c of the
switch143 contacts with the second fixed contact 143b of the
switch143 so that the prediction error signal is transmitted through the
switch143 to the
orthogonal transform device103.
During one-frame periods separated by equal intervals corresponding to a predetermined number of frames, that is, during periods corresponding to independent frames, the movable contact 142c of the
switch142 contacts with the first fixed
contact142a of the
switch142 contacts while the movable contact 143c of the
switch143 contacts with the first fixed
contact143a of the
switch143. During other periods, that is, during periods corresponding to dependent frames, the movable contact 142c of the
switch142 contacts with the second
fixed contact142b of the
switch142 while the movable contact 143c of the
switch143 contacts with the second fixed contact 143b of the
switch143.
The video signal of the independent-frame or the prediction error signal of the dependent-frame outputted from the
subtracter102 is subjected to an orthogonal transform by the
orthogonal transform device103. The orthogonal transform increases the efficiency of encoding. The output signal from the
orthogonal transform device103 is quantized by a
quantizer104. The output data from the
quantizer104 is encoded by an
encoder105 into variable-length codes such as Huffman codes. The codes outputted from the
encoder105 are transmitted via an
output terminal106.
The independent-frame video signal is transmitted via the
switch142 and is written into a
frame memory132. A movable contact 144c of a
switch144 is connected to an output terminal of the
frame memory132. A first fixed
contact144a of the
switch144 is connected to an input terminal of a
frame memory133. A second fixed
contact144b of the
switch144 has no connection with other circuits. The movable contact 144c of the
switch144 periodically contacts with and separates from the first fixed
contact144a and the second
fixed contact144b of the
switch144 in a manner and at a timing similar to those of the
switch142. Specifically, when the current independent-frame signal is transmitted via the
switch142 and is written into the
frame memory132, the movable contact 144e of the
switch144 connects with the first fixed
contact144a of the
switch144 so that the preceding independent-frame signal is transferred from the
frame memory132 to the
frame memory133 via the
switch144. In this way, the current independent-frame signal and the preceding independent-frame signal are prepared in the
frame memory132 and the
frame memory133 respectively.
The current independent-frame signal and the preceding independent-frame signal remain stored in the
frame memory132 and the
frame memory133 respectively until the
frame memory132 is fed with the subsequent independent-frame signal via the
switch142. The current independent-frame signal and the preceding independent-frame signal are repeatedly outputted from the
frame memory132 and the
frame memory133 to position
shifters134 and 135 respectively. The
position shifter134 two-dimensionally shifts the current independent-frame signal by a magnitude which is determined by a
shift calculator136. Similarly, the
position shifter135 shifts the preceding independent-frame signal by a magnitude which is determined by the
shift calculator136.
The
position shifter134 includes an address generator, and a temporal memory into and from which the current independent-frame signal is written and read in accordance with an address signal from the address generator. During the reading out the signal from the temporal memory, the address generator shifts addresses relative to the write addresses in accordance with a signal from the
shift calculator136 to provide the two dimensional shift of the current independent-frame signal. The
position shifter135 is designed similarly to the
position shifter134.
An output signal from a
vector difference detector141 which represents a motion vector is inputted into the
shift calculator136. The
shift calculator136 multiplies the motion vector by (N-i), generating a magnitude of shift fed to the
position shifter134. In addition, the
shift calculator136 multiplies the motion vector by (-i), generating a magnitude of shift fed to the
position shifter135. The character i denotes the order number of a predicted frame which is determined in view of the time relation between frames. Specifically, the number i is 0 for each independent frame and varies as i=1, 2, 3, . . . , (N-i) for dependent frames.
Output signals from the
position shifters134 and 135 are fed to
multipliers137 and 138 respectively. The
multiplier137 multiplies the output signal from the
position shifter137 by a weight coefficient a and feeds the resultant to an
adder139. The
multiplier138 multiplies the output signal from the
position shifter135 by a weight coefficient (1-α) and feeds the resultant to the
adder139. The
adder139 adds the resultants of the multiplications, generating a prediction signal fed to the
subtracter102. For example, according to a linear prediction, the weight coefficient α is defined as i/N.
The current independent-frame video signal is transmitted via the
switch142 to a
motion vector detector140. The preceding independent-frame video signal is fed from the
frame memory132 to the
motion vector detector140. The
motion vector detector140 detects motion vectors from the current independent-frame signal and the preceding independent-frame signal in a known way such as a block matching method. The
motion vector detector140 outputs a signal representative of the detected motion vectors to the
vector difference detector141.
As shown in FIG. 9, the
motion vector detector140 includes
RAMs153 and 154 receiving the current independent-frame signal and the preceding independent frame signal via
input terminals151 and 152 respectively. The current independent-frame signal and the preceding independent-frame signal are written into the
RAMs153 and 154 respectively. Each of the signals stored in the
RAMs153 and 154 is divided into spatial regions whose number is greater than the number of spatial regions corresponding to blocks of 8×8 or 16×16 pixels which are used in determining motion vectors.
Since motion vectors are used for dependent frames between the current independent frame and the preceding independent frame, it is necessary to handle the two independent frames similarly. Therefore, the two frames are moved symmetrically during the determination of motion vectors.
During the writing of the signal into each of the
RAMs153 and 154, addresses fed to each of the
RAMs53 and 154 are directly made by main addresses of pixels in blocks. During the reading of the signal from each of the
RAMs153 and 154, addresses fed to each of the
RAMs153 and 154 are modified by a value of motion vectors which are generated in a vector generator 55. The writing of the signal into each of the
RAMs53 and 154 is performed once for each independent frame, while the reading of the signal from each of the
RAMs153 and 154 is repeatedly performed a number of times which equals the number of motion vectors. The address shift is composed of a vector which is generated by a
vector generator155 in correspondence with intervals of N frames. For example, the
vector generator155 includes a counter.
Specifically, addresses to the
RAM153 for the current independent frame are generated by adding the vector output from the
vector generator155 to an output signal from a
main address generator156. This addition is performed by an
adder157. For example, the
main address generator156 includes a counter. Addresses to the
RAM154 for the preceding independent frame are generated by subtracting the vector output of the
vector generator155 from an output signal of the
main address generator156. This subtraction is performed by a
subtracter158.
A
subtracter159 calculates a difference between the output data from the
RAMs153 and 154. A squaring
device160 calculates the square of the data difference. An
integrator161 accumulates the output data from the squaring
device160 during an interval corresponding to a period of one block. The
integrator161 obtains a mean square error for a setting vector.
The section corresponding to a part surrounded by the broken line of FIG. 9 includes the
devices153, 154, 156, 157, 158, 159, 160, and 161. This section performs a process of detecting a mean square error. For each setting vector in each block, the detecting process is repeatedly performed a number of times which equals the number of pixels within one block.
The mean square errors of respective vectors are sequentially fed from the
integrator161 to a
vector selector162. The
vector selector162 detects the smallest among the mean square errors and outputs the motion vector corresponding to the selected smallest mean square error as a motion vector V' representing a motion corresponding to each of blocks of one frame. For example, the
vector selector162 includes a smallest value selector of the serial input type which is composed of a combination of latches and comparators. The; motion vector V' is generated for each block. The motion vectors of one frame are stored into a
frame memory163. During the prediction process for dependent frames between two adjacent independent frames, the motion vectors are repeatedly read out from the
frame memory163 and are transmitted via an
output terminal164 to the
vector difference detector141.
Returning to FIG. 8, the
vector difference detector141 performs a process of calculating differences between vectors in each predicted frame. As will be made clear hereinafter, some circuit components are common to the
vector difference detector141 and the
motion vector detector140.
The
vector difference detector141 is fed with the dependent-frame signal from the
multi-frame memory131. A first fixed
contact145d and a second fixed
contact145e of a
switch145 are connected to the output terminals of the
frame memories132 and 133 respectively. A
movable contact145f of the
switch145 is connected to the
vector difference detector141. The
movable contact145f of the
switch145 periodically changes between a first position and a second position at a predetermined period. When the
movable contact145f of the
switch145 assumes the first position, it contacts with the first fixed
contact145d of the
switch145 and separates from the second fixed
contact145e of the
switch145 so that the preceding independent-frame signal is fed from the
frame memory133 to the
vector difference detector141 via the
switch145. When the
movable contact145f of the
switch145 assumes the second position, it contacts with the second fixed
contact145e of the
switch145 and separates from the first fixed
contact145d of the
switch145 so that the current independent-frame signal is fed from the
frame memory132 to the
vector difference detector141 via the
switch145. In this way, the current independent-frame signal and the preceding independent-frame signal are alternately fed to the
vector difference detector141.
Specifically, the
switch145 is changed in response to the order number of a frame so that one of the current independent-frame signal and the preceding independent-frame signal is alternately and periodically selected and fed to the
vector difference detector141. Selected one of the current independent-frame signal and the preceding independent-frame signal relates to a larger weight in the predictive calculation.
As shown in FIG. 10, the
vector difference detector141 includes the
RAMs153 and 154 receiving the dependent-frame signal and the independent-frame signal via the
input terminals151 and 152 respectively. The dependent-frame signal and the independent-frame signal are written into the
RAMs153 and 154 respectively.
The motion vector V' determined between the current independent frame and the preceding independent frame is transmitted from the
motion vector detector140 to an
adder167 via an
input terminal165. The
adder167 adds the motion vector V' and an output value vd from a
vector difference generator166, generating a resultant vector. For example, the
vector difference generator166 includes a counter.
A known detection of motion vectors use a method in which first vectors are set at a low distribution density and then second vectors are set around the first vectors at a high distribution density. The output value vd from the
vector difference generator166 corresponds to a vector difference determined in respect of setting the second vectors in the known method.
A
multiplier168 multiplies the resultant vector by (-i) for the preceding independent frame. The
multiplier168 multiplies the resultant vector by (N-i) for the current independent frame. The character i denotes the order number of a predicted frame which is determined in view of the time relation between frames. Specifically, the number i is 0 for each independent frame and varies as i=1, 2, 3, . . . , (N-i) for dependent frames. The output signal from the
multiplier168 represents an address shift.
During the writing of the signal into each of the
RAMs153 and 154, addresses fed to each of the
RAMs153 and 154 are directly made by main addresses of pixels in blocks. During the reading of the signal from the
RAM153, addresses fed to the
RAM153 are also directly made by the main addresses. During the reading of the signal from the
RAM154, addresses fed to the
RAM154 are made by adding an address shift to the main addresses.
Specifically, reading addresses to the
RAM154 for the independent frame are generated by adding the address shift from the
multiplier168 to the output signal from the
main address generator156. This addition is performed by an
adder169. The output signal from the
main address generator156 are fed to the
RAM153 as reading and writing addresses.
The
subtracter159 calculates a difference between the output data from the
RAMs153 and 154. The squaring
device160 calculates the square of the data difference. The
integrator161 accumulates the output data from the squaring
device160 during an interval corresponding to a period of one block. The
integrator161 obtains a mean square error for a setting vector.
The mean square errors of respective vectors are sequentially fed from the
integrator161 to the
vector selector162. The
vector selector162 detects the smallest among the mean square errors and outputs the motion vector corresponding to the detected smallest mean square error as a motion vector dV' representing a motion corresponding to each of blocks of one frame. The motion vector dV' is generated for each block. An
adder170 adds the motion vector dV' and the motion vector V', generating a final motion vector V. The final motion vector V outputted from the
adder170 is transmitted via the
output terminal164 to the
shift calculator136 and an encoder 113 (see FIG. 8). The output data from the
vector difference detector141 is encoded by the
encoder113 into variable-length codes such as Huffman codes.
The motion vector V' or the motion vector dV' is selected from 9 different types while the final motion vector V is selected from 49 different vectors.
The processing by the
motion vector detector140 and the processing by the
vector difference detector141 are performed during different periods respectively. This design enables reliable operation of the
detectors140 and 141 although the circuit components are common to the
detectors140 and 141. It should be noted that the circuit components of the
detector140 may separate from the circuit components of the
detector141.
As understood from the previous description, in this invention, a signal of a frame is coded by referring to signals of frames which precede and follow that frame. This function enables the present invention to produce new advantages such as an increase in the efficiency of encoding.
While the reference frames (the independent frames) are independently coded in the previously-mentioned embodiments, other designs may be adopted. For example, the reference frames may be coded by use of the correlation between the reference frames to further increase the efficiency of encoding.