Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
METHOD FOR RESILIENT DECODING OF IMAGE SEQUENCES
Document Type and Number:
WIPO Patent Application WO/2024/103127
Kind Code:
A1
Abstract:
Methods for error resilient decoding of independently encoded video frames are disclosed which do not require a frame buffer to store the most reliable decoded values for each sample locaon. Instead, the methods described require only the buffering of original encoded code-blocks within a reference buffer. Original encoded blocks of a previous frame are accessed from a reference buffer. These blocks are decoded to retrieve sample values for the block from the previous frame. The decoded sample values from a current block and a corresponding reference block are paroned into cells having one or one correspondence. The corresponding cells are compared to produce an inial classificaon label for each cell, the classificaon label having one of at least 3 values, which idenfy the cell as ambiguous, consistent or inconsistent. Cells inially labelled as ambiguous or inconsistent are then resolved to consistent using labels assigned to other cells.

Inventors:
TAUBMAN DAVID SCOTT (AU)
NAMAN AOUS (AU)
Application Number:
PCT/AU2023/051174
Publication Date:
May 23, 2024
Filing Date:
November 17, 2023
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
KAKADU R & D PTY LTD (AU)
International Classes:
H04N19/66; H04N19/00; H04N19/119; H04N19/122; H04N19/124; H04N19/134; H04N19/169; H04N19/507; H04N19/593; H04N19/61
Foreign References:
JP2011082725A2011-04-21
US20060188025A12006-08-24
EP1806931A22007-07-11
Attorney, Agent or Firm:
GRIFFITH HACK (AU)
Download PDF:
Claims:
CLAIMS What is claimed is: 1. A method for resilient decoding of a sequence of video frames, each having been transformed to produce a plurality of sample blocks, that have been independently encoded, the encoded sample blocks being identified herein as code-blocks, the method involving a reference buffer that is capable of selectively storing previously received code-blocks from the encoded video sequence, where a code-block within a given frame, identified here as the current code-block of the current frame, is processed using the following steps: a) decoding the current code-block to produce a current block of sample values; b) obtaining a corresponding reference block of decoded sample values for a previous frame in the video sequence by accessing the reference buffer and performing any required decoding operations to obtain said reference samples; c) partitioning each of the current and reference blocks of decoded sample values into one or more cells, each cell of the current block being in one to one correspondence with a cell of the reference block; d) comparing corresponding cells of the current and reference block to produce an initial classification label for each cell pair, the label having at least three possible values that identify the cell as inconsistent, ambiguous or consistent; and e) resolving cell pairs initially labelled as ambiguous to consistent or inconsistent by a label adjustment step that uses the labels assigned to other cells. 2. The method of Claim 1, wherein the initial classification label for a cell pair has four possible values that identify the cell as one of: a) inconsistent; b) ambiguous; c) consistent, with a preference for using the reference cell in representations of the current frame; d) and consistent, with a preference for using the current cell in representations of the current frame. 3. The method of Claim 1 or Claim 2, wherein comparing corresponding cells is based on a comparison of the quantization intervals associated with each sample in the current and reference cell, such that: a) the cell pair is labelled inconsistent if the quantization interval associated with at least one decoded sample in the current cell is inconsistent with the quantization interval associated with the corresponding sample in the reference cell, where inconsistency here means that the quantization intervals do not intersect; b) the cell pair is labelled as ambiguous if there are no corresponding sample pairs that are non-zero in both the current and the reference cell; and c) the cell pair is otherwise assigned a consistent label. 4. The method of Claim 3 where dependent on Claim 2, wherein preference for using the reference cell or preference for using the current cell, in representations of the current frame, is determined based on information identifying which of the current and reference cells is more finely quantized. 5. The method of Claim 2, wherein comparing corresponding cells is based on: 30 20386987_2 (GHMatters) P120441.PCT a) a distortion reduction value that assesses the maximum amount by which quantization distortion can be reduced within the more finely quantized of the two cells, over sample locations for which the current and reference quantization intervals are consistent and both current and reference samples are non-zero; b) a distortion enlargement value that assesses the distortion between the current and reference cells over sample locations for which the current and reference quantization intervals are inconsistent; and c) an energy value that accumulates an energy statistic, equivalent to distortion between decoded sample values and 0, over all samples of the current and reference cells that are included in the distortion reduction and distortion enlargement value calculations; where the cell pair is labelled inconsistent if the distortion enlargement term exceeds a specified fraction of the energy value, ambiguous if the energy value is 0; else consistent, where preference for using the reference cell or preference for using the current cell, in representations of the current frame, is determined based on a comparison between the distortion reducing and distortion enlargement values, together with information identifying which of the current and reference cells is more finely quantized. 6. The method of any one of Claims 1 to 5, wherein the code-blocks are independently coded blocks of samples from sub-bands produced by discrete wavelet transformation of the frame sample values. 7. The method of any one of Claims 1 to 6, wherein ambiguity in the classification label assigned to a cell pair is resolved with the aid of temporal inconsistency tracking information, which identifies whether or not any temporal change has been detected over the cell’s region of support, in any frame following the one that is associated with the cell’s reference code-block, such that the ambiguous label is adjusted to an inconsistent label if such temporal change has been detected. 8. The method of Claim 7, wherein cells with a similar region of influence in the video frame, identified as co-located cells, are collected in groups and a co-inconsistency step modifies temporal inconsistency tracking information so as to identify temporal change for one cell within a group based on the detection of novel temporal inconsistency in any cell within the same group, where novel temporal inconsistency in a cell means that temporal change is detected for the first time in that cell since the frame associated the cell’s reference code-block. 9. The method of Claim 7 or Claim 8, wherein temporal inconsistency tracking is performed by assigning a binary state variable to cells in a frame, where the binary state variable for a cell is set to a prescribed value, its active state, after temporal change is identified, and is reset to the other value, its inactive state, when the reference buffer is updated so that future frames will employ a new reference code- block for the cell. 10. The method of Claim 8, wherein temporal inconsistency tracking information is performed jointly for all of the co-located cells within a group, using a single state 31 20386987_2 (GHMatters) P120441.PCT variable that serves to identify the most recent frame in which novel temporal inconsistency has been detected within any cell of the group. 11. The method of any one of Claims 8 to 10, wherein a co-consistency step adjusts the classification label of a cell from ambiguous to consistent, if the temporal inconsistency tracking information for the cell does not identify temporal change since the frame associated with the cell’s reference code-block, and another co-located cell in the same group, whose reference code-block comes from the same or an earlier frame, is labelled as consistent. 12. The method of any one of Claims 7 to 11, where a cell belonging to a lower resolution sub-band of a frame, whose region of influence in the video frame overlaps that of a cell from a higher resolution sub-band, is identified as an ancestor of the higher resolution cell, and an inter-consistency step adjusts the classification label of the higher resolution cell from ambiguous to consistent, if the temporal inconsistency tracking information for the higher resolution cell does not identify temporal change since the frame associated with that cell’s reference code-block, and the ancestor cell’s reference code-block comes from the same or an earlier frame and is labelled as consistent. 13. The method of Claim 12, wherein an inter-inconsistency step adjusts the temporal inconsistency tracking information for the higher resolution cell, so that it identifies temporal change in the higher resolution cell once novel temporal inconsistency has been detected in the ancestor cell. 14. The method of Claim 12, wherein an inter-inconsistency step adjusts the temporal inconsistency tracking information for the higher resolution cell, so that it identifies temporal change in the higher resolution cell once novel temporal inconsistency has been detected in an ancestor cell, or in an ancestor of said ancestor cell, but not if the immediate ancestor is ambiguous and a power inversion property is observed, meaning that a measure of the signal power within the higher resolution cell’s reference samples is larger than a measure of the signal power within the immediate ancestor’s reference samples. 15. The method of Claim 14 applied to groups of co-located cells, such that a group of co- located cells from higher resolution sub-bands has an identified ancestor group, consisting of the lower resolution ancestor cells of each cell in the higher resolution group, and the measures of signal power used to identify power inversion are calculated over all cells in the higher resolution group and over all cells in the lower resolution ancestor group. 16. The method of any one of Claims 1 to 15, wherein each ambiguous cell classification label remaining ambiguous is adjusted to a non-ambiguous label, where the adjusted label is inconsistent if temporal inconsistency tracking information for the associated cell identifies temporal change, else consistent if a power inversion property has been observed in connection with the cell, else determined using a default policy that 32 20386987_2 (GHMatters) P120441.PCT specifies either inconsistency or consistency as the default label for all cells that would otherwise remain ambiguous. 17. The method of any one of Claims 1 to 16, wherein the reference buffer is updated based on contextual hints that allow the resilient decoder to determine whether a code-block of the current video frame should be used as a reference code-block for future frames, such contextual hints including protocol information and designated marker codes that accompany the encoded video content received by the decoder. 18. The method of any one of Claim 2 to 17, wherein the reference buffer is updated based on cell classification labels, together with quantization interval length statistics, such that a current code-block that has a sufficient fraction of cells labelled as inconsistent or consistent with a preference for using the current cell in representations of the current frame, is entered into reference buffer to become a reference cell for future frames, where the sufficient fraction itself depends on the quantization interval length statistics. 19. The method of any of Claims 1 to 18, wherein a video frame is reconstructed from merged cells of the frame, such that a merged cell is derived from the current and reference cells of a cell pair by: a) using the decoded sample values of the current cell when the final classification label for the cell is inconsistent or when the classification label or quantization interval length statistics identify the current cell as the one that is preferred for representations of the current frame; else b) using the decoded sample values of the reference cell. 20. The method of any one of Claim 13 to Claim 19 when dependent on Claim 12, where code-blocks from selected high resolution sub-bands are assigned only one cell each and that cell’s final classification label is pre-determined based on the classification labels assigned to lower resolution cells, so that only the reference or the current version of each high resolution code-block needs to be decoded, rather than both versions, depending on the pre-determined classification labels, where the high resolution cells are initially assigned ambiguous labels, and the method of Claim 12, in combination with the methods of any of Claim 13 to Claim 16, are used to convert these ambiguous labels to final classification labels that are inconsistent or consistent. 33 20386987_2 (GHMatters) P120441.PCT
Description:
Method for Resilient Decoding of Image Sequences Field of the Invention This invention relates to image and video decoding. More specifically, it relates to the decoding of a sequence of related encoded images (frames) with non-uniform quality, such the quality of any given spatial region within a frame varies from frame to frame within the sequence. The disclosed method can be applied to video content that has been subjected to a multi-resolution spatial transformation, such as the wavelet transform, together with block- based encoding of the transform sub-bands, as used in JPEG 2000 and several other media encoding algorithms. Background When decoding of a sequence of related encoded images (frames) non-uniform quality or quality variation can be encountered. Such quality variation may arise due to the presence of errors in the coded data stream. Quality variations may, however, arise for other reasons such as changes in the communication bandwidth available for communicating the encoded image sequence, or the preferential encoding of regions of interest that change over the sequence. Decoding methods need to accommodate such quality variations. Error concealment strategies for video coding have been well studied. Typically, the video decoder incorporates one or more frame buffers that hold reference values for each sample in a frame. In traditional predictive video coding algorithms, samples of the current frame have been encoded with respect to the reference values held within frame buffers at the encoder, usually as quantized prediction residuals, where the predicted value of each sample is derived from the frame buffer(s), either directly or through motion compensating or other transformations. In this very common context, the frame buffer(s) used in the encoder are needed also during decoding and so there is little or no additional implementation cost associated with making the values stored in these frame buffer(s) available to an error concealment algorithm that estimates the samples of a current frame when encoded data is erroneous or missing for any reason. The same approach can be used even when video frames are encoded independently, by maintaining frame buffers (usually only one) within the decoder that keep track of recent reliable value for each sample in a frame. In this case, however, frame buffering is used only for error concealment, as opposed to the decoding procedure itself, so it can add significantly to the implementation cost of the decoder. Definitions The term “code-block” here is borrowed from JPEG 2000, as defined in ISO/IEC 15444-1 and other members of the JPEG 2000 family of standards, where a code-block is one independently coded block of samples from one sub-band produced by applying a spatial Discrete Wavelet Transform (DWT) to an image. However, the methods of this invention are applicable also if a code-block is an independently coded collection of samples produced by 1 20386987_2 (GHMatters) P120441.PCT other transformation methods, or even without transformation of the original frame sample data, and even when the code-block extends over the entire video frame. In particular, the term code-block simply refers to a unit within the coded representation that can potentially be decoded successfully even if another code-block within the same frame is corrupted. In this disclosure, the term JPEG 2000 is taken to refer to all members of the JPEG 2000 family of standards, and particularly includes both the coding extensions defined in ISO/IEC 15444- 2 and the High Throughput JPEG 2000 coding technology defined in ISO/IEC 15444-15. Summary of the Invention Embodiments of the present invention provide a method for resilient decoding of a sequence of video frames, each having been transformed to produce a plurality of sample blocks, that have been independently encoded, the encoded sample blocks being identified herein as code-blocks, the method involving a reference buffer that is capable of selectively storing previously received code-blocks from the encoded video sequence, where a code-block within a given frame, identified here as the current code-block of the current frame, is processed using the following steps: a) decoding the current code-block to produce a current block of sample values; b) obtaining a corresponding reference block of decoded sample values for a previous frame in the video sequence by accessing the reference buffer and performing any required decoding operations to obtain said reference samples; c) partitioning each of the current and reference blocks of decoded sample values into one or more cells, each cell of the current block being in one to one correspondence with a cell of the reference block; d) comparing corresponding cells of the current and reference block to produce an initial classification label for each cell pair, the label having at least three possible values that identify the cell as inconsistent, ambiguous or consistent; e) resolving cell pairs initially labelled as ambiguous to consistent or inconsistent by a label adjustment step that uses the labels assigned to other cells. In some embodiments, the initial classification label for a cell pair has four possible values that identify the cell as one of: a) inconsistent; b) ambiguous; c) consistent, with a preference for using the reference cell in representations of the current frame; d) and consistent, with a preference for using the current cell in representations of the current frame. In some embodiments, comparing corresponding cells is based on a comparison of the quantization intervals associated with each sample in the current and reference cell, such that: a) the cell pair is labelled inconsistent if the quantization interval associated with at least one decoded sample in the current cell is inconsistent with the quantization interval associated with the corresponding sample in the reference cell, where inconsistency here means that the quantization intervals do not intersect; b) the cell pair is labelled as ambiguous if there are no corresponding sample pairs that are non-zero in both the current and the reference cell; and c) the cell pair is otherwise assigned a consistent label. 2 20386987_2 (GHMatters) P120441.PCT In some embodiments preference for using the reference cell or preference for using the current cell, in representations of the current frame, is determined based on information identifying which of the current and reference cells is more finely quantized. In some embodiments comparing corresponding cells is based on: a) a distortion reduction value that assesses the maximum amount by which quantization distortion can be reduced within the more finely quantized of the two cells, over sample locations for which the current and reference quantization intervals are consistent and both current and reference samples are non-zero; b) a distortion enlargement value that assesses the distortion between the current and reference cells over sample locations for which the current and reference quantization intervals are inconsistent; and c) an energy value that accumulates an energy statistic, equivalent to distortion between decoded sample values and 0, over all samples of the current and reference cells that are included in the distortion reduction and distortion enlargement value calculations; where the cell pair is labelled inconsistent if the distortion enlargement term exceeds a specified fraction of the energy value, ambiguous if the energy value is 0; else consistent, where preference for using the reference cell or preference for using the current cell, in representations of the current frame, is determined based on a comparison between the distortion reducing and distortion enlargement values, together with information identifying which of the current and reference cells is more finely quantized. In some embodiments the code-blocks are independently coded blocks of samples from sub- bands produced by discrete wavelet transformation of the frame sample values. In some embodiments ambiguity in the classification label assigned to a cell pair is resolved with the aid of temporal inconsistency tracking information, which identifies whether or not any temporal change has been detected over the cell’s region of support, in any frame following the one that is associated with the cell’s reference code-block, such that the ambiguous label is adjusted to an inconsistent label if such temporal change has been detected. In some embodiments cells with a similar region of influence in the video frame, identified as co-located cells, are collected in groups and a co-inconsistency step modifies temporal inconsistency tracking information so as to identify temporal change for one cell within a group based on the detection of novel temporal inconsistency in any cell within the same group, where novel temporal inconsistency in a cell means that temporal change is detected for the first time in that cell since the frame associated the cell’s reference code-block. In some embodiments temporal inconsistency tracking is performed by assigning a binary state variable to cells in a frame, where the binary state variable for a cell is set to a prescribed value, its active state, after temporal change is identified, and is reset to the other value, its inactive state, when the reference buffer is updated so that future frames will employ a new reference code-block for the cell. In some embodiments temporal inconsistency tracking information is performed jointly for all of the co-located cells within a group, using a single state variable that serves to identify the most recent frame in which novel temporal inconsistency has been detected within any cell of the group. 3 20386987_2 (GHMatters) P120441.PCT In some embodiments a co-consistency step adjusts the classification label of a cell from ambiguous to consistent, if the temporal inconsistency tracking information for the cell does not identify temporal change since the frame associated with the cell’s reference code-block, and another co-located cell in the same group, whose reference code-block comes from the same or an earlier frame, is labelled as consistent. In some embodiments a cell belonging to a lower resolution sub-band of a frame, whose region of influence in the video frame overlaps that of a cell from a higher resolution sub- band, is identified as an ancestor of the higher resolution cell, and an inter-consistency step adjusts the classification label of the higher resolution cell from ambiguous to consistent, if the temporal inconsistency tracking information for the higher resolution cell does not identify temporal change since the frame associated with that cell’s reference code-block, and the ancestor cell’s reference code-block comes from the same or an earlier frame and is labelled as consistent. In some embodiments an inter-inconsistency step adjusts the temporal inconsistency tracking information for the higher resolution cell, so that it identifies temporal change in the higher resolution cell once novel temporal inconsistency has been detected in the ancestor cell. In some other embodiments an inter-inconsistency step adjusts the temporal inconsistency tracking information for the higher resolution cell, so that it identifies temporal change in the higher resolution cell once novel temporal inconsistency has been detected in an ancestor cell, or in an ancestor of said ancestor cell, but not if the immediate ancestor is ambiguous and a power inversion property is observed, meaning that a measure of the signal power within the higher resolution cell’s reference samples is larger than a measure of the signal power within the immediate ancestor’s reference samples. In an embodiment this method can be applied to groups of co-located cells, such that a group of co-located cells from higher resolution sub-bands has an identified ancestor group, consisting of the lower resolution ancestor cells of each cell in the higher resolution group, and the measures of signal power used to identify power inversion are calculated over all cells in the higher resolution group and over all cells in the lower resolution ancestor group. In some embodiments each ambiguous cell classification label remaining ambiguous is adjusted to a non-ambiguous label, where the adjusted label is inconsistent if temporal inconsistency tracking information for the associated cell identifies temporal change, else consistent if a power inversion property has been observed in connection with the cell, else determined using a default policy that specifies either inconsistency or consistency as the default label for all cells that would otherwise remain ambiguous. In some embodiments the reference buffer is updated based on contextual hints that allow the resilient decoder to determine whether a code-block of the current video frame should be used as a reference code-block for future frames, such contextual hints including protocol information and designated marker codes that accompany the encoded video content received by the decoder. In some embodiments the reference buffer is updated based on cell classification labels, together with quantization interval length statistics, such that a current code-block that has a sufficient fraction of cells labelled as inconsistent or consistent with a preference for using the current cell in representations of the current frame, is entered into reference buffer to 4 20386987_2 (GHMatters) P120441.PCT become a reference cell for future frames, where the sufficient fraction itself depends on the quantization interval length statistics. In some embodiments a video frame is reconstructed from merged cells of the frame, such that a merged cell is derived from the current and reference cells of a cell pair by: a) using the decoded sample values of the current cell when the final classification label for the cell is inconsistent or when the classification label or quantization interval length statistics identify the current cell as the one that is preferred for representations of the current frame; else b) using the decoded sample values of the reference cell. In some embodiments code-blocks from selected high resolution sub-bands are assigned only one cell each and that cell’s final classification label is pre-determined based on the classification labels assigned to lower resolution cells, so that only the reference or the current version of each high resolution code-block needs to be decoded, rather than both versions, depending on the pre-determined classification labels, where the high resolution cells are initially assigned ambiguous labels, and the method of Claim 12, in combination with the methods of any of Claim 13 to Claim 16, are used to convert these ambiguous labels to final classification labels that are inconsistent or consistent. Brief Description of the Drawings An embodiment, incorporating all aspects of the invention, will now be described by way of example only with reference to the accompanying drawings in which Figure 1: illustrates an overview of the invention, identifying some of the key concepts, notation and aspects of the invention; Figure 2: is an illustration of an example of temporal inconsistency tracking; and Figure 3: is an illustration of reference buffer management. Detailed Description Embodiments of the invention provide methods for error resilient decoding of independently encoded video frames that does not require a frame buffer in the traditional sense, which can be updated from frame to frame with the most reliable decoded values for each sample location. Instead, the methods described in this invention require only the buffering of original encoded code-blocks within a reference buffer. Embodiments of the method for resilient decoding are applied to a sequence of video frames, where each video frame has been transformed to produce a plurality of sample blocks, that have been independently encoded. The encoded sample blocks are referred to herein as code-blocks. The method involves a reference buffer that is capable of selectively storing previously received code-blocks from the encoded video sequence. A code-block within a given frame, identified here as the current code-block of the current frame, is processed using the following steps: 5 20386987_2 (GHMatters) P120441.PCT a) decoding the current code-block to produce a current block of sample values; b) obtaining a corresponding reference block of decoded sample values for a previous frame in the video sequence by accessing the reference buffer and performing any required decoding operations to obtain said reference samples; c) partitioning each of the current and reference blocks of decoded sample values into one or more cells, each cell of the current block being in one to one correspondence with a cell of the reference block; d) comparing corresponding cells of the current and reference block to produce an initial classification label for each cell pair, the label having at least three possible values that identify the cell as inconsistent, ambiguous or consistent; e) resolving cell pairs initially labelled as ambiguous to consistent or inconsistent by a label adjustment step that uses the labels assigned to other cells. Original encoded blocks of a previous frame are accessed from a reference buffer. These blocks are decoded to retrieve sample values for the block from the previous frame. The decoded sample values from a current block and a corresponding reference block are partitioned into cells having one or one correspondence between the current and reference frames. The corresponding cells are compared to produce an initial classification label for each cell, the classification label having one of at least 3 values, which identify the cell as ambiguous, consistent or inconsistent. Cells initially labelled as ambiguous or inconsistent are then resolved to consistent using labels assigned to other cells. The methods described in this disclosure are particularly beneficial when applied to video content whose frames have been encoded independently (also known as intra coding), since then errors or quality fluctuations encountered in one frame need not affect the decoding of other frames in the sequence. Moreover, in this context, the methods described in this invention do not require a frame buffer to keep track of decoded sample values from previous frames, which can be particularly beneficial for lightweight deployments on embedded hardware platforms. More particularly, the methods described in this invention are particularly beneficial when used with block-based encoding techniques that independently encode blocks of samples, either in the image domain or a transform domain such as the wavelet domain, since then errors or quality fluctuations encountered in one block do not necessarily affect the decoding of other blocks within the same frame of the sequence. Finally, the methods described in this disclosure are particularly beneficial when applied to video content that has been subjected to a multi-resolution spatial transformation, such as the wavelet transform, together with block-based encoding of the transform sub-bands, as used in JPEG 2000 and several other media encoding algorithms. Overview and context of the invention While error resilience is normally used only to combat unintentional corruption of the encoded media data stream, the same techniques can be used to improve the quality of a video sequence that has been encoded non-uniformly, such that some frames or regions of frames have significantly lower quality than others. In particular, lower quality code-blocks in a current frame can be processed as if they had been corrupted, so long as higher quality reference code-blocks are available from a previous frame. The main difference between non- uniform quality decoding applications and error resilient decoding is that the former 6 20386987_2 (GHMatters) P120441.PCT introduce more ambiguity concerning the choice of reference code-blocks to keep track of within the reference buffer. This is because the discrimination between corrupted and non- corrupted code-blocks is often trivial in comparison to the discrimination between higher and lower quality versions of a code-block as the video sequence evolves over time. In the case of JPEG 2000, however, error resilient decoding applications can wind up being equivalent to non-uniform quality decoding applications, since JPEG 2000 code-blocks are progressively encoded using an embedded block coding algorithm, such that errors in one code-block bit- stream lead only to quality reductions, as opposed to complete corruption of the decoded sample values. In some cases, an encoder or content distribution server may intentionally introduce substantial quality variations from code-block to code-block, as a method for reducing communication bandwidth. An excellent example of this is the “JSIV” video communication framework (for example, as described in A. Naman and D. Taubman, "JPEG2000-based scalable interactive video (JSIV)," IEEE Transactions on Image Processing, vol.20, no.5, pp. 1435-1449, 2011), in which a server dynamically optimizes the delivery of code-block bit- streams over a bandwidth constrained channel, based on the assumption that a client which receives entirely empty code-blocks (no coded bits at all) will use the most recent non-empty version of the same code-block from a previous frame. That is, JSIV assumes that the decoder has a reference buffer containing the most recent non-empty version of each code-block in a frame and that the decoder addresses the non-uniform quality of each frame by simply replacing empty code-blocks in a current frame with the corresponding code-block bit-stream from the reference buffer. JSIV can also be used with motion compensation (for example as described in A. Naman and D. Taubman, "JPEG2000-based scalable interactive video (JSIV) with motion compensation," IEEE Transactions on Image Processing, vol.20, no.9, pp.2650- 2663, 2011), in which case the use of the reference buffer is more complex, but the principles are basically the same. The methods described in this invention can be used to further enhance the decoded video quality associated with content delivered by a JSIV server. In its simplest form, JSIV is a “conditional replenishment” scheme, where non-empty code- blocks within the current frame are used to update (or “replenish”) the corresponding sample values, while empty code-blocks retain their previous values (i.e., they are not replenished). In JSIV, the conditional replenishment operates in the wavelet domain, since JPEG 2000 code- blocks partition wavelet sub-bands rather than the image data associated with each video frame. More generally, though, conditional replenishment tools have a very long history of application within video codecs. Since the earliest video coding standards, such as H.261, conditional replenishment has been an important mode for block-based motion compensated video codecs, which can explicitly identify (e.g., through mode flags) blocks that are not updated (not replenished) with new data in a given frame. These video codecs mostly operate with image-domain blocks, e.g., “macro-blocks,” that are subjected to block-based transform, quantization and coding operations. Apart from operating in the image domain, rather than the wavelet domain, the other distinction between conditional replenishment based video codecs and JSIV is that JSIV is an open-loop scheme that does not require the decoder to adopt a prescribed strategy for processing the content that it receives; by contrast, most video codecs employ a closed-loop approach, where the decoder progressively updates at least one frame buffer that is replicated within the encoder. The JSIV server makes rate- distortion optimizing decisions regarding the content that it sends to a remote client, based on an assumption that the decoder will employ a sensible method for reconstructing the non- 7 20386987_2 (GHMatters) P120441.PCT uniform quality content that it receives, but the decoder has the freedom to use its reference buffer in any manner it sees fit. As noted, the methods described in this invention can be used to address both error resilient decoding and the decoding of content in which code-blocks may be intentionally dropped from the transmitted video stream to save bandwidth, which can be seen as a form of conditional replenishment. More generally, however, the methods of this invention can be used in the less extreme case where some code-blocks have significantly lower quality than other code-blocks within a given frame, without necessarily being entirely empty or corrupted. A video encoder or distribution server may intentionally create such content, as a method for reducing communication bandwidth while preserving a minimum level of video quality even for non-resilient decoders. The purpose of this invention is to provide robust methods for decoding such content no matter how it has been created, whether by a JSIV- like encoder or server that carefully selects the code-blocks to be sent with higher quality based on temporal changes in the scene, or by a naïve encoder that employs a data independent strategy for cycling through code-blocks that are to be sent with higher quality. Importantly, all the methods of this invention can be applied using a reference buffer that consists only of original code-blocks from the encoded content that is received by the decoder, without the need to store decoded sample values from a previous frame. This property can be highly beneficial in embedded hardware systems where memory and memory bandwidth are heavily constrained resources. One way to address the memory and memory bandwidth concerns associated with maintaining a frame buffer is to employ lightweight data compression techniques for the frame buffer itself, as described, for example, in US patent application publication no. 2011/0310974, entitled “Frame buffer compression for video processing devices”. However, the reference buffer employed in this invention stores only original code-block bit-streams received by the decoder; no encoding is performed within the decoder to maintain the reference buffer. Figure 2 illustrates an overview of the invention, identifying some of the key concepts, notation and aspects of the invention. In the figure, frames 100 have been independently compressed using a Discrete Wavelet Transform (DWT) whose sub-bands 105 have been partitioned into independently coded code-blocks 110a-n, as is done in JPEG 2000. Arriving data consists of compressed code-blocks 110a-n, while a reference buffer holds previously received code-blocks ^^^^ ^^^^ ^^^^ ^^^^ 120a-n, with various frame indices ^^^^ ^^^^ ^^^^ ^^^^ . Lightly shaded regions in the current (arriving) frame’s sub-bands are intended to indicate code-blocks ^^^^ ^^^^ ^^^^ ^^^^ (i.e.110a, 110c) that have reduced quality (larger Π ^^^^ ^^^^ ^^^^ ^^^^ value), which may happen more often than not in some applications of the invention. Decoded code-blocks 130 are partitioned into cells 135; cells are grouped with co-located cells 135a-c from other sub-bands; cells 135, 155 are compared 160 between current 130 and reference code-blocks 150 to establish an initial set of labels Λ; ambiguity in the labelling process is resolved with the help of other cell comparisons 170; and a preferred version of each cell (current or reference) 180 is used to reconstruct the final video frame 190. The invention consists of the following aspects, many of which are illustrated in Figure 1, along with notation that is discussed in more detail in the following sections. 8 20386987_2 (GHMatters) P120441.PCT Aspect 1: A method for decoding and comparing a code-block ^^^^ ^^^^ ^^^^ ^^^^ from the current frame and a corresponding code-block ^^^^ ^^^^ ^^^^ ^^^^ from the reference buffer, involving partitioning of the decoded samples from ^^^^ ^^^^ ^^^^ ^^^^ into one or more cells ^^^^ ^^^^ ^^^^ ^^^^ and the assignment of a classification label Λ to each such cell, based on comparison with a corresponding cell ^^^^ ^^^^ ^^^^ ^^^^ that is derived from ^^^^ ^^^^ ^^^^ ^^^^ . In preferred embodiments, there are four labels Λ ∈ {−1,0,1,2}, where Λ = −1 identifies inconsistency between ^^^^ ^^^^ ^^^^ ^^^^ and ^^^^ ^^^^ ^^^^ ^^^^ , Λ > 0 identifies consistency and Λ = 0 means that the comparison is ambiguous. The resolution of such ambiguities is the subject of Aspects 2, 3, 4 and 5 of the invention. The distinction between labels Λ = 1 and Λ = 2 is that Λ = 2 identifies ^^^^ ^^^^ ^^^^ ^^^^ as preferred for use in the current frame’s reconstruction, while Λ = 1 identifies ^^^^ ^^^^ ^^^^ ^^^^ as the preferred one to use in reconstruction of the current frame. In some embodiments of the invention, there may be no need to distinguish between these two cases of temporal consistency Λ > 1 on a cell-by-cell basis, but the distinction is useful for a comprehensive description of the methods of the invention. Aspect 2: A method for tracking inconsistency across multiple frames of a video sequence, on a cell-by-cell basis, so as to identify novel frames in which inconsistency is first observed within a cell, between the frame from which the cell’s reference code-block ^^^^ ^^^^ ^^^^ ^^^^ is derived, and the current frame. This aspect is useful for resolving ambiguous cell classification outcomes Λ = 0. Aspect 3: A method for using cell classification outcomes and temporal inconsistency tracking information for cells belonging to one sub-band to enhance the cell classification outcomes and inconsistency tracking for co-located cells belonging to another sub-band. In some embodiments, this aspect of the invention also allows the temporal inconsistency tracking information associated with co-located cells to be combined so as to reduce the memory footprint associated with tracking state variables. Aspect 4: A method for using the classification outcomes and temporal inconsistency tracking information from cells belonging to lower resolution code-blocks, to enhance the cell classification outcomes and inconsistency tracking information for cells belonging to higher resolution code-blocks. The method is applicable where the encoded frames have been produced using a multi-resolution transform. Aspect 5: A method for resolving ambiguous cell classification outcomes Λ = 0, using temporal inconsistency tracking information and other contextual information. Aspect 6: A method for managing the content of the reference buffer, including methods for determining when a reference code-block ^^^^ ^^^^ ^^^^ ^^^^ should be replaced with a corresponding code-block ^^^^ ^^^^ ^^^^ ^^^^ from the current frame; these methods take into account any available information concerning communication errors, the lengths of the bit-streams or number of coding passes associated with ^^^^ ^^^^ ^^^^ ^^^^ and ^^^^ ^^^^ ^^^^ ^^^^ , and explicit information provided within the coded data or a communication protocol that provides hints regarding the relative utilities of ^^^^ ^^^^ ^^^^ ^^^^ and ^^^^ ^^^^ ^^^^ ^^^^ , and (potentially) the cell classification labels deduced via the methods of any of Aspects 1 through 5. Aspect 7: A method for decoding a video sequence from encoded video frames consisting of code-blocks of non-uniform quality, involving the cell classification and reference buffer management methods of any of Aspect 1 through to Aspect 6, together with a merging 9 20386987_2 (GHMatters) P120441.PCT procedure that combines the cells from each decoded pair of reference ^^^^ ^^^^ ^^^^ ^^^^ and current ^^^^ ^^^^ ^^^^ ^^^^ code-blocks. Aspect 8: A method to reduce block decoding resources in applications where the encoded sub-bands come from a multi-resolution transform of each video frame, by pre-determining the final classification label Λ for code-blocks belonging to one or more of the highest resolutions, using information obtained from lower resolution code-blocks, such that only the current or the reference version of each high-resolution code-block needs to be decoded, but not both. Description of aspects of the invention 1.1 1 st Aspect: Decoding and comparison of current and reference code-blocks The methods of this invention are described in terms of code-blocks, where a current video frame has been (optionally) transformed, producing a collection of sub-bands, then each sub- band has been partitioned into one or more code-blocks that are (optionally) subjected to quantization and coding operations to produce code-block bit-streams, such that the encoded representation of the frame consists of a collection of code-block bit-streams. An important example of interest is JPEG 2000, where the frames have been transformed using a Discrete Wavelet Transform (DWT) prior to any quantization of the wavelet sub-bands and then independent embedded block coding. However, as noted in Section 2, the methods of the invention may be applied with other coding strategies. 1.1.1 Sample values, quantization intervals and related quantities for a code-block ^^^^ ^^^^ ^^^^ ^^^^ Write ^^^^ ^^^^ ^^^^ ^^^^ for the index (or time) of a current frame within the video sequence and consider any given code-block ^^^^ ^^^^ ^^^^ ^^^^ within that frame, writing Ω ^^^^ ^^^^ ^^^^ ^^^^ for the collection of sub-band sample locations ^^^^ ∈ Ω ^^^^ ^^^^ ^^^^ ^^^^ that are spanned by the code-block. Associated with each location ^^^^ ∈ Ω ^^^^ ^^^^ ^^^^ ^^^^ , there is a reconstructed sample value ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ [ ^^^^] and a quantization interval ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ [ ^^^^] that can be obtained by decoding and dequantizing ^^^^ ^^^^ ^^^^ ^^^^ . The quantization interval ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ [ ^^^^] is important here; it is the set of all possible sample values ^^^^[ ^^^^] whose quantization is consistent with the encoded representation of ^^^^ ^^^^ ^^^^ ^^^^ . Usually, the reconstructed value ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ [ ^^^^] is the mid-point of the interval ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ [ ^^^^], but any value within the interval could conceivably be used as the reconstructed value. In JPEG 2000, the quantization interval ^^^^[ ^^^^] can effectively be embedded implicitly within the reconstructed sample value ^^^^[ ^^^^], by following the mid-point reconstruction rule and recognizing that JPEG 2000 uses a deadzone quantizer, so that ^^^^ [ ^^^^ ] can be expressed as Here, Δ is a sub-band wide quantization step size, ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ [ ^^^^] ∈ {0,1} identifies the sign of the reconstructed sample value, ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ [ ^^^^] holds the decoded magnitude bits and ^^^^[ ^^^^] is the number of least significant magnitude bit-planes not encountered within the decoded coding passes associated with sample location ^^^^. Then ^^^^[ ^^^^] and its quantization interval ^^^^[ ^^^^] can be 10 20386987_2 (GHMatters) P120441.PCT fully and compactly described in sign-magnitude form by ^^^^[ ^^^^] ≡ ( ^^^^[ ^^^^], ^̅^^^[ ^^^^]), where ^̅^^^[ ^^^^] is the modified magnitude value given by It is a simple matter to recover ^^^^[ ^^^^] from ^̅^^^[ ^^^^], by recognizing that whenever ^^^^[ ^^^^] ≠ 0, ^̅^^^[ ^^^^] contains at least two non-zero bits, so ^^^^[ ^^^^] = 0 if and only if ^̅^^^[ ^^^^] is a pure power of 2, which is equivalent to the condition that ^̅^^^ [ ^^^^ ] and ^̅^^^ [ ^^^^ ] − 1 have a zero logical AND. At the same time, for all values of ^^^^[ ^^^^], the least significant non-zero bit within ^̅^^^[ ^^^^] reveals the value of ^^^^[ ^^^^]. Thus, ^^^^[ ^^^^], together with the sub-band wide step size Δ, provides both i f ( ^̅^^^[ ^^^^] ∧ ( ^̅^^^[ ^^^^] − 1)) ≠ 0 otherwise and the ^^^^[ ^^^^], from which we get the length ‖ ^^^^[ ^^^^]‖ of the quantization interval as ‖ ^^^^[ ^^^^]‖ = Δ if y[ ^^^^] ≠ 0 otherwise (the deadzone is twice as large as other intervals) Since ^^^^ [ ^^^^ ] ∈ ^^^^[ ^^^^], the values of ^^^^[ ^^^^] and ^^^^[ ^^^^] uniquely determine the quantization interval ^^^^[ ^^^^]. It is useful also to introduce the notation ^ ^^^[ ^^^^] =� 1 if ^^^^[ ^^^^] ≠ 0 1 if ( ^̅^^^[ ^^^^] ∧ ( ^̅^^^[ ^^^^] − 1)) ≠ 0 if ^^^[ ] =� 0 ^ ^^^^ = 0 0 otherwise along with the term “significant,” where a sample is said to be significant if ^^^^[ ^^^^] = 1, or equivalently, if ^^^^ [ ^^^^ ] ≠ 0. Finally, we note that the otherwise impossible value ^^^^ [ ^^^^ ] = 0 is used for insignificant samples where the quantization interval length ^^^^[ ^^^^] is unknown. This is useful when the code-block ^^^^ ^^^^ ^^^^ ^^^^ is empty, so that no information is available for the decoder. In some embodiments of the invention, the value ^^^^ [ ^^^^ ] = 0 is used for all insignificant samples, as a means of simplifying the implementation, even if the quantization interval length ‖ ^^^^[ ^^^^]‖ could be deduced by the decoder. Similar strategies can be applied to quantization schemes other than the JPEG 2000 one discussed above, to describe both the reconstructed sample value and the quantization interval for each sample, via one or more global quantization step sizes and an integer ^^^^[ ^^^^] in sign-magnitude form. For this reason, we use ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ [ ^^^^] henceforth to collectively identify both ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ [ ^^^^] and ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ [ ^^^^] for a sample location ^^^^ ∈ Ω ^^^^ ^^^^ ^^^^ ^^^^ . In addition to ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ [ ^^^^], ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ [ ^^^^], ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ [ ^^^^] and ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ [ ^^^^], which are all sample-specific quantities, let Π ^^^^ ^^^^ ^^^^ ^^^^ be a measure of the average log-quantization interval length, log2� ^^^^ ^^^^ ^^^^ ^^^^ ^^^^[ ^^^^]�, over all ^^^^ ∈ Ω ^^^^ ^^^^ ^^^^ ^^^^ . This can be computed as log2 Δ plus the average of the ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ [ ^^^^] values, where ^^^^[ ^^^^] is the number of least significant magnitude bit-planes not encountered within the decoded coding passes associated with sample location ^^^^, as explained above. However, in the case of JPEG 2000 code-blocks, it is preferable to estimate a value for Π ^ ^^^ ^^^^ ^^^^ ^^^^ = Π ^^^^ ^^^^ ^^^^ ^^^^ − log 2 Δ simply by counting the number of finest coding passes that are missing from ^^^^ ^^^^ ^^^^ ^^^^ and dividing by 3. To be clear, JPEG 2000 code-blocks are extracted from J2K packets, where the packet headers identify the number of coding passes that are present for each block, along with the number 11 20386987_2 (GHMatters) P120441.PCT of most significant magnitude bit-planes ^^^^ that are “missing,” in the sense that the first coding pass encodes the next most significant magnitude bit-plane. Moreover, the total number of coding passes that could be present in a code-block, is equal to 3( ^^^^ ^^^^ − ^^^^) − 2, where ^^^^ ^^^^ is deduced from the relevant sub-band’s quantization parameters, as the maximum number of magnitude bit-planes that can be encoded for any sample. The “number of finest coding passes that are missing” from ^^^^ ^^^^ ^^^^ ^^^^ , mentioned above, is then the difference between the number of actual coding passes available for ^^^^ ^^^^ ^^^^ ^^^^ and this maximum value 3( ^^^^ ^^^^ − ^^^^) − 2. The reason for dividing by 3 is that there are three JPEG 2000 coding passes for each magnitude bit-plane. Other coding technologies may provide their own means for estimating the log-quantization interval within a code-block, based solely on parameters extracted from the code-stream. 1.1.2 Sample values and related quantities for a reference code-block ^^^^ ^^^^ ^^^^ ^^^^ We write ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ [ ^^^^], ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ [ ^^^^], ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ [ ^^^^], ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ [ ^^^^] and Π ^^^^ ^^^^ ^^^^ ^^^^ to identify the same quantities for a reference code-block, whose region of support Ω ^^^^ ^^^^ ^^^^ ^^^^ is the same as Ω ^^^^ ^^^^ ^^^^ ^^^^ , within the same sub-band of an earlier frame ^^^^ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ ^^^^ < ^^^^ ^^^^ ^^^^ ^^^^ in the video sequence. In preferred embodiments of the invention, these quantities are all obtained by decoding an original code-block ^^^^ ^^^^ ^^^^ ^^^^ that is co-located with ^^^^ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ within frame ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ . If there is no such code-block within the reference buffer, for any reason, ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ [ ^^^^] is taken to be 0 for all ^^^^ ∈ Ω ^^^^ ^^^^ ^^^^ ^^^^ , meaning that all of the code-block’s samples are taken to be insignificant and with unknown (unbounded) quantization interval length [ ^^^^]�, as explained above. In many embodiments of the invention, reference code-blocks ^^^^ ^^^^ ^^^^ ^^^^ are used only if they were encoded using the same underlying sub-band wide quantization step size Δ as ^^^^ ^^^^ ^^^^ ^^^^ , again setting ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ [ ^^^^] = 0 if this is not the case. This limitation is reasonable, especially for JPEG 2000 based deployments, since the underlying quantization step sizes do not normally change from frame to frame when JPEG 2000 is used as the codec – the encoded data rate is usually controlled separately through the number of coding passes that are emitted to the output codestream. In this important case, where Δ does not change over time, it is sufficient to calculate Π ^ ^ ^^ ^^^^ ^^^^ ^^^^ = Π ^^^^ ^^^^ ^^^^ ^^^^ − log 2 Δ rather than Π ^^^^ ^^^^ ^^^^ ^^^^ and Π ^^^^ ^^^^ ^^^^ ^^^^ , noting that Π ^ ^ ^^ ^^^^ ^^^^ ^^^^ by counting the number of finest coding passes that are missing from ^^^^ ^^^^ ^^^^ ^^^^ and ^^^^ ^^^^ ^^^^ ^^^^ , respectively, and dividing by 3. In some embodiments of the invention, the reference buffer may consist of decoded sample data instead of code-block bit-streams, in which case ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ [ ^^^^] and can be recovered directly from the reference buffer, without requiring these quantities to arise from a unique code-block ^^^^ ^^^^ ^^^^ ^^^^ within the earlier frame; indeed ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ [ ^^^^] and thence Π ^^^^ ^^^^ ^^^^ ^^^^ may even be derived using a motion compensation procedure in some embodiments of the invention. Nevertheless, preferred embodiments of the invention avoid the need to buffer actual decoded sample data by buffering reference code-block bit-streams instead, decoding any available ^^^^ ^^^^ ^^^^ ^^^^ concurrently with ^^^^ ^^^^ ^^^^ ^^^^ . 1.1.3 Partition of code-blocks into cells In this invention, the code-block region Ω ^^^^ ^^^^ ^^^^ ^^^^ is partitioned into one or more cells, ^^^^ ^^ ^ ^ ^ ^ ^ ^ ^ ^ ^^ ^^^^ , each with their own region of support Ω ^^^^ ^^ ^^ ^^ ^^ ^ ^^^ ^^^^ , such that 12 20386987_2 (GHMatters) P120441.PCT where denotes the set of cell indices ^^^^ that belong to code-block ^^^^ ^^^^ ^^^^ ^^^^ . In preferred embodiments of the invention, cells are 4 × 4 square blocks, but other cell dimensions may be preferred for some applications and indeed a single cell can span the entire code-block if desired. As a matter of convenience, we write ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ [ ^^^^] = ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ [ ^^^^] for the decoded sample values and quantization intervals of cell ^^^^ for each ^^^^ ∈ Ω ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ . Similarly, we write ^^^^ ^^^^ [ ^^^^] = ^^^^ [ ^^^^] for each ^^^^ ∈ Ω ^^^^ , we writ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ e ^^^^ ^^^^ ^^ ^^ ^^ ^^ ^^^^ ^^^^ = ^^^^ ^^^^ ^ ^ ^^ ^^ ^^ ^^^^ ^^^^ for the frame to which the reference code-block ^^^^ ^^^^ ^^^^ ^^^^ used with ^^^^ ^^^^ ^^^^ ^^^^ belongs, and we write = Π ^^^^ ^^^^ ^^^^ ^^^^ , for all ^^^^ in the cell partition ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ of ^^^^ ^^^^ ^^^^ ^^^^ . describe a pair of decoded sample values, along with their quantization intervals. A significant step in the invention is to compare the cells of a cell pair, which in turn involves the comparison of samples and quantization intervals of sample pairs. The purpose of cell comparison is to determine a classification label Λ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ ∈ {−1,0,1,2} for the cell pair. In this description of the invention, the term “cell pair” is often replaced simply by “cell”, where the involvement of a pair of cells ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^ ^^ ^^^^ and is obvious from the context. Also, the label Λ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^ explicitly identifies y the cell ^^^^ ^^^^ ^^^^ ^^^^ , even though the label is derived from both ^^^^ ^^^^ ^^^^ ^^^^ and ^^^^ ^^ ^ ^ ^ ^ onl ^^^^ ^^^^ ^ ^ ^ ^ ^ ^^^ ^^^^ , and the same is true for the notation used for other quantities in this description of the invention. As noted already, the classification labels of interest are: Λ=-1 (inconsistent); Λ=0 (ambiguous); Λ=1 (consistent, with a preference to reconstruct using ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ ) and Λ=2 (consistent, with a preference to reconstruct using ^^^^ ^^ ^ ^ ^ ^ ^ ^ ^ ^ ^^ ^^^^ ). There are multiple ways to do determine the label for each cell pair, depending in part upon the type of encoded video content that the decoder is expected to process. In order to explain these different labelling strategies, we first introduce the binary symbol ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ [ ^^^^], to indicate whether or not ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ [ ^^^^ ] and ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ [ ^^^^] describe a sample pair with incompatible quantization intervals. Specifically, ^ ^^^ ^^^^ 1 if ^^^^ ^^^^ [ ^^^^] ∩ ^^^^ ^^^^ [ ^^^^] = ∅ (non-intersecting intervals are incomp [ ^^^ ] ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ atible) ^ ^^^ ^^^^ ^^^^ ^^^^ ^ =� 0 otherwise (intersecting intervals are considered compatible) In the common case where frames ^^^^ ^^^^ ^^^^ ^^^^ and ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ use the same underlying set of sub-band wide quantization step sizes Δ, ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ [ ^^^^] can be deduced directly from the values of ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ [ ^^^^] ^ ^^^ alone. Also, if either of ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ [ ^^^^ ] or ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ [ ^^^^ ] is 0, then the corresponding quantization interval is unknown and taken to have unbounded size, so that ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ [ ^^^^] = 0; that is, an unbounded deadzone in one frame is taken to be compatible with any reconstructed value in the other frame. 1.1.4 Cell classification label generation for screen content decoding Screen content is special in that large regions of the image often remain entirely unchanged across many frames, so that any detectable change between frames can be taken as an indication of inconsistency. When a decoder expects to be processing screen content video sequences, cell ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^ ^^ ^^^^ can be identified as: a) inconsistent (Λ = −1) if ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ [ ^^^^ ] = 1 for any ^^^^ ∈ 13 20386987_2 (GHMatters) P120441.PCT Ω ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ ; b) ambiguous (Λ = 0) if ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ [ ^^^^] = 0 but ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ [ ^^^^] = 0 or ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ [ ^^^^] = 0 for all ^^^^ ∈ and otherwise consistent (Λ > 0). For consistent cells, Λ=2 (prefer ^^^^ ^^ ^ ^ ^ ^ ^ ^ ^ ^ ^^ ^^^^ ) applies if the quantization intervals of ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^ ^^ ^^^^ are no larger than those of ^^^^ ^^ ^ ^ ^ ^ ^^ ^ ^^^ ^^^^ ; here, this condition is expressed using the binary value If both code-blocks use the same quantization step size Δ, then ^^^^ ^^^^ ^^ ^^ ^ ^^ = 1 if and only if ′ ^ ^^^^ ^^^^ Π ^^^^ ^^ ^^ ^^ ^^ ^ ^^^ ^^^^ ≤ Π ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ , which is true if and only if the last coding pass in ^^^^ ^^^^ ^^^^ ^^^^ is also present within ^^^^ ^^^^ ^^^^ ^^^^ . In the case where no reference code-block ^^^^ ^^^^ ^^^^ ^^^^ is available, or in embodiments of the invention where a reference code-block cannot be used because the embodiment requires ^^^^ ^^^^ ^^^^ ^^^^ to have the same quantization step size Δ as ^^^^ ^^^^ ^^^^ ^^^^ , Π ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ can be understood as unbounded (or ∞) and so ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ is certainly 1. In summary, the cell classification labels for screen content decoding are set as follows: - 1 if ^^^^ ^^^^ ^ ^ ^^ ^^ ^^ ^^^^ ^^^^ [ ^^^^] = 1 for any else = � 0 if ^^^^ ^^^^ ^ ^ ^^ ^^ ^^ ^^^^ ^^^^ [ ^^^^] ⋅ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ [ ^^^^] = 0 for all ^^^^ ∈ Ω ^^^^ ^^ ^ ^^ ^^^ ^^^^ ^^^^ , else (2) 1+ ^^^^ ^^^^ ^ ^ ^^ ^^ ^^ ^^^^ ^^^^ otherwise It is worth noting that since ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ can usually be expected to take the same value for all cells in a code-block, there is no need to explicitly distinguish between Λ = 1 and Λ = 2 at the level of individual cells. 1.1.5 Cell classification label generation for photographic video decoding For photographic content, small inconsistencies between reconstructed quantization intervals for cells ^^^^ ^^ ^ ^ ^ ^ ^ ^ ^ ^ ^^ ^^^^ cannot necessarily be interpreted as evidence of significant temporal change, since photographic image sensors are subject to photon shot noise and other minor environmental factors that may occasionally lead to the appearance of inconsistent intervals, even if there is no structural change in the scene content over time. One way to identify temporal change is by applying a correlation technique to the reconstructed sample values ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ [ ^^^^] and comparing the result against a pre-determined threshold. For example, the label Λ=-1 could be assigned to the case in which the normalized cross correlation between ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ and ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ is less than 0.7. It is preferable, however, to first exclude all locations ^^^^, for which are both 0. These are the locations for which one or both of ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ [ ^^^^ ] and ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ [ ^^^^ ] are insignificant yet have compatible quantization intervals, which yield no significant information regarding the temporal consistency between and ^^^^ ^^ ^ ^ ^ ^ ^ ^ ^ ^ ^^ ^^^^ . From an implementation perspective at least, it is preferable to replace normalized correlation metrics with estimates of the relative distortion reduction associated with using instead of ^^^^ ^^ ^ ^ ^ ^ ^ ^ ^ ^ ^^ ^^^^ or vice-versa, whichever offers the smaller quantisation intervals. Specifically, preferred embodiments evaluate the following cell-wide quantities: 14 20386987_2 (GHMatters) P120441.PCT The cell classification label Λ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ for photographic video content is formed as follows: i f ^^^^ ^^^^ ^^ ^ ^^ ^^^ ^^^^ ^^^^ = 0, else + (4) if ^^^^ ^^^^ ^^ ^ ^^ ^^^ ^^^^ ^^^^ ≥ ^^^^ ^^^^ ^ ^ ^^ ^^ ^^ ^^^^ ^^^^ or ^^^^ ^^^^ ^^ ^ ^^ ^^^ ^^^^ ^^^^ = 1, else otherwise Here, 0 < ^^^^ < 1 is a threshold value, such as 0.9, for example. Note that when code-blocks ^^^^ ^^^^ ^^^^ ^^^^ and ^^^^ ^^^^ ^^^^ ^^^^ both involve the same quantization step size Δ, as may be required by many embodiments, the common factor Δ 2 can be eliminated from the calculation of ^^^^ + ^ ^^^ ^^ ^^ ^^ ^^ ^ ^^^ ^^^^ , ^^^^ ^ ^^^ ^^ ^^ ^^ ^^ ^ ^^^ ^^^^ and ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ , since these quantities are only compared with each other or with zero. It should be apparent to a reader skilled in the art, that there are many variations of the method described here that could be expected to have similar properties. For example, the squaring operations involved in the calculation of ^^^^ + − ^ ^^^ ^^ ^^ ^^ ^^ ^ ^^^ ^^^^ , ^^^^ ^^^^ ^^ ^^ ^^ ^^ ^ ^^^ ^^^^ and ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ , could all potentially be replaced by absolute values in order to reduce numerical complexity. 1.2 2 nd Aspect: Temporal inconsistency tracking The purpose of inconsistency tracking is to keep track of temporal change over the region of support of cell ^^^^ ^^ ^ ^ ^ ^ ^ ^ ^ ^ ^^ ^^^^ since the reference for cell ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^ ^^ ^^^^ was last updated – i.e., over frames ^^^^ in the range ^^^^ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^ ^^ ^^ ^^ ^^^^ ^^^^ < ^^^^ ≤ ^^^^ ^^^^ ^^^^ ^^^^ . The information of relevance is identified by two binary quantities ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ ∈ {0,1} and ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ ∈ {0,1}, where ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ = 1 means that inconsistency has been observed within a previous frame ^^^^ since the reference for cell ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^ ^^ ^^^^ was last updated, i.e., ^^^^ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^ ^^ ^^ ^^ ^^^^ ^^^^ < ^^^^ < ^^^^ ^^^^ ^^^^ ^^^^ , while ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ = 1 identifies a novel temporal inconsistency observation within the current frame ^^^^ ^^^^ ^^^^ ^^^^ , so that ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ will be 1 in the next frame, unless the reference for cell ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^ ^^ ^^^^ is updated. ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ can be interpreted as an “inconsistency trigger” signal, which is actually formed from the inclusive OR of one or more inconsistency trigger signals ^^^^ (1) ^ ^^^ ^^ ^^ ^^ ^^ ^^^^ ^^^^ , ^^^^ (2) ^ ^^^ ^^ ^^ ^^ ^^ ^^^^ ^^^^ , …, that can be produced by various methods of the invention. That is, 16 20386987_2 (GHMatters) P120441.PCT The first of these trigger signals is given by so that ^^^^ (1) = 1 identifies the frame as one in which cell ^^^^ ^ ^^^ ^^ ^^ ^^ ^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ is inconsistent with its reference ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ , where the inconsistency has not previously been observed, at least since the reference code-block associated with ^^^^ ^^ ^ ^ ^ ^ ^ ^ ^ (1) ^ ^^ ^^^^ was last updated. In this sense, ^^^^ ^^^^ ^^ ^^ ^^ ^^ ^^^^ ^^^^ = 1 does indeed represent a novel inconsistency observation in the current frame. Novelty is the key property that allows temporal inconsistency in one cell to be used to infer inconsistency in other cells, given that the cells can have very different reference frames ^^^^ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^ ^^ ^^ ^^ ^^^^ ^^^^ . Figure 2 illustrates some of the temporal inconsistency tracking concepts and notation used here and in later aspects of the invention. The figure shows how novel inconsistency triggers and induced inconsisency triggers ^^^^ (2) and ^^^^ (3) from th rd th ^ ^^^ ^^ ^^ ^^ ^^ ^^^^ ^^^^ ^^^^ ^^ ^^ ^^ ^^ ^^^^ ^^^^ e 3 and 4 aspects of the invention yield values for ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ and ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ , depending on each cell’s reference frame ^^^^ ^^^^ ^^ ^^ ^^ ^^ ^^^^ ^^^^ . Figure 3 is an illustration of temporal inconsistency tracking for 6 cells 210a-f, which form two cell groups 220, 230 in accordance with the 3 rd aspect of the invention, showing “co- inconsistency induction” within cell groups, and “inter-inconsistency induction” (Aspect 4 of the invention) between cell groups of different resolutions. Cell indices ^^^^ = 1,2,3,4,5,6 are used for simplicity here and = ^^^^ ^^^^ ^^^^ ^^^^ identifies the selection of the next reference frame for cell ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^ ^^ ^^^^ as the current frame ^^^^ ^^^^ ^^^^ ^^^^ , when ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^ ^^ ^^^^ comes from a high quality code-block that is entered into the reference buffer, as explained in Aspect 6 of the invention. A simple method for constructing ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ within the resilient decoder is to maintain a persistent binary state variable (a 1-bit memory) for each cell of a frame, such that ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ is the value of this state variable. The binary state variable whose value is ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ is initialized to 0 prior to any processing by the resilient decoder. Once all processing for cell ^^^^ ^^ ^ ^ ^ ^ ^ ^ ^ ^ ^^ ^^^^ is complete within a current frame, the binary state variable is updated according to where ^^^^ ^^^^ ^ ^ ^^^ ^^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^^^^_ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ≠ ^^^^ ^^^^ ^ ^ ^^^ ^^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^^^^ ^ ^^^ ^^^^ means that the reference code-block ^^^^ ^^^^ ^^^^ ^^^^ associated with cell ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^ ^^ ^^^^ is being updated within the reference buffer, as described in Section 3.6. Another method for constructing ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ is described in Section 3.3, which can be more efficient in terms of the memory required to maintain temporal inconsistency state information. In some embodiments of the invention, cells that are initially assigned an ambiguous label 0 by the methods described in Section 3.1 have their label reassigned immediately to Λ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ = −1 (inconsistent) if ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ = 1 or ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ = 1. However, preferred embodiments of the invention defer such reassignment until after the methods described in Section 3.4 have been applied; this allows for additional conditions to be imposed on the propagation of 17 20386987_2 (GHMatters) P120441.PCT temporal inconsistency across resolution levels. Aspect 5 of the invention ultimately uses ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ and ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ to resolve ambiguous cell classification labels, potentially making other adjustments to Λ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ , as described in Section 3.5. In some embodiments of the invention, if ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ = 1, an elevated threshold ^^^^ is used in (4) to identify cell pair inconsistency for photographic video content; this adds hysteresis to the cell labelling process, so that the assignment Λ ^^^^ ^^ ^^ ^^ ^^ ^ ^^^ ^^^^ = −1 becomes increasingly likely after temporal inconsistency has been detected in an earlier frame. 1.3 3 rd Aspect: Combining information from co-located cells In many embodiments of the invention, the compressed frame data consists of multiple sub- bands that can be partitioned into cells of the same size, such that a cell in one sub-band is associated with essentially the same image region as corresponding cells in other sub-bands. In the case of JPEG 2000 and other wavelet based encoding techniques, each resolution level of one component usually consists of three sub-bands, known as the HL, LH and HH sub- bands, which have this property, so that cells naturally come in triplets consisting of one HL cell, one LH cell and one HH cell, as illustrated in Figure 1 and Figure 2. Colour imagery also consists of multiple colour components (or colour planes), each of which have their own sub- bands such that a cell in a sub-band of one colour component can be associated with the same image region as corresponding cells within corresponding sub-bands of each of the other colour components. Thus, in many applications 9 sub-bands can be grouped together, consisting of the HL, LH and HH sub-bands at a given resolution, from each of 3 colour components, such that each cell in one sub-band is naturally associated with a corresponding cell in each of the other 8 sub- bands in the group. If some components are sub-sampled, such associations may require the use of different cell sizes in corresponding sub-bands of the different colour components, or the sets of grouped sub-bands may be smaller. In general, though, cell groups ^^^^ ^^^^ ^ ^^^ ^^^ can be identified, such that all cells ^^^^ ^^^^ ^ ^ ^ ^^ ^^^^ with ^^^^ ∈ ^^^^ ^ ^ ^^^^ ^ ^^ ^^^ ^ ^^^ ^^^^ ^^^^ are spatially co-located within different sub- bands. Since all cells in a group describe the same image region, they can be used to provide additional evidence of both temporal inconsistency and temporal consistency. This is done using two steps, known as “co-inconsistency induction” and “co-consistency induction,” which are applied in that order. The “co-inconsistency induction” step is performed first. This step produces a second inconsistency trigger signal for each cell ^^^^ ^^ ^ ^ ^ ^ ^ ^ ^ ^ ^^ ^^^^ in group , according to i f ^^^^ ^^^^ = 0 and ^^^^(1) ^ ^^ ^^^^ ^^^^ = 1 for any ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^ ^^^^ ∈ ^^^^ ^^^^ ^^^^ ^^^^ otherwise Following the methods of Section 3.2, this second inconsistency trigger signal augments the number of conditions under which the temporal inconsistency variables ^^^^ ^^^^ ^^ ^^ ^^ ^^ ^ ^^^ ^^^^ and ^^^^ ^^^^ ^^ ^^ ^^ ^^ ^ ^^^ ^^^^ can become non-zero. Similar to ^^^^ (1) ^ ^^^ ^^ ^^ ^^ ^^ ^^^^ ^^^^ = 1, the event ^^^^ (2) ^ ^^^ ^^ ^^ ^^ ^^ ^^^^ ^^^^ = 1 represents the observation of novel temporal inconsistency in the current frame. Figure 2 shows two examples of “co- inconsistency induction.” 18 20386987_2 (GHMatters) P120441.PCT The grouping of cells actually presents an opportunity to reduce the amount of persistent memory required to determine these temporal inconsistency variables. Rather than assigning one persistent binary state variable to each cell, for large groups it is preferable to assign a single ^^^^-bit state variable ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ to each group ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ , where ^^^^ ^^^^ ^^^^ ^^^^ − ^^^^ ^^^^ ^^^^ can be interpreted last frame for which ^^^^ ^ ^^^^ ^^^^ ^^^^ as the index ^^^^ of the ^^^ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ = 1 for any ^^^^ ∈ ^^^^ ^^^^ ^^^^ ^^^^ . In this approach, ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ itialized to the maximum possible value, 2 ^ ^ ^ ^^^ is in ^^ − 1, which is interpreted as meaning that no cell in group has observed temporal inconsistency since its reference was last updated. For each cell ^^^^ ^^ ^ ^ ^ ^ ^ ^ ^ ^ ^^ ^^^^ belonging to group ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ , ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ is derived from ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ according to: Finally, after all cells in group have been processed, ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ is updated according to: 1 if ^^^^ = 1 for ^^^^ ^^^^ any ^^^^ ∈ ^^^^ ^^^^ ^^^^ ^^^^ ^ ^ ^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^ ^ ^^^ ^^^^ ^^^^ ←� m in {2 ^^^^ − 1, ^^^^ ^^^^ + 1} otherwis . ^ ^^^ ^^^^ ^^^^ ^^^^ e This alternative method for tracking temporal inconsistency, produces the same values for ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ and ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ as the simple method described in Section 3.2, so long as the reference frame gap ^^^^ ^^^^ ^^^^ ^^^^ − ^^^^ ^^^^ ^ ^ ^^^ ^^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^^^^ ^ ^^^ ^^^^ for every cell in the frame is strictly less than 2 ^^^^ . If some cells have a larger reference frame gap, then the method can lose track of temporal inconsistency that has not been observed for many frames, but this is not inherently damaging to the performance of the resilient decoder, so long as ^^^^ is not too small. Preferred embodiments of the invention use ^^^^ = 4 or ^^^^ = 3, which significantly reduces the amount of persistent state memory required for temporal inconsistency tracking when the number of cells in a group is 9. The “co-consistency induction” step is performed after “co-inconsistency induction.” In this step, if a cell ^^^^ ^^^^ , ^^^^ ^ ^^^ ^^^^ ^^^^ belonging to group ^^^^ ^^^^ ^^^^ ^^^^ , has been assigned an ambiguous classification label Λ ^^^^ = ar ^^^^ ^ ^^^ ^^^^ ^^^^ ^^^^ e all 0, and if any other cell ^^^^ ^^^^ ^^^^ ^^^^ belonging to the same group is classified as consistent Λ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ > 0, with ^^^^ ^^^^ ^^^^ ≤ ^^^^ ^ ^^^ ^^^^ ^^^^ ^^^^ ^^ ^^ ^^ ^^ ^^^^ ^^^^ , then cell ^^^^ ^^^^ ^^^^ ^^^^ ’s classification label is changed to Λ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ = 1 + ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ , which is 1 (consistent with a preference for > Π ^^^^ , or 2 (consistent with a preference for preference for ^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^ ^^^^ ^^^^ ^^^^ ), if Π ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ ≤ Π ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ , as given by (1). This step recognizes the fact that observations of temporal consistency between another cell in the group and its own reference provide some assurance of temporal consistency for all cells in the group, whose reference is as recent or more recent. In some embodiments of this invention, induction is performed with a classification label Λ ^^^^ ^^ ^^ ^^ ^^ ^^^^ ^^^^ that has 3 states, given by {−1,0,1}, instead of the normal 4 states, which are {−1,0,1,2}; that is, state values of 1 and 2 are collapsed into one state value of 1 before induction. In this case, some or all induction operations, as explained in this aspect and in aspects 4 and 5 of this invention, can be performed using three-state classification labels. Once the classification label Λ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ is adjusted as explained in the 5th aspect of this invention, it is possible to convert the classification label value of 1 back to its original un-collapsed values of 1 and 2. 19 20386987_2 (GHMatters) P120441.PCT 1.4 4 th Aspect: Inferring cell consistency from lower resolution levels This 4 th aspect of the invention is applicable to the case in which the encoding of each video frame was based on a multi-resolution transform structure, so that each sub-band has an associated resolution level ^^^^, where ^^^^ = 0 is the lowest resolution, typically consisting of a single base sub-band (or LL band) and the sub-bands associated with each higher resolution ^^^^ can be understood as detail bands. In the common case of the DWT, with a classic Mallat structure, each resolution level ^^^^ > 0 has three detail sub-bands HL ^^^^ , LH ^^^^ and HH ^^^^ , whose dimensions are at least approximately proportional to 2 ^^^^ . As a result, assuming that the cell dimensions are the same in all sub-bands, most if not all, cells from an HL ^^^^ sub-band have a region of influence within the reconstructed image which spans that of four cells from the HL ^^^^+1 sub-band, if it exists, with similar statements applying to the LH ^^^^ and HH ^^^^ sub-bands. This is illustrated for two resolution levels in Figure 2. Reversing this relationship, for each resolution level ^^^^ > 1, we can say that each cell ^^^^ ^^ ^^ ^ ^ ^ ^ ^ from the HL ^^^^ sub-band has an “ancestor” ^^^^ ^^^^ ^^^ ^^^^ ^^^^ ^^ ^ ^ ^ ^ ^^ ^ ^ in the HL ^^^^−1 sub-band, whose region of influence covers that of ^^^^ ^^ ^ ^ ^ ^ ^ ^ ^ ^^ ^^^ ^^^^ . Similarly each cell ^^^^ ^ ^^^^ from the LH sub-band has an ancestor ^^^^ ^^^^ ^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^ ^ ^ ^ ^ ^^ ^ ^^^ in the LH ^^^^−1 sub-band; and each cell ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^^ ^ ^^ ^^^^ from the HH ^^^^ sub-band has an ancestor ^^^^ ^^^^ ^^^^ the HH sub-band. N ^^ ^^^^ ^^^^ in ^^ ^ ^^^−1 ot only this, but each group of cells ^^^^ from resolution level ^^^^ > 1 has an ancestor group ^^^^ ^^^^ ^^^^ ^ ^ from resolution level ^^^^ − 1, consisting of all the ancestors of the ells in group ^^^^ ^ ^^ ^^^^ ^^^^ c ^^^ ^ ^^^ ^^^^ ^^^^ . In general, ancestor relationships such as these can be found within any multi-resolution compressed representation, regardless of whether a Mallat-structured DWT is employed by the frame encoding technology, or some other multi-scale transform. Since the region of influence of a cell is covered by that of its ancestor, an ancestor can be used to provide additional evidence of both temporal inconsistency and temporal consistency. This is done using two steps, known as “inter-consistency induction” and “inter- inconsistency induction,” which are applied in that order. For cells ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^ ^^ ^^^^ that belong to a group ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ of multiple co-located cells at the same resolution, these steps are applied to the group as a whole, using all cells from ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^ to induce consistency nd then inconsistency for the cells of group ^^^^ ^ ^^ a ^^^ ^ ^^^ ^^^^ ^^^^ . The methods of this aspect of the invention are described from this cell-group perspective, but apply equally to the case in which cells are not grouped, in which case ^^^^ and ^^^^ ^^^^ ^^^^ each consist of one cell ^^^^ ^^^^ ^ only, corresponding to ^^^^ ^^^^ ^^^^ ^^^^ and its ancestor ^^^^ ^^^^ ^^^ ^^^^ ^^^^ ^^^ ^^ ^ ^^ ^ ^^^ . In multi-resolution embodiments of the invention, the methods of this aspect and all preceding aspects of the invention are not applied to a group ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^ at resolution level ^^^^ until they have been applied to the corresponding ancestor group ^^^^ ^^^^ ^ ^ ^^^ ^ ^^^ ^^^^ ^^^^ at resolution level ^^^^ − 1. In embodiments of the invention, the “inter-consistency induction” step is performed immediately after any “co-consistency” step described in Section 3.3. In this step, if a cell ^^^^ ^^ ^ ^ ^ ^ ^ ^ ^ ^ ^^ ^^^^ , belonging to group ^^^^ , has been assigned an ambiguous classification label Λ ^^^^ ^^^^ = 0, ^ ^^^ ^^^ ^^^^ ^^^^ ^^^^ and ^^^^ ^^^^ , ^^^^ (1) and ^^^^ (2) are all 0, and if any anc ^^^^ ^ ^^^^ ^ ^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^ ^^ ^^ ^^ ^^^^ ^^^^ ^^^^ ^^ ^^ ^^ ^^ ^^^^ ^^^^ estor cell ^^^^ ^^^^ ^^^^ ^^^^ belonging to ^^^^ ^^^^ ^^^^ ^^^^ is classified as consistent Λ ^^^^ ^^^^ ^^^^ > 0, with ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^ ≤ ^^^^ ^^^^ ^ ^ ^ ^ ^^ ^^^^ , then cell ^^^^ ^^^^ ’s classification label is changed to ^^^^ ^^^^ ^^^^ ^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^ ^^^^ ^^^^ ^^^^ ^^^^ (consistent with a preference for preference for ^^^^ ^^^^ ^^^^ ^^^^ ), if Π ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ ≤ Π ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ , as given by (1). 20 20386987_2 (GHMatters) P120441.PCT Note that the “inter-consistency induction” step and the “co-consistency induction” step described in Section 3.3 can actually be performed together, since they both affect cells for which Λ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ = 0, and ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ (1) (2) ^ ^^^ ^^^^ , ^^^^ ^^^^ ^^ ^^ ^^ ^^ ^^^^ ^^^^ and ^^^^ ^^^^ ^^ ^^ ^^ ^^ ^^^^ ^^^^ are all 0, providing complementary conditions under which the cell’s label is changed to Λ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ = 1 + ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ . After applying all relevant “co-inconsistency induction”, “co-consistency induction” and “inter-consistency induction” steps to group ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ , as described above, the “inter- inconsistency induction” step is performed last. This step forms a third inconsistency trigger signal each cell ^^^^ ^^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^^ ^ ^^ ^^^^ in group ^^^^ ^^^^ ^^^^ ^^^^ . In some embodiments of the invention, this third form of novel temporal inconsistency is observed if all cells ^^^^ ^^^^ in group have Λ ≤ 0 and any ancestor cell ^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^ ^^^^ ^^^^ ^^^^ in group ^^^^ ^^^^ ^^^^ ^^^^ has ^^^^ ^^^^ ^ = 1, ^^^^ (3) ^^^^ ^^^^ noting that ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ is the logical OR of all and ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^ ^^^^ ^^^^ . That is, ^ ^^^ ^^^^ ^^^^ ^^^^ (3) ^ ^^^ 1 if �Λ ^^^^ ^^ ^^ ^ ^^ ^ ^^ ≤ 0,∀ ^^^^ ∈ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ and ∃ ^^^^ ∈ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ such that ^^^^ ^^^^ ^^^^ = 1� ^ ^ ^^ ^^ ^^ ^^^^ ^^^^ =� ^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ . 0 otherwise In these embodiments, the ^^^^ (3) ^ ^^^ ^^ ^^ ^^ ^^ ^^^^ ^^^^ temporal inconsistency trigger signal can propagate from low resolution to higher resolutions recursively, being blocked only by the presence of temporally consistent cells (i.e., Λ > 0). Note, however, that the “co-consistency induction” and “inter-consistency induction” steps that are performed first, convert either all or none of the ambiguous cells in group ^^^^ to consistent cells; any such conversion both results in the recursive propagation of temporal consistency to higher resolution levels and blocks the recursive propagation of temporal inconsistency from lower to higher resolutions. Note also that ^^^^ (3) ^ ^^^ ^^^^ ^ ^ ^^ ^^ ^^ ^^^^ ^^^^ can be 1, even if ϕ ^^^^ ^^^^ ^^^^ ^^^^ = 1, meaning that temporal inconsistency has been detected in preceding frames for cell ^^^^ ^^ ^ ^ ^ ^ ^ ^ ^ ^ ^^ ^^^^ . This is not a violation of the novelty principle, since the event = 1 can only arise from the novel detection of temporal inconsistency at a lower resolution level. Figure 2 shows an example of such “inter-inconsistency induction.” In other embodiments of the invention, the ^^^^ (3) ^ ^^^ ^^ ^^ ^^ ^^ ^^^^ ^^^^ temporal inconsistency trigger signal is determined according to: otherwise Here, Θ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ ∈ {0,1} is used to identify a “power inversion” property, which is calculated from all cells within and their ancestors w ^^^^ ^^^^ ithin ^^^^ ^^^^ ^^^^ ^^^^ , based on averages of their squared reference sample values. Specifically, let ⊆ Ω ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ denote a collection of sample locations within the support of ancestor cell that most closely reflects the region of support of the higher resolution cell ^^^^ ^^ ^ ^ ^ ^ ^ ^ ^ ^ ^^ ^^^^ , noting that in typical multi-scale representations 21 20386987_2 (GHMatters) P120441.PCT the lower resolution sub-bands are sub-sampled by a factor of 2 in each direction relative to the next higher resolution sub-bands, so that the number of samples is only one quarter the number of that are in . Then write ^^^^ ^^^^ ^^^ ^ ^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ = ��Ω ^^^^ ^^ ^ ^ ^ ^^ ^^ ^ ^^^ ^^^^ � , ^^^^∈ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ so that represent the reference sample powers within and ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ , respectively, limited to just those parts of the ancestor cells whose region of influence in the image is most similar to the corresponding cells in ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ . Then where ^^^^ is a power-inversion threshold that is selected so that Θ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ = 1 corresponds to the relatively unusual condition in which the higher resolution samples have more power than their lower resolution counterparts. In these embodiments of the invention, the quantities ^^^^ ^^^^ ^^^^ and ^^^^ ^^^ ^^ ^ ^^ ^^^^ ^ ^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ can all be computed concurrently with the cell powers ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ defined in (3), without substantial implementation overhead. To understand the use of power inversion in the derivation of ^^^^ (3) ^ ^^^ ^^ ^^ ^^ ^^ ^^^^ ^^^^ above, we note firstly that power inversion is not usually stable under temporal change; even pure translation shifts can reverse power inversion, due to a lack of shift invariance resulting from the substantial spectral overlap between low- and high-pass sub-band filters in the transform. That is, even if a higher resolution cell exhibits higher power than a lower resolution ancestor at one point in time, we do not expect this to remain true if the frame data is undergoing significant geometric change. So, if temporal change exists in the higher resolution, one might reasonably expect to observe the temporal change at some point within the lower resolution also, especially because lower resolution sub-bands tend to be much more finely quantized. The above equation for ^^^^ (3) ^ ^^^ ^^^^ ^ ^ ^^^^ makes use of the power inversion attribute Θ ^^^^ only when ^^^^ ^^^^ ^^ = 1 and all cells in ^^ ^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ( 3) ^^ ^^^^ are ambiguous. If this happens then the temporal inconsistency ^^^^ ^^ ^ ^^^ ^^^^ ^^^^ we would expect to see at the lower resolution, in the event of geometric change, is not being observed, and so in the event of power inversion, it is reasonable to consider that the higher 22 20386987_2 (GHMatters) P120441.PCT power cells of ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ are likely to belong to a static texture. As a result, it is reasonable to block the propagation of temporal inconsistency from much lower resolutions, via ^^^^ (3) ^^^^ , when ^^^^ ^^^^ ^ ^^ ^^^^ ^^^^ ^^^^ Θ ^ ^^^^ = 1 an ^^^^ ^ ^^^ ^^^^ ^^^^ ^^^^ d none of the cells in ^^^^ ^^^^ ^^^^ ^^^^ has Λ < 0, which would explicitly suggest temporal inconsistency. It should be apparent to a reader skilled in the art, that there are many variations of the method described here for assessing “power inversion,” that could be expected to have similar properties. For example, the squaring operations involved in the calculation of ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ and ^^^^ ^^^ ^ ^^^ ^^ ^ ^^ ^ ^^^ ^^^^ ^^^^ , could potentially be replaced by absolute values in order to reduce numerical complexity. Note that in all embodiments of the invention discussed here, all cells ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^ ^^ ^^^^ in a group for which ^^^^ ^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ = 0, receive the same va (3) (2) ^ ^^ lues for ^^^^ ^^^^ ^^ ^^ ^^ ^^ ^^^^ ^^^^ and ^^^^ ^^^^ ^^ ^^ ^^ ^^ ^^^^ ^^^^ and hence ultimately the same value for ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ , so the memory efficient alternative method for keeping track of temporal inconsistency state information, as discussed in Section 3.3, is appropriate also to be used in conjunction with the methods of this 4 th aspect of the invention. 1.5 5 th Aspect: Adjustment of cell classification labels After applying the methods associated with the first four aspects of the invention, there are often some cells which have the ambiguous classification label Λ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ = 0. To address this, ambiguous cells have their labels converted to Λ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ = −1 (inconsistent) whenever either of the temporal inconsistency tracking variables ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ or ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ is 1. In cases where ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ and ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ are both 0, and Π ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ ≤ Π ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ , ambiguous cells may be resolved either to Λ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ = −1 or Λ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ = 2, both of which mean that the preferred way to reconstruct the frame is to use the samples of ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^ ^^ ^^^^ rather than ^^^^ ^^ ^ ^ ^ ^ ^^ ^ ^^^ ^^^^ . Otherwise, ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ and ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ are both 0 and Π ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ > Π ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ , so ambiguous cells should be resolved either to Λ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ = −1 or Λ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ = 1. In this last case, the distinction between these two possible ways to resolve ambiguity is important, since it affects the way in which the current video frame will be reconstructed, as discussed in Section 3.6. The remainder of this section is concerned with this case. When Λ ^^^^ ^^ ^^ ^^ ^^ ^ ^^^ ^^^^ = 0 and ^^^^ ^^^^ ^^ ^^ ^^ ^^ ^ ^^^ ^^^^ and ^^^^ ^^^^ ^^ ^^ ^^ ^^ ^ ^^^ ^^^^ are both 0 and Π ^^^^ ^^ ^^ ^^ ^^ ^ ^^^ ^^^^ > Π ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ , some embodiments of the invention take the conservative position that ^^^^ ^^ ^ ^ ^ ^ ^ ^ ^ ^ ^^ ^^^^ may be temporally inconsistent, converting the ambiguous label to Λ ^^^^ ^^ ^^ ^^ ^^ ^ ^^^ ^^^^ = −1. In preferred embodiments of the invention, however, the ambiguous label is converted instead to Λ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ = 1 (consistent) if the “power inversion” property holds, as described in 3.4, meaning that ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ belongs to a group ^^^^ ^^^^ ^^^^ ^^^^ for which Θ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ = 1. In applications where the resilient decoder has reason to believe that the encoded video content comes from a JSIV-like encoder or distribution server, which has intentionally assigned higher quality to code-blocks where significant temporal change was observed, preferred embodiments should convert all ambiguous labels for which ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ and ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ are both (consistent). In further embodiments of the invention, prior evidence of temporal inconsistency may be used to assist in resolving ambiguous cells for which ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ and ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ are both 0 23 20386987_2 (GHMatters) P120441.PCT Π ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ . In particular, when the memory efficient inconsistency tracking mechanism described in Section 3.3 is employed, such that each cell group ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ is assigned a persistent ^^^^-bit state- variable ^^^^ ^^^^ ^^^^ ^^^^ ^^ ^^^^ ^ ^^^ ^^^^ ^^^^ from which ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ is derived, the special case ^^^^ ^^^^ ^^ ^ ^^^ ^^^^ ^^^^ = 2 − 1 means that no recent temporal inconstency has been observed within any cell of the group. If this is true, these embodiments of the invention convert ambiguous cells within the group to Λ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ = 1 (consistent), as opposed to Λ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ = −1. Still further embodiments of the invention may use spatially neighbouring cells, whose classification labels have already been finalised, in the resolution of ambiguous cells for which ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ and ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ are both 0 and Π ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ > Π ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ . In particular, if any neighbour ^^^^ ^^^^ ^^^^ of cell ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^ ^^ ^^^^ has already been assigned a label indicating temporal consistency Λ ≤ ^^^^ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^ ^^ ^^ ^^ ^^^^ ^^^^ , then the ambiguous label for ^^^^ ^^ ^ ^ ^ ^ ^ ^ ^ ^ ^^ ^^^^ may be converted to Λ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ = 1 (consistent), as opposed In particularly conservative embodiments of the invention, when ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ = 1 or ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ = 1, the cell classification label produced by earlier aspects of the invention is always converted to regardless of the initial label assigned. In these embodiments, once temporal inconsistency has been observed in one frame, the cell will continue to be treated as inconsistent from its reference in all future frames, until the reference code-block for the cell is updated so that ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ returns to 0. In preferred embodiments, however, non-ambiguous classification labels are not modified by ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ or ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ , but the threshold ^^^^ used in (4) to identify temporal consistency for photographic video content may be adjusted by ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ , as described in Section 3.2. 1.6 6 th Aspect: Determination and management of reference code-blocks As explained previously, this invention relies upon a reference buffer to store code-blocks received from previous frames. While it is possible to store the decoded sample values for those code-blocks, preferred embodiments of the invention store only the reference code- block bit-streams received by the decoder, so as to minimize the memory size and memory bandwidth associated with reference buffer transactions. In the event that the reference buffer does store decoded sample values and these sample values are able to be updated on a cell-by-cell basis, it is sufficient simply to update the reference samples for cell ^^^^ ^^ ^ ^ ^ ^ ^ ^ ^ ^ ^^ ^^^^ with the values ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ [ ^^^^] whenever the final classification outcome Λ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ ≠ 1, since Λ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ = 1 is the only classification outcome for which the existing reference samples ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ [ ^^^^] can be considered superior to ^^^^ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ [ ^^^^]. In most embodiments of the invention, however, such cell-by-cell updates are not possible, since they require either a high memory bandwidth interface to the reference buffer or an encoded reference buffer whose content must be directly encoded within the decoder, both of which come with high implementation cost – the only exception to this is where a code-block consists of only one cell, but that special case is covered by the methods described below. In the reference buffer management methods described below, it is assumed that the reference buffer contains only code-block bit-streams previously received by the decoder. Moreover, in the event that code-block dimensions change from frame to frame, it is assumed 24 20386987_2 (GHMatters) P120441.PCT that reference code-blocks with incompatible dimensions are ignored or discarded. As a result, whenever a reference code-block exists, not only does ^^^^ ^^^^ ^^^^ ^^^^ have the same region of support as ^^^^ , but all fr ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ ames ^^^^ in the range ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ≤ ^^^^ ≤ ^^^^ ^^^^ ^^^^ ^^^^ necessarily contain code- blocks with the same region of support as ^^^^ ^^^^ ^^^^ ^^^^ . As a result, there is a well-defined sequence of average log-quantization interval length values Π ^ k ^ ^^ ^^^^ , corresponding to the Π ^^^^ ^^^^ ^^^^ ^^^^ values k r that were evaluated when ^^^^ was the current frame index, such that Π ^^^ e ^ f ^ ^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ = Π ^^^^ ^^^^ ^^^^ ^^^^ and Π ^^^^ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ = Some embodiments of the inventio k ^ ^^^ n use these Π ^^^^ ^^^^ ^^^^ ^^^^ values to help determine the point at which a reference code-block should be replaced within the reference buffer. In some embodiments of the invention, the reference buffer stores only one reference code- block ^^^^ ^^^^ ^^^^ ^^^^ for each code-block ^^^^ ^^^^ ^^^^ ^^^^ that may appear within a current video frame. In other embodiments of the invention, a rotating buffer may be employed to store all received code- blocks from the past ^^^^ frames, where ^^^^ is large enough to cover all reference-to-current frame separations ^^^^ ^^^^ ^^^^ ^^^^ − ^^^^ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ ^^^^ that are likely to be of interest. While such embodiments require more memory, they have essentially the same memory bandwidth requirement, since for each code-block ^^^^ ^^^^ ^^^^ ^^^^ in the current frame, there is still only one designated reference frame ^^^^ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ ^^^^ from which a reference code-block ^^^^ ^^^^ ^^^^ ^^^^ can be accessed by the resilient decoding methods of this invention. In general, this reference frame ^^^^ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ ^^^^ varies from block to block. Regardless of how the reference buffer is organized, the key consideration for this second aspect of the invention is the determination of ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ ^^^^ , identifying the frame index of the reference code-block that will be accessed by the resilient decoding procedure when it processes code-block ^^^^ ^^^^ ^^^^ ^^^^ within the next frame ^^^^ ^^^^ ^^^^ ^^^^ = ^^^^ ^^^^ ^^^^ ^^^^ + 1. That is, a resilient decoder that is processing a current frame ^^^^ , must determine the refer ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ ence frame ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ that will be used in the next frame. In some cases, the determination of is rendered trivial by the nature of the application, or equivalently the operating context of the resilient decoder. For example, if the invention is used only for error resilient decoding applications, then the natural choice is i f ^^^^ ^^^^ ^^^^ ^^^^ is error free if ^^^^ ^^^^ ^^^^ ^^^^ is corrupted by errors As another example, if the invention is used for resilient decoding of content delivered by a JSIV server, which actively selects the code-blocks that will be assigned non-empty bit- streams on the basis of a temporal change detection algorithm, the natural choice is n on-empty empty In other applications, an encoder or video distribution server may deliberately reduce the quality associated with certain code-blocks to conserve communication bandwidth, while including sufficient information in the communicated data to identify (mark) those code- blocks whose quality has not been reduced. This could be done as part of a communication protocol, by inserting markers into the codestream or otherwise. If the resilient decoder is aware of the protocol or marker codes, then it can use them to update as follows: 25 20386987_2 (GHMatters) P120441.PCT marked as a high quality code-block n ot marked as a high quality code-block All of these are examples of contextual hints, that allow the resilient decoder to determine ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ ^^^^ without any real decision process. The rest of this section considers the case in which there are no significant contextual hints. In the absence of contextual hints, embodiments of the invention determine ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ ^^^^ based on the cell labels Λ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ deduced in the current frame ^^^^ ^^^^ ^^^^ ^^^^ , for all of the cells ^^^^ ∈ associated with code-block ^^^^ ^^^^ ^^^^ ^^^^ , after taking into account any adjustments that are made by the various aspects of this invention. The information provided by the adjusted labels is supplemented by inherent measures of code-block quality such as the log-quantization interval statistics Π ^^^^ ^^^^ , Π ^^^^ ^^^^ ^^^^ ^^^^ and Π ^^^^ ^^^^ ^^^^ ^^^^ , as illustrated in Figure 3. Figure 4 is an illustration of reference buffer management. Code-blocks 310 arriving as part of the compressed data for a frame 320 are selectively used to update the reference buffer 330, based on contextual hints that are implied by or explicitly contained within the received compressed video data, or a decision process that takes into account the adjusted cell labels and log-quantization interval statistics To understand the decision process, note that only those cells for which Λ = 1 actually benefit from the currently installed reference code-block and this is likely to continue to be true in the future. Thus, if there are no such cells ^^^^ ∈ ^^^^ for which Λ c = 1, t ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ ^^^^ Ccur hen ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ should be updated to ^^^^ ^^^^ ^^^^ ^^^^ . Moreover, this condition is certain to occur if ^^^^ ^^^^ ^^^^ ^^^^ has the same or higher quality compared to ^^^^ ^^^^ ^^^^ ^^^^ , as identified by ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ = 1, or equivalently, Π ^^^^ ^^^^ ^^^^ ^^^^ ≤ Π ^^^^ ^^^^ ^^^^ ^^^^ . To address the more general case, in which some cells ^^^^ ∈ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ have Λ C c c ur = 1, while others do not, various heuristics can be employed to determine whether should be updated to ^^^^ ^^^^ ^^^^ ^^^^ or retained as ^^^^ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ ^^^^ , but preferred embodiments use the historical average log- quantization interval statistics introd � ^^^^ uced above. In particular, write Π ^^^^ ^^^^ ^^^^ ^^^^ for a measure of the average value of Π ^ k ^^^^ ^^^^ ^^^^ ^ ^^ ^^^^ ^^^^ ^^^^ over all frames from ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ + 1 to ^^^^. The decoder then uses Π � ^^^^ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ ^^^^ , Π ^^^^ ^^^^ ^^^^ ^^^^ and Π ^^^^ ^^^^ ^^^^ ^^^^ , together with the fraction Γ ^^^^ ^^^^ ^^^^ ^^^^ of cells in code-block ^^^^ ^^^^ ^^^^ ^^^^ for which Λ = 1, to determine whether or not the reference frame should be updated. Specifically, Π ^^^ Π� ^^^^ ^^^^ ^^^^ ^^^^ if Γ ^^^^ ^ ^^^^ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ ⋅ 4 +�1 − Γ ^^^^ ^^^^ ^^^^ ^^^^ � ⋅ 4 ^^^^ ^^^^ ^^^^ ≥ 4Π ^^^^ ^^^^ ^^^^ ^^^^ , otherwise where i f Λ ^^^^ ^^ ^^ ^^ ^^ ^^^^ ^^^^ = 1 otherwise and� ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ � is the number of cells in the partition of ^^^^ ^^^^ ^^^^ ^^^^ . 26 20386987_2 (GHMatters) P120441.PCT In the special case where ^^^^ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ ^^^^ = ^^^^ ^^^^ ^^^^ ^^^^ − 1, meaning that the reference block has only just been updated in the previous frame, Π � ^^^^ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ ^^^^ is identical to Π ^^^^ ^^^^ ^^^^ ^^^^ and so the reference selection outcome becomes In fact, whenever Π ^^^^ ^^^^ ^^^^ ^^^^ ≤ Π ^^^^ ^^^^ ^^^^ ^^^^ , no cell in the block can have Λ = 1, so that Γ ^^^^ ^^^^ ^^^^ ^^^^ is certainly 0 and the reference block is necessarily updated – ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^ ^^^ ^^^^ ^^^^ ^^^^ = ^^^^ ^^^^ ^^^^ ^^^^ . This condition also arises when there is no reference code-block, since then is taken to be unbounded (or ∞). On the contrary, if Π ^^^^ ^^^^ ^^^^ ^^^^ > Π ^^^^ ^^^^ ^^^^ ^^^^ and ^^^^ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ ^^^^ < ^^^^ ^^^^ ^^^^ ^^^^ − 1 then Π � ^^^^ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ ^^^^ is necessarily larger than Π ^^^^ ^^^^ ^^^^ ^^^^ , since the fact that the reference block was not updated at any of the frame indices ^^^^ > ^^^^ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ ^^^^ means that Π ^ k ^ ^^ ^^^^ ^^^^ ^^^^ must have been larger than for each ^^^^ contributing to the average in Π � ^^^^ ^^^^ ^^^^ ^^^^ . It follows that the reference block can only be updated � ^^^^ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ ^^^^ if Π ^^^^ ^^^^ ^^^^ ^^^^ > Π ^^^^ ^^^^ ^^^^ ^^^^ , meaning that the current instance of the code-block has higher quality than the average experienced since the reference block was last updated. Moreover, the increase in quality needs to be sufficient to justify an update, considering the extent to which the reference block’s cells are still being beneficially used by the resilient decoding procedure. In some embodiments of the invention, the average Π � ^^^^ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ ^^^ is computed by maintaining an accumulator Π ^ ^^ ^ ^ ^ ^ ^ ^^^^ ^^^^ ^ ^^^^ ^^^^ ^^^^ for each code-block ^^^^ ^^^^ ^^^^ ^^^^ , which is updated according to Π ^ ^^ ^ ^ ^ ^ ^ ^^^^ ^^^^ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ ← Π ^^^^ ^^^^ ^^^^ ^^^^ + Π ^^^^ ^^^^ ^^^^ ^^^^ during the processing of a current frame ^^^^ ^^^^ ^^^^ ^^^^, after which Π � ^^^^ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ ^^^^ is found from decision is subsequently made to set ^^^^ ^^^^ ^^^^ ^^^^_ ^^^^ ^^^^ ^^^^ ^ ^^^ ^^^^ ^^^^ ^^^^ = ^^^^ ^^^^ ^^^^ ^^^^ , then the accumulator Π ^^^^ ^^^^ ^^^^ ^^^^ is reset to 0. In other embodiments of the invention, a recursive averaging procedure may be employed, which may have slightly lower implementation complexity. In all such embodiments, however, one statistic related to Π ^ ^^ ^ ^ ^ ^ ^ ^^^^ ^^^^ must be stored for each code- block in a frame. 1.7 7 th Aspect: Cell merging and decoded frame reconstruction After generation and adjustment of cell labels according to all previous aspects of the invention, no ambiguities remain, so that each cell ^^^^ ^^ ^ ^ ^ ^ ^ ^ ^ ^ ^^ ^^^^ has one of the labels Λ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ = −1 (inconsistent), Λ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ = 1 (consistent, with a preference for = 2 (consistent, with a preference for ^^^^ ^^ ^ ^ ^ ^ ^ ^ ^ ^ ^^ ^^^^ ). Embodiments of this invention then reconstruct the frame using the sub-band samples y ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ [ ^^^^ ] if Λ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ = 1 and y ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ [ ^^^^ ] if Λ ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ ≠ 1, for all ^^^^ ∈ Ω ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ . The reconstruction is completed by applying any relevant inverse transformations (e.g., inverse DWT transformation, in the case of JPEG 2000) to the resulting reconstructed sub-band samples. In embodiments that process code-blocks produced by block-based transforms, as opposed to overlapped transforms such as the DWT, it is preferable to merge the two candidates y ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ and y ^^^^ ^^ ^^ ^ ^ ^ ^ ^ ^^^ ^^^^ for a cell’s reconstructed sample values in a manner that is less likely to create artificial discontinuities at the boundaries between transform blocks. One way to do this is to 27 20386987_2 (GHMatters) P120441.PCT use the well-known Laplacian Pyramid blending approach(for example as described in E. Adelson, C. Anderson, J. Bergen, P. Burt and J. Ogden, "Pyramid methods in image processing," RCA Engineer, vol.29, no.6, pp.33-41, 1984), but other methods will be known to those skilled in the art. 1.8 8 th Aspect: A method to reduce block-decoding resources The resilient decoding processes described in this invention require up to two separate code- blocks to be decoded for each actual code-block in the compressed representation of a video frame. This doubling of decoding resources is the main implementation cost of the approach, in comparison to a simple non-resilient decoder. Of course, two full code-blocks only need to be decoded if both ^^^^ ^^^^ ^^^^ ^^^^ and ^^^^ ^^^^ ^^^^ ^^^^ are non-empty, but even if one of the two is often empty, real-time hardware deployments normally need to provide sufficient resources to address the worst case. When the sub-bands arise from a multi-resolution transform of each video frame, as described in Section 3.4, one way to greatly reduce the decoding resources required for resilient decoding is to arrange for the code-blocks from one or more of the highest resolution sub-bands to be assigned classification labels that depend only on information that can be deduced from lower resolution levels, together with code-block attributes that can be inferred without actual decoding. Specifically, each such high-resolution code-block ^^^^ ^^^^ ^^^^ ^^^^ is assigned exactly one cell ^^^^ ^^^^ ^^^^ ^^^^ such that Ω ^^^^ ^^^^ ^^^^ ^^^^ = Ω ^^^^ ^^^^ ^^^^ ^^^^ , and an initially ambiguous classification label Λ ^^^^ ^^^^ ^^^^ ^^^^ = 0, together with Π ^^^^ ^^^^ ^^^^ ^^^^ = Π ^^^^ ^^^^ ^^^^ ^^^^ and Π ^^^^ ^^^^ ^^^^ ^^^^ = Π ^^^^ ^^^^ ^^^^ ^^^^ log- quantization interval properties that are deduced from the number of available coding passes in ^^^^ ^^^^ ^^^^ ^^^^ and its reference ^^^^ ^^^^ ^^^^ ^^^^ , as described in Section 3.1. The methods of Aspect 4 and Aspect 5 of the invention are then used to resolve the initially ambiguous label for each high- resolution cell into a non-ambiguous label ∈ {−1,1,2}, based on the more informative labels assigned to lower resolution cells, whose current and reference code-blocks are fully decoded. In this way, is determined without actually decoding ^^^^ ^^^^ ^^^^ ^^^^ or its reference ^^^^ ^^^^ ^^^^ ^^^^ , and since there is only one cell in each high-resolution code-block, the decoder proceeds as follows: • if Λ ^^^^ ^^^^ ^^^^ ^^^^ = 1, only ^^^^ ^^^^ ^^^^ ^^^^ is decoded to produce ^^^^ ^^^^ ^^^^ ^^^^ , which is then used in reconstructing the video frame; • otherwise, only ^^^^ ^^^^ ^^^^ ^^^^ is decoded to produce ^^^^ ^^^^ ^^^^ ^^^^ , which is then used in reconstructing the video frame, while also being used to update the reference buffer. It will be understood to persons skilled in the art of the invention that many modifications may be made without departing from the spirit and scope of the invention. In the claims which follow and in the preceding description of the invention, except where the context requires otherwise due to express language or necessary implication, the word “comprise” or variations such as “comprises” or “comprising” is used in an inclusive sense, i.e. to specify the presence of the stated features but not to preclude the presence or addition of further features in various embodiments of the invention. 28 20386987_2 (GHMatters) P120441.PCT It is to be understood that, if any prior art publication is referred to herein, such reference does not constitute an admission that the publication forms a part of the common general knowledge in the art, in Australia or any other country. 29 20386987_2 (GHMatters) P120441.PCT



 
Previous Patent: TSPO LIGANDS

Next Patent: BUIDING METHOD AND PANEL