Irreversible compression/decompression DWT: use 1/K constant as per standard
authorEven Rouault <even.rouault@spatialys.com>
Wed, 20 May 2020 16:00:45 +0000 (18:00 +0200)
committerEven Rouault <even.rouault@spatialys.com>
Wed, 20 May 2020 18:31:28 +0000 (20:31 +0200)
commit3cd1305596f191a01afdc11f9355f9c6590065dd
tree5aa7c46c81d6122f38100df2f590bc915b5275a9
parentf38c069547f1c41dc94ec4a273efb07997685c21
Irreversible compression/decompression DWT: use 1/K constant as per standard

The previous constant opj_c13318 was mysteriously equal to 2/K , and in
the DWT, we had to divide K and opj_c13318 by 2... The issue was that the
band->stepsize computation in tcd.c didn't take into account the log2gain of
the band.

The effect of this change is expected to be mostly equivalent to the previous
situation, except some difference in rounding. But it leads to a dramatic
reduction of the mean square error and peak error in the irreversible encoding
of issue141.tif !
src/lib/openjp2/dwt.c
src/lib/openjp2/dwt.h
src/lib/openjp2/t1.c
src/lib/openjp2/tcd.c
tests/nonregression/test_suite.ctest.in