JEG no-reference hybrid HEVC
JEG no-reference hybrid HEVC
JEG is currently developing a database of HEVC-encoded video sequences to eventually develop a Hybrid No Reference metric suited for HEVC.
- ENC : compressed video sequences
- FR : available quality measures (see below)
- SRC : uncompressed original video sequences used for encoding and quality measures
Sequences (from src01 to src10):
Key to compressed file naming
Meaning of parameters (Refer to https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/branches/HM-11.0-dev/doc/software-manual.pdf for further details on the encoder)
SEQN = sequence number, from 01 to 10
WIDTH = picture width, possibile values: 960, 1280, 1920
HEIGHT = picture height, possibile values: 544, 720, 1080
GOPTYPESIZE = GOP type and size, possibile values: GOP2, GOP4, GOP8, LDGOP4
GOP2 : GOP size 2 GOP4 : GOP size 4 GOP8 : GOP size 8 LDGOP4 : GOP size 4 in low-delay configuration
RATECONTROL = parameter for rate control, possibile values: 26, 32, 38, 46 or a number >= 500000 (ending with 0 or 1)
The number <= 46 corresponds to the -q encoder option The number >= 500000 corresponds to the --TargetBitrate encoder option The last digit (0 or 1) if number >= 500000 corresponds to the --LCULevelRateControl encoder option 26 : fixed QP = 26 32 : fixed QP = 32 38 : fixed QP = 38 46 : fixed QP = 46 500000 : rate control at the frame level, 500 kbit/s 500001 : rate control at the LCU level, 500 kbit/s 1000000 : rate control at the frame level, 1 Mbit/s 1000001 : rate control at the LCU level, 1 Mbit/s 2000000 : rate control at the frame level, 2 Mbit/s 2000001 : rate control at the LCU level, 2 Mbit/s 4000000 : rate control at the frame level, 4 Mbit/s 4000001 : rate control at the LCU level, 4 Mbit/s 8000000 : rate control at the frame level, 8 Mbit/s 8000001 : rate control at the LCU level, 8 Mbit/s 16000000 : rate control at the frame level, 16 Mbit/s 16000001 : rate control at the LCU level, 16 Mbit/s
REFRESH = intra refresh type, possible values: 1, 2
The number corresponds to the --DecodingRefreshType encoder option 1 : Applies a non-IDR clean random access point (open GOP). 2 : Applies an IDR random access point (closed GOP).
INTRAPERIOD = intra frame period, possible values: 8, 16, 32, 64
The number corresponds to the --IntraPeriod encoder option 8 : one intra every 8 16 : one intra every 16 32 : one intra every 32 64 : one intra every 64
SEARCHRANGE = search range for motion estimation (around the predictor), possibile values: only 64
The number corresponds to the --SearchRange encoder option
BITDEPTH = bit depth used for coding, possibile values: only 8
The number corresponds to the --InternalBitDepth encoder option
SLICEARGUMENT = specifies how slicing is performed for each picture, possibile values: 0, 2, 4, 1500
The number corresponds to the --SliceMode and --SliceArgument encoder options 0 : only one slice per frame (SliceMode = 0 and SliceArgument = 0) 2 : 2 slices per frame (max 270,120,75 CTU per slice when WIDTH=1920,1280,960) (SliceMode = 1 and SliceArgument = 270 if 1920, 120 if 1280, 75 if 960) 4 : 4 slices per frame (max 130,60,34 CTU per slice when WIDTH=1920,1280,960) (SliceMode = 1 and SliceArgument = 130 if 1920, 60 if 1280, 34 if 960) 1500 : each slice is maximum 1500 bytes (SliceMode = 2 and SliceArgument = 1500)
The database of the original and processed video sequences can be downloaded at:
Please note that the whole database is larger than 300 GB. In particular:
- the SRC folder contains the original video sequences in YUV format
- the ENC folder contains the compressed files (.265 extension)
Already computed objective scores are available in the FR folder:
- for PSNR, SSIM, VIFP: look for .csv files
FRmetricsAverage.csv contains the final average value for each sequence
FRmetricsFrameByFrame.csv contains the detailed values for each frame, for each sequence
- for VQM, PVQM: look for the .tar.bz2 files
results_vqm001_RES_converted.tar.bz2 contains the detailed values of VQM for each frame for each sequence at resolution RES (can be 960, 1280, 1920). One csv file per sequence is provided.
results_pvqm001_RES_converted.tar.bz2 contains the detailed values of PVQM for each frame for each sequence at resolution RES (can be 960, 1280, 1920). One csv file per sequence is provided.
In both cases, in each csv file, the line "final_value" contains the final value of the metric for that sequence.
For convienience, all final_value results are included in a single csv file available from FRmetricsAverage_pvqm_vqm_psnr054_ssim_vifp.csv.bz2. Note that in this case when the PSNR value of a single frame is infinite, the value has been clipped at 54.15 dB PSNR (hence the name psnr054). This happens for some sequences which have completely black frames which are encoded perfectly. The other SSIM and VIFP values are the same as the ones available in the FRmetricsAverage.csv file, and the data format is the same.
The tools used to compute the scores:
- to compute PSNR, SSIM, VIFP: VQMT
- to compute VQM: NTIA/ITS
- to compute PVQM: PVQM_EM
- T-Labs HEVC packet loss toolchain
Some preliminary analysis of the data contained in the database is available in this result page
The database is composed of 59520 sequences (10 different contents at three different resolutions each, encoded with 1984 different encoding parameters).
For all 59520 HEVC sequences in the database, the following objective quality metrics are available:
- PSNR (as computed by the VQMT software)
- SSIM (as computed by the VQMT software)
- VIF (as computed by the VQMT software)
- VQM (as computed by the software available at NTIA/ITS )
- PVQM (as computed by the PVQM_EM software)
The database is currently being extended by computing the previous measures on sequences which have loss impairments. 25 different loss patterns are being used.
Since the work of expanding the database by computing the previous measures on sequences which have loss impairments is huge, a priority list has been agreed in the periodic bi-weekly JEG conference calls.
It has been suggested to prioritize the work according to three classes:
- GOLD - should always be done
- SILVER - should be done if possible
- BRONZE - the rest of the work
Other criteria include testing the extreme values first whenever reasonable.
The current proposal is:
|Param/Values||Value 1||Value 2||Value 3||Value 4||Value 5||Value 6|
Enrico is computing PSNR, SSIM, VIF, VQM, PVQM on several combinations of such parameters. As of Jan 19, 2015, all 960x544, subject to 25 different loss patterns, are available. Only a subset of sequences at 1280x720 and 1920x1080 are currently available. More information at: http://media.polito.it/downloads/jeg/with_losses/ (see also README file there).
It is possible to visualize the quality metrics in the database by using the tool at http://app.rawgraphs.io/ Please note that all processing is done in the browser and processing about 60,000 entries requires 20-30 seconds per operation on a recent computer. Smaller sets (e.g., 2,000 lines, which are equivalent to all HRCs for one sequence at one resolution) are processed almost immediately.
To take the most advantage from the data available at ftp://ftp.ivc.polytech.univ-nantes.fr/VQEG/JEG/HYBRID/hevc_database/FR/FRmetricsAverage.csv the file has been preprocessed to isolate the main coding parameters in single columns. The preprocessed data, as well as some subsets based on the content type and resolution, has been made available by Enrico at http://media.polito.it/downloads/jeg/encoding_quality/FRmetricsParams/ (content should be self-explaining from the filename).
To use the tool, cut&paste the content of a csv file (or download and upload it) at http://app.rawgraphs.io/, then choose a type of graph (e.g., scatter plot) and the columns that you want to use on x, y, point size and point color. For instance, x=psnr, y=ssim, point size=intrarefresh, pointcolor=slicetype gives the following graph: