JEG no-reference hybrid HEVC

From VQEG JEG Wiki
Revision as of 08:57, 14 February 2017 by Emasala (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

JEG no-reference hybrid HEVC

JEG is currently developing a database of HEVC-encoded video sequences to eventually develop a Hybrid No Reference metric suited for HEVC.

Contents

Main idea

Project Setup

Database

Link: ftp://ftp.ivc.polytech.univ-nantes.fr/VQEG/JEG/HYBRID/hevc_database/

Folders:

  • ENC : compressed video sequences
  • FR : available quality measures (see below)
  • SRC : uncompressed original video sequences used for encoding and quality measures

Sequences (from src01 to src10):

src01 src02 src03 src04 src05 src06 src07 src08 src09 src10

Key to compressed file naming

Example filename:

 src01_960x544p25.yuv_LDGOP4_8000001_2_16_64_8_4.265

Convention

 src<SEQN>_<WIDTH>x<HEIGHT>p25.yuv_<GOPTYPESIZE>_<RATECONTROL>_<REFRESH>_<INTRAPERIOD>_<SEARCHRANGE>_<BITDEPTH>_<SLICEARGUMENT>.265

Meaning of parameters (Refer to https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/branches/HM-11.0-dev/doc/software-manual.pdf for further details on the encoder)

SEQN = sequence number, from 01 to 10

WIDTH = picture width, possibile values: 960, 1280, 1920

HEIGHT = picture height, possibile values: 544, 720, 1080

GOPTYPESIZE = GOP type and size, possibile values: GOP2, GOP4, GOP8, LDGOP4

 GOP2 : GOP size 2
 GOP4 : GOP size 4
 GOP8 : GOP size 8
 LDGOP4 : GOP size 4 in low-delay configuration

RATECONTROL = parameter for rate control, possibile values: 26, 32, 38, 46 or a number >= 500000 (ending with 0 or 1)

 The number <= 46 corresponds to the -q encoder option
 The number >= 500000 corresponds to the --TargetBitrate encoder option
 The last digit (0 or 1) if number >= 500000 corresponds to the --LCULevelRateControl encoder option
   26 : fixed QP = 26
   32 : fixed QP = 32
   38 : fixed QP = 38
   46 : fixed QP = 46
   500000 : rate control at the frame level, 500 kbit/s
   500001 : rate control at the LCU level, 500 kbit/s
   1000000 : rate control at the frame level, 1 Mbit/s
   1000001 : rate control at the LCU level, 1 Mbit/s
   2000000 : rate control at the frame level, 2 Mbit/s
   2000001 : rate control at the LCU level, 2 Mbit/s
   4000000 : rate control at the frame level, 4 Mbit/s
   4000001 : rate control at the LCU level, 4 Mbit/s
   8000000 : rate control at the frame level, 8 Mbit/s
   8000001 : rate control at the LCU level, 8 Mbit/s
   16000000 : rate control at the frame level, 16 Mbit/s
   16000001 : rate control at the LCU level, 16 Mbit/s

REFRESH = intra refresh type, possible values: 1, 2

 The number corresponds to the --DecodingRefreshType encoder option
   1 : Applies a non-IDR clean random access point (open GOP).
   2 : Applies an IDR random access point (closed GOP).

INTRAPERIOD = intra frame period, possible values: 8, 16, 32, 64

 The number corresponds to the --IntraPeriod encoder option
   8 : one intra every 8
   16 : one intra every 16
   32 : one intra every 32
   64 : one intra every 64

SEARCHRANGE = search range for motion estimation (around the predictor), possibile values: only 64

 The number corresponds to the --SearchRange encoder option

BITDEPTH = bit depth used for coding, possibile values: only 8

 The number corresponds to the --InternalBitDepth encoder option

SLICEARGUMENT = specifies how slicing is performed for each picture, possibile values: 0, 2, 4, 1500

 The number corresponds to the --SliceMode and --SliceArgument encoder options
   0 : only one slice per frame
       (SliceMode = 0 and SliceArgument = 0)
   2 : 2 slices per frame (max 270,120,75 CTU per slice when WIDTH=1920,1280,960)
       (SliceMode = 1 and SliceArgument =  270 if 1920, 120 if 1280, 75 if 960)
   4 : 4 slices per frame (max 130,60,34 CTU per slice when WIDTH=1920,1280,960)
       (SliceMode = 1 and SliceArgument =  130 if 1920, 60 if 1280, 34 if 960)
   1500 : each slice is maximum 1500 bytes
       (SliceMode = 2 and SliceArgument = 1500)

Downloads

Database

The database of the original and processed video sequences can be downloaded at:

ftp://ftp.ivc.polytech.univ-nantes.fr/VQEG/JEG/HYBRID/hevc_database/

Please note that the whole database is larger than 300 GB. In particular:

  • the SRC folder contains the original video sequences in YUV format
  • the ENC folder contains the compressed files (.265 extension)

Objective scores

Already computed objective scores are available in the FR folder:

  • for PSNR, SSIM, VIFP: look for .csv files

FRmetricsAverage.csv contains the final average value for each sequence

FRmetricsFrameByFrame.csv contains the detailed values for each frame, for each sequence

  • for VQM, PVQM: look for the .tar.bz2 files

results_vqm001_RES_converted.tar.bz2 contains the detailed values of VQM for each frame for each sequence at resolution RES (can be 960, 1280, 1920). One csv file per sequence is provided.

results_pvqm001_RES_converted.tar.bz2 contains the detailed values of PVQM for each frame for each sequence at resolution RES (can be 960, 1280, 1920). One csv file per sequence is provided.

In both cases, in each csv file, the line "final_value" contains the final value of the metric for that sequence.

For convienience, all final_value results are included in a single csv file available from FRmetricsAverage_pvqm_vqm_psnr054_ssim_vifp.csv.bz2. Note that in this case when the PSNR value of a single frame is infinite, the value has been clipped at 54.15 dB PSNR (hence the name psnr054). This happens for some sequences which have completely black frames which are encoded perfectly. The other SSIM and VIFP values are the same as the ones available in the FRmetricsAverage.csv file, and the data format is the same.

Tools

The tools used to compute the scores:

Preliminary Analysis

Some preliminary analysis of the data contained in the database is available in this result page

Current status

The database is composed of 59520 sequences (10 different contents at three different resolutions each, encoded with 1984 different encoding parameters).

For all 59520 HEVC sequences in the database, the following objective quality metrics are available:

  • PSNR (as computed by the VQMT software)
  • SSIM (as computed by the VQMT software)
  • VIF (as computed by the VQMT software)
  • VQM (as computed by the software available at NTIA/ITS )
  • PVQM (as computed by the PVQM_EM software)

The database is currently being extended by computing the previous measures on sequences which have loss impairments. 25 different loss patterns are being used.

Priority list

Since the work of expanding the database by computing the previous measures on sequences which have loss impairments is huge, a priority list has been agreed in the periodic bi-weekly JEG conference calls.

It has been suggested to prioritize the work according to three classes:

  • GOLD - should always be done
  • SILVER - should be done if possible
  • BRONZE - the rest of the work

Other criteria include testing the extreme values first whenever reasonable.

The current proposal is:

Param/Values Value 1 Value 2 Value 3 Value 4 Value 5 Value 6
WIDTH 960 1280 1920
GOPTYPESIZE GOP2 GOP4 GOP8 LDGOP4
RATECONTROL_QP 26 32 38 46
RATECONTROL_FRAME_kbit/s 500 1000 2000 4000 8000 16000
RATECONTROL_LCU_kbit/s 500 1000 2000 4000 8000 16000
REFRESH 1 2
INTRAPERIOD 8 16 32 64
SLICEARGUMENT 0 2 4 1500


Enrico is computing PSNR, SSIM, VIF, VQM, PVQM on several combinations of such parameters. As of Jan 19, 2015, all 960x544, subject to 25 different loss patterns, are available. Only a subset of sequences at 1280x720 and 1920x1080 are currently available. More information at: http://media.polito.it/downloads/jeg/with_losses/ (see also README file there).


Analysis

It is possible to visualize the quality metrics in the database by using the tool at http://app.rawgraphs.io/ Please note that all processing is done in the browser and processing about 60,000 entries requires 20-30 seconds per operation on a recent computer. Smaller sets (e.g., 2,000 lines, which are equivalent to all HRCs for one sequence at one resolution) are processed almost immediately.

To take the most advantage from the data available at ftp://ftp.ivc.polytech.univ-nantes.fr/VQEG/JEG/HYBRID/hevc_database/FR/FRmetricsAverage.csv the file has been preprocessed to isolate the main coding parameters in single columns. The preprocessed data, as well as some subsets based on the content type and resolution, has been made available by Enrico at http://media.polito.it/downloads/jeg/encoding_quality/FRmetricsParams/ (content should be self-explaining from the filename).

To use the tool, cut&paste the content of a csv file (or download and upload it) at http://app.rawgraphs.io/, then choose a type of graph (e.g., scatter plot) and the columns that you want to use on x, y, point size and point color. For instance, x=psnr, y=ssim, point size=intrarefresh, pointcolor=slicetype gives the following graph:

Src01 Res960 scatter plot example.png