GitHub

xtool

Xtool, part of the gl package, provides a utility to calculate differences between GRIB/FA files and produce the result in a new GRIB/NetCDF file. See xtool part of util/gl/README. The main commands are:

 
                          xtool                                 
 
 Simple usage:                                                  
  xtool -f1 FILE1 [ -fN FILEN] -op OPERATOR [ -o OFILE]       
  i.e. apply OPERATOR on FILEN and output to OFILE              
  operator is one of SUM/DIFF/AVE/STDV/RMS/MS                      
  1 file : SUM/AVE/SQR                                          
  2 files: SUM/AVE/PROD/DIFF/RMSE/STDV/SAL                      
  3 files: DIFF/SAL                                             
 
 Accumulated usage:                                             
  xtool -sdtg1 YYYYMMDDHH -edtg1 YYYYMMDDHH -ll1 LLL -ll2 LLL \ 
        -p1 PATH1 -p2 PATH2 -fcint HH -op OPERATOR              
  i.e. accumulate OPERATOR on FILE1 and FILE2 during the period 
  sdtg1 to edtg1 with step of fcint.                            
  sdtg2/edtg2=sdtg1/edtg2 - (LL1-LL2) unless given              
 
 Time information in the path should be given as                
  -p1 /data/test/fc@YYYY@@MM@@DD@@HH@+@LLL@ or something like   
  -p2 /data/@YYYY@/@MM@/@DD@/@HH@/fc@YYYY@@MM@@DD@_@HH@+@LLL@   
 
 Different parameters can be compared by using the xkey         
 variable in the namelist                                       
 
 Flag summary :                                                 
  -h                : Print this help                           
  -fN FILE          : Single file name                          
  -pN PATH          : Path name                                 
  -sdtgN YYYYMMDDHH : Start date/time                           
  -edtgN YYYYMMDDHH : End date/time                             
  -llN LLL          : Forecast length to use                    
  -fcint HH         : Forecast cycle interval in hours          
  -iN               : format of the input file GRIB/FA          
  -s                : Run silent                                
  -of               : Output format GRIB/FA/NETCDF/SCREEN       
  -g                : Prints ksec/cadre/lfi info                
  -p                : Use FA file without extension zone        
  -f                : Global FA switch                          
  -a FILE           : Accumulation file to be read              
  -n namelist       : Use namelist to set additional options    
  -igd              : Set lignore_duplicates=F                  
  -r VALUE          : Rescale the output with a constant VALUE

DIFF

One of the things xtool is useful for is to check if the result from two different experiments differ. This is done by applying the DIFF operator and writing the output to the screen like

 xtool [-f] -f1 FILE1 -f2 FILE2 -of SCREEN

The is heavily used in the Harmonie testbed to check the difference between versions of the system.

Below is a simple example of how to use xtool. You may also check scr/Fldver to find examples.

What is the difference between +24h and +48h MSLP forecasts during August 2008?

  1. Namelist for xtool, which lists the parameters (here mean sea level pressure) to be examined:

    &NAMINTERP
  PPPKEY%ppp = 001,
  PPPKEY%lll = 000,
  PPPKEY%ttt = 103,
/

  2. Run xtool.

    xtool -sdtg1 2008080100 -edtg1 2008083000 -ll1 48 \
      -sdtg2 2008073100 -edtg2 2008082900 -ll2 24 \
      -p1 /your_model_data/YYYY/MM/HH/fcYYYYMMDD_HH+LLL \
      -p2 /your_model_data/YYYY/MM/HH/fcYYYYMMDD_HH+LLL \
      -fcint 6 -op DIFF -n your_namelist \
      -o output.grb
    • -sdtg1, -edtg1, -ll1: The cycles to look for the +48h forecast.
    • -sdtg2, -edtg2, -ll2: The cycles to look for the +24h forecast.
    • -p1, -p2: Naming rules for the files in cycle 1 and 2, respectively.
    • -fcint: Interval between forecast cycles.
    • -op: Operation to be applied. Possible choices DIFF, SUM, AVE, STDV or SAL
    • -n: Namelist file.
    • -o: Name of the output grib file.

  3. Output file (output.grb) now contains one 2D-field with accumulated 48-24h difference of mean sea level pressure.

Deaccumulate with xtool, setting outkey%startstep

Accumulated fields, with tri=4, such as precipitation and radiation fields, are accumulated from the start of a cycle. Often it is more interesting to know for example the hourly amount. This can be also be done with xtool DIFF, using the steps below, assuming regular 1h output intervals:

  1. The first output file (+0h) does not contain accumulated fields, so skip that one
  2. The second file contains the amount between 0 and 1, so no need to subtract, simple convert with gl
  3. For all consecutive steps use xtool, e.g.:
    xtool -f -f1 $infile -f2 $infile_prev -op DIFF -of NETCDF -n $def_nam -igd

This subtracts the fields in $infile_prev from those in $infile. The namelist file should contain at least the names of the accumulated fields to use and the outkey parameters:

&naminterp
readkey%name = "YOUR_ACCUM_FIELD",
outkey%date = $YMD
outkey%time = ${HH}00
outkey%startstep = $FF
outkey%endstep = $FF_prev
/

where $YMD and $HH are the start of your cycle, $FF is the step of $infile and $FF_prev of the previous file. For example to calculate the amount of rain between hour 7 and 8 of a forecast started at 2017030300 use:

xtool -f -f1 ICMSHHARM+0008 -f2 ICMSHHARM+0007 -op DIFF -of NETCDF -n $def_nam -igd

def_nam contains:

&naminterp
readkey%name = "SURFACCPLUIE",
outkey%date = 20170303
outkey%time = 0000
outkey%endstep = 8
outkey%startstep = 7
/

This is used scr/convertFA to deaccumulate fields to NetCDF for climate simulations.

SAL

Structure Amplitude Location (SAL) is object based quality measure for the verification of QPFs (Wernli et al., 2008). SAL contains three independent components that focus on Structure, Amplitude and Location of the precipitation field in a specified domain.

  • S: Measure of structure of the precipitation area (-2 - +2). Large S, if model predicts too large precipitation areas.

  • A: Measure of strength of the precipitation (-2 - +2). Large A, if model predicts too intense precipitation.

  • L: Measure of location of the precipitation object (0 - +2). Large L, if modelled precipitation objects are far from the observed conterparts.

  • SAL can be activated in xtool by using -op SAL option. e.g.

     xtool -f1 model.grib -f2 observation.grib -op SAL -n namelist

  • Output of the SAL are 2 simple ascii-files:

    1. scatter_plot.dat containing date, S,A and L parameters.
    2. sal_output.dat containing more detailed statistics collected during the verification (location of center of mass, number of objects, measure of object size etc.).