5. Packaged Utilities

Griddr comes bundled with Python utilities which allow for simple and flexible production of images and the automation of movie frame generation.

griddr_csvimg

The tool for making images.

Has the following command line flags, each of which expect a parameter to follow (typically integer, floating point number or string. Strings do not require quotes):

  • --clmn
    • sets the column to be used as the main plotting dataset (integer argument)
  • --format
    • output image format, defaults to png
  • --vmin
    • the main dataset or value axis limit minimum value. If --log is set, this is the logged value
  • --vmax
    • the main dataset or value axis limit maximum value. If --log is set, this is the logged value
  • --colormap
    • the plotting colourmap, i.e. the colours which are mapped onto the main dataset defined by --clmn. Defaults to ‘spectral’
  • --log
    • sets whether the main dataset values are logged prior to plotting, expects 1 or 0. Defaults to 1
  • --interp
    • the type of interpolation or smoothing used between adjacent pixels. ‘nearest’ is technically correct and default, ‘bicubic’ is smoother
  • --preview
    • defines whether generated images are saved or displayed on screen in a preview. Expects 1 or 0 and defaults to 0
  • --echo
    • defines whether or not value axis limits are printed to the screen. Expects 1 or 0 and defaults to 1
  • --norm-clmn
    • sets the column to be used for normalisation of the main dataset to correct for kernel biases. The column to be used will typically be named N_x where x is the corresponding particle type
  • --weight-clmn
    • sets the column to be used for the weighting (opacity) of the colours defined by the main dataset
  • --log-weights
    • sets whether the weight values are logged prior to plotting, expects 1 or 0. Defaults to 1
  • --wmin
    • the weight axis limit minimum value. If --log-weights is set, this is the logged value
  • --wmax
    • the weight axis limit maximum value. If --log-weights is set, this is the logged value
  • --wnorm-clmn
    • sets the column to be used for normalisation of the weighting to correct for kernel biases. The column to be used will typically be named N_x where x is the corresponding particle type
  • --suffix
    • defines a string to be appended to the output filenames after the index
  • --ref-res
    • used to set a ‘reference resolution’ to maintain constant axis limits when changing resolutions
  • --mask
    • adds a circular mask to the outputted image if 1, for use with fisheye images
  • --redshift
    • sets a reference redshift with which star formation times are compared when colouring by age

The standard colourmaps available can be found listed in the matplotlib documentation.

Several custom colourmaps have been made (roughly described here) for use with specific simulation components

Component (coloured by) Colourmap name
Dark Matter (velocity dispersion) PurpYelWhite
Gas (temperature) RedYelWhiBlue
Stars (age) BlueRed1

transitions

The tool for making movie frames.

When using the simulation to track the evolution of an object, i.e. using it to automatically read in a series of snapshots, the filenames must conform to certain rules for the numerical index. The index must always have a constant number of digits (therefore smaller indices must be left-padded with zeros) and unless used very carefully, the file indices must be sequential with no gaps.

Additionally, the code has the ability to re-centre the simulation data automatically by reading an ASCII (plain text, three coordinates on a line separated with white space) file containing the coordinates to accompany each snapshot. The filenames by default have the filenames snap_????.com.asc, however these can be overridden through use of prefix and suffix parameters.

The main driver function is the execute/batch execute function which takes the instructions given describing the desired animation and converts this into a set of instructions for Griddr and griddr_csvimg, either on a local PC or via a batch queue on a supercomputer.

Its parameters are:
  • sequence
    • the list of transitions objects
  • runname
    • the run name, used as output filename root
  • iname = ''
    • if using only one file, the input file name (otherwise, filename patterns are constructed using a combination of prefixes and suffixes)
  • iformat = 1
    • the format of the input file(s) to be read by Griddr
  • rundir = './'
    • the directory to work in, the code will place output files here
  • overwrite = False
    • if an exiting image file matches the exact filename as a planned output, this sets whether the corresponding frame is skipped, or the files overridden
  • index_digits = 4
    • the number of digits in the index number portion of input filenames, e.g. snapshot_??? is 3, snapshot_????.hdf5 is 4
  • prefix = ''
    • any text to prepend to the input filename string such as the path to the directory containing files and the filename root
  • suffix = ''
    • any text to be appended after the input filename index, e.g. .e.hdf5 is commonly used for enhanced files
  • max_cores = 150
    • when used in a batch (supercomputer) system, the maximum number of processors to occupy
  • rendering_cmd = ''
    • the string containing all parameters to be passed to a script to perform the rendering of images after Griddr is called
  • rendering_script = '~/Codes/griddr/bin/griddr_csvimg.py'
    • the location of the rendering script
  • img_format = '.png'
    • the output rendered image format
  • executable = '~/Codes/griddr/Griddr-P'
    • the path to, and name of, the Griddr executable to be used
  • keep_csv = False
    • if False, intermediate .csv files are deleted once the rendering script has been called and an image should have been created
  • preview = False
    • when True, rather than producing an image file, the rendering script attempts to open the image in a window

Example Workflow

Warning

Under Construction

1. Sanity Check

Run Griddr for a single representative image.

./Griddr-P --p ParameterFile

Perhaps start with auto limits if not known (check the calculated ones are sensible) and maybe sanity check the .csv to make sure there are some data in there. You may wish to use a lower resolution to make sure the runtime is short.

2. Initial Image

Use griddr_csvimg many times to fine-tune what you want to show, setting limits and colour maps. Keep note of the reference resolution for future, --ref-res is equal to the product of the number of pixels on each dimension. Your first run will be relatively simple

~/Codes/griddr/griddr_csvimg.py /home/user/out/trans_000.csv --clmn 10 --norm-clmn 4

but can grow complex as you specify more options

~/Codes/griddr/griddr_csvimg.py /home/user/out/trans_000.csv --clmn 10 --interp bicubic --echo 1 --colormap PurpYelWhite  --weight-clmn 9  --norm-clmn 4 --wnorm-clmn 4 --log 1 --vmax 1.8 --vmin 0.1 --wmax -1.0 --wmin -6.0 --ref-res 2359236 --suffix _DM

Once you have the parameters for your first image decided, you may wish to move away from the use of a parameter file (if you have been using one so far) and use only the command line to ensure you know every parameter which is in use for the run and don’t need to worry about the parameter file being changed. You may end up with something like this

~/Codes/griddr/Griddr-P --lMode 2 --i /home/user/snapshot_125.e.hdf5 --rx0 13.3247 --ry0 34.8412 --rz0 40.8917 --theta 586.114239739 --phi 0 --gdx 4 --gdy -1 --gdz 4 --gx0 25 --gy0 25 --gz0 25 --o /home/user/out/trans_000.csv --iFormat 9 --Nx 4096 --Ny 4096

3. Test Movie

Using the image limits deduced in step 2, start building up a transitions file which will make up your movie. It is probably advisable to copy an existing script and modify it to your new needs. If using an existing file, you will update the input/output file names, locations and formatting as well as pasting the string of commands you wish passed to griddr_csvimg into the ‘execute’ command

rendering_cmd= ' /home/user/out/trans_000.csv --clmn 10 --interp bicubic --echo 1 --colormap PurpYelWhite  --weight-clmn 9  --norm-clmn 4 --wnorm-clmn 4 --log 1 --vmax 1.8 --vmin 0.1 --wmax -1.0 --wmin -6.0 --ref-res 2359236 --suffix _DM',

Other tips include using the lower image resolution and using less frames than you’ll want in the final movie, perhaps using 1/10 of the final frame rate.

4. Final Tests

Once you have built up the complexity of the transitions and checked that they link smoothly, you may want to render a sub-section of the movie at full frame rate and resolution before committing to the final rendering. This ensures that the machine in use has sufficient memory for the operation and that the limits are set correctly.

5. Rendering

Now you’re ready to render the whole movie, if you have access to a supercomputer, you may now wish to move the scripts to be produced in the batch queue, otherwise prepare for a long duration of hard use on your machine.