|
The OPFoF was build for the parallel Friend-of-Friend halo findings
when total system memory is too small to load all the particle data simultaneously.
Its routine assumes to read particle data saved by the GOTPM
which make each rank (or cpu) output its own particle data separately.
Below shows how it works with multiple cpus;
Nfile/Ncpu = M where Nfile is the number of particle data files, Ncpu is the number of CPUs applied to the OPFOF job, and M is the number of consecutive (in number) data files assigned to each CPU. It should be an integer larger than 1. So, Nfile should be divisible by Ncpu. Then each CPU has its own local domain from z_lower=MIN(z_min(i)) to z_upper = MAX(z_max(i)), where i is running index for the local file. Actually, each CPU read its designated files one by one after finding FoF halos for each file locally. There is one restriction on the integer value M. The FoF structure in each local domain should not cross both the boundaries of z_lower and z_upper. To check whether Ncpu violates this restriction, you should search for "num0=%ld num1=%ld num2=%ld num3=%ld" in the stdout logfile. "num0" is the number of FoF halos which are isolated (meaning they are not crossing any z-boundary), "num1" is the number of FoF halos which are only crossing lower boundary (z_lower), "num2" is the number of FoF halos which are only crossing upper boundary (z_upper), and "num3" is the number of FoF halos which are crossing both boundaries. Each cpu produces these information for each file and it is OK even though num3 is not 0 in cases the file is not the last one allocated for the cpu. Please use a sufficiently small number of M. It automatically assumes periodic boundary conditions in three dimensions. And there should a parameter file, "params.#####", to get simulation information, where OPFOF gets the periodic box size. There are two argument for opfof. The first integer argument is used to find files having a step number of the input integer. The other one is Nfile, the total number of files of particle data. The number of CPUs are read through the MPI_Comm_size function. And the linking length is set as a conventional value (0.2 times the mean particle separation). There are two output files; FoF_halo_cat.##### and FoF_mem_particle.##### where "#####" is the step number. The first one contains the basic information of the halo together with the number of member particles and the last one contains only the particle information (x,y,z,vx,vy,vz, and index number). So to read member particles, you absolutely need "FoF_halo_cat.#####" file too. To compile the source code, just run as "sh exe" in the directory. You can download source codes here ;
|