石山 智明

We have simulated, for the first time, the long term evolution of the Milky Way Galaxy using 51 billion particles on the Swiss Piz Daint supercomputer with our N-body gravitational tree-code Bonsai. Herein, we describe the scientific motivation and numerical algorithms. The Milky Way model was simulated for 6 billion years, during which the bar structure and spiral arms were fully formed. This improves upon previous simulations by using 1000 times more particles, and provides a wealth of new data that can be directly compared with observations. We also report the scalability on both the Swiss Piz Daint and the US ORNL Titan. On Piz Daint the parallel efficiency of Bonsai was above 95%. The highest performance was achieved with a 242 billion particle Milky Way model using 18600 GPUs on Titan, thereby reaching a sustained GPU and application performance of 33.49 Pflops and 24.77 Pflops respectively.

We present a method to couple N-body star cluster simulations to a cosmological tidal field, using AMUSE (Astrophysical Multipurpose Software Environment).We apply thismethod to star clusters embedded in the CosmoGrid dark matter only Lambda cold dark matter simulation. Our star clusters are born at z = 10 (corresponding to an age of the universe of about 500 Myr) by selecting a dark matter particle and initializing a star cluster with 32 000 stars on its location. We then follow the dynamical evolution of the star cluster within the cosmological environment. We compare the evolution of star clusters in two Milky Way size haloes with a different accretion history. The mass-loss of the star clusters is continuous irrespective of the tidal history of the host halo, but major merger events tend to increase the rate of massloss. From the selected two dark matter haloes, the halo that experienced the larger number of mergers tends to drive a smaller mass-loss rate from the embedded star clusters, even though the final masses of both haloes are similar. We identify two families of star clusters: native clusters, which become part of the main halo before its final major merger event, and the immigrant clusters, which are accreted upon or after this event native clusters tend to evaporate more quickly than immigrant clusters. Accounting for the evolution of the dark matter halo causes immigrant star clusters to retain more mass than when the z = 0 tidal field is taken as a static potential. The reason for this is the weaker tidal field experienced by immigrant star clusters before merging with the larger dark matter halo © 2013 The Authors Published by Oxford University Press on behalf of the Royal Astronomical Society.

We present the results of the "Cosmogrid" cosmological N-body simulation suites based on the concordance LCDM model. The Cosmogrid simulation was performed in a 30 Mpc box with 2048(3) particles. The mass of each particle is 1.28 x 10(5) M-circle dot, which is sufficient to resolve ultra-faint dwarfs. We found that the halo mass function shows good agreement with the Sheth & Tormen fitting function down to similar to 10(7) M-circle dot. We have analyzed the spherically averaged density profiles of the three most massive halos which are of galaxy group size and contain at least 170 million particles. The slopes of these density profiles become shallower than - 1 at the innermost radius. We also find a clear correlation of halo concentration with mass. The mass dependence of the concentration parameter cannot be expressed by a single power law, however a simple model based on the Press-Schechter theory proposed by Navarro et al. gives reasonable agreement with this dependence. The spin parameter does not show a correlation with the halo mass. The probability distribution functions for both concentration and spin are well fitted by the log-normal distribution for halos with the masses larger than similar to 10(8) M-circle dot. The subhalo abundance depends on the halo mass. Galaxy-sized halos have 50% more subhalos than similar to 10(11) M-circle dot halos have.

As an entry for the 2012 Gordon-Bell performance prize, we report performance results of astrophysical N-body simulations of one trillion particles performed on the full system of K computer. This is the first gravitational trillion-body simulation in the world. We describe the scientific motivation, the numerical algorithm, the parallelization strategy, and the performance analysis. Unlike many previous Gordon-Bell prize winners that used the tree algorithm for astrophysical N-body simulations, we used the hybrid TreePM method, for similar level of accuracy in which the short-range force is calculated by the tree algorithm, and the long-range force is solved by the particle-mesh algorithm. We developed a highly-tuned gravity kernel for short-range forces, and a novel communication algorithm for long-range forces. The average performance on 24576 and 82944 nodes of K computer are 1.53 and 4.45 Pflops, which correspond to 49% and 42% of the peak speed.

We report on the performance of our cold dark matter cosmological N-body simulation that was carried out concurrently using supercomputers across the globe. We ran simulations on 60-750 cores distributed over a variety of supercomputers in Amsterdam (The Netherlands, Europe), in Tokyo (Japan, Asia), Edinburgh (UK, Europe) and Espoo (Finland, Europe). Regardless of the network latency of 0.32 s and the communication over 30 000 km of optical network cable, we are able to achieve ∼87% of the performance compared to an equal number of cores on a single supercomputer. We argue that using widely distributed supercomputers in order to acquire more compute power is technically feasible and that the largest obstacle is introduced by local scheduling and reservation policies. © 2011 IOP Publishing Ltd.

The computational requirements of simulating a sector of the universe led an international team of researchers to try concurrent processing on two supercomputers half a world apart. Data traveled nearly 27,000 km in 0.277 second, crisscrossing two oceans to go from Amsterdam to Tokyo and back.

In this paper, we describe the implementation and performance of GreeM, a massively parallel TreePM code for large-scale cosmological N-body simulations. GreeM uses a recursive multi-section algorithm for domain decomposition. The size of the domains are adjusted so that the total calculation time of the force becomes the same for all processes. The loss of performance due to non-optimal load balancing is around 4%, even for more than 10(3) CPU cores. GreeM runs efficiently on PC clusters and massively-parallel computers, such as a Cray XT4. The measured calculation speed on Cray XT4 is 5 x 10(4) particles per second per CPU core, for the case of an opening angle of theta = 0.5, if the number of particles per CPU core is larger than 10(6).

We analyzed the statistics of subhalo abundance of galaxy-sized and giant-galaxy-sized halos formed in a high-resolution cosmological simulation of a 46.5 Mpc cube with the uniform mass resolution of 10(6) M(circle dot). We analyzed all halos with mass more than 1.5 x 10(12) M(circle dot) formed in this simulation box. The total number of halos was 125. We found that the subhalo abundance, measured by the number of subhalos with maximum rotation velocity larger than 10% of that of the parent halo, shows large halo-to-halo variations. The results of recent ultra-high-resolution runs fall within the variation of our samples. We found that the concentration parameter and the radius at the moment of the maximum expansion show fairly tight correlation with the subhalo abundance. This correlation suggests that the variation of the subhalo abundance is at least partly due to the difference in the formation history. Halos formed earlier have a smaller number of subhalos at present.

Recent high-resolution simulations of the formation of dark-matter halos have shown that the distribution of subhalos is scale-free, in the sense that if scaled by the velocity dispersion of the parent halo, the subhalo velocity distribution functions of galaxy-sized and cluster-sized halos are identical. For cluster-sized halos, the simulation results agreed well with the observations. The simulations, however, predicted far too many subhalos for galaxy-sized halos. Our galaxy has several tens of known dwarf galaxies. On the other hand, simulated dark-matter halos contain thousands of subhalos. We performed a simulation of a single large volume, and measured the abundance of subhalos in all massive halos. We found that the variation of the subhalo abundance is very large, and those with the largest number of subhalos correspond to the simulated halos in previous studies. The subhalo abundance depends strongly on the local density of the background. Halos in high-density regions contain a large number of subhalos. Our Galaxy is in the low-density region. For our simulated halos in low-density regions, the number of subhalos is within a factor of four to that of our Galaxy. We argue that the "missing dwarf problem" is not a real problem, but is caused by biased selections of the initial conditions in previous studies, which were not appropriate for field galaxies.