SESSION 3

NUMERICAL/COMPUTATIONAL ASPECTS

Chairman: T.-H. Song

APPLICATION OF SPATIAL AND ANGULAR DOMAIN BASED PARALLELISM TO A DISCRETE ORDINATES FORMULATION WITH UNSTRUCTURED SPATIAL DISCRETIZATION

Shawn P Burns
Engineering Sciences Center
Sandia National Laboratories, Albuquerque, New Mexico, USA

A parallel discrete ordinate formulation employing a general, unstructured finite element spatial discretization is presented for steady, gray, nonscattering radiative heat transport within a participating medium. Specifically, the application of interest involves numerical modeling of the heat and mass transport within heavily sooting hydrocarbon pool fires in complex three dimensional geometries. It is estimated that high resolution spatial grids with O(10⁶) to O(10⁸) elements will be required to resolve the important length and time scales in this coupled, multiphysics application. The size of the unstructured spatial grid as well as the overall expense of the radiative transport calculation requires that an efficient parallel radiative heat transport algorithm be developed.

The discrete ordinate formulation employed is based on the first order form of the Boltzmann transport equation assuming gray, nonscattering material properties. This is a reasonable first approximation for heavily sooting pool fires in which the material properties are dominated by the soot phase. Boundary conditions assume gray, diffuse boundaries. The radiative heat flux and flux divergence are obtained using a level weighted, even moment, symmetric angular quadrature set. Several sparse, iterative solvers are employed to evaluate the discrete form of the Boltzmann transport equation along each ordinate direction.

The parallel implementation of the discrete ordinate algorithm allows for any combination of angular and spatial domain based parallelism. Interprocessor communication is required to complete the matrix-vector products and vector-vector inner products at each iteration of the sparse matrix solver for spatial domain decomposition. Angular domain decomposition also requires interprocessor communication to complete the angular integration needed to evaluate the radiative heat flux and flux divergence at each node in the spatial grid.

The formulation is tested on the ASCI RED massively parallel multiple instruction, multiple data supercomputer using a standard three-dimensional benchmark calculation. The ASCI RED machine consists of 4,536 compute nodes in a distributed memory configuration. Each compute node consists of two 200 Mhz processors with a two dimensional split mesh interprocessor communication topology with an 800 Mbyte/sec bidirectional bandwidth. The rated performance of the ASCI RED machine is 1.8 trillion operations per second.

Timing results were obtained using up to 343 processors and 2.7x 106 elements with S₁₀ quadrature (120 ordinates). These results showed that angular domain based parallelism is a very natural decomposition paradigm capable of achieving scaled problem size parallel efficiencies in excess of 90%. However, angular decomposition provides limited parallelism for massively parallel architectures since the number of ordinates is typically much less than the number of available processors.

The timing results also showed that the scaled problem size parallel efficiency for spatial domain based parallelism is influenced both by interprocessor communication as well as the number of sparse matrix solver iterations required for convergence on each ordinate direction.

It was found that the number of iterations increased as approximately the number of nodes in the global mesh to the 1/4 to 1/3 power depending on the iterative solver being used. Expressing the parallel efficiency on a per iteration basis showed that the communication overhead is small for spatial decomposition provided that the surface area to volume ratio of the individual spatial subdomains is reasonable.

Ultimately, the total compute time required for a given simulation is the most important parallel performance metric. For the test problem considered in this work, overall cycle times of several micro seconds per element per ordinate direction were easily achieved. Furthermore, by combining spatial and angular decomposition, the algorithm described may exploit the high parallel efficiency of angular decomposition as well as the unlimited parallelism available with spatial decomposition.

Future work is planned to incorporate the algorithm described in this work into a larger turbulent, high temperature, reacting flow algorithm for full scale fire simulation testing. Continued development is also planned on the transport algorithm to incorporate scattering and/or nongray radiative properties. The effect of sophisticated preconditioning on the parallel performance is also planned.

PARALLELIZATION OF THE FINITE VOLUME METHOD

João GONÇALVES, Pedro COELHO
Instituto Superior Técnico, Technical University of Lisbon
Mechanical Engineering Department
Av. Rovisco Pais,1096 Lisboa Codex
Portugal

ABSTRACT. Two different parallelization strategies of the finite volume method (FVM) are described and applied to two test problems. One of the methods is based on angular decomposition and the other one is based on spatial decomposition. In the first case each processor performs the calculations for the whole domain but only deals with a few directions, while in the second case each processor treats all the directions but only for a subdomain. It is shown that the number of iterations is independent of the number of processors in the first parallelization strategy, but increases with the number of processors in the second case. Consequently, higher efficiencies are achievable using the angular decomposition approach. The influence of the angular discretization, grid size and optical thickness of the medium is also investigated.

BENCHMARK COMPARISONS WITH DISCRETE TRANSFER METHOD SOLUTIONS FOR RADIATIVE HEAT TRANSFER IN THREE-DIMENSIONAL, NONGRAY, SCATTERING MEDIA

J.C. HENSON, W.M.G. MALALASEKERA
Department of Mechanical Engineering,
Loughborough University,
Loughborough, LEl13TU, United Kingdom.

ABSTRACT. A bandwise formulation of the discrete transfer method is presented for the three- dimensional radiative transport in nongray, nonhomogeneous, absorbing, emitting and isotropically scattering media. Numerical solutions with this formulation are in good agreement with YIX, finite element and Monte Carlo solutions for several benchmark problems. These involve a homogeneous mixture of suspended carbon particles in gaseous CO₂ and N₂ in an L-shaped enclosure. Variations in the particle concentration and problem geometry are considered. The average deviation of the discrete transfer results with a second set of solutions, calculated from a hybrid Monte Carlo/diffusion algorithm, is less than 5% for both the boundary surface flux and the divergence of radiative flux within the gas/carbon particle mixtures.

DIRECT EXCHANGE AREAS FOR A RECTANGULAR BOX BY THE DIRECT DISCRETE-ORDINATES METHOD

K.H. Byun*, Theodore F. Smith**
*Department of Mechanical Engineering, Dongguk University, Seoul 100-715, Republic of Korea
**Department of Mechanical Engineering, The University of Iowa, Iowa 52242-1527, USA

ABSTRACT. The direct discrete-ordinates method (DSN) is applied to compute direct exchange areas (DEA) for rectangular box type enclosures. The discrete-ordinates weights and directions are derived using spherical coordinates. The segments are generated on the surface of unit sphere by uniform angular divisions. The discrete-ordinates directions are defined at the centers of each segments employing the circumferential angle in the base plane and the polar angle measured from the base plane. The discrete-ordinates weights are set equal to the solid angles of each segments.

A gray absorbing and emitting medium is enclosed by black walled rectangular box type enclosures. To show that the method is applicable to general rectangular box type geometry, the length, width, and height of the system are varied. A protrusion may exist in the system that alters the radiant exchange between some of the surfaces due to shading.

The effects of optical thickness as well as the number of spatial and angular divisions on the accuracy of DEA results are studied. For given spatial and angular divisions, there is an upper limit of optical thickness that ensures physically realistic results. In terms of the cell optical thickness, it is around 0.3 as suggested by the results of this study. The cell optical thickness is the product of the control volume height and absorption coefficient. The results are presented up to this limit. For a given optical thickness, the errors are reduced as the number of spatial and angular divisions increase. If there is a protrusion in the system, additional spatial and angular divisions are needed than used for the no-protrusion case at the same accuracy.

The results are compared with the DEA predictions by the S-_N discrete-ordinates methods of order up to S-₁₀. The results indicate that the higher order S-_N method than S-₁₀ is necessary to improve the accuracy of results. The exact values are computed numerically integrating the DEAs definition. The results by the DS_N method converge to the exact values as the number of spatial and angular division increase. The results show certain convergence characteristics with the spatial and angular divisions; thus, in some cases, the accuracy can be estimated.

In conclusion, whether there is a protrusion in the system or not, DEAs for rectangular box type enclosures can be accurately obtained by using the DS_N method within the optical thickness limit. The value of the optical thickness limit increases as the number of spatial and angular divisions increase. One of the advantages of the DS_N method is that it is easy to generate and implement the weights and directions. Also, the DEAs of the enclosure can be obtained simultaneously and the results also satisfy the enclosure, symmetry, and the reciprocity relations.