Laboratory of Thermal and Environmental Engineering
Combustion Research Center
Large Eddy Simulations of
Ethanol Spray Combustion
Dr.-Ing. Flavio Cesar Cunha Galeazzo, flavio.galeazzo@usp.br
Prof. Dr.-Ing. Guenther Carlos Krieger Filho
3. Workshop de High Perfomance Computing 17. April 2015
Laboratory of Thermal and Environmental Engineering
Combustion Research Center
Experimental setup
 PhD of Newton K. Fukumasu (2010 - 2014)
 Open burner derived from an automotive fuel injector
and a swirler
 Laser diagnostics (PDI, LIF e PIV)
3WHPC-LCCA Flavio Galeazzo 2
Laboratory of Thermal and Environmental Engineering
Combustion Research Center
100 Hz 250 Hz 400 Hz
Case Injection
frequency
Duty
cycle
Combustion pattern
ESF100 100 Hz 29% Long anchored flame
ESF250 250 Hz 43% Short anchored flame
ESF400 400 Hz 59% Lifted flame
Boundary conditions
Air flow 20 m3 / h
Ethanol flow 2 L / h
3WHPC-LCCA Flavio Galeazzo 3
Numerical setup
 OpenFOAM 2.1.1
 LES (Large Eddy Simulation)
 Smagorinsky subgrid scale model (SSG)
 Spray
 Lagrangian particle tracking
 Combustion model
 Simple “mixed is burned” approach
Laboratory of Thermal and Environmental Engineering
Combustion Research Center
3WHPC-LCCA Flavio Galeazzo 4
Laboratory of Thermal and Environmental Engineering
Combustion Research Center
Grid 1 Grid 2
Number of elements 600 000 1.6 million
Grid sensitivity
 Hexahedral elements
 Two grids with different
refinement
 Cold flow
 Better agreement with
refined grid
 Grid 2 was used for the
LES simulations
3WHPC-LCCA Flavio Galeazzo 5
• Cold flow
• Mean axial velocity UX at the
centerline
• Experiments
• RANS simulation using
Reynolds-stress turbulence
model
• LES using Smagorinky SSG
• The negative velocities indicate
the recirculation zone created by
the swirl flow
• Much better agreement with LES
Laboratory of Thermal and Environmental Engineering
Combustion Research Center
3WHPC-LCCA Flavio Galeazzo 6
• Combustion
• Axial velocity(m/s)
• Temperature (K)
Laboratory of Thermal and Environmental Engineering
Combustion Research Center
7
100 Hz 250 Hz 400 Hz
• Experiment
chemiluminescence
• Mean temperature (K)
Laboratory of Thermal and Environmental Engineering
Combustion Research Center
8
Laboratory of Thermal and Environmental Engineering
Combustion Research Center
Computational resources
 BlueGene/P – USP/Rice
 42 racks, 128 CPUs, 512
computing cores each
 4 GB per CPU
 Proprietary interconnect
 SGI Altix cluster – LETE
 12 nodes, dual CPU, 8
computing cores each
 24 GB per node / 12 GB per
CPU
 Infiniband interconnect
Problems using BG/P
 Compilation of OpenFOAM in
the BG/P
 Need GCC compiler – can’t use
native IBM XL compiler
 Need shared libraries – complicated
to setup in the BG/P
 Run the OpenFOAM
simulation in the BG/P
 Need aprox. 2.5 GB per grid partition
 Can’t use all computing cores of
each CPU
3WHPC-LCCA Flavio Galeazzo 9
Laboratory of Thermal and Environmental Engineering
Combustion Research Center
Performance comparison
 BlueGene/P – USP/Rice
 42 racks, 128 CPUs, 512
computing cores each
 4 GB per CPU
 Proprietary interconnect
 SGI Altix cluster – LETE
 12 nodes, dual CPU, 8
computing cores each
 24 GB per node / 12 GB per
CPU
 Infiniband interconnect
SwirlFlameLETE/1/H1/les_onlyFlow – cold flow
Grid = 1.6 million elements
Mode
Number of
partitions
Simulation
time (s)
Computing
time (s)
Relative
time
Blue
Gene/P
SMP 128 0,002 517 1
LETE openmpi 24 0,002 450 0,87
SwirlFlameLETE/1/H1/EPS250/les_fireFoam - combustion
Grid = 1.6 million elements
Mode
Number of
partitions
Simulation
time (s)
Computing
time (s)
Relative
time
Blue
Gene/P
SMP 128 0,002 8423 1
Blue
Gene/P
SMP 256 0,002 6077 0,72
Blue
Gene/P
SMP 384 0,002 5892 0,70
3WHPC-LCCA Flavio Galeazzo 10
Laboratory of Thermal and Environmental Engineering
Combustion Research Center
Conclusions
 Better agreement of LES results with the experimental data
 More refined computational grids and longer simulation time are
required for LES
 For more refined grids there is a need of more computational power
 OpenFOAM 2.1 was successfully compiled in the BlueGene/P
 The performance is not ideal
 Old GCC compiler (version 4.3.2, from 2008)
 Memory usage of OpenFOAM do not permit the use of all computing cores
 Use of OpenMP is an alternative
 Current development, up to now no speedup using Paralution OpenMP solver
11
Laboratory of Thermal and Environmental Engineering
Combustion Research Center
Thank you
Dr.-Ing. Flavio Cesar Cunha Galeazzo, flavio.galeazzo@usp.br
Prof. Dr.-Ing. Guenther Carlos Krieger Filho
12

“Large Eddy Simulations of Ethanol Spray Combustion”. Flavio Galeazzo – EP/USP.

  • 1.
    Laboratory of Thermaland Environmental Engineering Combustion Research Center Large Eddy Simulations of Ethanol Spray Combustion Dr.-Ing. Flavio Cesar Cunha Galeazzo, [email protected] Prof. Dr.-Ing. Guenther Carlos Krieger Filho 3. Workshop de High Perfomance Computing 17. April 2015
  • 2.
    Laboratory of Thermaland Environmental Engineering Combustion Research Center Experimental setup  PhD of Newton K. Fukumasu (2010 - 2014)  Open burner derived from an automotive fuel injector and a swirler  Laser diagnostics (PDI, LIF e PIV) 3WHPC-LCCA Flavio Galeazzo 2
  • 3.
    Laboratory of Thermaland Environmental Engineering Combustion Research Center 100 Hz 250 Hz 400 Hz Case Injection frequency Duty cycle Combustion pattern ESF100 100 Hz 29% Long anchored flame ESF250 250 Hz 43% Short anchored flame ESF400 400 Hz 59% Lifted flame Boundary conditions Air flow 20 m3 / h Ethanol flow 2 L / h 3WHPC-LCCA Flavio Galeazzo 3
  • 4.
    Numerical setup  OpenFOAM2.1.1  LES (Large Eddy Simulation)  Smagorinsky subgrid scale model (SSG)  Spray  Lagrangian particle tracking  Combustion model  Simple “mixed is burned” approach Laboratory of Thermal and Environmental Engineering Combustion Research Center 3WHPC-LCCA Flavio Galeazzo 4
  • 5.
    Laboratory of Thermaland Environmental Engineering Combustion Research Center Grid 1 Grid 2 Number of elements 600 000 1.6 million Grid sensitivity  Hexahedral elements  Two grids with different refinement  Cold flow  Better agreement with refined grid  Grid 2 was used for the LES simulations 3WHPC-LCCA Flavio Galeazzo 5
  • 6.
    • Cold flow •Mean axial velocity UX at the centerline • Experiments • RANS simulation using Reynolds-stress turbulence model • LES using Smagorinky SSG • The negative velocities indicate the recirculation zone created by the swirl flow • Much better agreement with LES Laboratory of Thermal and Environmental Engineering Combustion Research Center 3WHPC-LCCA Flavio Galeazzo 6
  • 7.
    • Combustion • Axialvelocity(m/s) • Temperature (K) Laboratory of Thermal and Environmental Engineering Combustion Research Center 7
  • 8.
    100 Hz 250Hz 400 Hz • Experiment chemiluminescence • Mean temperature (K) Laboratory of Thermal and Environmental Engineering Combustion Research Center 8
  • 9.
    Laboratory of Thermaland Environmental Engineering Combustion Research Center Computational resources  BlueGene/P – USP/Rice  42 racks, 128 CPUs, 512 computing cores each  4 GB per CPU  Proprietary interconnect  SGI Altix cluster – LETE  12 nodes, dual CPU, 8 computing cores each  24 GB per node / 12 GB per CPU  Infiniband interconnect Problems using BG/P  Compilation of OpenFOAM in the BG/P  Need GCC compiler – can’t use native IBM XL compiler  Need shared libraries – complicated to setup in the BG/P  Run the OpenFOAM simulation in the BG/P  Need aprox. 2.5 GB per grid partition  Can’t use all computing cores of each CPU 3WHPC-LCCA Flavio Galeazzo 9
  • 10.
    Laboratory of Thermaland Environmental Engineering Combustion Research Center Performance comparison  BlueGene/P – USP/Rice  42 racks, 128 CPUs, 512 computing cores each  4 GB per CPU  Proprietary interconnect  SGI Altix cluster – LETE  12 nodes, dual CPU, 8 computing cores each  24 GB per node / 12 GB per CPU  Infiniband interconnect SwirlFlameLETE/1/H1/les_onlyFlow – cold flow Grid = 1.6 million elements Mode Number of partitions Simulation time (s) Computing time (s) Relative time Blue Gene/P SMP 128 0,002 517 1 LETE openmpi 24 0,002 450 0,87 SwirlFlameLETE/1/H1/EPS250/les_fireFoam - combustion Grid = 1.6 million elements Mode Number of partitions Simulation time (s) Computing time (s) Relative time Blue Gene/P SMP 128 0,002 8423 1 Blue Gene/P SMP 256 0,002 6077 0,72 Blue Gene/P SMP 384 0,002 5892 0,70 3WHPC-LCCA Flavio Galeazzo 10
  • 11.
    Laboratory of Thermaland Environmental Engineering Combustion Research Center Conclusions  Better agreement of LES results with the experimental data  More refined computational grids and longer simulation time are required for LES  For more refined grids there is a need of more computational power  OpenFOAM 2.1 was successfully compiled in the BlueGene/P  The performance is not ideal  Old GCC compiler (version 4.3.2, from 2008)  Memory usage of OpenFOAM do not permit the use of all computing cores  Use of OpenMP is an alternative  Current development, up to now no speedup using Paralution OpenMP solver 11
  • 12.
    Laboratory of Thermaland Environmental Engineering Combustion Research Center Thank you Dr.-Ing. Flavio Cesar Cunha Galeazzo, [email protected] Prof. Dr.-Ing. Guenther Carlos Krieger Filho 12