About PARAM Brahma
The state-of-the-art supercomputing system is a supercomputing facility is being established at IISER, Pune under build approach of National Supercomputing Mission with a peak computing power of 1.75 PetaFlops. PARAM Brahma is designed and is being installed by C-DAC to cater to the computational needs of IISER, Pune and nearby Research, Engineering and scientific institutes. The system is built with the latest cutting-edge hardware and software technologies. The uniqueness of this system lies in its highly efficient cooling technology, which is based on direct contact liquid cooling.
PARAM Brahma Details
| System Specifications | |
|---|---|
| Theoretical Peak Floating-point Performance Total (Rpeak) | 1.75 PFLOPS |
| Base Specifications (Compute Nodes) | 2 X Intel Xeon Cascadelake 8268, 24 Cores, 2.9 GHz, Processors per node, 192 GB Memory, 480 GB SSD |
| Master/Service/Login Nodes | 10 nos. |
| CPU only Compute Nodes (Memory) | 108 nos. (192 GB) |
| GPU Nodes (Memory) | 32 nos. (192 GB) |
| High Memory Compute Nodes | 39 nos. (768 GB) |
| Total Memory | 56.8 TB |
| Interconnect | Primary: 100Gbps Mellanox Infiniband Interconnect Network 100% non blocking, fat tree topology Secondary: 10G/1G Ethernet Network Management Network: 1G Ethernet |
| Primary Storage | Lustre based Primary storage 750 TiB usable with 25 GB/Sec write throughput |
| Archival Storage | Archival Storage 250 TiB usable capacity based on DDN Gridscaler (GPFS) with 1GB/sec write throughput |
| CPU Only Compute Nodes | |
|---|---|
| Nodes | 108 |
| Cores | 5184 |
| Compute Power of Rpeak | 481.03 TFLOPS |
| Each Node with | 2 X Intel Xeon Cascadelake 8268,
24 cores, 2.9 GHz, processors 192 GB memory 480 GB SSD |
| GPU Only Compute Nodes | |
|---|---|
| Nodes | 32 |
| CPU Cores | 1280 |
| CUDA Cores | 655360 |
| Rpeak | CPU 102.4 TFLOPS + GPU 998.4 TF |
| Each Node with | 2 X Intel Xeon Skylake 6248, 20 cores,
2.5 GHz, processors 192 GB Memory 4 x NVIDIA V100 SXM2 GPU Cards (32 Nodes 480 GB SSD |
| High Memory Compute Nodes | |
|---|---|
| Nodes | 39 |
| CPU Cores | 1872 |
| Compute Power of Rpeak | 173.7 TFLOPS |
| Each Node with | 2 X Intel Xeon Cascadelake 8268,
24 cores, 2.9 GHz, processors 768 GB Memory 480 GB SSD |
Architecture Diagram:
Software Stack:
Installed Applications/Libraries
HPC Applications
- Bio-informatics: MUMmer, HMMER, MEME, PHYLIP, mpiBLAST, ClustalW
- Molecular Dynamics: NAMD (for CPU and GPU), LAMMPS, GROMACS
- CFD: OpenFOAM, SU2
- Material Modeling, Quantum Chemistry: Quantum-Espresso, Abinit, CP2K, NWChem
- Weather, Ocean, Climate: WRF-ARW, WPS (WRF), ARWPost (WRF), RegCM, MOM, ROMS
In-house Developed, Open-source Materials and Computational Chemistry (MCC) HPC Applications/Softwares
- AMDKIIT : Linear scaling hybrid-DFT code for ab initio molecular dynamics
- ANN-CI : Computational chemistry, code augmented by machine learning for studying complex biological systems
- LITESOPH: Layer Integrated Toolkit and Engine for Simulations of Photo-induced phenomena is a toolkit for simulations of photo-induced phenomena
- Mµ2Mech : It is a multiscale modeling approach combining atomistic and phase-field simulations for microstructure modeling during solid-state phase transformations
- MTA* : Quantum chemistry code based on the fragmentation-based molecular tailoring approach ( * MTA will be facilitated based on the Gaussian Institutional license. Kindly contact mscc-support@cdac.in for more details.)
Deep Learning Libraries
- cuDNN, TensorFlow, Theano
Dependency Libraries
- NetCDF, PNETCDF, Jasper, HDF5, Tcl, Boost, FFTW
Support
For any support, contact: brahmasupport@iiserpune.ac.in
PARAM Brahma Usage Report
Publication
There are a total of 107 publications that have been published using PARAM Brahma till 2023.