About PARAM Ganga
IIT Roorkee has established PARAM Ganga, a High-Performance Computing (HPC) system under the National Supercomputing Mission (NSM). This supercomputer uses a heterogeneous and hybrid configuration of Intel Xeon Cascade Lake processors and NVIDIA Tesla V100 GPU, with a total peak computing capacity of 1.66 (CPU+GPU+HM) PFLOPS performance. The system comprises 312 (CPU+GPU+HM) nodes that are connected using the Mellanox (HDR) InfiniBand interconnect network, with the Lustre parallel file system and CentOS 7.x operating system.
PARAM Ganga Details
| System Specifications | |
|---|---|
| Theoretical Peak Floating-point Performance Total (Rpeak) | 1.67 PFLOPS |
| Base Specifications (Compute Nodes) | 2 X Intel Xeon Cascadelake 8268, 24 Cores, 2.9 GHz, Processors per node, 192 GB Memory, 480 GB SSD |
| Master/Service/Login Nodes | 20 nos. |
| CPU only Compute Nodes (Memory) | 214 nos. (192GB) |
| GPU Nodes (Memory) | 20 nos. (192GB) |
| High Memory Compute Nodes | 78 nos. (768GB) |
| Total Memory | 104.832 TB |
| Interconnect | Primary: 100Gbps Mellanox Infiniband Interconnect network 100% non-blocking, fat tree topology Secondary: 10G/1G Ethernet Network Management network: 1G Ethernet |
| CPU Only Compute Nodes | |
|---|---|
| Nodes | 214 |
| Cores | 10272 |
| Compute Power of Rpeak | 953.2 TFLOPS |
| Each Node with | 2 X Intel Xeon Cascadelake 8268, 24 cores, 2.9 GHz, processors 192 GB memory 480 GB SSD |
| GPU Only Compute Nodes | |
|---|---|
| Nodes | 20 |
| CPU Cores | 800 |
| CUDA Cores | 204800 |
| Rpeak | CPU 64 TFLOPS + GPU 312 TF |
| Each Node with | 2 X Intel Xeon Skylake 6248, 20 cores, 2.5 GHz, processors 192 GB Memory 2 x NVIDIA V100 SXM2 GPU Cards 480 GB SSD |
| High Memory Compute Nodes | |
|---|---|
| Nodes | 78 |
| CPU Cores | 3744 |
| Compute Power of Rpeak | 347.33 TFLOPS |
| Each Node with | 2 X Intel Xeon Cascadelake 8268, 24 cores, 2.9 GHz, processors 768 GB Memory 480 GB SSD |
PARAM Ganga Architecture Diagram:
Software Stack:
Installed Applications/Libraries
HPC Applications
- Bio-informatics: MUMmer, HMMER, MEME, PHYLIP, mpiBLAST, ClustalW
- Molecular Dynamics: NAMD (for CPU and GPU), LAMMPS, GROMACS
- CFD: OpenFOAM, SU2
- Material Modeling, Quantum Chemistry: Quantum-Espresso, Abinit, CP2K, NWChem
- Weather, Ocean, Climate: WRF-ARW, WPS (WRF), ARWPost (WRF), RegCM, MOM, ROMS
In-house Developed, Open-source Materials and Computational Chemistry (MCC) HPC Applications/Softwares
- AMDKIIT : Linear scaling hybrid-DFT code for ab initio molecular dynamics
- ANN-CI : Computational chemistry, code augmented by machine learning for studying complex biological systems
- LITESOPH: Layer Integrated Toolkit and Engine for Simulations of Photo-induced phenomena is a toolkit for simulations of photo-induced phenomena
- Mµ2Mech : It is a multiscale modeling approach combining atomistic and phase-field simulations for microstructure modeling during solid-state phase transformations
- MTA* : Quantum chemistry code based on the fragmentation-based molecular tailoring approach ( * MTA will be facilitated based on the Gaussian Institutional license. Kindly contact mscc-support@cdac.in for more details.)
Deep Learning Libraries
- cuDNN, TensorFlow, Theano
Dependency Libraries
- NetCDF, PNETCDF, Jasper, HDF5, Tcl, Boost, FFTW
Support
For any support, contact: gangasupport@iitr.ac.in
PARAM Ganga Usage Report
Link To Be Added
*Note: The above data is coming from C-Chakshu (Multi Cluster Monitoring Platform)
Publication
There are a total of 61 publications that have been published using PARAM Ganga till 2023.