Category: infrastructure

PARAM Brahma

About PARAM Brahma

The state-of-the-art supercomputing system is a supercomputing facility is being established at IISER, Pune under build approach of National Supercomputing Mission with a peak computing power of 1.75 PetaFlops. PARAM Brahma is designed and is being installed by C-DAC to cater to the computational needs of IISER, Pune and nearby Research, Engineering and scientific institutes. The system is built with the latest cutting-edge hardware and software technologies. The uniqueness of this system lies in its highly efficient cooling technology, which is based on direct contact liquid cooling.

PARAM Brahma Details

System Specifications
Theoretical Peak Floating-point Performance Total (Rpeak) 1.75 PFLOPS
Base Specifications (Compute Nodes) 2 X Intel Xeon Cascadelake 8268, 24 Cores, 2.9 GHz, Processors per node, 192 GB Memory, 480 GB SSD
Master/Service/Login Nodes 10 nos.
CPU only Compute Nodes (Memory) 108 nos. (192 GB)
GPU Nodes (Memory) 32 nos. (192 GB)
High Memory Compute Nodes 39 nos. (768 GB)
Total Memory 56.8 TB
Interconnect Primary: 100Gbps Mellanox Infiniband Interconnect
Network 100% non blocking, fat tree topology
Secondary: 10G/1G Ethernet Network Management
Network: 1G Ethernet
Primary Storage Lustre based Primary storage 750 TiB usable with 25 GB/Sec write throughput
Archival Storage Archival Storage 250 TiB usable capacity based on DDN Gridscaler (GPFS) with 1GB/sec write throughput
CPU Only Compute Nodes
Nodes 108
Cores 5184
Compute Power of Rpeak 481.03 TFLOPS
Each Node with 2 X Intel Xeon Cascadelake 8268, 24 cores, 2.9 GHz, processors
192 GB memory
480 GB SSD
GPU Only Compute Nodes
Nodes 32
CPU Cores 1280
CUDA Cores 655360
Rpeak CPU 102.4 TFLOPS + GPU 998.4 TF
Each Node with 2 X Intel Xeon Skylake 6248, 20 cores, 2.5 GHz, processors
192 GB Memory
4 x NVIDIA V100 SXM2 GPU Cards (32 Nodes
480 GB SSD
High Memory Compute Nodes
Nodes 39
CPU Cores 1872
Compute Power of Rpeak 173.7 TFLOPS
Each Node with 2 X Intel Xeon Cascadelake 8268, 24 cores, 2.9 GHz, processors
768 GB Memory
480 GB SSD

Architecture Diagram:

Software Stack:

Installed Applications/Libraries

HPC Applications
  • Bio-informatics: MUMmer, HMMER, MEME, PHYLIP, mpiBLAST, ClustalW
  • Molecular Dynamics: NAMD (for CPU and GPU), LAMMPS, GROMACS
  • CFD: OpenFOAM, SU2
  • Material Modeling, Quantum Chemistry: Quantum-Espresso, Abinit, CP2K, NWChem
  • Weather, Ocean, Climate: WRF-ARW, WPS (WRF), ARWPost (WRF), RegCM, MOM, ROMS
In-house Developed, Open-source Materials and Computational Chemistry (MCC) HPC Applications/Softwares
  • AMDKIIT : Linear scaling hybrid-DFT code for ab initio molecular dynamics
  • ANN-CI : Computational chemistry, code augmented by machine learning for studying complex biological systems
  • LITESOPH: Layer Integrated Toolkit and Engine for Simulations of Photo-induced phenomena is a toolkit for simulations of photo-induced phenomena
  • Mµ2Mech : It is a multiscale modeling approach combining atomistic and phase-field simulations for microstructure modeling during solid-state phase transformations
  • MTA* : Quantum chemistry code based on the fragmentation-based molecular tailoring approach
    • ( * MTA will be facilitated based on the Gaussian Institutional license. Kindly contact mscc-support@cdac.in for more details.)
Deep Learning Libraries
  • cuDNN, TensorFlow, Theano
Dependency Libraries
  • NetCDF, PNETCDF, Jasper, HDF5, Tcl, Boost, FFTW

Support

For any support, contact: brahmasupport@iiserpune.ac.in

PARAM Brahma Usage Report

Publication

There are a total of 107 publications that have been published using PARAM Brahma till 2023.

PARAM Sanganak

About PARAM Sanganak

The supercomputing facility established under the build approach of National Supercomputing Mission with a peak computing power of 1.67 PetaFlops. Param Sanganak is designed and commissioned by C-DAC to cater to the computational needs of IIT, Kanpur and various Research and Engineering institutes of the region. It is based on Intel Xeon Cascade lake processors, and NVIDIA Tesla V100 and HDR100. The system is built with the latest cutting edge hardware and software technologies. Substantial components utilized to build this system are manufactured and assembled within India, which is a step towards the Make in India initiative of the Government.

PARAM Sanganak Details

System Specifications:
System Specifications
Theoretical Peak Floating-point Performance Total (Rpeak) 1.67 PFLOPS
Base Specifications (Compute Nodes) 2 X Intel Xeon Cascadelake 8268, 24 Cores, 2.9 GHz, Processors per node, 192 GB Memory, 480 GB SSD
Master/Service/Login Nodes 20 nos.
CPU only Compute Nodes (Memory) 214 nos. (192GB)
GPU Nodes (Memory) 20 nos. (192GB)
High Memory Compute Nodes 78 nos. (768GB)
Total Memory 104.832 TB
Interconnect Primary: 100Gbps Mellanox Infiniband Interconnect network 100% non-blocking, fat tree topology
Secondary: 10G/1G Ethernet Network
Management network: 1G Ethernet
CPU Only Compute Nodes
Nodes 214
Cores 10272
Compute Power of Rpeak 953.2 TFLOPS
Each Node with 2 X Intel Xeon Cascadelake 8268, 24 cores, 2.9 GHz, processors
192 GB memory
480 GB SSD
GPU Compute Nodes
Nodes 20
CPU Cores 800
CUDA Cores 204800
Rpeak CPU 64 TFLOPS + GPU 312 TF
Each Node with 2 X Intel Xeon Cascadelake 6248, 20 cores, 2.5 GHz, processors
192 GB Memory
2 x NVIDIA V100 SXM2 GPU Cards
480 GB SSD
High Memory Compute Nodes
Nodes 78
CPU Cores 3744
Compute Power of Rpeak 347.33 TFLOPS
Each Node with 2 X Intel Xeon Cascadelake 8268, 24 cores, 2.9 GHz, processors
768 GB Memory
480 GB SSD

Architecture Diagram:

Software Stack:

Installed Applications/Libraries

HPC Applications
  • Bio-informatics: MUMmer, HMMER, MEME, PHYLIP, mpiBLAST, ClustalW
  • Molecular Dynamics: NAMD (for CPU and GPU), LAMMPS, GROMACS
  • CFD: OpenFOAM, SU2
  • Material Modeling, Quantum Chemistry: Quantum-Espresso, Abinit, CP2K, NWChem
  • Weather, Ocean, Climate: WRF-ARW, WPS (WRF), ARWPost (WRF), RegCM, MOM, ROMS
In-house Developed, Open-source Materials and Computational Chemistry (MCC) HPC Applications/Softwares
  • AMDKIIT : Linear scaling hybrid-DFT code for ab initio molecular dynamics
  • ANN-CI : Computational chemistry, code augmented by machine learning for studying complex biological systems
  • LITESOPH: Layer Integrated Toolkit and Engine for Simulations of Photo-induced phenomena is a toolkit for simulations of photo-induced phenomena
  • Mµ2Mech : It is a multiscale modeling approach combining atomistic and phase-field simulations for microstructure modeling during solid-state phase transformations
  • MTA* : Quantum chemistry code based on the fragmentation-based molecular tailoring approach
    • ( * MTA will be facilitated based on the Gaussian Institutional license. Kindly contact mscc-support@cdac.in for more details.)
Deep Learning Libraries
  • cuDNN, TensorFlow, Theano
Dependency Libraries
  • NetCDF, PNETCDF, Jasper, HDF5, Tcl, Boost, FFTW

Support

For any support, contact: sanganakalert@iitk.ac.in

PARAM Sanganak Usage Report

Publication

There are total 107 publications have been published by using PARAM Sanganak till 2023.

PARAM Utkarsh

About PARAM Utkarsh

PARAM Utkarsh is a High Performance Computing System setup at C-DAC, Bangalore under the National Supercomputing Mission (NSM), Government of India. PARAM Utkarsh is based on Intel Cascade Lake processor and NVIDIA Tesla V100 GPU with 100Gbps infiniband non-blocking interconnect. Equipped with 50,000+ compute cores (CPU & GPU) and liquid cooling system for efficient PUE, PARAM Utkarsh offers peak computing power of 838 Teraflops.

PARAM Utkarsh HPC system is setup at CTSF under the National Supercomputing Mission (NSM) of Government of India. Current setup supports HPC simulations, Big Data Analytics and Cloud services mainly to meet the requirements of the Micro, Small and Medium Enterprises (MSME) sector.

MSME are the backbone of the Indian economy, and many could use high performance computing (HPC) to enhance their business. However, it is challenging for MSME to adopt HPC as part of their operation. They may have little in-house expertise, limited access to HPC specific hardware. To fill this gap only NSM project comes into to play a vital role by making it easier for MSME sectors mainly to support their ideas, simulations, application implementation on top of HPC PARAM Utkarsh resources of C-DAC. The overall objective of PARAM Utkarsh is to enhance the MSME business model into the next level of digital transformation.

PARAM Utkarsh Details

System Specifications
Theoretical Peak Floating-point Performance Total (Rpeak) 838 TFLOPS
Base Specifications (Compute Nodes) 2 X Intel Xeon Cascadelake 8268, 24 Cores, 2.9 GHz, Processors per node, 192 GB Memory, 480 GB SSD
Master/Service/Login Nodes 10 nos.
CPU only Compute Nodes (Memory) 107 nos. (192GB)
GPU Nodes (Memory) 10 nos. (192GB)
High Memory Compute Nodes 39 nos. (768GB)
Total Memory 52.416 TB
Interconnect Primary: 100Gbps Mellanox Infiniband Interconnect network 100% non-blocking, fat tree topology
Secondary: 10G/1G Ethernet Network
Management network: 1G Ethernet
Storage 1PiB PFS based storage
CPU Only Compute Nodes
Nodes 107
Cores 5136
Compute Power of Rpeak 476.6 TFLOPS
Each Node with 2 X Intel Xeon Cascadelake 8268, 24 cores, 2.9 GHz, processors
192 GB memory
480 GB SSD
GPU Only Compute Nodes
Nodes 10
CPU Cores 400
CUDA Cores 102400
Rpeak CPU 32 TFLOPS + GPU 156 TF
Each Node with 2 X Intel Xeon Skylake 6248, 20 cores, 2.5 GHz, processors
192 GB Memory
2 x NVIDIA V100 SXM2 GPU Cards
480 GB SSD
High Memory Compute Nodes
Nodes 39
CPU Cores 1872
Compute Power of Rpeak 173.7 TFLOPS
Each Node with 2 X Intel Xeon Cascadelake 8268, 24 cores, 2.9 GHz, processors
768 GB Memory
480 GB SSD

PARAM Utkarsh Architecture Diagram:
Software Stack:

Installed Applications/Libraries

HPC Applications
  • Bio-informatics: MUMmer, HMMER, MEME, PHYLIP, mpiBLAST, ClustalW
  • Molecular Dynamics: NAMD (for CPU and GPU), LAMMPS, GROMACS
  • CFD: OpenFOAM, SU2
  • Material Modeling, Quantum Chemistry: Quantum-Espresso, Abinit, CP2K, NWChem
  • Weather, Ocean, Climate: WRF-ARW, WPS (WRF), ARWPost (WRF), RegCM, MOM, ROMS
Deep Learning Libraries
  • cuDNN, TensorFlow, Theano
Dependency Libraries
  • NetCDF, PNETCDF, Jasper, HDF5, Tcl, Boost, FFTW

Support

For any support, contact: utkarsh-support@cdac.in

PARAM Utkarsh Usage Report

Link To Be Added

*Note: The above data is coming from C-Chakshu (Multi Cluster Monitoring Platform)

Publication

There are a total of 39 publications that have been published using PARAM Utkarsh till 2023.