Category: infrastructure

PARAM Shivay

About PARAM Shivay

The supercomputer of 838 TFLOPS capacity, built at the cost of Rs 32.5 crore under the National Super Computing Mission at Indian Institute of Technology (IIT), Banaras Hindu University (BHU).

PARAM Shivay Details

System Specifications
Theoretical Peak Floating-point Performance Total (Rpeak) 837 TFLOPS
Base Specifications (Compute Nodes) 2 X Intel Xeon Skylake 6148, 20 Cores, 2.4 GHz, Processors per node, 192 GB Memory, 480 GB SSD
Master/Service/Login Nodes 10 nos.
CPU only Compute Nodes (Memory) 192 nos. (192GB)
GPU Nodes (Memory) 11 nos. (22 Nvidia V100 PCIe)
High Memory Compute Nodes 20 nos. (768GB)
Total Memory 54.3 TB
Interconnect Primary: 100Gbps Mellanox Infiniband Interconnect network 100% non-blocking, fat tree topology
Secondary: 10G/1G Ethernet Network
Management network: 1G Ethernet
Primary Storage Lustre based Primary storage 750 TiB usable with 25 GB/Sec write throughput
Archival Storage Archival Storage 250 TiB usable capacity based on DDN Gridscaler (GPFS) with 1GB/sec write throughput
CPU Only Compute Nodes
Nodes 192
Cores 7680
Compute Power of Rpeak 589 TFLOPS
Each Node with 2 X Intel Xeon Skylake 6148, 20 cores, 2.4 GHz, processors
192 GB memory
480 GB SSD
GPU Only Compute Nodes
Nodes 11
CPU Cores 440
CUDA Cores 112640
Rpeak Compute Rpeak 187 TF
Each Node with 2 X Intel Xeon Skylake 6148, 20 cores, 2.4 GHz, processors
192 GB Memory
2 x NVIDIA V100 PCIe Accelerator Cards 16 GB HBM2
480 GB SSD
High Memory Compute Nodes
Nodes 20
CPU Cores 800
Compute Power of Rpeak 61 TFLOPS
Each Node with 2 X Intel Xeon Skylake 6148, 20 cores, 2.4 GHz, processors
768 GB Memory
480 GB SSD

Architecture Diagram:
Software Stack:

Installed Applications/Libraries

HPC Applications
  • Bio-informatics: MUMmer, HMMER, MEME, PHYLIP, mpiBLAST, ClustalW
  • Molecular Dynamics: NAMD (for CPU and GPU), LAMMPS, GROMACS
  • CFD: OpenFOAM, SU2
  • Material Modeling, Quantum Chemistry: Quantum-Espresso, Abinit, CP2K, NWChem
  • Weather, Ocean, Climate: WRF-ARW, WPS (WRF), ARWPost (WRF), RegCM, MOM, ROMS
Deep Learning Libraries
  • cuDNN, TensorFlow, Theano
Dependency Libraries
  • NetCDF, PNETCDF, Jasper, HDF5, Tcl, Boost, FFTW

Support

For any support, contact: shivaysupport@iitbhu.ac.in

PARAM Shivay Usage Report

Note: The above data is coming from C-Chakshu (Multi Cluster Monitoring Platform)

Publication

There are a total of 219 publications that have been published using PARAM Shivay till 2023.

PARAM Seva

About PARAM Seva

The supercomputer PARAM Seva is based on heterogeneous and hybrid configuration of Intel Xeon Cascade Lake processors, and NVIDIA Tesla V100.

The system was designed and implemented by HPC Technologies team, Centre for Development of Advanced Computing (C-DAC). PARAM Seva, a state-of-the-art supercomputing facility established under the build approach of the National Supercomputing Mission with a peak computing power of 838 TF.

PARAM Seva Details

System Specifications:
System Specifications
Theoretical Peak Floating-point Performance Total (Rpeak) 838 TFLOPS
Base Specifications (Compute Nodes) 2 X Intel Xeon Cascadelake 8268, 24 Cores, 2.9 GHz, Processors per node, 192 GB Memory, 480 GB SSD
Master/Service/Login Nodes 10 nos.
CPU only Compute Nodes (Memory) 107 nos. (192GB)
GPU Nodes (Memory) 10 nos. (192GB)
High Memory Compute Nodes 39 nos. (768GB)
Total Memory 52.416 TB
Interconnect Primary: 100Gbps Mellanox Infiniband Interconnect network 100% non-blocking, fat tree topology
Secondary: 10G/1G Ethernet Network
Management network: 1G Ethernet
Storage 1PiB PFS based storage
CPU Only Compute Nodes
Nodes 107
Cores 5136
Compute Power of Rpeak 476.6 TFLOPS
Each Node with 2 X Intel Xeon Cascadelake 8268, 24 cores, 2.9 GHz, processors
192 GB memory
480 GB SSD
GPU Only Compute Nodes
Nodes 10
CPU Cores 400
CUDA Cores 102400
Rpeak CPU 32 TFLOPS + GPU 156 TF
Each Node with 2 X Intel Xeon Skylake 6248, 20 cores, 2.5 GHz, processors
192 GB Memory
2 x NVIDIA V100 SXM2 GPU Cards
480 GB SSD
High Memory Compute Nodes
Nodes 39
CPU Cores 1872
Compute Power of Rpeak 173.7 TFLOPS
Each Node with 2 X Intel Xeon Cascadelake 8268, 24 cores, 2.9 GHz, processors
768 GB Memory
480 GB SSD
PARAM Seva Architecture Diagram:
Software Stack:

Installed Applications/Libraries

HPC Applications
  • Bio-informatics: MUMmer, HMMER, MEME, PHYLIP, mpiBLAST, ClustalW
  • Molecular Dynamics: NAMD (for CPU and GPU), LAMMPS, GROMACS
  • CFD: OpenFOAM, SU2
  • Material Modeling, Quantum Chemistry: Quantum-Espresso, Abinit, CP2K, NWChem
  • Weather, Ocean, Climate: WRF-ARW, WPS (WRF), ARWPost (WRF), RegCM, MOM, ROMS
In-house Developed, Open-source Materials and Computational Chemistry (MCC) HPC Applications/Softwares
  • AMDKIIT : Linear scaling hybrid-DFT code for ab initio molecular dynamics
  • ANN-CI : Computational chemistry, code augmented by machine learning for studying complex biological systems
  • LITESOPH: Layer Integrated Toolkit and Engine for Simulations of Photo-induced phenomena is a toolkit for simulations of photo-induced phenomena
  • Mµ2Mech : It is a multiscale modeling approach combining atomistic and phase-field simulations for microstructure modeling during solid-state phase transformations
  • MTA* : Quantum chemistry code based on the fragmentation-based molecular tailoring approach
    • ( * MTA will be facilitated based on the Gaussian Institutional license. Kindly contact mscc-support@cdac.in for more details.)
Deep Learning Libraries
  • cuDNN, TensorFlow, Theano
Dependency Libraries
  • NetCDF, PNETCDF, Jasper, HDF5, Tcl, Boost, FFTW

Support

For any support, contact: sevasupport@iith.ac.in

PARAM Seva Usage Report

Link To Be Added

For detailed usage reports, please visit the respective sections on our internal portal.

Publication

There are a total of 32 publication that have been published using PARAM Seva till 2023.

PARAM Shavak

PARAM Shavak - Supercomputing Solution in a Box

Param Shavak machine PARAM Shavak - solution, aims to provide computational resource (Capacity building) with advanced technologies to perform high-end computations for scientific, engineering and academic programs to address and catalyze the research using modelling, simulation and data analysis. This initiative is expected to create HPC aware skilled workforce (Capability building) and for promoting research by integrating leading-edge emerging technologies at grass root level.

As the scope and complexity of computational needs continue to increase at colleges and universities, professors and administrators are compelled to seek appropriate and affordable solutions. PARAM Shavak provides the computing power necessary to keep academic institutions on the leading edge in today's competitive market at affordable cost. This system is meant for research organizations and academic institutions that are on the verge of adopting HPC culture in their institutions/organizations.

Besides a handful of value additions from C-DAC, the system comes with most of the features that can be found in a full-blown HPC cluster including job schedulers, compilers, parallel libraries, MPI, resource managers, and some of the commonly used HPC applications in engineering and scientific domains.

HPC Usage Charging Policy

HPC Usage Charging Policy

User Category Academic and R&D Institutions MSME / Startups Commercial Users
Priority
High Normal / Free*
High High
CPUcore/hr
Rs. 0.96 free(limited)
Rs. 0.96 Rs. 1.50
GPUcard/hr
Rs. 42 free(limited)
Rs. 42 Rs. 100
Storage/month
100GB free/account
Rs. 1/GB thereafter 100GB free/account
Rs. 1/GB thereafter
100GB free/account
Rs. 1/GB thereafter		 100GB free/account
Rs. 1.50/GB thereafter
Billing Period
Monthly/Quarterly/Annually Monthly/Quarterly/Annually
Monthly/Quarterly/Annually Monthly/Quarterly/Annually

Upcoming NSM Systems

Upcoming NSM Systems


Theoretical Performance Location
20 PF C-DAC, Bangalore
838 TF IIT, Jammu

NSM Systems

15

PARAM Shivay

[838 TF]

IIT BHU Varanasi
13

PARAM Shakti

[1.66 PF]

IIT Kharagpur
Nsm 3

PARAM Brahma

[1.75 PF]

IISER Pune
14

PARAM Yukti

[1.8 PF]

JNCASR Bangalore
16

PARAM Sanganak

[1.67 PF]

IIT Kanpur
NSM 2

PARAM Pravega

[3.3 PF]

IISc Bangalore
5

PARAM Seva

[838 TF]

IIT Hyderabad
4

PARAM Smriti

[838 TF]

NABI Mohali
NSM 1

PARAM Utkarsh

[838 TF]

C-DAC Bangalore
6

PARAM Ganga

[1.67 PF]

IIT Roorkee
8

PARAM Ananta

[838 TF]

IIT Gandhinagar
9

PARAM Porul

[838 TF]

NIT Trichy
10

PARAM Himalaya

[838 TF]

IIT Mandi
12

PARAM Kamrupa

[838 TF]

IIT Guwahati
11

PARAM Siddhi-AI

[5.27 PF]

C-DAC Pune
7

PARAM Rudra

[3PF]

IUAC Delhi
7

PARAM Rudra

[838 TF]

SN Bose Kolkata
7

PARAM Rudra

[1.3 PF]

GMRT Narayangaon
7

PARAM Rudra

[200TF]

C-DAC Delhi
7

PARAM Rudra

[3PF]

IIT Bombay
7

PARAM Rudra

[3 PF]

IIT Madras
7

PARAM Rudra

[838 TF]

IIT Patna

Infrastructure

NSM Infrastructure

The National Supercomputing Mission aims at achieving the goals of attaining self-reliance in supercomputing, building the culture of using supercomputing for carrying out R&D and problem-solving in various domains of scientific and technological endeavours, and designing solutions for various societal applications, and positioning the supercomputing ecosystem in the country at a globally competitive level.

The mission envisions the creation of a national infrastructure of supercomputing systems and facilities of various sizes and scales, distributed across the country and seamlessly integrated through the National Knowledge Network.

The systems and facilities created as part of the infrastructure under this mission are divided into three phases of the National Supercomputing Mission: Phase I, Phase II, and Phase III.

Collective Last Week Utilization (Avg.)

81.15%

CPU

68.32%

GPU