Trinetra
Modern supercomputers are behind the scene powerhouses for scientific and human advancement applications such as weather prediction, Numerical simulations, bio-informatics, and material research. CDAC was entrusted with design and development of indigenous core technologies for supercomputing and has built several generations of PARAM supercomputing systems. One of the core components of a PARAM supercomputer is PARAMNet interconnect network. Under the NSM project, CDAC has developed three generations of latest PARAMNet interconnects called as Trinetra series of networks. Trinetra design and development effort spans multiple domains related with state of art hardware system design. It has multiple hardware and software components, such as NCC (Network Controller Chip: VLSI communication co-processor design), NIC (Network Interface Card), and LWP (Lightweight protocol) software. Together, these components realize a high bandwidth, low latency, scalable network fabric supporting industry standard programming interfaces.
Trinetra development has been done in three phases
- Trinetra-POC: This was proof of concept phase, with focus being getting know-how about switchless networks, 3D Torus topology, and onload/hybrid architecture of Network Controller Chip. The hardware used was based 40Gbps physical link layer for IO and PCIe Gen-3.0 2x interface for host interface. The PoC hardware was used for extensive validation of key architectural concepts and hardware/software co-verification.
- Trinetra-A is based on100Gbps physical link layer, PCIe Gen-3.0 x8 host interface, and uses 3D Torus as network topology. All hardware, software and firmware components of the network have been developed successfully. Trinetra-A is currently deployed in PARAM Rudra 1PF system at CDAC Pune.
- Trinetra-B is the latest hardware developed with performance and topology enhancements, leveraging experience gained during PoC and TrA development phases. TrB is based around PCIe Gen-3.0 x16 host interface, and 200Gbps physical link layer. Ten links @ 200Gbps allow for Dragonfly/ Supercluster topology. TrB planned to be deployed in PARAM 20PF system.
Trinetra development effort is expected to continue under forthcoming NSM2.0 with primary aim of indigenous development of Exascale systems.
- Use Cases
- HPC and Enterprise applications
- Salient Features
- High Performance: High bandwidth and low latency, supported by innovative hardware capabilities and a lightweight software stack.
- Scalability and Power Efficiency: Designed to ensure real performance, scalability, and power efficiency, while facilitating experimentation and application optimization.
- Topology Flexibility: Supports 3D Torus (Trinetra-A) and Dragonfly/Supercluster (Trinetra-B) topologies for large-scale scalability eliminating need for dedicated switches.
- Multi-Processor Support: Compatible with multiple processor and platform architectures.
- Industry Standards Compliance: Supports standard HPC programming interfaces like MPI and legacy socket interfaces.
Technical Specifications
Trinetra-A:
- Six 100Gbps full duplex interfaces
- PCIe GEN-3.0 x8 host interface
- 3D Torus topology
- NCC-I Co-processor
Trinetra-B:
- Ten 200Gbps full duplex interfaces
- PCIe GEN-3.0 x16 host interface
- Dragonfly/ Supercluster topology
- NCC-II Co-processor
System Software
- Includes device drivers and kernel/user components, with OFED support for seamless integration with existing systems.
