SGI 10-Gigabit Ethernet Network Adapter Uživatelský manuál stáhnout pdf (Strana 131)

MPI over IB-WAN: Obsidian Routers

Delay (us) Distance

(km)

10 2

100 20

1000 200

10000 2000

Cluster A

Cluster B

WAN Link

Obsidian WAN Router Obsidian WAN Router

(Variable Delay)

MPI Bidirectional Bandwidth

Impact of Encryption on Message Rate (Delay 0 ms)

Hardware encryption has no impact on performance for less communicating streams

131

CCGrid '11

S. Narravula, H. Subramoni, P. Lai, R. Noronha and D. K. Panda, Performance of HPC Middleware over

InfiniBand WAN, Int'l Conference on Parallel Processing (ICPP '08), September 2008.

1 2 ... 126 127 128 129 130 131 132 133 134 135 136 ... 149 150

A Tutorial at CCGrid ’11 1
Computing Systems 3
Cluster Computing Environment 4
(http://www.top500.org) 5
Grid Computing Environment 6
Compute cluster 8
Cloud Computing Environments 9
Hadoop Architecture 10
Memcached Architecture 11
• Hardware components 13
• Software components 13
• Ex: TCP/IP, UDP/IP 14
– E.g., PCI, PCI-X 15
– One bit per wire 15
– Not scalable: 15
Myrinet (1993 -) 1 Gbit/sec 16
IB Trade Association 19
• Network speed bottlenecks 21
• I/O interface bottlenecks 21
(not shown) 24
• Fast Messages (FM) 27
• Myricom GM 27
IB Hardware Acceleration 28
• Interrupt Coalescing 29
• Jumbo Frames 29
• Hardware Checksum Engines 29
TOE and iWARP Accelerators 30
2006 (v3.0), 2008 (v3.1) 34
PCI-Express (PCIe) 34
2003 (Gen1), 2007 (Gen2) 34
IB, HSE and their Convergence 36
Traditional Ethernet 37
IB Overview 38
Components: Channel Adapters 39
• Switches: intra-subnet 40
– Not directly addressable 41
IB Communication Model 43
Queue Pair Model 44
Memory Registration 45
Memory Protection 46
(Send/Receive Model) 47
InfiniBand Device 48
Hardware ACK 48
Hardware Protocol Offload 51
• Buffering and Flow Control 52
• Switching and Multicast 52
Buffering and Flow Control 53
Virtual Lanes 54
Service Levels and QoS 55
Traffic Segregation Benefits 56
Identifiers) 57
Switch Complex 58
– 3D Torus (Sandia Red Sky) 59
More on Multipathing 60
IB Multicast Example 61
IB Transport Services 63
Reliability 64
Transport Layer Capabilities 65
Data Segmentation 66
Transaction Ordering 67
Message-level Flow-Control 68
Concepts in IB Management 71
Subnet Manager 72
HSE Overview 74
Differences 75
– Multi Stream Semantics 76
Offloaded TCP 83
Myrinet Express (MX) 85
Datagram Bypass Layer (DBL) 86
• Solarflare approach: 87
Ethernet 90
IB Network 90
IB Verbs 90
Application 90
Hardware 90
IB Transport 91
IB iWARP/HSE RoE RoCE 92
IB Hardware Products 94
Tyan Thunder S2935 Board 95
IB Hardware Products (contd.) 96
– Nortel Networks 97
• Support for VPI and RoCE 98
– www.openfabrics.org 99
– OFED 1.6 is underway 99
Verbs Interface 100
(libibverbs) 100
• Within the hardware: 101
OpenFabrics Software Stack 102
InfiniBand in the Top500 103
Cray Interconnect 104
NUMAlink 104
SP Switch 104
CCGrid '11 106
• Other small clusters 107
• Integrated Systems 107
Other HSE Installations 108
Presentation Overview 109
1/10 GigE 110
InfiniBand 110
Case Studies 111
Latency (us) 112
Message Size (bytes) 112
Bandwidth (MBps) 113
MVAPICH/MVAPICH2 Software 115
One-way Latency: MPI over IB 116
Bandwidth: MPI over IB 117
Bandwidth: MPI over iWARP 119
Convergent Technologies: 121
Traditional Model 123
Possible SDP Model 123
InfiniBand CA 123
SDP vs. IPoIB (IB QDR) 124
IB on the WAN 126
Features 127
• Supports SONET OC-192 or 128
“appear” local 128
Sunnyvale 129
700 miles 130
3300 miles 4300 miles 130
Cluster B 131
Communication Options in Grid 132
Interface 133
Modern WAN 133
Performance of Memory Based 134
Data Transfer 134
ADTS-64MB 135
IPoIB-64MB 135
Application Level Performance 136
Memcached Design Using Verbs 139
Memcached Get Latency 140
Memcached Get TPS 141
Bandwidth with C version 142
Hadoop Sort Benchmark 145
Concluding Remarks 147
Funding Acknowledgments 148
Personnel Acknowledgments 149
Web Pointers 150

Komentáře k této Příručce

Žádné komentáře

SGI 10-Gigabit Ethernet Network Adapter Uživatelský manuál Strana 131

Komentáře k této Příručce

Související produkty a manuály pro Sítě SGI 10-Gigabit Ethernet Network Adapter