SGI 10-Gigabit Ethernet Network Adapter Uživatelský manuál Strana 142
- Strana / 150
- Tabulka s obsahem
- KNIHY
Hodnocené. / 5. Na základě hodnocení zákazníků
Hadoop: Java Communication Benchmark
• Sockets level ping-pong bandwidth test
• Java performance depends on usage of NIO (allocateDirect)
• C and Java versions of the benchmark have similar performance
• HDFS does not use direct allocated blocks or NIO on DataNode
S. Sur, H. Wang, J. Huang, X. Ouyang and D. K. Panda
“Can High-Performance
Interconnects Benefit Hadoop Distributed File System?”
, MASVDC „10 in conjunction with
MICRO 2010, Atlanta, GA.
0
200
400
600
800
1000
1200
1400
Bandwidth (MB/s)
Message Size (Bytes)
Bandwidth with C version
0
200
400
600
800
1000
1200
1400
1
4
16
64
256
1K
4K
16K
64K
256K
1M
4M
Bandwidth (MB/s)
Message Size (Bytes)
Bandwidth Java Version
1GE
IPoIB
SDP
10GE-TOE
SDP-NIO
CCGrid '11
142
- A Tutorial at CCGrid ’11 1
- Computing Systems 3
- Cluster Computing Environment 4
- (http://www.top500.org) 5
- Grid Computing Environment 6
- Compute cluster 8
- Cloud Computing Environments 9
- Hadoop Architecture 10
- Memcached Architecture 11
- • Hardware components 13
- • Software components 13
- • Ex: TCP/IP, UDP/IP 14
- – E.g., PCI, PCI-X 15
- – One bit per wire 15
- – Not scalable: 15
- Myrinet (1993 -) 1 Gbit/sec 16
- IB Trade Association 19
- • Network speed bottlenecks 21
- • I/O interface bottlenecks 21
- (not shown) 24
- • Fast Messages (FM) 27
- • Myricom GM 27
- IB Hardware Acceleration 28
- • Interrupt Coalescing 29
- • Jumbo Frames 29
- • Hardware Checksum Engines 29
- TOE and iWARP Accelerators 30
- 2006 (v3.0), 2008 (v3.1) 34
- PCI-Express (PCIe) 34
- 2003 (Gen1), 2007 (Gen2) 34
- IB, HSE and their Convergence 36
- Traditional Ethernet 37
- IB Overview 38
- Components: Channel Adapters 39
- • Switches: intra-subnet 40
- – Not directly addressable 41
- IB Communication Model 43
- Queue Pair Model 44
- Memory Registration 45
- Memory Protection 46
- (Send/Receive Model) 47
- InfiniBand Device 48
- Hardware ACK 48
- Hardware Protocol Offload 51
- • Buffering and Flow Control 52
- • Switching and Multicast 52
- Buffering and Flow Control 53
- Virtual Lanes 54
- Service Levels and QoS 55
- Traffic Segregation Benefits 56
- Identifiers) 57
- Switch Complex 58
- – 3D Torus (Sandia Red Sky) 59
- More on Multipathing 60
- IB Multicast Example 61
- IB Transport Services 63
- Reliability 64
- Transport Layer Capabilities 65
- Data Segmentation 66
- Transaction Ordering 67
- Message-level Flow-Control 68
- Concepts in IB Management 71
- Subnet Manager 72
- HSE Overview 74
- Differences 75
- – Multi Stream Semantics 76
- Offloaded TCP 83
- Myrinet Express (MX) 85
- Datagram Bypass Layer (DBL) 86
- • Solarflare approach: 87
- Ethernet 90
- IB Network 90
- IB Verbs 90
- Application 90
- Hardware 90
- IB Transport 91
- IB iWARP/HSE RoE RoCE 92
- IB Hardware Products 94
- Tyan Thunder S2935 Board 95
- IB Hardware Products (contd.) 96
- – Nortel Networks 97
- • Support for VPI and RoCE 98
- – www.openfabrics.org 99
- – OFED 1.6 is underway 99
- Verbs Interface 100
- (libibverbs) 100
- • Within the hardware: 101
- OpenFabrics Software Stack 102
- InfiniBand in the Top500 103
- Cray Interconnect 104
- NUMAlink 104
- SP Switch 104
- CCGrid '11 106
- • Other small clusters 107
- • Integrated Systems 107
- Other HSE Installations 108
- Presentation Overview 109
- 1/10 GigE 110
- InfiniBand 110
- Case Studies 111
- Latency (us) 112
- Message Size (bytes) 112
- Bandwidth (MBps) 113
- MVAPICH/MVAPICH2 Software 115
- One-way Latency: MPI over IB 116
- Bandwidth: MPI over IB 117
- Bandwidth: MPI over iWARP 119
- Convergent Technologies: 121
- Traditional Model 123
- Possible SDP Model 123
- InfiniBand CA 123
- SDP vs. IPoIB (IB QDR) 124
- IB on the WAN 126
- Features 127
- • Supports SONET OC-192 or 128
- “appear” local 128
- Sunnyvale 129
- 700 miles 130
- 3300 miles 4300 miles 130
- Cluster B 131
- Communication Options in Grid 132
- Interface 133
- Modern WAN 133
- Performance of Memory Based 134
- Data Transfer 134
- ADTS-64MB 135
- IPoIB-64MB 135
- Application Level Performance 136
- Memcached Design Using Verbs 139
- Memcached Get Latency 140
- Memcached Get TPS 141
- Bandwidth with C version 142
- Hadoop Sort Benchmark 145
- Concluding Remarks 147
- Funding Acknowledgments 148
- Personnel Acknowledgments 149
- Web Pointers 150
Související produkty a manuály pro Sítě SGI 10-Gigabit Ethernet Network Adapter
Sítě SGI network adapter Uživatelská příručka
(34 stránky)
(34 stránky)
Sítě SGI Ethernet Card Uživatelská příručka
(52 stránky)
(52 stránky)
Sítě SGI Ethernet Card Uživatelská příručka
(36 stránky)
(36 stránky)
© 2020, manymanuals.cz. Všechna práva vyhrazena. | 0.044 s |
Manymanuals.com
Manymanuals.de
Manymanuals.fr
Manymanuals.it
Manymanuals.pl
Manymanuals.cz
Manymanuals.es
Manymanuals-pt.com
Komentáře k této Příručce