Existing tools
PSC Speed Page:
PSC maintains a service with a web front end at http://speedpage.psc.edu that lists average data transfer rates between all XSEDE resources. More information regarding the service is available at https://sysops.xsede.org/xes-index/
Speedpage reports the results from automated GridFTP testing that runs in a mesh between XSEDE site storage systems twice daily, once at 2amET and once at 2pmET. The test command and output can be viewed by clicking on a test result number in the table. The table entries are the high throughput, the low throughput, and the average of those two values.
Maintained by: Sys Ops group - Bob Budden, Bryon Gill (PSC)
INCA testing:
Inca (http://inca.sdsc.edu/) detects cyberinfrastructure problems by executing periodic, automated, user-level testing of XSEDE software and services. The results of the periodic tests are stored and can be displayed via a web interface. Inca tests for XSEDE resources can be found at http://inca.teragrid.org/inca/view/status/sp.
Maintained by: Hardware: Sys Ops group - Andrew Ferbert (SDSC)
Software suite: Kate Kaya, Shava Smallen (SDSC)
perfSONAR:
perfSONAR is a widely-deployed network test and measurement infrastructure that is used by science networks and facilities around the world to monitor and ensure network performance. perfSONAR test results are useful for viewing network performance trends over time and identifying when performance problems may be occurring.
Available measurement and test tools:
pscheduler - pScheduler is responsible for managing the execution of network measurements, or more generally tasks, in perfSONAR. (see http://docs.perfsonar.net/pscheduler_intro.html for more information)
OWAMP - One way active measurement protocol that measures delay and loss (automated and manual)
traceroute - traceroute changes are logged (automated)
reverse traceroute and traceroute responder (manual)
reverse ping and ping responder (manual)
BWCTL - (Bandwidth Control) was used to schedule network tests on perfSONAR hosts prior to perfSONAR v4.0. BWCTL is still supported in v4.0, but is considered deprecated and will be unsupported in perfSONAR 4.1 and beyond (automated and manual)
The XSEDE mesh includes 10GbE-connected perfSONARs at: IU, LSU, NCAR, NCSA, NICS, PSC, Purdue, SDSC, and TACC. The currently running automated tests include:
iperf3 TCP
approximately 1 test every 6 hours for 30 seconds checking for throughput
iperf3 UDP
approximately 1 test every 24 hours for 10 seconds at 50Mb/s checking for loss
OWAMP - One Way Active Measurement Protocol measures latency between hosts. Settings in use are:
packet_interval 0.1 # Send 10 packets every second (i.e. pause 0.1 seconds between each packet
loss_threshold 10 # Wait no more than 10 seconds for a response
session_count 18000 # Refresh the test every half hour (once every 18000 packets)
sample_count 600 # Send results back every 60 seconds (once every 600 packets)
packet_padding 0 # The size of the packets (not including the IP/UDP headers)
bucket_width 0.0001 # The granularity of the measurements
force_bidirectional 1 # do bidirectional test
pingER
Test every 5 minutes with 10 packets, 1 second interval between packets
Loss and latency
traceroute
test every 10 minutes
A summary view of perfSONAR test results is available via the XSEDE Maddash dashboard. Clicking on a square in the grids will bring up graphs of test results in each direction between the two specific endpoints. The default view is for one week. The overall duration of view can be selected through the "Report Range" selector in the upper right-hand corner. The display also supports zoom-in and zoom-out and dragging the display left or right.
Current hardware is from ca. late 2011/early 2012:
- Supermicro X8DAH+-F, Dual XEON motherboard
- 2 CPUs 2.4GHz Intel Xeon E5645 Six Core
- 24 (6 * 4) GB DDR1333 ECC memory
- 240GB OCZ VERTEX 3 MIOP SSD (system disk)
- 2 Hard Drives 2TB Hitachi 7200RPM Enterprise (collected data storage), software-configured RAID1
- Dual Intel Ethernet 10/100/1000
- Myricom 10GbE, 10G-PCIE2-8B2-2S, Dual SFP+ PCIe Gen2
- DVD
- 3U Supermicro Chassis, CSE-835TQ-R800B (3U for multiple PCIe slots and any possible full-height cards in the future)
sysctl parameters
Parameter | Value |
net.ipv4.tcp_rmem | 4096 87380 134217728 |
net.ipv4.tcp_wmem | 4096 87380 134217728 |
net.core.rmem_max | 268435456 |
net.core.wmem_max | 268435456 |
net.core.netdev_max_backlog | 1000 |
net.ipv4.tcp_moderate_rcvbuf | 1 |
net.ipv4.tcp_no_metrics_save | 1 |
net.ipv4.tcp_timestamps | 1 |
net.ipv4.tcp_congestion_control | htcp |
XSEDE perfSONAR list
Site | Hostname | Host IPv4 | Host IPv6 | Connectivity |
IU | 149.165.238.190 | 10GbE | ||
LSU | 208.100.64.226 | 10GbE | ||
NCAR | 128.117.212.249 | 10GbE | ||
NCSA | 141.142.150.8 | 2620:0:c80:500::2 | 10GbE | |
NICS | 192.249.6.3 | 10GbE | ||
PSC | 192.231.243.57 | 2001:5e8:ffff:3::39 | 10GbE | |
Purdue | 128.211.136.42 | 10GbE | ||
SDSC | 198.202.105.14 | 10GbE | ||
TACC | 129.114.0.189 | 10GbE |
Login is provided upon request for networking staff at XSEDE perfSONAR host sites. Access is via user ssh key, with sudo privilege available.
Web admin for automated test scheduling is via password.
Myricom 10GbE adapter information
MYRICOM 10GBE | |
DRIVER | 1.5.3-1.534 |
FIRMWARE | 1.4.57 -- 2013/10/23 13:58:51 m |
MODEL | 10G-PCIE2-8B2-2S |
Vendor Contact
Joey P. Sims <JoeyS@senecadata.com>
National Account Manager
Seneca Data Distributors, Inc.
5895 Shiloh Rd, Suite 101
Alpharetta,GA 30005
P 877.291.8507
F 315.579.1300
More information regarding speedpage: