11 packets transmitted, 11 received, 0% packet loss, time 10005ms
rtt min/avg/max/mdev = 89.544/92.286/98.582/2.450 ms
It still isn't quite as fast as the Hurricane Electric tunnel (maybe 5ms slower). The path to the UK got a lot better and the native IPv6 is only around 2ms slower than the tunnel.
Original Story
I have been running IPv6 for a long time now by using a Hurricane Electric tunnel. It has worked well, and has been reliable. I managed to make Sage with the tunnel broker certification.I have been running a couple of NTP servers on IPv6 that are part of the pool.ntp.org pool and I started to wonder whether the amount of noise (jitter) that was being seen by the monitoring system was due to the tunnel. Happily, Comcast had rolled out native IPv6 as far as my cable modem. I have two IP addresses on my service (I was only using one) and this allowed me to bring up a new firewall (an Edgerouter Lite) and configure it to get a prefix delegation from Comcast and get native IPv6. [There will be significant complications when I want to run both prefixes on my LAN]
The obvious thing to do was to test the latency (using ping6) to get to various destinations over the tunnel and native connection. This is where things start to get confusing.
I'm based in Massachusetts and my tunnel is terminated in NYC.
Test1: www.mit.edu (hosted by akamai): 2001:559:11:183::255e
Native: 10 packets transmitted, 10 received, 0% packet loss, time 9014ms
rtt min/avg/max/mdev = 9.113/11.309/17.273/2.383 ms
Tunnel: 10 packets transmitted, 10 received, 0% packet loss, time 9000ms
rtt min/avg/max/mdev = 42.437/43.936/47.292/1.454 ms, pipe 2
Test2: banjo.employees.org 2001:1868:205::19 (west coast)
Native: 10 packets transmitted, 10 received, 0% packet loss, time 9012ms
rtt min/avg/max/mdev = 98.436/100.445/104.056/1.593 ms
Tunnel: 10 packets transmitted, 10 received, 0% packet loss, time 9010ms
rtt min/avg/max/mdev = 83.349/85.815/92.942/2.651 ms, pipe 2
Test3: 2a02:b80:0:6:7b::2 (some random NTP server in the UK)
Native: 10 packets transmitted, 10 received, 0% packet loss, time 9012ms
rtt min/avg/max/mdev = 118.134/119.084/119.891/0.650 ms
Tunnel: 10 packets transmitted, 10 received, 0% packet loss, time 8999ms
rtt min/avg/max/mdev = 88.790/91.656/95.290/2.253 ms, pipe 2
In two out of the three cases, the tunneled connection has less latency than the native connection. In fact, in most cases that I tried, the tunneled connection is better. The reason for the first test above may be that I did the lookup of www.mit.edu over the native connection, so it probably returned an address that was close to that exit point...
So what is going on here? Maybe a traceroute will help -- unfortunately there seems to be a dearth of reverse IP mappings registered in the DNS so that it is not possible to guess the entrie path that the packet is taking.
For employees.org via the tunnel, the IPv6 path is NYC->???->ORD->DEN->SV1->SCL. I'm guessing that SCL is Santa Clara and maybe SV1 is Silicon Valley. This seems a reasonable route. The IPv4 path to get to the tunnel endpoint is direct over Comcast's network to 111 8th Ave in NYC and then directly into Hurricane Electric's network. The Comcast path is almost entirely unnamed routers.
For the random NTP server, the tunneled path goes on Hurricane Electric's network over to the UK and then there are a couple of unnamed hops. The Comcast connection goes via Illinois before jumping across the pond.
One thing does stand out in the Comcast traceroutes:
3 te-0-14-0-0-ar01.woburn.ma.boston.comcast.net (2001:558:200:18c::1) 17.495 ms 20.326 ms 15.029 ms
4 2001:558:0:f6c1::1 (2001:558:0:f6c1::1) 51.400 ms 47.531 ms 47.752 ms
5 he-0-11-0-0-pe04.350ecermak.il.ibone.comcast.net (2001:558:0:f8ce::2) 43.888 ms 42.834 ms 40.432 ms
The hop from Woburn to 2001:558:0:f6c1::1 takes a long time.
The hops across the pond both take around 70ms (round trip). This is pretty reasonable given that the path is maybe 4,000 miles each way.
In short, I'm sticking with Hurricane Electric for now. I've reached out to people in Comcast to figure out what is going on, but I'm not having much luck (yet) and I am now actively engaged with (I think) someone who can understand the problem and take action.
