Parler may still be hosted in the US

Parler was recently kicked off of Amazon AWS for TOS violations. Parler claims that the big tech cloud providers weren’t willing to do business with them, and facing no place left to go, the site went offline.

Recently the site came back online in limited form. Some have observed they’re enlisting the help of a Russian-owned cloud service called DDoS-Guard. I have no prior understanding of DDoS-Guard, but the emerging narrative is that they’re now hosted in Russia and this is causing a bit of a stir.

Based on my analysis, the argument that they’re now hosted in Russia is not well supported. This is not an exhaustive analysis, rather treat it as a starting point for second guessing the dominant narrative.

One can look up parler.com in DNS and see that it resolves to 190.115.31.151, and one can look up that IP address in a WHOIS database to see what company owns the IP address. When I do this, I get a result that implies it’s a company based out of Belize.

% whois -h whois.arin.net 190.115.31.151
 
...
 
inetnum:     190.115.16.0/20
status:      allocated
aut-num:     AS262254
owner:       DDOS-GUARD CORP.
ownerid:     BZ-DALT-LACNIC
responsible: Evgeniy Marchenko
address:     1/2Miles Northern Highway, --, --
address:     -- - Belize - BZ
country:     BZ
phone:       +7 928 2797045
owner-c:     HUN8
tech-c:      HUN8
abuse-c:     HUN8
inetrev:     190.115.16.0/24
nserver:     NS1.DDOS-GUARD.NET
nsstat:      20210124 AA
nslastaa:    20210124
nserver:     NS2.DDOS-GUARD.NET
nsstat:      20210124 AA
nslastaa:    20210124

Verifying the authenticity of information in IP registrations is outside of my wheelhouse, but it would appear that DDoS-Guard is involved in some way, and there’s an agent in Belize that owns the IP address, and that they may be directed by Russia-owned DDoS-Guard.

Aside: 190.115.31.151 reverse resolves to ddos-guard.net. So, that’s a non-WHOIS-based clue that they may be involved. And if you do an HTTP HEAD on that IP address, the server identifies itself as being involved with ddos-guard.net. None of this so far is solid proof that parler.com has an actual relationship with DDoS-Guard the company, but there’s paperwork that points that way.

Anyway, does the fact that a Russian internet service is working with them mean they’re now hosted out of Russia? Not necessarily. I’m assuming DDoS-Guard, by its name, is a service you sign up with to protect your service from internet attacks. To be good at this, they would probably distribute your service to multiple datacenters (I’ll call them POPs from hereon out) across the world to eliminate single points of attack-failure. A similar service, Cloudflare, has an impressive map, for example.

In probing how widely distributed Parler’s service is, it seems they’re not distributed very widely at all.

When I run a traceroute from my residential internet service at home in Oregon, the traceroute starts going dark somewhere on NTT’s backbone around NYC.

oregon % traceroute 190.115.31.151
traceroute to 190.115.31.151 (190.115.31.151), 30 hops max, 60 byte packets
...
 6  be-10847-pe02.seattle.wa.ibone.comcast.net (68.86.86.226)  24.509 ms  20.180 ms  21.082 ms
 7  ae-31.a00.sttlwa01.us.bb.gin.ntt.net (129.250.66.149)  26.071 ms  26.060 ms  26.049 ms
 8  ae-14.r05.sttlwa01.us.bb.gin.ntt.net (129.250.5.133)  94.289 ms  94.278 ms  94.268 ms
 9  ae-2.r22.sttlwa01.us.bb.gin.ntt.net (129.250.2.44)  27.087 ms ae-1.r22.sttlwa01.us.bb.gin.ntt.net (129.250.2.9)  27.075 ms ae-2.r22.sttlwa01.us.bb.gin.ntt.net (129.250.2.44)  25.983 ms
10  ae-11.r20.nwrknj03.us.bb.gin.ntt.net (129.250.6.176)  94.453 ms  94.218 ms  94.210 ms
11  ae-1.r00.nycmny13.us.bb.gin.ntt.net (129.250.2.174)  94.202 ms  94.421 ms  94.425 ms
12  * * *
13  * * *
14  * * *

Note the “nyc” in the final hops along the route that reply. These names could be completely fabricated (naturally), but backbone ISPs tend to not screw around with that.

Of course, that doesn’t necessarily mean they’re entirely hosted in NYC. It also doesn’t mean that after the trail goes dark it doesn’t continue into Russia. Perhaps DDoS-Guard directs me to their presence in NYC because it’s closest to me in Oregon. Am I wrong to focus on this one IP address? Were I to resolve their name from another part of the world, would get a different IP and be sent to a different POP?

Nope!

From a cloud provider in Hong Kong:

hongkong % host parler.com
parler.com has address 190.115.31.151

From a cloud provider in Tokyo:

tokyo % host parler.com
parler.com has address 190.115.31.151

From a cloud provider in Chicago

chicago % host parler.com
parler.com has address 190.115.31.151

From a cloud provider in Dublin:

dublin % host parler.com
parler.com has address 190.115.31.151

All the same, which is pretty odd for a DDoS protection service.

While it’s possible to use a single IP address to publish a service across multiple locations, that only works for “session-less” services based on UDP. DNS can be replicated this way using anycast IP addresses. For session-oriented stuff, like a social networking web site (and mobile app), you need to use TCP, which prevents you from replicating a single IP globally. (TCP-based anycast exists, though in my experience it isn’t common for general purpose TCP)

What you would normally do in this situation is resolve a name to a different IP address based on where in the world the name was queried from. If someone looks up the IP from Hong Kong, you would reply with an IP that maps to a POP that’s closest to Hong Kong. And if someone looks up from Europe, you’d dispatch to an IP closest to Europe. That’s not happening here.

It’s probably not for lack of resources on DDoS-Guard’s part. Their network map (scroll down) shows they have POPs all over the world.

So, where in the world are they located?

A traceroute from Hong Kong also heads towards NYC:

hongkong % traceroute 190.115.31.151
traceroute to 190.115.31.151 (190.115.31.151), 30 hops max, 60 byte packets
...
11  ae-11.r20.nwrknj03.us.bb.gin.ntt.net (129.250.6.176)  191.798 ms  196.203 ms  190.617 ms
12  ae-1.r00.nycmny13.us.bb.gin.ntt.net (129.250.2.174)  192.586 ms  185.942 ms  192.661 ms
13  * * *
14  * * *
15  * * *

From Tokyo, the trail disappears near twelve99.net’s Hong Kong onramp:

tokyo % traceroute 190.115.31.151
traceroute to 190.115.31.151 (190.115.31.151), 30 hops max, 60 byte packets
...
18  snge-b2-link.telia.net (62.115.176.136)  81.262 ms  81.243 ms  82.168 ms
19  hnk-b1-link.ip.twelve99.net (62.115.116.147)  81.928 ms  81.851 ms  81.601 ms
20  * * *
21  * * *
22  * * *

From a host in Chicago, it also appears to go dark around twelve99.net’s NYC on-ramp:

chicago % traceroute 190.115.31.151
traceroute to 190.115.31.151 (190.115.31.151), 30 hops max, 60 byte packets
...
 5  te0-7-0-25.rcr21.b002281-5.ord03.atlas.cogentco.com (38.142.64.193)  1.528 ms  1.604 ms  1.639 ms
 6  be2409.ccr41.ord03.atlas.cogentco.com (154.54.29.65)  1.768 ms  1.558 ms  1.552 ms
 7  telia.ord03.atlas.cogentco.com (154.54.12.38)  1.354 ms  1.542 ms  1.339 ms
 8  * * *
 9  nyk-b3-link.ip.twelve99.net (62.115.139.151)  19.058 ms  19.041 ms nyk-b3-link.ip.twelve99.net (62.115.140.223)  19.876 ms
10  * * *
11  * * *

From the host in Dublin. Appears to go dark after routing through Amsterdam:

dublin % traceroute 190.115.31.151
...
25  slou-b1-link.telia.net (195.12.255.242)  25.141 ms  25.157 ms  25.143 ms
26  ldn-bb1-link.ip.twelve99.net (62.115.120.74)  18.279 ms ldn-bb4-link.ip.twelve99.net (62.115.117.122)  15.722 ms ldn-bb1-link.ip.twelve99.net (62.115.117.192)  18.512 ms
27  adm-bb4-link.ip.twelve99.net (62.115.134.26)  16.582 ms  17.376 ms adm-bb3-link.ip.twelve99.net (213.155.136.99)  18.577 ms
28  adm-b1-link.ip.twelve99.net (62.115.136.195)  18.286 ms adm-b1-link.ip.twelve99.net (62.115.137.65)  17.184 ms adm-b1-link.ip.twelve99.net (62.115.136.195)  18.804 ms
...

The above traceroutes alone don’t clearly prove that all traffic must head towards and terminate in NYC, but it’s not clearly pointing away from NYC either. You would expect if they were actually hosted in (e.g.) Russia the traceroutes wouldn’t look like this at all.

Here’s another technique. We can try to triangulate from a couple of locations based on ping latency. When I do a TCP-level “ping” on the HTTP port from each location, the round-trip latency is consistent with the amount of time it takes to access something on the east coast of the US. That is, about 100ms round-trip from Oregon, 100ms round-trip from Europe, and almost 400ms round-trip from Hong Kong.

Let’s assume NYC is the POP that’s serving parler.com. This isn’t a huge leap of faith; their network map mentioned earlier certainly indicates they have one in NYC. This still doesn’t mean it’s necessarily hosted in NYC. There are obviously lots of tricks providers can play. DDoS-Guard could be providing an on-ramp in NYC that then fetches the content from somewhere else in the world (Russia!?), and acting as a caching proxy for the content. Since Parler’s customers tend to be US centric there’s no reason to provision them on other POPs. But… if you’re already providing an on-ramp in NYC, why not also host the content in NYC?

At the minimum, I think the claim that Parler is now hosted in Russia is not very strong. That they might be still hosted in the US would line up with what I’m seeing here.

I don’t have a particular dog in this fight, though as an aside, it’d be kind of sad if you couldn’t be on the internet anymore just because Amazon, Google and Microsoft told you to get lost. Simply pointing out if we’re going to freak out about Russia being involved in some way, I think it should be accurate freaking out.

Further research

This is about all the energy I want to invest in this controversy. If you’re interested in continuing, rocks I would turn over include:

  1. Better traceroutes. In addition to ICMP based traceroutes, I also did tcptraceroute which wasn’t much different (so I didn’t mention it), though it’s possible newer tools and public registries can reveal more information about the topology.

  2. Confirming whether or not they’re using TCP anycast; it’s unlikely, but possible, which would blow a big hole in my claims. This BGP registration confirms DDoS-Guard is directly connected to a router that’s in NYC for that IP. Can you find another?

  3. Doing deeper HTTPS-level transactions into their site that bypasses caching, so you can possibly identify a lower bound latency on moving from their public endpoint to their content server; if it’s low enough, it could confirm they have authoritative content in, say, NYC, but if it’s too large it doesn’t disprove that it’s in NYC (they could simply be slow).

Leave a Reply

Your email address will not be published. Required fields are marked *