The Great Firewall of China: A Digital Black Hole

Recently we were working with one of our customers and saw some irregularity in the traceroute data using Catchpoint Network Insights. Although the traffic is destined to go to our client’s website, it was instead going to Twitter/Facebook IPs via their ASNs. We have seen this happen in the past as well. But this time we decided to do an in-depth analysis to understand what was happening behind the scenes.

In the above Sankey chart, we can see the traffic originating from our nodes destined to the website IP addresses on the right. But instead going through Akamai as intended, the traffic is being forwarded to by Guangdong Mobile Twitter & Facebook (Red/Green arrows in the viz). We can verify this with the destination IPs, which do belong to Facebook and Twitter.

Below are some more snaps, and we see more IPs for Facebook and Twitter instead of the actual website. There is a lot of packet loss associated on almost every AS, which seems to be quite common in China region for traffic going in or coming out.

Some more instances:

We knew the Great Firewall of China was responsible, which was confirmed by data from DNS Experience Tests:

In the images above, request for the same domain returns the same CNAME but with different IPs (sometimes the correct destination, and sometimes spoofed destination IP).

What actually is the “Great Firewall of China”?

The Great Firewall (GFW) is a powerful and sophisticated censorship tool unlike any the world has seen before. It uses a combination of DNS tampering and IP address blocking to completely seal off access to specific IP addresses. It uses an Intrusion Detection System (IDS)-like system to inspect traffic for blacklisted keywords and terminate connections by injecting reset (RST) packets. While these tools can cause significant collateral damage, they are extremely effective for blocking almost any website for the vast majority of Internet users in China. Here are some of the methods that the China firewall uses to scan incoming/outgoing traffic.

DNS Tampering

DNS tampering is used in conjunction with IP address blocking, as changing domain names is much harder than changing IP addresses. The first step in DNS tampering is DNS injection. When a user attempts to connect to a domain, the computer queries DNS servers for the IP address associated with the domain name. GFW monitors each DNS query originating from clients inside China at the border of the Chinese Internet. If it detects a query to a blocked domain name, it injects a fake DNS reply with an invalid IP, or in some rare cases, an IP to another website. This fake DNS reply then trickles down to internal recursive DNS servers in China, with the incorrect pairing cached along the way, achieving DNS poisoning. Thus, almost all DNS resolvers in China have poisoned caches.

Unfortunately, this causes large scale collateral damage, affecting communication beyond the censored networks when outside DNS traffic traverses censored links. In this post, we analyze the causes of the collateral damages comprehensively and measure the Internet to identify the injecting activities and their effect.

As per a report published by Siggcomm, around 39 ASNs in China were injecting forged replies even for transit DNS traffic, and 26% of 43,000 measured open resolvers outside China, distributed in 109 countries, may suffer some collateral damage. They found that most collateral damage arises from resolvers querying TLD nameservers that transit through China rather than the root servers (F, I, J, K, L) located in China.

Root Servers currently in Beijing, China:

k.root-servers.net

IPv4: 193.0.14.129

ASN: 25152

Operator: RIPE NCC

f.root-servers.net

IPv4: 192.5.5.241

ASN: 3557

Operator: Internet Systems Consortium, Inc.

l.root-servers.net

IPv4: 199.7.83.42

ASN: 20144

Operator: ICANN

j.root-servers.net

IPv4: 192.58.128.30

ASN: 26415

Operator: Verisign, Inc.

i.root-servers.net

IPv4: 192.36.148.17

ASN: 29216

Operator: Netnod

Apart from these, there are around 9 in Hong Kong as well.

Consider the scenario where a resolver requests for the domain ‘www.xyz.com’ (or any GFW tampered/listed domain name), the resolution process would look like this:

The name is not already in cache => The ‘com.’ zone (NS) is not cached either (48-hour TTL) => Ask F, I, or J-root instances => Get directed to a root instance in Beijing

Once the query is directed to Beijing, the race begins!

Query returns the ‘com.’ zone servers (NS) => GFW may return an incorrect A record => If GFW wins the race, the A record is added to cache => The DNS cache is now poisoned

TCP Reset

Most content inspection schemes work by pushing all traffic through a proxy that refuses to serve results for forbidden material. However, a proxy-based system that can cope with the traffic volumes of a major network, or an entire country, would be extremely expensive and difficult to scale. Instead, the GFW inspects traffic by sending copies to out-of-band devices based on Intrusion Detection Systems. The original packets are unaffected, while the IDS inspects the content of the packet and the requested URL, matching them against a blacklist of keywords.

Once the IDS detects blacklisted keywords, the GFW router injects multiple forged TCP RST packets to both endpoints, forcing the connection to be dropped. The GFW then maintains the flow state for the source and destination IP addresses, port number, and protocol of denied request to block all further communications for hours at a time

IP Address Blocking

IP address blocking is a simple, lightweight one, yet extremely effective censorship tool. By peering with the gateway routers of all Chinese ISPs, GFW injects a list of blacklisted destination addresses into BGP (Border Gateway Protocol) and hijacks all traffic to blocked websites. In other words, the GFW forces routers to drop all traffic for blocked IPs. This technique is called null routing and can only block outbound traffic from China and permits inbound traffic. This is sufficient in most cases, as most current Internet communication require a three-way handshake to function.

Conclusion

The GFW of China is a sophisticated censorship tool that uses multiple techniques to scan and filter every bit of data that goes in and out of the Chinese network. If any of the traffic from your application passes through the China firewall, then it can be manipulated and redirected impacting end-user experience. To avoid such a situation –

• Strictly use HTTPS Protocol for the website traffic. Even back in 2014-2015 when HTTPS was a must-have for any website, China was discouraging the use of HTTPS for Chinese websites.
• Avoid routing requests across the GFW (if you can help it), as it intercepts anything that passes through the China Domestic/National Network, and might even impact the traffic coming from sources outside China.
• DNS was not primarily designed with security in mind. It was designed to be a scalable, public database which did not restrict access to its data. This exposed a lot of vulnerabilities in the system and led to multiple exploits. Start using DNSSEC, which adds security to the Domain Name System by enabling the validation of DNS Responses. Correct implementation of DNSSEC makes DNS less vulnerable to a very common type of attack called DNS Spoofing.

In addition to these points discussed above, you must:

• Improve DNS Monitoring Strategy so that issues are detected and resolved quickly.
• Use BGP monitoring to keep track of any malicious or unnecessary change in network routes.
• Invest in synthetic DNS Monitoring, it allows you to look at the performance of all the nameservers involved and detect any errors along the entire DNS resolution chain. It also gives you a window into the database records used by DNS servers (from MX to CNAME Records), which enables the diagnosis of the specific cause(s) of an error such as misconfiguration, DNS Cache Poisoning, or insecure zone transfers.