Traceroute, CrackerJack’s Ignorance, and Network Information

26 02 2011

Crackerjack continues to amuse us with his ignorance. Last Wednesday he said something that is not strictly relevant to the RedZone debate, but it is amusing because he does not realise that. I repeat it partly for the amusement but partly because I know a few of my readers appreciate some geeky stuff, and the rest can ignore this post as irrelevant and move onto something more interesting.

On Wednesday Crackerjack said:

well the greenzoners are a bit stumped when it comes to tracking the link to the database when all their traceroute enquiries end at dyndns

Now I did not immediately correct him, because frankly who cares? We all know where zFire’s computer is, right down to his flat number! But even if we did not have this, we don’t need to use traceroute to find out the information (and in any case its just nonsense again. The packets do not touch the DynDNS network at all, as we will prove later on). Given any IP address, you can quickly look up the records on its registration. In many cases this will give you the area where the address is in use (although not always. Sometimes it is not even the right country).

I did a lookup on the IP address for online. Note a few things that this tells me:

IP Location: United States Los Angeles Comcast Cable Communications Inc

So zFire uses a Comcast cable connection. This gives him a fair bit of bandwidth, although it is essentially a home grade solution. It looks like it is the Washington State part of the Comcast service.

OrgAbuseHandle: NAPO-ARIN
OrgAbuseName: Network Abuse and Policy Observance
OrgAbusePhone: +1-856-317-7272

Now you know who to call to complain.

Also we have:

Reverse IP: 2 websites use this address. (examples:

The reverse ip misses the dyndns domains, but still we have zFire’s other hobbies (porn and pretending to be a pro gambler).

So I am really at a loss as to why Crackerjack says greenzoners (and I presume he really means all non spyware operators) are stumped. What precisely does he think we don’t know?

He was replying to this query:

Can you comment zFire, is it feasible to cloak the domain, effectively neutralising this viewer change?

Just for the record – the answer to that is no. If you want us to send data to the redzone database you can’t “cloak” it. That is like saying “can we make everyone take a train at Euston station, but not tell them where the stayion is?” It is just not going to happen. The only thing I am stumped about is what on earth Crackerjack thinks it is that he can hide here!

But…. never one to pass up a chance to get geeky, let’s actually try traceroute.

As expected my traceroute does not get stuck at all. The only difference between me and crackerjack is I don’t use windows! How is it that crackerjack’s traceroute fails and mine succeeds?

To understand this you need to understand a little about how traceroute works – and what it tells you.

Traceroute is a program that is designed to tell you the interface IPs of nearside[0] interfaces of the routers between the person running a traceroute and another node on the Internet. It does this by using a particular value in the IP datagram header called the TTL (or hop count in IPv6). The TTL – Time to Live – is a number in the datagram header that is decremented by 1 each time the datagram passes through a router.[1] If the TTL is decremented to 0 then the packet is discarded as undeliverable, and an ICMP message (think of it as an Internet error or control message) is sent back to the originating node to say the delivery failed for the reason “time exceeded”.

The purpose of the TTL is to protect the Internet from routing loops. Imagine if Router A was told to pass all traffic to router B, and router B is told to send all traffic to C, and C is told to pass all traffic to A…then once router A receives a packet it will spin around the routing loop A-B-C-A-B-C add infinitum, until everything collapses because all the other packets are doing the same thing.

Most real routing loops are more subtle than that, but they do happen.

So inserting a TTL ensures that if a packet cannot be delivered in a reasonable number of hops, it is thrown away to save bandwidth and we send back an error message (as long as this error does not also get stuck in the loop. If that happens the failed error message does not generate another error message). The initial TTL is usually set at 64, 128 or 255 hops before the packet is discarded. No routes on the Internet are that long, nor ever likely to be.

That is great, but then Steve Deering postulated in the 1980s that we could use the TTL value to recover information about the routing nodes a packet passes through on its way to the destination node. To do this is actually quite simple: we send a packet to the destination node but we set the initial TTL value to 1. This means that the very first router that receives our packet will decrement the TTL to 0 and thus discard it, sending back an ICMP time exceeded message.

That ICMP time exceeded message is sent by the router and thus has the IP address of the router on it. We have thus discovered the nearside IP address of the first router on the network path. Just in case we have packet loss, we repeat a couple more times to be sure.

Now how to find the next hop? I know – let’s make the TTL 2 now. So now we deliver our packet to the end node with a TTL of 2. The first router decrements it to 1, and the second hop now decrements to 0. Thus the second router sends back our error message and we have the address of hop 2.

We repeat the process until either our packet is delivered or until we have tried all the TTL values (255 is the largest TTL).

There is a problem. How do we know the packet was successfully delivered?

The traditional solution was to send the packet to the destination node to a port that is not listening. Choose an unlikely UDP port and send the node a packet…when the datagram is eventually delivered, the node we sent it to will send back a different ICMP error message. This time it is still a destination unreachable message, but the cause is that there is nothing listening on our chosen port (port unreachable). This is the default operation for Unix systems.

Microsoft Windows has a program called tracert that does much the same thing but does not try to send a UDP packet al all. Instead it sends and ICMP echo message (a ping packet). If it gets an echo reply (a reply to the ping) then the packet reached the destination.

Sometimes one method works, sometimes the other does. The problem here is firewalls. Many firewalls will filter out ICMP echo replies because of, for instance, the problems with the Nachi virus that used these packets to locate other machines to attack. Other firewalls block all traffic to all but a few UDP ports. If a firewall silently blocks packets (with no error message), then traceroute just receives no reply and cannot tell this from packet loss. The program inserts a * in the data it receives for lost packets.

Sometimes a router will not respond with ICMP destination unreachable packets, but it will pass them from following nodes. That is fine – that node just appears as a row of “* * *” in our output, but traceroute still discovers all other nodes. But if all traffic is filtered beyond a certain node, and we receive no more ICMP messages, we cannot tell when traceroute completes. All the program can do is work through all TTLs, and stop when the TTL reaches 255. This looks like a lot of “* * *” lines in the output, although usually the final destination is only a hop or two after this starts to happen.

If we get this problem, we can try to work around it. For instance, maybe trying traceroute with TCP packets or packets to port 80 will work. There are all kinds of options to traceroute, and you can retry the trace at a certain node to save time. With a bit of work, many apparently dark routes can be lightened.

Not necessary for ZFire’s route though… here is the result of my traceroute with ICMP (as windows does it by default). I only include the last few hops:

11 ( 213.193 ms 212.087 ms 229.980 ms
10 ( 215.151 ms 295.391 ms 215.994 ms
11 ( 295.998 ms 306.183 ms 218.683 ms
12 ( 217.853 ms 279.288 ms 217.092 ms
13 * * *
14 * * *
15 * * *
16 * * *

See it failing there? not at dyndns of course. DynDNS is used for the name look up but no packets are routed through their service. It fails in the seattle area of the comcast network.

Now here is my default UNIX traceroute (using UDP packets). I get exactly the same with TCP incidentally:

12 ( 264.004 ms 280.467 ms 216.272 ms
10 ( 295.643 ms 271.381 ms
11 ( 307.256 ms
12 ( 282.131 ms 217.690 ms 320.098 ms
13 * * *
14 ( 257.890 ms 307.147 ms 307.233 ms

So hop 13 did some ICMP filtering but zFire’s router seems to reply fine. In fact, I think maybe his router is a NAT box set up to forward all UDP/TCP traffic to the Mac, in which case the Mac replied. I could investigate further if I cared. But I don’t.


[0] All routers by definition must have at least two non loopback interfaces. The router takes packets from one interface, and makes a decision as to which of its other interfaces to pass the packet onto. Basically a router takes a packet and forwards it somewhere else. Think of it like the postal service. A router is a sorting office. Letters arrive at the office, and someone looks at the address on the envelope and throws the letter into one of a number of sacks. Some sacks go to, say, London. Some go to Birimingham and so on. On arrival at the destination there is more sorting and the process continues until a postman takes his sack to your door and posts the letter through your letterbox (straight into the waiting mouth of your dog, who swallows it).

Now traceroute tells you the address on the door of the sorting office as you go in, not the address on the sack as you go out. That is what I mean by nearside interface.

[1] The RFCs actually say the TTL also gets decremented for each second a router holds the datagram without forwarding it, but as this never actually happens, we can ignore that technicality.




One response

28 02 2011
Mysty Saunders

Sent an email to for violating their AUP

participate in the collection of very large numbers of e-mail addresses, screen names, or other
identifiers of others (without their prior consent), a practice sometimes known as spidering or
harvesting, or participate in the use of software (including “spyware”) designed to facilitate this

%d bloggers like this: