Routes and Rules

This section describes the use of alternative tools to the iproute2 suite for viewing the status of a network. They may be old, but route and netstat can still be useful when troubleshooting IPv4 routing tables.

Hosts, Gateways, and Routes

First, let's review the three (3) most common network connection types: Host, Gateway, and Route. Most routes point to a host or gateway. What is the difference?

A Host is a single, addressable device with a specific address on the network.

A Gateway is an upstream router that connects you to another network and performs address filtering. Gateways are not final destinations, but they are treated as both a host and router.

A Route is a pathway. Think of it like a highway. It's not a destination, but facilitates a packet reaching a destination (host). Routes are always a range of addresses.

Each route points to a destination device (host), a LAN (route), or an upstream router (gateway). By examining how each route in a routing table is structured, you can quickly identify whether it points to a single device (host address), another router (gateway), or a local route (an address range).

The Route Command: an Oldie, but a Goodie

Sometimes, all you need is a basic, low-frills tool to get the job done. When you need a quick snapshot of your network routes, what is your go-to resource? Although deprecated, the route command is still instrumental in obtaining useful information about a network.

One caveat: because it was created prior to the invention of Policy Based Routing, route only displays the main routing table. Therefore, if you have custom named routes or plan to add them, you won't be able to view them with the route command. This is obviously an important limitation to be aware of.

One advantage of route versus ip route lies in the former's flexibility in formatting how details are displayed. Both commands provide the same information, but route makes a few characteristics clearer and easier to follow, such as which routes point to gateways. You can output this information in a clear and consistent format that is easier to skim through. I encourage you to try it if you've never used route before.

While it's possible to use route to add new routes, I strongly discourage the practice. Instead, use the more recent ip route command, discussed here. Legacy compatibility challenges with the old route command may cause problems if you use it to create new routes.
Viewing existing routes with route is a non-issue.

To illustrate the point, let's first examine the main routing table using ip route.

ip route list table main

It will produce output similar to this:

default via 192.168.10.6 dev enp2s0
192.168.10.0/24 dev enp2s0  proto kernel  scope link  src 192.168.10.10

Now, let's compare the output when using the route command.

route

Its output looks like this:

Wow. What a difference. Which one would you rather look at?

The output from route is natively displayed in a format that's arguably easier to read, while simultaneously creating a more consistent output format. Which is best for legibility depends not just on personal preference, but also what information you need. For example, a drawback of route is its inability to display the local device's IPv4 address, while ip route's output does so. Take a closer look again at the end of the 2nd line of ip route's output.

... src 192.168.10.10

Notice the source (src) IPv4 address is displayed in the output from ip route (which in this case is the local device's IPv4 address). However, that information is nowhere to be found in the output from route. That said, does it matter? After all, the fact the source in this case is the local device is pretty obvious. Plus, you probably don't need to know the local device's IPv4 address when reviewing its routing tables and routes to begin with. And even if you did, there are other tools that will get you that information more succinctly (e.g. ip a or ifconfig if like me, you're into using old, deprecated tools)!

Another advantage to route is one can modify its output slightly via the -n switch, which produces a slightly different view.

route -n

Each line in the table above represents a network route.

Routes and hosts must be local; meaning they are connected to the same physical network as the local device.

Gateways are routes to a router. They act like both a route and a host. An example would be a router connecting your LAN to the Internet.

Now that the nuances between these connection types are clear, how do you know for sure what each line in the router table is pointing to? Answer: You check the flags.

Flags

Output from the route command uses flags to indicate the purpose of each route. Flags clarify what each object type is. Let’s begin with the most common flags.

Here is a list of other possible, but rare flags.

Flags provide information about the function of a route. Is it up (connected)? Is it a gateway? Is it a host? What do those things mean? Although the Flags column provides more information, it is the combination of four of the columns in the route output that definitively identify what things are.

Destination, Gateway, Genmask, and Flags

Let's break down the example above to explain the fields in route's output. Take a look at the headers:

Destination and Genmask

The Destination and Genmask are intertwined. The destination is the destination host address of a packet; the host address the packet needs to be delivered to. Genmask is just another name for Netmask; the latter being the industry-standard term for sub-network mask. The Netmask (Genmask) is evaluated against the destination address to derive a range of host addresses covered by the current route. The process of calculating the IPv4 range is explained in Netmask Filtering Deep Dive.

Gateway Address vs. Gateway Flag

The gateway column identifies the address of the router the packet will be sent to if it matches the specified route. The gateway address you see there will always be a router. A gateway flag indicates the address under the Gateway column is an upstream router. The gateway flag will always be set when the gateway address is pointing to an upstream router.

There is one special case - when the gateway address is 0.0.0.0 - that indicates the destination router is unspecified (i.e. there isn't one). When this is the case, it means the true destination address is local (the local device is directly, physically connected to the destination IPv4 address via the local network). Therefore, the route will dispatch the packet directly to the destination device. You will also notice the gateway flag is not set.

The Up Flag

The presence of an Up flag (U) indicates the routing table entry is a route, and it is currently active (Up). If you don't see the Up flag set for a route or gateway, it could be down. Take note of this possibility when troubleshooting network connectivity issues.

route vs. route -n

Above, we briefly saw some differences in table output between route and route -n. Revisiting those examples above, let's break down their differences.

Here, the route representation is shown again.

And here is its route -n equivalent.

Their differences are highlighted above. Breaking them down:

1. default vs. 0.0.0.0 representing the default destination path
2. * vs. 0.0.0.0 representing an unspecified or non-existent gateway address
3. The -n version displays host numbers only, while the command without arguments displays qualified hostnames

There are advantages and disadvantages to either method. Which is "better" is subjective. Some prefer viewing hostnames. If you have a large network, this is likely to be an easier method of monitoring what is what, versus remembering IPv4 addresses. On the other hand, some people find the -n version easier to clarify the relationship between routes. The latter can be particularly true if you incorporate Split Gateways in your routing table.

Default Routes

Examining the first line in either view, we can see it identifies an upstream router that appears to be the default route. How do we know?

The biggest differences between route and route -n are their handling of how the default route and unspecified addresses are displayed. When using route, you just need to remember * means unspecified. However, with route -n, there's a bit more evaluation involved. No matter which version you're using, you should examine all the clues in the routing table to verify the default route is what you think it is. Furthermore, routing tables can contain errors. When they do, you will sometimes be able to detect those errors by virtue of the route flags not looking quite right.

Normally, the default route is obvious. You can only have one default route. Or can you? Technically, there can be only one default route, but that doesn't mean there must only be one default route in a routing table. It means any given destination can only have one default route path. This is an advanced topic that is explained in the articles describing Gateways and Split Gateways.

When using route -n, here is the thought process to verify a default route:

1. Destination and Genmask are both 0.0.0.0. This indicates the full range of IPv4 addresses. The Genmask (Netmask) will be inverted to create an upper boundary IPv4 address that encompasses all IPv4 addresses (i.e. 255.255.255.255). If that logic is not familiar to you, check out Netmask Filtering Deep Dive to be helpful.
2. The Gateway flag is present (G), indicating the gateway address in the routing table is pointing to a gateway host.
3. Presence of the Up flag (U) is only possible on a route, and indicates the route is currently active (Up). Since a gateway acts as both a gateway and a host, this flag should be present.

What gives? A destination address can't be both a host and and a route, as a host is where the packet is delivered, but a route is handing the packet off to another router. Or can it? That is basically what a gateway is! An upstream router, which acts as both a route and a host. Certain packets need to be routed or directed to it, but it's not their final destination. A gateway is a host that acts like a route. It is an addressable device (host), yet it is not the destination, but moves the packet closer to its destination (a route).

Here is the first line from route -n one more time:

If you were to translate that line in the routing table to a human-readable format, it would sound something like:

"Route all traffic addressed between 0.0.0.0 and 255.255.255.255 to the gateway at 192.168.10.10 via the eth0 network interface."

When both the Destination and Genmask are 0.0.0.0, it means all traffic will be passed to that route unless a more specific route is identified in the table. This is the functional definition of a default route.

LANs

Have you noticed the similarities between routing table entries displayed by ip route vs. route commands? They present the same information in different ways. One advantage of ip route is it identifies the current computer's IPv4 address on the LAN (src=xxx.xxx.xxx.xxx).

Returning to route's output, the second line in the main routing table example describes the LAN the local server is connected to.

This route instructs the Linux kernel to send any network traffic with a destination address within the LAN's IPv4 address range (unicast) directly to that address, via the indicated interface. Likewise, if a broadcast message is called for it would be sent out over the indicated interface.

How do we know? The 2nd line's flags column has only an Up flag. Therefore, we know this is a route. So, what? The absence of any other flags indicate this is a network route connected directly to the specified interface. And, I can tell it is a Local Area Network (LAN). How do I know? I can deduce this from the following observed data points:

1. The local server is on a LAN;
2. The gateway is within the same IPv4 address range; and
3. A cursory review of ip route (or even ifconfig) shows this computer's IPv4 address src 192.168.10.10 is on the same network (Destination: 192.168.10.0)

Basically it means your server's routing table believes anything in that range is connected directly to the specified network interface. This informs the kernel there are no other hops required to reach an address in that range. Yet, a route has no awareness of which other hosts (if any) are truly attached to that route.

If you were to translate the 2nd line in the table to a human-readable format, it would sound something like:

"Route all traffic addressed between 192.168.10.0 and 192.168.10.255 via the eth0 network interface."

Iface (Interface)

The next most-important table column in route's output is the Iface column. "Iface" stands for Interface. Specifically, the outbound network interface. Notice all the table entry examples point to the same outbound network interface (eth0). Things get tricky when there's more than one network interface. For instance, imagine your server had two (2) network interfaces and you wanted to restrict all local (LAN) traffic through one network interface, and all outbound internet traffic through the other. You could easily accomplish through proper route configuration. In that case, routes would likely be the simplest method of achieving your goal, perhaps directing all packets outbound to the LAN via Interface #1 and all other outbound traffic through Interface #2.

Metric, Ref, and Use

I'll briefly explain the other, relatively unimportant data columns in route's output. Note these terms have different meanings and uses depending on which tool they are associated with. Make sure you understand their purpose with respect to route and don't get confused with other routing tools' functions. I say that simply because these are somewhat common names/terms in diagnostic routing tools.

Metric

Metric: [Deprecated] Measurement of distance to target in hops. No longer used by Linux kernel. Some routing daemons may still use it.

ip route has a parameter called metric that establishes a priority for routes with equal specificity. Normally, ip route's last-ditch method of resolving a tie in routes is to favor the route with the lowest route number in the current table. ip route's metric parameter supercedes that logic with the metric value. The lowest metric value wins. Zero (0) counts as the lowest possible value. The lower the metric value, the greater the importance of that route relative to other routes of equal weighting in the same table. It essentially allows a network administrator to prioritize different routes that are otherwise weighted equally by ip route's matching algorithm. A common use is for load balancing. For example, if you had multiple outbound routers with equal priority weighting based on route matching, and you wanted to prioritize the order in which they would be chosen based out of those which are reachable (up) at any given time. Routes that are down are removed from their routing table until the routing cache is refreshed (e.g. reboot or flushing the router table cache).

Ref

Ref: [Mostly Deprecated] Number of references to a route. Mostly obsolete. Not used by the Linux kernel. However, I have seen it used in Red Hat Linux implementations. It can be helpful (when implemented) for network administrators to measure how heavily used a particular route is.

Use

Use: Counter of lookups of this route by the local device. Can be made to represent either hits or misses (default = hits).

Utilizing route Successfully

Here are some additional tips to help you navigate your routing tables via route:

1. If you don't specify a particular table, you will only see the contents of the main routing table.
2. Routes are not prioritized in a top-down pattern as you might expect. They are evaluated on a longest-match wins basis. The routing table will isolate routes where the route prefix matches the packet prefix. It then chooses the route with the longest matching rule, followed by the longest matching route prefix. This is regardless of the order in which the routing instructions appear in the table.

3. A line containing a G flag and Gateway address, but both Destination and Genmask are 0.0.0.0 is the default gateway/router for the specified interface. Outbound traffic directed to the main routing table not captured by one of the other routes will be sent to this gateway.
4. You can have only ONE gateway per interface. Split gateway entries are possible (a gateway appears on more than one line, but has its full address range split into multiple line entries). This is an advanced topic, discussed under Split Gateways.
5. A Host can also be a Gateway. If you see an entry with both Host and Gateway flags set, this indicates something special about that entry. It could be a router, gateway, or the local gateway for a VPN (Virtual Private Network).

Reiterating point #2 again, because it is a subtle but very important nuance of Linux routing:

Of all the routes stored in the RPDB that matches the packet, the longest will trump every other branching decision factor.

In layman's terms, this means most specific route will usually be chosen, but that isn't always the case. It is imperative you pay attention to how your routes are entered into the RPDB, as it can impact how your packets are routed. Similar logic also applies to the network filtering rules in ip rule.

netstat

There's one more legacy tool in this genre worthy of discussion relative to route and ip route: netstat

Are you hunkering to learn more obscure and deprecated networking tools? Yes?! Let's continue with a look at netstat! Even though it was deprecated a long time ago, netstat (an acronym for "network statistics") remains one of my favorite networking tools for gathering quick information about a network or host. It has a number of nifty features, but for the purpose of this discussion I'll focus just on its use as an alternative to route.

Running netstat -r produces output nearly identical to route

Likewise, netstat -rn produces output roughly equivalent to route -n

So, what's so special about netstat versus route? Nothing really. I mention it simply because if you're interested in the output from route, you could alternatively run netstat instead.

Analyzing Outputs: route vs netstat

Let's take a look at how routing command output differs depending on the tool and command argument(s) used to generate the results. They all show the same data, but it is displayed slightly differently. There's no right or wrong approach here. Use whichever method you prefer.

The sample outputs below are all demonstrating the view of a network with the following characteristics:

• An upstream router at address 192.168.10.100 with hostname "matrix" on domain name "skynet"
• LAN #1 on IPv4 range 192.168.10.0 - 192.168.10.255
• LAN #2 on IPv4 range 10.10.10.0 - 10.10.10.255
• All connections accessed via physical interface "eth0"

Each result is slightly different, but the primary information describing the network is the same. Note netstat reports route flags just like route does, and they mean the same thing.

route

Notice the Gateway connection has a name instead of an IPv4 address and the routes representing the LANs just have an asterisk under the 'Gateway' column.

netstat -r

Again, the Gateway connection has a name instead of an IPv4 address and the routes representing the LANs just have an asterisk under the 'Gateway' column.

route -n

Here the Gateway connection is displayed as an IPv4 address, and when the route is a locally connected group (i.e. a LAN; packet not being sent to a router), you see the 0.0.0.0 IPv4 address.

netstat -rn

Again, the Gateway connection is displayed as an IPv4 address and LANs appear as 0.0.0.0 IPv4 addresses.

Parameters

Astute readers will have noticed there are differences in the output headings between route and netstat. What do those column headings mean? They are operational, real-time alerts about the network. For the most part they may be ignored, though if they are anything than zero you might have a network hiccup or two worth investigating.

Parameters related to route are discussed in more detail under Metric, Ref, and Use.

route

versus

netstat -r

Metric (iproute2) vs MSS (netstat)

Metric = [Deprecated] Sets route priority for routes with equal specificity; i.e. a user-defined tie-breaker. This will supercede the default last-ditch tie-breaker logic, which is to select the route with the lower route number.

MSS = Maximum Segment Size. TCP parameter. Used to split packets when the destination indicates it can’t handle larger segments. A value of 0 means “no changes”

Ref (iproute2) vs Window (netstat)

Ref = Number of references to this route. This is mostly obsolete. It is not used by the Linux kernel. However, I have seen it used in Red Hat Linux implementations. It can be helpful (when implemented) for network administrators to understand how heavily used a particular route is.

Window = Default window size (how many TCP packets can be sent before at least one has to be ACKnowledged); usually zero (no changes).

Use (iproute2) vs irtt (netstat)

Use = Counter of lookups of this route by the local device. Can be made to represent either hits or misses (default = hits).

irtt = Initial Round Trip Time and may be used by the kernel to guess about the best TCP parameters without waiting for slow replies. Not used much.