Routes and Rules

#
# reserved values
#
255local
254main
253default
0  unspec
#
# local
#
#1inr.ruhep

~# ip route show table {table name}

~# ip route show table local

~# ip route list

default via 192.168.10.10 dev eth0
192.168.10.0/24 dev eth0  proto kernel  scope link  src 192.168.10.11

~# ip route show table main

default via 192.168.10.10 dev eth0
192.168.10.0/24 dev eth0  proto kernel  scope link  src 192.168.10.11

Scope Values. "Scope" describes where the address is valid.

Proto Values. Also known as "RTPROTO" or "route protocol." It identifies the process that created the associated route.

As you can see, one can group several of these into a category of "drop the packet" (namely blackhole, prohibit, throw, and unreachable). Dropping and rejecting packets may also be accomplished via iptables. The unicast (forward packet to address such-and-such), local (send to localhost), and broadcast actions instruct the kernel where to send the current packet.

Adding New Routes

So, you've decided you want to add a new route. Perhaps you want to setup a VPN tunnel or add a new network interface to your server and you need to tell it how to use the new interface. Adding a new route boils down to instructing your server how to direct traffic from or to a specific IP address or range of addresses. You either want to create a route the server doesn't understand or see right now, or add a new one that changes how your server currently directs traffic.

Regardless of why you want to add a new route, the process is the same. Just remember routes affect both inbound and outbound network traffic.

There are three options when adding a new route:

1. Add a route to the default routing table (main)
2. Add a route to another existing table
3. Create a new route table and add at least one new rules that points to it

To add a new route, the ip route command argument structure is:

ip route add [type] [prefix] via [next hop] dev [interface name] table [table name/ID] src [source IP]

Prefix means the destination IP address, including netmask (if present). If type is omitted, unicast is presumed. Nearly all routes are unicast. For other types, refer to the charts above for information. Next hop is the gateway or upstream router the packet will be sent to.

If prefix is set to default it is the same as assigning an IPv4 address of 0/0 or 0.0.0.0/0 or 0.0.0.0/0.0.0.0.

The formatting boils down to the identification of the destination IP address or range, whether the route is direct or via a gateway, network device, and it may contain other values depending on whether or not the route points to a gateway, host, or a group of IP addresses (e.g. a LAN branch). The preceding sections (especially Examining Existing Routes) have more detailed explanations on the syntax.

Here's an example of how to add a new route to a route table called custom. The new route directs all traffic not routed by a more specific route to a gateway router at 192.168.1.100 via the eth1 network interface.

ip route add default via 192.168.1.100 dev eth1 table custom

If you translated this to English, it would read something like this,

"Add a new route to the table named custom that by default routes all traffic to a router at ipv4 address over device eth1."

You can verify the results of your handiwork with this command:

ip route show table custom

Even though your new routes are now created, they won't be honored by the server until the ip rule and route caches are flushed.[2]

ip route flush cache

All RPDB rules are loaded into the kernel’s memory when the server starts up. If you make changes to ip rules or ip routes and wish to utilize them prior to the next system reboot, you must flush their cache. This forces the kernel to reload the respective databases.

Main Routing Table Best Practices

Here are some brief concepts for you to keep in mind when adding new routing tables to your master routing table file.

1. Ensure table name and number are both unique!
2. Do not modify default values in the file
3. New table references should be numbered between 100 and 200
4. Remember the "default" table is actually named main

Adding New Routes to the Default (main) Routing Table (main)

To simply create a new route in the default routing table, you just need to insert a new route in the main table. This is accomplished via ip route.

First, take a look at the current state of your main routing table. Remember, your table labeled main is the actual default routing table.

ip route show

or

ip route show table main

or

route -n

If you ever see a route with a non-zero metric value, it is a priority value. Metric is an arbitrary 32-bit number that delineates route preferences. Smaller values denote higher priority. Zero is the highest priority metric. 65535 is the lowest.

Now, follow the instructions below to create new routes in the main table. To add a new route to the default main table, simply omit the table <table ID> portion of each command line.

Adding New Routes to Existing Routing Tables

Adding a new route to an existing table that is not the main table is a simple process. Routing tables have a very basic command structure. The possible routing types are described above in the routing type chart shown above under IP Route Syntax.

For the purpose of this discussion I'm going to focus only on the unicast route type. This narrows down the route configurations to the most common scenarios when the server is not a router. Thus, the command format to add a new route becomes:

ip route add {[destination ip/mask] [default]} {via [ip/mask]} {dev} [device] {table} [table ID] src [source ip]

Route Priority Processing

How does netfilter determine which route to choose from a routing table? What happens if a packet matches more than one route in the table?

Complex routing tables in large networks can contain hundreds of route entries. When the RPDB is ready to scan a route table and attempt to match a packet to one of the routes in that table, how does it decide which route to apply?

Recall that at this juncture, the RPDB has been directed to the current table by a rule. Routes are compared with the current packet based on the route's prefix and ToS value (if any). The prefix is a pair of values equal to the destination IP address and its netmask. The RPDB compares the current packet to each route in the table. The most specific match will be chosen, in this order:

1. Destination IP address of the packet matches the prefix, up to the length of the prefix
2. ToS bits of packet and route match
3. The route's ToS=0 (regardless of packet ToS value)

If more than one route matches the packet, the list is pruned in the following order until only a single route remains:

1. Longest matching (most specific) destination IP address including netmask
2. If a single route ToS and the packet ToS bits match, use that route
3. Routes where ToS=0
4. If no ToS=0 routes exist, fail on error (unreachable)
5. If multiple ToS=0 routes still exist, select route with best preference value
6. As a last resort, select the first route in chronological order

IP Route Command Examples

Here are some examples demonstrating how to structure the command line of common scenarios.

Point Destination to Gateway Router

These instructions direct the packet with indicated destination IP address to the indicated gateway router. Destination IP address may be single address or sub-net. CIDR (netmask) is optional.

ip route add [destination address/mask] via [gateway IP]

ip route add n.n.n.n/n.n.n.n via n.n.n.n/n.n.n.n

ip route add n.n.n.n/nn via n.n.n.n/nn

Set Default Route to Gateway Router

These instructions assign the default route to the indicated gateway router or next hop.

ip route add default via [gateway IP]

ip route add default via [next hop router IP]

ip route add default via n.n.n.n/n.n.n.n

ip route add default via n.n.n.n/nn

Point Destination to Next Hop

Destination IP address may be single address or sub-net; CIDR (netmask) optional. These instructions direct the packet with indicated destination IP address to the indicated router that is the next hop.

ip route add [destination address/mask] via [next hop IP]

ip route add n.n.n.n/n.n.n.n via n.n.n.n/n.n.n.n

ip route add n.n.n.n/nn via n.n.n.n/nn

Assign Route to an Interface

Add a new route tied to a specific interface.

ip route add [destination address/mask] dev [interface name]

ip route add default dev [interface name]

ip route add n.n.n.n dev [interface name]

ip route add n.n.n.n/nn dev [interface name]

ip route add n.n.n.n/n.n.n.n dev [interface name]

Set Default Route to Specific Interface

Change the device associated with the default route.

ip route add default dev [interface name]

Policy Routing (Routing Tables)

When applied to the RPDB policy routing model, the syntax is the same with the exception of the table specification. Simply append "table table ID" to the end of the command line. Table ID may be either the name or number of the table in the rt_tables file. Here are a few syntax examples:

ip route add default via [next hop] table [table ID]

ip route add default dev [interface name] table [table ID]

Set default route in table vpn as tun0 interface:

ip route add default dev tun0 table vpn

Set default route in table test to the gateway router (default) at IPv4 address 192.168.1.1:

ip route add default via 192.168.1.1 table test

Automatic Fallback/Down Detection and Redirection

This scenario involves two gateways and how to establish one of them as a priority path while using the other as a fallback. An example would be if your endpoint was attached to two independent internet service providers on the same network interface. How does new outgoing traffic get routed over one versus the other? There are multiple ways to decide. One method involves the use of UP and DOWN route commands in your server's network configuration.

Here is an example of setting up your router table using that method and branches network traffic to one of two new router tables if the destination port matches.

1. Create a new table in /etc/ip route2/rt_tables

   Name the new tables vpn1 and vpn2.

echo 1 vpn1 >> /etc/ip route2/rt_tables
echo 2 vpn2 >> /etc/ip route2/rt_tables

2. Create new routes that point to the new tables.

   For the table vpn1:

   ip route add default dev tun0 table vpn1

   For the table vpn2:

   ip route add default dev tun1 table vpn2

3. Create filter rules. Add rules to the mangle table (iptables) and the set-mark action to map specific port destinations to a particular routing table. The example below branches traffic for destination ports 22 or 80. Notice a different mark value is used depending on which table the traffic should be redirected to.

iptables -A PREROUTING -t mangle -i eth0 -p tcp --dport 22 -j MARK --set-mark 1
iptables -A PREROUTING -t mangle -i eth0 -p tcp --dport 80 -j MARK --set-mark 2

4. Add new ip rules to funnel marked traffic to the new route tables.

ip rule add from all fwmark 1 table vpn1
ip rule add from all fwmark 2 table vpn2

Deleting Routes

Deleting routes is the reverse of creating them. You must provide sufficiently specific information that the ip route command is able to isolate a single matching rule for deletion.

ip route delete {full route statement you wish to delete}

Don't forget to flush.

ip route flush cache

Creating Additional Routing Tables

New routing tables in the RPDB are created by simply adding a line to the file rt_tables
All that is required is an index value and table name. Each table must have a unique index value and unique name. Beyond that, the chronological order in the table doesn't matter. See The Master Routing Table for more information.

Adding a new route table is straightforward, but do you truly need to? If you're just adding a few new routes, it may make more sense to simply use the existing main table. Ask yourself: should I be adding a new routing table, or simply adding a new route? If you just want to add a new route, the main routing table can be utilized. However, if you are considering marking packets (fwmark) or creating a conditional branch such as a split VPN, a separate table can be very useful.

When you create a new routing table, you must also create a minimum of one corresponding rule that points to it, and you need to design the routes in the table to handle any possible outcomes. Three steps are required:

1. Add new entry to master routing table
2. Populate new routing table
3. Create at least one rule that points to the new table

Best Practices When Adding New Routing Tables

Here are some best practices to consider when creating new routing tables. Remember, you will be modifying the master routing table. If you decide to make your changes persistent, make sure they are working properly first or you may make your server inaccessible if something goes wrong.

1. Ensure the table name and number are both unique!
2. Do not modify default values in the file
3. New table references should be numbered between 100 and 200 (this allows you the ability to adjust route priorities in the future)
4. Use only lowercase characters for your table name
5. Do not use consecutive table numbers or bunch them together; leave some room between them to make future updates easier to implement

Sometimes, creating your own routing tables is a good idea. The main table is unique and has special features. Care needs to be taken not to disrupt it. An advantage to creating your own routing tables is you're much less likely to accidentally create a problem you can't easily fix. Regardless,
always test your routes thoroughly before making them persistent.

How To Create a New Routing Table

Adding a new routing table to a Linux server is a 3-step process.

Step 1. Manually edit the master routing table

You have two possible methods of completing this step.

1a. Method #1. Open the master routing table file for editing.

nano /etc/iproute2/rt_tables

Here's how your new table might look:

Remember, the order in which the indexes (table numbers) are presented in the master routing table doesn't matter; though the corresponding table numbers do. Route tables are processed in chronological order, beginning with number 0.

1b. Method #2. Use the command line to inject a new line at the end of the master routing table.

echo 100 custom >> /etc/iproute2/rt_tables

This command will place the new table entry at the end of the file with the name, "custom" and an index or table ID of 100. The order of the router table entries in the file is irrelevant. What is important is the routing table instructions are evaluated in order of the index of the routing tables. Table number zero (0) has the higest priority. Its routes are evaluated first, before all others.

Avoid duplicate table names or index numbers. Table numbers 0, 253, 254, and 255 are reserved. Do not use them.

Step 2. Populate your new table with one or more routes.

Now, we still have a problem, which is the fact the custom router table you just created is empty. Take a look at the syntax of your existing tables. Routing tables have a very basic command structure.

Here's an outline of how the routes are structured in a table (routing syntax):

[destination ip] | {via [ipv4]} | dev {device} | {proto [kernel or static]} | {scope [scope type]} | {src [ipv4]}

The formatting primarily boils down to the identification of the destination IP address or range, whether the route is direct or via a gateway, and the network device the packet will be sent out on. Recall these are outgoing routes only. Each route may contain other values depending on whether or not the route points to a gateway, host, or a group of IP addresses (e.g. a LAN branch). The preceding sections (especially Examining Existing Routes) have more detailed explanations on the syntax.

Here's an example of how to add a new route to your route table. The table will have just one route, which directs all traffic to a gateway router at 192.168.1.100 via the eth1 network interface.

ip route add default via 192.168.1.100 dev eth1 table custom

If you translated this to English, it would read something like this,

"Add a new route to the table named custom that routes all traffic to a gateway at ipv4 address 192.168.1.100 over device eth1."

You can verify the results of your handiwork with this command:

ip route show table custom

Step 3. Create at least one rule that points to your new table.

Routing rules are stored in the RPDB and created via the ip rule command.

This is arguably the most important part of this process. Even though this particular discussion is regarding routes, rules have a great impact on which routes a packet is capable of traversing. And therefore, you need to be cognizant of how new rules you create will behave in the context of other routing rules and other routing tables in the RPDB. Remember, the kernel won't pay any attention to your fancy new route table and its routes unless you have rules instructing it to do so.

Details of routing rules (ip rules) are discussed in their own section, but for the moment you should be clear on the most important factors when creating a new routing rule:

1. Rules are processed in priority order with 0 (zero) as the highest priority
2. Longest matching rule wins a tie
3. If more than one rule is matching and lengths are identical, the higher priority rule wins

Endnotes