When you add a secondary IP address that belongs to a different subnet than your primary IP, it usually comes with its own Gateway.
Linux, by default, supports only one Default Gateway. This leads to Asymmetric Routing issues: traffic enters via the secondary IP but tries to leave via the primary gateway, causing connections to fail.
To solve this on RHEL-based systems (AlmaLinux, Rocky, CentOS), we use Policy Based Routing via NetworkManager.
This guide is for systems using NetworkManager
For Ubuntu/Debian (Netplan), refer to the standard networking documentation.
Step 1: Taming Cloud-init (Crucial)
Before configuring anything, we must ensure Cloud-init does not overwrite our manual NetworkManager changes upon the next reboot.
- Log in as
rootvia SSH. - Run the following command to create a lock file:
echo "network: {config: disabled}" > /etc/cloud/cloud.cfg.d/99-disable-network-config.cfgThis tells Cloud-init to stop managing networking, while keeping other functions (like SSH key injection) active.
Step 2: Gather Information
First, find the name of your active connection.
nmcli connection show(Common names: System eth0, eth0, ens3, or ens18).
Have the following details ready:
- Connection Name: (e.g., "System eth0")
- Secondary IP: (e.g., 192.168.102.10)
- Secondary Gateway: (e.g., 192.168.102.1)
Even if you ordered a single IP, usually you must configure it with the subnet mask of the gateway (often /24).
Do not use /32, or the gateway will be unreachable.
Step 3: Configure Routing Rules
We will execute three commands to configure the interface. Replace the values in brackets with your data.
1. Add the Secondary IP:
# Syntax: nmcli con mod "NAME" +ipv4.addresses IP/MASK
nmcli con mod "System eth0" +ipv4.addresses 192.168.102.10/242. Create a Routing Policy Rule:
This tells Linux: "If traffic originates from 84.22.105.9, look at Routing Table 200".
# Syntax: ... "priority 100 from IP table 200"
nmcli con mod "System eth0" +ipv4.routing-rules "priority 100 from 192.168.102.10 table 200"3. Add the Route to Table 200:
This tells Linux: "In Table 200, the default gateway is 84.22.105.1".
# Syntax: ... "0.0.0.0/0 GATEWAY table=200"
nmcli con mod "System eth0" +ipv4.routes "0.0.0.0/0 192.168.102.1 table=200"Step 4: Apply Changes
Reload the connection to activate the new configuration. Warning: If you made a typo, you might lose connectivity.
nmcli con up "System eth0"Step 5: Verification
Check if the separate routing table works correctly.
1. Check the table:
ip route show table 200Output should show: default via 192.168.102.1 dev eth0...
2. Test outgoing traffic:
Force a ping specifically from your new IP address.
ping -I 192.168.102.10 -c 2 8.8.8.8If the ping succeeds, your Policy Based Routing is active and reboot-proof.