Bug: Interleaved iptables filter rules in compute pod

[cloneable]

Environment

compute pod

Steps to reproduce

iptables -L -vn

Actual result

I noticed a strange ordering in compute pods iptables filter rules:

Compute A:

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
num   pkts bytes target     prot opt in     out     source               destination
1       60  4800 ACCEPT     17   --  *      *       0.0.0.0/0            172.20.0.10          udp dpt:53
2        0     0 ACCEPT     6    --  *      *       0.0.0.0/0            172.20.0.10          tcp dpt:53
3        2   738 ACCEPT     17   --  *      *       0.0.0.0/0            0.0.0.0/0            udp spts:67:68 dpts:67:68
4     3836  334K ACCEPT     6    --  *      *       0.0.0.0/0            169.254.254.252/30   tcp dpt:22
5        0     0 ACCEPT     6    --  *      *       0.0.0.0/0            169.254.254.252/30   tcp dpt:25183
6        0     0 ACCEPT     1    --  *      *       0.0.0.0/0            169.254.254.252/30
7        0     0 ACCEPT     1    --  *      *       0.0.0.0/0            10.100.0.0/16
8        0     0 REJECT     0    --  *      *       0.0.0.0/0            169.254.254.252/30   reject-with icmp-port-unreachable
9        0     0 REJECT     0    --  *      *       0.0.0.0/0            10.100.0.0/16        reject-with icmp-port-unreachable
10   69210 8217K ACCEPT     6    --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate ESTABLISHED
11       0     0 ACCEPT     6    --  *      *       0.0.0.0/0            10.0.0.0/8           tcp dpt:20187
12       0     0 ACCEPT     6    --  *      *       0.0.0.0/0            172.16.0.0/12        tcp dpt:20187
13       2   120 ACCEPT     6    --  *      *       0.0.0.0/0            0.0.0.0/0            tcp dpt:443 /* apk commands */
14      34  1540 REJECT     6    --  *      *       0.0.0.0/0            0.0.0.0/0            reject-with tcp-reset
15       2    80 REJECT     0    --  *      *       0.0.0.0/0            0.0.0.0/0            reject-with icmp-port-unreachable
16       0     0 ACCEPT     6    --  *      *       127.0.0.1            169.254.254.254      tcp dpt:5432
17       0     0 ACCEPT     6    --  *      *       127.0.0.1            169.254.254.254      tcp dpt:3080
18       0     0 ACCEPT     6    --  *      *       127.0.0.1            169.254.254.254      tcp dpt:6432
19       0     0 ACCEPT     6    --  *      *       127.0.0.1            169.254.254.254      tcp dpt:9100
20       0     0 ACCEPT     6    --  *      *       127.0.0.1            169.254.254.254      tcp dpt:9127
21       0     0 ACCEPT     6    --  *      *       127.0.0.1            169.254.254.254      tcp dpt:9187
22       0     0 ACCEPT     6    --  *      *       127.0.0.1            169.254.254.254      tcp dpt:9399
23       0     0 ACCEPT     6    --  *      *       127.0.0.1            169.254.254.254      tcp dpt:9499
24       0     0 ACCEPT     6    --  *      *       127.0.0.1            169.254.254.254      tcp dpt:10301
25       0     0 ACCEPT     6    --  *      *       127.0.0.1            169.254.254.254      tcp dpt:10432

(ignore rule 13, i injected that one)

Compute B:

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     17   --  *      *       0.0.0.0/0            172.20.0.10          udp dpt:53
    0     0 ACCEPT     6    --  *      *       0.0.0.0/0            172.20.0.10          tcp dpt:53
    0     0 ACCEPT     6    --  *      *       127.0.0.1            169.254.254.254      tcp dpt:5432
    2   738 ACCEPT     17   --  *      *       0.0.0.0/0            0.0.0.0/0            udp spts:67:68 dpts:67:68
    0     0 ACCEPT     6    --  *      *       127.0.0.1            169.254.254.254      tcp dpt:3080
    0     0 ACCEPT     6    --  *      *       0.0.0.0/0            169.254.254.252/30   tcp dpt:22
    0     0 ACCEPT     6    --  *      *       0.0.0.0/0            169.254.254.252/30   tcp dpt:25183
    0     0 ACCEPT     6    --  *      *       127.0.0.1            169.254.254.254      tcp dpt:6432
    0     0 ACCEPT     1    --  *      *       0.0.0.0/0            169.254.254.252/30
    0     0 ACCEPT     6    --  *      *       127.0.0.1            169.254.254.254      tcp dpt:9100
    0     0 ACCEPT     1    --  *      *       0.0.0.0/0            10.100.0.0/16
    0     0 REJECT     0    --  *      *       0.0.0.0/0            169.254.254.252/30   reject-with icmp-port-unreachable
    0     0 ACCEPT     6    --  *      *       127.0.0.1            169.254.254.254      tcp dpt:9127
    0     0 REJECT     0    --  *      *       0.0.0.0/0            10.100.0.0/16        reject-with icmp-port-unreachable
    0     0 ACCEPT     6    --  *      *       127.0.0.1            169.254.254.254      tcp dpt:9187
31298 3732K ACCEPT     6    --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate ESTABLISHED
    0     0 ACCEPT     6    --  *      *       0.0.0.0/0            10.0.0.0/8           tcp dpt:20187
    0     0 ACCEPT     6    --  *      *       0.0.0.0/0            172.16.0.0/12        tcp dpt:20187
    0     0 ACCEPT     6    --  *      *       127.0.0.1            169.254.254.254      tcp dpt:9399
    0     0 REJECT     6    --  *      *       0.0.0.0/0            0.0.0.0/0            reject-with tcp-reset
    1    40 REJECT     0    --  *      *       0.0.0.0/0            0.0.0.0/0            reject-with icmp-port-unreachable
    0     0 ACCEPT     6    --  *      *       127.0.0.1            169.254.254.254      tcp dpt:9499
    0     0 ACCEPT     6    --  *      *       127.0.0.1            169.254.254.254      tcp dpt:10301
    0     0 ACCEPT     6    --  *      *       127.0.0.1            169.254.254.254      tcp dpt:10432

@sharnoff pointed out that this is likely due to init scripts and qemu setup running in parallel.

Slack discussion: https://neondb.slack.com/archives/C03TN5G758R/p1741261832679489

[cloneable]

Also, I wonder if this pairing in the rules ever matches anything: 127.0.0.1 169.254.254.254. Should not be routed, right?