2018年8月

If you are debugging IPTables, it is handy to be able to trace the packets while it traverses the various chains. I was trying to find out why port forwarding from the external NIC to a virtual machine attached to a virtual bridge device was not working.

You need to perform the following preparations:

Load the (IPv4) netfilter log kernel module:

# modprobe nf_log_ipv4

Enable logging for the IPv4 (AF Family 2):

# sysctl net.netfilter.nf_log.2=nf_log_ipv4

reconfigure rsyslogd to log kernel messages (kern.*) to /var/log/messages:

# cat /etc/rsyslog.conf | grep -e "^kern"
kern.*;*.info;mail.none;authpriv.none;cron.none                /var/log/messages

restart rsyslogd:

# systemctl restart rsyslog

Now check the raw tables – you’ll see that there are already entries coming from firewalld:

# iptables -t raw -L
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
PREROUTING_direct all -- anywhere anywhere

Chain OUTPUT (policy ACCEPT)
target prot opt source destination
OUTPUT_direct all -- anywhere anywhere

Chain OUTPUT_direct (1 references)
target prot opt source destination

Chain PREROUTING_direct (1 references)
target prot opt source destination

We’ll want to add our tracing rules before the existing rules. In this example we’ll trace everything related to HTTP (port 80)

# iptables -t raw -j TRACE -p tcp --dport 80 -I PREROUTING 1
# iptables -t raw -j TRACE -p tcp --dport 80 -I OUTPUT 1

The rules now look as follows:

# iptables -t raw -L
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
TRACE tcp -- anywhere anywhere tcp dpt:http
PREROUTING_direct all -- anywhere anywhere

Chain OUTPUT (policy ACCEPT)
target prot opt source destination
TRACE tcp -- anywhere anywhere tcp dpt:http
OUTPUT_direct all -- anywhere anywhere

Chain OUTPUT_direct (1 references)
target prot opt source destination

Chain PREROUTING_direct (1 references)
target prot opt source destination

Now access to that specific machine’s TCP port 80 are logged to /var/log/messages:

# tail /var/log/messages
May 27 19:57:54 storm3 kernel: TRACE: mangle:PRE_public:rule:3 IN=em1 OUT= MAC=ec:f4:bb:f1:4e:f0:00:25:46:70:2e:41:08:00 SRC=10.36.7.11 DST=10.32.105.30 LEN=64 TOS=0x00 PREC=0x00 TTL=59 ID=10953 DF PROTO=TCP SPT=54451 DPT=80 SEQ=1779626624 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (020404D8010303050101080A124E9DB70000000004020000)
May 27 19:57:54 storm3 kernel: TRACE: mangle:PRE_public_allow:return:1 IN=em1 OUT= MAC=ec:f4:bb:f1:4e:f0:00:25:46:70:2e:41:08:00 SRC=10.36.7.11 DST=10.32.105.30 LEN=64 TOS=0x00 PREC=0x00 TTL=59 ID=10953 DF PROTO=TCP SPT=54451 DPT=80 SEQ=1779626624 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (020404D8010303050101080A124E9DB70000000004020000)
May 27 19:57:54 storm3 kernel: TRACE: mangle:PRE_public:return:4 IN=em1 OUT= MAC=ec:f4:bb:f1:4e:f0:00:25:46:70:2e:41:08:00 SRC=10.36.7.11 DST=10.32.105.30 LEN=64 TOS=0x00 PREC=0x00 TTL=59 ID=10953 DF PROTO=TCP SPT=54451 DPT=80 SEQ=1779626624 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (020404D8010303050101080A124E9DB70000000004020000)
May 27 19:57:54 storm3 kernel: TRACE: mangle:PREROUTING:policy:4 IN=em1 OUT= MAC=ec:f4:bb:f1:4e:f0:00:25:46:70:2e:41:08:00 SRC=10.36.7.11 DST=10.32.105.30 LEN=64 TOS=0x00 PREC=0x00 TTL=59 ID=10953 DF PROTO=TCP SPT=54451 DPT=80 SEQ=1779626624 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (020404D8010303050101080A124E9DB70000000004020000)
May 27 19:57:54 storm3 kernel: TRACE: nat:PREROUTING:rule:1 IN=em1 OUT= MAC=ec:f4:bb:f1:4e:f0:00:25:46:70:2e:41:08:00 SRC=10.36.7.11 DST=10.32.105.30 LEN=64 TOS=0x00 PREC=0x00 TTL=59 ID=10953 DF PROTO=TCP SPT=54451 DPT=80 SEQ=1779626624 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (020404D8010303050101080A124E9DB70000000004020000)
May 27 19:57:54 storm3 kernel: TRACE: nat:PREROUTING_direct:rule:1 IN=em1 OUT= MAC=ec:f4:bb:f1:4e:f0:00:25:46:70:2e:41:08:00 SRC=10.36.7.11 DST=10.32.105.30 LEN=64 TOS=0x00 PREC=0x00 TTL=59 ID=10953 DF PROTO=TCP SPT=54451 DPT=80 SEQ=1779626624 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (020404D8010303050101080A124E9DB70000000004020000)
May 27 19:57:54 storm3 kernel: TRACE: mangle:FORWARD:rule:1 IN=em1 OUT=virbr1 MAC=ec:f4:bb:f1:4e:f0:00:25:46:70:2e:41:08:00 SRC=10.36.7.11 DST=192.168.101.10 LEN=64 TOS=0x00 PREC=0x00 TTL=58 ID=10953 DF PROTO=TCP SPT=54451 DPT=80 SEQ=1779626624 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (020404D8010303050101080A124E9DB70000000004020000)
May 27 19:57:54 storm3 kernel: TRACE: mangle:FORWARD_direct:return:1 IN=em1 OUT=virbr1 MAC=ec:f4:bb:f1:4e:f0:00:25:46:70:2e:41:08:00 SRC=10.36.7.11 DST=192.168.101.10 LEN=64 TOS=0x00 PREC=0x00 TTL=58 ID=10953 DF PROTO=TCP SPT=54451 DPT=80 SEQ=1779626624 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (020404D8010303050101080A124E9DB70000000004020000)
May 27 19:57:54 storm3 kernel: TRACE: mangle:FORWARD:policy:2 IN=em1 OUT=virbr1 MAC=ec:f4:bb:f1:4e:f0:00:25:46:70:2e:41:08:00 SRC=10.36.7.11 DST=192.168.101.10 LEN=64 TOS=0x00 PREC=0x00 TTL=58 ID=10953 DF PROTO=TCP SPT=54451 DPT=80 SEQ=1779626624 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (020404D8010303050101080A124E9DB70000000004020000)
May 27 19:57:54 storm3 kernel: TRACE: filter:FORWARD:rule:7 IN=em1 OUT=virbr1 MAC=ec:f4:bb:f1:4e:f0:00:25:46:70:2e:41:08:00 SRC=10.36.7.11 DST=192.168.101.10 LEN=64 TOS=0x00 PREC=0x00 TTL=58 ID=10953 DF PROTO=TCP SPT=54451 DPT=80 SEQ=1779626624 ACK=0 WINDOW=65535 RES=0x00 SYN URGP=0 OPT (020404D8010303050101080A124E9DB70000000004020000)

周杰伦 - 安静

词/曲:周杰伦

只剩下钢琴陪我

弹了一天

睡着的大提琴

安静的旧旧的

我想你已表现的

非常明白

我懂我也知道

你没有舍不得

你说你也会难过

我不相信

牵着你陪着我

也只是曾经

希望他是真的

比我还要爱你

我才会逼自己离开

你要我说多难堪

我根本不想分开

为什么还要我用

微笑来带过

我没有这种天份

包容你也接受他

不用担心的太多

我会一直好好过

你已经远远离开

我也会慢慢走开

为什么我连分开

都迁就着你

我真的没有天份

安静的没这么快

我会学着放弃你

是因为我太爱你

只剩下钢琴陪我

弹了一天

睡着的大提琴

安静的旧旧的

我想你已表现的

非常明白

我懂我也知道

你没有舍不得

你说你也会难过

我不相信

牵着你陪着我

也只是曾经

希望他是真的

比我还要爱你

我才会逼自己离开

你要我说多难堪

我根本不想分开

为什么还要我用

微笑来带过

我没有这种天份

包容你也接受他

不用担心的太多

我会一直好好过

你已经远远离开

我也会慢慢走开

为什么我连分开

都迁就着你

我真的没有天份

安静的没这么快

我会学着放弃你

是因为我太爱你

你要我说多难堪

我根本不想分开

为什么还要我用

微笑来带过

我没有这种天份

包容你也接受他

不用担心的太多

我会一直好好过

你已经远远离开

我也会慢慢走开

为什么我连分开

都迁就着你

我真的没有天份

安静的没这么快

我会学着放弃你

是因为我太爱你