Owen's: [Network] Set Ubuntu as switch with port mirroring feature

最近因為需要錄一些封包做分析(架構如下圖),可是手上只有便宜的router和switch,還有一台廢棄的PC,想一想這樣的功能應該對Linux來說只是一片小蛋糕,爬文後發現只要靠netfilter就可以實現這樣的需求...

工作環境:Ubuntu 10.04.1 LTS (Linux NAT Server)

準備工作

$ sudo apt-get install fakeroot build-essential kernel-package libncurses5 libncurses5-dev initramfs-tools

下載Ubuntu kernel source和iptables source和netfilter patch

$ mkdir ~/src
$ cd src
$ sudo apt-get source linux-image-$(uname -r)
$ sudo apt-get source iptables
$ wget http://ftp.netfilter.org/pub/patch-o-matic-ng/snapshot/patch-o-matic-ng-20091205.tar.bz2

下載iptables的patch

$ cd /path/to/patch-o-matic-ng
$ ./runme --download

Successfully downloaded external patch geoip
http://www.nucleus.it/pom-repo: bad patch name <?xml version="1.0" encoding="ISO-8859-1"?>, ignored
http://www.nucleus.it/pom-repo: bad patch name <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN", ignored
http://www.nucleus.it/pom-repo: bad patch name   "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">, ignored
http://www.nucleus.it/pom-repo: bad patch name <html xmlns="http://www.w3.org/1999/xhtml" lang="en" xml:lang="en">, ignored
http://www.nucleus.it/pom-repo: bad patch name <head>, ignored
http://www.nucleus.it/pom-repo: bad patch name <title>Object not found!</title>, ignored
http://www.nucleus.it/pom-repo: bad patch name <link rev="made" href="mailto:webmaster@start2000.net" />, ignored
http://www.nucleus.it/pom-repo: bad patch name <style type="text/css"><!--/*--><![CDATA[/*><!--*/ , ignored
http://www.nucleus.it/pom-repo: bad patch name     body { color: #000000; background-color: #FFFFFF; }, ignored
http://www.nucleus.it/pom-repo: bad patch name     a:link { color: #0000CC; }, ignored
http://www.nucleus.it/pom-repo: bad patch name     p, address {margin-left: 3em;}, ignored
http://www.nucleus.it/pom-repo: bad patch name     span {font-size: smaller;}, ignored
http://www.nucleus.it/pom-repo: bad patch name /*]]>*/--></style>, ignored
http://www.nucleus.it/pom-repo: bad patch name </head>, ignored
http://www.nucleus.it/pom-repo: bad patch name <body>, ignored
http://www.nucleus.it/pom-repo: bad patch name <h1>Object not found!</h1>, ignored
http://www.nucleus.it/pom-repo: bad patch name <p>, ignored
http://www.nucleus.it/pom-repo: bad patch name     The requested URL was not found on this server., ignored
http://www.nucleus.it/pom-repo: bad patch name     If you entered the URL manually please check your, ignored
http://www.nucleus.it/pom-repo: bad patch name     spelling and try again., ignored
http://www.nucleus.it/pom-repo: bad patch name </p>, ignored
http://www.nucleus.it/pom-repo: bad patch name <p>, ignored
http://www.nucleus.it/pom-repo: bad patch name If you think this is a server error, please contact, ignored
http://www.nucleus.it/pom-repo: bad patch name the <a href="mailto:webmaster@start2000.net">webmaster</a>., ignored
http://www.nucleus.it/pom-repo: bad patch name </p>, ignored
http://www.nucleus.it/pom-repo: bad patch name <h2>Error 404</h2>, ignored
http://www.nucleus.it/pom-repo: bad patch name <address>, ignored
http://www.nucleus.it/pom-repo: bad patch name   <a href="/">www.nucleus.it</a><br />, ignored
http://www.nucleus.it/pom-repo: bad patch name   <span>Fri Apr 22 12:55:03 2011<br />, ignored
http://www.nucleus.it/pom-repo: bad patch name   Apache/2.2.10 (Linux/SUSE)</span>, ignored
http://www.nucleus.it/pom-repo: bad patch name </address>, ignored
http://www.nucleus.it/pom-repo: bad patch name </body>, ignored
http://www.nucleus.it/pom-repo: bad patch name </html>, ignored
Successfully downloaded external patch IPMARK
Successfully downloaded external patch ROUTE
Successfully downloaded external patch connlimit
Successfully downloaded external patch ipp2p
Successfully downloaded external patch time
Successfully downloaded external patch ipv4options
Successfully downloaded external patch TARPIT
Successfully downloaded external patch ACCOUNT
Failed to get http://svn.berlios.de/svnroot/repos/portknocko/trunk/pom//index, skipping..
Hey! KERNEL_DIR is not set.
Where is your kernel source directory? [/usr/src/linux] /path/to/linux-2.6.32
Hey! IPTABLES_DIR is not set.
Where is your iptables source code directory? [/usr/src/iptables] /path/to/iptables-1.4.4
iptables-1.4.4 doesn't look like a iptables source code directory to me.

在iptables-1.4.x以後,必須先configure後才能正確patch

$ cd /path/to/iptables
$ ./configure
$ cd /path/to/patch-o-matic-ng
$ ./runme --download

...
Loading patchlet definitions......... done


Excellent! Source trees are ready for compilation.

執行iptables的ROUTE patch

./runme ROUTE

Testing ROUTE... not applied
The ROUTE patch:
   Author: C�dric de Launois 
   Status: Experimental

  
  This option adds a `ROUTE' target, which enables you to setup unusual
  routes. For example, the ROUTE lets you route a received packet through 
  an interface or towards a host, even if the regular destination of the 
  packet is the router itself. The ROUTE target is also able to change the 
  incoming interface of a packet.

  The target can be or not a final target. It has to be used inside the 
  mangle table.

  ROUTE target options:
  --oif   ifname    Send the packet out using `ifname' network interface.
  --iif   ifname    Change the packet's incoming interface to `ifname'.
  --gw    ip        Route the packet via this gateway.
  --continue        Route the packet and continue traversing the rules.
  --tee             Route a copy of the packet, but continue traversing
                    the rules with the original packet, undisturbed.

  Note that --iif, --continue, and --tee, are mutually exclusive.

  Examples :

  # To force all outgoing icmp packet to go through the eth1 interface 
  # (final target) :
  iptables -A POSTROUTING -t mangle -p icmp -j ROUTE --oif eth1
 
  # To tunnel outgoing http packets and continue traversing the rules :
  iptables -A POSTROUTING -t mangle -p tcp --dport 80 -j ROUTE --oif tunl1 --continue
 
  # To forward all ssh packets to gateway w.x.y.z, and continue traversing
  # the rules :
  iptables -A POSTROUTING -t mangle -p tcp --dport 22 -j ROUTE --gw w.x.y.z --continue
 
  # To change the incoming network interface from eth0 to eth1 for all icmp
  # packets (final target) :
  iptables -A PREROUTING -t mangle -p icmp -i eth0 -j ROUTE --iif eth1

  # To copy (duplicate) all traffic from and to a local ECHO server
  # to a second box (nonfinal target)
  iptables -A PREROUTING -t mangle -p tcp --dport 7 -j ROUTE --gw 1.2.3.4 --tee
  iptables -A POSTROUTING -t mangle -p tcp --sport 7 -j ROUTE --gw 1.2.3.4 --tee

-----------------------------------------------------------------
Do you want to apply this patch [N/y/t/f/a/r/b/w/q/?] y

config kernel,將ROUTE target support選項M起來

$ cd /path/to/kernel
$ cp /boot/config-`uname -r` .config
$ make menuconfig

-*- Networking support  ---> 
  Networking options  --->
    [*] Network packet filtering framework (Netfilter)  ---> 
      IP: Netfilter Configuration  --->  
        <M> ROUTE target support

重新編譯kernel

$ make-kpkg clean
$ fakeroot make-kpkg --initrd --append-to-version=-<some-string-here> kernel-image kernel-headers

漫長的等待後,會編出kernel-image-xxx.deb和kernel-headers-xxx.deb
安裝新的kernel-image和kernel-headers

$ cd ..
$ sudo dpkg -i linux-image-2.6.32.32+drm33.13-<some-string-here>_2.6.32.32+drm33.13-<some-string-here>-10.00.Custom_i386.deb

Selecting previously deselected package linux-image-2.6.32.32+drm33.14-route.
(Reading database ... 128393 files and directories currently installed.)
Unpacking linux-image-2.6.32.32+drm33.14-route (from linux-image-2.6.32.32+drm33.14-route_2.6.32.32+drm33.14-route-10.00.Custom_i386.deb) ...
Done.
Setting up linux-image-2.6.32.32+drm33.14-route (2.6.32.32+drm33.14-route-10.00.Custom) ...
Running depmod.
Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/nvidia-common 2.6.32.32+drm33.14-route /boot/vmlinuz-2.6.32.32+drm33.14-route
run-parts: executing /etc/kernel/postinst.d/pm-utils 2.6.32.32+drm33.14-route /boot/vmlinuz-2.6.32.32+drm33.14-route
Running postinst hook script update-grub.
Generating grub.cfg ...
Found linux image: /boot/vmlinuz-2.6.32.32+drm33.14-route
Found linux image: /boot/vmlinuz-2.6.32-24-generic
Found initrd image: /boot/initrd.img-2.6.32-24-generic
Found memtest86+ image: /boot/memtest86+.bin
done

$ sudo dpkg -i linux-headers-2.6.32.32+drm33.13-<some-string-here>_2.6.32.32+drm33.13-<some-string-here>-10.00.Custom_i386.deb

Selecting previously deselected package linux-headers-2.6.32.32+drm33.14-route.
(Reading database ... 131949 files and directories currently installed.)
Unpacking linux-headers-2.6.32.32+drm33.14-route (from linux-headers-2.6.32.32+drm33.14-route_2.6.32.32+drm33.14-route-10.00.Custom_i386.deb) ...
Setting up linux-headers-2.6.32.32+drm33.14-route (2.6.32.32+drm33.14-route-10.00.Custom) ...
Examining /etc/kernel/header_postinst.d.
run-parts: executing /etc/kernel/header_postinst.d/nvidia-common 2.6.32.32+drm33.14-route /boot/vmlinuz-2.6.32.32+drm33.14-route

建立initramfs image

$ sudo update-initramfs -c -k 2.6.32.32+drm33.14-route

update-initramfs: Generating /boot/initrd.img-2.6.32.32+drm33.14-route

更新GRUB

$ sudo update-grub

Ubuntu 9.10以後採用GRUB2,因此也可以靠修改/boot/grub/grub.cfg指到對應的kernel

重開機後看是否是已經以正確的kernel開機

$ uname -r

2.6.32-32-route

掛載ipt_ROUTE和x_tables核心模組

$ sudo insmod ipt_ROUTE

insmod: error inserting 'ipt_ROUTE.ko': -1 Unknown symbol in module

$ dmesg|tail -n 6

[  154.118451] ipt_ROUTE: Unknown symbol xt_register_target
[  154.119117] ipt_ROUTE: Unknown symbol xt_unregister_target
[  166.920710] ipt_ROUTE: Unknown symbol xt_register_target
[  166.921153] ipt_ROUTE: Unknown symbol xt_unregister_target
[  243.767101] ipt_ROUTE: Unknown symbol xt_register_target
[  243.767516] ipt_ROUTE: Unknown symbol xt_unregister_target

由於核心模組的載入有順序性,需要先insmod x_tables.ko再insmod ipt_ROUTE.ko,也可用modprobe解決相依性

$ sudo modprobe ipt_ROUTE
$ lsmod|grep -in route

ipt_ROUTE               2835  0 
x_tables               14299  1 ipt_ROUTE

安裝patch過的iptables

$ cd /path/to/iptables
$ ./configure --prefix=/path/to/install
$ make

...
libipt_ROUTE.c:17:44: warning: linux/netfilter_ipv4/ipt_ROUTE.h: No such file or directory
libipt_ROUTE.c: In function 『init':
libipt_ROUTE.c:73: error: dereferencing pointer to incomplete type
libipt_ROUTE.c:74: error: dereferencing pointer to incomplete type
libipt_ROUTE.c:75: error: dereferencing pointer to incomplete type
libipt_ROUTE.c:76: error: dereferencing pointer to incomplete type
...

由於libipt_ROUTE和libip6t_ROUTE需要對應的核心標頭檔,所以需在config時引入核心原始檔的位置

$ ./configure --prefix=/path/to/install --with-kernel=/path/to/kernel
$ make install

執行patch過的iptables

$ cd /path/to/iptables
$ sudo ./sbin/iptables -m ROUTE

/path/to/iptables/libexec/xtables/libipt_ROUTE.so: /path/to/iptables/libexec/xtables/libipt_ROUTE.so: undefined symbol: exit_error
iptables v1.4.4: Couldn't load match `ROUTE':(null)

Try `iptables -h' or 'iptables --help' for more information.

由於iptables 1.4.x以後exit_error()改成xtables_error(),因此需修改相關的函式

$ vim -p extensions/libipt_ROUTE.c extensions/libip6t_ROUTE.c
:1,$s/exit_error/xtables_error/g

再執行patch過的iptables

/usr/src/iptables/libexec/xtables/libipt_ROUTE.so: /usr/src/iptables/libexec/xtables/libipt_ROUTE.so: undefined symbol: check_inverse
iptables v1.4.4: Couldn't load match `ROUTE':(null)

Try `iptables -h' or 'iptables --help' for more information.

將check_inverse()改成xtables_check_inverse()

$ vim -p extensions/libipt_ROUTE.c extensions/libip6t_ROUTE.c
:1,$s/check_inverse/xtables_check_inverse/g

再執行patch過的iptables,確定沒有錯誤訊息了...

$ sudo ./sbin/iptables -m ROUTE

iptables v1.4.4: option `-m' requires an argument
Try `iptables -h' or 'iptables --help' for more information.

但是要能讓封包能forward還需要打開核心ip_forward功能,另外還需要讓eth0和eth1互通

$ vim set_nat_env.sh

#!/bin/bash
# set ip_forward
echo 1 > /proc/sys/net/ipv4/ip_forward

# set nat table
# $IPTABLES是iptables安裝的路徑
# $INNET是對內的網段
# $EXTIF是對外介面,本例為eth1
export IPTABLES=/path/to/iptables
export INNET="192.168.1.0/24"
export EXTIF="eth1"
$IPTABLES/sbin/iptables -t nat -A POSTROUTING -s $INNET -o $EXTIF -j MASQUERADE

最後終於可建立iptables規則了... 只要建立這兩條規則就可以完成我們的需求

$ sudo ./sbin/iptables -A PREROUTING -t mangle -j ROUTE --gw 192.168.1.50 --tee
$ sudo ./sbin/iptables -A POSTROUTING -t mangle -j ROUTE --gw 192.168.1.50 --tee

但要注意的是,建立的規則用iptables -L是看不到的,但是他依舊存在,可以用iptables-save導出規則來確認,而且該規則用iptables -F也是刪除不掉的,需要重新啟動iptables服務才會消失

reference:
* 請問 linux 如何做到 port mirror (已解決)
* How to compile a kernel on Ubuntu 10.04
* 修改Grub2開機選單的啟動順序[9.1,10.04 Or Newer]
* grub 和 menu.lst [論壇 - Ubuntu基本設定]
* [工作]安裝iptables extension
* Need help installing patch-o-matic-ng
* 請教Linux L7-filter無法作用的問題
* Subject: [PATCH 14/16] libxtables: prefix/order - move check_inverse to xtables.c - msg#00065
* 9.5 NAT 伺服器的設定

Owen's

2011年4月22日星期五

[Network] Set Ubuntu as switch with port mirroring feature

沒有留言:

關於我自己

Taiwan Open Source

陽光心靈診所

網誌存檔

標籤

##EasyReadMore##