It's not possible to put an ordinary switch in the middle, as the ethernet frames would not get through. What is needed is an ethernet tap or port mirroring. An old-fashioned ethernet hub would work here, mirroring the traffic from either end of the original wire to a third port that could be monitored e.g. by Wireshark. But can this done be without a special device, with plain linux?
It turns out port mirroring is quite doable provided you have a device with at least 2 NICs (if you want to capture the frames on the device that does the port mirroring) or at least 3 NICs (if you want a box resembling the tap, with the actual frame capture occurring on another box that is connected to the tap).
Googling around doesn't find many useful links but there are some. The best I found being a detailed explanation at \1. So you can just head there and read it. What follows is an practical application of that to building an ethernet tap.
It's possible to test this in a simple virtual environment with some help from network namespaces. Let's build a virtual network with the following topology:
The tap node is mirroring all incoming frames from either veth_a_tap or veth_b_tap to the veth_tap_tap interface and to the other interface than the incoming one. The network monitoring software is then used on veth_tap, which receives all the frames that either A or B sent.
The nodes are emulated by network namespaces, so there are namespaces eth_a and eth_b for the nodes A and B respectively. Then there is the namespace tap for the ethernet tap. The interface veth_tap is left in the default namespace to make it easily accessible.
First create the network namespaces (all these commands need to be run as root):
ip netns add eth_a ip netns add eth_b ip netns add eth_tap
Then add the virtual ethernet devices (for now they are in the default namespace):
ip link add veth_a type veth peer name veth_a_tap ip link add veth_b type veth peer name veth_b_tap ip link add veth_tap type veth peer name veth_tap_tap
The virtual ethernet devices get a random MAC address. To make our life easier in subsequent experimenting with the frames, we can set some easy to spot MAC addresses:
ip link set dev veth_a addr 88:00:00:00:00:00 ip link set dev veth_b addr 88:00:00:00:00:01
Now move the virtual ethernet interfaces into the appropriate namespaces:
ip link set veth_a netns eth_a ip link set veth_b netns eth_b ip link set veth_a_tap netns eth_tap ip link set veth_b_tap netns eth_tap ip link set veth_tap_tap netns eth_tap
There is one more thing: The interfaces are DOWN, so we need to set them UP. As it turns out this has to be done after the interfaces are moved to their namespaces as moving them sets them DOWN. Use ip netns exec to run the commands in the appropriate namespace:
ip netns exec eth_a ip link set dev veth_a up ip netns exec eth_b ip link set dev veth_b up ip netns exec eth_tap ip link set dev veth_a_tap up ip netns exec eth_tap ip link set dev veth_b_tap up ip netns exec eth_tap ip link set dev veth_tap_tap up
And finally:
ip link set dev veth_tap up(no ip netns exec needed as veth_tap is in the default namespace)
Now configure the tap node to do the actual port mirroring. After reading the article on \1 I chose to use the tc way. This approach is easy to set up and needs neither special software nor special kernel configuration. It's easy to set up even on an embedded device.
If you run a distro kernel you should have all that's needed already enabled in the kernel. If you build your own, you need to enable a bunch of options in the QoS and/or fair queueing menu: at least CONFIG_NET_SCHED , CONFIG_NET_SCH_INGRESS, CONFIG_NET_CLS_U32, CONFIG_NET_CLS_ACT and CONFIG_NET_ACT_MIRRED... and perhaps more, so just set them as modules. :-) Debian packages the tc user space tool in the package iproute2.
As the veth_a_tap and veth_b_tap interfaces have been moved to the eth_tap namespace, the following commands need to be run in the eth_tap namespace. One way to do it is to start a shell in the eth_tap namespace by ip netns exec eth_tap $SHELL and run the commands inside this shell. Alternatively, prefix each of the the following tc invocation by ip netns exec eth_tap.
Direction from the node A to the node B:
tc qdisc add dev veth_a_tap ingress tc filter add \ dev veth_a_tap \ parent ffff: \ protocol all \ u32 match u8 0 0 \ action mirred egress mirror dev veth_b_tap \ action mirred egress mirror dev veth_tap_tap
Direction from B to A:
tc qdisc add dev veth_b_tap ingress tc filter add \ dev veth_b_tap \ parent ffff: \ protocol all \ u32 match u8 0 0 \ action mirred egress mirror dev veth_a_tap \ action mirred egress mirror dev veth_tap_tap
Voilà! That should do.
Let's now test the setup by sending a non-IP frame from namespace eth_a. It should reach both the namespace eth_b and the veth_tap interface. I used etherwake as a tool to send a raw non-IP frame and Wireshark to capture incoming data in the other namespace.
First start three instances of Wireshark to keep an eye on what the nodes A and B are sending and receiving, and to check if it's correctly mirrored onto the veth_tap interface.
You should be able to run Wireshark as non-root, but root is needed to start a process in a different network namespace. So run something like the following (as a non-root user, provided the user has rights to do network packet capture):
sudo ip netns exec eth_a su $USER -c "wireshark -i veth_a" & sudo ip netns exec eth_b su $USER -c "wireshark -i veth_b" & wireshark -i veth_tap &
Start capture in all three Wireshark instances and then send a WoL frame from the node A to the node B:
ip netns exec eth_a etherwake -i veth_a 88:00:00:00:00:01
You should see the frame has been sent from veth_a and has been delivered to both veth_b and to veth_tap.
You can do similar test with sending a WoL frame from node B to node A:
ip netns exec eth_b etherwake -i veth_b 88:00:00:00:00:00
To move this setup from the virtual to the real world there is only one important change that needs to be done: the tap interfaces connected to nodes A and B need to be set to promiscuous mode. (For some reason the veth devices don't seem to care and always work as if they were in promiscuous mode.) Use ip link set dev $DEVICENAME promisc on. Otherwise all frames with destination address different from the MAC address of the tap interface would be dropped. This would make the tap useless.
I used this on an Alix 2d3 computer with 3 NICs: eth0 and eth1 are connected to the hosts that were originally connected directly via an ethernet cable. eth2 is the port to which all frames are duplicated. To this port I connected a laptop where I run Wireshark to monitor the frames passing through the tap.
Here is a script that sets up the tap:
#!/bin/bash set -x TAP_A=eth0 TAP_B=eth1 TAP=eth2 # remove any IP addresses that may be set on the interfaces ip address flush dev $TAP_A ip address flush dev $TAP_B ip address flush dev $TAP # set up mirror # A -> B, TAP tc qdisc add dev $TAP_A ingress tc filter add \ dev $TAP_A \ parent ffff: \ protocol all \ u32 match u8 0 0 \ action mirred egress mirror dev $TAP_B \ action mirred egress mirror dev $TAP # B -> A, TAP tc qdisc add dev $TAP_B ingress tc filter add \ dev $TAP_B \ parent ffff: \ protocol all \ u32 match u8 0 0 \ action mirred egress mirror dev $TAP_A \ action mirred egress mirror dev $TAP # disable IPv6 -> hopefully this kills any IPv6 autoconfiguration packets sysctl -w net.ipv6.conf.$TAP_A.disable_ipv6=1 sysctl -w net.ipv6.conf.$TAP_B.disable_ipv6=1 sysctl -w net.ipv6.conf.$TAP.disable_ipv6=1 # set the A, B tap interfaces into promiscuous mode to accept all incoming frames
ip link set dev $TAP_A up promisc on ip link set dev $TAP_B up promisc on
# set the TAP interface UP ip link set dev $TAP up
If you want to run Wireshark on the same machine that does the port mirroring, there is no need for a third interface. Just set up mirroring between two ports:
tc qdisc add dev $TAP_A ingress tc filter add \ dev $TAP_A \ parent ffff: \ protocol all \ u32 match u8 0 0 \ action mirred egress mirror dev $TAP_B
tc qdisc add dev $TAP_B ingress tc filter add \ dev $TAP_B \ parent ffff: \ protocol all \ u32 match u8 0 0 \ action mirred egress mirror dev $TAP_A
Then connect Wireshark to either of $TAP_A or $TAP_B.
Long post, but perhaps someone finds it helpful.
No comments:
Post a Comment