@ -30,7 +30,7 @@ When measuring the bandwidth of our networks with longer failure paths the resul
The addition of hops to the failure path did not have an effect on the bandwidth.
\subsection{Two concurrent data transfers}
Similar to the bandwidth results, the addition of a second measurement running concurrently on the looped path does reduce throughput for both data flows.
Similar to the the results for our minimal network in \cref{evaluation_minimal_bandwidth_link_usage}, the addition of a second measurement running concurrently on the looped path does reduce throughput for both data flows.
The longer failure path however implicates that the impact in a realistic environment might be much bigger. Because more links experience additional traffic, even more data flows might be affected by the failure. The usage of ShortCut would therefore be more beneficial the more links can be removed from looped paths.
\subsubsection{With FRR}
@ -49,14 +49,14 @@ The longer failure path however implicates that the impact in a realistic enviro
@ -87,7 +87,7 @@ As can be seen in \cref{fig:evaluation_minimal_bandwidth_sc} and \cref{fig:evalu
\subsection{Two concurrent data transfers}
\label{evaluation_minimal_bandwidth_link_usage}
In this test we evaluated the bandwidth between H1 and H4 with a concurrent data transfer on H2 to H1. Both transfers were run with a limitation of \SI{100}{Mbps}, which constitutes the maximum allowed bandwidth in this test.
In this test we evaluated the bandwidth between hosts H1 and H4 with a concurrent data transfer between hosts H2 and H1. Both transfers were run with a limitation of \SI{100}{Mbps}, which constitutes the maximum allowed bandwidth in this test.
\subsubsection{With FRR}
\begin{figure}
\centering
@ -116,9 +116,9 @@ In this test we evaluated the bandwidth between H1 and H4 with a concurrent data
\end{figure}
Before a failure, as can be seen in \cref{fig:evaluation_minimal_bandwidth_link_usage_wo_sc_a}, the throughput is at around \SI{100}{Mbps} which is our current maximum. While the additional transfer between H2 and H1 does in fact use some of the links that are also used in our \textit{iperf} test, namely the link between R1 to R2 and H1 to R1, it does so in a different direction. While the data itself is sent from H1 to H4 over H2, only the tcp acknowledgements are sent on the route back. Data from H2 to H1 is sent from R2 to R1 and therefore only the returning acknowledgements use the link in the same direction, not impacting the achieved throughput.
Before a failure, as can be seen in \cref{fig:evaluation_minimal_bandwidth_link_usage_wo_sc_a}, the throughput is at around \SI{100}{Mbps} which is our current maximum. While the additional transfer between hosts H2 and H1 does in fact use some of the links that are also used in our \textit{iperf} test, namely the link between routers R1 to R2 and host H1 to R1, it does so in a different direction. While the data itself is sent from hosts H1 to H4 over H2, only the tcp acknowledgements are sent on the route back. Data from hosts H2 to H1 is sent from routers R2 to R1 and therefore only the returning acknowledgements use the link in the same direction, not impacting the achieved throughput.
If a failure is introduced however, traffic from H1 loops over R2 using up bandwidth on a link that is also passed by the additional data flow. Therefore we experience a huge performance drop to around \SIrange{20}{30}{Mbps}, while the additional data flow drops in performance to around \SI{80}{Mbps} as can be seen in \cref{fig:evaluation_minimal_bandwidth_link_usage_wo_sc_b}. From a network perspective, this results in a combined loss of 50\% throughput. While the amount of traffic sent through the network before the failure amounted to \SI{200}{Mbps}, it now dropped down to a combined \SI{100}{Mbps}.
If a failure is introduced however, traffic from host H1 loops over router R2 using up bandwidth on a link that is also passed by the additional data flow. Therefore we experience a huge performance drop to around \SIrange{20}{30}{Mbps}, while the additional data flow drops in performance to around \SI{80}{Mbps} as can be seen in \cref{fig:evaluation_minimal_bandwidth_link_usage_wo_sc_b}. From a network perspective, this results in a combined loss of 50\% throughput. While the amount of traffic sent through the network before the failure amounted to \SI{200}{Mbps}, it now dropped down to a combined \SI{100}{Mbps}.
During the bandwidth measurement in \cref{fig:evaluation_minimal_bandwidth_link_usage_wo_sc_b} there are small drops in performance
@ -182,13 +182,13 @@ In the following sections we evaluate the latency measurements run on the minima
As each link adds \SI{5}{\milli\second} of delay and \textit{ping} logs the difference in time between sending a packet and receiving an answer, the approximate delay would be the amount of links passed \textit{N} multiplied with the delay per link. In our test network there are 6 links between H1 and H4. Because these links are passed twice, one time to H4 and one time back to H1, this results in an approximate delay of \SI{60}{\milli\second}.
As each link adds \SI{5}{\milli\second} of delay and \textit{ping} logs the difference in time between sending a packet and receiving an answer, the approximate delay would be the amount of links passed \textit{N} multiplied with the delay per link. In our test network there are 6 links between hosts H1 and H4. Because these links are passed twice, one time to host H4 and one time back to host H1, this results in an approximate delay of \SI{60}{\milli\second}.
The test run confirmed these assumptions. As can be seen in \cref{fig:evaluation_minimal_latency_wo_sc_a} a ping on the network without failure took an average of around 65 milliseconds with slight variations. The additional \SI{5}{\milli\second} are most likely caused in the routing process on the router.
When introducing a failure however, additional links are passed on the way from H1 to H4. Instead of 6 links passed per direction, the network now sends the packets on a sub-optimal path which adds 2 passed links from R1 to R2 and back. These are only passed when sending packets to H4, packets returning from H4 will not take the sub-optimal path. This would, in theory, add around \SI{10}{\milli\second} of delay to our original results.
When introducing a failure however, additional links are passed on the way from host H1 to H4. Instead of 6 links passed per direction, the network now sends the packets on a sub-optimal path which adds 2 passed links from router R1 to R2 and back. These are only passed when sending packets to host H4, packets returning from H4 will not take the sub-optimal path. This would, in theory, add around \SI{10}{\milli\second} of delay to our original results.
As can be seen in \cref{fig:evaluation_minimal_latency_wo_sc_b} this is also the case. With an average of around \SI{76}{\milli\second} of latency the results show an additional delay of around \SI{11}{\milli\second} when taking the sub-optimal path. The discrepancy between our assumption of \SI{10}{\milli\second} and the actual added \SI{11}{\milli\second} might be caused by the additional router that is passed in the direction to H4.
As can be seen in \cref{fig:evaluation_minimal_latency_wo_sc_b} this is also the case. With an average of around \SI{76}{\milli\second} of latency the results show an additional delay of around \SI{11}{\milli\second} when taking the sub-optimal path. The discrepancy between our assumption of \SI{10}{\milli\second} and the actual added \SI{11}{\milli\second} might be caused by the additional router that is passed in the direction to host H4.
When the failure is introduced concurrent to a running test, the latency spikes to around \SI{94}{\milli\second} for one packet as can be seen in \cref{fig:evaluation_minimal_latency_concurrent_wo_sc}. This might be caused by the deactivation of interfaces using \textit{ifconfig} and a packet arriving just at the moment of reconfiguration, as packets are sent every \SI{0.5}{\second} and the failure is introduced exactly \SI{15}{\second} after starting the measurement. Depending on the time \textit{ifconfig} takes to reconfigure this would cause the packet to remain in the queue until the reconfiguration is finished, adding to the latency measured in this one instance.
@ -229,7 +229,7 @@ The spike in latency which can be seen in \cref{fig:evaluation_minimal_latency_c
\subsection{Packet flow - TCP}
\label{evaluation_minimal_tcp_packet_flow}
To show the amount of TCP packets being forwarded on each router, we measured the packet flow on all routers of this topology. This is done by counting TCP packets with \textit{nftables} while a concurrent data transfer is started from H1 to H4. The results include the amount of packets forwarded on each router per second. This was done with an intermediate and concurrent failure for a network with FRR in \cref{minimal_packet_flow_with_frr}, as well as a network with an additional implementation of ShortCut in \cref{minimal_packet_flow_with_frr_and_shortcut}.
To show the amount of TCP packets being forwarded on each router, we measured the packet flow on all routers of this topology. This is done by counting TCP packets with \textit{nftables} while a concurrent data transfer is started from host H1 to H4. The results include the amount of packets forwarded on each router per second. This was done with an intermediate and concurrent failure for a network with FRR in \cref{minimal_packet_flow_with_frr}, as well as a network with an additional implementation of ShortCut in \cref{minimal_packet_flow_with_frr_and_shortcut}.
\subsubsection{With FRR}
\label{minimal_packet_flow_with_frr}
@ -239,14 +239,14 @@ To show the amount of TCP packets being forwarded on each router, we measured th
The results in the network before a failure are as to be expected and can be seen in \cref{fig:evaluation_minimal_packet_flow_wo_sc_a}. Each router on the route from H1 to H4, which includes R1, R2 and R4, report the same amount of packets at each point of measurement. While the packet count fluctuates during the measurement no packet loss was reported and the bandwidth was at an average of \SI{95}{Mbps} during the whole run of the test. This is why we assume that the fluctuations can be attributed to the mechanisms used in \textit{iperf}.
The results in the network before a failure are as to be expected and can be seen in \cref{fig:evaluation_minimal_packet_flow_wo_sc_a}. Each router on the route from host H1 to H4, which includes routers R1, R2 and R4, report the same amount of packets at each point of measurement. While the packet count fluctuates during the measurement no packet loss was reported and the bandwidth was at an average of \SI{95}{Mbps} during the whole run of the test. This is why we assume that the fluctuations can be attributed to the mechanisms used in \textit{iperf}.
After a failure all four routers receive packets as can be seen in \cref{fig:evaluation_minimal_packet_flow_wo_sc_b}, but router R1 now receives most packets with an average of around 1500 packets while router R3 and R4 receive roughly the same amount of packets as before the failure at an average of around 1000 packets. Router R2 receives the least packets with an average of around 500 packets.
After a failure all four routers receive packets as can be seen in \cref{fig:evaluation_minimal_packet_flow_wo_sc_b}, but router R1 now receives most packets with an average of around 1500 packets while routers R3 and R4 receive roughly the same amount of packets as before the failure at an average of around 1000 packets. Router R2 receives the least packets with an average of around 500 packets.
This is most likely caused by the looped path and the implications for packet travel this has. Router R1 receives all packets that are sent to H4 from H1 twice, once sending them to R2 and the second time when receiving the packets back from R2 to send them to R3. But while all packets sent from H1 pass R1 twice, acknowledgements sent back by the \textit{iperf} server on H4 will only pass R1 once, as R1 would not send packets with H1 as destination to R2. Router R2 on the other hand only receives packets sent to H4 but none of the ACKs sent back. This is why, when compared to the average packet count of all routers in \cref{fig:evaluation_minimal_packet_flow_wo_sc_a}, R2 receives roughly half of all packets a router would normally receive as TCP specifies that for each received packet TCP will send an ACK as answer. This also explains why router R1 forwards an average of around 1500 packets per second, forwarding data packets with around 500 packets per second twice and forwarding acknowledgement packets once with also 500 packets per second, producing an additional 50\% load on the router.
This is most likely caused by the looped path and the implications for packet travel this has. Router R1 receives all packets that are sent to host H4 from H1 twice, once sending them to router R2 and the second time when receiving the packets back from router R2 to send them to router R3. But while all packets sent from host H1 pass router R1 twice, acknowledgements sent back by the \textit{iperf} server on H4 will only pass R1 once, as R1 would not send packets with H1 as destination to R2. Router R2 on the other hand only receives packets sent to H4 but none of the ACKs sent back. This is why, when compared to the average packet count of all routers in \cref{fig:evaluation_minimal_packet_flow_wo_sc_a}, R2 receives roughly half of all packets a router would normally receive as TCP specifies that for each received packet TCP will send an ACK as answer. This also explains why router R1 forwards an average of around 1500 packets per second, forwarding data packets with around 500 packets per second twice and forwarding acknowledgement packets once with also 500 packets per second, producing an additional 50\% load on the router.
Aside from the changed path and therefore the inclusion of router R3 in this path, routers R3 and R4 are unaffected by the failure, forwarding each packet once.
@ -303,7 +303,7 @@ Reconfiguration of routers in Mininet does not reset the \textit{nftables} count
When running the TCP packet flow measurements with an implementation of ShortCut running on the network however, the results change drastically, and as expected all packets sent by the \textit{iperf} transfer are forwarded by router R2 on the original route. After the failure was introduced the router R2 does not forward any packets. ShortCut has effectively cut out router R2 from the route, forwarding packets from R1 to R3 directly. All remaining routers R1, R3 and R4 now receive all packets and no router forwards any packets twice.
When running the TCP packet flow measurements with an implementation of ShortCut running on the network however, the results change drastically, and as expected all packets sent by the \textit{iperf} transfer are forwarded by router R2 on the original route. After the failure was introduced the router R2 does not forward any packets. ShortCut has effectively cut out router R2 from the route, forwarding packets from router R1 to R3 directly. All remaining routers R1, R3 and R4 now receive all packets and no router forwards any packets twice.
\subsection{Packet flow - UDP}
@ -364,7 +364,7 @@ Because UDP does not send ACKs the results observed after a failure in \cref{fig
\caption{UDP packet flow on all routers measured with \textit{nftables} counters using ShortCut}
\end{figure}
When using ShortCut in a UDP packet flow measurement, the negative consequences of the failure disappear. While in \cref{fig:evaluation_minimal_packet_flow_udp_wo_sc_a} routers R1, R2 and R4 receive all packets on the original route, the load switches after a failure from R2 to R3. As expected, the ShortCut implementation has cut out the looped path and restored the original functionality on an alternative route.
When using ShortCut in a UDP packet flow measurement, the negative consequences of the failure disappear. While in \cref{fig:evaluation_minimal_packet_flow_udp_wo_sc_a} routers R1, R2 and R4 receive all packets on the original route, the load switches after a failure from router R2 to R3. As expected, the ShortCut implementation has cut out the looped path and restored the original functionality on an alternative route.
@ -42,7 +42,7 @@ Older FRMs have already been evaluated thoroughly and even though they do work i
In this context we use Mininet (\cite{LantzBobandtheMininetContributors.}), a tool to create virtual networks, to implement multiple topologies with routings. We then implement a simple FRR mechanism, re-routing returning packets to an alternative path.
To test ShortCut we provide a prototype for an implementation using \textit{nftables} (\cite{AyusoPabloNeiraandKadlecsikJozsefandLeblondEricandWestphalFlorianandGonzalezArtur.}) and python 3.8 (\cite{vanRossum.2009}) that can be installed on any linux based router.
To test ShortCut we provide a prototype for an implementation using \textit{nftables} (\cite{Ayuso.2019}) and python 3.8 (\cite{vanRossum.2009}) that can be installed on any linux based router.
We developed a testing framework that can be used to automatically create Mininet topologies, formulate tests in python dictionaries using existing measurement functions, set network wide bandwidth limits or delays and to run automatic sets of tests.
The framework can be called using an argument based \textit{command line interface} (CLI).
Frederik Maaßen
2 years ago