From 1613e0121c78172be75c88604d79dd2604f9ec33 Mon Sep 17 00:00:00 2001 From: Jouni Malinen Date: Sun, 15 Oct 2017 23:27:12 +0300 Subject: [PATCH] Add notes on how to do cipher suite and key management testing This describes how various wpa_supplicant/hostapd extensions can be used to test IEEE 802.11 functionality in other devices. Signed-off-by: Jouni Malinen --- tests/cipher-and-key-mgmt-testing.txt | 404 ++++++++++++++++++++++++++ 1 file changed, 404 insertions(+) create mode 100644 tests/cipher-and-key-mgmt-testing.txt diff --git a/tests/cipher-and-key-mgmt-testing.txt b/tests/cipher-and-key-mgmt-testing.txt new file mode 100644 index 000000000..5030ca869 --- /dev/null +++ b/tests/cipher-and-key-mgmt-testing.txt @@ -0,0 +1,404 @@ +Cipher suite (CCMP, TKIP, GCMP, ..) and key management testing +============================================================== + +wpa_supplicant and hostapd include number of extensions that allow +special test builds to be used for testing functionality related to +correct implementation of IEEE 802.11. These extensions allow behavior +to be modified and invalid operations to be performed to verify behavior +of other devices in unexpected situations. While most of the testing +extensions are focused on the fully automated testing framework with +mac80211_hwsim (see tests/hwsim subdirectory), many of these can be used +for over-the-air testing of the protocol as well. + +Since some of the testing extensions can result in exposing key +information or allowing non-compliant behavior, these changes are +disabled in default wpa_supplicant and hostapd builds for production +purposes. Testing functionality can be enabled by adding +CONFIG_TESTING_OPTIONS=y into build configuration (hostapd/.config and +wpa_supplicant/.config). + + +Testing setup +------------- + +These tests can be run as black-box testing without having to modify the +tested device at all or without knowing details of its +functionality. The test commands in wpa_supplicant/hostapd control +interfaces are used to perform unexpected operations and normal data +traffic is used to verify reaction of the tested device to such +operations. + +In theory, the test functionality is available with most drivers +supported by wpa_supplicant/hostapd, but the most reliable results are +likely available through ath9k-based devices. If you are using something +else, it is strongly recommended that you'll run the first tests with +sniffer captures and verify that the test tools are behaving correctly. + +wpa_supplicant is used to control a test device in station mode to test +an AP and hostapd is similarly used to control a test device in AP mode +to test a station. + +Various data traffic generators could be used to test the behavior, but +this document focuses on using ping to test unicast traffic and arping +to test broadcast traffic. To keep things simple and to reduce +interference from unrelated traffic, the steps here assume static IPv4 +addresses are used and IPv6 is disabled. + +The tests here use WPA2-Personal for simplicity. WPA2-Enterprise and +other cipher suites can also be tested for more complete coverage. + +Example hostapd.conf for the test tool in AP mode: + +driver=nl80211 +hw_mode=g +channel=1 +ieee80211n=1 +interface=wlan0 +ctrl_interface=/var/run/hostapd +ctrl_interface_group=adm +ssid=test-psk +wpa=2 +wpa_key_mgmt=WPA-PSK +wpa_pairwise=CCMP +wpa_passphrase=12345678 + +Example wpa_supplicant.conf for the test tool in station mode: + +ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=adm + +network={ + ssid="test-psk" + key_mgmt=WPA-PSK + psk="12345678" +} + +The examples in this document assume following IPv4 address +configuration: + +Test tool (either AP or station mode): 192.168.1.1/24 +Device under test: 192.168.1.2/24 + + +Data traffic tests +------------------ + +ping is used to test whether unicast frames go through on the data +link. It should be noted that ping may need to use broadcast ARP at the +beginning if the other device is not yet in the ARP table, so working +broadcast and unicast connectivity may be needed to get this started. + +Example: + +$ ping -n -c 5 192.168.1.2 +PING 192.168.1.2 (192.168.1.2) 56(84) bytes of data. +64 bytes from 192.168.1.2: icmp_seq=1 ttl=64 time=43.7 ms +64 bytes from 192.168.1.2: icmp_seq=2 ttl=64 time=67.9 ms +64 bytes from 192.168.1.2: icmp_seq=3 ttl=64 time=900 ms +64 bytes from 192.168.1.2: icmp_seq=4 ttl=64 time=5.81 ms +64 bytes from 192.168.1.2: icmp_seq=5 ttl=64 time=135 ms + +--- 192.168.1.2 ping statistics --- +5 packets transmitted, 5 received, 0% packet loss, time 4004ms +rtt min/avg/max/mdev = 5.811/230.605/900.223/337.451 ms + +This shows working unicast data connectivity. + +$ ping -n -c 5 192.168.1.2 +PING 192.168.1.2 (192.168.1.2) 56(84) bytes of data. + +--- 192.168.1.2 ping statistics --- +5 packets transmitted, 0 received, 100% packet loss, time 4033ms + +This shows not working unicast data connectivity. + + +arping is used to test broadcast connectivity. + +Example: + +$ arping -b -I wlan0 192.168.1.2 -c 5 +ARPING 192.168.1.2 from 192.168.1.1 wlan0 +Unicast reply from 192.168.1.2 [] 119.695ms +Unicast reply from 192.168.1.2 [] 144.496ms +Unicast reply from 192.168.1.2 [] 166.788ms +Unicast reply from 192.168.1.2 [] 2.283ms +Unicast reply from 192.168.1.2 [] 2.234ms +Sent 5 probes (5 broadcast(s)) +Received 5 response(s) + +This shows working broadcast data connectivity. + +$ arping -b -I wlan0 192.168.1.2 -c 5 +ARPING 192.168.1.2 from 192.168.1.1 wlan0 +Sent 5 probes (5 broadcast(s)) +Received 0 response(s) + +This shows not working broadcast data connectivity. + +If testing results do not look consistent, the testing state can be +cleared by disconnection and reconnecting the station (the test tool or +the DUT) to the network. + + +Sniffer and wlantest +-------------------- + +It is useful to get a wireless sniffer capture from the operating +channel of the AP to be able to confirm DUT behavior if any of the data +tests indicate reason to believe something is not working as expected. + +wlantest (from the wlantest directory of hostap.git) can be used to +decrypt and analyze a sniffer capture. For example: + +wlantest -r wlan0.pcap -n decrypted.pcap -p 12345678 + +The debug prints and comments in the generated file indicate where +unexpected behavior has been detected, e.g., when the test tool ends up +clearing its packet number to test replay protection. That can help in +checking whether the DUT actually replies to a frame that it was +supposed to drop due replay. + + +Testing replay protection on a station device +--------------------------------------------- + +Start hostapd and use hostapd_cli on the test device to control testing +operations. Connect the DUT to the network. + +<3>AP-STA-CONNECTED + +This indicates that the connection was completed successfully. + +Verify that broadcast and unicast traffic works correctly (if not, +something is wrong in the test setup and that needs to be resolved +before being able to run any tests). + +Verify that unicast traffic works and issue the following command in +hostapd_cli: + +> raw RESET_PN +OK + +Verify that unicast traffic does not work anymore. If it does, the DUT +does not implement replay protection correctly for unicast frames. Note +that unicast traffic can recover once the packet number from the test +device increases beyond the value used prior to that RESET_PN command. + + +Verify that broadcast traffic works and issue the following command in +hostapd_cli: + +> raw RESET_PN ff:ff:ff:ff:ff:ff +OK + +Verify that broadcast traffic does not work anymore. If it does, the DUT +does not implement replay protection correctly for broadcast +frames. Note that broadcast traffic can recover once the packet number +from the test device increases beyond the value used prior to that +RESET_PN command. + + +Testing replay protection on an AP device +----------------------------------------- + +Start the AP (DUT) and start wpa_supplicant on the test device to +connect to the network. Use wpa_cli to control the test device. + +<3>SME: Trying to authenticate with (SSID='test-psk' freq=5240 MHz) +<3>CTRL-EVENT-REGDOM-CHANGE init=CORE type=WORLD +<3>Trying to associate with (SSID='test-psk' freq=5240 MHz) +<3>Associated with +<3>WPA: Key negotiation completed with [PTK=CCMP GTK=CCMP] +<3>CTRL-EVENT-CONNECTED - Connection to completed [id=0 id_str=] + +Verify that unicast traffic works and issue the following command in +wpa_cli: + +> raw RESET_PN +OK + +Verify that unicast traffic does not work anymore. If it does, the DUT +does not implement replay protection correctly. Note that unicast +traffic can recover once the packet number from the test device +increases beyond the value used prior to that RESET_PN command. + +IEEE 802.11 protocol uses unicast frames in station-to-AP direction, so +there is no need to test AP replay protection behavior separately with +the broadcast IPv4 traffic (which would be converted to unicast frames +on the link layer). + + +Testing GTK reinstallation protection on a station device (group handshake) +--------------------------------------------------------------------------- + +Use the procedure describe above for testing replay protection, but with +the following hostapd_cli commands: + +Test broadcast connectivity; should work + +> raw RESET_PN ff:ff:ff:ff:ff:ff +OK + +Test broadcast connectivity; should not work; if it does, replay +protection is completely broken and the following step cannot be +executed reliably. The following command needs to be run before there +has been large enough number of new frames to increment the PN on the +test tool. It would also be possible to execute "raw RESET_PN +ff:ff:ff:ff:ff:ff" again after the initial sanity testing to get back to +PN 0 for the next step. + +> raw RESEND_GROUP_M1 +OK + +Test broadcast connectivity; should not work; if it does, the device +does not implement protection for delayed retransmission of Group Key +Message 1/2. + + +Testing GTK reinstallation protection on a station device (4-way handshake) +--------------------------------------------------------------------------- + +Use the procedure described above for testing replay protection for +broadcast traffic, but with the following hostapd_cli commands: + +Test broadcast connectivity; should work + +> raw RESET_PN ff:ff:ff:ff:ff:ff +OK + +Test broadcast connectivity; should not work; if it does, replay +protection is completely broken and the following step cannot be +executed reliably. The following command needs to be run before there +has been large enough number of new frames to increment the PN on the +test tool. It would also be possible to execute "raw RESET_PN +ff:ff:ff:ff:ff:ff" again after the initial sanity testing to get back to +PN 0 for the next step. + +> raw RESEND_M3 +OK + +Test broadcast connectivity; should not work; if it does, the device +does not implement protection for delayed retransmission of 4-way +handshake EAPOL-Key Message 3/4. + +Variant 1: Include extra Message 1/4 + +Otherwise same as above, but replace RESEND_M3 command with: + +> raw RESEND_M1 +OK +> raw RESEND_M3 +OK + +Variant 2: Include two extra Message 1/4 + +Otherwise same as above, but replace RESEND_M3 command with: + +> raw RESEND_M1 change-anonce +OK +> raw RESEND_M1 +OK +> raw RESEND_M3 +OK + + +Testing TK reinstallation protection on a station device (4-way handshake) +-------------------------------------------------------------------------- + +Use the procedure described above for testing replay protection for +unicast traffic, but with the following hostapd_cli commands: + +Test unicast connectivity; should work + +> raw RESET_PN +OK + +Test unicast connectivity; should not work; if it does, replay +protection is completely broken and the following step cannot be +executed reliably. The following command needs to be run before there +has been large enough number of new frames to increment the PN on the +test tool. It would also be possible to execute "raw RESET_PN " again after the initial sanity testing to get back to PN 0 for +the next step. + +> raw RESEND_M3 +OK + +Test unicast connectivity; should not work; if it does, the device +does not implement protection for delayed retransmission of 4-way +handshake EAPOL-Key Message 3/4. + +Variant 1: Include extra Message 1/4 + +Otherwise same as above, but replace RESEND_M3 command with: + +> raw RESEND_M1 +OK +> raw RESEND_M3 +OK + +Variant 2: Include two extra Message 1/4 + +Otherwise same as above, but replace RESEND_M3 command with: + +> raw RESEND_M1 change-anonce +OK +> raw RESEND_M1 +OK +> raw RESEND_M3 +OK + + +Testing ANonce generation on an AP device +----------------------------------------- + +Start the AP (DUT) and start wpa_supplicant on the test device to +connect to the network. Use wpa_cli to control the test device. + +<3>SME: Trying to authenticate with (SSID='test-psk' freq=5240 MHz) +<3>CTRL-EVENT-REGDOM-CHANGE init=CORE type=WORLD +<3>Trying to associate with (SSID='test-psk' freq=5240 MHz) +<3>Associated with +<3>WPA: Key negotiation completed with [PTK=CCMP GTK=CCMP] +<3>CTRL-EVENT-CONNECTED - Connection to completed [id=0 id_str=] + +Show the ANonce from the first 4-way handshake, request PTK rekeying, +and show the ANonce from the second 4-way handshake: + +> GET anonce +df8c61d1f1f7aca9f1739dd888199547f4af2b8b07f8bf15b45ea271da0072b2 +> raw KEY_REQUEST 0 1 +OK +> GET anonce +d8ddcb716f28abfdf1352a05d51e7a70f58802122e99d13c730c3c0f09594aac + +If the ANonce values are same, the AP did not update the ANonce for +rekeying (it should have as shown in the example above). + + +Testing FT Reassociation Request frame retransmission on an AP device +--------------------------------------------------------------------- + +This test case requires a sniffer to be used and manually analyzed. + +Enable FT on the DUT AP (likely two AP devices needed), connect test +tool to the AP using FT protocol (e.g., connect to another AP first and +then use the "ROAM " command), and do the following steps: + +- verify unicast traffic from the AP to test station (either ping from + the AP or from a device behind the AP); this needs to work +- wpa_cli "raw RESEND_ASSOC" +- verify unicast traffic from the AP to test station (either ping from + the AP or from a device behind the AP); this is likely to fail, but + the real analysis is done based on the sniffer capture + +In the sniffer capture, find the last Reassociation Request frame from +the test station (this is more or less identical to the previous one and +the only one that should not have Authentication frame exchange before +it). Look at the last used PN in a unicast Data frame from the AP to the +test station before the last Reassociation Request frame and the PN in +the following unicast Data frame after the last Reassociation Request +frame. If the PN goes down (e.g., is reset to 1), this would be a sign +of a likely security vulnerability. The AP's TK configuration should be +verified (i.e., whether it is configuring the same TK again and then +allowing it to be used with reused PN values).