Commit graph

1274 commits

Author SHA1 Message Date
Jouni Malinen
584e4197bd tests: Make ap_wpa2_eap_aka_ext faster and more robust
Use SELECT_NETWORK instead of REASSOCIATE for the first reconnection to
avoid unnecessary long wait for temporary network disabling to be
cleared. In addition, wait for the disconnect event after issuing the
DISCONNECT commands to avoid issues due to any pending events during the
immediately following reconnection attempt.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-17 16:59:40 +02:00
Jouni Malinen
27f527e0e2 tests: ap_hs20_fetch_osu: Print osu-providers.txt in debug log
This makes it easier to figure out what happened if the test case fails
due to not finding all the needed OSU-PROVIDER information.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-17 16:03:43 +02:00
Jouni Malinen
a4a15cf1f9 tests: Skip some scan tests if iw does not support scan flush
The external cfg80211 scan flushing operation requires a relatively
recent iw version and not all distributions include that. Avoid false
failure reports by marking these test cases skipped if the iw command
fails.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-17 13:05:34 +02:00
Jouni Malinen
e246d7d5b3 tests: Fix test skipping for some DFS/VHT cases
Due to a typo and missing hapd variable initialization, some of the DFS
and VHT test cases were marked as failures even though they were
supposed to be marked as skipped in case the kernel and wireless-regdb
did not have sufficient support for these modes.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-17 13:04:11 +02:00
Jouni Malinen
fb9adae466 tests: Fix dbus_probe_req_reporting_oom if already registered
If dbus_probe_req_reporting was run before dbus_probe_req_reporting_oom,
the SubscribeProbeReq() method succeeded since the memory allocation
that was supposed to fail in the OOM test case was not even tried.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-17 12:39:00 +02:00
Jouni Malinen
90b10d4edd tests: EAP-TNC fragmentation
Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-17 12:12:33 +02:00
Jouni Malinen
ee9533eb0e tests: EAP-MD5 server error cases
Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-17 12:02:11 +02:00
Jouni Malinen
802bf82482 tests: Add optional -1 argument to parallel-vm.py
This can be used to skip rerunning of failed test cases
(e.g., with "./parallel-vm.py 1 -1 <test case>").

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-17 11:28:46 +02:00
Jouni Malinen
e912e4bc92 tests: Interworking auto_interworking=1 with mismatching BSS
This is a regression test case to detect a failure that resulted in an
up to five second busy loop through wpa_supplicant_fast_associate() when
interworking_find_network_match() and wpa_supplicant_select_bss() get
different matching results.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-17 01:53:44 +02:00
Jouni Malinen
6ace81ea77 tests: Disconnect-Request with no session identification attributes
Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-16 16:16:28 +02:00
Jouni Malinen
9921689759 tests: Use a helper function to send and check RADIUS DAS messages
No need to have this same sequence of steps duplicated in multiple
places.

Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
2015-01-16 16:14:54 +02:00
Jouni Malinen
05dad77c8f tests: RADIUS DAS and Disconnect-Request removing PMKSA cache entry
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
2015-01-16 16:07:52 +02:00
Jouni Malinen
e94a3f626d tests: RADIUS DAS with Acct-Multi-Session-Id
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
2015-01-16 13:10:48 +02:00
Jouni Malinen
9142b4dd45 tests: Disconnect-Request multi-session-match
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
2015-01-16 12:50:16 +02:00
Jouni Malinen
c2b48088f6 tests: Fix radius_das_disconnect match + non-match case
If Calling-Station-Id matches, but CUI does not, NAS is expected to
reject the request instead of accepting it. Verify that Disconnect-NAK
is returned for this.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-16 12:50:16 +02:00
Jouni Malinen
201c9ad77f tests: STA not getting response to SA Query
This verifies that wpa_supplicant reconnects if PMF is enabled,
unprotected Deauthentication/Disassociation frame is received, and the
AP does not reply to SA Query.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-16 01:13:59 +02:00
Jouni Malinen
e1f8fe8851 tests: INTERWORKING_CONNECT after having found hidden SSID AP
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
2015-01-15 12:29:02 +02:00
Jouni Malinen
061cbb258f tests: domain_match checking against server certificate
Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-14 15:45:18 +02:00
Jouni Malinen
2099fed400 tests: dbus_connect_eap to verify dNSName constraint configuration
This verifies that Certification signals include the expected
information on peer certificates and that dNSName constraint can be
configured based on that and is working both in matching and not
matching cases.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-14 15:45:18 +02:00
Jouni Malinen
496c4e45d8 tests: Subset of VHT functionality on 2.4 GHz
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
2015-01-14 01:12:56 +02:00
Jouni Malinen
615d8a9705 tests: Add room for more vendor elems in wpas_ctrl_vendor_elem
This test case was verifying that the first unused VENDOR_ELEM value
above the current maximum is rejected. That makes it a bit inconvenient
to add new entries, so increase the elem value to leave room for new
additions without having to continuously modify this test case.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-14 01:12:56 +02:00
Jouni Malinen
37b4a66ce6 tests: Valid OCSP response with revoked and unknown cert status
This increases testing coverage for OCSP processing by confirming that
valid OCSP response showing revoked certificate status prevents
successful handshake completion. In addition, unknown certificate status
is verified to prevent connection if OCSP is required and allow
connection if OCSP is optional.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-12 00:19:21 +02:00
Jouni Malinen
279a0afffb tests: Generate a fresh OCSP response for each test run
GnuTLS has a hardcoded three day limit on OCSP response age regardless
of the next update value in the response. To make this work in the test
scripts, try to generate a new response when starting the authentication
server. The old mechanism of a response without next update value is
used as a backup option if openssl is not available or fails to generate
the response for some reason.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-12 00:19:21 +02:00
Jouni Malinen
0ff7afbc42 tests: Verify mesh support for wpas_add_set_remove_support
This test case fails if wpa_supplicant is built without mesh support, so
need to check for this.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-12 00:19:21 +02:00
Jouni Malinen
2f0347954a tests: Check mesh capability based on the modes capabilities list
This is more robust than checking the driver capability because it is
also possible for the wpa_supplicant build to be configured without mesh
support regardless of whether the driver supports it.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-12 00:19:21 +02:00
Jouni Malinen
b9749b6aaa tests: Verify that SAE is supported for test cases requiring it
This makes it more convenient to run tests with wpa_supplicant builds
that do not support SAE (e.g., due to crypto library not providing
sufficient functionality for this).

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-12 00:19:21 +02:00
Jouni Malinen
4bf4e9db86 tests: Skip ap_wpa2_eap_ttls_server_cert_hash if probing not supported
The ca_cert="probe://" functionality is currently supported only with
OpenSSL.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-12 00:19:21 +02:00
Jouni Malinen
62750c3e80 tests: Use RSA key format in ap_wpa2_eap_tls_blob
This format as a DER encoded blob is supported by both OpenSSL and
GnuTLS while the previous OpenSSL specific format did not get accepted
by GnuTLS.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-12 00:19:21 +02:00
Jouni Malinen
24579e7047 tests: Split domain_suffix_match test cases based on match type
With GnuTLS, domain_suffix_match is currently requiring full match, so
split the test cases in a way that can be reported more cleanly as PASS
or SKIP based on TLS library behavior.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-12 00:19:21 +02:00
Jouni Malinen
524c6c33bc tests: Add ca_cert to username/password Hotspot 2.0 credentials
Proper configuration should be used here to get server validation
enabled, so update the test cases to provide the ca_cert parameter. This
was included in number of existing test cases, but not all.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-12 00:19:21 +02:00
Jouni Malinen
506b2f05f2 tests: Split subject_match and altsubject_match to separate test cases
These parameters are supported only with OpenSSL, so split any test case
that used those for a successful connection into two test cases. Skip
all test cases where these are used without the selected TLS library
supporting them to avoid reporting failures incorrectly. Though, verify
that subject_match and altsubject_match get rejected properly if TLS
library does not support these.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-12 00:19:21 +02:00
Jouni Malinen
3b51cc6359 tests: Skip EAP-pwd and EAP-FAST test cases if not supported
Check wpa_supplicant EAP capability and skip EAP-pwd and EAP-FAST test
cases if the build did not include support for these. This is cleaner
than reporting failures for such test cases when the selected TLS
library does not support the EAP method.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-12 00:19:21 +02:00
Jouni Malinen
a783340d04 tests: Skip OpenSSL cipher string tests with other TLS libraries
Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-12 00:19:21 +02:00
Jouni Malinen
37d6135507 tests: Increase altsubject_match testing coverage
Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-11 00:27:01 +02:00
Jouni Malinen
c2096d9981 tests: bssid_blacklist and bssid_whitelist
Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-10 17:35:53 +02:00
Jouni Malinen
70437ae55a tests: Drop mesh_ht_mode parameter from mesh test cases
This network profile parameter will be removed with the cleanup that
makes mesh use shared functions for setting channel parameters. That
will allow HT to be enabled automatically based on driver capabilities.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-10 17:35:53 +02:00
Jouni Malinen
de1d5049dc tests: Mesh in 5 GHz band
Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-10 17:35:53 +02:00
Johannes Berg
2e1d7386e2 tests: Refactor tshark running
Refactor the code to run tshark into its own submodule. This allows
even remembering whether -Y or -R needs to be used for filtering.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
2015-01-10 00:53:57 +02:00
Jouni Malinen
0e126c6dca tests: Out-of-memory cases for D-Bus operations
This increases testing coverage on various error paths.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-09 02:46:26 +02:00
Jouni Malinen
795b6f57a8 tests: Additional D-Bus error path coverage
Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-09 02:32:15 +02:00
Jouni Malinen
cd9548c02f tests: Add out-of-memory loop tests for hostapd startup
These test cases run hostapd interface setup multiple times with
TEST_ALLOC_FAIL commands triggering memory allocation failures one by
one at each possible location in the setup sequence. Effectively, these
test cases will hit most error paths for memory allocation issue cases
(i.e., only the cases requiring more than one allocation failure in a
sequence are not covered).

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-08 22:43:48 +02:00
Jouni Malinen
f7af5a48f4 tests: hostapd BSS addition failures
Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-08 22:43:48 +02:00
Jouni Malinen
fff95bb614 tests: hostapd interface setup and memory allocation failures
Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-08 22:43:48 +02:00
Jouni Malinen
f12240a371 tests: Memory allocation failure in wpa_supplicant blacklist
Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-08 22:43:48 +02:00
Jouni Malinen
81e787b750 tests: Convert test skipping to use exception
Instead of returning "skip" from the test function, raise the new
HwsimSkip exception to indicate a test case was skipped.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-08 22:43:47 +02:00
Johannes Berg
9c8779daf6 tests: Use rfkill python module
Instead of calling the rfkill binary, use the built-in module.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
2015-01-08 20:56:54 +02:00
Johannes Berg
bd31b92a68 tests: Add rfkill module
This can be used instead of invoking the rfkill binary.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
2015-01-08 20:56:47 +02:00
Jouni Malinen
6a964b7560 tests: Make scan_random_mac more robust
There is a race condition between wlantest having received and written
the sniffer log and this test case using tshark to process it. Wait one
second before running tshark to make it less likely to get truncated
results that can result in the test case failing.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-07 16:31:22 +02:00
Jouni Malinen
51c5aeb42a tests: Convert "hwsim-SKIP" exception to use a custom class
This makes the design a bit cleaner for catching the exceptions.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-07 16:31:22 +02:00
Jouni Malinen
910eb5b525 tests: Import dbus module in a cleaner way
There is no need to use prepare_dbus() function to check whether the
dbus module can be imported.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-07 13:29:25 +02:00
Jouni Malinen
c975e1c1da tests: Additional bgscan coverage
Verify error cases in bgscan module parameters. Increase learn bgscan
module coverage with parsing of the saved data and probe frequency
selection.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-06 02:49:13 +02:00
Jouni Malinen
8aeb6e3dbc tests: MAC_RAND_SCAN with valid and invalid arguments
This verifies that random MAC address is used as requested in Probe
Request frames.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-04 23:06:58 +02:00
Jouni Malinen
5434f07c20 tests: TDLS channel switching
Verify correct behavior with invalid commands. In addition, allow minor
mac80211_hwsim modifications to be used to enable testing of the driver
interface to enable and disable channel switching.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-04 21:46:38 +02:00
Eliad Peller
41bf76447b tests: WMM-AC reassociation-to-same-BSS test
Make sure the tspecs are kept on reassociation to the same BSS.

Signed-off-by: Eliad Peller <eliadx.peller@intel.com>
2015-01-04 18:59:07 +02:00
Eliad Peller
bc32f830f4 tests: wmm_ac_status and roaming case with WMM-AC
Make sure the wmm_ac_status command reflects correctly the existing
tspecs after add_ts/del_ts commands. Add a new test to verify all tspecs
are removed on roaming (while FT is not used).

Signed-off-by: Eliad Peller <eliadx.peller@intel.com>
2015-01-04 18:57:23 +02:00
Jouni Malinen
750904dd42 tests: Extend EAP key lifetime in memory to cover MSK and EMSK
Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-03 15:49:57 +02:00
Jouni Malinen
ce8ca2f9a0 tests: Make wpas_mesh_max_peering more robust
The previous version was enabling all three stations at the same time
and left dev[1] and dev[2] competing on getting connected with dev[0]
that allowed only one pairing. This was not exactly robust and the pass
criteria depended on an extra event from either dev[1] or dev[2]. Fix
that by first connecting dev[0] and dev[1] and only after that, start
dev[2]. This allows proper validation of both the peering limit on
dev[0] and no extra event on dev[2].

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-02 22:50:27 +02:00
Jouni Malinen
29bac7cb89 tests: Make scan_hidden more robust under heavy CPU load
It is possible for the Probe Response frame wait to time out when active
scanning is used under heavy CPU load. Make this test case more robust
by trying multiple times before declaring the scan for a hidden SSID to
have failed.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-02 22:50:27 +02:00
Jouni Malinen
2ec82e67c1 tests: wpa_supplicant D-Bus interface
This adds new files with test cases to verify both the old and new
wpa_supplicant D-Bus interface.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-02 22:50:27 +02:00
Jouni Malinen
b0839232fa tests: Enable wpa_supplicant D-Bus support for hwsim tests
This allows automated testing of the wpa_supplicant D-Bus interface. The
instance controlling wlan0 registers with D-Bus if dbus-daemon was
started successfully. This is only used in VM testing, i.e., not when
run-tests.sh is used on the host system with D-Bus running for normal
system purposes.

Signed-off-by: Jouni Malinen <j@w1.fi>
2015-01-02 22:50:27 +02:00
Jouni Malinen
5b3c40a65b tests: Verify that wpa_supplicant clears keys from memory
Check that PMK and PTK and not left in memory (heap or stack)
unnecessarily after they are not needed anymore.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-30 10:37:02 +02:00
Jouni Malinen
13113d127c tests: Filter out extra files from codecov reports
wpa_cli and hostapd_cli are not currently tested for code coverage, so
filter the files specific to those components away from the code
coverage reports. *_module_tests.c are not included in normal builds, so
drop them as well. In addition, drop the system header file (byteswap.h)
that gets somehow unnecessarily included in the reports for couple of
lines.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-29 18:27:23 +02:00
Jouni Malinen
7fe84009f8 tests: Reorder build.sh operations
This leaves the build tree with valid wpa_supplicant object files in the
end and makes it a bit easier to do additional custom builds when
needed.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-29 15:49:05 +02:00
Jouni Malinen
185275228d tests: Use QUIET=1 option to make build.sh output much shorter
Report only fatal errors when building binaries for hwsim testing.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-29 15:49:05 +02:00
Jouni Malinen
31705bf4fc tests: Skip mesh test cases if kernel does have CONFIG_MAC80211_MESH
Previously, these test cases were marked as failures, but it is nicer to
mark as skipped if the kernel does not include support for mesh.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-29 15:49:05 +02:00
Jouni Malinen
e2a8db7785 tests: Report missing erp support more nicely in wpas_ctrl_get_capability
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-29 15:49:05 +02:00
Jouni Malinen
84512f8c9b tests: Support older tshark versions
The -Y command line argument may need to be replaced with -R when an
older tshark version is used.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-29 15:49:05 +02:00
Jouni Malinen
8c6f6ac100 tests: Allow DFS test cases to be skipped
Some of the newer dfs_radar* test cases did not allow hostapd
startup to fail. Since these require relatively recent kernel
support, mark the test cases with skip rather than fail based
on that step failing.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-29 15:49:05 +02:00
Jouni Malinen
752d58c775 tests: Allow SMPS test case to be skipped
ap_ht_smps needs relatively recern kernel support, so allow it to be
skipped rather than claimed as failure, in case hostapd startup fails.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-29 15:49:04 +02:00
Jouni Malinen
6532a84ed3 tests: Fix --codecov cases to find correct wpa_cli/hostapd_cli
It was possible for the separate builds to not include
wpa_cli/hostapd_cli in the default location. Make sure hostapd_cli gets
built for --codecov cases and update both WPACLI and HAPDCLI paths to
match the alternative location.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-29 15:49:04 +02:00
Jouni Malinen
de677292d5 tests: Verify that hostapd-as-RADIUS-server started
Large number of test cases will fail if hostapd fails to start as the
RADIUS server. To make this more obvious, verify that the RADIUS server
instance is running and do not even start test execution if the setup if
not work properly.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-26 13:21:00 +02:00
Jouni Malinen
ed8a1fa496 tests: Limit number of failed test cases in the curses output
It was possible for the scr.addstr() operations to fail and terminate
parallel-vm.py if the number of failed test cases increased beyond what
fits on the screen.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-26 13:21:00 +02:00
Jouni Malinen
c715b31f18 tests: Clear active PBC mode from cfg80211 BSS table in p2p_channel
This reduces number of test combinations in which PBC session overlap
could have been indicated incorrectly.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-25 16:37:12 +02:00
Jouni Malinen
dd859e20ac tests: Make p2p_channel test cases more robust
Wait for a CTRL-EVENT-REGDOM-CHANGE even before returning from
set_country() to avoid issues with test operations being executed before
all components have had chance to update their regulatory domain
information. Some of these test cases could fail under heavy load.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-25 16:37:12 +02:00
Jouni Malinen
1f4de34ec1 tests: Maintain test result counters through the run
This avoids having to go through the full output buffer from each
VM for each update during a test run.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-25 16:37:12 +02:00
Jouni Malinen
8041102874 tests: Process VM stdout output in full lines
Merge partial lines together before processing them in parallel-vm.py.
This avoids issues in cases where the stdout read gets split into pieces
that do not include the full READY/PASS/FAIL/SKIP information. In
addition, strip unnecessary whitespace (mainly, '\r') from the log
lines.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-25 16:37:12 +02:00
Jouni Malinen
1942b68172 tests: Disable terminal echo for stdin control (-i)
This makes vm-run.sh output cleaner to parse for parallel-vm.py.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-25 16:37:12 +02:00
Jouni Malinen
2e0d48e5c8 tests: Do not print to stdout during test execution
This cleans up vm-run.sh output and makes information more easily
available from the *.log files.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-25 16:37:12 +02:00
Jouni Malinen
d3fb9c1403 tests: Add debug logging for parallel-vm.py
parallel-vm.log is now written with details of test execution steps and
results. This makes it easier to debug if something goes wrong in VM
monitoring. The --debug option can be used to enable verbose debugging.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-25 16:36:51 +02:00
Jouni Malinen
3eb1db0377 tests: Retry failed cases automatically in parallel VM run
parallel-vm.py is now retrying failed cases once at the end of the run.
If all the failed test cases passed on the second attempt, that is noted
in the summary output. Results are also indicated as the exit value from
the run: 0 = all cases passed on first run, 1 = some cases failed once,
but everything passed after one retry, 2 = some cases failed did not
succeed at all.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-23 22:25:29 +02:00
Jouni Malinen
a628eeb42b tests: Add more long duration test cases to parallel-vm.py list
This adds the remaining test cases that took more than 15 seconds to run
into the list of test cases to run at the beginning of the execution to
avoid these being left at the end when only some of the VMs may be
running.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-23 20:34:13 +02:00
Jouni Malinen
6d930a1eef tests: Optimize ap_qosmap test cases
Avoid unnecessary DATA_TEST_CONFIG calls and wlantest_cli invocations to
speed up the test cases. This drops ap_qosmap execution time from about
14 seconds to under 3 seconds.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-23 17:46:34 +02:00
Jouni Malinen
4905f1ebfe tests: Optimize multi_check() waits in ap_bss_add* and ap_multi_bss*
There is no need to wait for the longer fixed duration for each STA when
an event message or an earlier wait for another STA can be used instead.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-23 17:20:18 +02:00
Jouni Malinen
610cfff5d7 tests: Make scan_hidden* more robust
Clear cfg80211 BSS table more carefully after the scan_hidden* test
cases. At least scan_hidden_many could have left behind a hidden SSID
entry that could cause problems for following scan_bss_operations test
case.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-23 16:15:11 +02:00
Jouni Malinen
e0cfd223c0 tests: Fix mesh no_auto_peer=1 test cases to allow enough time for retry
If the initial Authentication frame was too early for the peer (i.e.,
NEW_PEER_CANDIDATE event arrived only after the Authentication frame),
wpas_mesh_open_no_auto and wpas_mesh_secure_no_auto test cases were
failing since they waited only for 10 seconds for the connection to be
completed while the retry timer was set to 10-20 seconds on the
authenticator side.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-23 13:10:22 +02:00
Jouni Malinen
9828b70ebd tests: Fix wpas_ctrl_data_test with Linux 3.19-rc1
The kernel commit 'packet: make packet_snd fail on len smaller than l2
header' started rejecting <= 14 octet raw packet socket transmission.
This test case was testing with 14 ocets and that is now rejected by the
kernel. While this may be a kernel side issue, use one octet longer test
data for now to avoid undesired FAIL cases in hwsim tests.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-23 12:27:14 +02:00
Jouni Malinen
b2eb3fe21c tests: Fix ap_ht_40mhz_intolerant_ap
The previous design did not actually break from the wait loop when the
AP changed back to 40 MHz channel and as such, ended up waiting the full
30 second time. Furthermore, the five second delay time for returning
back to 40 MHz was not sufficiently long to test behavior correctly
since the STA did not have any chances of returning the next coex report
before the AP had returned to 40 MHz. Increase the AP wait time to 15
seconds so that the once per 10 seconds OBSS scan from the STA gets in
before changing back to 40 MHz channel (after the 40 MHz intolerant AP
gets disabled).

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-22 22:07:54 +02:00
Jouni Malinen
6ac593ec47 tests: Optimize run-tests.py test case dict iteration
Use a single iteration through the module dictionary rather than
iterating over the keys and separately fetching values to get the
function.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-22 20:08:11 +02:00
Jouni Malinen
4cfe369b98 tests: Optimize stop.sh wait times
There is no need to wait for fixed 0.5 seconds for the processes to
exit.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-22 19:37:08 +02:00
Jouni Malinen
b350009f8c tests: Optimize start.sh execution time
There is no need to wait for one extra second to chown log files when
running as root which is always the case in VM execution.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-22 19:22:42 +02:00
Jouni Malinen
a69d0cdc72 tests: Merge grpform_cred_ready_timeout test cases into a single one
These test cases had a long 120 seconds wait for the GO Negotiation
initiator to time out. This can be done using two devices in parallel to
save two minutes from total test execution time.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-22 15:54:30 +02:00
Jouni Malinen
9be2b811d4 tests: Mesh max peering limit
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-22 11:53:02 +02:00
Jouni Malinen
4b9481bcd8 tests: Remove unnecessary wrap_wpas_mesh_test() complexity
This function did not seem to do anything else apart from making it less
obvious that hwsim_utils.test_connectivity() is called.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-22 11:52:54 +02:00
Jouni Malinen
8b2600329f tests: Mesh beacon_int parameter
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-21 23:45:29 +02:00
Jouni Malinen
5554fbef46 tests: Make parallel VM start a bit more resource friendly
Wait one second between each kvm start to avoid hitting large number of
processes trying to start in parallel. This allows the VMs to be started
more efficiently for parallel-vm.py runs with large number of VMs.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-21 22:04:54 +02:00
Jouni Malinen
b3f4134912 tests: Clear cfg80211 BSS table at the end of scan_hidden
It was possible for the hidden SSID entry to leak into following test
cases and result in failures, e.g., in this sequence:

scan_hidden
scan_reqs_with_non_scan_radio_work
radius_auth_unreachable
wep_open_auth

BSS <BSSID> picked incorrect BSS entry within wpa_supplicant due to the
old zero-length SSID showing up from the earlier hidden SSID case.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-21 21:58:47 +02:00
Jouni Malinen
2ffbf34db0 tests: Optimize test case execution order for multiple VMs
Move test cases with long duration to the beginning as an optimization
to avoid last part of the test execution running a long duration test
case on a single VM while all other VMs have already completed their
work.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-21 18:20:15 +02:00
Jouni Malinen
01c87519a3 tests: Do not mark proxyarp_open failed if ebtables is not available
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-21 17:14:00 +02:00
Jouni Malinen
f24489d94f tests: Use new scripts for vm-run.sh codecov
Now there is only one set of commands to maintain. The separate reports
for individual components have not been of much use in the past, so they
are dropped for now.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-21 16:41:59 +02:00
Jouni Malinen
cb0555f7be tests: EAP-IKEv2 protocol tests
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-21 16:16:26 +02:00
Jouni Malinen
f41f670ea5 tests: ERP with EAP-IKEv2
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-20 23:48:53 +02:00
Jouni Malinen
f1381f99e4 tests: Mesh ht_mode options
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-20 21:27:22 +02:00
Jouni Malinen
f86868c253 tests: wdev_id in STATUS-DRIVER
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-20 21:18:26 +02:00
Jouni Malinen
542452a93d tests: hostapd behavior with interface in different bridge
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-20 21:01:31 +02:00
Jouni Malinen
a42f88f8e7 tests: bg_scan_period parameter configuration
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-20 19:44:40 +02:00
Jouni Malinen
6343f24d35 tests: IBSS cfg80211 workarounds
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-20 19:36:41 +02:00
Jouni Malinen
cece9b7df7 tests: Dynamic BSS add operation reusing existing interface
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-20 19:04:18 +02:00
Jouni Malinen
8916857da6 tests: TDLS with VHT
This adds some more coverage, but the TDLS setup is not for VHT due to
missing support in mac80211.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-20 18:44:48 +02:00
Jouni Malinen
4466fe6541 tests: VHT and 40 MHz channel
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-20 18:44:48 +02:00
Jouni Malinen
d4d80e9a14 tests: SMPS AP configuration options
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-20 18:44:48 +02:00
Jouni Malinen
598555c7e8 tests: IEEE 802.1X with static WEP
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-20 18:44:48 +02:00
Jouni Malinen
1e7528298a tests: cfg80211 scan-for-auth workaround with WEP keys
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-20 17:28:53 +02:00
Jouni Malinen
a5687274e1 tests: cfg80211 scan-for-auth workaround failing
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-20 17:22:18 +02:00
Jouni Malinen
b97a7bf672 tests: Large number of BSS add operations with hostapd
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-20 17:13:00 +02:00
Jouni Malinen
863a54093d tests: rfkill block prior to wpa_supplicant start
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-20 16:46:39 +02:00
Jouni Malinen
d92da8a23e tests: Radar detection on non-HT and on HT40- channels
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-20 13:50:19 +02:00
Jouni Malinen
cac5526390 tests: Report ROAM failure in pmksa_cache_opportunistic_multiple_sta
There is no need to wait for the timeout if the ROAM command itself
failed. This could happen if an earlier test case had left hidden SSIDs
in the cfg80211 BSS table.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-20 13:37:08 +02:00
Jouni Malinen
1264b68355 tests: Clear cfg80211 scan results after wext_hidden
Leaving hidden SSIDs in the cfg80211 BSS table can result in errors in
the following test cases, so use special care to clear all BSS entries
at the end of the wext_hidden test case.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-20 13:36:01 +02:00
Jouni Malinen
5f35a5e27f tests: Add wait_connected() and wait_disconnected() helpers
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-20 13:10:09 +02:00
Jouni Malinen
e0cccf26a4 tests: Add --codecov support to parallel-vm.py
This allows code coverage report to be generated must faster with the
help of parallel VMs executing test cases.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-20 01:24:28 +02:00
Jouni Malinen
42ce1a27e0 tests: Connection with large number of scan_ssid=1 network blocks
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-19 23:30:58 +02:00
Jouni Malinen
ed930f064c tests: P2P_LISTEN and scan
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-19 23:17:06 +02:00
Jouni Malinen
558ac77c58 tests: setband and scans
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-19 23:08:10 +02:00
Jouni Malinen
7749b1c982 tests: Additional coverage for filter_ssids=1
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-19 20:26:07 +02:00
Jouni Malinen
91cc6f918a tests: AP_SCAN 2 operation
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-19 20:08:58 +02:00
Jouni Malinen
83e80d32f1 tests: SCAN requests while in non-scan radio work
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
2014-12-19 15:23:07 +02:00
Jouni Malinen
7c5c1ba7b4 tests: P2P_LISTEN behavior with offchannel TX
This verifies that offchannel TX (PD in this specific case) does not
stop ongoing long P2P_LISTEN operation.

Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
2014-12-18 16:41:40 +02:00
Jouni Malinen
89cd4355bb tests: Fix proxyarp_open IPv4-from-bridge
There were couple of typos in the IP addresses and there was no coverage
for the normal unicast ARP response from the bridge (since non-wireless
device does not get proxied by the AP). In addition, it is better to
change the IP address used here to be unique to make the sniffer logs
easier to interpret.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-18 16:41:40 +02:00
Jouni Malinen
6d5332948c tests: Use send_ns() helper for the earlier proxyarp_open NS packets
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
2014-12-18 16:41:40 +02:00
Jouni Malinen
40e4c9c8b0 tests: Add traffic test to proxyarp_open
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
2014-12-18 16:41:40 +02:00
Jouni Malinen
5a3ce80265 tests: ProxyARP behavior with IPv6 frames
This adds transmission of number of NS/NA frames to test ProxyARP
behavior. The actual validation of the AP behavior is still manual,
i.e., a separate inspectation of the capture files is needed.

Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
2014-12-18 16:41:40 +02:00
Jouni Malinen
19cbe06238 tests: Add ebtables rules for ProxyARP
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
2014-12-18 16:41:40 +02:00
Jouni Malinen
a95c6973aa tests: Use non-promiscuous mode and packet-buffered tcpdump capture
This helps in avoiding truncated capture files in the proxyarp_open test
case.

Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
2014-12-18 16:41:40 +02:00
Jouni Malinen
1d0b5a04e6 tests: Dump monitor messages before hwsim_utils test
This may be needed in some corner cases where broadcast frames with two
associated stations are received by both devices.

Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
2014-12-18 16:41:40 +02:00
Jouni Malinen
db1d66d879 tests: Fix wifi_display_persistent_group event checks
The final invitation case in this test was verified incorrectly. The GO
was already operatign in that case, so there was not supposed to be a
new P2P-GROUP-STARTED message from GO. That happened to be show up in
the pending event messages from the last instance, but that was just by
accident and any additional dump_monitor() operation added here would
cause this test to fail. Fix this by handling the final invitation
separately and verifying that only the client side indicates
P2P-GROUP-STARTED.

Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
2014-12-18 16:41:40 +02:00
Jouni Malinen
c8836a4f03 tests: Allow dump_monitor() to be used without global monitor interface
It is possible for WpaSupplicant instance to be used without the global
control socket, so allow dump_monitor() to handle this case cleanly.

Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
2014-12-18 16:01:41 +02:00
Jouni Malinen
9d7fdac5b3 tests: Try to clear scan results after regulatory domain changes
It was possible for regulatory domain changes to "leak" into following
test cases in number of cases where the cfg80211 BSS table remained
after regulatory domain had been restored to world roaming. Try to make
this less likely to occur by explicitly clearing BSS table at the end of
test cases that use different regulatory domain. This makes P2P test
cases that verify channel selection based on world roaming rules more
robust.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-15 01:32:01 +02:00
Jouni Malinen
970a23f665 tests: Make flush_scan_cache() more robust
It was possible for BSS entries on the scanned channel (2412 MHz) to be
left behind after flush_scan_cache() call. Use a less likely channel
2417 MHz as the default channel to scan. This will hopefully get rid of
most problematic BSS entries from previous test cases. For example,
ap_hs20_oen followed by ap_hs20_random_mac could fail due to
INTERWORKING_CONNECT seeing non-RSN scan result from the previous test
case.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-15 01:13:03 +02:00
Jouni Malinen
d48b64ba6c tests: Use helper function for adding open mesh network
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-14 23:34:43 +02:00
Jouni Malinen
7e3614df5c tests: Mesh with dynamic interface
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-14 23:28:36 +02:00
Jouni Malinen
5881873b86 tests: Make pmksa_cache_opportunistic_connect more robust
Use scan_for_bss() instead of scan(freq) to avoid false positives due to
active scan failing under heavy load.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-14 19:15:48 +02:00
Jouni Malinen
acb63c753b tests: SAE with missing password
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-14 17:09:14 +02:00
Jouni Malinen
0e40d7daa4 tests: ERP when server has dropped the keys
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-14 15:47:04 +02:00
Jouni Malinen
777bbe7a3c tests: P2P_PRESENCE_REQ on group interface
This ends up using the special offchannel.c code path where a different
interface is selected for TX. In addition, the P2P-PRESENCE-RESPONSE
event is verified to be delivered on the group interface.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-14 15:47:04 +02:00
Jouni Malinen
11ace2ed1f tests: Mesh and missing SAE password
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-14 15:47:04 +02:00
Jouni Malinen
fba65f7240 tests: Mesh BSS data
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-14 12:55:43 +02:00
Jouni Malinen
54cacef5eb tests: Mesh sae_groups configuration
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-14 12:55:40 +02:00
Jouni Malinen
e9026725f7 tests: Extra coverage for command line arguments
The results for these are not currently verified, but this allows
--codecov runs to get more coverage for the command line argument
parsers.

Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-14 12:24:15 +02:00
Jouni Malinen
e869f812d3 tests: DEAUTHENTICATE/DISASSCIATE/CHAN_SWITCH error cases
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-14 02:28:03 +02:00
Jouni Malinen
69f9912310 tests: Additional FETCH_OSU and CANCEL_FETCH_OSU coverage
Signed-off-by: Jouni Malinen <j@w1.fi>
2014-12-14 02:28:03 +02:00