The previous version was enabling all three stations at the same time
and left dev[1] and dev[2] competing on getting connected with dev[0]
that allowed only one pairing. This was not exactly robust and the pass
criteria depended on an extra event from either dev[1] or dev[2]. Fix
that by first connecting dev[0] and dev[1] and only after that, start
dev[2]. This allows proper validation of both the peering limit on
dev[0] and no extra event on dev[2].
Signed-off-by: Jouni Malinen <j@w1.fi>
It is possible for the Probe Response frame wait to time out when active
scanning is used under heavy CPU load. Make this test case more robust
by trying multiple times before declaring the scan for a hidden SSID to
have failed.
Signed-off-by: Jouni Malinen <j@w1.fi>
This allows automated testing of the wpa_supplicant D-Bus interface. The
instance controlling wlan0 registers with D-Bus if dbus-daemon was
started successfully. This is only used in VM testing, i.e., not when
run-tests.sh is used on the host system with D-Bus running for normal
system purposes.
Signed-off-by: Jouni Malinen <j@w1.fi>
wpa_cli and hostapd_cli are not currently tested for code coverage, so
filter the files specific to those components away from the code
coverage reports. *_module_tests.c are not included in normal builds, so
drop them as well. In addition, drop the system header file (byteswap.h)
that gets somehow unnecessarily included in the reports for couple of
lines.
Signed-off-by: Jouni Malinen <j@w1.fi>
This leaves the build tree with valid wpa_supplicant object files in the
end and makes it a bit easier to do additional custom builds when
needed.
Signed-off-by: Jouni Malinen <j@w1.fi>
Previously, these test cases were marked as failures, but it is nicer to
mark as skipped if the kernel does not include support for mesh.
Signed-off-by: Jouni Malinen <j@w1.fi>
Some of the newer dfs_radar* test cases did not allow hostapd
startup to fail. Since these require relatively recent kernel
support, mark the test cases with skip rather than fail based
on that step failing.
Signed-off-by: Jouni Malinen <j@w1.fi>
ap_ht_smps needs relatively recern kernel support, so allow it to be
skipped rather than claimed as failure, in case hostapd startup fails.
Signed-off-by: Jouni Malinen <j@w1.fi>
It was possible for the separate builds to not include
wpa_cli/hostapd_cli in the default location. Make sure hostapd_cli gets
built for --codecov cases and update both WPACLI and HAPDCLI paths to
match the alternative location.
Signed-off-by: Jouni Malinen <j@w1.fi>
Large number of test cases will fail if hostapd fails to start as the
RADIUS server. To make this more obvious, verify that the RADIUS server
instance is running and do not even start test execution if the setup if
not work properly.
Signed-off-by: Jouni Malinen <j@w1.fi>
It was possible for the scr.addstr() operations to fail and terminate
parallel-vm.py if the number of failed test cases increased beyond what
fits on the screen.
Signed-off-by: Jouni Malinen <j@w1.fi>
Wait for a CTRL-EVENT-REGDOM-CHANGE even before returning from
set_country() to avoid issues with test operations being executed before
all components have had chance to update their regulatory domain
information. Some of these test cases could fail under heavy load.
Signed-off-by: Jouni Malinen <j@w1.fi>
Merge partial lines together before processing them in parallel-vm.py.
This avoids issues in cases where the stdout read gets split into pieces
that do not include the full READY/PASS/FAIL/SKIP information. In
addition, strip unnecessary whitespace (mainly, '\r') from the log
lines.
Signed-off-by: Jouni Malinen <j@w1.fi>
parallel-vm.log is now written with details of test execution steps and
results. This makes it easier to debug if something goes wrong in VM
monitoring. The --debug option can be used to enable verbose debugging.
Signed-off-by: Jouni Malinen <j@w1.fi>
parallel-vm.py is now retrying failed cases once at the end of the run.
If all the failed test cases passed on the second attempt, that is noted
in the summary output. Results are also indicated as the exit value from
the run: 0 = all cases passed on first run, 1 = some cases failed once,
but everything passed after one retry, 2 = some cases failed did not
succeed at all.
Signed-off-by: Jouni Malinen <j@w1.fi>
This adds the remaining test cases that took more than 15 seconds to run
into the list of test cases to run at the beginning of the execution to
avoid these being left at the end when only some of the VMs may be
running.
Signed-off-by: Jouni Malinen <j@w1.fi>
Avoid unnecessary DATA_TEST_CONFIG calls and wlantest_cli invocations to
speed up the test cases. This drops ap_qosmap execution time from about
14 seconds to under 3 seconds.
Signed-off-by: Jouni Malinen <j@w1.fi>
There is no need to wait for the longer fixed duration for each STA when
an event message or an earlier wait for another STA can be used instead.
Signed-off-by: Jouni Malinen <j@w1.fi>
Clear cfg80211 BSS table more carefully after the scan_hidden* test
cases. At least scan_hidden_many could have left behind a hidden SSID
entry that could cause problems for following scan_bss_operations test
case.
Signed-off-by: Jouni Malinen <j@w1.fi>
If the initial Authentication frame was too early for the peer (i.e.,
NEW_PEER_CANDIDATE event arrived only after the Authentication frame),
wpas_mesh_open_no_auto and wpas_mesh_secure_no_auto test cases were
failing since they waited only for 10 seconds for the connection to be
completed while the retry timer was set to 10-20 seconds on the
authenticator side.
Signed-off-by: Jouni Malinen <j@w1.fi>
The kernel commit 'packet: make packet_snd fail on len smaller than l2
header' started rejecting <= 14 octet raw packet socket transmission.
This test case was testing with 14 ocets and that is now rejected by the
kernel. While this may be a kernel side issue, use one octet longer test
data for now to avoid undesired FAIL cases in hwsim tests.
Signed-off-by: Jouni Malinen <j@w1.fi>
The previous design did not actually break from the wait loop when the
AP changed back to 40 MHz channel and as such, ended up waiting the full
30 second time. Furthermore, the five second delay time for returning
back to 40 MHz was not sufficiently long to test behavior correctly
since the STA did not have any chances of returning the next coex report
before the AP had returned to 40 MHz. Increase the AP wait time to 15
seconds so that the once per 10 seconds OBSS scan from the STA gets in
before changing back to 40 MHz channel (after the 40 MHz intolerant AP
gets disabled).
Signed-off-by: Jouni Malinen <j@w1.fi>
Use a single iteration through the module dictionary rather than
iterating over the keys and separately fetching values to get the
function.
Signed-off-by: Jouni Malinen <j@w1.fi>
There is no need to wait for one extra second to chown log files when
running as root which is always the case in VM execution.
Signed-off-by: Jouni Malinen <j@w1.fi>
These test cases had a long 120 seconds wait for the GO Negotiation
initiator to time out. This can be done using two devices in parallel to
save two minutes from total test execution time.
Signed-off-by: Jouni Malinen <j@w1.fi>
This function did not seem to do anything else apart from making it less
obvious that hwsim_utils.test_connectivity() is called.
Signed-off-by: Jouni Malinen <j@w1.fi>
Wait one second between each kvm start to avoid hitting large number of
processes trying to start in parallel. This allows the VMs to be started
more efficiently for parallel-vm.py runs with large number of VMs.
Signed-off-by: Jouni Malinen <j@w1.fi>
It was possible for the hidden SSID entry to leak into following test
cases and result in failures, e.g., in this sequence:
scan_hidden
scan_reqs_with_non_scan_radio_work
radius_auth_unreachable
wep_open_auth
BSS <BSSID> picked incorrect BSS entry within wpa_supplicant due to the
old zero-length SSID showing up from the earlier hidden SSID case.
Signed-off-by: Jouni Malinen <j@w1.fi>
Move test cases with long duration to the beginning as an optimization
to avoid last part of the test execution running a long duration test
case on a single VM while all other VMs have already completed their
work.
Signed-off-by: Jouni Malinen <j@w1.fi>
Now there is only one set of commands to maintain. The separate reports
for individual components have not been of much use in the past, so they
are dropped for now.
Signed-off-by: Jouni Malinen <j@w1.fi>
There is no need to wait for the timeout if the ROAM command itself
failed. This could happen if an earlier test case had left hidden SSIDs
in the cfg80211 BSS table.
Signed-off-by: Jouni Malinen <j@w1.fi>
Leaving hidden SSIDs in the cfg80211 BSS table can result in errors in
the following test cases, so use special care to clear all BSS entries
at the end of the wext_hidden test case.
Signed-off-by: Jouni Malinen <j@w1.fi>
This allows code coverage report to be generated must faster with the
help of parallel VMs executing test cases.
Signed-off-by: Jouni Malinen <j@w1.fi>
This verifies that offchannel TX (PD in this specific case) does not
stop ongoing long P2P_LISTEN operation.
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
There were couple of typos in the IP addresses and there was no coverage
for the normal unicast ARP response from the bridge (since non-wireless
device does not get proxied by the AP). In addition, it is better to
change the IP address used here to be unique to make the sniffer logs
easier to interpret.
Signed-off-by: Jouni Malinen <j@w1.fi>
This adds transmission of number of NS/NA frames to test ProxyARP
behavior. The actual validation of the AP behavior is still manual,
i.e., a separate inspectation of the capture files is needed.
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
This may be needed in some corner cases where broadcast frames with two
associated stations are received by both devices.
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
The final invitation case in this test was verified incorrectly. The GO
was already operatign in that case, so there was not supposed to be a
new P2P-GROUP-STARTED message from GO. That happened to be show up in
the pending event messages from the last instance, but that was just by
accident and any additional dump_monitor() operation added here would
cause this test to fail. Fix this by handling the final invitation
separately and verifying that only the client side indicates
P2P-GROUP-STARTED.
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
It is possible for WpaSupplicant instance to be used without the global
control socket, so allow dump_monitor() to handle this case cleanly.
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
It was possible for regulatory domain changes to "leak" into following
test cases in number of cases where the cfg80211 BSS table remained
after regulatory domain had been restored to world roaming. Try to make
this less likely to occur by explicitly clearing BSS table at the end of
test cases that use different regulatory domain. This makes P2P test
cases that verify channel selection based on world roaming rules more
robust.
Signed-off-by: Jouni Malinen <j@w1.fi>
It was possible for BSS entries on the scanned channel (2412 MHz) to be
left behind after flush_scan_cache() call. Use a less likely channel
2417 MHz as the default channel to scan. This will hopefully get rid of
most problematic BSS entries from previous test cases. For example,
ap_hs20_oen followed by ap_hs20_random_mac could fail due to
INTERWORKING_CONNECT seeing non-RSN scan result from the previous test
case.
Signed-off-by: Jouni Malinen <j@w1.fi>
This ends up using the special offchannel.c code path where a different
interface is selected for TX. In addition, the P2P-PRESENCE-RESPONSE
event is verified to be delivered on the group interface.
Signed-off-by: Jouni Malinen <j@w1.fi>
The results for these are not currently verified, but this allows
--codecov runs to get more coverage for the command line argument
parsers.
Signed-off-by: Jouni Malinen <j@w1.fi>
This avoids one more cleanup step between most test cases by clearing
the default PMF behavior change only in case it was actually modified
during a test.
Signed-off-by: Jouni Malinen <j@w1.fi>
This avoids one more cleanup step between most test cases by stopping ER
only in case it was actually used during a test.
Signed-off-by: Jouni Malinen <j@w1.fi>
This uses a new testing mode in hostapd to allow RRM neighbor request
transmittion to be tested. For the second part of the test case to be
executed, mac80211_hwsim needs to be modified to claim support for the
required RRM capabilities (that change is not yet in Linux kernel).
Signed-off-by: Jouni Malinen <j@w1.fi>
This verifies processing of various ARP messages at an AP that enables
ProxyARP. All the validation steps have not yet been scripted, i.e., the
sniffer traces need manual analysis for full coverage.
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
This provides a regression test that would have caught the recent
issue with tls_openssl.c change breaking EAP-FAST.
Signed-off-by: Jouni Malinen <j@w1.fi>
This verifies that the corner case of a duplicated, retransmitted
Invitation Response frame ends up being dropped instead of being
processed twice for the case of Invitation Request getting resend with
social channel as an operating channel in case of no common channels
found.
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
This adds P2P_SERV_DISC_REQ, P2P_SERVICE_ADD, and P2P_SERVICE_DEL error
cases and P2P_SERVICE_FLUSH and P2P_SERC_DISC_EXTERNAL testing.
Signed-off-by: Jouni Malinen <j@w1.fi>
A single channel scan just before WPS_REG, WPS_PBC, and WPS_PIN commands
can be used to avoid having to run a full scan. This saves significant
amount of time in the WPS test cases.
Signed-off-by: Jouni Malinen <j@w1.fi>
This increases wpa_supplicant_ie_txt(), print_bss_info(), and
wpa_supplicant_ctrl_iface_scan_result() testing coverage to include the
previously missing key management options.
Signed-off-by: Jouni Malinen <j@w1.fi>
This verifies that IBSS_RSN <peer> return OK for the case where a valid
and already connection peer address is given.
Signed-off-by: Jouni Malinen <j@w1.fi>
There is no need to pass the test case names to the VMs when using
parallel-vm.py. Removing those from the command line helps in avoiding
kernel panic if maximum number of kernel parameters limit is hit.
Signed-off-by: Jouni Malinen <j@w1.fi>
Use single channel scan instead of full scan to save time. In addition,
use EAP-GPSK which takes significantly less CPU that EAP-EKE with
default parameters.
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
This tests RP EAP-Initiate/Re-auth-Start transmission, ERP key
derivation, and EAP-Initiate/Re-auth + EAP-Finish/Re-auth exchange and
rMSK derivation.
Signed-off-by: Jouni Malinen <j@w1.fi>
It was apparently possible for this test case not to do what it was
supposed to do, i.e., get nl80211 Authenticate command failing due to
cfg80211 BS entry missing. With the external radio work blocking fixed,
this can be cleaned up by explicitly waiting for the scan event. In
addition, a less used channel can be selected to avoid finding other BSS
entries.
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
This makes it more convenient and consistent to clear the cached scan
results from cfg80211 and wpa_supplicant.
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
Do not leave the station trying to connect at the end of the test case
since that can have an effect to the following test case. Such sequences
should be tested in test cases that are specifically designed for that
rather than randomly between test cases.
Signed-off-by: Jouni Malinen <j@w1.fi>
Since run-tests.py is now required to run as root, there is no need to
use the somewhat awkward "sudo tee" construction; instead, just write
directly to the file.
Signed-off-by: Jouni Malinen <j@w1.fi>
Clear scan results to avoid PBC overlap issues caused by earlier test
cases. For example, go_neg_with_bss_connected followed by
go_neg_with_bss_on_disallowed_chan resulted in failure before this
change.
Signed-off-by: Jouni Malinen <j@w1.fi>
This test case was failing every now and then due to dev1
(no_auto_peer=1) not receiving the new-peer-candidate event in time
before dev0 has already stopped retries on mesh peering open message.
This sounds somewhat expected with the default 4 * 40 ms = 160 ms
retries and 1000 TU beacon interval. Use maximum timeout 16 * 255 ms =
4080 ms to make this test case less likely to fail.
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
Both the wpa_supplicant and kernel configuration need wext to
run the wext testcase, enable those in the default/example
configurations.
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
Keep full channel scans in autogo for test coverage, but use single
channel scan in all other autogo* test cases to remove unnecessary
waiting that does not add any test coverage. This removes more than one
minute from the total test execution time.
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
Replace fixed sleep with waiting for a disconnection event. In addition,
remove unnecessary use of sudo.
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
It was possible for this test case to fail due to PBC overlap that was
detected based on previous test case having used PBC. Make that false
positive less likely to happen by explicitly clearing the scan cache on
the device.
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
p2p_oper_reg_class and p2p_oper_channel need to be restored to 0, not an
arbitrary 2.4 GHz channel to avoid issues for following test cases. For
example, p2p_channel_random_social_with_op_class_change followed by
p2p_autogo_pref_chan_not_in_regulatory ended up with the latter test
case failing.
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
hwsim_utils.test_connectivity() is already bidirectional test, so there
is no need to run it twice with the devices swapped for the second
iteration.
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
Clear wpa_supplicant and cfg80211 scan results at the end of hidden SSID
test cases to avoid potential issues for following test cases. For
example, ap_hs20_session_info could fail if executed immediately after
ssid_hidden due to the AP configuration change from open to RSN was not
noticed.
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
This is more robust and extensible than configuring IPv6 addresses on
the interfaces and trying to use ping6 or some other external tools to
generate suitable IPv6 frames.
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
The 50 ms wait time for Action frame response was not sufficient when
the tests are run under heavy load (e.g., multiple VMs in parallel).
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
Wait for GO to start before sending invitation frames in the protocol
test. Without this, it was possible to hit the 5 second timeout on
management frame RX under load.
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
It was possible for this test case to fail due to PBC overlap that was
detected based on previous test case having used PBC. Make that false
positive less likely to happen by explicitly clearing the scan cache on
the devices.
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
The "Not fully connected" report did not clearly identify what went
wrong, so make this more verbose in hope of being able to determine what
happened should this test case fail again.
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
It was possible to hit the 10 second timeout in some test cases under
heavy load (e.g., with large number of VMs running tests in parallel).
These timeouts are not really indicating any real error, so make them
less likely to show up in reports by increasing the connection timeout
to 30 seconds.
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
There are valid status (and possibly status-driver) responses
that don't have a name=value format, ignore those in the test
framework parser.
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
