It was possible for the scr.addstr() operations to fail and terminate
parallel-vm.py if the number of failed test cases increased beyond what
fits on the screen.
Signed-off-by: Jouni Malinen <j@w1.fi>
Wait for a CTRL-EVENT-REGDOM-CHANGE even before returning from
set_country() to avoid issues with test operations being executed before
all components have had chance to update their regulatory domain
information. Some of these test cases could fail under heavy load.
Signed-off-by: Jouni Malinen <j@w1.fi>
Merge partial lines together before processing them in parallel-vm.py.
This avoids issues in cases where the stdout read gets split into pieces
that do not include the full READY/PASS/FAIL/SKIP information. In
addition, strip unnecessary whitespace (mainly, '\r') from the log
lines.
Signed-off-by: Jouni Malinen <j@w1.fi>
parallel-vm.log is now written with details of test execution steps and
results. This makes it easier to debug if something goes wrong in VM
monitoring. The --debug option can be used to enable verbose debugging.
Signed-off-by: Jouni Malinen <j@w1.fi>
parallel-vm.py is now retrying failed cases once at the end of the run.
If all the failed test cases passed on the second attempt, that is noted
in the summary output. Results are also indicated as the exit value from
the run: 0 = all cases passed on first run, 1 = some cases failed once,
but everything passed after one retry, 2 = some cases failed did not
succeed at all.
Signed-off-by: Jouni Malinen <j@w1.fi>
This adds the remaining test cases that took more than 15 seconds to run
into the list of test cases to run at the beginning of the execution to
avoid these being left at the end when only some of the VMs may be
running.
Signed-off-by: Jouni Malinen <j@w1.fi>
Avoid unnecessary DATA_TEST_CONFIG calls and wlantest_cli invocations to
speed up the test cases. This drops ap_qosmap execution time from about
14 seconds to under 3 seconds.
Signed-off-by: Jouni Malinen <j@w1.fi>
There is no need to wait for the longer fixed duration for each STA when
an event message or an earlier wait for another STA can be used instead.
Signed-off-by: Jouni Malinen <j@w1.fi>
Clear cfg80211 BSS table more carefully after the scan_hidden* test
cases. At least scan_hidden_many could have left behind a hidden SSID
entry that could cause problems for following scan_bss_operations test
case.
Signed-off-by: Jouni Malinen <j@w1.fi>
If the initial Authentication frame was too early for the peer (i.e.,
NEW_PEER_CANDIDATE event arrived only after the Authentication frame),
wpas_mesh_open_no_auto and wpas_mesh_secure_no_auto test cases were
failing since they waited only for 10 seconds for the connection to be
completed while the retry timer was set to 10-20 seconds on the
authenticator side.
Signed-off-by: Jouni Malinen <j@w1.fi>
The kernel commit 'packet: make packet_snd fail on len smaller than l2
header' started rejecting <= 14 octet raw packet socket transmission.
This test case was testing with 14 ocets and that is now rejected by the
kernel. While this may be a kernel side issue, use one octet longer test
data for now to avoid undesired FAIL cases in hwsim tests.
Signed-off-by: Jouni Malinen <j@w1.fi>
The previous design did not actually break from the wait loop when the
AP changed back to 40 MHz channel and as such, ended up waiting the full
30 second time. Furthermore, the five second delay time for returning
back to 40 MHz was not sufficiently long to test behavior correctly
since the STA did not have any chances of returning the next coex report
before the AP had returned to 40 MHz. Increase the AP wait time to 15
seconds so that the once per 10 seconds OBSS scan from the STA gets in
before changing back to 40 MHz channel (after the 40 MHz intolerant AP
gets disabled).
Signed-off-by: Jouni Malinen <j@w1.fi>
Use a single iteration through the module dictionary rather than
iterating over the keys and separately fetching values to get the
function.
Signed-off-by: Jouni Malinen <j@w1.fi>
There is no need to wait for one extra second to chown log files when
running as root which is always the case in VM execution.
Signed-off-by: Jouni Malinen <j@w1.fi>
These test cases had a long 120 seconds wait for the GO Negotiation
initiator to time out. This can be done using two devices in parallel to
save two minutes from total test execution time.
Signed-off-by: Jouni Malinen <j@w1.fi>
This function did not seem to do anything else apart from making it less
obvious that hwsim_utils.test_connectivity() is called.
Signed-off-by: Jouni Malinen <j@w1.fi>
Wait one second between each kvm start to avoid hitting large number of
processes trying to start in parallel. This allows the VMs to be started
more efficiently for parallel-vm.py runs with large number of VMs.
Signed-off-by: Jouni Malinen <j@w1.fi>
It was possible for the hidden SSID entry to leak into following test
cases and result in failures, e.g., in this sequence:
scan_hidden
scan_reqs_with_non_scan_radio_work
radius_auth_unreachable
wep_open_auth
BSS <BSSID> picked incorrect BSS entry within wpa_supplicant due to the
old zero-length SSID showing up from the earlier hidden SSID case.
Signed-off-by: Jouni Malinen <j@w1.fi>
Move test cases with long duration to the beginning as an optimization
to avoid last part of the test execution running a long duration test
case on a single VM while all other VMs have already completed their
work.
Signed-off-by: Jouni Malinen <j@w1.fi>
Now there is only one set of commands to maintain. The separate reports
for individual components have not been of much use in the past, so they
are dropped for now.
Signed-off-by: Jouni Malinen <j@w1.fi>
There is no need to wait for the timeout if the ROAM command itself
failed. This could happen if an earlier test case had left hidden SSIDs
in the cfg80211 BSS table.
Signed-off-by: Jouni Malinen <j@w1.fi>
Leaving hidden SSIDs in the cfg80211 BSS table can result in errors in
the following test cases, so use special care to clear all BSS entries
at the end of the wext_hidden test case.
Signed-off-by: Jouni Malinen <j@w1.fi>
This allows code coverage report to be generated must faster with the
help of parallel VMs executing test cases.
Signed-off-by: Jouni Malinen <j@w1.fi>
This verifies that offchannel TX (PD in this specific case) does not
stop ongoing long P2P_LISTEN operation.
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
There were couple of typos in the IP addresses and there was no coverage
for the normal unicast ARP response from the bridge (since non-wireless
device does not get proxied by the AP). In addition, it is better to
change the IP address used here to be unique to make the sniffer logs
easier to interpret.
Signed-off-by: Jouni Malinen <j@w1.fi>
This adds transmission of number of NS/NA frames to test ProxyARP
behavior. The actual validation of the AP behavior is still manual,
i.e., a separate inspectation of the capture files is needed.
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
This may be needed in some corner cases where broadcast frames with two
associated stations are received by both devices.
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
The final invitation case in this test was verified incorrectly. The GO
was already operatign in that case, so there was not supposed to be a
new P2P-GROUP-STARTED message from GO. That happened to be show up in
the pending event messages from the last instance, but that was just by
accident and any additional dump_monitor() operation added here would
cause this test to fail. Fix this by handling the final invitation
separately and verifying that only the client side indicates
P2P-GROUP-STARTED.
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
It is possible for WpaSupplicant instance to be used without the global
control socket, so allow dump_monitor() to handle this case cleanly.
Signed-off-by: Jouni Malinen <jouni@qca.qualcomm.com>
It was possible for regulatory domain changes to "leak" into following
test cases in number of cases where the cfg80211 BSS table remained
after regulatory domain had been restored to world roaming. Try to make
this less likely to occur by explicitly clearing BSS table at the end of
test cases that use different regulatory domain. This makes P2P test
cases that verify channel selection based on world roaming rules more
robust.
Signed-off-by: Jouni Malinen <j@w1.fi>
It was possible for BSS entries on the scanned channel (2412 MHz) to be
left behind after flush_scan_cache() call. Use a less likely channel
2417 MHz as the default channel to scan. This will hopefully get rid of
most problematic BSS entries from previous test cases. For example,
ap_hs20_oen followed by ap_hs20_random_mac could fail due to
INTERWORKING_CONNECT seeing non-RSN scan result from the previous test
case.
Signed-off-by: Jouni Malinen <j@w1.fi>
This ends up using the special offchannel.c code path where a different
interface is selected for TX. In addition, the P2P-PRESENCE-RESPONSE
event is verified to be delivered on the group interface.
Signed-off-by: Jouni Malinen <j@w1.fi>
The results for these are not currently verified, but this allows
--codecov runs to get more coverage for the command line argument
parsers.
Signed-off-by: Jouni Malinen <j@w1.fi>
This avoids one more cleanup step between most test cases by clearing
the default PMF behavior change only in case it was actually modified
during a test.
Signed-off-by: Jouni Malinen <j@w1.fi>
This avoids one more cleanup step between most test cases by stopping ER
only in case it was actually used during a test.
Signed-off-by: Jouni Malinen <j@w1.fi>
This uses a new testing mode in hostapd to allow RRM neighbor request
transmittion to be tested. For the second part of the test case to be
executed, mac80211_hwsim needs to be modified to claim support for the
required RRM capabilities (that change is not yet in Linux kernel).
Signed-off-by: Jouni Malinen <j@w1.fi>
