Commit graph

136 commits

Author SHA1 Message Date
Jouni Malinen
829f3cd2b2 tests: Fix the previous update of the regulatory database to VMs
The last update of the wireless-regdb database to the wireless-regdb.git
version of 2023-02-13 in commit c4034a69fe ("tests: Update regulatory
database to VMs") forgot to update regulatory.db.p7s. Update it as well.

Signed-off-by: Jouni Malinen <quic_jouni@quicinc.com>
2023-06-06 20:11:06 +03:00
Jouni Malinen
c4034a69fe tests: Update regulatory database to VMs
Update the wireless-regdb database to the wireless-regdb.git version of
2023-02-13.

Signed-off-by: Jouni Malinen <quic_jouni@quicinc.com>
2023-04-26 23:10:03 +03:00
Jouni Malinen
a70a4672d8 tests: Allow more VMs to be used that there are screen lines
curses prints were causing parallel-vm.py to terminate if there were too
many VMs to fit into the screen. For now, simply hide any VMs from the
live status if there is not sufficient room for them.

Signed-off-by: Jouni Malinen <quic_jouni@quicinc.com>
2023-01-25 23:47:33 +02:00
Johannes Berg
48c7e04be6 tests: Add mode for running UML kernel under gdb
The new --gdb option can be used when KERNELDIR (and optionally
MODULEDIR) are set and we therefore run UML. It runs the entire
VM under the debugger, with a script to load the right modules
into gdb so you can debug easily.

This needs CONFIG_GDB_SCRIPTS=y to be used in the kernel build.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
2022-11-28 15:43:00 +02:00
Johannes Berg
550ab94e83 tests: vm: Simplify parameter passing
Since the kernel  actually passes the command-line parameters
as environment variables to the init script, there's no need
to parse them out of /proc/cmdline.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
2022-09-16 21:28:23 +03:00
Johannes Berg
6f90aa5a63 tests: hwsim: Allow configuring MODULEDIR
It can be useful to configure a different module directory, so you don't
need to install the kernel modules in the host /lib/modules/ location.
Allow configuring it in the config file.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
2022-09-16 21:16:10 +03:00
Johannes Berg
6f844fa787 tests: Reload correct regdb if possible/needed
If cfg80211 is built into the kernel, then it may/will have
loaded the regdb before we mount our own /lib/firmware. This
may result in using the wrong regulatory data. Fix this by
using iw to reload the regdb after mounting it.

Fixes: a29c2399a7 ("tests: Add regulatory database to VMs")
Signed-off-by: Johannes Berg <johannes.berg@intel.com>
2022-09-16 21:13:20 +03:00
Jouni Malinen
dde7f90a41 tests: Update VM setup example to use Ubuntu 22.04 and UML
Signed-off-by: Jouni Malinen <quic_jouni@quicinc.com>
2022-05-24 00:34:08 +03:00
Jouni Malinen
72641f924e tests: Clean up failed test list in parallel-vm.py
Instead of printing a very long line of the failed tests, print the test
case names on separate lines up to the number of available lines at the
bottom of the screen. This avoids some issues with curses and overlong
lines. Furthermore, display the last failed test cases instead of
somewhat confusing sequence of test case names from the VMs.

Signed-off-by: Jouni Malinen <j@w1.fi>
2022-05-22 11:43:38 +03:00
Jouni Malinen
794011d465 tests: Update regulatory database to VMs
Update the wireless-regdb database to the wireless-regdb.git version of
2022-04-08.

Signed-off-by: Jouni Malinen <quic_jouni@quicinc.com>
2022-04-29 11:19:43 +03:00
Johannes Berg
2f336ca580 tests: Pretend the RNG is initialized withinthe VM
We don't particularly care about the quality of random numbers
during the test. So far, there hasn't been an issue with the
RNG not being initialized completely, we only get a few prints
about uninitialized reads from urandom. However, if some tool
were to actually use /dev/random, it might get stuck. Call the
RNDADDTOENTCNT ioctl to unblock this.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
2022-04-16 16:51:54 +03:00
Jouni Malinen
73b41762d0 tests: Fetch commitid on the host when running tests in a VM
git has started rejecting repositories owned by other users and refusing
to run the "git rev-parse HEAD" command in this type of cases. That
resulted in issues with the VM testing model where the VM is practically
running everything as root while the host is a normal development
environment and likely a non-root user owned files.

Fix this by fetching the commitid on the host and pass it to the VM so
that no git operations need to be run within the VM itself.

Signed-off-by: Jouni Malinen <quic_jouni@quicinc.com>
2022-04-14 11:59:22 +03:00
Jouni Malinen
d5fba20c1e tests: Add a script for find a minimal failing test sequence
min-seq.py can be used to find a minimal test sequence that can be used
to reproduce test failures. This is meant for being able to process the
recently added "Failure sequence:" entries from parallel-vm.log to
reduce manual work needed to debug commonly failing test case sequences.

Signed-off-by: Jouni Malinen <quic_jouni@quicinc.com>
2022-02-24 00:23:25 +02:00
Jouni Malinen
2e166d9e5b tests: Write full failure sequences into parallel-vm.log
This can be helpful in discovering test case sequences that result in
failures frequently.

Signed-off-by: Jouni Malinen <quic_jouni@quicinc.com>
2022-02-15 20:54:24 +02:00
Johannes Berg
a29c2399a7 tests: Add regulatory database to VMs
Add a common well-known regulatory database to the test VMs during runs
to remove one thing to have correct in the host.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
2021-10-22 17:17:28 +03:00
Arowa Suliman
ed5e1b7223 Replace "dummy" with "stub" in comments/documentation
Replace the word "dummy" with the inclusive word "stub".

Signed-off-by: Arowa Suliman <arowa@chromium.org>
2021-10-11 20:52:50 +03:00
Jouni Malinen
eaf925df00 tests: Speed up tshark operations
Hide /usr/share/wireshark from hostfs to prevent tshark from loading all
the data from there since that can take significant amount of time and
is not really needed for the test cases. In addition, set HOME to point
to local tmpfs to avoid unnecessary references through hostfs.

Signed-off-by: Jouni Malinen <jouni@codeaurora.org>
2021-03-17 23:36:52 +02:00
Jouni Malinen
38e700198e tests: Display unexpected stdout and stderr prints in parallel-vm.py
Make it more difficult to miss issues that were previously only printed
out in /tmp/hwsim-test-logs/*-parallel.log. This covers things like
memory leaks and test script failures or forgotten development time
prints to stdout.

Signed-off-by: Jouni Malinen <j@w1.fi>
2021-02-28 22:31:58 +02:00
Thomas Pedersen
fe4c0ea723 tests: Enable MACsec testing by default in the VM kernel
Include support for MACsec testing in the (vm) kernel by default.

Don't include support in the example hostapd or wpa_supplicant configs
yet since that would potentially break the build on older distributions
like Ubuntu 16.04, which is supported until April 2021.

Signed-off-by: Thomas Pedersen <thomas@adapt-ip.com>
2020-04-04 17:25:30 +03:00
Thomas Pedersen
34708b4893 tests: Include UML defconfig
Include a defconfig for building kernel as UML. Also update the README
with a few notes related to UML.

Signed-off-by: Thomas Pedersen <thomas@adapt-ip.com>
2020-03-29 21:15:16 +03:00
Thomas Pedersen
eedf08f7a9 tests: Convert kernel-config to defconfig
Make the included kernel-config a little more minimal by checking in the
defconfig instead.

Generate the defconfig by checking out a linux at tag wt-2020-03-17,
copy kernel-config to .config, run
'yes "" | make oldconfig && make savedefconfig',
and copy resulting defconfig to kernel-config.

Signed-off-by: Thomas Pedersen <thomas@adapt-ip.com>
2020-03-29 21:15:16 +03:00
Johannes Berg
478bc00b5f tests: Assume --long for UML
If we use user-mode-linux, we have time-travel, and then the --long
argument doesn't really make a difference, so just assume that's the
case.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
2020-03-29 21:15:16 +03:00
Jouni Malinen
72fd30c4b9 tests: WPS PBC/PIN timeout on AP
Verify that AuthorizedMACs advertisement is removed when a PBC session
or a PIN times out.

Signed-off-by: Jouni Malinen <jouni@codeaurora.org>
2020-03-04 23:28:45 +02:00
Jouni Malinen
9c244b542e tests: DFS channel bandwidth downgrade from VHT80 to VHT40
Signed-off-by: Jouni Malinen <jouni@codeaurora.org>
2020-03-03 18:58:08 +02:00
Jouni Malinen
31a3d4c234 tests: Recognize more test SKIP reasons
Signed-off-by: Jouni Malinen <j@w1.fi>
2020-01-26 16:03:31 +02:00
Johannes Berg
f124367afb tests: parallel-vm: allow running without curses
Allow running without curses, in which case the log is simply written to
stdout instead of a file. This is useful for automated (but parallel)
testing. Note that in most cases, you'd want to specify --debug, and so
I added a .rstrip() there on the lines to clean that up a bit.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
2020-01-20 21:17:24 +02:00
Jouni Malinen
a6ba5cca45 tests: Allow more VMs to be started in parallel
Check the number of CPUs to determine how many VMs can be started in
parallel.

Signed-off-by: Jouni Malinen <j@w1.fi>
2019-12-27 20:31:33 +02:00
Jouni Malinen
c64b6f62cd tests: Use python selector in the parallel-vm.py main loop
This gets rid of the loop that was polling for things to do every 0.25
seconds and instead, reacts to any data from VMs as soon as it becomes
available. This avoids unnecessary operations when no new data is
available and avoids unnecessary waits when new data becomes available
more quickly.

Signed-off-by: Jouni Malinen <j@w1.fi>
2019-12-27 17:12:34 +02:00
Jouni Malinen
fd0465b85e tests: Print a summary of SKIP reasons
This makes it easier to determine reasons for test cases being skipped.

Signed-off-by: Jouni Malinen <j@w1.fi>
2019-12-27 16:03:50 +02:00
Jouni Malinen
179279eb34 tests: Move VM starting delay control into parallel-vm.py
This is more efficient since we can now start only the necessary number
of VMs instead of always forcing all VMs to start with one second delay.
This can also control the starting delay by keeping at most two VMs
starting at a time instead of using the hardcoded one second wait for
each consecutive VM.

Signed-off-by: Jouni Malinen <j@w1.fi>
2019-12-27 16:03:50 +02:00
Jouni Malinen
9dc07a1d55 tests: Fix 'unexpected exit' detection in parallel-vm.py
Now that parallel-vm.py is actually stopping VMs as soon as they are not
needed for retries, it is not really an unexpected exit to see a VM exit
while test cases remain in the queue as long as at least that many VMs
remain running. Get rid of confusing 'unexpected exit' status from the
UI in such cases.

Fixes: 4aaddecdd8 ("tests: Handle test retries through the same queue")
Signed-off-by: Jouni Malinen <j@w1.fi>
2019-11-23 17:09:56 +02:00
Pradeep Kumar Chitrapu
0a809529d0 tests: HE AP Functionality
Signed-off-by: Pradeep Kumar Chitrapu <pradeepc@codeaurora.org>
2019-10-25 19:29:53 +03:00
Jouni Malinen
d137275131 tests: Remove openssl header files from code coverage report
Signed-off-by: Jouni Malinen <jouni@codeaurora.org>
2019-08-06 13:21:56 +03:00
Jouni Malinen
4aaddecdd8 tests: Handle test retries through the same queue
This removes the separate rerun step from the parallel-vm.py processing
and instead, simply requeues the failed test cases into the same queue
that is used for the initial run. This is simpler and more efficient
since reruns start as soon as any VM is ready for processing them
instead of having to wait for all VMs to complete the first round.
Furthermore, this allows VMs to be stopped sooner when no more test
cases remain and that is helpful especially with the time travel patches
that make the wait-for-next-test step in the VM use all available CPU.

Signed-off-by: Jouni Malinen <j@w1.fi>
2019-07-27 20:19:28 +03:00
Johannes Berg
2080f4c779 tests: uml: Request non-raw serial ports
When the serial ports are set into raw mode on stdio (fd:0,fd:1)
then Ctrl-C is sort of passed through, but not effective. Request
non-raw mode to avoid that and let us cancel test execution with
Ctrl-C properly (both in parallel-vm.py and vm-run.sh cases).

Note that this requires a currently out-of-tree patch, but so
does the virtual time. If the patch is not applied, the command
line argument is ignored.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
2019-05-28 13:34:44 +03:00
Jouni Malinen
d55035c935 tests: Use time-travel=inf-cpu with UML
This is the current proposed way of configuring virtual time.

Signed-off-by: Jouni Malinen <j@w1.fi>
2019-05-27 22:43:04 +03:00
Jouni Malinen
8528ad0c60 tests: Enable virtual time in UML
This speeds up test execution significantly by removing unnecessary
waiting for things to happen since the kernel log is allowed to jump
forward whenever there is nothing to do.

Signed-off-by: Jouni Malinen <jouni@codeaurora.org>
2019-05-06 20:29:16 +03:00
Jouni Malinen
1cefd89b37 tests: Support UML kernel as a replacement for KVM
This allows UML builds to be used in running user mode without having to
run the full x86 kernel in virtual machine.

Signed-off-by: Jouni Malinen <j@w1.fi>
2019-05-05 00:44:04 +03:00
Jouni Malinen
fab49f6145 tests: Python coding style cleanup (pylint3 bad-whitespace)
Signed-off-by: Jouni Malinen <jouni@codeaurora.org>
2019-03-16 18:52:09 +02:00
Jouni Malinen
a2d300765b tests: Make parallel-vm.py log easier to use for debugging failures
Print a list of full paths to log files from failed test cases both the
parallel.log and stdout so that they can be easily opened for analysis.
In addition, renumber the VM lines in the <timestamp>-parallel.log to
match the i+1 numbering used in the log directories and UI that is
tracking test execution.

Signed-off-by: Jouni Malinen <jouni@codeaurora.org>
2019-03-15 21:10:00 +02:00
Lubomir Rintel
192d061add tests: Drop testing of the old D-Bus interface
This old interface has been obsoleted and should not have been used
since 2010, so remove testing for it in preparation to dropping the
interface completely from wpa_supplicant.

Signed-off-by: Lubomir Rintel <lkundrak@v3.sk>
2019-03-09 18:23:09 +02:00
Masashi Honma
432496cdd8 tests: Switch default python version to 3
Signed-off-by: Masashi Honma <masashi.honma@gmail.com>
2019-02-17 17:26:01 +02:00
Masashi Honma
51c83edfca tests: Catch only specific exceptions
This allows unexpected cases to terminate parallel-vm.py without being
hidden by the exception handler.

Signed-off-by: Masashi Honma <masashi.honma@gmail.com>
2019-02-11 12:42:52 +02:00
Masashi Honma
45d3e2edbd tests: Explicitly flush stdin for python3
Without this flush(), test does not run.

Signed-off-by: Masashi Honma <masashi.honma@gmail.com>
2019-02-10 01:43:50 +02:00
Masashi Honma
3069be8fe3 tests: Encode VM input for python3
Signed-off-by: Masashi Honma <masashi.honma@gmail.com>
2019-02-10 01:43:50 +02:00
Masashi Honma
689a956049 tests: Decode VM output for python3
Signed-off-by: Masashi Honma <masashi.honma@gmail.com>
2019-02-10 01:43:50 +02:00
Masashi Honma
0da6d93dbf tests: Change handling of reading non blocked empty stream for python3
The result of reading non blocked empty stream is different between
python2 and 3. The python2 sends "[Errno 11] Resource temporarily
unavailable" exception. The python3 could read "None" without
exception, so handle this "None" case as well.

Signed-off-by: Masashi Honma <masashi.honma@gmail.com>
2019-02-10 01:43:50 +02:00
Johannes Berg
e5a410f472 tests: Make /etc/alternatives work in VM
In recent Debian versions, ebtables is an alias managed by
the alternatives(8) mechanism. This means /usr/sbin/ebtables
is a symlink to /etc/alternatives/ebtables, which in turn
links to the real binary.

As we mount a tmpfs over /etc, we cannot access this.

Fix this by bind-mounting the real /etc to /tmp/etc and
adding a symlink from /etc/alternatives to this.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
2019-02-05 23:51:37 +02:00
Johannes Berg
e339511007 tests: Optionally start telnet server inside VMs
If telnetd is installed and --telnet <port> is passed on the
vm-run.sh command line, start a telnet server (directly connected
to bash, no login) inside the VM(s) to be able to look into them
when something is wrong. Use a user network in qemu with a single
host forward from the specified port for this, listening only on
'localhost'.

Please note that this provides unauthenticated access to the guest
system from anything that can open a TCP connection on the host system.
The guess system does have access to reading all files on the host that
the user account running kvm has access to (and even write access if the
default ROTAG ,readonly parameter is cleared). In other words, this
option should not be used on any multiuser systems where kvm is run
under user accounts that are not dedicated for testing purposes (i.e.,
do not have access to any files that should not be readable to
everyone).

This needs CONFIG_VIRTIO_NET=y in the guest kernel.

For parallel-vm.py, the --telnet argument specifies the base port
and each VM index (0, 1, ...) is added to it.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
2019-02-05 14:26:36 +02:00
Johannes Berg
b0797ec955 tests: Suppress annoying console reset from VMs
Recently, qemu/seabios grew an annoying console/terminal reset,
which also causes my terminal to be left in a state where long
lines don't work well and less gets confused because of this.

Suppress this by suppressing all output from qemu before a new
magic string printed from inside.sh.

Signed-off-by: Johannes Berg <johannes.berg@intel.com>
2019-02-05 13:35:15 +02:00