--- /dev/null
+Automated hostapd/wpa_supplicant testing with mac80211_hwsim
+------------------------------------------------------------
+
+This directory contains testing infrastructure and test cases to run
+automated tests of full hostapd and wpa_supplicant functionality. This
+testing is done with the help of mac80211_hwsim which is Linux kernel
+driver that simulates IEEE 802.11 radios without requiring any
+additional hardware. This setup most of the hostapd and wpa_supplicant
+functionality (and large parts of the Linux cfg80211 and mac80211
+functionality for that matter) to be tested.
+
+mac80211_hwsim is loaded with five simulated radios to allow different
+device combinations to be tested. wlantest is used analyze raw packets
+captured through the hwsim0 monitor interface that capture all frames
+sent on all channels. wlantest is used to store the frames for
+analysis. Three wpa_supplicant processes are used to control three
+virtual radios and one hostapd process is used to dynamically control
+the other two virtual radios. wpa_supplicant/hostapd test functionality
+is used to verify that data connection (both unicast and broadcast)
+works between two netdevs.
+
+The python scripts and tools in this directory control test case
+execution. They interact wpa_supplicant and hostapd through control
+interfaces to perform the operations. In addition, wlantest_cli is used
+to verify that operations have been performed correctly and that the
+network connection works in the expected way.
+
+These test cases are run automatically against the hostap.git commits
+for regression testing and to help in keeping the hostap.git master
+branch in stable state. Results from these tests are available here:
+http://buildbot.w1.fi/hwsim/
+
+
+Building binaries for testing
+-----------------------------
+
+You will need to build (or use already built) components to be
+tested. These are available in the hostap.git repository and can be
+built for example as follows:
+
+cd ../../wpa_supplicant
+cp ../tests/hwsim/example-wpa_supplicant.config .config
+make clean
+make
+cd ../hostapd
+cp ../tests/hwsim/example-hostapd.config .config
+make clean
+make hostapd hlr_auc_gw
+cd ../wlantest
+make clean
+make
+
+Alternatively, the build.sh script here can be used to run these steps
+with conditional creation of .config files only if they do not exist.
+
+The test scripts can find the binaries in the locations where they were
+built. It is also possible to install wlantest_cli somewhere on the path
+to use pre-built tools.
+
+Please note that some of the configuration parameters used to enable
+more testing coverage may require development packages that may not be
+installed by default in many distributions. For example, following
+Debian/Ubuntu packages are likely to be needed:
+- binutils-dev
+- libsqlite3-dev
+- libpcap-dev
+
+example-setup.txt provides more complete step-by-step example on how a
+test setup can be built.
+
+
+wpaspy
+------
+
+The python scripts use wpaspy.py to interact with the wpa_supplicant
+control interface, but the run-tests.py script adds the (relative)
+path into the environment so it doesn't need to be installed.
+
+
+mac80211_hwsim
+--------------
+
+mac80211_hwsim kernel module is available from the upstream Linux
+kernel. Some Linux distributions enable it by default. If that's not the
+case, you can either enable it in the kernel configuration
+(CONFIG_MAC80211_HWSIM=m) and rebuild your kernel or use Backports with
+CPTCFG_MAC80211_HWSIM=m to replace the wireless LAN components in the
+base kernel.
+
+
+sudo
+----
+
+Some parts of the testing process requires root privileges. The test
+scripts are currently using sudo to achieve this. To be able to run the
+tests, you'll probably want to enable sudo with a timeout to not expire
+password entry very quickly. For example, use this in the sudoers file:
+
+Defaults env_reset,timestamp_timeout=180
+
+Or on a dedicated test system, you could even disable password prompting
+with this in sudoers:
+
+%sudo ALL=NOPASSWD: ALL
+
+
+Other network interfaces
+------------------------
+
+Some of the test scripts are still using hardcoded interface names, so
+the easiest way of making things work is to avoid using other network
+devices that may use conflicting interface names. For example, unload
+any wireless LAN driver before running the tests and make sure that
+wlan0..4 gets assigned as the interface names for the mac80211_hwsim
+radios. It may also be possible to rename the interface expectations in
+run-tests.py to allow other names to be used.
+
+Please also note that some commonly enabled tools, like NetworkManager,
+may end up trying to control new network interfaces automatically. This
+can result in conflicts with the test scripts and you may need to
+disable such network services or at least mark the mac80211_hwsim wlan#
+interfaces as umanaged. As an example, this can be done in
+/etc/NetworkManager/NetworkManager.conf with following addition:
+
+[keyfile]
+unmanaged-devices=mac:02:00:00:00:00:00;mac:02:00:00:00:01:00;mac:02:00:00:00:02:00;mac:02:00:00:00:03:00;mac:02:00:00:00:04:00
+
+
+Running tests
+-------------
+
+Simplest way to run a full set of the test cases is by running
+run-all.sh in tests/hwsim directory. This will use start.sh to load the
+mac80211_hwsim module and start wpa_supplicant, hostapd, and various
+test tools. run-tests.sh is then used to run through all the defined
+test cases and stop.sh to stop the programs and unload the kernel
+module.
+
+run-all.sh can be used to run the same test cases under different
+conditions:
+
+# run normal test cases
+./run-all.sh
+
+# run normal test cases under valgrind
+./run-all.sh valgrind
+
+# run normal test cases with Linux tracing
+./run-all.sh trace
+
+# run normal test cases with multi channel support (see details below)
+./run-all.sh channels=<num of channels>
+
+run-all.sh directs debug logs into the logs subdirectory (or $LOGDIR if
+present in the environment). Log file names include the current UNIX
+timestamp and a postfix to identify the specific log:
+- *.log0 = wpa_supplicant debug log for the first radio
+- *.log1 = wpa_supplicant debug log for the second radio
+- *.log2 = wpa_supplicant debug log for the third radio
+- *.hostapd = hostapd debug log
+- hwsim0 = wlantest debug log
+- hwsim0.pcapng = capture with all frames exchanged during the tests
+- *.log = debug prints from the test scripts
+- trace.dat = Linux tracing record (if enabled)
+- hlr_auc_gw - hlr_auc_gw (EAP-SIM/AKA/AKA' authentication) log
+- auth_serv - hostapd as RADIUS authentication server log
+
+
+For manual testing, ./start.sh can be used to initialize interfaces and
+programs and run-tests.py to execute one or more test
+cases. run-tests.py output verbosity can be controlled with -d (more
+verbose debug output) and -q (less verbose output) on the command
+line. "-f <module name>" (pointing to file test_<module name>.py) can be
+used to specify that all test cases from a single file are to be
+run. Test name as the last command line argument can be specified that a
+single test case is to be run (e.g., "./run-tests.py ap_pmf_required").
+
+Notice that some tests require the driver to support concurrent
+operation on multi channels in order to run. These tests will be skipped
+in case the driver does not support multi channels. To enable support
+for multi channel, the number of supported channel is passed as an
+argument to run-all.sh or start.sh
+
+
+Adding/modifying test cases
+---------------------------
+
+All the test cases are defined in the test_*.py files. These are python
+scripts that can use the local helper classes to interact with the test
+components. While various python constructs can be used in the scripts,
+only a minimal level of python knowledge should really be needed to
+modify and add new test cases. The easiest starting point for this is
+likely to take a look at some of the example scripts. When working on a
+new test, run-tests.py with -d and the test case name on the command
+line is a convenient way of verifying functionality.
+
+run-tests.py will automatically import all test cases from the test_*.py
+files in this directory. All functions starting with the "test_" prefix
+in these files are assumed to be test cases. Each test case is named by
+the function name following the "test_" prefix.
+
+
+Results database
+----------------
+
+run-tests.py can be requested to write results from the execution of
+each test case into an sqlite database. The "-S <path to database>" and
+"-b <build id>" command line arguments can be used to do that. The
+database must have been prepared before this, e.g., with following:
+
+cat | sqlite3 /tmp/example.db <<EOF
+CREATE TABLE results (test,result,run,time,duration,build,commitid);
+CREATE INDEX results_idx ON results (test);
+CREATE INDEX results_idx2 ON results (run);
+CREATE TABLE tests (test,description);
+CREATE UNIQUE INDEX tests_idx ON tests (test);
+CREATE TABLE logs (test,run,type,contents);
+CREATE INDEX logs_idx ON logs (test);
+CREATE INDEX logs_idx2 ON logs (run);
+EOF
projects / mech_eap.git / blobdiff