Automated Webclient API testbase.
1. Goals.
The main goal of this testbase is test of Webclient API.
Tests covers all interfaces(so far implemented) than can be queried by application or received to application in any other way.
The "Webclient"(EMWindow Test application that is run from /mozilla/java/..src_moz/runwm script) is not a subject of this testbase.
2. How it works.
The perl script autorun.pl is used to run test application in cycles.
The simple test application runs and executes test class.
Test class queries corresponding interface, performs procedure call
and verifies result when possible.
3. Directory struct.
Source files, build files , configuration files and utils are placed in different directories. Please see the diagram below for more details.
TOP(0)
|
|-build(1)
| |-classes(1.1)
| |
| |-test(1.2)
| | |-basic(1.2.1)
| |-run(1.3) |-api(1.2.1.1)
|-src(2) |-mixed(1.2.1.2)
| |-classes(2.1)
| |
| |-test(2.2)
| | |-basic(2.2.1)
| | |-api(2.2.1.1)
| | |-mixed(2.2.1.2)
| |-run(2.3)
| |
| |-Killer(2.4)
|-utils(3)
|-config(4)
|-doc(5)
|-log(6)
TOP 0) directory with testbase, in usually this dir named "automated"
build 1) build directory. All class and data files(exclude html and cgi files) compiled and copied to it.
1.1) classes directory.
Directory for test classes files . User must add this directory to the CLASSPATH before executing tests.
1.2, 1.2.1, 1.2.1.1, 1.2.1.2) In these directories data and properties files are copied at make stage
1.3) Execution script and default list file are copied to this dir.
src 2) All source files placed to this dir.
2.1) Directory for all Java files
2.2, 2.2.1, 2.2.1.1, 2.2.1.2) Directories for data and properties files.
All common files can be placed to 2.2.1, 2.2.1.1, 2.2.1.2
For specific test files additional dirs created in 2.2.1.1 and 2.2.1.2
2.3) Directory for autorun.pl and miscellaneous list files
2.4) Directory where sources of "Killer" util is placed
utils 3) Directory for miscellaneous utils, that is used at "make" and "run" stages.
config 4) Directory for configuration files. CommonProperties file.
PLEASE EDIT IT BEFORE RUNNING A MAKE
doc 5) Directory for miscellaneous documents. Documentation for testcases placed here.
log 6) Directory for placing execution logs.
Also additional HTML_ROOT_DIR and CGI_ROOT_DIR directories used to placing CGI and HTML files.
4. Requirements.
- User must have successfully built Webclient and Mozilla
- User must have a configured and working http/Java Web server. Otherwise user must use file:/// protocol
to specify HTML_ROOT .Tests( very small part of all), that uses CGI will not executed properly in such cases.
5. How to build.
- Please setup environment as for running Webclient.
- Please edit CommonProperties file in config(4) dir. This file is self documented.
- Go to build(1) dir and do make
6. How to run.
- Set JAVAHOME environment variable to path to java executable.
- Make sure CLASSPATH is included <TOP_DIR>/build/classes
- Go to build/run(1.3) directory and execute autorun.pl script.
6.1 Usage.
User can execute tests by different ways:
To execute ONE testcase just do: "perl autorun.pl -t test_case_id"
To execute LIST of tests just do: "perl autorun.pl -f file_with_test_ids"
Also user can turn TestRunner to the "mixed" mode via -m switch. "perl autorun.pl -m file file_with_mixed_tests".
In this mode 2 buttons "FAILED" and "PASSED" added to TestWindow
and tester can fail or pass test in any time.
This mode is NECESSARY for running tests from "mixed" sections
And via -i switch user can turn autorun to "ignorecore" mode. In
this mode autorun.pl ignore core and doesn't fail tests, that
really coredumps. This mode is useful for Solaris testing. "perl autorun.pl -i -f file_with_testids"
7. Logging.
- All logs are placed to the log(6) directory. A unique directory created
in the log(6) directory for each execution.
Files, created in this directory are:
BWTest.html - the main log file with results of execution in HTML format
BWTest.txt - the results of execution in TXT format
BWTestRun.log - the log of test execution with additional info
<testID>.log \
<testID>.log |- files with output of each testcase.
<testID>.log /
8. Sections.
Now testbase contains 3 different sections of tests: api, stress and mixed.
"api" Tests from this section performs calls to api methods and verify results when
possible. They are fully automated and user mustn't interact
with application.
"stress" Tests from this section performs stress actions. E.g loading many pages,
calls api methods many times, etc. They are fully automated
and user need not interact with application.
"mixed" Tests from this section performs calls of api methods, that can change
rendering/viewing behavior. Eg setBounds, moveWindowTo. User should read
instructions on TestWindow and pass or fail tests via corresponding buttons.
9. Things, that must be done, known bugs.
- Several tests on CurrentPage failed with NP exception.
Problem with system clipboard.
- Solaris: Some times no urls loaded by TestRunner at all.
- Webclient tests have been tested (so far) with PR3 Webclient bundled with Netscape6 SVR4pkg from SUN
on Sparc Solris and Intel Solaris Platforms only.
- Webclient Tests have not been tested with PR3 Webclinet on Win32 or Linux platoforms yet.
Last Modified 08/23/2000 by avm
Last Modified 09/08/2000 by gv
Last Modified 12/15/2000 by GV