Some Basic Testing Concepts

Some Basic Testing Concepts Tests are Tools A test is simply a tool that is used to measure something. To narrow that definition a little – after all, we do measure stuff all the time without having any interest in testing it – a test is usually formal, in the sense that is it created and applied with a purpose and intentionally. The "something" that a test is measuring can often be summarized with a question:

1. What are the subject's characteristics or properties? This kind of measurement looks at the test subject itself.
2. Does the test subject pass or fail the test? This kind of measurement compares the subject, or the subjects performance or behavior, against a concrete definition of what success means. The test evaluates the subject based on that definition; if that requirement is met, the subject passes the test.
3. How does the subject respond to the test? This kind of measurement evaluates some arena of performance or behavior, and is usually intended to improve understanding of the test subject.

A test requires more than just asking one of these questions, however. Tests must be planned and thought out a head of time; you have to decide such things as what exactly you are testing and testing for, the way the test is going to be run and applied, what steps are required, etc. A test is usually based on some kind of understanding of what a good result would be, or a specific definition of what "good" means. Using the example above, say I find a television that will fit, so I based on the fact that it passed my measurement test. I get home and set it up and then realize – oops, it doesn't come with a remote control – that I hadn't specified all of my requirements, and as a result hadn't tested for the all of the correct things that I needed. A misunderstood or inadequately planned test can waste time and provide bad information, because the interpretation of the test results will be flawed and misleading. Oops again, I brought a metric tape measure, and I don't know how to convert. Darn, was I supposed to measure with or without the antenna? Did my wife tell me to look for a projection TV, or a Web TV? Before running any test, you should be able to answer the following rough questions:

1. What are you testing? Define the test subject, whether it is a thing, a process, a behavior, a threshold, etc. Define the scope of the test subject. For example, if you are testing a web site's links, will you test every link, or only links to static pages, or only links to other pages as opposed to internal links, etc?
2. From what point-of-view are you testing? If your test is supposed to mimic the interaction of a specific agent or user, then you must have a strong understanding of that agent or user.
3. What are you testing for? Be as specific as possible. If you are going to test one aspect of the test subject, make that limitation clear.
4. How are you going to test? Define the test plan, the specific test(s), test methodologies, etc.

Testing, Quality Control and Quality Assurance Testing is often confused with the processes of quality control and quality assurance. Testing is the process of creating, implementing and evaluating tests. If you are shopping for a new television, you can call that process "testing for the best TV for you"... it's kind of pretentious, but that is what you're doing as you compare prices and features to find what will work best for you. Testing usually has a limited scope and duration – you're just looking at TVs, and only in your town, you're not going to spend a year shopping, are you? Testing is predicated on the use of a standard of quality: you have a specification of what's allowable (no broken links? ALT tags have values? maximum page weight is 10K?) and you compare the site to the standard, noting deviations and shortcomings. This seems simple, but your testing is only valuable if your standard of quality is comprehensive, well thought-out, and reasonable. If your standard has holes, then your testing process has blind spots. Quality control is a refinement of testing, involving the formal and systematic use of testing and a precise definition of what quality means for the purposes of the test. You aren't just testing, you are testing and then doing something with the results. Quality control is used for testing a product or output of a process, with the test measuring the subject's ability to meet a certain benchmark or threshold of quality. The tests usually take the form of "does this product meet requirement X?", and are often pass-fail. Effective quality control testing requires some basic goals and understanding:

1. You must understand what you are testing; if you're testing a specific functionality, you must know how it's supposed to work, how the protocols behave, etc.
2. You should have a definition of what success and failure are. In other words, is close enough good enough?
3. You should have a good idea of a methodology for the test, the more formal a plan the better; you should design testcases.
4. You must understand the limits inherent in the tests themselves.

Any true attempt at quality control requires a great deal of planning before any tests are ever applied, and extensive documentation of quality standards, test plans, test scenarios, test cases, test results -- anything that goes into the testing must be carefully tracked and written down. In fact, for companies that manufacture products, as well as for software companies, a series of formal accreditation programs exist to measure and certify the company's adherence to some very strict standards, for example the ISO 9000 series of rules. No such certification systems exist for web sites, perhaps because sites are more experiences and resources than products to buy. The distinctions between testing and quality control are important for an understanding of the roles and purposes of testing, but they are especially important to anyone involved in testing or creating large a web site. Based on my own experiences, I strongly recommend that testing for site quality be a priority for anyone who

1. works as part of a team that is building and/or maintaining a big web site, and whose responsibility is for testing, quality control, or quality assurance
2. delivers a site to a customer
3. receives site code from a contractor, agency, or technology partner

Testing -- and by extension quality control -- is reactive; that is, you test to find deviations from a standard. If you systematically employ a formal battery of tests on a consistent schedule, you will be able to pass a product with fairly stable quality. The shortcoming here is that this kind of testing does nothing to improve the quality of output; as far as user-experience is concerned, you're just running in place. Testing and quality control do nothing to raise the level of quality beyond perhaps tweaking the standard to "raise the bar". Quality assurance goes beyond quality control to examine the processes that create and shape the product: quality assurance looks at the quality of output, as well as at the quality of the inputs.

SOA Testing

Sometime back, I had given a presentation on SOA Testing. You can view/download the ppt from following link:

SOA Testing PPT

Comments are welcome!

Getting Current Time on DB

To verify a bugs I had a need to get the current time on the Database
Server. Here are some SQL which might be useful.

Note: This is on Oracle DB

To get the server Date:

SELECT sysdate FROM dual;

This will give you the system date in this format, Ex: 18-AUG-09

To get the server TimeZone:

SELECT sessiontimezone FROM dual;

This will give you the system timezone in this format, Ex: Asia/Calcutta

To get the server time:

SELECT systimestamp FROM dual;

This will give you the system time in this format,
Ex: 18-AUG-09 04.49.43.648480000 AM -07:00

Microsoft Releases IE8

Microsoft to release IE8. Read more at the following links.

* http://www.microsoft.com/Presspass/press/2009/mar09/03-18IE8AvailablePR.mspx
* http://blogs.msdn.com/ie/archive/2009/03/19/internet-explorer-8-final-available-now.aspx
* http://www.pcworld.com/businesscenter/article/161522/microsoft_releases_ie8_stresses_security.html

List Files in a directory

I remembered a good question which I was asked in one of my interviews some time back...

Write a program to list the files in a directory. The listed files should be greater than 1000 KB and it should be sorted.

This should take care of recursion also. Also write test cases for the same.

Here is what I came up with...

Java Program to List Files in a Directory.

/*
This program prints all the files in a directory(including sub directories recursively)
- more than 1000 KB - 1 KB = 1000 Bytes, 1000 KB = 1024000 bytes
- in descending order
*/

import java.io.*;
import java.util.*;

public class listFiles{
// Vector to hold all the list of files.
static Vector v1 = new Vector();
static boolean validdir = true;

public static void main(String[] args) {
// Get the directory name from the command line
if (args.length > 0 ) {
File dirName = new File(args[0].trim());
listFiles.listFilesinDir(dirName);
} else {
System.out.println("Enter a directory name.\n Usage: listFiles dirName");
validdir = false;
}

if (validdir) {
// Sort the Vector v1
Comparator comparator = Collections.reverseOrder();
// System.out.println("The List of files before sorting: " + v1);

Collections.sort(v1,comparator);
//System.out.println("The List of files after sorting in descending order : " + v1);
System.out.println("The Total Number of Files are: " + v1.size());

for (Enumeration e = v1.elements(); e.hasMoreElements();) {
System.out.println(e.nextElement());
}
}
} // end main()

public static void listFilesinDir(File fin) {
// Throw error if the directory does not exist.
if (!fin.exists()) {
validdir = false;
System.out.println(fin.getName() + " does not exist!!!");
return;
}

if (fin.isFile()) {
// 1 KB = 1000 Bytes, 1000 KB = 1024000 bytes
if (fin.length() > 1024000) {
// To print all files remove the comment in the below line.
// System.out.println(fin.getName());
// Add all the files into the vector.
listFiles.v1.add(fin.getName());
}
}
else if (fin.isDirectory()) {
String[] files = fin.list();
if (files == null) return;

for (int i=0; i

Here is the list of TestCases

	A	B	C	D	E
1	Nu	Test Case For	Description	Expected Results	Remarks
2	1	Valid Dir Name	Give a Valid Dir Name	Should give the list of files	Assume that size and ascending is taken care
3	2	Valid Dir Name - Order	Check for Descending order	The listed files should be in descending order	Assume that size is taken care
4	3	Valid Dir Name - Size	Check for the Size of Files	The listed files should be greater than 1000 KB only	Assume that ascending is taken care
5	4	Size - Boundary values	Have a file of Exactly 1000 KB	This file should be listed	Assume that ascending is taken care
6	5	Size - Boundary values	Have a file of Exactly 1001 KB	This file should be listed	Assume that ascending is taken care
7	6	Size - Boundary values	Have a file of Exactly 999 KB	This file should NOT be listed
8	7	Size - Low size	Have a empty file/0 size	This file should NOT be listed
9	8	Size - Low size	Have a file of 1 KB size	This file should NOT be listed
10	9	Size - Huge Size	Have a file of 20000 KB	This file should be listed	Assume that ascending is taken care
11	10	Order	Have files with names - a.txt, aa.txt, aaa.txt	Observe that these should be sorted in descending order	Assume that size is taken care
12	11	Order	Have files with names - a.txt, b.txt, c.txt	Observe that these should be sorted in descending order	Assume that size is taken care
13	12	Dir Name -Space	Let the directory name have space - ex: "Program Files"	Should give the list of files	Assume that size and ascending is taken care
14	13	Dir Name - Case Sensitive	Have directories with case sensitive - ex: myDir1, mYdIR2 etc	Should give the list of files	Assume that size and ascending is taken care
15	14	Sub Directories	Have many sub directories	The sub directory names should NOT be listed
16	15	Sub Directories	Have many sub directories	The files inside the sub directories should be listed	Assume that size and ascending is taken care
17	16	Sub Directories	Have a sub directory with the "same name" as parent	The files inside the sub directories should be listed	Assume that size and ascending is taken care
18	17	Hidden Files	Have some hidden files in the dir	These should be listed	Assume that size and ascending is taken care
19	18	Hidden Directory	Specify a hidden dir as input	The files inside a hidden dir should be listed	Assume that size and ascending is taken care
20	19	Hidden sub directories	Have some hidden sub directories inside the parent dir	The files inside a hidden dir should be listed	Assume that size and ascending is taken care
21	20	Read-Only Files	Have some read only files in the dir	These should be listed
22	21	Read-only Directory	Specify a read-only dir as input	The files inside a read-only dir should be listed	Assume that size and ascending is taken care
23	22	Read-only Sub directories	Have some read-only sub directories inside the parent dir	The files inside a read-only dir should be listed	Assume that size and ascending is taken care
24	23	Empty Dir	Give a valid dir name - but empty one	No Files should be listed
25	24	Dir Name - Special Chars	Have some special chars in the name of the dir	Should give the list of files	Assume that size and ascending is taken care
26	25	Sub Directories - Breadth	Have many sub directories in the parent dir… Have a breadth of say 10-20 dirs. and files inside those directories	Should give the list of files	Assume that size and ascending is taken care
27	26	Sub Directories - Depth	Have a sub directoy inside a sub directory inside a sub dir…. Have a depth of say 10-20 dirs. and files inside those directories	Should give the list of files	Assume that size and ascending is taken care
28	27	Shared Dir in Windows	Specify a shared directory	Should give the list of files	Assume that size and ascending is taken care
29	28	Specify the URI as dir name	A dir on the intranet - ex: \\10.192.23.212\myDir1	Should give the list of files if accessible	Assume that size and ascending is taken care
30	29	Dir Name - "."	Specify "." as the dir name	Should give the list of files in the current dir	Assume that size and ascending is taken care
31	30	Dir Name - ".."	Specify ".." as the dir name	Should give the list of files in the parent dir	Assume that size and ascending is taken care
32	31	Dir Name - Absolute Path - Forward Slash	Specify "C:/Testing/MyDir1" as the dir name	Should give the list of files	Assume that size and ascending is taken care
33	32	Dir Name - Absolute Path - Back Slash	Specify "C:\Testing\MyDir1" as the dir name	Should give the list of files	Assume that size and ascending is taken care
34	33	Dir Name - Relative Path - Forward slash	Specify "MyDir1/MyDir2/MyDir3" as the dir name	Should give the list of files	Assume that size and ascending is taken care
35	34	Dir Name - Relative Path - Backward slash	Specify "MyDir1\MyDir2\MyDir3" as the dir name	Should give the list of files	Assume that size and ascending is taken care
36	35	Dir Name - Relative Path	Specify "..\..\MyDir2\MyDir3" as the dir name	Should give the list of files	Assume that size and ascending is taken care
37	36	Dir Name - integer	The dir name is "12345"	Should give the list of files	Assume that size and ascending is taken care
38	37	Dir Name - very long name	Ex: The dir name is "Thisismydirectoryandthenameisverybig"	Should give the list of files	Assume that size and ascending is taken care
39	38	Negative Test	Specify some file name as input	Throw error message
40	39	Negative Test	Specify some non existing dir name	Throw error message
41	40	Negative Test	Specify null	Throw error message
42	41	Formats	All possible format files should be listed - ex: ".zip" ".doc" ".pdf" etc	Should give the list of files	Assume that size and ascending is taken care
43	42	Negative Test	Specify a zip file as input	Throw error message
44	43	Negative Test	Specify the dir name + some blanks	Should trim
45
46		Unix/Linux
47	44	Perform all the above tests for Linux/Unix; Also perform the below special cases for Unix
48
49	45	Dir Name - Mounted dir	Specify a mounted dir as the input dir name	Should give the list of files	Assume that size and ascending is taken care
50	46	Dir Name - Case Sensitive	Have directories with case sensitive - ex: myDir1, mYdIR2 etc	Should give the list of files	Assume that size and ascending is taken care
51	47	Read-Only Files	Have some read only files in the dir	These should be listed
52	48	Read-only Directory	Specify a read-only dir as input	The files inside a read-only dir should be listed	Assume that size and ascending is taken care
53	49	Read-only Sub directories	Have some read-only sub directories inside the parent dir	The files inside a read-only dir should be listed	Assume that size and ascending is taken care
54	50	Dir Name - Special Chars	Have some special chars in the name of the dir	Should give the list of files	Assume that size and ascending is taken care
55	51	Dir Name - "."	Specify "." as the dir name	Should give the list of files in the current dir	Assume that size and ascending is taken care
56	52	Dir Name - ".."	Specify ".." as the dir name	Should give the list of files in the parent dir	Assume that size and ascending is taken care
57	53	Dir Name - Absolute Path - Forward Slash	Specify "/opt/testing/MyDir1" as the dir name	Should give the list of files	Assume that size and ascending is taken care
58	54	Dir Name - Absolute Path - Back Slash	Specify "opt\testing\MyDir1" as the dir name	Should give the list of files	Assume that size and ascending is taken care
59	55	Dir Name - Relative Path - Forward slash	Specify "MyDir1/MyDir2/MyDir3" as the dir name	Should give the list of files	Assume that size and ascending is taken care
60	56	Dir Name - Relative Path - Backward slash	Specify "MyDir1\MyDir2\MyDir3" as the dir name	Should give the list of files	Assume that size and ascending is taken care
61	57	Dir Name - Relative Path	Specify "..\..\MyDir2\MyDir3" as the dir name	Should give the list of files	Assume that size and ascending is taken care
62
63		Performance Testing
64	58	50,000 Files( all > 1000 KB)	Have a dir with 50,000 Files	Should give the list of files - check for breakdown	Assume that size and ascending is taken care
65	59	50,000 Files > 1000 KB + 50,000 Files < 1000 KB	Have a dir with 50,000 Files > 1000 KB and 50,000 Files < 1000 KB	Should give the list of files - check for breakdown	Assume that size and ascending is taken care
66
67	60	10,000 Sub directories	Have 10,000 sub directories with files	Should give the list of files - check for breakdown	Assume that size and ascending is taken care

Testing Private methods

Hi,
Here is a good article on testing private methods...

http://www.artima.com/suiterunner/private.html

-ahamad

Code coverage tool for UI

I was looking for a code coverage tool for Java based UI, I came across this tool – ECLEmma.

EclEmma is a free Java code coverage tool for Eclipse, available under the Eclipse Public License.

home page : http://www.eclemma.org/

Below are the steps that are to be followed to setup EclEmma for code coverage:

1. Open Eclipse; Goto "Help->Software Updates->Find and Install" menu
2. Click "Next >" In Install window and add a new remote site by clicking on "New Remote Site" button
3. Enter any name and specify the URL as http://update.eclemma.org
4. Click finish to search and install the EclEmma Java Code Coverage plug-in
5. Restart Eclipse when prompted
6. Now a new icon can be seen next to Run or Debug icons.

7. Click on arrow next to Coverage icon and select "Open Coverage Dialog"
8. Select “Eclipse Application” and click on “New” to create a new configuration.
9. Goto "Coverage" tag and select only the packages in which the source for your feature is checked-in. You can contact the developer or see the folders in changelists to know the list of all packages used by the feature.
10. Click on "Apply" after all the required packages are selected and then on "Coverage" button to launch your app in "Code Coverage" mode.
11. Conduct Manual / Automated testing to cover the feature that you are trying to test and close your app.
12. Now you can see a new tab with the name "Coverage" near "Console" or "Problems" tab
13. Navigate to exact Java files where the source of your feature is written and check out the percentage of coverage done.
14. You can double-click on Class or Function to navigate to corresponding Java file and see what parts of code were not exercised and write more test cases to cover the un-exercised code.
15. You can export the whole of Coverage results to HTML format by right-clicking on any row and selecting "Export Session"
16. You can have multiple sessions and then you can merge them together.