Monday, June 07, 2021

Some Basic Testing Concepts

Some Basic Testing Concepts Tests are Tools A test is simply a tool that is used to measure something. To narrow that definition a little – after all, we do measure stuff all the time without having any interest in testing it – a test is usually formal, in the sense that is it created and applied with a purpose and intentionally. The "something" that a test is measuring can often be summarized with a question:
  1. What are the subject's characteristics or properties? This kind of measurement looks at the test subject itself.
  2. Does the test subject pass or fail the test? This kind of measurement compares the subject, or the subjects performance or behavior, against a concrete definition of what success means. The test evaluates the subject based on that definition; if that requirement is met, the subject passes the test.
  3. How does the subject respond to the test? This kind of measurement evaluates some arena of performance or behavior, and is usually intended to improve understanding of the test subject.
A test requires more than just asking one of these questions, however. Tests must be planned and thought out a head of time; you have to decide such things as what exactly you are testing and testing for, the way the test is going to be run and applied, what steps are required, etc. A test is usually based on some kind of understanding of what a good result would be, or a specific definition of what "good" means. Using the example above, say I find a television that will fit, so I based on the fact that it passed my measurement test. I get home and set it up and then realize – oops, it doesn't come with a remote control – that I hadn't specified all of my requirements, and as a result hadn't tested for the all of the correct things that I needed. A misunderstood or inadequately planned test can waste time and provide bad information, because the interpretation of the test results will be flawed and misleading. Oops again, I brought a metric tape measure, and I don't know how to convert. Darn, was I supposed to measure with or without the antenna? Did my wife tell me to look for a projection TV, or a Web TV? Before running any test, you should be able to answer the following rough questions:
  1. What are you testing? Define the test subject, whether it is a thing, a process, a behavior, a threshold, etc. Define the scope of the test subject. For example, if you are testing a web site's links, will you test every link, or only links to static pages, or only links to other pages as opposed to internal links, etc?
  2. From what point-of-view are you testing? If your test is supposed to mimic the interaction of a specific agent or user, then you must have a strong understanding of that agent or user.
  3. What are you testing for? Be as specific as possible. If you are going to test one aspect of the test subject, make that limitation clear.
  4. How are you going to test? Define the test plan, the specific test(s), test methodologies, etc.
Testing, Quality Control and Quality Assurance Testing is often confused with the processes of quality control and quality assurance. Testing is the process of creating, implementing and evaluating tests. If you are shopping for a new television, you can call that process "testing for the best TV for you"... it's kind of pretentious, but that is what you're doing as you compare prices and features to find what will work best for you. Testing usually has a limited scope and duration – you're just looking at TVs, and only in your town, you're not going to spend a year shopping, are you? Testing is predicated on the use of a standard of quality: you have a specification of what's allowable (no broken links? ALT tags have values? maximum page weight is 10K?) and you compare the site to the standard, noting deviations and shortcomings. This seems simple, but your testing is only valuable if your standard of quality is comprehensive, well thought-out, and reasonable. If your standard has holes, then your testing process has blind spots. Quality control is a refinement of testing, involving the formal and systematic use of testing and a precise definition of what quality means for the purposes of the test. You aren't just testing, you are testing and then doing something with the results. Quality control is used for testing a product or output of a process, with the test measuring the subject's ability to meet a certain benchmark or threshold of quality. The tests usually take the form of "does this product meet requirement X?", and are often pass-fail. Effective quality control testing requires some basic goals and understanding:
  1. You must understand what you are testing; if you're testing a specific functionality, you must know how it's supposed to work, how the protocols behave, etc.
  2. You should have a definition of what success and failure are. In other words, is close enough good enough?
  3. You should have a good idea of a methodology for the test, the more formal a plan the better; you should design testcases.
  4. You must understand the limits inherent in the tests themselves.
Any true attempt at quality control requires a great deal of planning before any tests are ever applied, and extensive documentation of quality standards, test plans, test scenarios, test cases, test results -- anything that goes into the testing must be carefully tracked and written down. In fact, for companies that manufacture products, as well as for software companies, a series of formal accreditation programs exist to measure and certify the company's adherence to some very strict standards, for example the ISO 9000 series of rules. No such certification systems exist for web sites, perhaps because sites are more experiences and resources than products to buy. The distinctions between testing and quality control are important for an understanding of the roles and purposes of testing, but they are especially important to anyone involved in testing or creating large a web site. Based on my own experiences, I strongly recommend that testing for site quality be a priority for anyone who
  1. works as part of a team that is building and/or maintaining a big web site, and whose responsibility is for testing, quality control, or quality assurance
  2. delivers a site to a customer
  3. receives site code from a contractor, agency, or technology partner
Testing -- and by extension quality control -- is reactive; that is, you test to find deviations from a standard. If you systematically employ a formal battery of tests on a consistent schedule, you will be able to pass a product with fairly stable quality. The shortcoming here is that this kind of testing does nothing to improve the quality of output; as far as user-experience is concerned, you're just running in place. Testing and quality control do nothing to raise the level of quality beyond perhaps tweaking the standard to "raise the bar". Quality assurance goes beyond quality control to examine the processes that create and shape the product: quality assurance looks at the quality of output, as well as at the quality of the inputs.

Wednesday, July 21, 2010

SOA Testing

Sometime back, I had given a presentation on SOA Testing. You can view/download the ppt from following link:

SOA Testing PPT

Comments are welcome!

Tuesday, August 18, 2009

Getting Current Time on DB

To verify a bugs I had a need to get the current time on the Database
Server. Here are some SQL which might be useful.

Note: This is on Oracle DB

To get the server Date:

SELECT sysdate FROM dual;

This will give you the system date in this format, Ex: 18-AUG-09

To get the server TimeZone:

SELECT sessiontimezone FROM dual;

This will give you the system timezone in this format, Ex: Asia/Calcutta

To get the server time:

SELECT systimestamp FROM dual;

This will give you the system time in this format,
Ex: 18-AUG-09 04.49.43.648480000 AM -07:00

Monday, February 02, 2009

List Files in a directory


I remembered a good question which I was asked in one of my interviews some time back...

Write a program to list the files in a directory. The listed files should be greater than 1000 KB and it should be sorted.

This should take care of recursion also. Also write test cases for the same.

Here is what I came up with...

Java Program to List Files in a Directory.

/*
This program prints all the files in a directory(including sub directories recursively)
- more than 1000 KB - 1 KB = 1000 Bytes, 1000 KB = 1024000 bytes
- in descending order
*/

import java.io.*;
import java.util.*;

public class listFiles{
// Vector to hold all the list of files.
static Vector v1 = new Vector();
static boolean validdir = true;
public static void main(String[] args) {
// Get the directory name from the command line
if (args.length > 0 ) {
File dirName = new File(args[0].trim());
listFiles.listFilesinDir(dirName);
} else {
System.out.println("Enter a directory name.\n Usage: listFiles dirName");
validdir = false;
}
if (validdir) {
// Sort the Vector v1
Comparator comparator = Collections.reverseOrder();
// System.out.println("The List of files before sorting: " + v1);

Collections.sort(v1,comparator);
//System.out.println("The List of files after sorting in descending order : " + v1);
System.out.println("The Total Number of Files are: " + v1.size());

for (Enumeration e = v1.elements(); e.hasMoreElements();) {
System.out.println(e.nextElement());
}
}
} // end main()
public static void listFilesinDir(File fin) {
// Throw error if the directory does not exist.
if (!fin.exists()) {
validdir = false;
System.out.println(fin.getName() + " does not exist!!!");
return;
}
if (fin.isFile()) {
// 1 KB = 1000 Bytes, 1000 KB = 1024000 bytes
if (fin.length() > 1024000) {
// To print all files remove the comment in the below line.
// System.out.println(fin.getName());
// Add all the files into the vector.
listFiles.v1.add(fin.getName());
}
}
else if (fin.isDirectory()) {
String[] files = fin.list();
if (files == null) return;
for (int i=0; i


Here is the list of TestCases

A B C D E
1 Nu Test Case For Description Expected Results Remarks
2 1 Valid Dir Name Give a Valid Dir Name Should give the list of files Assume that size and ascending is taken care
3 2 Valid Dir Name - Order Check for Descending order The listed files should be in descending order Assume that size is taken care
4 3 Valid Dir Name - Size Check for the Size of Files The listed files should be greater than 1000 KB only Assume that ascending is taken care
5 4 Size - Boundary values Have a file of Exactly 1000 KB This file should be listed Assume that ascending is taken care
6 5 Size - Boundary values Have a file of Exactly 1001 KB This file should be listed Assume that ascending is taken care
7 6 Size - Boundary values Have a file of Exactly 999 KB This file should NOT be listed
8 7 Size - Low size Have a empty file/0 size This file should NOT be listed
9 8 Size - Low size Have a file of 1 KB size This file should NOT be listed
10 9 Size - Huge Size Have a file of 20000 KB This file should be listed Assume that ascending is taken care
11 10 Order Have files with names - a.txt, aa.txt, aaa.txt Observe that these should be sorted in descending order Assume that size is taken care
12 11 Order Have files with names - a.txt, b.txt, c.txt Observe that these should be sorted in descending order Assume that size is taken care
13 12 Dir Name -Space Let the directory name have space - ex: "Program Files" Should give the list of files Assume that size and ascending is taken care
14 13 Dir Name - Case Sensitive Have directories with case sensitive - ex: myDir1, mYdIR2 etc Should give the list of files Assume that size and ascending is taken care
15 14 Sub Directories Have many sub directories The sub directory names should NOT be listed
16 15 Sub Directories Have many sub directories The files inside the sub directories should be listed Assume that size and ascending is taken care
17 16 Sub Directories Have a sub directory with the "same name" as parent The files inside the sub directories should be listed Assume that size and ascending is taken care
18 17 Hidden Files Have some hidden files in the dir These should be listed Assume that size and ascending is taken care
19 18 Hidden Directory Specify a hidden dir as input The files inside a hidden dir should be listed Assume that size and ascending is taken care
20 19 Hidden sub directories Have some hidden sub directories inside the parent dir The files inside a hidden dir should be listed Assume that size and ascending is taken care
21 20 Read-Only Files Have some read only files in the dir These should be listed
22 21 Read-only Directory Specify a read-only dir as input The files inside a read-only dir should be listed Assume that size and ascending is taken care
23 22 Read-only Sub directories Have some read-only sub directories inside the parent dir The files inside a read-only dir should be listed Assume that size and ascending is taken care
24 23 Empty Dir Give a valid dir name - but empty one No Files should be listed
25 24 Dir Name - Special Chars Have some special chars in the name of the dir Should give the list of files Assume that size and ascending is taken care
26 25 Sub Directories - Breadth Have many sub directories in the parent dir… Have a breadth of say 10-20 dirs.
and files inside those directories
Should give the list of files Assume that size and ascending is taken care
27 26 Sub Directories - Depth Have a sub directoy inside a sub directory inside a sub dir…. Have a depth of say 10-20 dirs.
and files inside those directories
Should give the list of files Assume that size and ascending is taken care
28 27 Shared Dir in Windows Specify a shared directory Should give the list of files Assume that size and ascending is taken care
29 28 Specify the URI as dir name A dir on the intranet - ex: \\10.192.23.212\myDir1 Should give the list of files if accessible Assume that size and ascending is taken care
30 29 Dir Name - "." Specify "." as the dir name Should give the list of files in the current dir Assume that size and ascending is taken care
31 30 Dir Name - ".." Specify ".." as the dir name Should give the list of files in the parent dir Assume that size and ascending is taken care
32 31 Dir Name - Absolute Path - Forward Slash Specify "C:/Testing/MyDir1" as the dir name Should give the list of files Assume that size and ascending is taken care
33 32 Dir Name - Absolute Path - Back Slash Specify "C:\Testing\MyDir1" as the dir name Should give the list of files Assume that size and ascending is taken care
34 33 Dir Name - Relative Path - Forward slash Specify "MyDir1/MyDir2/MyDir3" as the dir name Should give the list of files Assume that size and ascending is taken care
35 34 Dir Name - Relative Path - Backward slash Specify "MyDir1\MyDir2\MyDir3" as the dir name Should give the list of files Assume that size and ascending is taken care
36 35 Dir Name - Relative Path Specify "..\..\MyDir2\MyDir3" as the dir name Should give the list of files Assume that size and ascending is taken care
37 36 Dir Name - integer The dir name is "12345" Should give the list of files Assume that size and ascending is taken care
38 37 Dir Name - very long name Ex: The dir name is "Thisismydirectoryandthenameisverybig" Should give the list of files Assume that size and ascending is taken care
39 38 Negative Test Specify some file name as input Throw error message
40 39 Negative Test Specify some non existing dir name Throw error message
41 40 Negative Test Specify null Throw error message
42 41 Formats All possible format files should be listed - ex: ".zip" ".doc" ".pdf" etc Should give the list of files Assume that size and ascending is taken care
43 42 Negative Test Specify a zip file as input Throw error message
44 43 Negative Test Specify the dir name + some blanks Should trim
45




46
Unix/Linux


47 44 Perform all the above tests for Linux/Unix; Also perform the below special cases for Unix


48




49 45 Dir Name - Mounted dir Specify a mounted dir as the input dir name Should give the list of files Assume that size and ascending is taken care
50 46 Dir Name - Case Sensitive Have directories with case sensitive - ex: myDir1, mYdIR2 etc Should give the list of files Assume that size and ascending is taken care
51 47 Read-Only Files Have some read only files in the dir These should be listed
52 48 Read-only Directory Specify a read-only dir as input The files inside a read-only dir should be listed Assume that size and ascending is taken care
53 49 Read-only Sub directories Have some read-only sub directories inside the parent dir The files inside a read-only dir should be listed Assume that size and ascending is taken care
54 50 Dir Name - Special Chars Have some special chars in the name of the dir Should give the list of files Assume that size and ascending is taken care
55 51 Dir Name - "." Specify "." as the dir name Should give the list of files in the current dir Assume that size and ascending is taken care
56 52 Dir Name - ".." Specify ".." as the dir name Should give the list of files in the parent dir Assume that size and ascending is taken care
57 53 Dir Name - Absolute Path - Forward Slash Specify "/opt/testing/MyDir1" as the dir name Should give the list of files Assume that size and ascending is taken care
58 54 Dir Name - Absolute Path - Back Slash Specify "opt\testing\MyDir1" as the dir name Should give the list of files Assume that size and ascending is taken care
59 55 Dir Name - Relative Path - Forward slash Specify "MyDir1/MyDir2/MyDir3" as the dir name Should give the list of files Assume that size and ascending is taken care
60 56 Dir Name - Relative Path - Backward slash Specify "MyDir1\MyDir2\MyDir3" as the dir name Should give the list of files Assume that size and ascending is taken care
61 57 Dir Name - Relative Path Specify "..\..\MyDir2\MyDir3" as the dir name Should give the list of files Assume that size and ascending is taken care
62




63
Performance Testing


64 58 50,000 Files( all > 1000 KB) Have a dir with 50,000 Files Should give the list of files - check for breakdown Assume that size and ascending is taken care
65 59 50,000 Files > 1000 KB + 50,000 Files < 1000 KB Have a dir with 50,000 Files > 1000 KB and 50,000 Files < 1000 KB Should give the list of files - check for breakdown Assume that size and ascending is taken care
66




67 60 10,000 Sub directories Have 10,000 sub directories with files Should give the list of files - check for breakdown Assume that size and ascending is taken care








Sunday, January 18, 2009

Testing Private methods

Hi,
Here is a good article on testing private methods...

http://www.artima.com/suiterunner/private.html

-ahamad

Thursday, September 18, 2008

Code coverage tool for UI

I was looking for a code coverage tool for Java based UI, I came across this tool – ECLEmma.

EclEmma is a free Java code coverage tool for Eclipse, available under the Eclipse Public License.

home page : http://www.eclemma.org/

Below are the steps that are to be followed to setup EclEmma for code coverage:

  1. Open Eclipse; Goto "Help->Software Updates->Find and Install" menu
  2. Click "Next >" In Install window and add a new remote site by clicking on "New Remote Site" button
  3. Enter any name and specify the URL as http://update.eclemma.org
  4. Click finish to search and install the EclEmma Java Code Coverage plug-in
  5. Restart Eclipse when prompted
  6. Now a new icon can be seen next to Run or Debug icons.

  1. Click on arrow next to Coverage icon and select "Open Coverage Dialog"
  2. Select “Eclipse Application” and click on “New” to create a new configuration.
  3. Goto "Coverage" tag and select only the packages in which the source for your feature is checked-in. You can contact the developer or see the folders in changelists to know the list of all packages used by the feature.
  4. Click on "Apply" after all the required packages are selected and then on "Coverage" button to launch your app in "Code Coverage" mode.
  5. Conduct Manual / Automated testing to cover the feature that you are trying to test and close your app.
  6. Now you can see a new tab with the name "Coverage" near "Console" or "Problems" tab
  7. Navigate to exact Java files where the source of your feature is written and check out the percentage of coverage done.
  8. You can double-click on Class or Function to navigate to corresponding Java file and see what parts of code were not exercised and write more test cases to cover the un-exercised code.
  9. You can export the whole of Coverage results to HTML format by right-clicking on any row and selecting "Export Session"
  10. You can have multiple sessions and then you can merge them together.