A

Google

Google

Quick Test Professional

1. How QTP scripts are advantageous when need to re-work the same business scenarios?A. To record the Quick Test Professional Script on one instance and be able to execute it on any other instance. The assumption is there are no major GUI changes between the recorded and the execution instances.

2. How can you make the scripts generic and what architecture needs to be followed?A. In order to achieve the above objective, we need to plan the Quick Test Professional script. It should have two parts:1. Script – that is generic Quick test script.2. Data – from the parameter file that is customer instance specific. Eg. Imagine a business flow has a scenario- Login to the web page- Check mail- Logout1. Create data/parameter file (can be flat file [.txt] or an excel file) is instance specific. 2. Create (record/program) the QTP initialization 3. The initialization script which calls the Object repository, common function library and all QTP actions/scripts

3. How to create an individual script?A. Record the Quick Test Professional script and modify it to make it a generic script using the following steps:a. Set the testing options in the Test Settings Dialog box.b. Record the script c. Modify the script to make it generic:i. Parameterize the hard coded values.ii. Modify the Object Repository file with regular expressions.iii. Add unique properties to items in the Object Repository to make recognition simpler.iv. Insert synchronization points as required.v. Insert checkpoints for pass/fail validation.vi. Insert additional programming logic.vii. Use the Common Functions.

4. What is the testing process in QTP?
A. The testing process consists of 3 main phases:1. Gather relevant test information – Input data should be gathered.2. Create the basic test – Record/Program the scripts in actions3. Enhance the basic testUse Data Driven scripts to use the script to test with different sets of dataReusable Actions- Identify the common scenarios and make the scripts generic and reuse the scripts across different business scenarios.

5. What are different types of recording modes in QTP?
A. There are two recording modes 1. Low-level 2. Analog
Low – Level Recording:Use Low-Level Recording when you need to record the exact location of the object on your application screen. To record in Low-Level Recording, select “Low-Level Recording” from the “Test” menu while recording Analog:Use Analog Recording for applications in which the actual movement of the mouse is what you want to record. To record in Analog mode, select “Analog Recording” from the “Test” menu while recording.

6. What is Object repository?
A. The Object Repository dialog box displays a test tree of all objects in the current action or the entire test (depending on the object repository mode you choose when you create your test). You can use the Object Repository dialog box to view or modify the properties of any test object in the repository or to add new objects to your repository.Object Repository ModesPer – Action Object Repository:This is the default setting, and all tests created in QuickTest 5.6 or earlier use this mode. In this mode, QuickTest automatically creates an object repository file for each action in your test so that you can record and run tests without creating, choosing, or modifying object repository files.Shared Object Repository:In this mode, you can use one object repository file for multiple tests if the tests include the same objects. Object information that applies to many tests is kept in one central location. When the objects in your application change, you can update them in one location for multiple tests

7. How to select the Object repository mode?
A. To select the object repository modeGoto Test Settings > Resource Tab to change the Object Repository Mode The default object repository file name is default.tsrYou can change object repository mode when the Test contain no actions.

8. What is Active screen? What are the advantages of Active screen?A. Active screen captures all the properties of the application and makes available even when offline/ when you are not connected to the application. The main advantage is checkpoints can be added without connecting to the application

9. What are different Screen capture options available for Active screen?
A. Complete—Captures all properties of all objects in the application’s active window/dialog box/Web page in the Active Screen of each step. This level saves Web pages after any dynamic changes and saves Active Screen files in a compressed format.Partial—(Default). Captures all properties of all objects in the application’s active window/dialog box/Web page in the Active Screen of the first step performed in an application’s window, plus all properties of the recorded object in subsequent steps in the same window. This level saves Web pages after any dynamic changes and saves Active Screen files in a compressed format.Minimum—Captures properties only for the recorded object and its parent in the Active Screen of each step. This level saves the original source HTML of all Web pages (prior to dynamic changes) and saves Active Screen files in a compressed format.None—Disables capturing of Active Screen files for all applications and Web pages.

10. How QTP identifies the objects in the application during runtime?
A. QTP uses different properties to identify the objects in the applications. They are:a. Mandatory Propertiesb. Assistive Propertiesc. Object Identifiesd. Smart Identification

11. Explain all Object identification properties.
A. Mandatory and Assistive Properties:During the test run, QuickTest looks for objects that match all properties in the test object description - it does not distinguish between properties that were learned as mandatory properties and those that were learned as assistive propertiesSmart Identification: QuickTest uses a very similar process of elimination with its Smart Identification mechanism to identify an object, even when the recorded description is no longer accurate. Even if the values of your test object properties change, QuickTest’s TestGuard technology maintains your test’s reusability by identifying the object using Smart Identification.

12. What are Ordinal identifies. Explain in detail.
A. Ordinal Identifiers are Index:Indicates the order in which the object appears in the application code relative to other objects with an otherwise identical description.Location:Indicates the order in which the object appears within the parent window, frame, or dialog box relative to other objects with an otherwise identical description. Values are assigned from top to bottom, and then left to right. The Web Browser object has a third ordinal identifier type: Creation Time:Indicates the order in which the browser was opened relative to other open browsers with an otherwise identical description.

13. What is Smart Identification?
A. Smart Identification:If QuickTest is unable to find any object that matches the recorded object description, or if it finds more than one object that fits the description, then QuickTest ignores the recorded description, and uses the Smart Identification mechanism to try to identify the object. While the Smart Identification mechanism is more complex, it is more flexible, and thus, if configured logically, a Smart Identification definition can probably help QuickTest identify an object, if it is present, even when the recorded description fails.

14. What are the properties available in Smart identification?
A. Base filter properties:The most fundamental properties of a particular test object class; those whose values cannot be changed without changing the essence of the original object. For example, if a Web link’s tag was changed from Optional filter properties:Other properties that can help identify objects of a particular class as they are unlikely to change on a regular basis, but which can be ignored if they are no longer applicable.

15. What is Object Spy? How is it used in QTP?
A. Using the Object Spy, you can view the run-time or test object properties and methods of any object in an open application. You use the Object Spy pointer to point to an object. The Object Spy displays the selected object’s hierarchy tree. It displays the run-time or test object properties and values of the selected object in the Properties tab. It displays the run-time or test object methods associated with the selected object in the Methods tab

16. What are Run-Time Object Properties / Run-Time Object Methods?
A. Run-Time Object Properties / Run-Time Object Methods:You can use the Object property to access the native properties of any run-time object. For example, you can retrieve the current value of the ActiveX calendar’s internal Day property as follows: Eg. Sample codeDim MyDay Set MyDay = Browser('index').Page('Untitled').ActiveX('MSCAL.Calendar.7').Object.Day

17. What are Test Object Properties / Test Object Methods?
A. Test Object Properties / Test Object Methods: You can use the GetTOProperty and SetTOProperty methods to retrieve and set the value of test object properties for test objects in your test. You can use the GetROProperty to retrieve the current property value of objects in your application during the test run.

18.What are User-Defined Test Object Classes. How are they mapped?
A User-Defined Test Object Classes: The Object Mapping dialog box enables you to map an object of an unidentified or custom class to a Standard Windows class. For example, if your application has a button that cannot be identified, this button is recorded as a generic WinObject. You can teach QuickTest to identify your object as if it belonged to a standard Windows button class. Then, when you click the button while recording a test, QuickTest records the operation in the same way as a click on a standard Windows button. When you map an unidentified or custom object to a standard object, your object is added to the list of Standard Windows test object classes as a user-defined test object. You can configure the object identification settings for a user defined object class just as you would any other object class

19. What are checkpoints?
A. A checkpoint is a verification point that compares a current value for a specified property with the expected value for that property. This enables you to identify whether your Web site or application is functioning correctly. When you add a checkpoint, Quick Test adds a checkpoint with an icon in the test tree and adds a Check Point statement in the Expert View. When you run the test, Quick Test compares the expected results of the checkpoint to the current results. If the results do not match, the checkpoint fails. You can view the results of the checkpoint in the Test Results window.

20. What is a standard checkpoint?
A. You can check that a specified object in your application or on your Web page has the property values you expect, by adding a standard checkpoint to your test. To set the options for a standard checkpoint, you use the Checkpoint Properties dialog box.

21. What is Text or Text Area Checkpoint?
A. Text or Text Area Checkpoint ResultsBy adding text or text area checkpoints to your tests, you can check that a text string is displayed in the appropriate place in your application or on your Web page. When you run your test, Quick Test compares the expected results of the checkpoint to the actual results of the test run. If the results do not match, the checkpoint fails.

23. What is Bitmap Checkpoint?
A. Bitmap Checkpoints:You can check an area of a Web page or application as a bitmap. While creating a test, you specify the area you want to check by selecting an object. You can check an entire object or any area within an object. Quick Test captures the specified object as a bitmap, and inserts a checkpoint in the test. You can also choose to save only the selected area of the object with your test in order to save disk space.

24. What is Table and Database Checkpoint?
A. Table and Database Checkpoints:By adding table checkpoints to your tests, you can check that a specified value is displayed in a cell in a table on your Web page or in your application. By adding database checkpoints to your tests, you can check the contents of databases accessed by your Web page or application. The results displayed for table and database checkpoints are similar. When you run your test, Quick Test compares the expected results of the checkpoint to the actual results of the test run. If the results do not match, the checkpoint fails.

25. What is Accessibility Checkpoint?
A. Accessibility Checkpoints:You can add accessibility checkpoints to help you quickly identify areas of your Web site that may not conform to the W3C (World Wide Web Consortium) Web Content Accessibility Guidelines. You can add automatic accessibility checkpoints to each page in your test, or you can add individual accessibility checkpoints to individual pages or frames.

26. What is XML Checkpoint?
A. XML Checkpoint:The XML Checkpoint Properties dialog box displays the element hierarchy and values (character data) of the selected XML file.Select the element(s), attribute(s), and/or value(s) that you want to check. For each element you want to check, select the checks you want to perform. For each attribute or value you want to check, select the checks you want to perform, or the parameterization options you want to set.

27. What is Synchronization?
A. When you run tests, your application may not always respond with the same speed. For example, it might take a few seconds: for a progress bar to reach 100%§ for a status§ message to appear for a button to become enabled§ for a window or§ pop-up message to openYou can handle these anticipated timing problems by synchronizing your test to ensure that Quick Test waits until your application is ready before performing a certain step.

28. What are different functions available for Synchronization?
A. There are several options that you can use to synchronize your test: You can insert a synchronization point, which instructs Quick Test to pause the test until an object property achieves the value you specify. When you insert a synchronization point into your test, Quick Test generates a WaitProperty statement in the Expert View.

29. What is the difference in Exists/wait statements?
A. Exist ()/ Wait()You can insert Exist or Wait statements that instruct QuickTest to wait until an object exists or to wait a specified amount of time before continuing the test.Eg. Browser('Yahoo”).Page('CheckMail”).Button(“CheckMail”).Exists(10)QTP waits for 10 seconds till the button exists in the page. The script proceeds if the button even exits before 10 seconds unlike wait() statement – it waits for 10 seconds no matter the button exits before 10 seconds. 30. What is Default Time Out?A. Default Time Out:You can also increase the default timeout settings in the Test Settings and Options dialog boxes in order to instruct Quick Test to allow more time for certain events to occur

31. What is Parameterization (Data Table Wizard)?
A. You can supply the list of possible values for a parameter by creating a Data Table parameter. Data Table parameters enable you to create a data-driven test (or action) that runs several times using the data you supply. In each repetition, or iteration, Quick Test substitutes the constant value with a different value from the Data Table.

32. What are Method Arguments?
A. Using Method arguments you parameterize method arguments in the Method Arguments dialog box. to open the Method Arguments dialog box, right-click a step containing a method in the test tree and choose Method Arguments. The Method Arguments dialog box opens and displays the method arguments in the step.

33. Well, I would like to run my test with different sets of data, How can I make it with the options available in QTP?
A. Listed are the different Data Table IterationsRun one iteration only:Runs the test only once, using only the first row in the global Data Table. Run on all rows:Runs the test with iterations using all rows in the global Data Table. Run from row __ to row __ :Runs the test with iterations using the values in the global Data Table for the specified row range.34. What are different data tables available?A. 1. Global Sheet The Global sheet contains the data that replaces parameters in each iteration of the test.2. Action Sheets Each time you add a new action to the test, a new action sheet is added to the Data Table. Action sheets are automatically labeled with the exact name of the corresponding action. The data contained in an action sheet is relevant for the corresponding action only.

35. What is an Action?
A. An Quick test script contains different actions. An action contains the script ie. A piece of business scenario like, login to application, logout etc.Well again It depends on how you create your framework ( If you would like to know more about frame work check out this link. Good one. http://www-128.ibm.com/developerworks/rational/library/591.html) for testing the applications. I would suggest every action has a piece of business scenario which would help to re-use the script in a better way. Before deciding what are re-usable scripts. Firstly, identify all the common scenarios that occur in different business flows across different modules.Then prepare the scripts and make generic. You can call all these functions by making this common function library available at Test options > Resourses.

36. What is Copy of action?
A. Copy of Action:When you insert a copy of an action into a test, the action is copied in its entirety, including checkpoints, parameterization, and the corresponding action tab in the Data Table. If the test you are copying into uses per-action repository mode, the copied action’s action object repository will also be copied along with the action.

37. What are re-usable actions?
A. Reusable Actions:Determines whether the action is a reusable action. A reusable action can be called multiple times within a test and can be called from other tests. Non-reusable actions can be copied and inserted as independent actions, but cannot be inserted as calls to the original action.

38. what about Call of Action?
A. You can insert a call (link) to a reusable action that resides in your current test (local action), or in any other test (external action). 39. When to Insert transactions?A. Inserting Transactions: During the test run, the Start Transaction§ signals the beginning of the time measurement. You define the beginning of a transaction in the Start Transaction dialog box The End Transaction§ signals the end of the time measurement

40. What are reular expressions?
A. Regular Expressions:Regular expressions enable QuickTest to identify objects and text strings with varying values. You can use regular expressions when: • Defining the property values of an object • Parameterizing a step • Creating checkpoints with varying values A regular expression is a string that specifies a complex search phrase. By using special characters such as a period (.), asterisk (*), caret (^), and brackets ([ ]), you can define the conditions of a search. When one of these special characters is preceded by a backslash (\), QuickTest searches for the literal character.Here is an example:The actual pattern for the regular expression search is set using the Pattern property of the RegExp object. The RegExp.Global property has no effect on the Test method. The Test method returns True if a pattern match is found; False if no match is found. The following code illustrates the use of the Test method. Function RegExpTest(patrn, strng)Dim regEx, retVal ' Create variable.Set regEx = New RegExp ' Create regular expression.regEx.Pattern = patrn ' Set pattern.regEx.IgnoreCase = False ' Set case sensitivity.retVal = regEx.Test(strng) ' Execute the search test.If retVal ThenRegExpTest = 'One or more matches were found.'ElseRegExpTest = 'No match was found.'End IfEnd FunctionMsgBox(RegExpTest('is.', 'IS1 is2 IS3 is4'))41. Create a script to print the messageA. Dim MyVarMyVar = MsgBox ('Hello World!', 65, 'MsgBox Example')' MyVar contains either 1 or 2, depending on which button is clicked.42. List all the run time errors in VB script.A. VBScript run-time errors are errors that result when your VBScript script attempts to perform an action that the system cannot execute. VBScript run-time errors occur while your script is being executed; when variable expressions are being evaluated, and memory is being dynamic allocated.

Error Number Description
429 ActiveX component can't create object
507 An exception occurred
449 Argument not optional
17 Can't perform requested operation
430 Class doesn't support Automation
506 Class not defined11 Division by zero
48 Error in loading DLL5020 Expected ')' in regular expression
5019 Expected ']' in regular expression4
32 File name or class name not found during Automation operation92 For loop not initialized5008 Illegal assignment
51 Internal error
505 Invalid or unqualified reference
481 Invalid picture5 Invalid procedure call or argument
5021 Invalid range in character set94 Invalid use of Null
448 Named argument not found
447 Object doesn't support current locale setting
445 Object doesn't support this action
438 Object doesn't support this property or method
451 Object not a collection504 Object not safe for creating
503 Object not safe for initializing
502 Object not safe for scripting
424 Object required
91 Object variable not set
7 Out of Memory
28 Out of stack space
14 Out of string space
6 Overflow
35 Sub or function not defined
9 Subscript out of range
5017 Syntax error in regular expression
462 The remote server machine does not exist or is unavailable10 This array is fixed or temporarily locked
13 Type mismatch
5018 Unexpected quantifier
500 Variable is undefined
458 Variable uses an Automation type not supported in VBScript
450 Wrong number of arguments or invalid property assignment

Database Question

What is As the difference among “dropping a table”, “truncating a table” and “deleting all records” from a table.
Answer :
Drop Table - will remove the existence of the table from the database along with its data and structure and all the constraints. The table will be no longer available.

A Truncate Table - will remove all the rows (only the rows) from a table. it will not delete the table. Its a DDL statement that means the deleted rows cannot be reverted back by ROLLBACK statement;

Delete Table - is a DML statement which will delete rows from a table according to the matching criteria mentions in the ‘where’ clause. And these rows can be reverted back by ‘ROLLBACK’ statement if ‘COMMIT’ is not fired.

What are the Large object types supported by Oracle?
Answer :
These are the large object type supported bye oracle
• CLOB and LONG for large fixed-width character data
• NCLOB for large fixed-width national character set data
• BLOB and LONG RAW for storing unstructured data
• BFILE for storing unstructured data in operating system files

What are the different types of joins?
Answer :

Different Types of Joins-
Inner Join
Outer Join - Right Outer Join & Left Outer Join
Cross Join

What is snapshot log?
Answer :

It is a table that maintains a record of modifications to the master table in a snapshot. It is stored in the same database as master table and is only available for simple snapshots. It should be created before creating snapshots.

Explain the difference between MyISAM Static and MyISAM Dynamic.
Answer :
In MyISAM static all the fields have fixed width. The Dynamic MyISAM table would include fields such as TEXT, BLOB, etc. to accommodate the data types with various lengths. MyISAM Static would be easier to restore in case of corruption, since even though you might lose some data, you know exactly where to look for the beginning of the next record.

How can you Enforce Referential Integrity in snapshots?
Answer :

Time the references to occur when master tables are not in use. Perform the reference the manually immediately locking the master tables. We can join tables in snapshots by creating a complex snapshots that will based on the master tables.

What is Two-Phase Commit?
Answer :

Two-phase commit is mechanism that guarantees a distributed transaction either commits on all involved nodes or rolls back on all involved nodes to maintain data consistency across the global distributed database. It has two phase, a Prepare Phase and a Commit Phase.
What is a cluster Key?
Answer :
The related columns of the tables are called the cluster key. The cluster key is indexed using a cluster index and its value is stored only once for multiple tables in the cluster.

What is Read-Only Transaction?
Answer :

A Read-Only transaction ensures that the results of each query executed in the transaction are consistent with respect to the same point in time.

What is Rule-based approach to optimization?
Answer :

Choosing an executing plan based on the access paths available and the ranks of these access paths.

What are constraints? Explain different types of constraints?
Answer
# 1 constraint is a rule which can not be voilated by end users.
Different types of constraints are available.They are:-
1)default constraint:-which is used to define a default
value.
2)primary key:-the key which does not allows duplication
and null values.
3)foreign key:-the key used to refer primary key defined
field in another table and it allows duplication.
4)null:-which allows NULL values.
5)not null:-which does not allows NULL values.
6)unique key:-which does not allows duplication but allows
NULL values.

----------------------------
Testing

Question Find the Cyclometric complexity on the following pseudocode


Do while records remain
read record;
if record field#1=0
then process record;store in buffer;
incremental counter;store in file.
reset counter;end if end do

please let me know the procedure too.

Q.Find the Cyclometric complexity on the following p seudocode Do while records remain read record; if record field#1=0 then process record;store in buffer; incremental counter;store in file. reset counter;end if end do please let me know the procedure too.
Answer
# 1 Listen i will tell u a basic funda:

Cylometric complexity is THE NO. OF LOOPS + 1

so in the question u have to check how many loops r present.

if its 1 then the ans is 2
if its 0 then the nas is 1
-----------
Answer
# 2 The answer for this is 3

3 formula's to calculate Cyclomatic complexity. You can
choose either of them...
C.C = No of bounded regions +1
C.C = No of edges - No of nodes +1
C.C = No of predicate Nodes +1

Now let's consider the first formula to calculate as the
rest 2 diagrams require flow graphs to be drawn.

Do while loop forms 1 bounded region
and IF condition forms 1 bounded region.

So as per this formula, C.C = No of bounded regions +1
2 + 1 = 3
-------------
What is Retesting?
Answer
# 1 Executing the same test cases on the same build but with
different inputs..

Answer
# 2 re execution of the test cases on the same build with
diffrent set of inputs

Answer
# 3 re executing the cases to get the confirmation that defects
are reproducable

What is defect leaking and defect release
Answer
# 1 While verifying the defect if it has been found tobe not
fixed, then it is called defect leaking.

If it has been fixed, then the defect is said to be release

Answer
# 2 when a version is released say ex:version2.0 and is any
modifications have been made to this version and named to
version 2.1 and the u find any (small) defect in the new
version which was left in 2.0 the tht is called defect leakage

A Succesful testcase Means..?
Answer
# 1 it should cover the functionality,it doed not contain if
staement,high probabilty of finding defect

Answer
# 2 A good testcase should satisfy the following two basic
requirements:
1.It should cover all the requirements as mentioned in the
srs
2.For each requirement different scenarios that are
possible should also be mentioned.

Answer
# 3 should cover functionality
should be easy to understand
should be maintanable

Answer
# 7 a Successful test case uncovers as an yet undiscovered
defect

Answer
# 8 to cover all functionality and intended to find bugs

Question Suppose " You are given a build with 5 modules to test as
per the Customer request after finalization of SRS, But the
SRS contains only documentation about 4 modules. On what
basis you will continue the testing of 5th module." ?

Answer
# 1 is developer problem

Answer
# 2 If it is not in SRS, it is bug. Any extra sec. is a bug
also.

Answer
# 3 I think we do Adhoc Testing. Upto 4 modules we first check
the functionality. After that we do the Adhoc on the 5th
module.
Generally we do Adhoc Testing, when there is no clear
specifications or when the time is less.

Answer
# 4 SRS is prepared based on requirements. Even SRS is not
there and have requirements, we analyze the requirements
first, Take the inputs from domain experts,end users.
Consolidate and prepare the SRS on our own and will send
the same for customer approval. based on feed back we can
go with testing.

Answer
# 5 Along with the Build release to the Testing team
Devolopment people releases Build release document, by
studing the release document we are going to conduct
testing on 5 th module.

Answer
# 6 if srs is available prepare test cases and to conduct
formal testing
if SRS is not available in this time we conduct adhoc testing.



Question Testing of Non-Functional requirements does not involove
a) Performance tests
b) Reliability tests
c) usability tests
d) Configuration tests

Answer
# 1 Configuration test


Answer
# 2 reliability tests & configuration tests

Answer
# 3 non functional reuirements includes all ie. performance,
reliability , usability & configiration tests.

Answer
# 4 all are non functional req. i guess...well done poonam

Answer
# 5 Reliability , Usability and Configuration comes under
Functional requirements, and Performance comes under non
functional requirements..

Answer
# 6 In Non-functionality requirements .We consider 1)performance
2)security that's it

What is metrics? explian the type of metrices.
Answer
# 3 Metrics are generally classified into 2 types
1. Process metrics
2. Product metrics
b a metric used to measure the haracteristic of the
methods, techniques and tools employed in eveloping,
implementing and maintaining the software system.


Product metric: a metric used to measure the characteristic
of the documentation and code. The metrics for the test
process would include status of test activities against the
plan, test coverage achieved so far, amongst others. An
important metric is the number ofdefects found in internal
testing compared to the defect found in customer tests,
which indicate the effectiveness of the test process itself.

Answer
# 4 Metrics meaures the strength of testing process

In otherwords it measures how much percentage of testing is
complted

types of metrics

Functional covreage

Softwate release metric

Error discovery rate
stability trend chart

what are the Levels of testing?
Answer
# 1 1. Unit Testing
2. Integration Testing
3. System Testing
4. Acceptance Testing
5. Regression Testing

Answer
# 2 there r 4 levels of testing
level 0 :: Sanity Testing test whether build is testable
or not
level 1 :: Unit testing, Integraion Testing, System Testing
level 2 :: Regression Testing
level 3 :: Final Regression testing or Postmartom Testing

Answer
# 3 There are only 4 levels of testing

1. Unit Testing
2. Integration Testing
3. System Testing
4. User Acceptance Testing

Answer
# 4 1.unit testing
2.Integration testing
a)Top down integration testing
b)Bottom up integration testing
c)Big Bang integration testing
d)Isolation integration testing
e)Incremental integration testing
3.System testing
4.Acceptance testing.

Answer
# 5 Levels of testing
1.Unit testing
2.Module testing
3.Interation testing
a)Top down integration testing
b)Bottom up integration testing
c)Big Bang integration testing
d)Isolation integration testing
e)Incremental integration testing
4.System testing
5.User acceptance testing

What kinds of testing have you done?
Answer
# 1 I performed Installation testing, browser compatability
testing, regression testing, parallel testing.

Answer
# 2 Testers can perform Balck box testing,Grey box
testing,Smoke testing,Sanity testing,endurance
testing,automation testing,volume testing,Database
testing,Stress testing,End-end testing,alpha testing,adhoc
testing,concurrency testing ....List will not end!!!!!!!

Answer
# 3 Tester can perform Black box testing,GUI testing,DataBase
testing,Integration(Tester+Developer)System and UAT.As per
Client requirements.

Answer
# 4 Moreover testers perform sanity testing, black box,
regression, performance, volume, system testing and installtion.

Answer
# 5 I would like to add something here

Black Box & White box testing are two approaches of testing
Most commonly every tester perform
1.Functional testing
2.Regression testing
3.Re-testing
4.Sanity testing
5.Compatibility testing
6.UI testing
7.Exploratory tesing (this is depending upon the type of
project)
8.end to end testing
9.adhoc testing
10.Installation testing

Answer
# 7 Kinds of testing means:

1. Static testing & Dynamic testing.
2. Structural testing & Functional testing.
3. Automation testing & Manual testing.

Testers will do only:

Dynamic testing
Functional testing
Manual testing

what type of documents can u prepare during Testing?
Answer
# 1 Test case document, Bug report, Status reports, Tracebility
Matrix, Use cases, Metric preparation, Release report

Answer
# 2 1.Test Plan : A formal test plan is a document that
provides and records important information about a test
project, for example: Resources, Schedule & Timeline, Test
Milestones, Use cases and/or Test cases
2.Test Case: A set of test inputs, execution conditions and
expected results.
3.Test Data: The actual (set of) values used in the test or
that are necessary to execute the test.
4.Test Scripts: is used to test a particular functionality
(business rule). It may consist of 1 or more Test Cases.
5.Test Log: A chronological record of all relevant details
about the execution of a test
6.Bug Reports: Contains a summary of the bug, its priority
and other details regarding the bug.

Answer
# 3 BRD business requirment document.
srs system requirment specifications.
frs functional requirment specifications,
Test plan: it include what to test,when to test ,where to
test, whom to test,how to test.
test cases : functionality to be test
defwct reporting :
bug report :

Answer
# 4 Here I ma giving all the documents,which are using in our
project and the authorized person,who is preparing them.It
varies from company to company.

Project Plan:Prepared by PM.
QA-Project Plan:Prepared by QA Manager
Test Reports:Generally prepared by Team lead for every
release.
Software Metrics: Team Lead
Module based Test Plan: Testing engineer
Test Results: Test Engineer
Testcase document: Test Engineer
Defect Matrix: Prepared by Team Lead
Traceability Matrix: Team Lead or team member
Bu Reports: Prepared by test engineer.

Define Brain Storming and Cause Effect Graphing? With Eg?
Answer
# 1 Hi...........
BS:
A learning technique involving open group discussion
intended to expand the range of available ideas
OR
A meeting to generate creative ideas. At PEPSI Advertising,
daily, weekly and bi-monthly brainstorming sessions are
held by various work groups within the firm. Our monthly I-
Power brainstorming meeting is attended by the entire
agency staff.
OR
Brainstorming is a highly structured process to help
generate ideas. It is based on the principle that you
cannot generate and evaluate ideas at the same time. To use
brainstorming, you must first gain agreement from the group
to try brainstorming for a fixed interval (eg six minutes).

CEG :
A testing technique that aids in selecting, in a systematic
way, a high-yield set of test cases that logically relates
causes to effects to produce test cases. It has a
beneficial side effect in pointing out incompleteness and
ambiguities in specifications.

What is software development Life Cycle (SDLC)? where exactly the Testing activity begin in SDLC?
Answer
# 1 SDLC is a framework for understanding and developing
information and software succesfully.

Answer
# 2 SDLC is a framework for understanding and developing
information and software succesfully.

Testing activity begins right from the requirement analysis
phase.




For Web Applications what type of tests r u going to do .. tell me the Importent ones..
Answer
# 2 The following are the tests need to be done for Web
application.


Static Testing
Dynamic Testing
Navigation Testing
Compatabilty Testing
Functionality Testing
Link Testing

Answer
# 3 for an web application we need 2 check

URL of appl
navigation(Normal links,broken links_
scalability
reliabilty(recovery testing)
usability
compatbility
forms,cookies

Answer
# 4 for web application mainly tests as follows

userinterface means everyone understand the application is
mportant
funcationality
navigation
performence means speed of acces is imp
links

Answer
# 5 Functional Testing
Smoke Testing(for UI and API's-- Through Automation tool)
Regression Testing
Batch Process Testing(for simulators)

---------
Web-based applications present new challenges, these challenges include:
- Short release cycles;
- Constantly Changing Technology;
- Possible huge number of users during initial website launch;
- Inability to control the user's running environment;
- 24-hour availability of the web site.

The quality of a website must be evident from the Onset. Any difficulty whether in response time, accuracy of information, or ease of use-will compel the user to click to a competitor's site. Such problems translate into lost of users, lost sales, and poor company image.

To overcome these types of problems, use the following techniques:
1. Functionality Testing
Functionality testing involves making Sure the features that most affect user interactions work properly. These include :
· forms
· searches
· pop-up windows
· shopping carts
· online payments

2. Usability Testing
Many users have low tolerance for anything that is difficult to use or that does not work. A user's first impression of the site is important, and many websites have become cluttered with an increasing number of features. For general-use websites frustrated users can easily click over a competitor's site.

Usability testing involves following main steps
· identify the website's purpose;
· identify the indented users;
· define tests and conduct the usability testing
· analyze the acquired information

3. Navigation Testing
Good Navigation is an essential part of a website, especially those that are complex and provide a lot of information. Assessing navigation is a major part of usability Testing.

4. Forms Testing
Websites that use forms need tests to ensure that each field works properly and that the forms posts all data as intended by the designer.

5. Page Content Testing
Each web page must be tested for correct content from the user perspective for correct content from the user perspective. These tests fall into two categories: ensuring that each component functions correctly and ensuring that the content of each is correct.

6. Configuration and Compatibility Testing
A key challenge for web applications is ensuring that the user sees a web page as the designer intended. The user can select different browser software and browser options, use different network software and on-line service, and run other concurrent applications. We execute the application under every browser/platform combination to ensure the web sites work properly under various environments.

7. Reliability and Availability Testing
A key requirement o a website is that it Be available whenever the user requests it, after 24-hours a day, every day. The number of users accessing web site simultaneously may also affect the site's availability.

8. Performance Testing
Performance Testing, which evaluates System performance under normal and heavy usage, is crucial to success of any web application. A system that takes for long to respond may frustrate the user who can then quickly move to a competitor's site. Given enough time, every page request will eventually be delivered. Performance testing seeks to ensure that the website server responds to browser requests within defined parameters.

9. Load Testing
The purpose of Load testing is to model real world experiences, typically by generating many simultaneous users accessing the website. We use automation tools to increases the ability to conduct a valid load test, because it emulates thousand of users by sending simultaneous requests to the application or the server.

10. Stress Testing
Stress Testing consists of subjecting the system to varying and maximum loads to evaluate the resulting performance. We use automated test tools to simulate loads on website and execute the tests continuously for several hours or days.

11. Security Testing
Security is a primary concern when communicating and conducting business- especially sensitive and business- critical transactions - over the internet. The user wants assurance that personal and financial information is secure. Finding the vulnerabilities in an application that would grant an unauthorized user access to the system is important.

Following is the strategy we used in one of my projects:
1. From the requirements, list the priorities of testing.
2. Use a Link Checker to check all the broken text and image links. This will alert you on missing or broken links but not links that have been pointed to the unintended locations/files. But ignore such links for now this is just one of the cursory checks. Report the results.
3. Collect as many scenarios of real users browsing your site. If you are testing an already existing site you can get them from the site Stats. These will probably cover all your existing user base. If your site needs to target additional user base that is a whole other task.
4. Test using the scenarios.
5. From your testing priorities, list the test cases not touched in your previous test scenarios. If you want a complete list or have time to create one, just go ahead or else you can also list them and note the result from your tests earlier in here. We just listed a short description of each test case, it's input and output. It'll be helpful to record your results if you've a column in there to record your results when testing.
6. Start testing using the test cases and record the results.
7. Prioritize and Report the defects. If you don't have time for testing, there might not be sufficient time to fix them all and re-test the defects either. Prioritizing them will help fix at least the major bugs and/or issues. The rest will be fixed or ignored depending on their priority and your project deadlines.

What are the test methodologies??
Answer
# 1 Test Methodolgies defines how to approach the
testing .project type,system domain,risks,at which phase
what type of testing occurs,

all these comes under Methodology

What are the contents in "Requirements Traceability Matrix" not in " Test Responsibility Matrix"?
Answer
# 1 Contents of Requirements Traceability Matrix / RTM :

Release Name
Test Case Name
High level design
Low Level Design
Test Case Execution
Automation Execution

Difference between regression testing and re testing?
Answer
# 1 Retesting- Re test the defects after fixed.
Regression testing-- Nothing but retest, After fix the
defects we are checking the functinality,if the fixed
defect is effected to another functionality.

Answer
# 2 Retesting : Testing the application on the same built
soppose tester found the bugs and developer fixed it.
tested the fixed issues on the same built is called
retesting.
Regression: Testing the application on the different built
Suppose: if the tester found the bugs, and Developer fixed
it and they relised the new version of the build. so
testing on the new version on the built is called Regerssion
Testing

Answer
# 3 re testing:testing the build with multiple test data
regression testing:if the tester find any defect he /she
inform this defect to development team.developement team
fixes that bug, aagin send to the testing team for testing
again whether bug is fixed or not,or the newly developed
build giving any side affects to other modules or not.

Answer
# 4 Re Testing: Once the Bug is fixed Tester has to test the
same bug with multiple number of times with different data
and different scenario. Also once tester find the bug he
has to test multiple number of times bcoz whether the bug
is reproduceable or not.

Regression Testing: Once the Bug is fixed tester has to
test whether bug is fixed or not also test whether the
impact of the bug is shows in other application

What is the diffrence between BUILD & RELEASE ?
Answer
# 1 Difference number one:Builds refer to software that is
still in testing, release refers to software that is
usually no longer in testing.

Difference number two: Builds occur more frequently;
releases occur less frequently.

Answer
# 2 Generally build is a inegrated module,which contains the
fixes for your previous builds bugsGenerally we will
recievie the builds in daily basis,weekly basis.

Once we completed the testing on our software,before we
rolling out that software to our client,we will select a
final build,which is almost defect free and we will take
that build as release build.That release build will be
introduced in the market for end-users.

What is risk analysis,what type of risk analysis u did in u r project
Answer
# 1 hi.................(Must say u have a very good question )
Risk Analysis :
A systematic use of available information to determine how
often specified events and unspecified events may occur and
the magnitude of their likely consequences

OR

procedure to identify threats & vulnerabilities, analyze
them to ascertain the exposures, and highlight how the
impact can be eliminated or reduced

Types :

1.QUANTITATIVE RISK ANALYSIS
2.QUALITATIVE RISK ANALYSIS

what is test case? what it consists? what are the types of test cases?
Answer
# 1 Test Case is a document that describes the input or action
or event and expected response to determine the feature of
the application is working correctly. It contains Test case
No: Action/Input and Expected and Actual Result and Status
(Pass/Fail).


Answer
# 2 Test case is a document specifying inputs, predicted
results, and a set of execution conditions for a test
item.We can name test cases as system test
cases,integration test cases,Unit test cases (that include
validations)

Answer
# 3 Sharavati,
The Test cases naming which you have mentioned here is said
the be levels of testing in testing life cycle of a
software. Hence you cannot name/catogorize the test cases.

Answer
# 4 Test case is a document.It contains inputs,outputs and
outcome .There are three types of test cases
they are
1)Business logic based testcases
2)Input domain based testcases
3)user interface based testcases

Answer
# 5 Test case is a document.It contains inputs,outputs and
outcome .There are three types of test cases
they are
1)Business logic based testcases
2)Input domain based testcases
3)user interface based testcases

Answer
# 6 For me test cases is a document which describes ,
What is we r going to test ,i.e (test area)
followed by test description ,then by the test steps , then
the expected result ,actual result , status(pass/fail_)

Answer
# 7 Test Case:
Definition: To verify the possible values for the
attribute.

It Cosists:
positive and negative values for the attribute

Types:
GUI test case
Functional test case
System level test case

Answer
# 9 Test case is descrption f what is to be tested, what the
data to be used ,what are the actions done to check the
actual results against expected result



What is the difference between QC and QA?
Answer :
Quality assurance is the process where the documents for the product to be tested is verified with actual requirements of the customers. It includes inspection, auditing , code review , meeting etc.
Quality control is the process where the product is actually executed and the expected behavior is verified by comparing with the actual behavior of the software under test. All the testing types like black box testing, white box testing comes under quality control.
Quality assurance is done before quality control.
What is a scenario?
Answer :
A scenario defines the events that occur during each testing session. For example, a scenario defines and controls the number of users to emulate, the actions to be performed, and the machines on which the virtual users run their emulations.





What is Difference betweem Manual and Automation Testing?
Answer :
This answer is quite simple, Manual is when user needs to do many things based on the test case specified, say like click some tab and check if the tab is working fine or click on a particular URL and check if the web site specified opens.

The above stuff can also be done automatically, i.e. through some automated tools like Winrunner , Silk Test etc the things can be automated , so user just has to trigger the tool and the tool will take care of executing , only thing needed is we need to know how to make the testcases automated through that tool which is not very difficult.

What is L10 Testing?
Answer :
L10 Testing is Localization Testing, it verifies whether your products are ready for local markets or not.

What is I18N Testing?
Answer :
I18N Testing is “Internationalization testing”
Determine whether your developed product’s support for international character encoding methods is sufficient and whether your product development methodologies take into account international coding standards.

What is SEI? CMM? CMMI? ISO? IEEE? ANSI?
Answer :

SEI = ‘Software Engineering Institute’ at Carnegie-Mellon University; initiated by the U.S. Defense Department to help improve software development processes.
CMM = ‘Capability Maturity Model’, now called the CMMI (’Capability Maturity Model Integration’), developed by the SEI. It’s a model of 5 levels of process ‘maturity’ that determine effectiveness in delivering quality software. It is geared to large organizations such as large U.S. Defense Department contractors. However, many of the QA processes involved are appropriate to any organization, and if reasonably applied can be helpful. Organizations can receive CMMI ratings by undergoing assessments by qualified auditors.

Level 1 - characterized by chaos, periodic panics, and heroic efforts required by individuals to successfully complete projects. Few if any processes in place; successes may not be repeatable.
Level 2 - software project tracking, requirements management, realistic planning, and configuration management processes are in place; successful practices can be repeated.
Level 3 - standard software development and maintenance processes are integrated throughout an organization; a Software Engineering Process Group is is in place to oversee software processes, and training programs are used to ensure understanding and compliance.
Level 4 - metrics are used to track productivity, processes, and products. Project performance is predictable, and quality is consistently high.
Level 5 - the focus is on continouous process improvement. The impact of new processes and technologies can be predicted and effectively implemented when required.

ISO = ‘International Organisation for Standardization’ - The ISO 9001:2000 standard (which replaces the previous standard of 1994) concerns quality systems that are assessed by outside auditors, and it applies to many kinds of production and manufacturing organizations, not just software. It covers documentation, design, development, production, testing, installation, servicing, and other processes. The full set of standards consists of: (a)Q9001-2000 - Quality Management Systems: Requirements; (b)Q9000-2000 - Quality Management Systems: Fundamentals and Vocabulary; (c)Q9004-2000 - Quality Management Systems: Guidelines for Performance Improvements.

To be ISO 9001 certified, a third-party auditor assesses an organization, and certification is typically good for about 3 years, after which a complete reassessment is required. Note that ISO certification does not necessarily indicate quality products - it indicates only that documented processes are followed.

IEEE = ‘Institute of Electrical and Electronics Engineers’ - among other things, creates standards such as ‘IEEE Standard for Software Test Documentation’ (IEEE/ANSI Standard 829), ‘IEEE Standard of Software Unit Testing (IEEE/ANSI Standard 1008), ‘IEEE Standard for Software Quality Assurance Plans’ (IEEE/ANSI Standard 730) etc

ANSI = ‘American National Standards Institute’, the primary industrial standards body in the U.S.; publishes some software-related standards in conjunction with the IEEE and ASQ (American Society for Quality).

Other software development/IT management process assessment methods besides CMMI and ISO 9000 include SPICE, Trillium, TickIT, Bootstrap, ITIL, MOF, and CobiT.

What is documentation change management?
Answer :
Documentation change management is part of configuration management (CM). CM covers the tools and processes used to control, coordinate and track code, requirements, documentation, problems, change requests, designs, tools, compilers, libraries, patches, changes made to them and who makes the changes

What is the role of documentation in QA?
Answer :

Documentation plays a critical role in QA. QA practices should be documented, so that they are repeatable. Specifications, designs, business rules, inspection reports, configurations, code changes, test plans, test cases, bug reports, user manuals should all be documented. Ideally, there should be a system for easily finding and obtaining of documents and determining what document will have a particular piece of information. Use documentation change management, if possible.

How do you perform integration testing?
Answer :
First, unit testing has to be completed. Upon completion of unit testing, integration testing begins. Integration testing is black box testing. The purpose of integration testing is to ensure distinct components of the application still work in accordance to customer requirements. Test cases are developed with the express purpose of exercising the interfaces between the components. This activity is carried out by the test team.
Integration testing is considered complete, when actual results and expected results are either in line or differences are explainable/acceptable based on client input.

What is clear box testing?
Answer :
Clear box testing is the same as white box testing. It is a testing approach that examines the application’s program structure, and derives test cases from the application’s program logic.

What is closed box testing?
Answer :
Closed box testing is same as black box testing. Black box testing a type of testing that considers only externally visible behavior. Black box testing considers neither the code itself, nor the “inner workings” of the software.

What is open box testing?
Answer :
Open box testing is same as white box testing. It is a testing approach that examines the application’s program structure, and derives test cases from the application’s program logic.

What is ‘Software Quality Assurance’?
Answer :
Software QA involves the entire software development PROCESS - monitoring and improving the process, making sure that any agreed-upon standards and procedures are followed, and ensuring that problems are found and dealt with. It is oriented to ‘prevention’.

What is Security Testing?
Answer :
Security Testing

Application vulnerabilities leave your system open to attacks, Downtime, Data theft, Data corruption and application Defacement. Security within an application or web service is crucial to avoid such vulnerabilities and new threats.

While automated tools can help to eliminate many generic security issues, the detection of application vulnerabilities requires independent evaluation of your specific application’s features and functions by experts. An external security vulnerability review by Third Eye Testing will give you the best possible confidence that your application is as secure as possible.

Q: How do I test a database?A: As far as I know there are four different things that could beconsidered database testing1) using the database as a repository for some user-generatedfront-end activity (client-server)2) using the database as a repository for some application-generatedactivity (transaction logging)3) database migration (bringing one repository from one schema toanother or merging two or more schemas)4) database development (you work for Oracle, MySQL, or a company thatmakes databases)Which one are you trying to do?How to Thoroughly Test a Data WarehouseAuthor(s): Rebecca Cooper/Suellen Arbucklehttp://www.stickyminds.com/r.asp?F=DART_5941Temporal Data Integrity ValidationAuthor(s): Doron Drusinskyhttp://www.stickyminds.com/r.asp?F=DART_5938Testing Merged DatabasesAuthor(s): Bernie Berger/Lawrence NuanezSTQE Issue: Nov/Dec 2002 (Vol. 4 Issue 6)http://www.stickyminds.com/r.asp?F=DART_6181This article requires a PowerPass. It is summarised in the abstract,which I'll paraphrase for you here:This addresses four different problems that have to be overcome in adatabase merger:1) duplicate records (The challenge of duplicate records);2) mismatched columns (Ways to check for mismatched column definitions);3) data corruption (Tips for finding corrupt data);and 4) front-end assumptions (How to verify front- and back-end congruence).

How can u do regression testing tell me procedure?

Answer
# 1 Regression test will be done when we got modified build there we execute test cases to determine whether defects closed or not? or checking 2 ensure that resolved bug has made any impact
on the remaining part of code

Answer
# 2 Regression Testing is as discussed in above answer is done
after the first build release, and in regression testing we
test if the bugs that exited in the Build are actually fixed
or not, we also apply testcases for new requirements (if
any) to check if the system is working fine.

Answer
# 3 Regarding the regression testing procedure, it is same as
testing, we just re-run all those test that were previously
run, but this time we make sure that the test cases match
the new chnages, or new requirements that were not there in
the system.

Answer
# 4 Regreesion testing is nothing but whenever we find out the
bug , it has to be fixed ,given to developers they will fix
the bug , this is known as regression testing
Answer
# 5 Regression Testing-"Testing to ensure that introduction of
new functionality has not affected existing functionality"

Answer
# 6 Whenever tester gets the next fixed build:
1)He tests for the Fixed Bugs.
2)Test the whole application to check whether these fixes,
have not introduced new bugs.
3)If there are some update in the application, then test
cases need to be updated , and executed the same.

All of the above steps used, while doing regression testing.

Answer
# 7 Reexecution of testcases on modified form of build is
called regression testing.

Answer
# 8 regression testing:when ever the bug is fixed from
developement team we can verify whether the bug is fixed or
not and their related functionality is also working fine or not.

What’s a Test Case?

What’s a Test Case?
Let’s start with the basics. What’s a test case?
IEEE Standard 610 (1990) defines test case as follows:
“(1) A set of test inputs, execution conditions, and expected results developed for a
particular objective, such as to exercise a particular program path or to verify compliance
with a specific requirement.
“(2) (IEEE Std 829-1983) Documentation specifying inputs, predicted results, and a set
of execution conditions for a test item.”
According to Ron Patton (2001, p. 65),
“Test cases are the specific inputs that you’ll try and the procedures that you’ll follow
when you test the software.”
Boris Beizer (1995, p. 3) defines a test as
“A sequence of one or more subtests executed as a sequence because the outcome and/or
final state of one subtest is the input and/or initial state of the next. The word ‘test’ is
used to include subtests, tests proper, and test suites.
defines test case:
“A test case specifies the pretest state of the IUT and its environment, the test inputs or
conditions, and the expected result. The expected result specifies what the IUT should
produce from the test inputs. This specification includes messages generated by the IUT,
exceptions, returned values, and resultant state of the IUT and its environment. Test cases
may also specify initial and resulting conditions for other objects that constitute the IUT
and its environment.”
In practice, many things are referred to as test cases even though they are far from being fully
documented.
Brian Marick uses a related term to describe the lightly documented test case, the test idea:
“A test idea is a brief statement of something that should be tested. For example, if you're
testing a square root function, one idea for a test would be ‘test a number less than zero’.
The idea is to check if the code handles an error case.”
In my view, a test case is a question that you ask of the program. The point of running the test is
to gain information, for example whether the program will pass or fail the test.
It may or may not be specified in great procedural detail, as long as it is clear what is the idea of
the test and how to apply that idea to some specific aspect (feature, for example) of the product.
If the documentation is an essential aspect of a test case, in your vocabulary, please substitute the term “test idea” for “test case” in everything that follows.
An important implication of defining a test case as a question is that a test case must be
reasonably capable of revealing information.
􀂃 Under this definition, the scope of test cases changes as the program gets more
stable. Early in testing, when anything in the program can be broken, trying the
largest “legal” value in a numeric input field is a sensible test. But weeks later, after
the program has passed this test several times over several builds, a standalone test
of this one field is no longer a test case because there is only a miniscule probability
of failure. A more appropriate test case at this point might combine boundaries of ten
different variables at the same time or place the boundary in the context of a longsequence
test or a scenario.
􀂃 Also, under this definition, the metrics that report the number of test cases are
meaningless. What do you do with a set of 20 single-variable tests that were
interesting a few weeks ago but now should be retired or merged into a combination?
Suppose you create a combination test that includes the 20 tests. Should the metric
report this one test, 20 tests, or 21? What about the tests that you run only once?
What about the tests that you design and implement but never run because the
program design changes in ways that make these tests uninteresting?
Another implication of the definition is that a test is not necessarily designed to expose a defect.
The goal is information. Very often, the information sought involves defects, but not always. (I
owe this insight to Marick, 1997.) To assess the value of a test, we should ask how well it
provides the information we’re looking for.
Information Objectives
So what are we trying to learn or achieve when we run tests? Here are some examples:
􀂃 Find defects. This is the classic objective of testing. A test is run in order to trigger
failures that expose defects. Generally, we look for defects in all interesting parts of the
product.
􀂃 Maximize bug count. The distinction between this and “find defects” is that total number
of bugs is more important than coverage. We might focus narrowly, on only a few highrisk
features, if this is the way to find the most bugs in the time available.
􀂃 Block premature product releases. This tester stops premature shipment by finding bugs
so serious that no one would ship the product until they are fixed. For every releasedecision
meeting, the tester’s goal is to have new showstopper bugs.
􀂃 Help managers make ship / no-ship decisions. Managers are typically concerned with
risk in the field. They want to know about coverage (maybe not the simplistic code
coverage statistics, but some indicators of how much of the product has been addressed
and how much is left), and how important the known problems are. Problems that appear
significant on paper but will not lead to customer dissatisfaction are probably not relevant
to the ship decision.
􀂃 Minimize technical support costs. Working in conjunction with a technical support or
help desk group, the test team identifies the issues that lead to calls for support. These are
often peripherally related to the product under test--for example, getting the product to
work with a specific printer or to import data successfully from a third party database
might prevent more calls than a low-frequency, data-corrupting crash.
􀂃 Assess conformance to specification. Any claim made in the specification is checked.
Program characteristics not addressed in the specification are not (as part of this
objective) checked.
􀂃 Conform to regulations. If a regulation specifies a certain type of coverage (such as, at
least one test for every claim made about the product), the test group creates the
appropriate tests. If the regulation specifies a style for the specifications or other
documentation, the test group probably checks the style. In general, the test group is
focusing on anything covered by regulation and (in the context of this objective) nothing
that is not covered by regulation.
􀂃 Minimize safety-related lawsuit risk. Any error that could lead to an accident or injury is
of primary interest. Errors that lead to loss of time or data or corrupt data, but that don’t
carry a risk of injury or damage to physical things are out of scope.
􀂃 Find safe scenarios for use of the product (find ways to get it to work, in spite of the
bugs). Sometimes, all that you’re looking for is one way to do a task that will consistently
work--one set of instructions that someone else can follow that will reliably deliver the
benefit they are supposed to lead to. In this case, the tester is not looking for bugs. He is
trying out, empirically refining and documenting, a way to do a task.
􀂃 Assess quality. This is a tricky objective because quality is multi-dimensional. The nature
of quality depends on the nature of the product. For example, a computer game that is
rock solid but not entertaining is a lousy game. To assess quality -- to measure and report
back on the level of quality -- you probably need a clear definition of the most important
quality criteria for this product, and then you need a theory that relates test results to the
definition. For example, reliability is not just about the number of bugs in the product. It
is (or is often defined as being) about the number of reliability-related failures that can be
expected in a period of time or a period of use. (Reliability-related? In measuring
reliability, an organization might not care, for example, about misspellings in error
messages.) To make this prediction, you need a mathematically and empirically sound
model that links test results to reliability. Testing involves gathering the data needed by
the model. This might involve extensive work in areas of the product believed to be
stable as well as some work in weaker areas. Imagine a reliability model based on
counting bugs found (perhaps weighted by some type of severity) per N lines of code or
per K hours of testing. Finding the bugs is important. Eliminating duplicates is important.
Troubleshooting to make the bug report easier to understand and more likely to fix is (in
the context of assessment) out of scope.
􀂃 Verify correctness of the product. It is impossible to do this by testing. You can prove
that the product is not correct or you can demonstrate that you didn’t find any errors in a
given period of time using a given testing strategy. However, you can’t test exhaustively,
and the product might fail under conditions that you did not test. The best you can do (if
you have a solid, credible model) is assessment--test-based estimation of the probability
of errors. (See the discussion of reliability, above).
􀂃 Assure quality. Despite the common title, quality assurance, you can’t assure quality by
testing. You can’t assure quality by gathering metrics. You can’t assure quality by setting
standards. Quality assurance involves building a high quality product and for that, you
need skilled people throughout development who have time and motivation and an
appropriate balance of direction and creative freedom. This is out of scope for a test
organization. It is within scope for the project manager and associated executives. The
test organization can certainly help in this process by performing a wide range of
technical investigations, but those investigations are not quality assurance.
Given a testing objective, the good test series provides information directly relevant to that
objective.
Tests Intended to Expose Defects
Let’s narrow our focus to the test group that has two primary objectives:
􀂃 Find bugs that the rest of the development group will consider relevant (worth reporting)
and Get these bugs fixed.
Even within these objectives, tests can be good in many different ways. For example, we might
say that one test is better than another if it is:
􀂃 More powerful. I define power in the usual statistical sense as more likely to expose a
bug if it the bug is there. Note that Test 1 can be more powerful than Test 2 for one type
of bug and less powerful than Test 2 for a different type of bug.
􀂃 More likely to yield significant (more motivating, more persuasive) results. A problem
is significant if a stakeholder with influence would protest if the problem is not fixed. (A
stakeholder is a person who is affected by the product. A stakeholder with influence is
someone whose preference or opinion might result in change to the product.)
􀂃 More credible. A credible test is more likely to be taken as a realistic (or reasonable) set
of operations by the programmer or another stakeholder with influence. “Corner case” is
an example of a phrase used by programmers to say that a test or bug is non-credible:
“No one would do that.” A test case is credible if some (or all) stakeholders agree that it
is realistic.
􀂃 Representative of events more likely to be encountered by the customer. A population of
tests can be designed to be highly credible. Set up your population to reflect actual usage
probabilities. The more frequent clusters of activities are more likely to be covered or
covered more thoroughly. (I say cluster of activities to suggest that many features are
used together and so we might track which combinations of features are used and in what
order, and reflect this more specific information in our analysis.) For more details, read
Musa's (1998) work on software reliability engineering.
􀂃 Easier to evaluate. The question is, did the program pass or fail the test? Ease of
Evaluation. The tester should be able to determine, quickly and easily, whether the
program passed or failed the test. It is not enough that it is possible to tell whether the
program passed or failed. The harder evaluation is, or the longer it takes, the more likely
it is that failures will slip through unnoticed. Faced with time-consuming evaluation, the
tester will take shortcuts and find ways to less expensively guess whether the program is
OK or not. These shortcuts will typically be imperfectly accurate (that is, they may miss
obvious bugs or they may flag correct code as erroneous.)
􀂃 More useful for troubleshooting. For example, high volume automated tests will often
crash the system under test without providing much information about the relevant test
conditions needed to reproduce the problem. They are not useful for troubleshooting.
Tests that are harder to repeat are less useful for troubleshooting. Tests that are harder to
perform are less likely to be performed correctly the next time, when you are
troubleshooting a failure that was exposed by this test.
􀂃 More informative. A test provides value to the extent that we learn from it. In most cases,
you learn more from the test that the program passes than the one the program fails, but
the informative test will teach you something (reduce your uncertainty) whether the
program passes it or fails.
o For example, if we have already run a test in several builds, and the program
reliably passed it each time, we will expect the program to pass this test again.
Another "pass" result from the reused test doesn't contribute anything to our
mental model of the program.
o The notion of equivalence classes provides another example of information value.
Behind any test is a set of tests that are sufficiently similar to it that we think of
the other tests as essentially redundant with this one. In traditional jargon, this is
the "equivalence class" or the "equivalence partition." If the tests are sufficiently
similar, there is little added information to be obtained by running the second one
after running the first.
o This criterion is closely related to Karl Popper’s theory of value of experiments
(See Popper 1992). Good experiments involve risky predictions. The theory
predicts something that many people would expect not to be true. Either your
favorite theory is false or lots of people are surprised. Popper’s analysis of what
makes for good experiments (good tests) is a core belief in a mainstream approach
to the philosophy of science.
Perhaps the essential consideration here is that the expected value of what you
will learn from this test has to be balanced against the opportunity cost of
designing and running the test. The time you spend on this test is time you don't
have available for some other test or other activity.
􀂃 Appropriately complex. A complex test involves many features, or variables, or other
attributes of the software under test. Complexity is less desirable when the program has
changed in many ways, or when you’re testing many new features at once. If the program
has many bugs, a complex test might fail so quickly that you don’t get to run much of it.
Test groups that rely primarily on complex tests complain of blocking bugs. A blocking
bug causes many tests to fail, preventing the test group from learning the other things
about the program that these tests are supposed to expose. Therefore, early in testing,
simple tests are desirable. As the program gets more stable, or (as in eXtreme
Programming or any evolutionary development lifecycle) as more stable features are
incorporated into the program, greater complexity becomes more desirable.
􀂃 More likely to help the tester or the programmer develop insight into some aspect of the
product, the customer, or the environment. Sometimes, we test to understand the
product, to learn how it works or where its risks might be. Later, we might design tests to
expose faults, but especially early in testing we are interested in learning what it is and
how to test it. Many tests like this are never reused. However, in a test-first design
environment, code changes are often made experimentally, with the expectation that the
(typically, unit) test suite will alert the programmer to side effects. In such an
environment, a test might be designed to flag a performance change, a difference in
rounding error, or some other change that is not a defect. An unexpected change in
program behavior might alert the programmer that her model of the code or of the impact
of her code change is incomplete or wrong, leading her to additional testing and
troubleshooting. (Thanks to Ward Cunningham and Brian Marick for suggesting this
example.)
Test Styles/Types and Test Qualities
Within the field of black box testing, Kaner & Bach (see our course notes, Bach 2003b and
Kaner, 2002, posted at www.testingeducation.org, and see Kaner, Bach & Pettichord, 2002) have described eleven dominant styles of black box testing:
􀂃 Function testing
􀂃 Domain testing
􀂃 Specification-based testing
􀂃 Risk-based testing
􀂃 Stress testing
􀂃 Regression testing
􀂃 User testing
􀂃 Scenario testing
􀂃 State-model based testing
􀂃 High volume automated testing
Exploratory testing
Bach and I call these "paradigms" of testing because we have seen time and again that one or two of them dominate the thinking of a testing group or a talented tester. An analysis we find
intriguing goes like this:
If I was a "scenario tester" (a person who defines testing primarily in terms of application
of scenario tests), how would I actually test the program? What makes one scenario test
better than another? Why types of problems would I tend to miss, what would be difficult
for me to find or interpret, and what would be particularly easy?
Here are thumbnail sketches of the styles, with some thoughts on how test cases are “good”
within them.
Function Testing
Test each function / feature / variable in isolation.
Most test groups start with fairly simple function testing but then switch to a different style, often
involving the interaction of several functions, once the program passes the mainstream function
tests.
Within this approach, a good test focuses on a single function and tests it with middle-of-theroad
values. We don’t expect the program to fail a test like this, but it will if the algorithm is
fundamentally wrong, the build is broken, or a change to some other part of the program has
fowled this code.
These tests are highly credible and easy to evaluate but not particularly powerful.
Some test groups spend most of their effort on function tests. For them, testing is complete when
every item has been thoroughly tested on its own. In my experience, the tougher function tests
look like domain tests and have their strengths.
Domain Testing
The essence of this type of testing is sampling. We reduce a massive set of possible tests to a
small group by dividing (partitioning) the set into subsets (subdomains) and picking one or two
representatives from each subset.
In domain testing, we focus on variables, initially one variable at time. To test a given variable,
the set includes all the values (including invalid values) that you can imagine being assigned to
the variable. Partition the set into subdomains and test at least one representative from each
subdomain. Typically, you test with a "best representative", that is, with a value that is at least as
likely to expose an error as any other member of the class. If the variable can be mapped to the
number line, the best representatives are typically boundary values.
Most discussions of domain testing are about input variables whose values can be mapped to the
number line. The best representatives of partitions in these cases are typically boundary cases.
A good set of domain tests for a numeric variable hits every boundary value, including the
minimum, the maximum, a value barely below the minimum, and a value barely above the
maximum.
􀂃 Whittaker (2003) provides an extensive discussion of the many different types of
variables we can analyze in software, including input variables, output variables, results
of intermediate calculations, values stored in a file system, and data sent to devices or
other programs.
􀂃 Kaner, Falk & Nguyen (1993) provided a detailed analysis of testing with a variable
(printer type, in configuration testing) that can’t be mapped to a number line.
These tests are higher power than tests that don’t use “best representatives” or that skip some of the subdomains (e.g. people often skip cases that are expected to lead to error messages).
The first time these tests are run, or after significant relevant changes, these tests carry a lot of
information value because boundary / extreme-value errors are common.
Bugs found with these tests are sometimes dismissed, especially when you test extreme values of several variables at the same time. (These tests are called corner cases.) They are not necessarily credible, they don’t necessarily represent what customers will do, and thus they are not necessarily very motivating to stakeholders.
Specification-Based Testing
Check the program against every claim made in a reference document, such as a design
specification, a requirements list, a user interface description, a published model, or a user
manual.
These tests are highly significant (motivating) in companies that take their specifications
seriously. For example, if the specification is part of a contract, conformance to the spec is very
important. Similarly products must conform to their advertisements, and life-critical products
must conform to any safety-related specification.
Specification-driven tests are often weak, not particularly powerful representatives of the class of
tests that could test a given specification item.
Some groups that do specification-based testing focus narrowly on what is written in the
document. To them, a good set of tests includes an unambiguous and relevant test for each claim
made in the spec.
Other groups look further, for problems in the specification. They find that the most informative
tests in a well-specified product are often the ones that explore ambiguities in the spec or
examine aspects of the product that were not well-specified.
Risk-Based Testing
Imagine a way the program could fail and then design one or more tests to check whether the
program will actually fail that in way.
A “complete” set of risk-based tests would be based on an exhaustive risk list, a list of every way
the program could fail.
A good risk-based test is a powerful representative of the class of tests that address a given risk.
To the extent that the tests tie back to significant failures in the field or well known failures in a
competitor’s product, a risk-based failure will be highly credible and highly motivating.
However, many risk-based tests are dismissed as academic (unlikely to occur in real use). Being
able to tie the “risk” (potential failure) you test for to a real failure in the field is very valuable,
and makes tests more credible.
Good Test Cases Copyright © Cem Kaner 2003. All rights reserved. Page 9
Risk-based tests tend to carry high information value because you are testing for a problem that
you have some reason to believe might actually exist in the product. We learn a lot whether the
program passes the test or fails it.
Stress Testing
There are a few different definition of stress tests.
􀂃 Under one common definition, you hit the program with a peak burst of activity and see it
fail.
􀂃 IEEE Standard 610.12-1990 defines it as "Testing conducted to evaluate a system or
component at or beyond the limits of its specified requirements with the goal of causing
the system to fail."
􀂃 A third approach involves driving the program to failure in order to watch how the
program fails. For example, if the test involves excessive input, you don’t just test near
the specified limits. You keep increasing the size or rate of input until either the program
finally fails or you become convinced that further increases won’t yield a failure. The fact
that the program eventually fails might not be particularly surprising or motivating. The
interesting thinking happens when you see the failure and ask what vulnerabilities have
been exposed and which of them might be triggered under less extreme circumstances.
Jorgensen (2003) provides a fascinating example of this style of work.
I work from this third definition.
These tests have high power.
Some people dismiss stress test results as not representative of customer use, and therefore not
credible and not motivating. Another problem with stress testing is that a failure may not be
useful unless the test provides good troubleshooting information, or the lead tester is extremely
familiar with the application.
A good stress test pushes the limit you want to push, and includes enough diagnostic support to
make it reasonably easy for you to investigate a failure once you see it.
Some testers, such as Alberto Savoia (2000), use stress-like tests to expose failures that are hard
to see if the system is not running several tasks concurrently. These failures often show up well
within the theoretical limits of the system and so they are more credible and more motivating.
They are not necessarily easy to troubleshoot.
Regression Testing
Design, develop and save tests with the intent of regularly reusing them, Repeat the tests after
making changes to the program.
This is a good point (consideration of regression testing) to note that this is not an orthogonal list
of test types. You can put domain tests or specification-based tests or any other kinds of tests into
your set of regression tests.
So what’s the difference between these and the others? I’ll answer this by example:
Suppose a tester creates a suite of domain tests and saves them for reuse. Is this domain testing or
regression testing?
􀂃 I think of it as primarily domain testing if the tester is primarily thinking about
partitioning variables and finding good representatives when she creates the tests.
􀂃 I think of it as primarily regression testing if the tester is primarily thinking about
building a set of reusable tests.
Regression tests may have been powerful, credible, and so on, when they were first designed.
However, after a test has been run and passed many times, it’s not likely that the program will
fail it the next time, unless there have been major changes or changes in part of the code directly
involved with this test. Thus, most of the time, regression tests carry little information value.
A good regression test is designed for reuse. It is adequately documented and maintainable. (For
suggestions that improve maintainability of GUI-level tests, see Graham & Fewster, 1999;
Kaner, 1998; Pettichord, 2002, and the papers at www.pettichord.com in general). A good
regression test is designed to be likely to fail if changes induce errors in the function(s) or area(s)
of the program addressed by the regression test.
User Testing
User testing is done by users. Not by testers pretending to be users. Not by secretaries or
executives pretending to be testers pretending to be users. By users. People who will make use of the finished product. User tests might be designed by the users or by testers or by other people (sometimes even by lawyers, who included them as acceptance tests in a contract for custom software).
The set of user tests might include boundary tests, stress tests, or any other type of test.
Some user tests are designed in such detail that the user merely executes them and reports
whether the program passed or failed them. This is a good way to design tests if your goal is to
provide a carefully scripted demonstration of the system, without much opportunity for wrong
things to show up as wrong.
If your goal is to discover what problems a user will encounter in real use of the system, your
task is much more difficult. Beta tests are often described as cheap, effective user tests but in
practice they can be quite expensive to administer and they may not yield much information. For some suggestions on beta tests, see Kaner, Falk & Nguyen (1993).
A good user test must allow enough room for cognitive activity by the user while providing
enough structure for the user to report the results effectively (in a way that helps readers
understand and troubleshoot the problem).
Failures found in user testing are typically credible and motivating. Few users run particularly
powerful tests. However, some users run complex scenarios that put the program through its
paces.
Scenario Testing
A scenario is a story that describes a hypothetical situation. In testing, you check how the
program copes with this hypothetical situation.
The ideal scenario test is credible, motivating, easy to evaluate, and complex.
In practice, many scenarios will be weak in at least one of these attributes, but people will still
call them scenarios. The key message of this pattern is that you should keep these four attributes in mind when you design a scenario test and try hard to achieve them.
An important variation of the scenario test involves a harsher test. The story will often involve a
sequence, or data values, that would rarely be used by typical users. They might arise, however,
out of user error or in the course of an unusual but plausible situation, or in the behavior of a
hostile user. Hans Buwalda (2000a, 2000b) calls these "killer soaps" to distinguish them from
normal scenarios, which he calls "soap operas." Such scenarios are common in security testing or
other forms of stress testing.
In the Rational Unified Process, scenarios come from use cases. (Jacobson, Booch, &
Rumbaugh, 1999). Scenarios specify actors, roles, business processes, the goal(s) of the actor(s),
and events that can occur in the course of attempting to achieve the goal. A scenario is an
instantiation of a use case. A simple scenario traces through a single use case, specifying the data
values and thus the path taken through the case. A more complex use case involves
concatenation of several use cases, to track through a given task, end to end. (See also Bittner &
However they are derived, good scenario tests have high power the first time they’re run.
Groups vary in how often they run a given scenario test.
􀂃 Some groups create a pool of scenario tests as regression tests.
􀂃 Others (like me) run a scenario once or a small number of times and then design another
scenario rather than sticking with the ones they’ve used before.
Testers often develop scenarios to develop insight into the product. This is especially true early
in testing and again late in testing (when the product has stabilized and the tester is trying to
understand advanced uses of the product.)
In general, comparisons of software behavior to the model are done using automated tests and so
the failures that are found are found easily (easy to evaluate).
In general, state-model-based tests are credible, motivating and easy to troubleshoot. However,
state-based testing often involves simplifications, looking at transitions between operational
modes rather than states, because there are too many states (El-Far 1995). Some abstractions to operational modes are obvious and credible, but others can seem overbroad or otherwise odd to some stakeholders, thereby reducing the value of the tests. Additionally, if the model is
oversimplified, failures exposed by the model can be difficult to troubleshoot (Houghtaling,
2001).
Talking about his experiences in creating state models of software, Harry Robinson (2001)
reported that much of the bug-finding happens while doing the modeling, well before the
automated tests are coded. Elisabeth Hendrickson (2002) trains testers to work with state models
as an exploratory testing tool--her models might never result in automated tests, their value is
that they guide the analysis by the tester.
El-Far, Thompson & Mottay (2001) and El-Far (2001) discuss some of the considerations in
building a good suite of model-based tests. There are important tradeoffs, involving, for
example, the level of detail (more detailed models find more bugs but can be much harder to read
and maintain). For much more, see the papers at www.model-based-testing.org.
High-Volume Automated Testing
High-volume automated testing involves massive numbers of tests, comparing the results against one or more partial oracles.
􀂃 The simplest partial oracle is running versus crashing. If the program crashes, there must
be a bug. See Nyman (1998, 2002) for details and experience reports.
􀂃 State-model-based testing can be high volume if the stopping rule is based on the results
of the tests rather than on a coverage criterion. For the general notion of stochastic statebased
testing, see Whittaker (1997). For discussion of state-model-based testing ended
by a coverage stopping rule, see Al-Ghafees & Whittaker (2002).
􀂃 Jorgensen (2002) provides another example of high-volume testing. He starts with a file
that is valid for the application under test. Then he corrupts it in many ways, in many
places, feeding the corrupted files to the application. The application rejects most of the
bad files and crashes on some. Sometimes, some applications lose control when handling
these files. Buffer overruns or other failures allow the tester to take over the application
or the machine running the application. Any program that will read any type of data
stream can be subject to this type of attack if the tester can modify the data stream before
it reaches the program.
􀂃 Kaner (2000) describes several other examples of high-volume automated testing
approaches. One classic approach repeatedly feeds random data to the application under
test and to another application that serves as a reference for comparison, an oracle.
Another approach runs an arbitrarily long random sequence of regression tests, tests that
the program has shown it can pass one by one. Memory leaks, stack corruption, wild
pointers or other garbage that cumulates over time finally causes failures in these long
sequences. Yet another approach attacks the program with long sequences of activity and
uses probes (tests built into the program that log warning or failure messages in response
to unexpected conditions) to expose problems.
High-volume testing is a diverse grouping. The essence of it is that the structure of this type of
testing is designed by a person, but the individual test cases are developed, executed, and
interpreted by the computer, which flags suspected failures for human review. The almostcomplete
automation is what makes it possible to run so many tests.
􀂃 The individual tests are often weak. They make up for low power with massive numbers.
􀂃 Because the tests are not handcrafted, some tests that expose failures may not be
particularly credible or motivating. A skilled tester often works with a failure to imagine
a broader or more significant range of circumstances under which the failure might arise,
and then craft a test to prove it.
􀂃 Some high-volume test approaches yield failures that are very hard to troubleshoot. It is
easy to see that the failure occurred in a given test, but one of the necessary conditions
that led to the failure might have been set up thousands of tests before the one that
actually failed. Building troubleshooting support into these tests is a design challenge that
some test groups have tackled more effectively than others.
Exploratory Testing
Exploratory testing is “any testing to the extent that the tester actively controls the design of the
tests as those tests are performed and uses information gained while testing to design new and
better tests” (Bach 2003a).
Bach points out that tests span a continuum between purely scripted (the tester does precisely
what the script specifies and nothing else) to purely exploratory (none of the tester’s activities
are pre-specified and the tester is not required to generate any test documentation beyond bug
reports). Any given testing effort falls somewhere on this continuum. Even predominantly prescripted