Most of the product companies still have test case creation in place where tests are documented with detailed steps and expected results. Well, too much of documentation deviates the creative exploration of tester, documented test cases are still recommended a few reasons –

  1. Repeatability of the test cases
  2. Documented evidence of what has been tested and what has been not

Enterprise Products, while adopting latest technologies, are still leaning towards documenting test cases. ISV’s and edge-of-the technology organizations are preferring less documentation implementing Agile. Few organizations do not have test cases in place at all, where user stories are tested and updated on the management tool.

In traditional scripted approach, we take a user story, document different ways of achieving the story (build flows), create detailed steps with expected results, which needs the tester to document each and every test that we do on the story.

How do we incorporate exploratory testing into above format? Session based test management (SBTM) is efficient in terms of exploratory testing, with help of a tracking tool like Rapid reporter or Session tester. These tools create a log of actions (minute tests) performed during testing. If we can create a hybrid test case format, it may look something like below –

2016-09-14_125950

(Zoom in to view picture)

Repeatability will still remain a challenge, but not to the extent of no documented test cases. In the world of agile, we always automate the repeatable. One other challenge will be to trace the session report that failed when a bug is found in production.

I expect test management tools to move towards incorporating exploratory testing, rather than using traditional scripted test cases. I see Cervaya, by Paul gerrard is already focussing on this.

Advertisements

One of the key characteristics of a great tester is to know how to simplify tasks. A great tester will use right tools at the right time to simplify tasks. Tasks like taking screenshots & videos, test case creation, collaboration, communication, Report creation, test execution can be simplified.

Below are few tools that simplify tasks –

Screen capture & Video recording– Snipping tool, Faststone screen capture, jing, Camatasia, Chrome web page screenshot. Fast stone and Chrome web page screenshot (add-in) provides the ability to capture scrollable page.

Portableapps.com– Allows you to carry any (can be limited for now) app in a pen drive and launch without installation.

Fileinfo.com – provides information on file with any extension. It also provides information like software which can be used to create a particular type of file.

Colorcop (http://colorcop.net/download/) – this gives the ability to specifically mention the RGB values of a color, makes it more specific when  you report.

Supportdetails.com – gives basic environment details of the machine, useful in bug reporting.

Mindmaps like Xmind, FreeMind –brainstorming tools for test idea generation.

Test management – Testrail, Versionone, Microsoft Test Manager, HP Quality Center

Excel – Excel can be used for test management, bug reporting, test result reporting in the absence of specific tools. Excel is a wonderful tool.

Dropbox, Google drive, online mind mapping (MindMeister) – for collaboration

Slack, Skype, Lync or any IM – for communication

Tools like Selenium, UFT, VSTS coded UI, TestComplete, Sahi, Fiddler, Jmeter, SoapUI, Webinspect, Appscan,  – to simplify test execution

Sometimes, your tool expertise can be your USP. Find right tools that suit your needs at different areas to simplify your tasks. Do add to the list if you know more tools that can make tester’s life easy.

 

This is one of the frequently asked questions by stakeholders to testers. Below is a high-level checklist to ponder to answer this question –

  • Do you know in & out of the product functionality? Do you have the same understanding of the product as the product owner?
  • Do you know the users who use the software under test?
  • Do you know what are ‘important’ bugs?
  • Do you know all existing problems with the system?
  • Do you know if the test machine is same as in configuration & data as production system? Do you have control of changing this?
  • Do you have frequency & complete information on changes being pushed to the test system?
  • Do you know what your deadlines are? Do you have control of having more time for testing?

While the list extends based on the context and current situation of the software, above is the list of basic questions to look at.

Testing is beyond checking!

Posted: September 5, 2014 in Uncategorized

Testing is beyond checking!

Target audience: Testers

Hello everyone!

I always think checking a software is just not enough. Going strictly by requirements, creating tests around requirements and executing them – I think, does not add enough value to clients. Testing is something beyond checking. I’ll try to illustrate this with a real time example.

Let me set the context first

Communication plays a key role in outsourced business and many of us do communicate with clients via Emails, IMs & Web conferences. I use GoToMeeting software (company subscribed edition) for web conferencing to interact with clients. One of my usual practises is to record all meetings I attend, so that, I can refer them to

  • Review and understand the summary of discussion as needed
  • Get clarifications on any conflict in the understanding with other attendees
  • Prepare meeting minutes, and work on action items
  • Prepare myself before attending a follow up meeting

I’ve been using GoToMeeting (GTM) for quite some time now (for couple of years now), and I have GoToMeeting desktop agent installed on my computer. I usually login from the system tray icon, access the meetings and use GoToMeeting.

1 

I’ve got a new laptop recently, 4-5 months back, and it is amazing. It has got touch screen, Windows 8 OS, solid state drive –which boots in lightning speed (in ~10 seconds). I’ve installed GoToMeeting on my new computer with default settings and using it.

The problem

I was doing my meetings nicely, but, I was unable to play recorded files using Windows media player. The player refused to play the file indicating that the required video codec is not installed on my computer.

2

I have verified that the file is in WMV format and the default player is set to Windows media player (WMP). I have no clue on why this is happening and wanted to find more information on this. I clicked on the ‘Web Help’ in the error message, below is what I found –

3

Interesting! The website doesn’t have any specific information about the error!

I have verified the codecs installed on my computer with help of ‘frequently asked questions’ section on the site and found the codec for WMV files installed. Phew!

Okay, WMP is not worried about playing GoToMeeting files, but, would GoToMeeting be interested in making their files play with WMP??

Browsed GoToMeeting website to see if I can get some information. The site refers to a setting in preferences, where you can choose if the recorded file can be in –

  • GoToMeeting format
  • Windows media player format

 4

 The setting is currently set to GTM format and I’ve changed this preference to have all my meeting recordings in WMP format. Phew! The recorded file now plays in WMP.

Let me do some analysis now

I’ve tried to find any differences in the recorded files (GTM format & WMP format) – both are with extension .wmv & no differences observed in ‘Properties’ of the files. May have to explore more on how codecs make a difference!

I’ve uninstalled the software, re-installed and found that the preferences are set to GoToMeeting recording format by default. Is there a problem here? What would have been the reason this preference is set to GoToMeeting format by default? I could think of two situations –

  • GoToMeeting wants to promote their media player rather than Windows Media player – Possible! Considering the fact that Windows media player is the leading media player on Windows computers (at least when compared to GoToMeeting player), GoToMeeting should have handled this differently. One possible solution is to display an alert message when user attempts to record a meeting which can state ‘Meeting will be recorded in GTM format, Please change your preferences to record in Windows media player format’.
  • GoToMeeting didn’t really care about this – Testers could have brought this up to GoToMeeting attention and see if they really want to do this. If not, set the default option to Windows media player format.

Few points to note

You might refer to the requirements document which suggests the default option to be set to ‘GTM format’. Do you think that is right? I see a need for questioning the requirements. If the goal of testers is just to check if the user could change the format – yes, the check has passed. Do you still have a problem? Yes indeed! Testing is something beyond checking.

Testing is not just verifying if we have an option available, but, is about asking questions – Is there a problem here? Is this adding business value? What would add better business value than what we currently have?

This needs better understanding of context. Testing is not just knowing requirements, but also understand the current situation of the product, user base that uses the product and expectations from the product owner.

Testing is not just reporting bugs, but is also about providing any information that will add value to the product.

Happy Testing!!

Versions of software I use: GoToMeeting – Version 6.3.1 Build 1468, Windows media player – Version 12.0.9600.17031

Some information on Xpaths

Xpath

Image  —  Posted: December 29, 2013 in Uncategorized

Many a time, I come across a situation where I have to test the application without any documentation from the client. You can of course question and get the information. However, if the client/product owner is not next to you, it is difficult to get information on the product instantly. Where do I get more details about the product?? Can I get started without documentation??

Below are few sources I start searching to know more details of the product –

Website  

Client website speaks a lot about their product. Website is a good source to start with to understand what the application is.

 FAQs

FAQs are built to answer questions asked by majority of the users, like what is this application, how I start with, how many types of subscriptions etc. I personally find a lot of value in reading FAQs on an application. It gives me the main intent on which the application is built for and what the application builder thinks the user may think of.

 Youtube

You always find someone from the product department or Sales talk about your product on youtube saying – my product will do that, my product will do this, blah blah blah.. These are good source of information. Of course, can be useful to do some claims testing.

 Customer service portals

Customers talking about the AUT is a rich source of information for testers. You actually know what the end user feels like a problem with the AUT. Each ticket logged on Customer service portal is a new idea for testing. Each ticket accepted by the Product owner on customer service portal is a source for testers to know the modules that would undergo changes – rich source isn’t it?

 Help document

A detailed & Fantastic source of information. No explanation needed.

 Press releases

It helps me a bit to understand the context in which the product is. Not always helpful though. Sometimes helpful in learning about pre-promised deadlines for the product release.

 I am sure there will be more sources you can find information about the product and All sources may not be available for all products you test. Even if you have requirements and Specs, search for different sources to help you understand the product better..!! Happy testing..!!

Date : 18th April, 2011
Time : 4:00 AM
Duration : 1 hour

On a lazy Monday morning, I’m happy that I spent my time on testing. I’ve paired today with Ajay to learn & practise paired testing.

Ajay set us a mission to test the ‘Contact us’ module on the site Flipkart.

Ajay mentioned that we have to set an environment ready for paired testing to communicate the partner about the mode, mission and objective. We usually have a chat over skype but we’ve created a document in typewith.me for our convenience as Ajay mentioned an important point to note on paired testing – “No point in having different channels of communication during paired testing”. This only leads to confusion and end up altering the means of communication instead of testing.

So, “Set up a single environment for communication during paired testing

Next step is to Share test ideas. We have started listing down the test ideas on ‘Contact us’ module of Flipkart site. What can we test in this module and in what all ways?

We’ve shared the test ideas till 4:30AM and all set ready to test now. We’ve made an assumption that we could not test all test ideas shared in the 30 mins we have.

We took test ideas from the list, executed them and listed down Issues & Bugs for 25 mins.
During testing, Ajay found a bug and we made efforts to reproduce the bug and nail it down to the root cause of the bug.

Sharing test ideas is the advantage I see with paired testing – two brains working on a single application in different views to yield more & different test ideas.
I also feel we will be more focussed when paired.

Thanks Ajay for the experience :-). Hope we would pair up frequently!

Happy testing.

-Balaji