naveen automationlabs

Where we learn technology

Day: 21 November 2017

How To Figure Out What To Test

21 November 2017 / / 0 Comments
How To Figure Out What To Test





The Problem

There is a lot to figure out when it comes to automated web testing, but where do you start? And if you’ve already started, how do you know you’re on the right track? And how do you avoid testing everything in every browser without missing important issues?

A Solution

A great way to increase your chances of automated web testing success is to map out a testing strategy. And the best way to do it is to answer these four questions:
  1. How does your business make money?
  2. How do your users use your application?
  3. What browsers are your users using?
  4. What things have broken in the application before?
NOTE: for organizations that don’t deal directly in dollars and cents (e.g. non-profits, federal agencies, etc.) you should focus on how the application generates value for the end user.
After answering these questions you’ll have a firm understanding of the critical functionality and relevant browsers for the application you’re testing. This will help you focus your test automation efforts on the things that matter most. 
You’ll want to keep track of these items somehow — the recommended approach is to use a prioritized backlog.

What To Do With The Answers

Question 1 – Money/Value

Every company’s application makes money (or generates value) through a core set of functionality — a.k.a. a ‘funnel’. Your answers to this question will help you determine what functionality makes up the funnel. This will be your highest priority for test automation.

Question 2 – Usage Data

There can be a big difference between how you think your application is used, and how yours users actually use it. And odds are your application offers a robust set of functionality that grows well beyond the core functionality of the funnel.
Your answers to this question will help you determine what features are highly used and lightly used. Tack the highly used items onto your automation backlog based on order of use below the answers to question 1.

Question 3 – Browsers

Now that you know what functionality is business critical and widely adopted by your users, you need to determine what browsers to focus your automated web testing efforts on. Your usage data will tell you this as well. It will help you determine which browsers you can reasonably avoid testing in (e.g. based on non-existent or low usage numbers). 
Note the top 2 browsers (or 3 depending on your numbers), but focus on the top 1 for now. This is the browser you will start writing your automated tests in.

Question 4 – Risky Bits

To round out the strategy it is best to think about what things have broken in the application before. You might be able to glean some of this information from a defect tracker. But the best information is often in the minds of your colleagues. Ask around and see what they say.
What you come up with will likely read like a laundry list of browser specific issues or functionality that has been flaky or forgotten about in the past. Be sure to check this list against your automation backlog. If you’ve come across something that’s not already in the backlog, add it and put it at the bottom. If it is there, make a note in the backlog item that this has been an issue in the past. And if the issue has happened numerous times and has the potential to occur again, move the item up in the backlog priority.
If issues keep cropping up that are related to a specific browser, compare this browser to your short list from question #3. If it’s a browser that’s not in your list but it’s a small pocket of high value users, track it on the backlog but put it at the bottom.

Now Your Are Ready

Having answered these questions you should now have a prioritized backlog of critical functionality that’s backed up by usage data, a short list of browsers to focus on, and an understanding of the risky parts of your application. With it you can make sure you are on the right track with your test automation (regardless if you’re just getting started with test automation or already have a mature test automation practice).

Outro

Hopefully this strategy will help you focus your testing efforts, avoid wasting time, and increase your confidence in the approach you are taking.
This strategy works best for applications with existing functionality and does not speak directly to testing new functionality that’s being actively developed. That’s not to say that the two couldn’t co-exist. It largely depends on your available resources and pace of development. But in order to reach high quality at speed, you first have to go slow.
Cheers,
Naveen Khunteta

Where To Look for Selenium Information

21 November 2017 / / 17 Comments

Where To Look for Selenium Information

The Problem

There is information all around us when it comes to Selenium. But it it can be challenging to sift through it, or know where to look.

A Solution

Here is a list breaking down a majority of the Selenium resources available, and what they’re useful for.

Available Resources

Documentation & Tips

This is the official Selenium project documentation site. It’s a bit dated, but there is loads of helpful information here. You just have to get the hang of how the navigate site to find what you need.
This is where all the good stuff is — mainly, documentation about the various language bindings and browser drivers. If you’re not already familiar with it, take a look. It’s not perfect, but it’s better than the Selenium HQ site for a lot of topics.
Every tip I’ve written is freely available on my tips archive page. There are 70 different Selenium problems and solutions covered.

Blogs

This is where news of the Selenium project gets announced. There’s also the occasional round-up of what’s going on in the tech space (as it relates to testing). Definitely worth a look.
At some point, someone rounded up a large list of blogs from Selenium practitioners and committers. It’s a pretty good list.

Books

This book is by David Burns and it is a thorough resource. It outlines how to use Selenium, top-to-bottom, using Java. It’s a good resource to keep handy.
This a good book that covers more advanced ways to leverage Selenium. While I haven’t had a chance to finish reading it, it’s clear that Gundecha has a very pragmatic approach that will yield great results for any automated testing practice that uses Selenium.
This is the book I wish existed when I first started out. It teaches you the basics of programming needed for automated testing, how to decompose an existing web app, how to write tests so they’re reliable and maintainable, how to package things up into a test harness, and how to automate and scale your tests. I think it’s pretty good, but then again, I wrote it. It’s covers both Java and Ruby.

Meetups

This is an entirely online meetup that is co-run by myself and David Burnswhere we talk about the latest in the Selenium community (e.g., Selenium Conf, the status of Selenium 3, the WebDriver W3C spec, etc.) and answer questions from the community. Videos are recorded and made available (along with notes) on the official Selenium blog.
A listing of all in-person Selenium Meetups are available on Meetup.com. If you’re near a major city, odds are there’s one waiting for you.
If there’s not a Selenium Meetup near you, start one! Sauce Labs has a great write up on how to do it.

Conferences

This is the annual conference where practitioners and Selenium Core Committers gather and share knowledge. All of the talks are top notch, as is the hallway track. If I were to attend just one conference a year, it would be this one.
This is a self-organized Selenium conference that’s held annually in Kiev, Ukraine. It’s a well organized event with some great talks. It’s the most advanced test automation conference I’ve attended. Definitely worth the trip.

Videos

Every year talks from The Selenium Conference are recorded and made freely available online. This is one of the best resources around.
Some Selenium Meetups take it upon themselves to record their talks and publish them afterwards. Here are some of them. They are a great way to see what other pockets of the community are up to (outside of the annual Selenium Conf).
All of the Selenium Hangout Meetups are recorded and made available. A lot of great stuff is discussed in them.

Mailing Lists

The signal to noise ratio in these groups can be challenging at times. But you can occasionally find some insights and answers to your questions.
This list is less about Selenium and more about “testing” (in the checking versus testing sense). It’s a great place to ask for sage advice, and to hang out and listen.

Forums

These are the usual forums where you can go looking for answers to questions you’re facing (in addition to the mailing lists above).

Issues

If you’re running into a specific and repeatable issue that just doesn’t make sense, you may have found a bug in Selenium. You’ll want to check the Selenium Issue Tracker to see if it has already been reported. If not, then create a new issue. But be sure to read this post before you do (so you can be sure that you’ve provided enough information for the Core Committers to work on it).

IRC

The Selenium IRC Chat Channel is arguably the best way to connect with the Selenium community and get questions answered. This is where committers and practitioners hang out day-in and day-out. You can read more about what it is and how to access it here.
The is the single best resource available.

Outro

As you can see there’s a lot out there. But hopefully this list has helped guide you to the best resources for your context.
Happy Testing!

© 2024 naveen automationlabs

Theme by Anders Norén — Up ↑