Agile Testing

by Stan Taylor

Automating android cordova/phonegap applications with appium

by Stan on 2015/10/16, no comments

I am just putting this out there for others to find, because I had such a difficult time locating this crucial tidbit of information: if you select the Android WebView context in your hybrid mobile application, you can run your DOM-based Selenium tests against it using appium.

I am tasked with creating UI automation for an angularJS-based application that we are deploying as a web application and as a mobile application using cordova/phonegap. I wrote my Selenium-based tests for the web deployment and then wanted to use the same tests for the Android mobile application using appium. When I launched my mobile application in the Android emulator and then used Android’s UIAutomator to view the UI objects, all I saw was the Android-based objects, no DOM-based objects–even when I selected the WebView context. My heart sunk because I thought I would have to write separate automation for the web deployment and the Android app. After quite a bit of Googling, though, I found the nugget of information above (I can’t find the source now). So, I’m able to write my tests against the web deployment using Selenium and then run them against the Android app using appium.

Identifying form elements by the <label> tag contents

by Stan on 2015/10/16, one comment

In a previous post, I explained how I use the text associated with UI objects in my cucumber tests. My steps look something like this:

Given I go to the login page
And I enter "johndoe" in the field with id "username"
And I enter "password1" in the field with id "password"

If the application under test is using the <label> tag to identify form elements (and it should! The <label> tag was designed specifically for this purpose), then your application under test has UI objects that look something like this:

<label for="username">Username</label>
<input name="username" type="text"></input>

Writing a cucumber step to interact with the <input> field based on the text in the <label> tag consists of locating the label element and then using the value of its for= attribute to locate the input element. Using the Selenium bindings, my code looks like this:

  this.Given(/^I enter "([^"]*)" in the "([^"]*)" field$/, function (textToEnter, labelText, callback) {
    xpath = '//label[contains(.,"' + labelText + '")]';
    this.getAttribute(xpath, 'for').then(
      function (value) {
      // get the input tag with the name= {value} and enter the text
        xpath = '//input[@name="' + value + '"]';
        that.setValue(xpath, textToEnter).then(
          function () {
          function (err) {
      function (err) {

Words have meanings!

by Stan on 2015/10/10, no comments

I recently received the following message via LinkedIn:

Dear Stan, We are a young silicon-valley-like startup …
… developing disruptive products for sensing, cognition and communication for the Internet of Things (IoT) market;
… fully funded with an exclusive Fortune 200 customer already secured;
… who is working closely with us to specify the product and take it to market;
… led by a top-notch team of seasoned start-up engineers and executives with successful prior startup exits to multi-national corporations; and
… all right here in Austin.

We are currently looking for a top-notch automation expert and looking at your resume and background I thought you might be a good fit.    I hope you are interested in hearing more and would be glad to discuss this opportunity further via a call or f2f meeting. Thanks, [name redacted]

I was really curious to know what he meant by ‘silicon-valley-like,’ so I answered:

What does ‘silicon-valley-like’ denote? That could mean a lot of different things–both positive and negative.

His response:

Good point re: the silicon-valley reference – esp. being a long-time Austinite (by choice) I can understand why it could be considered negative! I was referring to the fact that we have an exciting mission in a hot industry area that can have a big impact with a top-notch team to work with. And the particular role I’d like to go over has some very interesting challenges – for example capturing, storing, analyzing, labeling and retrieving very large data sets.

And my answer again:

I’m pretty sure that “an exciting mission in a hot industry area that can have a big impact with a top-notch team to work with” isn’t a characteristic unique to the Bay Area. I understand that you probably can’t reveal many details, but the quote above doesn’t tell me anything more than “silicon-valley-like” So far, you’ve basically told me nothing at all about the opportunity.

If this ‘silicon-valley-like’ startup hired this guy to do their recruiting, I can only come to one of two conclusions:

  1. He doesn’t know the business well enough to give meaningful details, or
  2. He doesn’t understand recruiting well enough to get to a candidate’s concerns quickly and answer them.

Based on what I saw on LinkedIn (LI only let me see this guy’s name and title), however, I suspect that he is one of the founders or early employees. If that’s the case, then possible explanations above for his behavior incline me even less to treat his offer seriously. Do I want to work in a company where this person has a leading role? Hell no.

Writing Selenium test steps for cucumber tests

by Stan on 2015/10/03, 2 comments

In my current job, I’m responsible for developing a framework for performing UI testing of web and mobile applications with Selenium WebDriver/Appium, using the bindings for node.js Javascript. We have adopted cucumber as the format for defining the tests.

Conventional wisdom regarding UI testing holds that you should always strive to select UI objects by the attribute that is least likely to change. With web UIs, you’ll typically see a hierarchy of preferred selectors as the following:

  1. Element ID
  2. Element name
  3. CSS class(es) associated with the element
  4. Text associated with the element

If you used this strategy with cucumber, you would get tests that looked something like this:

Given I go to the login page
And I enter "johndoe" in the field with id "username"
And I enter "password1" in the field with id "password"
Then the input element with type "submit" should become enabled
And I click the input element with type "submit"

But such steps are counter to the principles of behavior-driven development. The primary purpose of cucumber is that it allows your business owner to write requirements that can also directly serve as tests, and identifiers such as element ID or name are artifacts of implementation details, not part of the basic requirements. Or more simply–your business owner should be able to describe interaction with the application using only what’s visible to them in the UI.

More typically, cucumber steps for the login scenario above would look something like this:

Given I go to the login page
And I enter "johndoe" in the "Username" field
And I enter "password1" in the "Password" field
The the "Log in" button should become enabled
And I click the "Log in" button

Note that the steps use the visible text associated with each element to select the element.

The argument against this approach is: “But whenever your UI text changes–and let’s face it, it often changes–you’ll have to go update every single test!” While this is true, my rebuttal is that it’s a simple global search and replace change, and more importantly, it’s not a change to the code behind the cucumber, only to the cucumber test definitions, which, in my opinion, is a lower risk change than an actual code change.

To be fair, as the number of tests grew, I saw the redundancy between tests as a maintenance risk, so I’ve abstracted out frequently used functions, such as login and navigation, into single steps. Steps like the examples above no longer exists in our codebase. Instead, we have single steps such as:

Given I log in as username "johndoe" and password "password1"

So, now, if the text on our login page changes, we only have to change it in one place.

In my next blog post, I’ll show some of the code behind these steps.

UPDATE: Here is the next blog post.

Underestimating myself

by Stan on 2015/07/23, no comments

I’m a self-taught programmer; I’ve never done it full-time. Most of the QA jobs that I have had over the years have included some amount of scripting and/or automated testing work, but it’s always been done alongside other QA tasks. When I was job hunting last year, I concluded that if I really wanted to remain competitive in my career, my next job needed to focus solely or primarily on writing test automation. Plus, I love to code, and I thought I would really enjoy such a job.

During my job hunt, my standard line about my coding skills was: I’m a decent coder, but because I’ve never done it full-time, I’m probably not qualified to fill a role where I’m the person primarily responsible for the development of an automation framework. When I interviewed here at Netspend for a senior automation developer job, I was told that they were seeking someone with a strong automation background but that they were less concerned whether the candidate knew the language being used, node.js/Javascript. Since node.js is a fairly new framework, and since Javascript is rarely used in automation, they (accurately, in my opinion) didn’t expect to find a candidate who had node.js/Javascript experience. Furthermore, they assumed that any qualified candidate could pick up the language.

It turned out that those were perfect expectations for me. During my interviews here, the interviewers were impressed with my extensive knowledge and experience with automation, and I passed their general programming tests. In regard to programming skills, they were more interested in my thought processes and logic than producing accurate code in the spur of the moment on a whiteboard. Plus, I did have a distant background with Javascript (not just in the browser; it was also the programming language of an automation tool that I used many years ago).

Now that I’ve been here at Netspend for about 8 months and developed a framework and automated tests for UI-testing, I’ve discovered that I misapprehended the skills that are necessary for building an automated testing framework. When I said I was probably not qualified to build an automation framework, I was thinking primarily of coding skills. I have learned the Javascript syntax and the node-specific knowledge necessary to build an automation framework, and I’ve built a robust framework and set of tests that adds value to our development process.

Last week, I worked with another senior QA engineer here at Netspend to define the desired skill sets of our different types of QA jobs: general QA, QA of our back-end systems, QA for our user-facing applications, and automated test developer (I’ll do another blog post about the results of this work). Here is the list of skills that we feel are necessary for an automated test developer in our organization:

  • Source Control Management (we use git)
    • How and Why
  • Continuous Integration (we use Bamboo)
  • Programming Basics, e.g.,
    • Conditionals
    • Data Types
    • Program flow
  • Advanced Programming, e.g.,
    • Development Patterns / Architecture
    • Abstractions / Refactoring
    • Contributing to Framework development
  • X-Path / DOM manipulation (for UI testing)
  • Automation Test Concepts, e.g.,
    • How to Validate what
    • Types of automation
    • Reducing to minimum set of tests
    • Cost / Benefit analysis
  • Database basics
  • Linux basics

As it turns out (and as the people who hired me here–one of whom was the guy I worked with to create this list–astutely realized), being an expert coder of the language of choice doesn’t even make the list. I’ve been successful with the development of an automated testing framework here due to my knowledge of and experience with all of these areas.

Not a maker

by Stan on 2015/01/25, no comments

I recently met with a friend of a friend to offer some ideas on a career change. This guy has a BA in English and has worked in retail management for 26 years. It has worn him out and he is looking for something new. When he heard that I work in software development, he initially didn’t imagine I had much to offer him since the only job he knew of was coding. During our talk, I outlined to him all the different roles involved in software development and sales–not just programming, but QA, tech writing, product management, business analysis, sales, sales engineering, partner management, etc.–and I briefly outlined the desired skills sets of each of these roles.

I would say that his is a typical view of software development: it’s all about the coding. When I read Debbie Chachra’s essay, Why I Am Not a Maker, it immediately struck me that what she describes is exactly what goes on with high tech in general: “As Kate Losse has noted, coders get high salary, prestige, and stock options. The people who do community management—on which the success of many tech companies is based—get none of those.” Substitute any of the job roles I mentioned above (and many others I overlooked) for ‘community management’ in that sentence and you get what I mean.

Debbie Chachra’s essay focuses on the ‘caregiving’ behind making, and in many ways, I don’t think that aspect of her thinking applies to QA and the other roles in software development. On the other hand, it’s usually the job of some non-coding member(s) of the software development team–QA, scrum master, program manager, etc.–to make sure that the processes are followed: bugs get tracked through correctly through their lifecycle, the team board accurately reflects work in progress, etc. You could certainly consider these to be caregiving roles. To cite Debbie’s example, the team members who get those tasks are the mothers who make sure that the housecleaning takes place–if not doing all the housecleaning themselves. Interesting.

Update: There’s a very long and thought-provoking MeFi discussion about this essay. For instance, “[P]rogrammers (I’m one of them) tend to over-estimate how hard their primary skills are and under-estimate skills others learn for their roles, as well, as you note, the half of the population that is “naturally” expected to be better at them (also in that half.)” On the one hand, it’s true that being a programmer typically requires more specialized training than many of the ‘supporting’ roles in software development. On the other hand, programmers overestimating the difficulty of their primary skills is a product of an environment where that work has been given preference over other work since it was dominated by males. It’s a self-reinforcing circle.

Perks in the high-tech workplace

by Stan on 2014/06/11, no comments

My friend Rafe Colburn and his Etsy coworker Melissa Santos have written an insightful article on perks in high-tech companies in which they explore a lot of issues that I’ve thought about over the years. Their thesis:

In an industry where culture is often allowed to be defined by perks, managers need to [be] mindful of the fact that for many people, perks underscore the differences between members of the team rather than bringing them together. They also need to think about what a company’s perks indicate to potential employees about the culture.

I’m one of those employees who often feels ambivalent about company perks–especially team-building events, and within that category, especially events that take place outside of work hours. Balancing my responsibilities to my family with my work responsibilities is hard enough when work is confined to defined hours and expectations. An after-hours event often just adds to that delicate balancing act.

Placeholder text goes here

by Stan on 2014/02/24, no comments

content-loremEarly on in my career as a software tester, I learned–the hard way–not to use profanity in my test text when an executive asked me to demo our software to someone outside the company and I had to use my test environment. In the same vein, Rian van der Merwe has collected  instances of placeholder text that have unintentionally  made it into the real world–both in new and old media. It’s great.

Assigning colors to a BIRT stacked bar chart based on series values

by Stan on 2013/04/04, no comments

This has nothing to do with QA, but I’ve been writing a lot of BIRT reports lately; I just solved this problem and wanted to make sure my solution is out there for others who need it.
This is the stacked bar chart that I created:
There are three possible values: PASS, FAIL and NOTRUN, and I wanted to make sure that each value got a specified color: green for PASS, red for FAIL and grey for NOTRUN (not tests showed this value on the chart that I used for this post)
The solution was to select the chart, click on the ‘Script’ tab and enter the code below. The beforeDrawDataPoint function draws the bars. the beforeDrawLegendItem function draws the legend:

* Called before drawing each datapoint graphical representation or marker.
* @param dph
* DataPointHints
* @param fill
* Fill
* @param icsc
* IChartScriptContext
function beforeDrawDataPoint( dph, fill, icsc )
var Condition = dph.getSeriesDisplayValue();
if(Condition == "PASS") {fill.set(0,153,0);}
if(Condition == "FAIL") {fill.set(255,0,0);}
if(Condition == "NOTRUN") {fill.set(204,204,204);}
* Called before drawing the legend item.
* @param lerh
* LegendEntryRenderingHints
* @param bounds
* Bounds
* @param icsc
* IChartScriptContext
* @since Version 2.2.0
function beforeDrawLegendItem( lerh, bounds, icsc )
var Condition = lerh.getLabel().getCaption().getValue();
if(Condition == "PASS") { lerh.getFill().set(0,153,0);}
if(Condition == "FAIL") { lerh.getFill().set(255,0,0);}
if(Condition == "NOTRUN") { lerh.getFill().set(204,204,204);}