In 1971, The Who released the single Won’t Get Fooled Again from their upcoming album Who’s Next. Fast forward to 2002 when this tune was introduced to a new set of listeners as the opening theme to CSI: Miami (cue witty remark by David Caruso’s Horatio Caine).

Apart from the sheer rock-and-roll brilliance of this song, I’ve always been drawn to the lyric “meet the new boss, same as the old boss”. This lyric strikes me as both poignant and revelatory: sometimes, a new thing ends up being basically the same as the previous thing. It’s come up many times in my life and my career. I frequently quote it when I’m experiencing the “same old situation, yet again”.

This is particularly true with questions I see about choosing a test automation tool.

Back around 2014, I created a talk inspired by questions I saw in LinkedIn automation and testing groups. Many of the questions started with a phrase similar to “What’s the best tool for…”. Now, don’t get me wrong, there’s nothing wrong with asking questions, not even these kinds of questions. The challenge we face with these questions is the lack of context; without at least some context, it’s extremely to give responsible guidance. What’s worse, in my opinion, are the answers that I’ve seen. One word/phrase answers like “Selenium” or “Playwright” by supposed experienced colleagues can inadvertently lead the question asker down a path that is inappropriate for them, though it might be totally appropriate for the question answerer.

The year is now 2022 and the same questions are still being asked. I know because I checked; it’s why my talk from 2014, You Bet Your Life – Playing The Automation Tool Selection Game still gets accepted at conferences. So, what do we do about this?

One thing that will help is when asking and answering questions about “the best tool”, we should keep in mind the automation ecosystem; this ecosystem is our context, specific to us. I describe the automation ecosystem in terms of three aspects: strategy, audience (direct and indirect), and environment.

If we can couch our questions with at least a portion of our ecosystem, responsible question answerers will consider that context when replying to questions. The whole feedback loop will become more valuable this way.

For example, instead of asking, “what’s the best tool for web browser testing?”, something like the following can produce more valuable results:

I’m looking for some help selecting an automation tool. I work in a highly regulated industry for a company whose tech stack is .Net. We handle PII and are unable to use external cloud-based tools, though an on-prem solution could be workable. The testing team has a few members with programming experience, but we don’t anticipate the ability to hire additional expertise in the foreseeable future. We need to create some browser-based test automation in C# to test against Chrome and Edge; we also need to test REST API calls. We need consolidated reporting of all automation runs. We’re adding automation to increase what we can cover during our testing intervals; application defects have a high cost to us and our users. What tools might be appropriate for us?

Though my wording is a bit forced, we can observe that each of the three automation ecosystem aspects is briefly mentioned. For the strategy, this person wants to reduce defects to reduce cost. For the audience, the testing team has limited programming experience. For the environment, the company is in a highly regulated environment that handles PII, has a .Net tech stack, and likely can’t take advantage of commodity, cloud-based solutions.

Certainly, it took a lot longer to think about our ecosystem and type our description of it, but not that long. Also, most assuredly, some readers will skip it because it’s “too long”. Perhaps, though, the people who skip it due to its length aren’t the ones we want answering these questions anyway. Questions like the ones we’re asking here require deliberate thought and conscientious answers. Someone simply replying with the name of their favorite tool or the tool they’re currently using isn’t all that helpful.

And for those of us that do take time out of our lives to answer these questions, please, let’s put some thought into our answers. As I previously mentioned, terse answers lacking any context aren’t all that helpful. If we want to be of help but the question asker hasn’t provided any context regarding their ecosystem, perhaps we should state the ecosystem from which we are answering the question. Better yet, we can ask follow-up questions of the asker to help ascertain their ecosystem.

For what it’s worth, I’ve noticed similar questions on development-related forums about frameworks and tools. I suspect those types of tools also have their own ecosystem that should be considered when choosing an appropriate framework for application development as well.

So, how about we make the new automation question, different from the old automation question? We can do that by adding some of our ecosystem context to our questions as well as our answers.

Like this? Catch me at an upcoming event!