How Can You Leverage AI in Testing

How Can You Leverage AI in Testing?

There's little doubt that generative AI (gen AI) and large language models (LLMs) are changing the game. But how do they really fit into our working lives? And how can you best understand and harness what they have to offer? Here are five areas where I think they're changing the way we work as testers right now.

AI Won't (And Can't) Take Your Job

The popular media perception is that gen AI is coming for all our jobs. Let's be clear: it can't, and it won't. Gen AI and LLMs draw on vast amounts of information and data - more information and data than a human could ever digest (or remember). What they lack is the next level - the wisdom and experience that humans bring to that information and data.

I liken it to Neo and Morpheus in The Matrix. Neo knows everything there is to know about kung fu, but he's never done it in the real world. It's why Morpheus, who has no knowledge but real-world wisdom and experience, can beat him so easily.

It means AIs can do a lot of simple tasks incredibly powerfully and therefore turbo-charge your productivity. These abilities will only increase as more commercial models come on stream. (See Keysight Eggplant's GAI and STE for perfect examples.) But fundamentally, the human-in-the-loop will always be needed because they bring the wisdom that's needed.

But let's not underestimate the power of gen AI to do the simple things well. This leads us to the next point…

LLMs Help You Achieve Work-Life Balance

Think for a moment about how brilliantly collaborative the world of testing is. If you're struggling with a line of code or don't know the answer to something, you just ask. You might ask Google and trawl through the results. You might ask a colleague on Slack. You might ask a question in a user community.

Whichever way you choose, 99% of the time you'll get the answer you need. The trouble is, it takes time. You've had to wait to get the answer you need, so your task has taken longer than you'd like. Your colleague or the people in the user community who answered your question took time out of their days to help you.

You can think of LLMs as the latest iterations of this way of working. You ask them question and they give you an answer - instantaneously. It gives us the potential to achieve a level of work-life balance that's eluded us in the past.

But to achieve that work-life balance, you have to know what you're doing - something we'll cover in the next point.

Getting the Best from Gen AI Depends on How You Talk to It

As we just explored, gen AIs lack your context-specific wisdom. If you want to get the most from them, you need to know how to talk to them to give them that context. This is where prompt tuning comes in.

Firstly, you'll need to give them the persona they need - by telling them you're a Gherkin engineer or an MIT professor, they know the level they need to be operating on. You'll likely need to refine your questions - or prompt tune - to get to the real answer you need. At the same time, you can relax knowing you can ask it a stupid question without fear of being judged.

What's interesting, though, is how open LLMs are evolving - and getting lazier. Where earlier iterations would give you the entire code base you needed, today's iterations may only give you a snippet. To get the full code base, you need to explicitly ask for it, even to the point of saying, "I'm feeling lazy today, I don't want to write any of the code and want you to do it for me."

Think of Gen AI as the Specialist That Complements You as a Generalist (And Vice Versa)

Throughout software development history, the pendulum has swung from specialist to generalist and back again. One minute, everyone wants generalist full stack developers. The next you need to be a specialist security engineer, a software tester, or a front-end developer.

Gen AI and LLMs can help you be better at being both.

When we first started to develop GAI, we fed it a wealth of testing-specific information. We then needed to help it understand relationships and dependencies so we could use RAG (retrieval augmented generation) architectures to enable it to deliver users the right information at the right time.

This means GAI is a generalist in that it knows - and can access - everything testing-related. When you're new to a test field, it's the specialist you need to get you up to speed. What's more, it's quicker than Googling and less embarrassing than asking a colleague.

It also means that when you're a specialist in a field it can be the generalist you need when something crops up that's outside your knowledge domain.

This brings us back to the collaborative nature of our world - and my final point…

An Office Buddy in a Post-Pandemic World

Pre-2020, we were all sat together in offices. We could celebrate the wins and commiserate the fails with our colleagues around us.

These days, we're likely remote workers. Who do you celebrate and commiserate with? Answer: your gen AI buddy. It's endlessly patient when you're working through the problem together, and it'll celebrate with you when you get something to work. Your gen AI stops being the monster that's going to take your job and starts being the selfless colleague who'll help get you your promotion.

Because, to come full circle: an AI won't take your job, but a human using an AI will.

Want to Explore More on Leveraging AI in Testing?

I discussed all these themes and more with Michael Ritchson, who spent 17 years as a NASA developer and is an innovator, principal engineer, and full stack developer who designs and develops software solutions for the aerospace and commercial industries. You can listen to our conversation in the test-talks.com podcast Gen AI: The Final Frontier. Enjoy!

Public link

Please use the above public link if you want to share this noodl on another website.