7 Key Areas to Consider When Selecting a Test Automation Tool

This month’s CPD session is focussed on choosing the right Test Automation Tool for your workflow and the evaluation process with Provar. Don’t forget to join us for demo’s and live Q&A’s with them during our CPD Office Hours on 17th August.

The Supermums CPD Programme focusses on equipping Salesforce talent with essential skills to enhance their CV and salary.

Sign up to our CPD programme to benefit from our weekly top tip bulletin, monthly office hours events with free Demo’s and Q&As, exclusive discounts, events and training.

During this month’s Supermums CPD Office Hours we’re focusing on how to choose the right test automation tool, the evaluation process and testing for beginners including a live demo from Provar.

• CPD Office Hours – 17th August at 9am PST / 12pm ET / 5pm BST

How to Choose the Right Test Automation Tool for your Business

If you’re ready to take the next step in your Salesforce testing journey, you may be considering adding a test automation tool to your workflow.

Among the myriad of options available, it can be daunting to select the right tool for your team’s needs. There are many aspects to consider before making your selection, and every test automation tool has its benefits and drawbacks.

They all provide different levels of ease of use, training, scalability, future-proofness, security, level of support, pricing, and more … but how to know what factors will make or break your team’s strategy?

It will depend on the size and scale of your team, as well as what stage of the testing journey you are on. If you’re just getting started with manual testing, for instance, you may prioritize a vendor with stellar post-sale support and training to get you up and running, but if you’re well into automated testing already, it may be more important to look for a solution that makes your existing tests more efficient, saving you time and money.

The evaluation process simplified into 7 key areas:

1. Test resilience or fragility
2. Polymorphism and reusability
3. Ease of use and learning
4. Testing adjacent systems
5. Support
6. Training
7. Customer success

Here’s a rundown of what to look for in each category and suggested questions and demo requests you can ask for.

Test Resilience or Fragility

To what degree are updates to the Salesforce system or updates to customizations likely to break tests and cause rework

Salesforce updates its platform multiple times per year, which can cause automated tests to break unless your testing tool was designed to operate with Salesforce. Between Salesforce updates, changes to customizations can also cause automated tests to break. Tests that break lead to rework, higher costs, and other negative consequences.

Here are some demos to request during your evaluation to test this area:

• Create a simple customization to create an opportunity.
• Create a test script that tests that the opportunity is successfully created.
• Make changes to the page layout, such as adding fields or changing the location of the fields on the page, and run the test case again.
• Create a test and show me how it continues to work when you move a button on the layout, for example.
• Show me step by step how your testing solution handles shadow DOM elements.

Polymorphism and Reusability

To what extent does the solution morph to the needs of the rest of the organization by utilizing a single test that can run across numerous contexts

The technology that makes tests resilient also enables polymorphism (when one test can run across numerous contexts) and reusability. Polymorphic tests that are reusable across your workflow reduce the time, effort, and expense of creating and maintaining tests. Fewer tests with more coverage mean a higher quality solution that can evolve more quickly and easily.

Here is a demo to request during your evaluation to test this area:

• Using the test from the prior demo (test resiliency), run that same test in a different environment (e.g., validation vs. development) in a different language (e.g., French), on a different browser, for a different user profile (e.g., a marketing VP/director) with special permissions (e.g., that reveal a sensitive field like total bookings companywide).

Ease of Use and Learning

What skills and experience levels are required to create, maintain, and run tests

Many people involved in Salesforce customizations are considered “citizen developers” – folks who may not necessarily be experts in programming languages and frameworks. It is important that the skills required to test Salesforce customizations are similar to the citizen developer skills needed to create the customization, so that these testers can easily work with their counterparts as they participate in, or even fully own, the testing process.

Here are some demos to request during your evaluation to test this area:

• Build a test case with the solution to see how easy and intuitive it is.
• Try to interact with the test case in real time: add, delete, update steps, pause, rewind, etc.
• Run a test script for a test that needs to be debugged and see if it passes/fails in real time or if you must run the test over again once it is fixed to confirm the issue is resolved.

Testing Adjacent Systems

To what extent is the system capable of running end-to-end tests that reach adjacent non-Salesforce systems

Every Salesforce platform connects to other enterprise systems and custom integration points, and it’s important that a Salesforce testing solution can reach out to other systems and test workflows end-to-end.

Here is a demo to request during your evaluation to test this area:

• Test an end to end scenario such as: email to case, website lead to Salesforce, lead to cash, etc.

Support

To what degree does a vendor offer ongoing support through demonstrations, one-on-one consultations, online help centers, trade show presence, etc.

The level of ongoing support a vendor provides their customers after closing the sale is essential. This support can come in a number of verticals: demos, one-on-one consultations, online help centers and community forums, a consistent trade show presence, and more. The more willing a vendor is to help their customers in the long haul, the stronger the relationship will become and the more satisfaction and trust you can have in the product.

Here are some questions you can ask during your evaluation to test this area:

• After the initial setup consultation, how do you provide continued support to your customers?
• Do you have an easily accessible online help center or a community forum?
• How quickly will I be able to connect with an expert from your team with questions, and what hours are you available?
• Are you attending any upcoming trade shows or conferences in my area?

Training

Whether or not a vendor offers additional or ongoing training services beyond the initial consultation or setup

A good vendor will prioritize continued education. As product updates are made and new features are introduced, vendors should offer readily accessible courses that users can take on their own time to brush up their skills. They can also offer continued training through webinars, blog posts, and white papers.

Here are some questions you can ask during your evaluation to test this area:

• Do you offer training courses for users to expand their product knowledge?
• Do you offer free webinars for customers?
• Do you have a blog?
• Do you have a library of white papers available for browsing?

Customer Success

How many customers have shared success stories using the vendor’s products and the caliber of reviews from reputable industry review sites, peers, and analysts

The best indication of whether a vendor is the right fit for your team is reflected in their customer success stories. Look for the number of customers who have shared positive success stories using the vendor’s products (and also make note of any negative feedback), and spend time gauging the caliber of reviews from reputable industry review sites, peers, and analysts.

Here are some questions you can ask during your evaluation to test this area:

• Do you have customer case studies or success stories available from my industry?
• When I was researching your company, I noticed that there was negative feedback published on [site] regarding [topic]. Can you speak to this feedback, such as ways your company has improved this area of business?
• Do you have any reviews you could provide from reputable industry review sites, peers, and/or analysts?
• Would any of your existing or past customers be willing to answer a few questions?

Interested in learning more about the evaluation process? Join Provar on Wednesday, August 17th at 9am PST / 5pm BST for open office hours! The team will discuss testing for beginners and will provide a demo of Provar’s offerings. For more, please visit www.provartesting.com.

This blog was written by Sydnie Mulcahy, Content Lead at Provar Testing.

Join us for our August CPD Office Hours

During this month’s Supermums CPD Office Hours we’re focusing on how to choose the right test automation tool, the evaluation process and testing for beginners including a live demo from Provar. Join them during these office hours:

• USA CPD Office Hours – 17th August at 9am PST / 12pm ET / 5pm BST