Transforming Software Testing Landscape
By Sachin Mulik, VP & Chief Solutions Architect, Amdocs Testing Services
Embracing Cloud Computing
I see the main benefits of cloud computing for software testing in the following areas:
1. Virtualization of testing environments
2. Cloud-based analytics
3. Cloud-based device and lab infrastructure
4. Cloud-based Test tools
At Amdocs, we embraced cloud computing for our testing practice a few years back and have made excellent progress in all four areas –from tools for creating virtual test environments on demand, to defect prediction analytics and NextGen mobile and IoT device lab services on the cloud that can be used by any of our customers.
Amdocs BEAT, the platform consolidating our testing best-practice methodologies and tools, is cloud-enabled, helping to facilitate collaboration across Amdocs and customer project teams, with full transparency of priorities and testing status, as well as introducing agile workflow methodologies as we shift into continuous testing.
Challenges in Big Data Analytics, I agree that it’s challenging and the main challenges are twofold:
1. Gathering the right data
2. Harnessing the collected data and putting it to real world use to solve real problems resulting in real benefits
The talented Testing R&D team at Amdocs accepted this challenge 2 years ago and cracked this problem. Here’s how:
1. They narrowed the best possible use for data analytics in testing such as defect prediction:
a. Simply collecting massive amounts of data is useless if you do not know why and how you are going to harness it. So it’s important to identify a few objectives upfront, and then identify what kind of data you may need.
b. Our R&D team put forward a few objectives, and then analyzed the data that was available (or could have been collected). They conducted POCs to test this hypothesis.
2. They then developed communications industry-specific algorithms resulting in highly accurate defect prediction models.
We now have a way to predict which parts of code or which applications (e.g. billing, CRM, and activation, etc.) are likely to have a higher number of critical defects for any given change. This feeds into our test planning, resource planning and test data modelling, and helps us optimize it. But most importantly, it reduces the number of severe production defects that occur right after any major launch. This has direct impact on the end customer experience and resulting satisfaction, as well as on the overall IT costs.
Focus on End-User Experience
Software testing professionals need to focus on the end-user experience at every stage of testing process. They need to think like end-users, behave like end-users, and test like end-users, detaching their mindset from the development/IT organization that they are part of, while acting on behalf of end-users. They then need to share their findings with product and application designers.
Software testing professionals need to focus on the end-user experience at every stage of testing process
This will make a huge difference to the end user experience and drive further growth in this connected digital economy.
The Future Lies in Agile Practices
Agile practices have come a long way and have become mainstream. More than 60 percent of the organizations are undergoing some form of digital transformation. DevOps is proving to be very effective and no longer considered to be an ‘experimental’ method – almost every single organization I work with has at least one massive DevOps program.
This has a huge impact on test automation. Traditional test automation tools and techniques are quickly becoming irrelevant. There is a desperate need for NextGen test automation tools and techniques that share the landscape with development and operations homogeneously. Test automation needs to build on basic blocks created during Agile sprints, reusing them to stitch together end-to-end test scripts.
At Amdocs, we have developed a NextGen test automation framework that can be used by development teams to create smaller automated unit tests, useful for DevOps / SDET testers to build end-to-end automation test scripts. These automated scripts can be kicked off automatically every time there’s a new version of code available.
The same automation concepts need to be adopted across all application platforms and channels–digital, legacy, mobile, storefronts, IoT, etc.
Considering Basics as CIOs
Interestingly, regardless of the dramatic changes the industry is undergoing, or maybe because of them, I think it’s important to go back to considering the basics:
1. Is it working?
a. Are my applications doing what they are supposed to do?
b. Are my applications not doing what they are not supposed to do?
2. Is it secure, fast and reliable?
3. Is it loved by your audience?
a. Is it usable?
b. Does it meet legal requirements for disability / accessibility?
c. Are user friendliness of your applications, services, platforms in sync with the industry?
4. Faster, better, cheaper?
a. Can we do this more cost-effectively, at a higher quality and quicker?
b. Are we innovating enough? Are we staying ahead of the curve?
Technologies Impacting Costs
I see the following technology advances directly impacting software testing costs (driving them down):
1. Cloud technologies: Virtualized test environments that can be created and deleted quickly as needed and shared, cloud-based infrastructure labs that are accessible globally
2. NextGen test automation tools and techniques: Automate early, automate more, automate with non-specialty skilled personnel with ease
3. Analytics: The reality of software testing is that you will always have constraints on time and resources. So the only solution is to optimize the time and resources you have to deliver optimal quality. Analytics help to identify optimization solutions. You know exactly where to focus, how much to focus, how much to test and when to stop testing.
4. Data solutions: Make the right quantity and right quality of test data available in the quickest possible time.
IoT in Software Testing
Imagine every single device/piece of equipment around you running on software and always connected–with the Internet and other devices– constantly sending and receiving data. Everyone expects these devices to work flawlessly, be secure, quick and user-friendly. Some of these devices will be used in mission/ life-critical industries such as healthcare and transportation where failure is not an option. These are exciting times in Software Testing. There is a need to expand breadth and depth of software testing multiple folds, while still reducing the overall cost and elapsed time needed for testing.
Extending Automated Testing
Steve Berczuk, Software Developer and Team Lead, Fitbit
To Shift Left or to Shift Right? Continuous Testing is Everyone's Responsibility
Jeff Scheaffer, General Manager, Continuous Delivery, CA Technologies