Skip to content
Tutorials Center
My learning

Topics

Lesson 52 of 60 ยท Automation Concepts and Process

Cloud Testing and AI in Testing

Learn cloud testing (testing in and with the cloud, device clouds, IaaS PaaS SaaS) and how AI helps testing: test generation, self-healing, risk, and its limits.

Two trends changed how teams test. First, software moved to the cloud, so tests must cover systems that scale up and down, run in many regions and depend on managed services. And the cloud also became a place to run tests with rented machines and device farms. Second, AI tools arrived that generate tests, repair broken ones and prioritize what to run. This lesson explains both in practical, tool-neutral terms, including their limits.

Two meanings of "cloud testing"

  1. Testing cloud-based applications. Checking software that runs in a cloud environment (a SaaS product, a system built on cloud services).
  2. Testing using the cloud. Renting infrastructure on demand to run tests: virtual machines, containers, browser farms and device clouds.

Real projects do both at once.

Cloud service models in one minute

  • IaaS (infrastructure as a service): you rent virtual machines, networks and storage and run everything on them.
  • PaaS (platform as a service): you deploy code to a platform that manages servers, runtime and scaling.
  • SaaS (software as a service): you use a finished application over the internet.

The model decides who is responsible for what, and therefore what you can test. With IaaS you test almost everything; with SaaS you mainly test configuration, integrations, data migration and your own customizations, because the vendor tests the core product. Always check the provider's rules, as some forbid penetration or load testing without permission.

Testing a cloud-based application

Beyond ordinary functional testing, pay attention to these areas:

  • Scalability and elasticity: does adding instances add capacity, and does the system scale back down? (See Performance Testing Concepts.)
  • Availability and failover: what happens when an instance, zone or region fails? (See Reliability, Recovery and Disaster Testing.)
  • Security and compliance: identity and access settings, encryption, exposed storage buckets, data residency. (See Security Testing Basics.)
  • Multi-tenancy: one customer must never see another's data, and a noisy neighbor must not slow others.
  • Integration and API behavior: timeouts, rate limits, retries and eventual consistency.
  • Configuration: infrastructure as code, environment variables, secrets and feature flags. Many cloud outages are configuration mistakes.
  • Cost: a test that scales to a thousand instances also scales the bill; set limits and shut things down afterwards.
  • Latency by region: users far from the data center may have a much worse experience.

Testing in the cloud

Cloud infrastructure helps the test process itself:

  • On-demand environments: create a full copy of the system for a branch, test it, then delete it. This fixes the scarcity of shared test environments.
  • Parallel execution: run a thousand automated tests across dozens of machines to cut hours to minutes.
  • Browser and device clouds: rent hundreds of real browser/OS/device combinations without owning them (see Compatibility and Localization Testing and Mobile Testing Basics).
  • Load generation at scale: produce traffic from many regions.
  • Elastic CI: build servers appear when needed and vanish afterwards.

Benefits are speed, scale and lower upfront cost. Risks are data privacy (do not put production personal data into a shared cloud), unstable networks between your test code and the provider, vendor lock-in, and uncontrolled spending.

flowchart LR
  C["Code change"] --> CI["CI pipeline"]
  CI --> E["Create temporary<br/>cloud environment"]
  E --> P["Run tests in parallel<br/>on many machines"]
  P --> R["Collect results"]
  R --> D["Destroy environment"]

Environments are disposable: created for a run and destroyed afterwards, so costs stay low and tests start clean.

Try it: how parallel test runs cut time and cost

This stand-in schedules 40 tests of different lengths across a number of cloud workers and reports wall-clock time and the cost units used.

JS Parallel workers: time versus cost

Press Run (or Ctrl + Enter) to see the console output here.

No output.

AI in testing

Artificial intelligence, especially large language models and machine-learning techniques, is now used in testing. The claims are loud; the realistic uses are narrower.

Where AI helps today

  • Generating test ideas and cases from requirements, user stories, code or API specifications. The output is a draft for a human to review.
  • Generating test data: realistic names, addresses, edge-case values, synthetic records (see Test Data and Environments).
  • Writing and explaining automation code: drafting scripts, locators, assertions and boilerplate, and explaining failures.
  • Self-healing locators: when the UI changes and an element's identifier no longer matches, the tool tries other attributes (text, position, neighbors) to find the element anyway, and proposes updating the script. This reduces brittle-test maintenance.
  • Visual and layout checks: ML-based comparison ignores meaningless rendering noise (see Visual and GUI Testing).
  • Test selection and prioritization: models learn from history which tests are likely to fail for a given code change, so the pipeline runs the riskiest first.
  • Defect triage: clustering duplicate reports, suggesting severity or owners, summarizing logs.
  • Anomaly detection: spotting unusual behavior in production telemetry or load-test curves.
  • Exploration: AI agents that crawl an application and try actions, like a smart monkey.

Limits and risks

  • Confident mistakes. Generated tests can assert the wrong behavior, check nothing, or duplicate each other. They look plausible, which makes errors easy to miss.
  • No real oracle. The AI does not know what the product should do unless the requirements are precise. Garbage requirements produce garbage tests.
  • Non-determinism. The same prompt may give different tests tomorrow. Treat outputs as drafts and commit reviewed versions.
  • Privacy and security. Do not send confidential code, customer data or secrets to external services without approval.
  • Self-healing can hide bugs. If a button was removed by mistake and the tool heals around it, a real defect disappears. Review healed changes.
  • Testing AI itself is different. Applications with machine-learning components need extra checks: data quality, bias and fairness, robustness to unusual input, drift over time, and evaluation of output quality with test sets, because the output is not a single correct string.

A good rule: AI speeds up drafting and maintenance, but responsibility for judgment, deciding what matters and whether the result is right, stays with the tester.

Try it: review an AI-drafted test list

Here a pretend AI has drafted test cases for a discount function. The stand-in runs them against the real function and checks whether each draft test is correct, meaningful and not a duplicate. This is what human review of generated tests looks like.

JS Reviewing AI-generated test drafts

Press Run (or Ctrl + Enter) to see the console output here.

No output.

Three of seven drafts needed to be rejected. If nobody had reviewed them, the suite would have shown a false failure, duplicated effort and a useless test.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. Which describes 'testing in the cloud' (as opposed to 'testing a cloud app')?

    Testing in the cloud uses cloud infrastructure as a tool: parallel execution, device farms and disposable environments.
  2. 2. What is a self-healing locator?

    Self-healing tools try other attributes to find the element and suggest an update, reducing maintenance. Healing should still be reviewed.
  3. 3. Why must AI-generated tests be reviewed?

    Generated tests are drafts. The tool does not know the true expected behavior, so a human must judge them.
  4. 4. Which risk is specific to running tests in a shared cloud?

    Production personal data should not be copied to shared or less-secure test infrastructure; use masked or synthetic data.

Summary

  • Cloud testing means both testing cloud applications (scalability, failover, security, tenancy, configuration, cost) and using the cloud to run tests (disposable environments, parallelism, device farms).
  • The service model (IaaS, PaaS, SaaS) determines what you can and must test.
  • AI helps with drafting tests and data, maintaining scripts, prioritizing runs and triaging defects, but its output needs human review.
  • Watch for hidden costs, privacy risk, confident mistakes and self-healing that hides real defects.

Next up: Becoming a Software Tester: Skills and Certifications.