Skip to content
Tutorials Center
My learning

Topics

Lesson 28 of 37 ยท Cloud Device Farms

Why Cloud Device Farms? Real Devices at Scale

Understand device farms for Appium: when emulators are enough, what real-device clouds add, the main vendors, cost models, and how to choose and compare them.

Your laptop can run one Android emulator and, on a Mac, a few iOS simulators. Your users run thousands of different phones: different screen sizes, manufacturers, Android skins, OS versions, memory, cameras and carrier networks. A bug that never appears on your emulator can crash the app on a mid-range phone from two years ago. A device farm (or device cloud) is a service that gives you remote access to many real devices, plus emulators and simulators, so Appium tests can run across that variety without a drawer full of phones.

This lesson explains when emulators are enough, what a farm adds, how to compare vendors and what it costs. No account, device or Appium server is available on this site, so everything here is conceptual, and the runnable examples are calculators and planners.

Emulators, simulators and real devices

Emulator or simulator Real device
Cost Free Hardware or a rental fee
Start time Seconds to a minute Ready immediately in a farm
Reset One command A farm cleans between sessions
Fidelity Good for logic and layout Real CPU, GPU, battery, sensors, camera
Vendor skins and quirks Not reproduced Present (Samsung, Xiaomi, Pixel...)
Push notifications, biometrics, payments Limited Closest to production
Parallel scale Limited by your machine Limited by your plan

A sensible mix: run the majority of tests on emulators in CI for fast feedback, then run a smaller smoke and regression set on a selected matrix of real devices before each release.

What a farm gives you

  • Many devices on demand, including old and brand-new models and OS versions.
  • Remote Appium endpoint: your tests connect to the vendor's URL instead of 127.0.0.1:4723.
  • Artifacts: video of each session, device and Appium logs, screenshots and network logs.
  • Parallelism: run many sessions at once, subject to your plan's limit.
  • iOS without owning Macs: the main reason many teams adopt a farm.
  • Special features: app upload storage, camera image injection, network throttling, geolocation, biometric and payment simulation.

Two kinds of farms

  1. Public cloud vendors. BrowserStack App Automate, Sauce Labs Real Device Cloud, LambdaTest, Kobiton, Perfecto, AWS Device Farm, Firebase Test Lab (Android and some iOS, with its own runner model) and others. You pay per parallel session or per minute.
  2. Private farms. Your own devices connected to your own Appium servers or to an open-source grid, run on your premises. This keeps data inside your network and may be cheaper at high volume, but you maintain USB hubs, batteries, OS updates and device cleanliness yourself.

A hybrid is common: public farm for breadth, a small private shelf for devices that must never leave your office or for tests that need special hardware.

flowchart LR
    T["Your tests (CI)"] --> E{"Where to run?"}
    E -->|"fast feedback"| EM["Emulators and simulators"]
    E -->|"release gate"| C["Public device cloud"]
    E -->|"secure or special hardware"| P["Private device shelf"]

Choose per test stage, not once for the whole project.

Choosing a vendor: questions to ask

  • Which devices and OS versions do my users have? Start from analytics, not from the vendor's headline number.
  • How does the vendor bill: parallels, minutes, seats or devices? What happens when the queue is full?
  • Does it support Appium 2/3 and the drivers I use, and which Appium versions can I pin?
  • Are session videos, logs and network captures included?
  • Can I test apps behind a firewall (a secure tunnel)? How is data stored and deleted?
  • Is there an SLA for device availability, and how long do sessions stay open?
  • Are my special features (biometrics, camera, SIM, payments) supported on real devices?

Stand-in: choose a device matrix from analytics

Start your matrix from usage data. This runnable stand-in picks the smallest set of devices that covers a target share of your users, a simple greedy algorithm. The numbers are invented for the example.

Python Device coverage planner (stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Estimating cost

Vendors price differently, but the main levers are the number of parallel sessions and the total device minutes. Rough planning math helps you negotiate and compare.

Python Run time and cost estimate (stand-in)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The prices are made up; always read the current price list. The lesson of the numbers is general: adding parallels shortens the wall-clock time almost linearly, but the fee also grows, and per-minute billing makes slow, flaky tests directly expensive.

Pitfalls

  • Treating the farm as a debugger. A remote device is slow to inspect. Reproduce and fix locally first.
  • Hard-coding vendor details in tests. Keep vendor URLs and capabilities in configuration so you can switch (see Connecting to a Cloud Grid).
  • Forgetting to quit sessions, which keeps a paid device busy until the idle timeout.
  • Uploading builds by hand. Upload the app in CI with the vendor's REST API so the tested build matches the commit.

Check your understanding

Quick check ยท 4 questions

of 4 answered You scored / 4 ยท perfect!

  1. 1. What do real devices in a farm give you that emulators typically do not?

    Real devices reproduce actual CPU, GPU, sensors and vendor customizations.
  2. 2. A sensible default testing strategy is:

    Mix the two to balance speed, cost and coverage.
  3. 3. Where should the first draft of your device matrix come from?

    Cover the devices your actual users have.
  4. 4. Why quit every session promptly on a cloud farm?

    Leaked sessions waste minutes and block parallel slots.

Summary

  • Emulators are cheap and fast; real devices find hardware and vendor-specific bugs.
  • A farm provides a remote Appium endpoint, many devices, videos, logs and parallelism.
  • Private farms give control; public ones give breadth; many teams use both.
  • Compare vendors on your device list, Appium support, billing, security and artifacts.
  • Build the device matrix from analytics and estimate cost from parallels and minutes.

Next: Connecting to a Cloud Grid