I have a ton of personal Github repositories because I'm always tinkering with something. I often use them to build out proof of concepts for larger projects as libraries that I think will be re-usable for other people or myself later on.
For android projects and libraries in particular, I find that its often useful to have some tests running directly on real phones (instrumented tests), and not just on emulators. This is often the case when testing things like BLE advertising and discovery, Wi-Fi direct connectivity, etc.
There are a few options for this, things like Firebase Testlab, which can be quite expensive if you run many tests often. You can also self-host a runner and attach your own phones to it.
One major downside as a solo developer is that Github doesn't let you share a runner across all of your person repositories. You can do so if you have an org account or a business account, but not with a solo account. This creates a big problem as soon as you have more than one repository which wants to use self-hosted runners with the same phones. Suppose you put both runners on the same machine, and have the same phones attached. Now suppose the CI for both triggers around the same time. You will end up in a state where one of the tests is likely to fail because it will find the devices are already in use by the other CI job. There are some workarounds, like having each set of CI jobs only use some subset of the phones that aren't overlapping - but I wanted to use all the phones on all the tests.
So, I built a project to handle this: [SAIR - Shared Android Instrumented Runner](https://github.com/compscidr/sair). Effectively this project creates an ADB proxy which all the CI runners use instead of directly using ADB. Then this proxy co-ordinates with an orchestrator which handles locking mechanisms for the phones. Before the tests or any ADB commands run, the CI obtains the lock for the phones, then runs the tests / commands, then releases the lock. Instead of tripping over each other, there is now co-ordination so that each CI runner is able to temporarily lock access to the phones, run the tests, and release the lock so the next runner queued up can run its tests - which completely avoids the failure mode.
Now that its working, its also so much cheaper than Firebase TestLab which costs $5/device-hour. Here's some assumptions about some costs running with TestLab:
Assumptions:
• 100 tests/day
• Average test duration: ~5 minutes (typical for instrumented tests)
• 100 tests × 5 minutes = 500 minutes = 8.33 device-hours/day
Monthly cost:
• 8.33 hours/day × $5/device-hour = $41.65/day
• × 30 days = ~$1,250/month
Breakdown for different test loads:
| Tests/day | Avg duration | Device-hours/day | Monthly cost |
| --------- | ------------ | ---------------- | ------------ |
| 50 | 5 min | 4.17 | $625 |
| 100 | 5 min | 8.33 | $1,250 |
| 200 | 3 min | 10 | $1,500 |
| 500 | 2 min | 16.67 | $2,500 |
It's also far more efficient use of the phones. Most android devs usually have a couple of phones sitting on their desk which they use for testing locally. Its possible with this setup, that you could have android studio use the proxy as well, and have a runner on your dev machine so that you can split the testing between your local dev and the CI jobs (often the phones are just sitting there anyway - and you can have the CI runner going in a docker container which you don't even notice while you're building an debugging in your own branch).