Shopify's Mobile Device Testing Lab Gets Wheels

With a never-ending variety of devices and browsers being used around the world, it’s more important than ever to test our projects on real devices. At Shopify’s Montreal office, we recently took our mobile device testing lab off the wall and strapped on some wheels. It’s helped us solve a number of problems other offices have encountered when running a device lab, and introduced some pretty cool new features. In this article, we’ll cover: How this new breed of mobile device testing lab helps teams build better projects. What each incorporated feature is and how it solves mobile testing problems. How to build your own mobile device testing lab on wheels.

With a never-ending variety of devices and browsers being used around the world, it’s more important than ever to test our projects on real devices. Anyone who has tried operating a mobile device testing lab for a large number of users knows it’s a serious undertaking.

With so many moving pieces, the device lab quickly falls into disorder without constant maintenance and oversight; cables go missing, devices aren’t returned, and batteries die.

The Shopify Montreal office is now the proud owner of a fancy new mobile device testing lab. By taking the mobile device testing lab off the wall and strapping on some wheels, we’ve been able to solve a number of problems other offices have encountered when running a device lab, and also introduce some pretty cool new features.

Let's take a look at what makes this new breed of device lab so awesome, and find out how to build one (or more) of your own!

Test wherever works best for you

The most obvious feature of the mobile device testing lab on wheels is its wheels! These little rollers allow the device lab to be wheeled directly to where users need it.

The device lab can be rolled to your desk when you need to tweak project settings on your personal laptop as you’re performing cross-device testing. Or, it can be rolled to meetings or workshops to help provide first-hand experience to participants on topics like accessibility and performance.

"The device lab can be rolled to your desk when you need to tweak project settings on your personal laptop as you’re performing cross-device testing. Or, it can be rolled to meetings or workshops to help provide first-hand experience to participants on topics like accessibility and performance."

But wait! Devices go missing on device labs all the time, now we need to deal with an entire missing device lab? How do we know which floor, pod, or meeting room our device lab is in?

We’re currently working on adding location tracking through our office WiFi network via the open-source library Find. Eventually, users will be able to locate the exact location of the device lab through an easy to access URL.

Are you building products for an international audience? Learn how pseudo-localization can make the process easier.

All the devices you need at the same time

Devices are securely attached to the lab by velcro, but can be removed so they can be tested in a user's hands. Users are discouraged from separating individual devices from the lab for extended periods of time to reduce the chance of a device going missing or left uncharged. We’re hoping that the convenience of a rolling device lab, along with the fixed positioning of the devices leads to fewer missing devices and cables.

The device lab on wheels comes preconfigured with a variety of devices that provide full coverage to the top 15 browsers used with Shopify Checkout. There’s no need for the user to pick and choose the devices they want from a overstocked device wall, or to wait for that one super popular device that is never available. When it comes time to test a project, you can test on all devices at the same time.

To save time while testing across multiple devices, the device lab’s MacBook comes preconfigured with programs like Ghostlab and Browsersync. Users can specify the URL they wish to test on Ghostlab, and click its icon on each device to open that same URL. They can then proceed to interact with the page and the experience will remain synchronized across all devices.

The device lab’s MacBook also comes with a variety of virtual machines for Windows-related testing, from Internet Explorer 8 to Edge. It also includes accessibility screen readers, like JAWS and NVDA.

You might also like: How to Build Usability Testing Into Everything You Do.

Scalable design means our labs can grow with our needs

Scheduling conflicts will become more and more common as the device lab grows in popularity. That’s why we integrated our device lab with Google Calendars, so it acts just like a meeting room. A user can reserve the device lab for whatever time period they need it for, without worrying about its availability.

The best part about taking a device lab off the wall is that you are no longer limited by its size and immobility. If we discover that the device lab is being overbooked, we can simply build a second one. However, one problem that we’ve come to recognize is that the device lab is designed for cross platform testing. If a team needs a single device for everyday use, they should have their own device separate from the device lab.

The fact that we can make more device labs means that we can not only scale the solution within our office, but to other Shopify offices as well (there are six in total). Our hope is that if more offices use the same solution, we’ll have more people contributing to the design and features of the lab, ultimately meaning a better overall product!

How we built our device lab on wheels

The construction of the device lab was pretty simple and mostly consisted of sticking cables, devices, and power supplies to a drafting table using velcro and sticky cable clips. A shopping list of materials we chose and links to where they were purchased can be found in the Device Lab on Wheels Shopping List.

A few notes on why we chose the items we did:

• Desk — We needed a surface that could tilt to a nice angle and that had wheels. A bonus feature we hoped to find was a desk with adjustable height, to allow use while either standing or sitting. The desk we chose fit the bill but left a lot of room for improvement. The adjustable height is impossible to do on your own and the included cheap plastic wheels wobble when rolling. We ended up replacing the included wheels with those mentioned in the shopping list.
• USB hub — One thing stressed by Lara Hogan in her guide to a device lab is to do your research when it comes to power supplies. We ended up choosing the Anchor 10-port Hub because it has three high amperage ports for charging tablets, and seven low amperage ports for charging phones. We’ve noticed that when all the devices are empty, it does take a while for them to charge, but they nevertheless all charge over the span of 24 hours.
• Cables — We were able to find all of the matching cables we needed from Anchor. Buy the longer ones so that you have lots of room to play with.
• Cable Clips — Probably the best $18 we’ve ever spent is for this pack of 200 sticky cable clips. Let your obsession for cable management consume you, and organize those cables to your heart’s content.

The devices we chose to include on the cart are based on the top 15 browsers used with Shopify Checkout. In addition, we made sure that we included a variety of both old and new devices to represent users with phones of different computing power, which can have a huge impact on the performance and usability of a project.

As with any device lab, the devices you choose for your lab should be a representation of the userbase being tested. The majority of your users are probably not using the newest, most expensive devices on the market, so don’t feel like you need to spend tons of money to have them in your device lab. Make due with the devices you have around and invest wisely in devices you really need. For example, we decided to purchase a Microsoft Surface because a both IE11 and Edge are in our list of top 15 browsers. In addition, the Surface enables us to test touch interactions at desktop resolution — something we often forget about handling when creating new UIs.

What we’re thinking about for the next iteration

The device lab pictured above is a proof of concept, and was planned and assembled in just a few weeks. We’re currently in the process of iterating on the design and build of the device lab. Some of the improvements we’re considering are:

• Custom table top — The current approach to cable management resembles that of a mullet — business in the front and party in the back. Initially you’re greeted by nice cable management and device layout, but upon further investigation, you find a mess of cables strapped to the back. We’d like to hide all of these cables, including the retractable power cable inside a custom table. In addition, we’d like to find a better method of attaching our devices to the table — maybe a pegboard surface?
• Custom frame — We would love a frame that rolls smoothly and allows the tabletop to be used at various angles and heights.
• Onboard testing server — All of our devices are currently attached to a data and power USB hub. It would be cool if we could take full advantage of this data connection, and setup automated testing for iOS and Android.
• WiFi tracking — As mentioned above, we want to make it as easy as possible for users to find the device lab, no-matter where it is in the office. Using the open-source Find library, users will be able to locate the exact location of the device lab through an easy to access URL.

You might also like: Growing Your Agency’s Revenue: 3 Metrics to Help You Plan for the Future.

Build your own device lab on wheels

Device labs shouldn’t be a burden to use and maintain. The less friction we have in device lab setups, the more likely our colleagues will use them to make even better projects. Device labs on wheels provide all the benefits of traditional, wall-mounted device labs with the added benefits of mobility, convenience, and scalability.

Talk to some of your colleagues about your device lab setup. Could it benefit from an upgrade?

Have you built or worked with a mobile device testing lab? Tell us about your experiences in the comments section below!