Cloud Infrastructure

Setting a baseline

When I first joined the Cloud Infrastructure team in October 2016, I wanted to quickly get a baseline of our portfolio’s usage. I analyzed traffic and behavior data for Softlayer by creating graphs on Google Analytics. I used this and Hotjar analytics to compile a report for Q4 2016 for Virtual Server (VSI) and Bare Metal instances. I then worked with design lead Janelle Arita to build data-informed as-is journey maps for the VSI “discover, try, and buy”, “manage”, and “support” experiences, including prioritized pain points and suggested interventions.

However, I soon found many discrepancies in the data being collected, leading us to overhaul our analytics gathering policy and switch to using Amplitude. Not all pages where most of the traffic and ordering happened was fully instrumented until much later. Until then, I needed to find new ways of collecting and attributing confidence to my data and insights for others to make use of.

Cloud CDN

I also used this baseline data and Hotjar analytics to compile a report for our Content Delivery Network (CDN) service. My designer colleague Steven Raden led the design while I helped with both heuristic evaluation and user testing, both live and remote. We combined this with another test for new UI elements used to compare card-based and table-based layouts for comparing offering options.

Following its beta launch, we prepared a constant evaluation plan with offering manager Annemarie Fitterer, including surveys, interviews, and usability testing. The offering was also instrumented with NPS and Amplitude for more aggregate usage analytics.

Cloud Gateways and security

One of the most popular network infrastructure offerings was our Vyatta device that could be used for any number of applications, particularly as a gateway and firewall device. However, it was limited as a physical device to individual pods within data centers. This made it less flexible than many customers desired, prompting a new offering: a virtual gateway as a service. I worked with offering managers, architects, and designers across all network offerings to align our research efforts on setting a baseline, planning for iterations with testing, and continuous evaluation.

My first efforts were to work with offering managers in helping coordinate beta programs, starting with improving their questionnaire and survey design to ensure their findings are better controlled for bias and more likely to be fully completed by participants. I also worked with sales and support contacts to recruit sponsor users and gather feedback they had heard from customers into a map.

On the design side, Steven and I worked together with offering manager Meeyoung Yoon and the development team to synthesize the team’s experience and market knowledge. In a short session, I helped them build a stakeholder map and personas to align on who we were designing for.

Genesis

As part of Genesis, a major project for IBM Cloud to update our infrastructure, virtual servers were the key offering that prompted our hiring a dedicated designer, Megan Baxter, for the task as well as to lead the Compute team. I worked with her on the many iterations of new interfaces using unmoderated remote user testing, surveys, and card sorting sessions.

They were a regular part of our sprints, although they were often every other sprint due to the workload on Megan to build prototypes for both dev teams and user testing scenarios. We co-developed research questions, protocols, and analysis plans for each feature, flow, or comprehension topic that we focused on at each sprint.

Closer to our beta launch, we finally had enough stability in design and completeness in scenarios that we could better research cross-offering flows that test the overall user experience. One example was in understanding how people made sense of our offerings in relation to each other. I collaborated with each offering’s lead to develop an open and closed card sorting test to accomplish that.

Snapshot of results from an open card sort on IaaS offerings

The results were very useful in helping us work with other Cloud teams who were focused on the overall navigation and cross-selling opportunities for potential customers.

Bridging Softlayer and Cloud

We worked with a growth team for a few months to create an experience that would increase conversion between Softlayer and Cloud domains and open a new channel for Cloud-only users to create virtual servers. We did have some conflicting opinions on how the page should be structured to arrive at this goal, though.

Working with Janelle Arita, one of our design leads who owned the VSI provisioning form, I took her mockup and sliced it into horizontal strips of one form step each, including section headers. I then built a sort of closed card sort test, including a think-aloud protocol, where participants would drag elements into a position they would like, allowing them to leave some unsorted. We then had some push-back on a particular header, so I ran the test again without it but found little useful other than greater overall variability.

Analysis of closed sort for steps in provisioning process

We found very useful information about how people grouped certain things together as well as roughly where overall certain steps should fall. This outcome directly affected our final provisioning form experience. Follow-up usability studies as well as live usage data (including errors) helped us discover more usability and technical issues we continue to improve on.

Budgeting

Due to an extremely long development cycle, there was actually little change on the Softlayer side until August 2017 when there were discussions of new pricing options. Offering changes move more quickly and I had caught wind of this later than desirable, so I decided on a fast study of why people make certain budgeting decisions. I immediately went to my account data to determine who were the top accounts with the parameters that would most be affected by these decisions. Using that, I investigated why those clients use certain pricing parameters by asking my sales and support team contacts.

I compiled a questionnaire informed by these unstructured interviews to be distributed to customers and their IBM representatives. The results helped inform my offering managers and allowed for a fast turnaround for my design team in presenting these choices to users in a way that better matched their concerns.

Feature image courtesy wocintechchat.com (CC-BY 2.0)