Who do call if you’ve got an app to repurpose, a team of developers starting any day, a behavioral science department raring to go and a ton of transaction data to sort out? Hopefully me!
With experience of quality assurance, data analysis, a psychology degree and personal climate activism, this job was a perfect fit. I started work a few weeks before the dev team and documented every nook and cranny of the old eco shopping app. Over the next year we took the app apart screen by screen and rebuilt it as a personal carbon footprint tracker using open banking to bring in transactions. Through this time I remained the end-to-end app expert, managed all the quality assurance and was responsible for data quality and analysis
Just let me break stuff!
Using a mix of having fun trying to break the app and methodologically testing app behavior in different scenarios, I provided thorough quality assurance of the app as it was redeveloped. I’ve always been fairly good at breaking software, I just get curious about “what happens if…” or can smell a problem somewhere and next thing you know I’ve found a juicy bug that needs fixing! I would also proactively keep an eye on user data looking for bugs, especially after new versions we’re released. For features under development I would discuss each bug with the development lead and write it up as they preferred, and for issues in the production app I would write a bug report.
Complex user journeys
The breaking stuff was always fun, but I enjoyed building out user journeys too. I created and tested user journeys for many complex scenarios, for example, behavioral science wanted to test letting users make a commitment to change their diet. Users could swap out beef, go veggie or go plant based for a period of time. This would then reduce their carbon footprint on food transactions during that time and show a total saving. Users could also change their diet between the types over time, for example, maybe they normally just avoid beef but try Veganuary then swap back. Transactions in each diet period should show the correct carbon foortpint, saving, graphic and link to the correct prompt for the an even better option. Then we did a similar plan for public transport instead of fuel, train instead of plane, second hand clothes instead of buying new clothes. As you can imagine there we’re hundreds of scenarios to test and I found several bugs.
Driving the data!
The app needed to collect the data generated by user behavior, as that would allow testing the behavior change hypotheses. As an oversight of the project set up, no-one was responsible for specifying how the data collection wold work so I took that on. I designed the data collection requirements for each new feature during development, this involved reviewing all the feature specifications to understand what was being measured and the design briefs to understand what would users do in the app and writing list of actions that would be recorded in Amplitude. I would then pre-bulid the charts for initial analyses so data would feed in straight away. I would also test each action was feeding through correctly into Amplitude and into the graphs before launch.
Quality data
I took lead on improving data quality in Amplitude and also on Equifax transaction data and how it mapped to categories in the app, and the proprietary carbon model where relating to the app. I deleted or merged thousands of old and broken events in amplitude and was gatekeeper for what new data would get collected. Every stage was documented meticulously so that, for example, sql queries in Metabase could be adjusted as event names changed, saving significant amount of time for the data science team.
Communication
The project structure was quite complicated and so my communication skills were really useful in coordinating between the various departments. I worked under engineering and liaised with the product and research teams daily, becoming the go-to expert in app functionality. I championed documentation at every stage. I advised the data science and impact teams on the adequacy of data generated by the app users and carbon model in relation to the app. I also set up a virtual board game club!
Sadly the project was wrapped up during the silicon valley layoffs in 2022 and my contract was ended. I’ve since been on maternity leave (last updated Oct 2024).