Overview
The Pittsburgh Study, housed within the University of Pittsburgh Department of Pediatrics, created a new framework for measuring child thriving in communities, considering not only individual well-being, but also the social and environmental conditions that shape it.
Our team was initially asked to revive a custom-built Child Thriving Matrix (CTM) application that had been taken offline due to unsustainable hosting costs. However, through research and user testing, we discovered that restoring the application would not address the deeper challenges facing the research team. Rather than rebuilding a tool, we ultimately designed a sustainable community research ecosystem centered around accessible facilitation, low-cost technology, and long-term data management.
Challenges
The Existing Platform Was Unsustainable
The custom-built application was taken offline due to AWS hosting costs exceeding $1,000/year, and restarting it required dedicated technical staff the team did not have.
Community Access To Technology Varied
CTM community sessions span a wide age range with highly variable tech literacy. Not every participant owns a mobile device, and not every session location has reliable Wi-Fi or service. Our solution had to work for everyone in the room, not just the tech-confident.
Initial Research
Secondary Research:
Child Thriving Matrix Document, The Pittsburgh Study Papers, etc.
We analyzed the Child Thriving Matrix document, which listed community themes and their sub-topics on a 0-7 scale with descriptions for the 4 Categories of Thriving for community members to rate which level they believe their community is at.
Expert Interviews:
Weekly meetings with clients and community session research staff
Clients valued anonymity, simple language, and the ability to spark community discussion.
They explained the constraints of the existing system:
Participation spans a wide age range with highly variable tech literacy.
The solution had to be low-cost and maintainable without dedicated technical staff.
Facilitators needed tools that worked reliably alongside they juggle other responsibilities.
Competitive Analysis:
Evaluated 20 polling platforms to identify needs fulfilled by criteria
Criteria:
Ease of joining
Ease of use
Anonymity
Admin dashboard
Data logs
Live results
Community facilitation
3 platforms emerged as finalists:
Mentimeter, CrowdPurr, and AhaSlides
Revising Project Scope
Initial Project Goal: Restore the existing custom platform and improve it to align with the Child Thriving Matrix values —> However, "Fixing" the existing app was the wrong goal
Revised Goal:
Build a sustainable community polling system that works for the research team AND community members
Prototyping & User Testing
Round 1
Testing facilitators to validate platform choices before involving community members
Participants were facilitators and research assistants who run community sessions.
They tested the 3 finalist platforms in a think-aloud process and reflected that although all platforms were considered learnable, Mentimeter required the least facilitator effort.
More importantly, facilitators repeatedly anticipated participant difficulties, particularly among older adults and individuals with lower technical confidence. This shifted our attention beyond the digital interface itself and toward the complete onboarding experience.
Round 2
Testing a sample of members who attended a community session
10 members tested 4 variations of polling styles (differing question types, data visualizations).
Participants strongly preferred a hybrid format: live polling on devices supplemented by a paper copy of the questions
Technological Difficulties
Some participants did not own devices, QR-code knowledge was limited, unstable internet access
Facilitators need to multi-task
All participants required hands-on assistance from facilitators while joining polls
Participants liked seeing results live
They expressed strong interest in seeing and discussing collective results, since in regular community sessions, they cannot see collected data until the final session and cannot easily access them after
Paper Document needs to be more readable
Although the format with the supplemental paper document was preferred, the text was too small and the colorful scale was confusing
Final Round
Designing a Supplemental Paper Packet
1) Streamline the survey joining process by adding instructions
2) Make the thriving scale color-blind accessible by using a monochromatic scale which also makes the paper packet accessible for black and white printers
3) Make the survey questions more clear by reducing the number of questions on each page
Key Findings
Device Preparation is Necessary
We prepared extra devices for participants who did not have any, as well as extra hotspots in case the location's Wi-Fi was not stable.
Some participants did end up using these devices, showing that part of the facilitation process needed to be device preparation.
Participants highly preferred Multiple Choice
We tested two question types: sliding scale, multiple choice. Approximately 86% of participants preferred multiple-choice questions over sliding scales, citing greater ease of use and visibility of answers.
The Hybrid Format serves as a middle ground for varying technology backgrounds
This community had a younger and more technology-familiar group. Around 60% preferred a hybrid paper-and-digital experience, the rest wished it was more digital.
Final Prototype System
A Comprehensive System for Community Polling
Before Sessions
What do facilitators need to prepare for community sessions?
An all encompassing facilitator manual that covers how to use Mentimeter, build a Child Thriving Matrix survey, and run hybrid-format sessions, among other tasks, which ensures stable transition for future facilitators.
During Sessions
How can community members participate and have discussions in sessions?
We created all 8 Child Thriving Matrix surveys in Mentimeter and their paper packets with survey instructions and thriving scales. This hybrid format improved accessibility for participants with varying levels of technology comfort while preserving the benefits of live data visuals and analysis.
After Sessions
How can researchers record and analyze data from live surveys after sessions?
We created a data conversion tool that transforms Mentimeter data exports into a format compatible with REDCap, the team's platform for data collection, enabling long-term analysis without requiring manual data restructuring