Could digital communities become research arenas?
How a Facebook group served as a tool for learning about Pods.NYC users
In the early days of Pods (before it was even called Pods), back in July 2020, my partner and I were trying to gather information about NYC parents’ needs, frustrations and motivations, in an ambitious attempt to address their most pressing problem — their children’s education amidst the world pandemic.
As we were in a very early stage of this initiative and didn’t have any budget on hand, we were trying to think about creative, low-cost, and effective ways that would enable us to “feel” our users.
Starting a Facebook group for our NYC neighborhood called “Pods in XXX” was an idea that popped in our minds early on, and without giving it too much thought we went for it, thinking “worst case we would help people around us”, which was our idea anyway.
We started this group in mid-July, and it reached over 750 members in less than 2 months when remote partial school days for students in public schools started. During these 2 months, the group became our research arena for the product we were building.
Organic, unmediated data
As someone who conducted user research before in more traditional manners, the most immediate advantage of the group that occurred to me was the fact that the data we collected through it was entirely organic: we had unmediated access to the users we were trying to solve for. The group almost felt like an active social listening tool, or modern days ethnographic research, only in this case we were the ones creating the stage for it.
Many insights about parents’ frustrations and needs in creating alternative educational programs presented themselves in the form of posts and comments, which we didn’t raise on our own.
Another important insight about the different characteristics of our users emerged from group discussions, and confirmed our hypothesis, that parents have a range of perspectives regarding educational approaches and safety standards during the pandemic, and so our product would need to be accommodating to that range.
This last insight brought us to think about Pods almost like a matchmaking app. If dating apps aim to match potential romantic partners, our job was to know each family’s characteristics in order to match them with a potential family to form a learning pod with.
The Meta-Data
On top of the organic data, our FB group allowed the collection of much data that helped us understand the profile of people who joined it. For example, our preliminary membership questions required joiners to state if they were parents or facilitators, state kids’ ages, and list their neighborhood.
Another great meta-data insight was derived from the percentage of posts we’ve seen from facilitators, who used the group as a platform to reach their targeted audience of local families. We realized that facilitators — many of them teachers who had lost their jobs due to Covid — lack a platform to market their skills in such a way. This was a pain point we tried to address (among others) in Pods.NYC.
Providing user feedback along the life-cycle of the product design
As research questions and needs change along the life cycle of product development, the lively and engaged community was an extremely handy tool in the different stages of design. It not only served us well through the preliminary research — defining pain points, needs, personas — but also for concepts testing, usability testing, and MVP launch.
Over and above that, we were able to easily source users for conducting in-depth interviews and surveys.
Designing the right environment
When conducting a focus group, the facilitator needs to provide a comfortable setting (including refreshments!) and lay concise ground rules.
In the same way, in order to allow those desired organic interactions to happen in our group, we had to encourage group members and make sure we’re facilitating a variable of experiences.
We wanted to set a tone and create a safe space where people would feel comfortable connecting and saying what’s on their hearts and minds.
Here are a few examples of means we used to do so:
- Group rules to which any member had to agree upon once joining reflected the values we wanted to emphasize. One of the main ones being “we’re all in this together”, so the conversation would be constructive and respectful.
- We used personal introduction posts (of ourselves and others) as well as “spotlight” posts about members of the community as a way to encourage empathy, personal connections, and inclusiveness.
- Self-promotion was only allowed on a designated thread, to make sure the group feed is authentic and diverse.
- “It takes a village” — By posting and bringing to awareness information about local initiatives that concerned the greater good and encouraging creative cooperation we wanted to empower social capital and feelings of togetherness.
The possible problems
Even though conducting research through the group turned out to have many advantages, as this was very experimental, we wanted to be as minded as possible to any issues that might arise from collecting data in such a way.
We identified 3 main weaknesses that were inherent to the nature of the community:
- Biases — the fact that group members know that other members of their local community read their posts and comments and the information is not anonymous may sway what they choose to say. We addressed this possible problem by conducting 1 on 1 interviews with users.
- Anecdotal data — while anecdotes can be useful in telling a story or proving a point, we wanted to make sure we didn’t rely solely on those. To tackle this issue, we’ve conducted 2 surveys that had structured questions about the topics we wanted to better understand.
- Open-ended data — to address the possibility that data would be out of focus we used our role as group admins to moderate discussions and raise questions we wanted to be answered.
Conclusions
Despite their weaknesses, and if done right, digital communities are a powerful space for conducting user research. In times when human interactions are held in the digital space more than ever, we should move away from conventional methodologies' comfort zone and use this space in order to learn about users.