Description
A research project to develop a framework for Community Health, create a PoC data collection tool, implement the tool in MolochDAO/DAOHause and collect interaction and pulse-survey data to validate the framework and provide the insights to the community.
Manifesto/Vision
This is a research project of RnDAO, who’s mission is to Empower Humane Collaboration. Specifically, the Community Health research project seeks to answer the question of how to conceive and measure Community Health to provide non-financial metrics to evaluate and improve DAOs resilience and effectiveness.
Problem
Like many others, Moloch DAO depends on the health of its community and its vibez—yet understanding and measuring these factors (what we call Community Health) is challenging.
Today, Web3 communities are left to rely on infrequent transactions data (on-chain records), while data from the significantly more frequent social interactions is limited to basic indicators of Discord or Discourse, Web2 oriented bots like Statsbot, or a patchwork of “homemade” surveys to fill in the gaps. These solutions are time-consuming for community managers and contributors to use, and the results, hampered by poor indicators and/or poor sampling, are unreliable.
Perhaps even direr, the lack of real-time analytics leaves community leaders and members without established baselines to measure against to understand the impact of community-focused initiatives, monitor shocks to the system, or rapidly gauge the effects of system-wide changes (such as market crashes).
Solution
RnDAO will develop a framework based on scientific research to conceptualise and assess Community Health. And prototype a way to gather and compute the data that requires minimal administrative effort from DAO community managers and minimal disruption to DAO members. We believe this approach ensures more representative insights and paves the way for broad adoption of real-time analytics and benchmarking across DAOs.
Beyond Traditional Surveys
This research proposal focuses on the use of two critical techniques as a starting point:
Organizational Network Analysis (ONA) is a structured way to visualize how communications, information, and value creation occur through an organization based on interaction graphs. ONA has been shown to provide a wide range of insights to improve contributor retention, avoid member burnout, predict team performance and community resilience, identify key contributors, enable decentralization, and improve coordination. Although relatively new, ONA is gaining in popularity over traditional survey tools.
Pulse Surveys are frequent and automated micro-surveys that provide qualitative and quantitative insights. In traditional organizations, they have been shown to increase employee response rate and employee engagement with related initiatives. They’re also used as a tool for culture design and implementing culture change. Lastly, Pulse Surveys significantly reduce admin work for community managers and related roles.
Although our initial focus is Community Health metrics, this sets the foundation for further applications. The combination of ONA and pulse surveys offers unprecedented actionable insights in real-time. Some of the potential applications and insights for DAOs are:
- Generate baseline metrics for Community Health / vibez to quantify and better understand the impact that a specific event is having on a community and/or sub-groups within the community
- Predict which contributors are likely to leave the DAO and take preventive action (without breaching privacy)
- Build funnels to track member onboarding and identify areas for improvement
- Identify measurements of decentralization to serve as KPIs or Insights metrics
- Monitor specific topics like contributor wellbeing, alignment, community experience, etc. in near real-time
- Attract talent and investment with objective Community Health metrics instead of proxy metrics like member count or proposal count, or financial metrics such as TVL
- Help new contributors find context-rich mentors outside of the existing pool of well-known but time-poor candidates
In addition to the initial research on Community Health, the potential applications mentioned above (and others to be found) can enable more effective and targeted efforts to build healthier DAO communities.
This research also helps reduce the tooling gap in DAOs compared to the employee and stakeholder experience at traditional corporations.
Product
This project includes two components:
- the research and synthesis of findings into a conceptual framework for assessing Community Health
- a data collection tool that enables passive data collection on Discord and a pulse survey functionality
Validation
We interviewed 15 community leaders across DAOs and hosted a workshop with 25 community managers to identify painpoints and desires.
Progress
A team of 7 contributors has been assembled and has been working in the project for 7 weeks. We also received grants from Aragon and Polygon, allowing us to advance the conceptual framework research and data collection tool to a PoC stage.
Differentiation
Although research has been carried out on communities and virtual communities for years, no research we could find addresses DAO Community Health. More general research (not focused on Community) has developed long-form surveys (e.g. TalenDAO’s DAO Health and Bankless’ Contributor Sentiment) but have focused on providing ecosystem insights as opposed to specific, actionable insights within a community.
Other tools exist for basic community analytics (Statbot, Discord stats, Orbit). However none are DAO focused, as such they fail to account for the diversity of stakeholders (contributors, investors, users all mixed), don’t include indicators of resilience, have no mechanism to collect qualitative data. To fill the gap, community managers create custom made surveys (with pitfalls around validity and sampling) and engage data science teams to create custom dashboard (that still remain limited to basic indicators with KPI application but no predictive capacity).
Team Leads
(the forum didn’t allow me to post enough links so let me know for any socials you might want to check…)
Katerinabc
Ph.D. in Team Dynamics using Social Network Analysis, Teaching Collaboration, and Organizational Performance at Northwestern University (since 2016).
Co-organized Learning in Networks sessions at the International Conference of Social Network Analysis (2018 - 2020), and previously advised a people analytics company on social network metrics.
Thegadget.eth
Software Engineer. Previously, Product Manager at Neolyze (Business Intelligence Dashboard for Instagram).
Danielo
Previously, Head of Governance at Aragon, 8 years experience in Organization Design consulting (clients include Google, BCG, Daymler, The UN, and multiple startups), and visiting lecturer at Oxford University.
See additional team members here:
sobol.io/d/rndao/structure?view=circles
Grant Request
$10K
(we’re applying for this grant and for a few others to build a coalition and fully fund this project, thus dividing the costs and providing maximum value to participating members. The total budget is $90k, and we’ve already received $35k from Aragon and Polygon).
Ideally (optional): access to MolochDAO/DAOHause community to implement the data collection tool in Discord, validate the framework and provide insights to the community so it can continue to improve.
Additional Resources
Examples of previous Research by RnDAO:
- rndao.mirror.xyz/1zGqbsh1YZNi3I9yvtk_2VMcpyg_dvHF1GlZ_LAO3p4
- rndao.mirror.xyz/gNwffffROGdAp1tEBhkRPEL8OAQqvUOlV5HuM9VBaoM
- rndao.mirror.xyz/Qn0Y71EYRUB-6Kn0jH47baWulXoIe-dmTYIVuJeEOt8
What’s next for Community Health
This conceptual framework and Community Health measurements are just the beginning. Building on this foundation, we can develop a dashboard to visualize the data in real-time, and develop functionalities that leverage the network analysis to e.g.:
- automatically request feedback to those who disengaged in a privacy-preserving manner
- prompt those working closely together to provide feedback to one another on a recurring basis
- identify the most helpful members of the community by combining network and semantic analysis
- etc.
Contact for additional questions:
Discord: danielo#2815
Telegram: @mrjackalop