Grant Proposal: PepeScore

Community Grant Applications
#1

Grant ID: 8686b0036

Summary:

PepeScore is a forum integration tool, that introduces new badges and streak mechanics for stimulating and highlighting contributing users in Graph Advocates DAO Forum.

  1. Badges and Streaks use cases:

Over the years, gamification mechanisms in online communities in the forum badges and streaks have proven to be powerful tools to boost contributions from users across various online collaborative, learning, and Q&A communities. The most notable and researched are Stack Overflow (Badges) and DuoLingo.

Similar to the Q&A and learning platforms, DAOs can benefit from deploying such gamification elements to encourage active participation and contribution from their members. The stimulus provided by badges and streaks is recognized as helpful tools for fostering increased engagement, driving members to contribute more deeply within the DAO.

  1. Existing solutions:

Badges in DAOs have the purpose of stimulating activity, and be used by members to distinguish between members of Governance teams, the “general population” or users with years of experience behind their backs.

DAOs today usually operate on built-in badge systems (Discourse) with regular and easy-to-obtain rewards to provide users with a constant flow of positive feedback. Such systems are effective for mass user onboarding and are a good fit for communities with a broader membership and a low required level of involvement from an individual member.

Meanwhile, DAO platforms with their complex relationships, much more limited user base, and higher expected involvement, require a more intricate and precise, hierarchical badge system. What we’ve described as a desired system is in its essence much closer to that employed by StackOverflow and alike, which can serve as our model further.

Thus, a more structured hierarchical badge system is a desired solution for DAO Forums to provide motivation for fostering growth and activity in the DAO communities.

  1. PepeScore’s solution

PepeScore proposes a tier-based badge format (ex: Bronze/Silver/Gold/Platinum) with an implemented streak system to create continuous motivated activity and form a habit of contribution to the DAO Forum.

Coupling basic quantifiable metrics with sophisticated ML/UI-based analytics of users’ contributions to offer a more precise leveling within the badge system, PepeScore provides a clearer hierarchy and value for the badge.

The dynamic aspect of Streaks ensures that users stay actively engaged and that badges remain relevant in reflecting real-time user contributions.

Badge and Streak holders can also be grouped into valued-contributor communities, facilitating stronger lateral communication between members of the DAO, which can then be leveraged for various purposes including marketing, community leaders discovery, grantor or delegate roles.

On platforms like the DAO community forums, where high-stake decisions are routine, badges also serve a critical role in curbing inadvertent activity. This protection comes in the form of spam filtering and regular user vetting, ensuring that the quality of forum engagements is maintained.

Possible criteria for upholding a streak could be: Writing a proposal review once a week, providing a detailed response to questions in General, etc.

Result: a more active user base with a habit of contribution, incentivized by the benefits of the streaks and badges.

It is important to remember: The success of any behavioral design and gamification intervention is very much dependent on the context and audience, so to test ideas and mitigate risks an iterative design process is crucial.

The badge & streak system will have:

  • Clear Goal:
    Streak is earned and held by weekly participation in discussions on the platform. The target could be for members to create or contribute to a certain number of discussions each week.
  • Tangible benefits. Example: Upholding a streak provides users with exclusive access to the “Streak society” of the Forum. This incentive will galvanize users to actively maintain their streaks.
  • Triggers and Reminders: PepeScore implies having reminders sent out to users of the platform to encourage users to uphold their streaks.
  • Two-fold feedback: The “completion feedback” will notify users whether their weekly actions are contributing to their streak. The “progress feedback” would visibly illustrate individuals’ ongoing journey towards completing and maintaining their streak on a weekly basis.
  • Value detection: The contributions should be monitored to exclude low-quality responses (AI-generated or general filler).

I have read and agree to the privacy policy:
true

Describe what your project does or is aiming to do in 50 characters or less:
Badges and Streaks for Graph Advocates DAO Forum

How far along are you?:
Our system is composed of the backend and a channel of communication with the forum (API). Currently, we have refined LLM processing methods in place and a well-trained text recognition system that uses data collected from over 20 DAOs for optimal performance. Furthermore, we have established source feeds that maintain the model’s relevance and accuracy.

Why did you pick this idea to work on? Do you have domain expertise in this area? How do you know people need what you’re making?:
We chose to work on PepeScore because we’re genuinely invested in the Graph community, and want to see it thrive.
Studying numerous DAOs, we observed that experienced participants often experience ‘burnout’, gradually visiting forums less frequently. This has motivated us to focus on retention as an important area for system development.
Given the successful implementation of similar gamification strategies in other platforms, and drawing from our expertise in data scraping from DAO and AI/Machine Learning analysis, we recognized how potent an optimized badge system can be in enhancing user interaction. PepeScore is an opportunity to leverage this, using our skills to contribute to a space we value.

What’s new about what you’re making? What substitutes do people resort to because it doesn’t exist yet (or they don’t know about it)?:
PepeScore stands out by offering a tier-based badge system alongside a streak mechanic specifically tailored for DAOs - a first in this space. Most DAOs utilize built-in badge systems, like those in Discourse, but these lack the customization needed to fully engage a DAO membership and don’t offer streak mechanics. Despite having these built-in systems, the engagement and ongoing contribution from members often lag due to a lack of incentives for continuous participation. PepeScore fills this gap by integrating a more robust and continuous engagement mechanic suited to the specific needs of DAOs.

What’s your vision for how your project will impact The Graph and/or web3?:
PepeScore is a tool for empowering and strengthening the Graph community. By introducing tailored badges and streak mechanics, PepeScore will foster increased and sustained participation. As members get more involved, we anticipate a boost in knowledge sharing, elevating understanding and innovation around web3. These regular and meaningful engagements can drive subgraph usage, attract more developers, stimulate more vibrant events, and overall, contribute to the expansion of The Graph and the larger web3 ecosystem.

How will you measure success?:
The success of the PepeScore project can be effectively measured through a combination of qualitative and quantitative metrics. To begin with, we will measure the level of user engagement, specifically looking at the increase in active participation and contributions in the DAO forums over a specified period. For a more granular review, we’ll track the rate of badge earning and streak maintenance among users. On a quantitative note, we will utilize the AI/ML analysis to evaluate the quality and relevance of these contributions, ensuring that they are not just frequent but also valuable.

What milestones will you achieve in the coming weeks/months, with or without the grant?:

  1. Backend Development:
  • Implement a custom aggregation tool to collect user activity.
  • Establish a balanced scoring mechanism for accurate badge distribution and streak continuation.
  • Develop and train Llm scoring circuit for rating user activity.
  • Develop an API-access client to facilitate the transfer of scored data back to the forum/frontend
  • Private Access page for controlling the system
  1. Integration Elements:
  • Design and implement a visually appealing, engaging, and intuitive badge and streak system, complete with notifications.
  • Public community page in Notion

Preferred Payment Structure: $5,000 will be an upfront payment required for the initiation of the project, with the remaining $5,000 to be remitted upon proof of complete and functional system execution. This milestone assures both parties of a mutual commitment to the successful completion of the project.)

Please select who you think will benefit from your project?:
Other

How much funding are you looking for? (USD):
10000

Provide a breakdown of your requested funding:

  1. Backend Development:
  • Implement a custom aggregation tool to collect user activity.
  • Establish a balanced scoring mechanism for accurate badge distribution and streak continuation.
  • Develop and train Llm scoring circuit for rating user activity.
  • Develop an API-access client to facilitate the transfer of scored data back to the forum/frontend
  • Private Access page for controlling the system
  1. Integration Elements:
  • Design and implement a visually appealing, engaging, and intuitive badge and streak system, complete with notifications.
  • Public community page in Notion

Preferred Payment Structure: $5,000 will be an upfront payment required for the initiation of the project, with the remaining $5,000 to be remitted upon proof of complete and functional system execution. This milestone assures both parties of a mutual commitment to the successful completion of the project.)

Have you applied for (or already received) funding from somewhere else?:
No

Why are you interested in contributing to The Graph and/or web3?:
We are interested in contributing to The Graph and web3 due to their broad impacts on data decentralization and transparency. Engaging with Graph and its community is an inspiring opportunity to influence future digital norms.

Are you applying as a team or individual?:
Team

What type of grant are you pursuing?:
Project (one-off)

Please select the category your project best fits into:
Dashboards & Analytics




Primary Community Poll

Do you fundamentally support this grant application, knowing that the final scope and grant amount may differ from the requested proposal?
  • Yes
  • No

0 voters




Grant Committee Vote

Do you recommend this grant to the DAO?
  • Yes
  • No

0 voters

3 Likes
#2

Hello! I have some questions regarding your proposal:

  • How do you plan to ensure that the gamification elements do not over-incentivize quantity over quality in user contributions?
  • Could you clarify the criteria for the different badge tiers and how they will be implemented in the system?
  • What mechanisms will be in place to prevent the exploitation of the badge and streak system (e.g., gaming the system)?
  • How will the success of the PepeScore project be tracked in terms of user retention and long-term engagement?
  • What are the expected tangible benefits for users who are part of the “Streak society,” and how do you plan to maintain its exclusivity?
  • What are the specific qualitative metrics that will be used to measure the project’s success?
  • Is there a plan to scale PepeScore for other DAOs or platforms, or is it exclusive to the Graph Advocates DAO Forum?
  • Trivial: Regarding the badges, will there be any customization options available for users, such as colour or design personalization?

Thanks!

4 Likes
#3

Great questions @doxe01.eth - looking forward to the responses!

2 Likes
#4

These are all indeed great and fair questions! We’d also like to ask the moderators to include responses to these in the FAQ section of our proposal.

How do you plan to ensure that the gamification elements do not over-incentivize quantity over quality in user contributions?

This obviously is one of the most anticipated challenges for PepeScore. Our solution - a system of evaluation for users’ contributions, that goes beyond merely quantitative metrics. With a track record of managing and analyzing substantial datasets collected from various DAOs, we are confident that a robust system for evaluation is possible with the use of LLM.

Our team is experienced at utilizing LLM for sophisticated analysis of data derived from multiple DAOs, including assessing the significance and sentiment of content. A testament to our expertise is the DAOTerminal, which provides an analytical overview of DAO activities.

Could you clarify the criteria for the different badge tiers and how they will be implemented in the system?

We’re implementing a badge system based on percentile rankings to introduce badge scarcity and drive ambition among users. Tiers are set by looking at the current top performers’ contributions— for example, the top 10/20/30 percentile. This creates a clear target for the rest of the community to aim for, encouraging active participation and progression.

What mechanisms will be in place to prevent the exploitation of the badge and streak system (e.g., gaming the system)?

LLMs are used to analyze contributions for quality and relevance regarding the proposal. We also propose surveying reviewers with control questions as a part of the validation process, where proposal authors can query the reviewer. If the reviewer (contributor) fails to respond appropriately, this could indicate suspicious activity. We will also establish a feedback loop for continuous system improvement and to develop various authenticity checks to deter gaming the system.

How will the success of the PepeScore project be tracked in terms of user retention and long-term engagement?

Proposed metrics:

  • tracking the migration of users from one cohort (badge tier) to another,
  • indicating improved engagement and retention.
  • overall level of participation
  • median number of valuable contributions and views per proposal, ensuring that interactions are meaningful and contribute to the forum’s health.

What are the expected tangible benefits for users who are part of the “Streak society,” and how do you plan to maintain its exclusivity?

Members of the ‘Streak Society’ will enjoy benefits such as streamlined proposal submission processes, enabling more proactive community participation. We’re also proposing to test a financial compensation model for active members — for instance, providing an equivalent of a $100 in GRT reward for valuable contributions towards 7-10 proposals a month. The ‘Streak Society’ thus serves as a clear path to enrich the community, offering a well-defined narrative for engagement. Additionally, it can foster interpersonal connections, serving as an icebreaker, while also providing reputation-based benefits. It acts as a social tool for validating community commitment and contribution.

What are the specific qualitative metrics that will be used to measure the project’s success?

Again, as we’ve stated in our answer to the 4th question, the proposed metrics are:

  • tracking the migration of users from one cohort (badge tier) to another,
  • indicating improved engagement and retention.
  • overall level of participation
  • median number of valuable contributions and views per proposal, ensuring that interactions are meaningful and contribute to the forum’s health.

Is there a plan to scale PepeScore for other DAOs or platforms, or is it exclusive to the Graph Advocates DAO Forum?

For now, we’re sticking with Graph Advocates as it’s our area of expertise. Though PepeScore is currently tailored for this community, the framework is adaptable for future expansion to other DAOs.

Trivial: Regarding the badges, will there be any customization options available for users, such as colour or design personalization?

Of course! We’re firmly for a very aesthetically pleasing and variable experience. It will be fun. We’ll show you the draft shortly :wink:

#5

I believe that before delving into something as sophisticated, it is essential to investigate the real problems faced by DAO advocates and identify the gaps in acquiring and retaining an active community. It’s important to understand the pains; for example, in the Arab community, there is an advocate who creates tweets but has not yet generated engagement with users interested in learning how to create subgraphs. In my case in Peru, Latin America, I am still in search of users who want to be part of my community and are eager to learn. To achieve this, I need to identify the developer’s journey.

1 Like
#6

Thank you for your input.

This is indeed another important goal for the Graph Advocates community.

PepeScore is designed to complement outreach efforts by rewarding ongoing participation, creating stickiness for newcomers, and highlighting a clearer path to obtaining reputational goals within the Graph Advocates community.

We’re open to adapting our tool to fit the specific needs of different communities within The Graph network. Your feedback is valuable as we fine-tune PepeScore to better serve The Graph’s diverse global community.

#7

Great to see this wonderful proposal and thanks for taking this to the forum, I have few questions after checked all your info and design here.

1: how this benefit fot The Graph ecosystem, is the source code will be open source or?

2: How you will spend the $10000 and is the a detailed milestone or prototype?

3: What’s your team consists of, and once this tools implemented, how big impact it will bring to the whole gov process.

#8

Hi, Jonathan! Thanks a lot for your feedback.

  1. Regarding your first question, there is an articulated request for community-building and strengthening within the Graph Advocates DAO. PepeScore is designed to address this by increasing stickiness of DAO Governance, gamifying users’ experience to encourage consistent and valuable contributions, and perhaps even identifying more community leaders in the process. As for the transparency of PepeScore, we plan to create a public Notion page for easy access by community. All codebase will be open source as well.

  2. To answer your second question, as mentioned in our grant application, the funds will be allocated for backend development, which includes an aggregator, an LLM-based scoring algorithm, and a client for returning the scored data to the forum. The visual element is also crucial, as effective gamification also relies heavily well-balanced and appealing design.
    Furthermore, as a reminder, we have stated that we are willing to receive the second half of the grant retroactively, after delivering a functioning product.

  3. Our exceptional team comprises the following members:
    Founder: Nikita K: served in and led marketing teams for crypto Launchpads on Polygon/BNB/Avax networks from 2020.
    Dmitri: Python and C++. mao13132 ¡ GitHub.
    Dmitri: Python, JavaScript, and TypeScript. Solutions in database organization. redboo ¡ GitHub

While it is difficult to forecast the exact impact, we have well-founded expectations to:

  • increase a median number of valuable comments and reviews per proposal
  • double the number of users, eligible for streaks (double the number of regular monthly contributors)
#9

I voted against this proposal because I strongly believe that the Graph community is not about creating a hierarchy but rather finding your own niche, contributing according to your own capacity by doing what you’re best at (We always say: not quantity but quality, don’t we?). The impact of hierarchy on communities can be positive too, but I will admit that I can see more disadvantages of this approach:

  • Hierarchy can provide a sense of achievement and recognition for active and engaged community members but it may also create feelings of exclusion for those who are not at the top levels, potentially discouraging participation, too.

  • Gamification elements, such as badges and streaks, are proven motivators in online communities. They enhance user engagement, create a competitive spirit, establish a sense of accomplishment, but they also lead to focusing on earning badges rather than contributing meaningfully.

  • Hierarchy can contribute to a more organized and structured community where roles and expertise levels are clearly defined - agreed! This enables smoother collaboration and communication. On the other hand, it may stifle creativity and discourage newcomers from participating, as they might perceive a high barrier to entry.

  • Hierarchy can lead to the exclusion of diverse voices and perspectives. In our community, we want everyone to feel valued regardless of their badge or streak level.

But I appreciate that you expressed your willingness to adapt based on community feedback and overall it is an interesting proposal. I believe it’s important that systems like this are implemented with flexibility to continuously pulse-check the evolving needs and preferences of the community and I’m sure there are communities out there where systems like this can be used successfully.

1 Like
#10

Hi, Julianna. Thanks for the feedback! Now to address your points:

  1. Don’t be discouraged by the use of the term Hierarchy - in fact it is already present in the current badge system Graph Advocates. Also, having gone though multiple researches on badge systems across various platforms, we haven’t bumped into any specifically significant evidence of these systems discouraging users from participation

We think that the best way to address these concerns is to do a trial run of the system.

  1. This is a great point, and it’s in fact why our system requires such a sophisticated approach: the streak system is fine-tuned to only work if the user leaves a contribution/comment which is:
  • relevant to the proposal/theme.
  • is made within the specific timeframe (currently, having done extensive research of Graph Advocates Forum, we’ve established this timeframe as one month)
  • is not fabricated / AI generated

Badges work in a similar way, but on a permanent manner.

We have multiple factors we can take or not take into account for the most sophisticated approach (including: likes, comment spread, sentiment, etc,) all in a point-based system, which is yet TO BE fine-tuned, but is very promising. Community feedback will help development of PepeScore, we are open to changes and tweaks!

  1. Streak system provides stickiness. Leveling up and getting new badges provides sense of accomplishment and motivation. Stream of positive Feedback from both the default discourse platform rewards and PepeScore badges encourage users to contribute more. We feel that the best answer is to arise from the trial run of PepeScore.

  2. We do not see our system as a thread towards diverse perspectives, but rather as motivation to cultivate a various array of contributions from our diverse community. Relevancy is key. Trial run is the best answer to this concerns as we see it from our perspective.

#11

Greetings @PepeScore!

Please check your email! Since there is a letter to schedule a presentation so that we can move forward with your grant proposal.

Thank you!

1 Like
#12

Thank you for the answers!

To add to the first point - The Graph AdvocatesDAO doesn’t have any hierarchy system, we have advocate roles and they are equal. But if you say we have, why do you think that we need another one? If you mean the discord or discourse badges, then yes they are provided by the tools we’re using but there is no benefits attached to them from our side.

If you say that you haven’t “bumped” into problem with discouraging participation, does it also mean that you measured participation? If so, can you share any statistics showing how the participation rate grew after launching your project?

#13

Looking forward to your presentation @PepeScore .

While I think what you’ve built is impressive and could be a useful tool, I’m voting no for now because I think the levels of Advocate inactivity and engagement that we’ve seen won’t be solved by implementing PepeScore. From my perspective those issues are a layer deeper and require more structural changes to some of the DAOs processes and strategy. If we were to solve that as a DAO I think what you’re providing would be of more interest to me.

Thank you for applying and responding with such detailed information, it is much appreciated.

#14

Greetings!

The presentation of the project will be in the public-voice-chat channel on the Graph AdvocatesDAO Discord server on November 28th at 10:00 UTC. Entrance is open and everyone is welcome to come in and ask their questions during the presentation!

Thanks everyone for your attention!

1 Like
#15

Hi there!

We have a force majeure situation - a really unfortunate flight delay, which overlaps the call we have scheduled. Really sorry for the inconvenience, but we would have to request a rescheduling of the call for 18:00 UTC today, or to 10:00 next Tuesday.

Thank you for understanding.

#16

Hi there!

Getting back to you after our recent conversation with a more detailed cost breakdown for PepeScore:

Backend Development
Forum data extraction/ API API/extractors and resources to extract data from forums. $700
LLM processing Training, fine-tuning the AI model, and computational resources for executing daily scans. $1300
LLM operator a human operator, proof-checking and finetuning the LLM system $500
backend and DB storage + API-access client Building the client for API communication with Discourse forum $450
Scoring processing DB-based Jobs: Developing and fine-tuning the algorithm to rate contributions with different variables $500
Access Page Designing and coding the entry public access to of the system. $500
Integration Elements
Design Establishing a visually appealing, intuitive, clear visual element for the system $750
Community Page Development of a community page, incorporating the design principles set out for the project $300

Our previous endeavor, DAOterminal, was an AI-driven platform designed to produce objective, concise summaries of governance events. The process involved analyzing proposals and correlating discussions from DAO forums, executing AI analyzing iterations with diverse parameters, and integrating the results to generate a polished, comprehensive post for the everyday summary.

The archives of the projects are available at Telegram: Contact @DAOTerminalMain

#17

The majority decision of the Grant Committee concludes to not recommend this proposal for DAO funding. Please recognize that this is not a statement on the proposal itself, but rather an aggregated reflection by committee members taking into account a number of factors, such as alignment to The Graph and web3 ecosystem, budget available to support grants and prioritization relative to other proposed grants.

Below you find brief feedback brought up within the committee to not recommend DAO funding that may help you to submit successful grant applications in the future:

For future proposals, we recommend focusing more on the DAO’s immediate needs, simplifying the approach, engaging with the community for feedback. Overall the proposal is interesting, but what we already have in the program things that solve most of the existing issues and, just as was said, the problem is much deeper than that.

Thank you for submitting this proposal, and you are encouraged to stay engaged in the Graph community. You are also welcome to submit new grant applications in the future with proposals that may generate stronger support from the committee.

Sincerely,
Graph AdvocatesDAO

#18

This will help building a stronger, more connected future!

#19