Description
The Optimism Foundation is working with Open-Source Observer (OSO) to determine impact metrics to measure, evaluate, and determine how much public goods projects should receive from RetroFunding. You can see this announcement for round 4, further explanation, and the a preview of details for round 7 for information about this. Open-Source Observer has posted a data challenge and request for impact metrics, which you can see below.
This document contains information about submitting Respect as an impact metric to Open Source Observer (OSO), which has partnered with the Optimism Foundation and will likely be deciding about 20 metrics that badgeholders will use to evaluate projects on RetroFunding Round 4. This is really important for Optimism Fractal because it gives us the opportunity to integrate Respect, Respect Votes, and Respect Trees (or Cignals) directly into the design of Retro Funding. There is currently a data challenge open to submit impact metrics and there is a limited amount of time to apply, so it may be worth applying as soon as possible. In addition to the data challenge, there is also a request for impact metrics and other requested contributions related to
This task/project is a work in progress and includes subtasks such as joining the Kariba Data Collective, asking about displaying Respect and Respect Trees in SQL, and information that OSO is seeking. Additionally, it mentions exploring ways of providing helpful resources for impact scientists.
Table of Contents
- Description
- To Do
- Review Announcement from Optimism Foundation
- Listen to Citizen’s House Call
- Review Citizen’s House Call Recap
- Experiment: Metrics-based Evaluation
- Related Tasks
- Respond to OSO Data Challenge 01
- Apply for Kariba Data Collective
- Respond to Request for Impact Metrics
- Share Personal Introduction in the lobby
- Submit Impact Metrics
- Requested Metrics
- Onchain Reputation
- Explore Ways of Contributing
- Building a Network of Impact Scientists
- Relationship to Optimism Fractal
- Related
- Integrate Respect Votes as an Impact Metric
- Create Auxiliary Tokens for the Optimism Collective
- Decide the best way to brand and explain the concept of voting with Respect
- Inspiration
To Do
Create seasonal structure for Optimism Fractal and - I think we probably want to apply for the both the Data Challenge 01 and the Request to Submit Impact Metrics, though I’m not sure what role is for the Request to Submit Impact Metrics in comparison to the Data Challenge 01
- The Data Challenge 01 was retweeted by Optimism and Jonas and it specifically states that they’re seeking impact metrics for Optimism Retro Funding, so this seems most important for us.
- The Request to Submit Impact Metrics also seems very aligned and helpful with our goals, Maybe we should just apply for this first then we can look into the Request to Submit Impact Metrics after that
Optimism Fractal Projects so that more people can see the synced notesReview Announcement from Optimism Foundation
Listen to Citizen’s House Call
Joint House #39: April 23th, 2024
Recording: April 23rd joint house call recording.mp4 - Google Drive
Slides: April 23rd, joint house call slides
- I think jonas said that he’d share a link about this but didn’t, so we could following up with him after listening. i asked two questions at around 40 and 55 minutes
- Actually i think the link he was talking about is what he tweeted in relation to OSO Data Challenge 01, which is below
Review Citizen’s House Call Recap
Retro Funding 4: Onchain Builders 5 was announced:
- The round sizing will be of 10M OP, the impact generated between October 2023 - June 2024 will be rewarded.
- One of the topics on the forums post on what impact will be rewarded is “Interactions from Optimism users with high trust scores” this refers to onchain reputation. Data sets are being currently collected and the aim is to try to find an index of reputation scores or experiment with different reputation data to determine this.
- Anybody will be free to create and propose impact metrics, a survey workshop will be coming up with Badegeholders to get input on what type of metrics the voting process should feature.
- If you want to know better how the design for the voting interface looks like, there is a calendy link to schedule a call for user testing in the badgeholder discord/telegram channel.
Experiment: Metrics-based Evaluation
- Thesis: By leveraging quantitive metrics, citizens are able to more accurately express their preferences for the types of impact they want to reward, as well as make more accurate judgements of the impact delivered by individual projects.
In Retro Funding 3, badgeholders voiced that non-standardized impact metrics added difficulty to the voting process and expressed desire to directly vote based on metrics. Examples of this approach include Buidl Guidl’s impact calculator and OpenSource Observers impact vectors. Metrics-based evaluation will be used to reward downstream impact, as this category can benefit from a quantitive approach to measuring impact.
Related Tasks
Respond to OSO Data Challenge 01
Apply for Kariba Data Collective
Respond to Request for Impact Metrics
Share Personal Introduction in the lobby
Submit Impact Metrics
- What is Colab or Jupyter Notebook?
- Maybe ask this in the discord
Requested Metrics
They’re looking for Impact Metrics in the following categories where we can help:
Onchain Reputation
Metrics that attempt to differentiate onchain users based on behavior and trust scores.
This is a hot topic right now, with a number of projects attempting to create reputation systems for onchain users. We're integrating with many of the leading projects and bringing the data they generate into the OSO data warehouse. From there, there are all sorts of directions you can take the analysis!
The Onchain Activity metric is a good starting point for new analysts. Have a look at our working definitions for onchain users and help us create some better ones. The Network Loyalty is a spicy metric that will get even spicier as we add more networks to the OSO dataset!
Impact Metric | Description | Comments |
Onchain Activity | How active is the user onchain? | Good first issue |
Network Loyalty | How much of a user's activity is on a single network? | Good first issue |
User Segment | What kind of dapps is the user most interested in? | Good first issue |
ENS History | Does the user have a rich profile on ENS? | Requires indexing of ENS data |
Gitcoin Passport | Does the user have a high Gitcoin passport score? | Requires indexing of Gitcoin Passport data |
When working with these metrics, because they deal with LOTS of transactions, it may be helpful to grab a bunch of data in a single query request and then do further analysis in a notebook. For example, this query will grab each user's transactions with each project on each network in the last 90 days:
-- Get the number of txns by user / project / network / day
SELECT
project_id,
from_name AS user_address,
to_namespace AS network,
time,
SUM(amount) AS txns
FROM `oso.int_events_to_project`
WHERE time >= DATE_SUB(CURRENT_TIMESTAMP(), INTERVAL 90 DAY)
AND event_type = 'CONTRACT_INVOCATION_DAILY_COUNT'
GROUP BY 1,2,3,4
tip: Check out our starter notebooks for impact metrics including onchain activity and network loyalty.
Explore Ways of Contributing
Here is another section of the OSO website that may be helpful to explore. I don’t think we need to do anything other than submit Impact Metrics for now, but other ways may be helpful to consider at some point.
Building a Network of Impact Scientists
- There’s nothing we need to do about this now, but it could be helpful to review
- See related thoughts at RegenLearnings.xyz and
Plan Optimism Fractal Season 3
Relationship to Optimism Fractal
Related
Integrate Respect Votes as an Impact Metric
Review Methods for Voting with Respect in Notion and Snapshot via the Notion API
Create Auxiliary Tokens for the Optimism Collective
Create OPC Respect Token for Optimism Collective
Decide the best way to brand and explain the concept of voting with Respect
Ideas:
- Respect Voting
- Respect Votes
- Voting with Respect
- Respectful Votes
- Cignals
- Respect Trees