Year 1 review
It's been a year! š„³ Here's the "executive summary" of our year 1 review.
TYPE III AUDIO narrates the best writing about AI safety, AI governance, and the long-term future. We also help people find the best things to listen to.
In our first year, we:
- Created 119 hours of human narration and ~500 hours of AI narration.
These narrations received ~26,000 hours of listening time, and will receive a further ~28,000 hours by October 2024 (without further work from us).
- Developed three assets:
a. AI narration service - easy way to create AI narrations, embed them on a website & release them on podcast feeds.
b. TYPE III LIBRARY - website to help people find the best things to listen to.
c. TYPE III PLAYER - embed audio on websites while collecting detailed listening data.
Our clients included: 80,000 Hours, LessWrong, EA Forum, AI Safety Fundamentals, Center for AI Safety, Joe Carlsmith (Power-seeking AI report), Utilitarianism.net and Jeffrey Ding (ChinAI newsletter). They were delighted with our service.
Our average team size was 2.4 FTE, so each hour of staff time created ~5 hours of listening time this year.
We were funded by a $300K grant from the Effective Altruism Infrastructure Fund, plus ~$50K of revenue from paying clients. Our total project costs were ~$320K. So: if you write the value of assets (2a), (2b) and (2c) down to $0, then our total cost per listening hour was ~$12, which is within the $1-20 range that we were aiming for. Many projects within the āeffective altruismā community are willing to purchase additional engagement time at a range of $1-10, and at a range of $10-20 for the highest-value content, so at the macro-level, our first year looks pretty good.
Making narrations is only worth doing if people listen to them. Distribution (āgetting narrations to listenersā) emerged as our key bottleneck during the year.
We helped several clients create and promote new podcasts, but our main solution was to work with clients that have large existing audiences. Long term, we think the upside vision for TYPE III AUDIO involves creating our own discovery platform, and building our own audience. Thatās the vision for TYPE III LIBRARY.
A key question, for assessing our strategy and cost-effectiveness this year, is how you value the assets we developed:
a. Our AI narration system looks set to generate 20,000-200,000 listening hours over the next 2 years, with as little as $10-30K of further investment during that period. It cost roughly $150K to develop and run this year, so if we assume a 3-year cost of $200K, then it looks like this programme will land in the range of $1-10 per listening hour.
b. The value of the TYPE III LIBRARY is very uncertain at this stage. Weāve made an interesting alpha version and run several user tests, but the project is early, and I donāt yet see strong evidence of product-market fit.
c. Our audio player enables our narration services, and will be a key user acquisition channel for the TYPE III LIBRARY. It can also be used by anyone with audio to embed, e.g. podcasters. For now we have only offered this to our clients and a few others (e.g. Hear This Idea), but it could become a service in its own right.
The value of these assets, along with some alternative strategies we might have pursued, are discussed in the main review.
Overall I am pleased with how things went this year. We delighted our clients, served thousands of listeners, and built an AI narration service which now narrates >500 posts per month for EA Forum and LessWrong. In September, our narrations were listened to for roughly 4000 hours (or 130 hours per day).
Plan for year 2
Our AI narration service will continue operating in some form throughout 2024, serving clients like EA Forum, LessWrong and 80,000 Hours.
We will wind down our human narration service in 2024 Q4. Perrin Walker will continue providing narration services independently of TYPE III.
I am unsure whether to invest more in TYPE III LIBRARY. The promise of the project is quite uncertain, and there are many other projects, unrelated to TYPE III AUDIO, that the team and I could work on in 2024.
In 2023Q4, I will spend ā of my time on TYPE III AUDIO, and ā of my time exploring other opportunities. In the winter, I will review my options, and decide whether to make TYPE III AUDIO my main focus for 2024.
Our full year 1 review includes a more detailed impact assessment and a reflection on key lessons from the year. You can find it here.
I'm always keen to hear thoughts or answer questions about our workāwrite to [email protected].
Appendix 1. Narration outputs
We also made AI narrations and podcasts for:
And we made human narrations for Joe Carlsmith (Power-seeking AI report), effectivealtruism.org (Intro to Effective Altruism), and Effektiver Altruismus (several German translations).
We also created ~50 hours of additional AI narrations for the TYPE III LIBRARY, e.g. Paul Christianoās best writing on AI.
Appendix 2. Narration hours and listening hours by client and service
80,000 Hours accounts for ~55% of total listening hours, mainly because they released narrations on their popular website and podcast. LessWrong is a further 20%, for similar reasons. The big surprise here is the AI Safety Fundamentals course, which has proved much more popular than expected. This is in part because they have large, highly engaged cohorts, and in part because the podcast ranks #1 on Apple Podcasts and Spotify for āAI Safetyā.
Our AI narration service generated only ~10% of listening hours. This is mainly because:
(a) Human narration service began releasing narrations in October 2022, while the AI service began operating in June 2023.
(b) Human narrations were released on the 80,000 Hours podcast.
The impact of a listening hour varies by content and audience type. My guess is that the AI Safety Fundamentals is the most valuable content per listening hour.
See also: How we estimate listening times.
Appendix 3. Our team
Total labour input for the year was 4400 hours, or 2.4 FTE.
Our core team members are listed below.
Exec team:
- Peter Hartree (CEO)
Narration team:
- Perrin Walker (Senior narrator)
Proof-listening team:
- Katy Moore (Head of QA)
- Marcus Sommers (Proof-listener)
Tech team:
- Igor Stajic (Senior Developer)
- Srdjan Stajic (Senior Developer)
We also worked with a further 10 short-term contractors, of which six were people we trialled for core-team roles.
Appendix 4. Financials
We spent roughly $320K during the year, $290K of which was salary payments.
We made salary payments to 16 freelance contractors. Eight invoiced less than $10K; four invoiced $10K-$25; and four invoiced >$40K (Perrin, Igor, Srdjan and myself).
Igor and Srdjan, our developers, were paid $90 / hour. Other staff were paid $30-50 / hour.
CEO pay & benefits
My effective hourly rate was roughly $30 / hour (not counting overtime, which was unpaid). I was paid ~$48K during the year.
TYPE III AUDIO also covered ~$10K of my expenses related to business travel, equipment and office rental.
Our team retreat cost ~$10K, including travel expenses for our all-remote team (Australia, France, Iceland, Serbia).
My total salary was less than half the amount I suggested in our EA Funds grant application. I made this change because I wanted to invest more in our AI narration system and the TYPE III LIBRARY, and wasn't able to quickly fundraise to enable this.
The AI narration system is an asset I now own, which will generate some future income for me. I doubt that it will generate enough income to significantly change the effective hourly compensation I received this year.
Appendix 5. Client testimonials
TYPE III AUDIO narrated more than 75 articles and academic papers for our AI Safety Fundamentals courses. We couldnāt be happier with the ease of working with them, the quality of Perrin's narrations, and the speed with which the team delivers new narrations as we update our curriculum.
āJamie Bernardi of Blue Dot Impact
We asked the Type III Audio team to produce AI-narrated audio of our webpages at utilitarianism.net. They were great to work with: reliable, communicative, and pro-active about suggesting possible improvements to our planned implementation. I especially appreciated that they identified pre-existing public domain narrations of our content, where available, and offered to integrate these at no additional cost. Highly recommended!
āRichard Y. Chappell, co-editor of utilitarianism.net.
TYPE III AUDIO has produced high-quality narrations of our articles that have increased their reach for comparatively little effort. Working with them has been incredibly smooth Ā for us, and they keep coming up with attractive additions to their services weāre interested in too!
āArden Koehler, 80,000 Hours
TYPE III AUDIOās efficiency and professionalism make them wonderful partners. By partnering with them, weāve been able to test audio narration on the EA Forum quickly and develop a strategy to boost engagement further.
āSharang Phadke, Effective Altruism Forum
TYPE III AUDIO created a full, 3+ hour, high-quality narration of a report I wrote about power-seeking AI, and theyāve been helping improve the quality of the narrations of other things Iāve written as well. In all of this, theyāve been enthusiastic, detail-oriented, communicative, serious about the quality of their product, and keen to be as helpful as possible. Iāve been very satisfied with the narrations, and itās been great working with them more broadly.
āJoseph Carlsmith, Open Philanthropy
2023-10-03 0700 BST: I updated this post to correct the figure for human narration output. The correct figure is 119 hours; I previously stated it as 140 hours due to a miscalculation.