Or: “Why EA biosecurity epistemics are whack”

The effective altruism (EA) biosecurity community focuses on reducing the risks associated with global biological catastrophes (GCBRs). This includes preparing for pandemics, improving global surveillance, and developing technologies to mitigate the risks of engineered pathogens. While the work of this community is important, there are significant challenges to developing good epistemics, or practices for acquiring and evaluating knowledge, in this area.

One major challenge is the issue of infohazards. Infohazards are ideas or information that, if widely disseminated, could cause harm. In the context of biosecurity, this could mean that knowledge of specific pathogens or their capabilities could be used to create bioweapons. As a result, members of the EA biosecurity community are often cautious about sharing information, particularly in online forums where it could be easily disseminated.  [1]

The issue of infohazards is not straightforward. Even senior biosecurity professionals may have different thresholds for what they [...]

---

Outline:

(01:57) Challenges for cause and intervention prioritisation

(04:01) Challenges for transparent and trustworthy advocacy

(06:06) What are known best practices?

The original text contained 3 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

I collected my favorite public pieces of research on AI strategy, governance, and forecasting from 2023 so far.

If you're a researcher, I encourage you to make a quick list of your favorite pieces of research, then think about what makes it good and whether you're aiming at that with your research.

To illustrate things you might notice as a result of this exercise:

I observe that my favorite pieces of research are mostly aimed at some of the most important questions[1]– they mostly identify a very important problem and try to answer it directly.

I observe that for my favorite pieces of research, I mostly would have been very enthusiastic about a proposal to do that research– it's not like I'd have been skeptical about the topic but the research surprised me with how good its results were.[2]

1. Model evaluation for extreme risks (DeepMind, Shevlane et al., May)

Current approaches to building general-purpose AI [...]

---

Outline:

(00:48) 1. Model evaluation for extreme risks (DeepMind, Shevlane et al., May)

(02:35) 2. Towards best practices in AGI safety and governance: A survey of expert opinion (GovAI, Schuett et al., May)

(04:42) 3. What does it take to catch a Chinchilla? Verifying Rules on Large-Scale Neural Network Training via Compute Monitoring (Shavit, March)

(06:53) 4. Survey on intermediate goals in AI governance (Rethink Priorities, Räuker and Aird, March)

(09:12) 5. Literature Review of Transformative AI Governance (LPP, Maas, forthcoming)

(10:44) 6. “AI Risk Discussions” website: Exploring interviews from 97 AI Researchers (Gates et al., February)

(12:46) 7. What a compute-centric framework says about AI takeoff speeds - draft report (OpenPhil, Davidson, January)

The original text contained 2 footnotes which were omitted from this narration.

---

First published:

Source:

Linkpost URL:
https://blog.aiimpacts.org/p/my-favorite-ai-governance-research

Narrated by TYPE III AUDIO.

(Cross-posted from my website.)

I recently resigned as Columbia EA President and have stepped away from the EA community. This post aims to explain my EA experience and some reasons why I am leaving EA. I will discuss poor epistemic norms in university groups, why retreats can be manipulative, and why paying university group organizers may be harmful. Most of my views on university group dynamics are informed by my experience with Columbia EA. My knowledge of other university groups comes from conversations with other organizers from selective US universities, but I don’t claim to have a complete picture of the university group ecosystem.

Disclaimer: I’ve written this piece in a more aggressive tone than I initially intended. I suppose the writing style reflects my feelings of EA disillusionment and betrayal.

My EA Experience

During my freshman year, I heard about a club called Columbia Effective Altruism. Rumor on [...]

---

Outline:

(00:57) My EA Experience

(06:38) Epistemic Problems in Undergraduate EA Communities

(08:33) My Best Guess on Why AI Safety Grips Undergraduate Students

(11:56) Caveats

(12:19) How Retreats Can Foster an Epistemically Unhealthy Culture

(12:43) Against Taking Ideas Seriously

(13:54) Why Do People Take Ideas Seriously in Retreats?

(15:51) Other Retreat Issues

(16:58) University Group Organizer Funding

(17:18) Why I Think Paying Organizers May Be Bad

(18:17) Potential Solutions

(19:08) Final Remarks

---

First published:

Source:

Narrated by TYPE III AUDIO.

EA-motivated specialists sometimes offer free or subsidized versions of normally expensive services to EA projects. I think this is often counterproductive and outline my reasoning in this post. 

The key problem with free services is that we don’t have market-based information about their quality, so the beneficiaries of a free service might be getting less value than it might appear they are getting. As a result, service providers waste their time providing expensive services to people who wouldn’t pay the full price (instead, providers could charge and donate[1] or spend that time on other impactful work). Additionally, community-level overestimates of the quality of free services are more likely and might lead people who need good services to use the free versions even when they’re worse suited to their needs. 

If you’re offering, taking, or advertising a free service like this, I think you should believe the situation is an exception to the general heuristic. [...]

---

Outline:

(03:59) Problems

(04:02) It’s harder to evaluate the quality/value of free services, so (1) providers might keep offering free services that are a bad use of their time, and (2) people might use services that are not what they need

(08:46) It might be more effective to charge the full cost and donate the profits

(10:30) More minor/complicated issues with services that are offered for free

(12:04) Situations in which it’s potentially reasonable to offer/advertise free services to EAs

(16:30) Other nuances and counterpoints

The original text contained 14 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

I'm stepping down as chair of the Long-Term Future Fund. I'm writing this post partially as a loose set of reflections on my time there, and partially as an overall update on what's going on with the fund, as I think we should generally be transparent with donors and grantees, and my sense is the broader community has fairly little insight into the fund's current operations. I'll start with a brief history of what's happened since I joined the fund, and its impact, and move to a few reflections on ways the fund is working now.

(Also-- you can donate to the Long-Term Future Fund here, and let us know here if you might be interested in becoming a fund manager. (The Long-Term Future Fund is part of EA Funds, which is a fiscally sponsored project of Effective Ventures Foundation (UK) (EV UK) and Effective Ventures Foundation USA Inc. (EV US). [...]

---

Outline:

(00:56) A brief history of my time on the Long-Term Future Fund

(03:29) The fund’s impact

(05:42) Reflections

(05:45) Problems with scale

(11:22) Transparency

---

First published:

Source:

Narrated by TYPE III AUDIO.

Views expressed in this article are my own and do not necessarily reflect those of my employer SecureBio.

Summary

• Far-UVC has great promise, but a lot of work still needs to be done
  • There still are many important open research questions that need to be answered before the technology can become widely adopted
  • Right now, a key priority is to grow the research field and improve coordination
• The main reason far-UVC is so promising is that widespread installation could passively suppress future pandemics before we even learn that an outbreak has occurred 
• Higher doses mean more rapid inactivation of airborne pathogens but also more risk for harm to skin, eyes, and through indoor air chemistry. Therefore, the important question in safety is, “How high can far-UVC doses go while maintaining a reasonable risk profile?”
• Existing evidence for skin safety within current exposure guidelines seems pretty robust, and I expect that skin safety won't be the bottleneck for far-UVC deployment at [...]

  ---

First published:

Source:

Narrated by TYPE III AUDIO.

On Tuesday, the US Senate Judiciary Subcommittee on Privacy, Technology and the Law held a hearing on AI. The hearing involved 3 witnesses – Dario Amodei (CEO of Anthopic), Yoshua Bengio (Turing Award winner, and the second-most cited AI resarcher in the world), and Stuart Russell (Professor of CS at Berkeley, and co-author of the standard textbook for AI). The hearing wound up focusing a surprising amount on AI X-risk and related topics. I originally planned on jotting down all the quotes related to these topics, thinking it would make for a short post of a handful of quotes, which is something I did for a similar hearing by the same subcommittee 2 months ago. Instead, this hearing focused so much on these topics that I wound up with something that’s better described as a partial transcript.All the quotes below are verbatim. Text that is bolded is simply stuff I thought readers might find particularly interesting. If you want to listen to the hearing, you can do so here (it’s around 2.5 hours). You might also find it interesting to compare this post to the one from 2 months ago, to see how the discourse has progressed.Opening remarksSenator Blumenthal:What I have [...]

---

First published:

Source:

Linkpost URL:
https://medium.com/@daniel_eth/ai-x-risk-at-senate-hearing-7104f371ca0b

Narrated by TYPE III AUDIO.

TL;DR: When I say “General EA” I am referring to the cluster including the term “effective altruism,” the idea of “big-tent EA,” as well as branding and support of those ideas. This post is a response in opposition to many calls for renaming EA or backing away from an umbrella movement. I make some strategic recommendations and take something of a deep dive using my own personal history/cause prioritization as a case study for why “General EA” works (longpost is long, so there's TL;DRs for each major section).

I’m primarily aiming to see if I’m right that there's a comparatively silent group that supports EA largely as it. If you’re in that category and don’t need the full rationale and story, the call to action is to add a comment linking your favorite “EA win” (success story/accomplishment you’d like to have people associate with EA).

Since long [...]

---

Outline:

(03:09) “Effective Altruism”

(07:26) In support of “Big Tent EA”

(13:39) In support of maintaining the EA brand

(18:10) Conclusion

The original text contained 4 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

TL;DR: When I say “General EA” I am referring to the cluster including the term “effective altruism,” the idea of “big-tent EA,” as well as branding and support of those ideas. This post is a response in opposition to many calls for renaming EA or backing away from an umbrella movement. I make some strategic recommendations and take something of a deep dive  using my own personal history/cause prioritization as a case study for why “General EA” works (longpost is long, so there's TL;DRs for each major section). I’m primarily aiming to see if I’m right that there’s a comparatively silent group that supports EA largely as it. If you’re in that category and don’t need the full rationale and story, the call to action is to add a comment linking your favorite “EA win” (success story/accomplishment you’d like to have people associate with EA).Since long before last fall's reckoning, I've been following discussions in the EA community both for or against the "effective altruism" name, debates about rebranding, and calls to splinter the various worldviews currently covered by the EA umbrella into separate groups. There are too many to link, and since this post is ultimately in opposition to them, [...]

The original text contained 4 footnotes which were omitted from this narration.

---

First published:

Source:

Narrated by TYPE III AUDIO.

SummaryWe are launching the Meta Charity Funders, a growing network of donors sharing knowledge and discussing funding opportunities in the EA meta space. Apply for funding by August 27th or join the circle as a donor. See below or visit our website to learn more!If you are doing EA-aligned “meta” work, and have not received substantial funding for several years, you might be worried about funding. Over the past 10 years, Open Philanthropy and EA Funds comprised a large percent of total meta funding and are far from independent of each other. This lack of diversity means potentially effective projects outside their priorities often struggle to stay afloat or scale, and the beliefs of just a few grant-makers can massively shape the EA movement’s trajectory. It can be difficult for funders within meta as well. Individual donors often don’t know where to give if they don’t share EA Funds’ approach. Thorough vetting is scarce and expensive, with only a handful of grant-makers deploying tens of millions per year in meta grants, resulting in sub-optimal allocations.  This is why we are launching the Meta Charity Funders, a growing network of donors sharing knowledge, discussing funding opportunities, and running joint open grant rounds in the EA meta [...]

---

First published:

Source:

Narrated by TYPE III AUDIO.