The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • AF - A strong mind continues its trajectory of creativity by Tsvi Benson-Tilsen
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A strong mind continues its trajectory of creativity, published by Tsvi Benson-Tilsen on May 14, 2023 on The AI Alignment Forum.
    [Metadata: crossposted from. First completed January 29, 2023.]
    A very strong mind is produced by a trajectory of creativity. A trajectory of creativity that produces a very strong mind is hard to separate from the mind's operation. So a strong mind continues on its trajectory of creativity as long as it is active.
    A strong mind comes from a trajectory of creativity
    If a mind is highly capable, it got to that point by gaining understanding in a voyage of novelty. If the mind gains understanding that is novel for all the surrounding minds (e.g., preexisting humans), it does so through creativity: generating novelty, rather than merely copying it. If the mind becomes much more capable than the surrounding minds, it does so by being on a trajectory of creativity: something about the mind implies that it generates understanding that is novel to the mind and its environment. If the mind is on a trajectory of creativity that brought it to the point of being highly capable, its trajectory of creativity probably carries the mind much further, making the mind much more capable than it already is.
    The ex quo of a mind's creativity is the element (collection of elements) out of which comes novel structure. The ex quo of a modern AI system is almost entirely dependent on the search (i.e. training) apparatus, which is clearly separated out from running the found system. (The ex quo isn't entirely dependent on the search apparatus. Some non-zero creativity happens in the collision of elements that happens in, say, a single run of a stable diffusion image model or a large transformer language model. But it's not much creativity, and the found structure is about as temporarily grasped as possible.)
    The proximal ex quo is that out of which novel structure comes directly. The distal ex quo is that out of which novel structure comes indirectly. So the mental context that's set up when a particular idea comes to you, and the other dark matter that goes into that abduction, is the proximal ex quo; human evolution is the distal ex quo; and the history of the development of your brain is an intermediate ex quo.
    Trajectory and operation are hard to separate
    An AI can simply be shut down, until it's able to and wants to stop you from shutting it down. But can an AI's improvement be shut down, without shutting down the AI? This can be done for all current AI systems in the framework of finding a fairly limited system by a series of tweaks. Just stop tweaking the system, and it will now behave as a fixed (perhaps stochastic) function that doesn't provide earth-shaking capabilities.
    I suspect that the ex quo that puts a mind on a trajectory to being very strong, is hard to separate from the operation of the mind. Some gestures at why:
    Making doesn't imply understanding
    Just because you can make something, doesn't mean you understand how it works.
    Evolution is a distal ex quo of human understanding. But there's clearly an ex quo more proximal than evolution for, say, scientific understanding: human thought and investigation. Setting up an evolution that can produce humans doesn't imply that you understand how humans do science.
    The way we make neural networks today is by setting up a distal ex quo (the search process). A more proximal ex quo for a neural net comes from the accumulated hidden features: they set up the context in which the next little tweak is beneficial. We can know how to make neural nets that work well without knowing much about how the series of tweaks in context build up the computations that end up performing well at the given task.
    We can nevertheless turn off the ex quo of current AI systems because the ex quo is almost entirely dependent on the ...
    11 min
  • EA - Announcing the African EA Forum Competition - $4500 in prizes for excellent EA Forum Posts by Luke Eure
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Announcing the African EA Forum Competition - $4500 in prizes for excellent EA Forum Posts, published by Luke Eure on May 14, 2023 on The Effective Altruism Forum.
    The Effective Altruism Forum serves as the main platform for thought leadership and experience sharing within the effective altruism movement. It is the virtual gathering spot for EAs and the space where most EAs will go to check for resources, materials and experiences for various areas of interest.
    However, there are low levels of authorship by EAs from Africa, which gives the impression that not much is happening in the African space with regards to EA - despite there being 10+ local EA groups in Africa.
    There is a need for increased thought leadership, experience sharing, and engagement from an African perspective that reflects the growth of EA in Africa and brings African perspectives to the wider EA community.
    To get more Africans writing on the forum, I'm excited to announce the African EA Forum Competition.
    How it works
    Prizes will be awarded for winners and runners up in each of three categories:
    Cause exploration: Explorations of new cause areas or contribution to thinking on existing cause areas
    African perspectives on EA: Posts that challenge or complement existing EA perspectives and/or cause areas from an African worldview
    Summaries of existing work: Informing the EA community about ongoing/completed research projects, sharing experiences within community groups, or reflections on being in EA
    Top prize in each category will be awarded $1,000. Runner up in each category will get $500.
    No need to include the category in your submission.
    Judging
    Posts will be judged based on the following rubric:
    Originality of insight
    Clarity
    Discussion provoked: (judged by the post’s forum score and number of comments)
    Persuasiveness of argument
    This is replaced by Relevance to forum readers for summaries of existing work
    The members of our judging panel are:
    Alimi Salifou - Events and Outreach Coordinator for EA Nigeria
    Jordan Pieters - independent community builder
    Kaleem Ahmid - Project Manager at Effective Ventures
    Dr. Kikiope Oluwarore - co-founder of Healthier Hens
    Zainab Taonga Chirwa - Chairperson for Effective Altruism UCT
    Support offered to writers
    We will support writers in the following two ways:
    Virtual training on EA Forum writing: there will be a ~1-2 hour workshop to build the capacity of individuals interested in writing forum posts
    Offering mentorship: curating a list of individuals who are happy to offer feedback to new writers to bolster the confidence of those writing about their posts, especially those who may feel intimidated by the seemingly high bar of forum posts
    Writers do not have to attend the forum or receive mentorship to be eligible for the competition. They only have to make a forum post within the competition period, and meet the eligibility criteria below.
    Please fill out this form if you would like to join the workshop or receive mentorship.
    Who is eligible
    Anybody who is:
    a citizen of an African country
    a children of a citizen of an Africa country
    Posts should be made to the EA forum with the tag ‘Africa EA Forum Competition’ to make them easy to find, and then the writer should identify themself by filling out this form.
    Posts with multiple authors are eligible as long as 50% or more of the authors meet the above criteria.
    Timeline
    The competition will run for 3 months. Any post made between 14 May 2023 - 11:59pm on Friday 18 August 2023 is eligible.
    Please reach out to me with any questions or feedback on how this competition could be better! ([email protected])
    Thanks to Daniel Yu for funding the prizes, Waithera Mwangi for support in writing this post, and to our judges for offering their time
    Thanks for listening. To help us out with The Nonlinear Library or to learn mo...
    4 min
  • EA - Blog update: Reflective altruism by David Thorstad
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Blog update: Reflective altruism, published by David Thorstad on May 14, 2023 on The Effective Altruism Forum.
    About me
    I’m a research fellow in philosophy at the Global Priorities Institute. Starting in the Fall, I'll be Assistant Professor of Philosophy at Vanderbilt University. (All views are my own, except the worst. Those are to be blamed on my cat.).
    There are many things I like about effective altruism. I’ve started a blog to discuss some views and practices in effective altruism that I don’t like, in order to drive positive change both within and outside of the movement.
    About this blog
    The blog features long-form discussions, structured into thematic series of posts, informed by academic research. Currently, the blog features six thematic series, described below.
    One distinctive feature of my approach is that I share a number of philosophical views with many effective altruists. I accept or am sympathetic to all of the following: consequentialism; totalism; fanaticism; expected value maximization; and the importance of using science, reasons and evidence to solve global problems. Nevertheless, I am skeptical of many views held by effective altruists including longtermism and the view that humanity currently faces high levels of existential risk. We also have a number of methodological disagreements.
    I've come to understand that this is a somewhat distinctive approach within the academic literature, as well as in the broader landscape. I think that is a shame. I want to say what can be said for this approach, and what can be learned from it. I try to do that on my blog.
    About this document
    The blog is currently five months old. Several readers have asked me to post an update about my blog on the EA Forum. I think that is a good idea: I try to be transparent about what I am up to, and I value feedback from my readers.
    Below, I say a bit about existing content on the blog; plans for new content; and some lessons learned during the past five months.
    Existing series
    Series 1: Academic papers
    The purpose of this blog is to use academic research to drive positive change within and outside of the effective altruism movement. This series draws insights from academic papers related to effective altruism.
    Sub-series A: Existential risk pessimism and the time of perils
    This series is based on my paper “Existential risk pessimism and the time of perils”. The paper develops a tension between two claims: Existential Risk Pessimism (levels of existential risk are very high) and the Astronomical Value Thesis (efforts to reduce existential risk have astronomical value). It explores the Time of Perils hypothesis as a way out of the tension.
    Status: Completed. Parts 1-6 present the main argument of the paper. Part 7 discusses an application to calculating the cost-effectiveness of biosecurity. Part 8 draws implications. Part 9 responds to objections.
    Sub-series B: The good it promises
    This series is based on a volume of essays entitled The good it promises, the harm it does: Critical essays on effective altruism. The volume brings together a diverse collection of scholars, activists and practitioners to critically reflect on effective altruism. In this series, I draw lessons from papers contained in the volume.
    Status: In progress. Part 1 introduces the series and discusses the foreword to the book by Amia Srinivasan. Part 2 looks at Simone de Lima’s discussion of colonialism and animal advocacy in Brazil. Part 3 looks at Carol J Adams' care ethical approach.
    Series 2: Academics review WWOTF
    Will MacAskill’s book What we owe the future is one of the most influential recent books about effective altruism. A number of prominent academics have written insightful reviews of the book. In this series, I draw lessons from some of my favorite academic reviews of What we owe the future....
    15 min
  • LW - An Analogy for Understanding Transformers by TheMcDouglas
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: An Analogy for Understanding Transformers, published by TheMcDouglas on May 13, 2023 on LessWrong.
    Thanks to the following people for feedback: Tilman Rauker, Curt Tigges, Rudolf Laine, Logan Smith, Arthur Conmy, Joseph Bloom, Rusheb Shah, James Dao.
    TL;DR
    I present an analogy for the transformer architecture: each vector in the residual stream is a person standing in a line, who is holding a token, and trying to guess what token the person in front of them is holding. Attention heads represent questions that people in this line can ask to everyone standing behind them (queries are the questions, keys determine who answers the questions, values determine what information gets passed back to the original question-asker), and MLPs represent the internal processing done by each person in the line. I claim this is a useful way to intuitively understand the transformer architecture, and I'll present several reasons for this (as well as ways induction heads and indirect object identification can be understood in these terms).
    Introduction
    In this post, I'm going to present an analogy for understanding how transformers work. I expect this to be useful for anyone who understands the basics of transformers, in particular people who have gone through Neel Nanda's tutorial, and/or understand the following points at a minimum:
    What a transformer's input is, what its outputs represent, and the nature of the predict-next-token task that it's trained on
    What the shape of the residual stream is, and the idea of components of the transformer reading from / writing to the residual stream throughout the model's layers
    How a transformer is composed of multiple blocks, each one containing an MLP (which does processing on vectors at individual sequence positions), and an attention layer (which moves information between the residual stream vectors at different sequence positions).
    I think the analogy still offers value even for people who understand transformers deeply already.
    The Analogy
    A line is formed by a group of people, each person holding a word. Everyone knows their own word and position in the line, but they can't see anyone else in the line. The objective for each person is to guess the word held by the person in front of them. People have the ability to shout questions to everyone standing behind them in the line (those in front cannot hear them). Upon hearing a question, each individual can choose whether or not to respond, and what information to relay back to the person who asked. After this, people don't remember the questions they were asked (so no information can move backwards in the line, only forwards). As individuals in the line gather information from these exchanges, they can use this information to formulate subsequent questions and provide answers.
    How this relates to transformer architecture:
    Each person in the line is a vector in the residual stream
    They start with just information about their own word (token embedding) and position in the line (positional embedding)
    The attention heads correspond to the questions that people in the line ask each other:
    Queries = question (which gets asked to everyone behind them in the line)
    Keys = how the people who hear the question decide whether or not to reply
    Values = the information that the people who reply pass back to the person who originally asked the question
    People can use information gained from earlier questions when answering / asking later questions - this is composition
    The MLPs correspond to the information processing / factual recall performed by each person in the sequence independently
    The unembedding at the end of the model is when we ask each person in the line for a final guess at what the next word is (in the form of a probability distribution over all possible words)
    Key Concepts for Transformers
    In...
    17 min
  • EA - The Implications of the US Supreme Court upholding Prop 12 by ishankhire
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Implications of the US Supreme Court upholding Prop 12, published by ishankhire on May 13, 2023 on The Effective Altruism Forum.
    Background: Farming Conditions and Prop 12
    Currently in the US, most breeding pigs live in factory farmers, where they are confined in gestation crates which are small metal cages so small that pigs can’t even turn around, while egg-laying hens live in tiny, cramped battery cages that cause a range of psychological and physiological harm. The crowded conditions also have potential health harms by increasing the stress levels of pigs and weakening their immune systems, which can make them more susceptible to zoonotic diseases that may spread to humans.
    Starting in the early 2000s, a few animal welfare groups including the Humane Society of the Unites States aimed to ban the farming system of cages for hens, breeding pigs and veal calves. In 2008, Proposition 2 was passed which put in place a “production” ban on cages, which said that producers had to ensure pigs, hens, and calves could lie down, turn around, and extend their limbs or wings without hitting the side of an enclosure. However, this specific language allowed some egg farms to circumvent the law by using bigger cages. In 2010, California passed AB 1437 which was a “sales” ban requiring all eggs sold in California had to meet those standards. These laws have brought about results — the share of hens that are cage-free has been rising and is expected to continue doing so.
    In 2018, over 62% of California voters passed Proposition 12, the strongest law to improve conditions for farmed animals. Under Prop 12, some of the gaps in these laws are covered — for one, it extends the cage-free ban to cover not just the eggs that are sold in the grocery store (shell eggs) but also liquid eggs, which are sold to restaurants, cafeterias and food manufacturers (liquid eggs).
    Opposition from Pork Industry
    The law is expected to be especially impactful to the pork industry which has been more resistant to change in doing away with confinement systems. Progress has been very mixed in terms of companies following through with their commitments to phase out gestation crates. So far, 10 states have banned them, but Prop 12’s space requirements are stricter and close some gaps that allow for loopholes. The law also makes it illegal for eggs and pork to be sold in California if the animals in other states are put in gestation crates (pigs) or battery cages (for chickens). California consumes 14% of the US’s pork and 12% of eggs and veal, so pork and egg producers would be forced to modify barns or construct new ones (only 1% of existing sow housing meets Prop 12’s standards according to the National Pork Producers Council (NPCC)), which would be costly and time taking, causing various meat trade groups to be opposed to it.
    Interestingly, some industries such as Whole Foods, aren’t concerned with the law as they claim they already meet animal welfare requirements. I think this is a crucial reason why the phase out of battery cages did not get as much opposition to phasing out pork crates — many companies already have commitments to phase out battery cage. In fact, these companies may have the incentive to increase regulations to raise costs on competitors.
    For this reason, the law was attacked by various meat industry trade groups, which filed three separate lawsuits to overturn it. The Supreme Court declined to take two of them, and in October 2022, the case National Pork Producers v. Ross began.
    Explaining the Supreme Court Ruling
    On May 11th, 2023, the Supreme Court upheld Prop 12 in a 5-4 decision of the case National Pork Producers v. Ross. Interestingly, the verdict was not split along conservative-liberal lines, with 3 conservative judges and 2 liberal judges in the majority.
    The pork industry ...
    8 min
  • EA - I want to read more stories by and about the people of Effective Altruism by Gemma Paterson
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: I want to read more stories by and about the people of Effective Altruism, published by Gemma Paterson on May 12, 2023 on The Effective Altruism Forum.
    TL; DR
    I want to read more stories by and about the people of the Effective Altruism movement
    But like, fun ones, not CVs
    I’ve added a tag for EA origin stories and tagged a bunch of relevant posts from the forum
    If I’ve missed some, please tag them
    The community experiences tag has a lot of others that don’t quite fit
    I think it is important to emphasise the personal in the effective altruism movement - you never know if your story is enough to connect with someone (especially if you don’t fit the stereotypical EA mold)
    I would also be very interested in reading folks’ answers to the “What is your current plan to improve the world?” question from the EA Global application - it’s really helpful to see other people’s thought processes (you can read mine here)
    Why?
    At least for me, what grabbed and kept my attention when I first heard about EA were the stories of people on the ground trying to do effective altruism.
    The audacity of a group of students looking at the enormity of suffering in the world but then pushing past that overwhelm. Recognising that they could use their privileges to make a dent if they really gave it a go.
    The folks behind Charity Entrepreneurship who didn’t stop at one highly effective charity but decided to jump straight into making an non-profit incubator to multiply their impact - building out, in my opinion, some of the coolest projects in the movement.
    I love that the 80,000 hours podcast takes the concept behind Big Talk seriously
    It’s absurd but amazing!
    I love the ethos of practicality within the movement. It isn’t about purity, it isn’t about perfection, it’s about actually changing the world.
    These are the people I’d back to build a robust Theory of Change that might just move us towards Fully Automated Luxury Gay Space Communism
    Maybe that google doc already exists?
    I have never been the kind of person who had role models. I have always been a bit too cynical to put people on a pedestal. I had respect for successful people and tried to learn what I could from them but I didn’t have heroes.
    But my response to finding the EA movement was, “Fuck, these people are cool.”
    I think there is a problem with myth making and hero worshipping within EA. I do agree that it is healthier to Live Without Idols. However, I don’t think we should live without stories.
    The stories I’m more interested in are the personal ones. Of people actually going out and living their values. Examples of trades offs that real people make that allow them to be ambitiously altruistic in a way that suits them. That show that it is fine to care about lots of things. That it is okay to make changes in your life when you get more or better information.
    I think about this post a lot because I agree that if people think that “doing effective altruism” means they have to live like monks and change their whole lives then they’ll just reject it. Making big changes is hard. People aren’t perfect.
    I can trace huge number of positive changes in my life to my decision to take EA seriously but realistically it was my personal IRL and parasocial connections to the people of EA that gave me the space and support to make these big changes in my life. In the footnotes and in this post about my EA story, I’ve included a list of podcasts, blog posts and other media by people within EA that were particularly influential and meaningful to me (if you made them then thank you <3)
    While I do see as EA as the key source of purpose in my life, it is a core value among many (I like Valuism - doing the intrinsic values test was really helpful for me). Like everyone else in the EA movement, I’m not an impact machine, I’m a person. I love throwing the...
    7 min
  • EA - Prioritising animal welfare over global health and development? by Vasco Grilo
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Prioritising animal welfare over global health and development?, published by Vasco Grilo on May 13, 2023 on The Effective Altruism Forum.
    Summary
    Corporate campaigns for chicken welfare increase wellbeing way more cost-effectively than the best global health and development (GHD) interventions.
    In addition, the effects on farmed animals of such interventions can influence which countries they should target, and those on wild animals might determine whether they are beneficial or harmful.
    I encourage Charity Entrepreneurship (CE), Founders Pledge (FP), GiveWell (GW), Open Philanthropy (OP) and Rethink Priorities (RP) to:
    Increase their support of animal welfare interventions relative to those of GHD (at the margin).
    Account for effects on animals in the cost-effectiveness analyses of GHD interventions.
    Corporate campaigns for chicken welfare increase nearterm wellbeing way more cost-effectively than GiveWell’s top charities
    Corporate campaigns for chicken welfare are considered one of the most effective animal welfare interventions. A key supporter of these is The Humane League (THL), which is one of the 3 top charities of Animal Charity Evaluators.
    I calculated the cost-effectiveness of corporate campaigns for broiler welfare in human-years per dollar from the product between:
    Chicken-years affected per dollar, which I set to 15 as estimated here by Saulius Simcikas.
    Improvement in welfare as a fraction of that of median welfare range when broilers go from a conventional to a reformed scenario, assuming:
    The time broilers experience each level of pain defined here (search for “definitions”) in a conventional and reformed scenario is given by these data (search for “pain-tracks”) from the Welfare Footprint Project (WFP).
    The welfare range is symmetric around the neutral point, and excruciating pain corresponds to the worst possible experience.
    Excruciating pain is 1 k times as bad as disabling pain.
    Disabling pain is 100 times as bad as hurtful pain.
    Hurtful pain is 10 times as bad as annoying pain.
    The lifespan of broilers is 42 days, in agreement with section “Conventional and Reformed Scenarios” of Chapter 1 of Quantifying pain in broiler chickens by Cynthia Schuck-Paim and Wladimir Alonso.
    Broilers sleep 8 h each day, and have a neutral experience during that time.
    Broilers being awake is as good as hurtful pain is bad. This means being awake with hurtful pain is neutral, thus accounting for positive experiences.
    Median welfare range of chickens, which I set to RP's median estimate of 0.332.
    Reciprocal of the intensity of the mean human experience, which I obtained supposing humans:
    Sleep 8 h each day, and have a neutral experience during that time.
    Being awake is as good as hurtful pain is bad. This means being awake with hurtful pain is neutral, thus accounting for positive experiences.
    I computed the cost-effectiveness in the same metric for the lowest cost to save a life among GW's top charities from the ratio between:
    Life expectancy at birth in Africa in 2021, which was 61.7 years according to these data from OWID.
    Lowest cost to save a life of 3.5 k$ (from Helen Keller International), as stated by GW here.
    The results are in the tables below. The data and calculations are here (see tab “Cost-effectiveness”).
    Intensity of the mean experience as a fraction of the median welfare range
    Broiler in a conventional scenario
    Broiler in a reformed scenario
    Human
    5.7710^-6
    2.5910^-5
    3.3310^-6
    Broiler in a conventional scenario relative to a human
    Broiler in a reformed scenario relative to a human
    Broiler in a conventional scenario relative to a reformed scenario
    7.77
    1.73
    4.49
    Improvement in chicken welfare when broilers go from a conventional to a reformed scenario as a fraction of...
    The median welfare range of chickens
    The intensity of the mean human experience
    2....
    31 min
  • LW - Thoughts on LessWrong norms, the Art of Discourse, and moderator mandate by Ruby
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Thoughts on LessWrong norms, the Art of Discourse, and moderator mandate, published by Ruby on May 11, 2023 on LessWrong.
    A couple of weeks ago I asked Should LW have an official list of norms? and I appreciate the responses there. Here I want to say what I'm currently thinking following that post, and continue having a public conversation about it.
    I think saying more on this topic actually gets into a bunch of interesting questions around LessWrong's purpose, userbase, de facto norms and culture, moderation mandate, etc. Without locking in things as "Officially How It Is Forever", I'll opine on my current thinking on this topics and how I relate to them in practice. It's possible that further public discussion will shift some things here, and after more back-and-forth, it'd make sense to "ratify" some of it more.
    With all that said...
    LessWrong and The Art of Discourse
    LessWrong was founded to be a place for perfecting the Art of Human Rationality, i.e., generally thinking in ways which more reliably result in true beliefs, etc. Similarly, I think there's a closely related "Art of Discourse": communicating in ways that more reliably result in those conversing (and reading along) having more true beliefs. Perhaps it's a sub-art of the Art of Human Rationality.
    The real rules of which communication most efficiently gets you towards truth lives in reality. You can choose your norms, but whether those norms are conducive to truth isn't up to you.
    The LessWrong community, over its 10-15 year existence, has assembled a number of beliefs about the Art of Discourse. Things like communicating degrees of belief quantitatively, preference for asymmetric weapons, an interest in local validity, etc. We of course don't have the complete art and may be mistaken about pieces of it, but we feel strongly about some of the pieces we believe we possess of this art.
    Different people in our community have somewhat different senses of the Art of Discourse, and these even form clusters. But there's a pretty solid common core set of norms on the site, such that if someone is not conforming to them, most people would want them to change their behavior or go elsewhere.
    The core point I want to make here is: The Art of [Truth-seeking] Discourse lives in the territory, and we community members attempt to discover it and practice it.
    Moderators moderate according to their own understanding of The Art
    A thing you could imagine doing is the community comes together, writes down its sense of how you ought to behave, and enshrines that as The Law. The moderators (judges/police) then interpret and enforce the law. I think this sometimes gets called "Rule of Law".
    I think that gets you some advantages, but requires infrastructure and investment LessWrong can't realistically have, both for enshrining the initial law and then updating it over time in cases of incompleteness and ambiguity.
    (edit: "Rule of Man" as an existing phrase means something crucially different from what I wanted to described. See my comment here for clarification.
    Instead LessWrong operates by a "Rule of Man" ~~"hybrid Rule of Law/Man" system where the moderators apply our own understanding of the Art of Discourse to making moderation decisions about which behaviors are okay or not, and what to do with users who behave badly according to us. This has quite a few benefits: it allows us to be flexible and adaptable to new cases, it means we ask a direct question of "does this seem good or not?" rather than "did it violate the enshrined law?", and it allows us to smoothly improve the enforced policy as our understanding of the Art of Discourse improves over time.
    This approach does run the risk that moderators have bad calls (or could be corrupt or biased), which is why I favor moderation being transparent where doing so isn't too c...
    8 min
  • EA - Proposed - 'How Much Does It Cost to Save a Life?' Quiz, calculator, tool by david reinstein
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Proposed - 'How Much Does It Cost to Save a Life?' Quiz, calculator, tool, published by david reinstein on May 12, 2023 on The Effective Altruism Forum.
    Epistemic basis/status: I've talked this over with Grace and others at GWWC, and people seem generally interested. I'm posting this to get feedback and gauge interest before potentially pushing it further.
    Basic idea
    I'd like to get your thoughts on a "How Much Does It Cost to Save a Life?"[1] quiz and calculator. I've been discussing this with Giving What We Can; it's somewhat modeled off their how rich am I calculator, which drives a lot of traffic to their site.
    This would mainly target non-EAs, but it would try to strike a good balance between sophistication and simplicity. It could start as a quiz to get people's attention. People would be asked to guess this cost. They could then be asked to reconsider it considering some follow-up questions. This might be a good opportunity for a chatbot to work its magic.
    After this interaction, the 'correct answer' and 'how well did I do' would take you to an interactive page, presenting the basic calculation and reasoning. (Before or after presenting this) it could also allow users to adjust their moral and epistemic parameters and the scope of their inquiry. This might be something to unfold gradually, letting people specify first one thing, and then maybe more, if they like.
    E.g.,
    Target: Rich or poor countries, which age groups, etc.
    Relative value of a child or adults life
    How much do you weight life-years for certain states
    Which evidence do you find more plausible
    Do you want to include or exclude certain types of benefits
    Discount rate
    We would aim to go viral (or at least bacterial)!
    Value/ToC
    I believe that people would be highly interested in this: it could be engaging and pique curiosity and competitiveness (a bit click-baity, maybe, but the payoff is not click bait)!
    It could potentially make news headlines. It’s an “easy story” for media people, asks a question people can engage with, etc. . ’how much does it cost to save a life? find out after the break!) giving the public a chance to engage with the question: "How much does it cost to save a life?"
    It could help challenge misconceptions about the cost of saving lives, contributing to a more reality-based, impact-focused, and evidence-driven donor community. If people do think it’s much cheaper than it is, as some studies suggest, it would probably be good to change this misconception. It may also be a stepping stone towards encouraging people to think more critically about measuring impact and considering EA-aligned evaluations.
    > Greater acceptance and understanding of EA, better epistemics in the general public, better donation and policy choices
    Implementation
    While GiveWell does have a page with a lot of technical details, it doesn't quite capture the interactive and compelling aspects I'm envisioning for this tool.
    Giving What We Can's response has been positive, but they understandably lack the capacity within their core team to take on such a project. They suggest it could make for an interesting volunteer project if a UX designer and an engineer were interested in participating.
    Considering the enthusiasm and the potential for synergy with academic research (which could be supported by funds for Facebook academic ads), I'm contemplating the best approach to bring this idea to life. I tentatively propose the following steps:
    Put out a request for a volunteer to help develop a proof of concept or minimum viable product. Giving What We Can has some interested engineers, and I could help with guidance and encouragement.
    2 Apply for direct funding for the project, possibly collaborating with groups focused on quantitative uncertainty and "build your own cost-effectiveness" initiatives, or perhaps with SoGi...
    4 min
  • LW - Towards Measures of Optimisation by mattmacdermott
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Towards Measures of Optimisation, published by mattmacdermott on May 12, 2023 on LessWrong.
    We would like a mathematical theory which characterises the intuitive notion of ‘optimisation’.
    Before Shannon introduced his mathematical theory of communication, the concept of ‘information’ was vague and informal. Then Shannon devised several standardised measures of it, with useful properties and clear operational interpretations. It turned out the concept of information is universal, in that it can be quantified on a consistent scale (bits) across various contexts.
    Could something similar be true for ‘optimisation’?
    In this post we review a few proposed ways to measure optimisation power and play around with them a bit.
    Our general setup will be that we’re choosing actions from a set A to achieve an outcome in a set X. We have some beliefs about how our actions will affect outcomes in the form of a probability distribution pa∈ΔX for each a∈A. We also have some preferences, either in the form of a preference ordering over outcomes, or a utility function over outcomes. We will also assume there is some default distribution p∈ΔX, which we can interpret either as our beliefs about X if we don’t act, or if we take some default action[1].
    Yudkowsky
    Yudkowsky’s proposed definition just makes use of a preference ordering ⪰ over X. To measure the optimisation power of some outcome x, we count up all the outcomes which are at least as good as x, and divide that number by the total number of possible outcomes. It’s nice to take a negative log to turn this fraction into a number of bits: OP(x)=−log|{x′∈X∣x′⪰x}X|. If I achieve the second best outcome out of eight, that's −log28=2 bits of optimisation power.
    If the outcome space is infinite, then we can't count the number of outcomes at least as good as the one we got, so we need a measure to integrate over. If we make use of our default probability distribution here, the resulting quantity has a nice interpretation. ∫x′⪰xp(x′)dx′∫p(x′)dx′ is just ∫x′⪰xp(x′)dx′: the default probability of doing as well as we did. Since we're always assuming we've got a default distribution, we might as well define OP like this even in the finite-domain case. Again we’ll take a log to get OP(x)=−log∫x′⪰xp(x′)dx′. Now 2 bits of optimisation power means the default probability of doing this well was 14.
    So far we’ve just been thinking about the optimisation power of achieving a specific outcome. We can define the optimisation power of an action as the expected optimisation power under the distribution it induces over outcomes: OP(a)=∫pa(x)OP(x)dx. The above definitions just make use of a preference ordering. If we do have a utility function u:XR then we’d like our definitions to make use of that too. Intuitively, achieving the second best outcome out of three should constitute more optimisation power in a case where it’s almost as good as the first and much better than the third, compared to a case where it’s only slightly better than the third and much less good than the first[2].
    Analogously to how we previously asked ‘what fraction of the default probability mass is on outcomes at least as good as this one?’ we could try to ask ‘what fraction of the default expected utility comes from outcomes at least as good as this one?’.
    But making use of utility functions in the above definition is tricky. Recall that utility functions originate from the Von Neumann-Morgenstern theorem, which says that if an agent choosing between probabilistic mixtures of options satisfies some weak rationality criteria then it acts as if it maximises expected utility according to a utility function u:XR. The utility function produced by the VNM-theorem is only defined up to positive affine transformations, meaning that the utility function u′=au+b, for any a∈R>0 and b∈R, equally...
    7 min