The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • LW - Optical Illusions are Out of Distribution Errors by vitaliya
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Optical Illusions are Out of Distribution Errors, published by vitaliya on August 15, 2023 on LessWrong.
    Our visual black box is trained on what we see in the real world. We don't process raw sensory data - we keep a more abstracted model in our heads. This is a blessing and a curse - it may either be more useful to have a sparser cognitive inventory, and work on a higher level, or to ignore preconceived notions and try to pick up details about What Actually Is. Marcus Hutter would have us believe this compression is itself identical to intelligence - certainly true for the representation once it hits your neurons. But as a preprocessing step? It's hard to be confident. Is that summarisation by our visual cortex itself a form of intelligence?
    Ultimately, we reduce a three-dimensional world into a two-dimensional grid of retinal activations; this in turn gets reduced to an additional layer of summarisation, the idea of the "object". We start to make predictions about the movement of "objects", and these are usually pretty good, to the point where our seeming persistence of vision is actually a convenient illusion to disguise that our vision system is interrupt-based. And so when we look at a scene, rather than even being represented as a computer does an image, as any kind of array of pixels, there is in actuality a further degree of summarisation occurring; we see objects in relation to each other in a highly abstract way. Any impressions you might have about an object having a certain location, or appearance, only properly arise as a consequence of intentional focus, and your visual cortex decides it's worth decompressing further.
    This is in part why blind spots (literal, not figurative) can go unidentified for so long: persistence of vision can cover them up. You're not really seeing the neuronal firings, it's a level lower than "sight". It probably isn't possible to do so consciously - we only have access to the output of the neural network, and its high-level features; these more abstracted receptive fields are all we ever really see.
    This is what makes optical illusions work. Your visual cortex is pretrained to convert an interrupt stream from your eyeballs into summarised representations. And in doing so it makes certain assumptions - ones which, once correctly characterised, can be subverted with a particular adversarial stimulus. Each optical illusion shows its own failure of the visual cortex to properly represent "true reality" to us. The underlying thing they're pointing at, intentionally or otherwise, is some integrative failure. A summarisation by the visual system that, on further inspection and decompression, turns out to be inaccurate.
    We can intentionally exploit these integrative failures, too. Autostereograms have us trick our visual cortex into the interpretation of a flat printed surface as having some illusory depth. Here, that interpretation is intended; if we were forced to see the world as it "truly is", such illusions would be impossible to detect. And so correcting all possible interpretation errors would have other knock-on effects - almost by necessity, there would be a family of phenomena that we would be incapable of detecting, phenomena which rely on the abstraction.
    This isn't just "like" an out of distribution error in neural networks, it's the archetypal example of it as wired in biology. And so if you've ever wondered how those errors feel "from the inside" - well, go take a look at some optical illusions, and you'll understand a little better.
    Are these errors a bad thing? Not necessarily.
    People with grapheme-color synesthesia are arguably people with especially aggressive pattern-matching visual networks. When such a person looks at the above image on the left, it might appear similar to that on the right, in the same vague kind...
    4 min
  • LW - My checklist for publishing a blog post by Steven Byrnes
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: My checklist for publishing a blog post, published by Steven Byrnes on August 15, 2023 on LessWrong.
    Introduction
    Checklists are good. I don't use checklists much for my job though. (My to-do list is stylistically a kanban, not a checklist - details here & here.)
    But I have one exception: My checklist for publishing blog posts (an activity that I've been doing with some regularity - you're reading my 114th blog post just on this forum!)
    I am sharing that checklist here, not because it's particularly good, nor because I'm recommending that other people use it (obviously it's tailored to my idiosyncratic needs), but because I'm interested in sharing ideas and getting feedback!
    Related things on this forum include a 2012 essay-publishing checklist by gwern, and Justis's writing advice list which is not directly a checklist but could be made into one (and indeed I copied a few items from it). Please comment with other references and suggestions! How would your own checklist differ from mine?
    A couple more bits of commentary before we begin:
    Checklist workflow: Good news is that pretty much every productivity-related app (e.g. logseq, roam, obsidian, emacs-org-mode, trello, etc.) has a very nice workflow for checklists - where you make a reusable checklist template, and then insert a fresh (non-checked-off) copy into the appropriate context, and then check off the items one-by-one. If you don't know the details, google it.
    "Consider doing X" items: You'll notice that many of these checklist items are of the form "Consider doing X". Often what that means for me in practice is:
    I get to the checklist item "Consider doing X";
    I consider doing X, and decide not to;
    I happily check it off.
    That's fine! It's not always a good use of time to make a blog post higher-quality. The one you're reading right now is a great example: I am writing this post very quickly, and I stand by that decision.
    OK, that's enough commentary! The rest of the post is the checklist itself.
    The actual checklist! (2023-08-15 version)
    Copyediting items
    Check for unexplained or unnecessary jargon & acronyms.
    Check for jargon & acronyms that are defined in one part of the post and then used in a distant part of the post without repeating the definition.
    Check for unnecessarily obscure words and cultural references (for non-native English speakers)
    Check for vague "this"
    Check for over-hedging
    Consider checking that all the hyperlinks actually go to the intended destination
    Consider adding more hyperlinks, references, and footnotes
    Consider adding a self-contained summary / table-of-contents / tl;dr to the top
    Consider adding humorous things
    Consider looking at each section and asking: "Can I delete this?"
    Consider looking at each paragraph and asking: "Can I delete this?"
    Consider whether there's anything I can move out of the main text and into a footnote (or hyperlink)
    Consider replacing (or at least supplementing) strawman arguments with better versions (even in the context of a "common misperceptions" discussion)
    Consider replacing criticism with "let's try to do better" type language
    Consider replacing criticism of individuals / groups with criticism of papers / ideas / plans
    Consider adding pictures, possibly including AI-generated.
    Consider adding concrete examples
    Consider "not being lazy / rushed" (e.g. if the text says "I don't know X" or "I didn't check Y" etc., consider whether I should sort that out before publishing)
    Make sure images / tables / etc. look OK in both light mode and dark mode (e.g. diagrams probably need a white background, not transparent).
    Check that the sidebar outline looks right
    Check that "> blah" sections have been reformatted as proper quote blocks (after copy-pasting from google docs)
    Consider asking GPT for copyediting advice (see below)
    Consider sharing the draft with ...
    6 min
  • EA - Price-, Taste-, and Convenience-Competitive Plant-Based Meat Would Not Currently Replace Meat by Jacob Peacock
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Price-, Taste-, and Convenience-Competitive Plant-Based Meat Would Not Currently Replace Meat, published by Jacob Peacock on August 15, 2023 on The Effective Altruism Forum.
    Also available on the Rethink Priorities website.
    Executive summary
    Plant-based meats, like the Beyond Sausage or Impossible Burger, and cultivated meats have become a source of optimism for reducing animal-based meat usage.
    Public health, environmental, and animal welfare advocates aim to mitigate the myriad harms of meat usage.
    The price, taste, and convenience (PTC) hypothesis posits that if plant-based meat is competitive with animal-based meat on these three criteria, the large majority of current consumers would replace animal-based meat with plant-based meat.
    The PTC hypothesis rests on the premise that PTC primarily drive food choice.
    The PTC hypothesis and premise are both likely false.
    A majority of current consumers would continue eating primarily animal-based meat even if plant-based meats were PTC-competitive.
    PTC do not mainly determine food choices of current consumers; social and psychological factors also play important roles.
    Although not examined here, there may exist other viable approaches to drive the replacement of animal-based meats with plant-based meats.
    There is insufficient empirical evidence to more precisely estimate or optimize the current (or future) impacts of plant-based meat. To rectify this, consider funding:
    Research measuring the effects of plant-based meat sales on displacement of animal-based meat.
    Research comparing the effects of plant-based meats with other interventions to reduce animal-based meat usage.
    Informed (non-blinded) taste tests to benchmark current plant-based meats and enable measurements of taste improvement over time.
    Introduction
    Plant-based meats, like the Beyond Sausage or Impossible Burger, and cultivated meats[1] have been identified as important means of reducing the public health, environmental, and animal welfare harms associated with animal-based meat production (Rubio et al., 2020). By providing competitive alternatives, these products might displace the consumption of animal-based meats. Since cultivated meats are not currently widely available on the public market, this paper will focus on plant-based meats, although many of the arguments might also apply to cultivated meats.
    Animal welfare, environmental, and public health advocates believe plant-based meats present a valuable opportunity to mitigate significant negative externalities of industrial animal agriculture, like animal suffering, greenhouse gas emissions, and antimicrobial resistance. For example, Animal Charity Evaluators lists "[cultivated] and plant-based food tech" as a priority cause area (Animal Charity Evaluators, 2022b), and a 2018 survey of 30 animal advocacy leaders and researchers ranked creating plant-based (and cultivated) meats third (after only research and corporate outreach) in their top priorities (Savoie, 2018). Non-profits working to research and support plant-based and cultivated meat production have received millions of dollars in funding (Animal Charity Evaluators, 2022a; New Harvest, 2021). Hu et al. (2019) describes plant-based meats as a potentially "vital" means to reduce the risks of diabetes, cardiovascular disease, and some cancers.
    Others have focused on reducing the climate impact of food production and "the need to de-risk global food systems" (Zane Swanson et al., 2023). The private and public sectors have taken note as well; in 2022, the "plant-based meat, seafood, eggs, and dairy companies" foods industry attracted at least $1.2 billion in private investment activity and at least $874 million in public funding (The Good Food Institute, 2022, pp. 55, 85-88).
    This enthusiasm has been propelled in some significant part by the informa...
    1 hr 12 min
  • EA - Unveiling the Longtermism Framework in Islam: Urging Muslims to Embrace Future-Oriented Values through 'Islamic Longtermism' by Zayn A
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Unveiling the Longtermism Framework in Islam: Urging Muslims to Embrace Future-Oriented Values through 'Islamic Longtermism', published by Zayn A on August 15, 2023 on The Effective Altruism Forum.
    I. Introduction
    Longtermism is the notion that those who will live in the long-term future matter just as much, morally, as those beings that are alive today; that, although current conceptions of who society ought to care about appears restricted to living people, this understanding must change because there is, on society at large, a moral obligation to care, and to ensure that future people are able to live their lives satisfactorily.[1] MacAskill coined the term Longtermism in 2017, in a bid to help concretize the idea of caring about future people into one word that helped make it sound like a consequential movement.[2]
    This, however, is not to say that the ideas that longtermism espouses are particularly brand new. To the contrary, it has been admitted that longtermism builds on a rather long historical concern for future people.[3]
    A perception that may be gleaned is that longtermism is a 'Western' idea, so to speak. In this regard, one need not look further than the leading scholarship in this area,[4] where most of the literature comes from, who it talks about, and the solutions espoused[5] - all seem to originate from Western perspectives. Consequently, longtermism could perhaps mistakenly be seen as a completely novel concept emerging from the West, and its recent surge in popularity only serves as evidence to this notion.[6] .
    In this piece, I seek to illustrate how Islam, and the initial flagbearers of Islam,[7] have historically preached tenets of longtermism, despite the religion finding inception some several centuries ago.[8] In fact, I will show in this piece that, not only is there a moral obligation on societies to care about future people, but that Islam, the Prophet, the Caliphs and even its prominent leaders place this obligation as one that is of great importance. This will be done with the following end in mind: that to convince Muslims to think in longtermist ways, the best method will be to find evidence that the idea finds some backing within Islamic teaching.
    II. Justifying the Study and Laying Groundwork
    At the very onset, one may question the very utility of this piece, by questioning why involving an Islamic perspective is at all necessary. In other words, what are the ramifications of an Islamic viewpoint of longtermism? In response to such argumentative resistance, one may merely resort to stating that this discussion is important for its own sake. That may be true. However, there are certainly a deeper reasons.
    Since its inception in 2017, when MacAskill offered the term 'longtermism', the main impetus pushing him to coin this word, essentially, was to make it appealing as a movement.[9] He opined that the previous renditions of phrases and terms that were used to appeal to longtermism, as we now know it, were 'a mishmash'. The entire point of the term and its meaning, was to make it 'attractive' as a movement for the wider public.[10] From the foregoing, it becomes clear that the conceptualizers of the term sought to find ways to make longtermism appeal to the public, presumably so that the movement gains support. In fact, it could be said that various undertakings have since occurred to do just that: increase the movements following. Whether using moral arguments, or other ideological arguments, much has been done to try and popularise the longtermist thought.
    Some discussion, on the Effective Altruism Forum, as well as other informal platforms, has taken place on how it may be possible to bring in the Christian contingent onboard to the idea of Effective Altruism and longtermism.[11] But, this avenue, using the Christian religion to ga...
    36 min
  • EA - Should I train to become a mediator for the EA ecosystem? by Severin
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Should I train to become a mediator for the EA ecosystem?, published by Severin on August 15, 2023 on The Effective Altruism Forum.
    Mediators help individuals and organizations resolve conflict. They serve as a buffer when strong emotions come up, help the conflict parties clarify their wants (which are often enough more compatible than it seems at first sight), and help make clear agreements.
    Over the years, I've heard of conflicts within a variety of EA organizations. Professional mediation might have helped resolve many of them earlier and more smoothly.
    I already bring a solid background in facilitation, counseling, and minimal training in mediation. In addition, my next career step is open.
    Accordingly, I'm currently thinking of training up to become a mediator for the EA/rationalist ecosystem. Services I'd include in that role would be:
    Offering conflict mediation for individuals and organizations.
    Giving workshops for EAs on how to disagree better.
    Offering communication trainings for organizations to a) build healthy team cultures that prevent conflict in the first place, and b) transform friction into clarity and cooperation. (I'm already working on formats for this with the team behind AuthRev and the Relating Languages.)
    Do you think this is a good idea? If yes, my next step would be to apply for 6-12 months of transition funding in order to do specialized skill-building and networking.
    Here are the reasons for and against this that I've come up with so far:
    Reasons for
    Especially after last year, there are a some boiling conflict lines within EA. And, the agile startup environments of EA orgs offer plenty potential for friction.
    It may be valuable to have an "in-house" mediater who has an in-depth understanding of EA culture, the local conflict lines, etc.
    As far as I know, no one else currently specializes in this.
    While the average EA likes to have controversial intellectual debates, I perceive the community as relatively conflict-averse when things get emotional. I tend to enjoy conflict and have an easy time trusting the process. I think that's useful for filling this role.
    Trust in EA leadership seems to be at an all-time low. While I've heard that CEA's community health team is remarkably good at not being partisan, some people might be more comfortable with having an EA mediator who is not directly involved with CEA.
    Reasons against
    It may be hard to convince those who'd profit from mediation to actually make use of it. (Just as with therapy or coaching.) I.e., there might not actually be a market for this.
    Subcultural knowledge may be less important than I think. External mediators may be able to fulfill this role just fine.
    The community health team, as well as the current coaches and therapists in EA, might already be sufficiently skilled and a sufficiently obvious address in case of conflict.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org
    3 min
  • EA - Why some people disagree with the CAIS statement on AI by David Moss
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Why some people disagree with the CAIS statement on AI, published by David Moss on August 15, 2023 on The Effective Altruism Forum.
    Summary
    Previous research from Rethink Priorities found that a majority of the population (59%) agreed with a statement from the Center for AI Safety (CAIS) that stated "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war." 26% of the population disagreed with this statement. This research piece does further qualitative research to analyze this opposition in more depth.
    The most commonly mentioned theme among those who disagreed with the CAIS statement was that other priorities were more important (mentioned by 36% of disagreeing respondents), with climate change particularly commonly mentioned.
    This theme was particularly strongly occurring among younger disagreeing respondents (43.3%) relative to older disagreeing respondents (27.8%).
    The next most commonly mentioned theme was rejection of the idea that AI would cause extinction (23.4%), though some of these respondents agreed AI may pose other risks.
    Another commonly mentioned theme was the idea that AI was not yet a threat, though it might be in the future.
    This was commonly co-occurring with the 'Other priorities' theme, with many arguing that other threats were more imminent.
    Other less commonly mentioned themes included that AI would be under our control (8.8%) and so would not pose a threat, while another was that AI was not capable of causing harm, because it was not sentient or sophisticated or autonomous (5%).
    Introduction
    Our previous survey on attitudes on US public perception of the CAIS statement on AI risk found that a majority of Americans agree with the statement (59%), while a minority (26%) disagreed. To gain a better understanding of why individuals might disagree with the statement, we ran an additional survey, where we asked a new sample of respondents whether they agreed or disagreed with the statement, and then asked them to explain why they agreed or disagreed. We then coded the responses of those who disagreed with the statement to identify major recurring themes in people's comments. We did not formally analyze comments from those who did not disagree with the statement, though may do so in a future report.
    Since responses to this question might reflect responses to the specifics of the statement, rather than more general reactions to the idea of AI risk, it may be useful to review the statement before reading about the results.
    "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war."
    Common themes
    This section outlines the most commonly recurring themes. In a later section of this report we'll discuss each theme in more detail and provide examples from each. It is important, when interpreting these percentages to remember that they are percentages of those 31.2% respondents who disagreed with the statement, not of all respondents.
    The dominant theme, by quite a wide margin, was the claim that 'Other priorities' were more important, which was mentioned by 36% of disagreeing respondents. The next most common theme was 'Not extinction', mentioned in 23.4% of responses, which simply involved respondents asserting that they did not believe that AI would cause extinction. The third most commonly mentioned theme was 'Not yet', which involved respondents claiming that AI was not yet a threat or something to worry about. The 'Other priorities' and 'Not yet' themes were commonly co-occurring, mentioned together by 7.9% of respondents, more than any other combination.
    Some less commonly mentioned themes were 'Control', the idea that AI could not be a threat because it would inevitably be under our...
    28 min
  • EA - EA Organization Updates: August 2023 by JP Addison
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: EA Organization Updates: August 2023, published by JP Addison on August 15, 2023 on The Effective Altruism Forum.
    We're featuring some opportunities and job listings at the top of this post. Some have (very) pressing deadlines.
    You can see previous updates on the "EA Organization Updates (monthly series)" topic page, or in our repository of past newsletters. Notice that there's also an "org update" tag, where you can find more news and updates that are not part of this consolidated series.
    These monthly posts originated as the "Updates" section of the EA Newsletter. Organizations submit their own updates, which we edit for clarity.
    (If you think your organization should be getting emails about adding their updates to this series, please apply here.)
    Opportunities and jobs
    Opportunities
    Consider also checking opportunities listed on the EA Opportunities Board.
    Applications are open for a number of conferences
    EA Global: Boston (27-29 October) is for people who have a solid understanding of effective altruism, and who are taking significant actions on the basis of key ideas in the movement. Apply by 13 October.
    EAGxBerlin, (8-10 September) is aimed at people in Western Europe. Tickets cost €0-80. Apply by 18 August.
    EAGxAustralia (22 - 24 September) is for people in Australia and New Zealand. Tickets are $75-150 (AUD). Apply by 8 September.
    EAGxPhilippines (20 - 22 October) is for people in South East and East Asia. Tickets are $0-100. Apply by 30 September.
    The Good Food Conference 2023 will be back in-person this year at the historic Fort Mason Center in San Francisco on Sept. 18-20.
    This year's program is bursting with insights and inspiration from big-picture plenaries and fascinating flash talks to in-depth technical sessions and solutions-focused workshops led by a stellar lineup of scientists, policymakers, private sector leaders, and other brilliant humans.
    Check out the detailed program to learn more, and register here to help build a future where alternative proteins are no longer alternative.
    Opportunities to take action
    SoGive is conducting a survey of people who (have the capacity to) give £10k+. They are trying to help the community fill the gaps in support so more major donors can give more, and give more effectively. Read more here.
    Job listings
    Consider also exploring jobs listed on "Job listing (open)."
    Centre for the Governance of AI
    Head of Operations (Mostly Oxford, £60K - £80K +benefits, apply by 21 August)
    Epoch
    ML Hardware Researcher (Remote, $70K - $120K apply by 20 August)
    Family Empowerment Media
    Project Director (Remote with international travel, apply by 21 August)
    Fish Welfare Initiative
    Operations Lead/Associate (Remote in a time-zone compatible with IST, apply by 3 September)
    GiveWell
    Senior Researchers (Remote or Oakland, California; $193.1K - $209K)
    Content Editors (Remote or Oakland, California; $90.6K - $98K)
    IDinsight
    Associate and Senior Associate (Philippines)
    Technical Delivery Manager/Director (India or Kenya)
    Social Scientist/Economist - (Morocco or Senegal)
    Open Philanthropy
    Operations Associate - Biosecurity & Pandemic Preparedness (Washington, DC, $114.2k)
    Recruiter (Remote, San Francisco/Washington, DC, $108.3k)
    Organization Updates
    80,000 Hours
    This month on The 80,000 Hours Podcast, Rob interviewed:
    Ezra Klein on existential risk from AI and what DC could do about it
    Holden Karnofsky's four-part playbook for dealing with AI
    And Luisa interviewed:
    Hannah Boettcher on the mental health challenges that come with trying to have a big impact
    Markus Anderljung on how to regulate cutting-edge AI models
    80,000 Hours also re-released an updated version of a classic three-part article series written by Gregory Lewis, exploring how many lives a doctor saves. They also updated their article on working in US AI policy.
    Anima Internati...
    15 min
  • EA - EA Poland 2023 - the first update by EA Poland
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: EA Poland 2023 - the first update, published by EA Poland on August 15, 2023 on The Effective Altruism Forum.
    TL;DR: EA Poland is active and growing. Our aim is to build and maintain a supportive and empowering community that enables individuals to plan their next steps to make a positive impact. We want to achieve this by focusing on three areas: Community Building, Career Planning, and Project Incubation.
    Introduction
    At the beginning of June we had our first Polish EAGx, in Warsaw. We think this is a good opportunity to introduce EA Poland. Before we start, we would like to mention that this post's structure was inspired by EA Estonia's post and was prepared with the support of ChatGPT and DeepL/Writer.
    Quick Facts About Poland
    Located in Central Europe, Poland has a population of ~38 million and has been classified by the World Bank as a high-income country since 2009. Additionally, there are up to 21 million Polish people living across the world. We love potatoes, onions, and dumplings with onions and potatoes.
    Quick Facts About EA Poland
    The first EA meetup we know of took place in 2014, and the first group was established in Cracow, in 2015. In 2018, the Polish Foundation for Effective Altruism (FEA) was registered. We have members from all over Poland: Cracow, Warsaw, Wroclaw, Poznan, Gdansk, Katowice, Lodz...
    Current status
    EA Poland in numbers:
    3 employees, with a total of 1.5 FTE (full-time equivalents; funded by the EA Infrastructure Fund)
    ~30 volunteers (incl. group organisers)
    ~60 highly-engaged EAs - HEAs (incl. most of the volunteers)
    178 members registered on our community Slack
    Of these, ~70 users are active on a weekly basis (figure 2)
    336 newsletter subscribers
    Social media:
    2,700 Facebook fanpage followers
    1,100 Facebook group members
    515 Instagram followers
    893 Linkedin followers
    211 YouTube subscribers
    3 local groups (Warsaw, Cracow, Wroclaw)
    1 active university group (University of Warsaw) and 1 university group expected to start in 2023/2024 (AGH in Cracow)
    An online book club (we plan to publish a separate forum post about it)
    2 AI Safety Groups (on-line): technical and governance
    2 EA-aligned organisations (Otwarte Klatki [Anima International Poland] and Optimum Pareto Foundation)
    What we did last year (June 2022 - June 2023)
    Organisational Structure and Operations
    Over the course of the past year, we channelled a lot of effort into strengthening our organisational structure and operational capacity. This includes establishing basic processes and procedures in areas such as IT, finance, and legal; reintroducing Asana for task management; and revitalising our social media and newsletter. We collectively agreed to form an executive board where two Co-Directors represent FEA for a period of two years. After this period, re-election of the Co-Directors requires approval, granted by the Active Members via a vote. To evaluate and support the work of the Co-Directors, we have set up a Supervisory Board consisting of five HEAs, elected by the Active Members (employees, volunteers, interns, groups organisers, projects coordinators).
    Mission, Vision, and Strategy
    In 2022, we defined our strategy through a workshop and multiple group discussions. In November 2023, we will revise it and determine an action plan for the next two years. We intend to write a separate post on this topic.
    Organisational Culture
    Creating a safe and inclusive environment is a priority for EA Poland. We developed a Code of Conduct through workshops, and introduced the Safety Person role to ensure the well-being and comfort of all community members. We also conducted a comprehensive survey to examine our organisational culture, with the aim of continuous improvement and fostering a supportive community. The survey, consisting of 72 questions, had an average score of 3.55 on a s...
    16 min
  • LW - A short calculation about a Twitter poll by Ege Erdil
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A short calculation about a Twitter poll, published by Ege Erdil on August 14, 2023 on LessWrong.
    Recently, the following poll was posted on Twitter:
    Everyone responding to this poll chooses between a blue pill or red pill.
    if > 50% of ppl choose blue pill, everyone lives
    if not, red pills live and blue pills die
    Which do you choose?
    If we linearize the problem of how much you value the lives of other poll respondents compared to your own, there's actually a way to quantify whether you should vote red or blue depending on your beliefs about how the rest of the population will vote.
    Suppose that we normalize the value of choosing the red pill at zero, and let N be the total number of people responding to the poll excluding you. Then, choosing the blue pill only makes a difference in the following two cases:
    If the number of people who vote blue excluding you, hereafter denoted B, is exactly ⌊N/2⌋, then your vote saves the lives of ⌊N/2⌋ people.
    If B<⌊N/2⌋, then you die and get nothing in return.
    If you value the lives of other poll respondents at a constant multiple ν of your own life, then you should pick blue if
    P(B=⌊N/2⌋)(ν⌊N/2⌋)>P(B<⌊N/2⌋)
    and you should pick red if the inequality goes in the other direction.
    A rough heuristic here is that if the distribution of B is unimodal with a maximum to the right of ⌊N/2⌋, you expect
    ⌊N/2⌋P(B=⌊N/2⌋)≥P(B<⌊N/2⌋)
    so if ν=1 (meaning you value the lives of other poll respondents equally to your own) and you expect a blue majority in the precise sense defined above, you should always take the blue pill yourself. The exact cutoff on ν for you to take the blue pill is
    ν≥P(B<⌊N/2⌋)P(B=⌊N/2⌋)⌊N/2⌋
    A general heuristic is that if we think a red victory is a distinct possibility, so that P(B<⌊N/2⌋)=O(1), and the distribution doesn't decay too sharply around N/2, in general, we'll have P(B=⌊N/2⌋)=O(1/N) so that the cutoff ends up being ν≥O(1). In other words, you pick blue in realistic circumstances if you value other people's lives similarly to your own: ν can't be too far away from 1 if picking blue is to make sense as an individually altruistic decision, ignoring social pressures to pick blue et cetera.
    Approximating B/N by a beta distribution for large N gives the following rough results:
    Probability of a red victory Expected share of blue votes Cutoff value of 1/ν, i.e. minimum value of your life in units of others' lives to choose red 10% 55% 22.4 10% 65% 7.1 10% 75% 3.7 20% 55% 11.6 20% 65% 3.5 20% 75% 1.4
    I think the degree of altruism implied by choosing blue in this context is pretty strong for plausible distributions over B/N. For that reason, I think picking red is easily defensible even from an individually altruistic point of view (would you sacrifice your life to save the life of five strangers?)
    There are higher-order considerations that are relevant beyond individual altruism, of course: society might have a set of norms to impose penalties on people who choose to take the red pill. However, the possible cost of not taking the red pill is losing your life, which suggests that such penalties would have to be quite steep to change these basic calculations as long as there is a non-negligible probability of a red victory that survives.
    I suspect that if this were a real situation most people would choose to take the red pill in a situation where monitoring costs are high (e.g. which pill you take is a secret decision unless red wins) and social punishments are therefore difficult to enforce.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org
    4 min
  • LW - Decomposing independent generalizations in neural networks via Hessian analysis by Dmitry Vaintrob
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Decomposing independent generalizations in neural networks via Hessian analysis, published by Dmitry Vaintrob on August 14, 2023 on LessWrong.
    In our joint SERI MATS project, we came up with a series of equations and experiments to mechanistically understand and steer the generalization behavior of neural nets. The core conceit is to locate the circuits (which we call "modules") responsible for implementing different generalizations using a toolbox of techniques related to Hessian eigenvectors. This is a general-audience distillation of our work.
    We hope most of the ideas and high-level goals are understandable to a non-expert, though, for most of our experiments, we attempt to include "supplementary" material with the main mathematical intuitions and concrete equations that would allow someone to reproduce our work. We plan in the coming weeks to write multiple follow-up distillations and discussions, both of some of the more technical parts of our work and of a few new insights into generalization behavior and phase transitions in general that came out of experiments involving our Hessian toolbox.
    Introduction
    A central problem for inner alignment is understanding how neural nets generalize off-distribution. For example, a powerful AI agent trained to make people happy can generalize either by choosing actions that deceptively look good to its overseers or those that truly align with human values. The same diversity of generalization is already seen in existing real-world tasks both in minor ways (image classifiers classifying cars by learning to recognize wheels vs. windows) and in serious ways (language models appearing honest by agreeing with the user vs. insisting on consensus opinions).
    One approach to steer between generalizations is activation steering, which Nina is investigating as her other SERI MATS project. This aims to encourage the neural net to implement one possible generalization (in this case, honestly reflecting the LLM's internal world model) instead of the other generalization (in this case, sounding good and correct to a particular user).
    While activation steering, supervised finetuning, and RLHF work well in practice and can make systems behave better, there is still a risk that powerful models generalize in unpredictable and potentially undesirable ways in out-of-distribution examples. In particular, for subtle alignment-related questions like deception or power-seeking, activation steering or RLHF may fix the "symptoms" of the problem on examples similar to the training corpus but may fail to fix the "underlying cause" and achieve aligned behavior.
    A somewhat ambitious alternative way to get at the "root" of a generalization problem instead of fixing its symptoms is to try to access it on a mechanistic level. Namely, imagine that on the level of the "internal architecture" of the neural net (something that is notoriously hard to access but can sometimes be partially interpreted), the two generalizations get executed by at least somewhat independent modules (i.e., parallel circuits: the term comes from this paper). If we were able to identify and split up these two modules cleanly, we might be able to find weight perturbation vectors that destroy ("ablate") one of them while preserving the other. The resulting method is now provably robust: it prevents one of the generalizations (understood mechanistically as the underlying module) from getting executed at any level, thus solving both the symptom and the underlying cause.
    This algorithm for tuning generalizations can be possible only if the underlying mechanistic model (of different independent generalization "modules" which can be consistently found and independently ablated) is correct or partially correct to a relevant approximation. In order to even begin to engage with it, we need answe...
    25 min