The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • EA - What might FTX mean for effective giving and EA funding by Jack Lewars
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What might FTX mean for effective giving and EA funding, published by Jack Lewars on November 15, 2022 on The Effective Altruism Forum.
    FTX's demise is understandably dominating EA discussion at the moment. Like everyone in the community, I’m primarily just really sad about the news. A lot of people have been hurt; and a lot of funding that could have been used to do a staggering amount of good is no longer available.
    I’m also aware of the dangers of publishing reactions too quickly. Many of the facts about FTX are not clear. However, the major effect of recent events on EA funding seems fairly unambiguous: there is substantially less funding available to EA today than there was a week ago.
    Accordingly, I think we can lay out some early food for thought from the last year or so; and hopefully start an evolving conversation around funding in EA, and effective giving more broadly, as we learn more.
    (P.s. Much of what is below has been said before but it seems helpful to summarize it in one place. If it’s your work I’m drawing on, I hope you see this as validation of what you saw that others didn’t. I’ll link to you where I can!)
    We need to aim for greater funding diversity and properly resource efforts to achieve this
    Saying “we should diversify our funding” is almost cringingly obvious in the current circumstances. But a lot of the discussion about funding over the last year seemed not to acknowledge the risks when ~85% of expected funding is supplied by two pots of money; and when those pots are effectively reliant on the value of 2-3 highly volatile assets. Even when these issues were explicitly acknowledged, they didn’t seem to influence the way funding was forecasted sufficiently - discussions rarely seemed to consider a future where one or both sources of funding rapidly declined in value. We may find out in the coming months that grantees didn’t consider this risk sufficiently either, for example if they bought fixed assets with significant running costs.
    Funding diversity is important not just to protect the overall amount of funding available to EA. It’s also vital to avoid a very small number of decision-makers having too much influence (even if they don’t want that level of influence in the first place). If we have more sources of funding and more decision-makers, it is likely to improve the overall quality of funding decisions and, critically, reduce the consequences for grantees if they are rejected by just one or two major funders. It might also reduce the dangers of money unduly influencing moral and philosophical views in a particular direction.
    In short: it's still important to find new donors, even if the amount of existing funding is very large in expectation.
    Of course, it is significantly easier to say we should diversify than actually to diversify. So what could we do to try to mitigate the risks of overly concentrated sources of funding?
    Bring more donors at every level into EA
    It seems wise to support organizations that can broaden the funding base of EA. Recently, there has been a move to prioritize securing major donors (and, in some extreme cases, to suggest that anything attracting small donors is a waste of time). Seeking more super wealthy donors is a sensible move (see below), but there is a reasonable case for seeking more donors in general at the same time. More donors in EA means more reliable, less volatile funding overall. Additionally, even if the amounts they give seem trivial by comparison, small donors can still make a difference, for example by funging dollars back to larger donors, who can then give more to things that are less popular, more technical or higher risk.
    To this end, we should celebrate, encourage and (most importantly) fund the wide range of projects that attract effective givers - pledge organizations (One for th...
    15 min
  • EA - Brainstorming ways to make EA safer and more inclusive by richard ngo
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Brainstorming ways to make EA safer and more inclusive, published by richard ngo on November 15, 2022 on The Effective Altruism Forum.
    After some recent discussion on the forum and on twitter about negative experiences that women have had in EA community spaces, I wanted to start a discussion about concrete actions that could be taken to make EA spaces safer, more comortable, and more inclusive for women. The community health team describes some of their work related to interpersonal harm here, but I expect there's a lot more that the wider community can do to prevent sexual harrassment and abusive behavior, particularly when it comes to setting up norms that proactively prevent problems rather than just dealing with them afterwards. Some prompts for discussion:
    What negative experiences have you had, and what do you wish the EA community had done differently in response to them?
    What specific behaviors have you seen which you wish were less common/wish there were stronger norms against? What would have helped you push back against them?
    As the movement becomes larger and more professionalized, how can we enable people to set clear boundaries and deal with conflicts of interest in workplaces and grantmaking?
    How can we set clearer norms related to informal power structures (e.g. people who are respected or well-connected within EA, community organizers, etc)?
    What codes of conduct should we have around events like EA Global? Here's the current code; are there things which should be included in there that aren't currently (e.g. explicitly talking about not asking people out in work-related 1:1s)?
    What are the best ways to get feedback to the right people on an ongoing basis? E.g. what sort of reporting mechanisms would make sure that concerning patterns in specific EA groups get noticed early? And which ones are currently in place?
    How can we enable people who are best at creating safe, welcoming environments to share that knowledge? Are there specific posts which should be written about best practices and lessons learned (e.g. additions to the community health resources here)?
    I'd welcome people's thoughts and experiences, whether detailed discussions or just off-the-cuff comments. I'm particularly excited about suggestions for ways to translate these ideas to concrete actions going forward.
    EDIT: here's a google form for people who want to comment anonymously; the answers should be visible here. And feel free to reach out to me in messages or in person if you have suggestions for how to do this better.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    3 min
  • AF - The economy as an analogy for advanced AI systems by rosehadshar
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The economy as an analogy for advanced AI systems, published by rosehadshar on November 15, 2022 on The AI Alignment Forum.
    Eric Drexler’s Comprehensive AI Services (CAIS), particularly as set out in his 2019 report Reframing Superintelligence, is a complex model with many different assumptions and implications. It’s hard to grasp the whole thing at once, and existing summaries are brief and partial.[1]
    One way of trying to understand CAIS is to seek generative intuitions for the whole model. These intuitions will be imprecise, but they can also make it easier to see why one might end up thinking that something like CAIS made sense.
    In this post, we offer one such generative intuition for CAIS: using the economy rather than rational agents as an analogy for thinking about advanced AI systems.
    Note that:
    We are not making the historical claim that thinking about economies was in fact the main generator of Drexler’s thinking on CAIS.[2]
    There are other generative intuitions for CAIS, and other bodies of theory which the CAIS model is rooted in.[3]
    The basic analogy
    An economy is an abstraction for the sum total of ‘the production, distribution and trade, as well as consumption of goods and services’. Prescriptively, we want the economy to serve human needs and preferences - and it does this at least to some extent.
    Prescriptively, we also want advanced AI systems to serve human needs and preferences.
    In worlds where we get advanced AI systems right, they would therefore be serving a similar function to the economy: serving human needs and preferences.
    Whether we get AI right or not, it seems likely that advanced AI systems will become heavily integrated with the economy, such that it might become hard to distinguish them.
    It therefore seems reasonable to imagine advanced AI systems in analogy with the economy, and to use what we know about economic dynamics to reason about dynamics which might shape those systems.
    In the modern economy, specialised services are provided by a range of entities, mostly companies and governments. We don’t see one giant global monopoly providing all services.
    Thinking analogically about advanced AI systems, the CAIS model expects an array of specialised AI services working on decomposed tasks, rather than a single generally superintelligent agent (a global monopoly in the base metaphor).
    This can be further unpacked. The reason that the human economy isn’t structured as a global monopoly is that specialisation is efficient. It’s often cheaper for an organisation to outsource a particular service, than to develop that capability in house: imagine family run businesses trying to manufacture their own smartphones from scratch, or big companies all hiring software engineers to develop their own internet search engines. So we end up with a range of different companies providing different services.
    Note that specialisation isn’t always the most efficient thing, because of economies of scope: cases where the unit cost of producing something decreases as the variety of products increases. Maybe you’ve already built a petrol station to sell petrol, and selling snacks too is cheap at the margin. Or you have a factory which makes women’s shoes, and starting a men’s line is pretty efficient. Here, there are joint costs which get shared across the different products, and so you get economies of scope.
    But economies of scope don’t seem to apply across the whole economy - otherwise we’d see the giant global monopoly. (Part of the reason here is that coordination costs increase with the size of an organisation, such that decentralised ways of sharing information like price signals are more efficient than centralised information transfer.)[4]
    In the advanced AI analogy, Drexler argues that it will be more efficient for a given AI service to coordinate with...
    10 min
  • EA - Want advice on management/organization-building? by Ben Kuhn
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Want advice on management/organization-building?, published by Ben Kuhn on November 15, 2022 on The Effective Altruism Forum.
    Someone observed to me recently that there are a lot of new EA organizations whose founders don't have much experience building teams and could benefit from advice from someone further along in the scaling process. Is this you, or someone you know? If so, I'd be interested in talking about your management/org-building challenges :) You can reach out through the forum's messaging feature or email me (ben dot s dot kuhn at the most common email address suffix).
    About me / credentials: I'm the CTO of Wave, a startup building financial infrastructure for unbanked people in sub-Saharan Africa. I joined as an engineer in the very early days (employee #6), became CTO in 2019, and subsequently grew the engineering team from ~2 to 70+ engineers while the company overall scaled to ~2k people. Along the way I had to address a large number of different team-scaling problems, both within engineering and across the rest of Wave. I also write a blog with some advice posts that people have found useful.
    Example areas I might have useful input on:
    Hiring: clarifying roles, writing job descriptions, developing interview loops, executing hiring processes, headcount planning...
    People management: coaching, feedback, handling unhappy or underperforming people, designing processes for things like performance reviews
    Organizational structure: grouping people into teams, figuring out good boundaries between teams, adding management layers
    About you: a leader at an organization that's experiencing (or about to experience) a bunch of growth, and could use advice on how to navigate the scaling problems that come with that.
    Structure: pretty uncertain since I've never done this before, but I'm thinking some sort of biweekly or weekly checkin (after an initial convo to determine fit—I'll be doing this on a volunteer basis with a smallish chunk of time, which means I may need to prioritize folks based on where I feel the most useful).
    Disclaimer: this is an experiment—I've never done this before, and giving good advice is hard, so I can't guarantee that I'll be useful :)
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    3 min
  • EA - The NY Times Interviewed SBF on Sunday by Lauren Maria
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The NY Times Interviewed SBF on Sunday, published by Lauren Maria on November 14, 2022 on The Effective Altruism Forum.
    The NY times did an interview with SBF yesterday that "stretched past midnight". Here are some quotes from the article: “Had I been a bit more concentrated on what I was doing, I would have been able to be more thorough,” he said. “That would have allowed me to catch what was going on on the risk side.”
    Mr. Bankman-Fried, who is based in the Bahamas, declined to comment on his current location, citing safety concerns. Lawyers for FTX and Mr. Bankman-Fried did not respond to requests for comment."Meanwhile, at a meeting with Alameda employees on Wednesday, Ms. Ellison explained what had caused the collapse, according to a person familiar with the matter. Her voice shaking, she apologized, saying she had let the group down. Over recent months, she said, Alameda had taken out loans and used the money to make venture capital investments, among other expenditures.
    Around the time the crypto market crashed this spring, Ms. Ellison explained, lenders moved to recall those loans, the person familiar with the meeting said. But the funds that Alameda had spent were no longer easily available, so the company used FTX customer funds to make the payments. Besides her and Mr. Bankman-Fried, she said, two other people knew about the arrangement: Mr. Singh and Mr. Wang."I "gifted" this article here, but if it doesn't work for some reason, I can post it again in the comments. Edited to add an extra quote
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min
  • LW - Will we run out of ML data? Evidence from projecting dataset size trends by Pablo Villalobos
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Will we run out of ML data? Evidence from projecting dataset size trends, published by Pablo Villalobos on November 14, 2022 on LessWrong.
    Summary: Based on our previous analysis of trends in dataset size, we project the growth of dataset size in the language and vision domains. We explore the limits of this trend by estimating the total stock of available unlabeled data over the next decades.
    Read the full paper in arXiv.
    Our projections predict that we will have exhausted the stock of low-quality language data by 2030 to 2050, high-quality language data before 2026, and vision data by 2030 to 2060. This might slow down ML progress.
    All of our conclusions rely on the unrealistic assumptions that current trends in ML data usage and production will continue and that there will be no major innovations in data efficiency. Relaxing these and other assumptions would be promising future work.
    Historical projectionCompute projectionLow-quality language stock2032.4[2028.4 ; 2039.2] 2040.5[2034.6 ; 2048.9]High-quality language stock2024.5[2023.5 ; 2025.7]2024.1[2023.2 ; 2025.3]Image stock2046[2037 ; 2062.8]2038.8[2032 ; 2049.8]Table 1: Median and 90% CI exhaustion dates for each pair of projections.
    Background
    Chinchilla's wild implications argued that training data would soon become a bottleneck for scaling large language models. At Epoch we have been collecting data about trends in ML inputs, including training data. Using this dataset, we estimated the historical rate of growth in training dataset size for language and image models.
    Projecting the historical trend into the future is likely to be misleading, because this trend is supported by an abnormally large increase in compute in the past decade. To account for this, we also employ our compute availability projections to estimate the dataset size that will be compute-optimal in future years using the Chinchilla scaling laws.
    We estimate the total stock of English language and image data in future years using a series of probabilistic models. For language, in addition to the total stock of data, we estimate the stock of high-quality language data, which is the kind of data commonly used to train large language models.
    We are less confident in our models of the stock of vision data because we spent less time on them. We think it is best to think of them as lower bounds rather than accurate estimates.
    Results
    Finally, we compare the projections of training dataset size and total data stocks. The results can be seen in the figure above. Datasets grow much faster than data stocks, so if current trends continue, exhausting the stock of data is unavoidable. The table above shows the median exhaustion years for each intersection between projections.
    In theory, these dates might signify a transition from a regime where compute is the main bottleneck to growth of ML models to a regime where data is the taut constraint.
    In practice, this analysis has serious limitations, so the model uncertainty is very high. A more realistic model should take into account increases in data efficiency, the use of synthetic data, and other algorithmic and economic factors.
    In particular, we have seen some promising early advances on data efficiency, so if lack of data becomes a larger problem in the future we might expect larger advances to follow. This is particularly true because unlabeled data has never been a constraint in the past, so there is probably a lot of low-hanging fruit in unlabeled data efficiency.
    In the particular case of high-quality data, there are even more possibilities, such as quantity-quality tradeoffs and learned metrics to extract high-quality data from low-quality sources.
    All in all, we believe that there is about a 20% chance that the scaling (as measured in training compute) of ML models will significantly slow down b...
    5 min
  • EA - Suggestion - separate out the FTX threads somehow by Arepo
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Suggestion - separate out the FTX threads somehow, published by Arepo on November 14, 2022 on The Effective Altruism Forum.
    In general the frontpage has felt overloaded to me, but that's a broader topic of which this is an acute example. For now, could we just quickly set up a way to stop other subjects from getting drowned out? Maybe just filter FTX-related posts from off the frontpage, with an announcement at the top that you've done so and linking to the FTX crisis tag?
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    1 min
  • EA - Money Stuff: FTX’s Balance Sheet Was Bad by Elliot Temple
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Money Stuff: FTX’s Balance Sheet Was Bad, published by Elliot Temple on November 14, 2022 on The Effective Altruism Forum.
    Matt Levine explains that FTX's balance sheet (which has been leaked) is a nightmare. Sample quote:
    And then the basic question is, how bad is the mismatch [between liabilities and assets on the balance sheet]. Like, $16 billion of dollar liabilities and $16 billion of liquid dollar-denominated assets? Sure, great. $16 billion of dollar liabilities and $16 billion worth of Bitcoin assets? Not ideal, incredibly risky, but in some broad sense understandable. $16 billion of dollar liabilities and assets consisting entirely of some magic beans that you bought in the market for $16 billion? Very bad. $16 billion of dollar liabilities and assets consisting mostly of some magic beans that you invented yourself and acquired for zero dollars? WHAT? Never mind the valuation of the beans; where did the money go? What happened to the $16 billion? Spending $5 billion of customer money on Serum would have been horrible, but FTX didn’t do that, and couldn’t have, because there wasn’t $5 billion of Serum available to buy.
    FTX shot its customer money into some still-unexplained reaches of the astral plane and was like “well we do have $5 billion of this Serum token we made up, that’s something?” No it isn’t!
    Mirror to avoid paywall: (Loading the article in a private browsing window would probably work too.)
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min
  • EA - Stop Thinking about FTX. Think About Getting Zika Instead. by jeberts
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Stop Thinking about FTX. Think About Getting Zika Instead., published by jeberts on November 14, 2022 on The Effective Altruism Forum.
    (Or about getting malaria, or hepatitis C (see below), or another exciting disease in an artisanal, curated list of trials in the UK, Canada, and US by 1Day Sooner.)
    Hi! My name is Jake. I got dysentery as part of a human challenge trial for a vaccine against Shigella, a group of bacteria that are the primary cause of dysentery globally. I quite literally shtposted through it on Twitter and earned fifteen minutes of Internet fame.
    I now work for 1Day Sooner, which was founded as an advocacy group in early 2020 for Covid-19 human challenge trials. (Who knew dysentery could lead to a career change?) 1Day is also involved in a range of other things, including pandemic preparedness policy and getting hepatitis C challenge trials off the ground.
    What I want to focus on right now is Zika. Specifically, I want to convince people assigned female at birth aged 18-40 in the DC-Baltimore/DMV area reading this post to take a few minutes to consider signing up for screening for the first-ever human challenge trial for Zika virus (ZIKV) at Johns Hopkins University in Baltimore.
    Even if you fall outside that category, I figure this might be something interesting to ponder over, and probably less stressful than the cryptocurrency debacle that shall not be named.
    1Day Sooner is not compensated in any way by the study/Hopkins for this, nor do I/we represent the study in any official sense. I happen to have become very fascinated by this one in particular because it represents how challenge trials can be used for pandemic prevention with comparatively few resources. This post is meant to inform you of the study with a bit more detail from an EA perspective, but does not supplant information provided by the study staff.
    (If you’re a DMV male like me bummed you can’t take part, ask me about current or upcoming malaria vaccine and monoclonal antibody trials taking place at the University of Maryland — I'll be screening next week! If you’re not in the DMV or otherwise just can’t do anything for Zika or malaria, you can still sign up for our volunteer base and newsletter, which will help you keep tabs on future studies. Something we’re very excited about is the emerging push for hepatitis C challenge studies, see the link above.)
    Zika 101
    Zika is a mainly mosquito-borne disease that has been known since 1947 — The 2015-2016 western hemisphere epidemic showed that Zika could cause grave birth defects (congenital Zika syndrome, CZS) — The disease is very mild in adults at present
    The Zika virus (ZIKV) was discovered in the Zika forest of Uganda in 1947, and a few years later, we learned it could cause what was universally assumed to be extremely mild disease in humans. The 2015-16 Zika epidemic that started in South America, and which was particularly severe in Brazil, proved otherwise. This is when it became clear that Zika was linked to horrific birth defects. Zika has since earned its place on the WHO priority pathogen list.
    To briefly review the basics of Zika:
    The Zika virus is a flavivirus, in the same family as dengue fever, yellow fever, and Chikungunya, among others. Flaviviruses are RNA viruses, which generally lack genomic proofreading ability and are thus more prone to mutation.
    Zika infection sometimes causes Zika fever, though asymptomatic cases are very common. Zika fever is usually very, very mild, and direct deaths are extremely rare.
    Zika is much more of a concern if you are pregnant — in about 7% of infections during pregnancy, the virus infects a fetus and causes serious, even fatal, birth defects. These defects, which most frequently include microcephaly, are referred to as congenital Zika syndrome (CZS).
    Zika is also thought to increase t...
    23 min
  • EA - Proposals for reform should come with detailed stories by Eric Neyman
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Proposals for reform should come with detailed stories, published by Eric Neyman on November 14, 2022 on The Effective Altruism Forum.
    Following the collapse of FTX, many of us have been asking ourselves:
    What should members of the EA community have done differently, given the information they had at the time?
    I think that proposed answers to this question are considerably more useful if they meet (and explicitly address) the following three criteria:
    (1) The (expected) benefits of the proposal outweigh the costs, given information available at the time.
    (2) Implementing the proposal would have been realistic.
    (3) There is a somewhat detailed, plausible story of how the proposal could have led to a significantly better outcome with regard to the FTX collapse.
    Hopefully it's clear why we want the first two of these. (1) is EA bread and butter. And (2) is important to consider given how decentralized EA is: if your proposal is "all EAs should refuse funding from a billionaire without a credible audit of their finances", you face a pretty impossible coordination problem.
    But most of the proposals I've seen have been failing (3). Here's a proposal I've heard:
    The people SBF was seeking out to run the FTX Foundation should have requested an audit of FTX's finances at the start.
    This isn't enough for me to construct a detailed, plausible story for things going better. In particular, I think one of two things would have happened:
    The FTX leadership would have agreed to a basic audit that wouldn't have uncovered underlying problems (much as the entire finance industry failed to uncover the problems).
    The FTX leadership would have refused to accede to the audit.
    In the first case, the proposal fails at accomplishing its goal. In the second case, I want more details! What should the FTX Foundation people have done in response, and how could their actions have led to a better outcome?
    A more detailed proposal:
    The people SBF was seeking out to run the FTX Foundation should have requested a thorough, rigorous audit of FTX's finances at the start. If FTX leadership had refused, they should have refused to run the FTX Foundation.
    This still isn't detailed enough: SBF would have just asked someone else to run the Foundation. It seems like the default outcome is that someone who's a bit worse at the job would have been in charge instead.
    A more detailed proposal:
    The people SBF was seeking out to run the FTX Foundation should have requested a thorough, rigorous audit of FTX's finances at the start. If FTX leadership had refused, they should have refused to run the FTX Foundation and made it public that FTX leadership had refused the audit. Then, EA leaders should have discouraged major EA organizations from taking money from the FTX Foundation and promoted a culture of looking down on anyone who took money from the Foundation.
    This is starting to get to the point where something could have realistically changed as a result of the actions. Maybe the pressure for transparency would have been strong enough that SBF would have acceded to an audit -- though I still think the audit wouldn't have uncovered anything. Or maybe he wouldn't have acceded, and for a while EA organizations would have refused his money, before eventually giving in to the significant incentives to take the money. Or maybe they would have refused his money for many years -- in that case, I would question whether criterion (1) is still satisfied (given only information we would have had at the time, remember). But at least we have a somewhat detailed story to argue about.
    Without a detailed story, it's easy to convince yourself that the change you propose would have plausibly averted the disaster we face. A detailed story forces you to spell out your assumptions in a way that makes it easier for other people to poke ...
    5 min