The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • EA - Some problems in operations at EA orgs: inputs from a dozen ops staff by Vaidehi Agarwalla
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Some problems in operations at EA orgs: inputs from a dozen ops staff, published by Vaidehi Agarwalla on March 16, 2023 on The Effective Altruism Forum.
    This is a brief summary of an operations brainstorm that took place during April 2022. It represents the views of operations staff at 8-12 different EA-aligned organizations (approximately). We split up into groups and brainstormed problems, and then chose the top problems to brainstorm some tentative solutions.
    The aim of the brainstorming session was to highlight things that needed improvement, rather than to evaluate how good EA operations roles are relative to the other non-profit or for-profit roles. It’s possible that EA organizations are not uniquely bad or good - but that doesn’t mean that these issues are not worth addressing. The outside world (especially the non-profit space) is pretty inefficient, and I think it’s worth trying to improve things.
    Limitations of this data: Meta / community building (and longtermist, to a lesser degree) organizations were overrepresented in this sample, and the tallies are estimates. We didn’t systematically ask people to vote for each and every sub-item, but we think the overall priorities raised were reasonable.
    General Brainstorming
    Four major themes came up in the original brainstorming session: bad knowledge management, unrealistic expectations, bad delegation, and lack of respect for operations. The group then re-formed new groups to brainstorm solutions for each of these key pain points.
    Below, we go into a breakdown of each large issue into specific points raised during the general brainstorming session. Some points were raised multiple times and are indicated by the “(x n)” to indicate how many times the point was raised.
    Knowledge management
    Problems
    Organizations don’t have good systems for knowledge management. Ops staff don’t have enough time to coordinate and develop better systems. There is a general lack of structure, clarity and knowledge.
    Issues with processes and systems (x 4)
    No time on larger problems
    Lack of time to explore & coordinate
    Lack of time to make things easier ([you’re always] putting out fires)
    [Lack of] organizational structure
    Line management
    Capacity to cover absences [see Unrealistic Expectations]
    Covering / keeping the show running
    Responsibilities
    Working across time zones
    Training / upskilling
    Management training [see improper delegation]
    Lack of Clarity + Knowledge
    Legal
    Compliance
    HR
    Hiring
    Wellbeing (including burnout)
    Lack of skill transfer
    Lack of continuity / High turn-over of junior ops specialists
    Potential Solutions
    Lowering the bar - e.g. you don’t need a PhD to work in ops. Pick people with less option value.
    Ask people to be nice and share with others
    Best practice guides shared universally. [Make them] available to people before hiring so they can understand the job better before applying, so [there’s] less turn-over.
    Database? (Better ops Slack?)
    Making time to create Knowledge Management Systems - so less fire-fighting.
    People higher in the organization [should have] better oversight of processes/knowledge.
    Unrealistic expectations
    Problems
    Employers have unrealistic expectations for ops professionals. Ops people are expected to do too much in too little time and always be on call.
    Lack of capacity / too much to do (x2)
    [Lack of] capacity to cover absences [from above]
    Ops people [are expected to be] “always on call”
    Timelines for projects [are subject to the] planning fallacy, [and there are] last minute changes
    Ops team [are] responsible for all new ideas that people come [up] with - could others do it?
    Unrealistic expectations about
    coordination capacity
    skillset
    organizational memory
    Solutions
    Bandwidth (?)
    Increase capacity
    Have continuity
    [give ops staff the] ability to push back on too-big asks
    Recognition
    Create...
    12 min
  • EA - [Linkpost] Why pescetarianism is bad for animal welfare - Vox, Future Perfect by Garrison
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [Linkpost] Why pescetarianism is bad for animal welfare - Vox, Future Perfect, published by Garrison on March 16, 2023 on The Effective Altruism Forum.
    In my debut for Vox, I write about why switching to a pescetarian diet for animal welfare reasons is probably a mistake.
    I was motivated to reduce animal consumption by EA reasoning. I initially thought that the moral progression of diets was something like vegan > vegetarian > pescetarian > omnivore. But I now think the typical pescetarian diet is worse than an omnivorous one. (I was actually convinced in part by an EA NYC talk by Becca Franks on fish psychology.)
    Why?
    Fish usually eat other fish, and they're smaller on average than typical farmed animals.
    The evidence for their sentience is much stronger than I previously thought. I think my credence is now something like P(pig/cow sentience) = 99.99%, P(chicken/fish sentience) = 99%
    Given that there are ~30k fish species, generalizing about them is a bit tricky, but I think the evidence of fish sentience is about as strong as the evidence for chicken sentience, something I would guess more people accept.
    I also spend time discussing:
    environmental impacts of fishing
    consumer choice vs. systemic change
    shrimp welfare
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min
  • LW - Here, have a calmness video by Kaj Sotala
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Here, have a calmness video, published by Kaj Sotala on March 16, 2023 on LessWrong.
    This is a bit of an unusual post.
    I have gotten the impression that a lot of people are kind of freaked out, either by AI or weird Bay Area social dynamics in general.
    I also think that a lot of freak-out reactions are driven at least as much by social contagion as any fact-based assessment of what's happening. When you see people around you freak out, you too are much more likely to freak out.
    Conversely, if the people around you are calm, then you're also much more likely to stay calm.
    There's also a selection effect where freakouts tend to spread much more online than calmness does. If you're calm, you don't necessarily feel the need to post anything. You might be content to just be.
    Whereas if you're freaking out, you're much more likely to post stuff about how you're freaking out or how we're all going to die.
    So there's easily a cycle where the most distressed views predominate, that freaks people out and causes there to be more distressed posts, which freaks out more people, and so on. And this might be mostly uncorrelated with how much of a reason there was to actually freak out.
    But if we were all in the same physical space, we might all notice that only some people are freaking out and a lot are a lot more calm. And then the distress wouldn't spread as much, and we could think more clearly.
    I too am concerned about AI, but I'm not freaked out. (In part because I don't think freaking out would be a useful reaction to have, in part because I'm somewhat more optimistic than most, in part because I spend a lot of time with people who aren't freaking out.) If I were physically located in the same place as others who were freaking out, I think that my calm could help with their freakout.
    However, I'm not. And as stated, it's kinda hard to convey calmness over text, the same way you can convey distress.
    So I thought of making a video where I'm calm. Maybe that would help convey it better.
    It's here. In Finnish, but with English subtitles.
    I know it's low video quality; I recorded it in Zoom, and only noticed afterward that there's an "HD quality" button I could have clicked in the settings. Oops.
    But that was part of the intended vibe too. I could have spent a lot of time optimizing the video quality and everything. Instead, I just recorded it in one shot, because it's not such a big deal whether the video quality is great or not.
    I'll probably make another calmness video with better quality.
    No earlier than tomorrow.
    Because I don't feel like I'm in a rush.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    3 min
  • AF - What organizations other than Conjecture have (esp. public) info-hazard policies? by David Scott Krueger
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What organizations other than Conjecture have (esp. public) info-hazard policies?, published by David Scott Krueger on March 16, 2023 on The AI Alignment Forum.
    I believe Anthropic has said they won't publish capabilities research?OpenAI seems to be sort of doing the same (although no policy AFAIK).I heard FHI was developing one way back when...I think MIRI sort of does as well (default to not publishing, IIRC?)
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    1 min
  • EA - Announcing the ERA Cambridge Summer Research Fellowship by Nandini Shiralkar
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Announcing the ERA Cambridge Summer Research Fellowship, published by Nandini Shiralkar on March 16, 2023 on The Effective Altruism Forum.
    The Existential Risk Alliance (ERA) has opened applications for an in-person, paid, 8-week Summer Research Fellowship focused on existential risk mitigation, taking place from July 3rd to August 25th 2023 in Cambridge, UK, and aimed at all aspiring researchers, including undergraduates.
    To apply and find out more, please visit the ERA website.
    If you are interested in mentoring fellows on this programme, please submit your name, email and research area here, and we will get in touch with you in due course.
    If you know other people who would be a good fit, please encourage them to apply (people are more likely to apply if you recommend they do, even if they have already heard of the opportunity!) If you are a leader or organiser of relevant community spaces, we encourage you to post an announcement with a link to this post, or alternatively a printable poster is here.
    Applications will be reviewed as they are submitted, and we encourage early applications, as offers will be sent out as soon as suitable candidates are found. We will accept applications until April 5, 2023 (23:59 in US Eastern Daylight Time).
    The ERA Cambridge Fellowship (previously known as the CERI Fellowship) is a fantastic opportunity to:
    Build your portfolio by researching a topic relevant to understanding and mitigating existential risks to human civilisation.
    Receive guidance and develop your research skills, via weekly mentorship from a researcher in the field.
    Form lasting connections with other fellows who care about mitigating existential risks, while also engaging with local events including discussions and Q&As with experts.
    Why we are running this programme
    Our mission as an organisation is to reduce the probability of an existential catastrophe. We believe that one of the key ways to reduce existential risk lies in fostering a community of dedicated and knowledgeable x-risk researchers. Through our summer research fellowship programme, we aim to identify and support aspiring researchers in this field, providing them with the resources and the mentorship needed to succeed.
    What we provide
    A salary equivalent to £31,200 per year, which will be prorated to the duration of the summer programme.
    Mentorship from a researcher working in a related field.
    Complimentary accommodation, meal provisions during working hours, and travel expense coverage
    Dedicated desk space at our office in central Cambridge.
    Opportunity to work either on a group research project with other fellows or individually.
    Networking and learning opportunities through various events, including trips to Oxford and London.
    What we are looking for
    We are excited to support a wide range of research, from the purely technical to the philosophical, as long as there is direct relevance to mitigating existential risk. This could also include social science or policy projects focusing on implementing existential risk mitigation strategies.
    Incredibly successful projects would slightly reduce the likelihood that human civilisation will permanently collapse, that humans will go extinct, or that the future potential of humanity will be permanently reduced. A secondary goal of this project is for fellows to learn more about working on existential risk mitigation, develop relevant skills, and test their fit for further research or work in this field.
    Who we are looking for
    Anyone can apply to the fellowship, though we expect it to be most useful to students (from undergraduates to postgraduates) and early-career individuals looking to test their fit for existential risk research. We particularly encourage undergraduates to apply, to develop their research experience.
    We are looking to support proactive i...
    5 min
  • EA - Offer an option to Muslim donors; grow effective giving by GiveDirectly
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Offer an option to Muslim donors; grow effective giving, published by GiveDirectly on March 16, 2023 on The Effective Altruism Forum.
    Summary
    In order to offer Muslim donors a way to give their annual religious tithing (zakat) to an EA-aligned intervention, GiveDirectly launched a zakat-compliant fund, delivered as cash to Yemeni families displaced by the civil war. Muslims give ~$600B/year in Zakat to the global poor, though much of this is given informally or to less-than-effective NGOs.
    Through this unconditional cash transfer option, we’re offering Muslims the opportunity to redirect a portion of their giving to a measurably high-impact intervention and introduce more Muslims to EA’s theory of effective giving. We invite readers to share thoughts in the comments and to share the campaign far and wide.
    Muslims are the fastest-growing religious group and give annually
    As Ahmed Ghoor observed, Muslims make up about 24% of the world population (1.8B people) and Islam is the fastest growing religion. Despite having a robust tradition of charitable giving, little has been done proactively to engage the Muslim community on the ideas of effective altruism. An important step to inclusion is offering this pathway for effectively donating zakat.
    Zakat is a sacred pillar of Islam, a large portion of which is given to the needy
    For non-Muslim readers: one of the five pillars of Islam, zakat is mandatory giving; Muslims eligible to pay it donate at least 2.5% of their accumulated wealth annually for the benefit of the poor, destitute, and others – classified as mustahik. Some key points:
    A major cited aim of Zakat is to provide relief from and ultimately eradicate poverty.
    It is generally held that zakat can only be given to other Muslims.
    A large portion of zakat is given informally person-to-person or through mosques and Islamic charities.
    Zakat is a sacred form of charity; it’s most often given during the holy month of Ramadan.
    Direct cash transfers are a neglected zakat option
    Zakat giving is estimated at $1.8B in the U.S. alone with $450M going to international NGOs, who mostly use their funds for in-kind support like food, tents, and clothing. Dr. Shahrul Hussain, an Islamic scholar, argues that cash transfers “should be considered a primary method of zakat distribution,” as, according to the Islamic principle of tamlīk (ownership), the recipients of the zakat have total ownership over the money, and it is up to them (not an intermediary third-party organization or charity) how it is spent. He also notes “the immense benefits of unconditional cash transfer in comparison to in-kind transfer."
    This is a simple, transparent means of transferring wealth that empowers the recipients. However, other than informal person-to-person giving, there are limited options to give zakat as 100% unconditional cash.
    GiveDirectly now allows zakat to be given as cash to Muslims in extreme poverty
    As an opportunity for Muslims to donate zakat directly as cash, GiveDirectly created a zakat-compliant fund to give cash through our program in Yemen. While GiveDirectly is a secular organization, our Yemen program and Zakat policy have been reviewed and certified by Amanah Advisors. In order to achieve this, we’re assured that 100% of donations will be delivered as cash, using non-zakat funds to cover the associated delivery costs.
    Donations through our page are tax-deductible in the U.S. and our partners at Giving What We Can created a page allowing donors to give 100% of their gift to GiveDirectly’s zakat-compliant fund, tax-deductible in the Netherlands and the U.K. Taken together, this provides a tax-deductible option for 8.6M Muslims across three countries.
    As a secular NGO, GiveDirectly may struggle to gain traction with Muslim donors
    GiveDirectly is a credible option for zakat donors: we’ve...
    8 min
  • AF - [ASoT] Some thoughts on human abstractions by leogao
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [ASoT] Some thoughts on human abstractions, published by leogao on March 16, 2023 on The AI Alignment Forum.
    TL;DR:
    Consider a human concept such as "tree." Humans implement some algorithm for determining whether given objects are trees. We expect our predictor/language model to develop a model of this algorithm because this is useful for predicting the behavior of humans.
    This is not the same thing as some kind of platonic ideal concept of what is “actually” a tree, which the algorithm is not incentivized to develop by training on internet text, and trying to retarget the search at it has the same supervision problems as RLHF against human scores on whether things look like trees.
    Pointing at this “actually a tree” concept inside the network is really hard; the ability of LMs to comprehend natural language does not allow one to point using natural language, because it just passes the buck.
    Epistemic status: written fast instead of not at all, probably partially deeply confused and/or unoriginal. Thanks to Collin Burns, Nora Belrose, and Garett Baker for conversations.
    Will NNs learn human abstractions?
    As setup, let's consider an ELK predictor (the thing that predicts future camera frames). There are facts about the world that we don't understand that are in some way useful for predicting the future observations. This is why we can expect the predictor to learn facts that are superhuman (in that if you tried to supervised-train a model to predict those facts, you would be unable to generate the ground truth data yourself).
    Now let's imagine the environment we're predicting consists of a human who can (to take a concrete example) look at things and try to determine if they're trees or not. This human implements some algorithm for taking various sensory inputs and outputting a tree/not tree classification. If the human does this a lot, it will probably become useful to have an abstraction that corresponds to the output of this algorithm. Crucially, this algorithm can be fooled by i.e a fake tree that the human can't distinguish from a real tree because (say) they don't understand biology well enough or something.
    However, the human can also be said to, in some sense, be "trying" to point to the "actual" tree. Let's try to firm this down. The human has some process they endorse for refining their understanding of what is a tree / "doing science" in ELK parlance; for example, spending time studying from a biology textbook. We can think about the limit of this process. There are a few problems: it may not converge, or may converge to something that doesn't correspond to what is "actually" a tree, or may take a really really long time (due to irrationalities, or inherent limitations to human intelligence, etc). This suggests that this concept is not necessarily even well defined. But even if it is, this thing is far less naturally useful for predicting the future human behaviour than the algorithm the human actually implements! Implementing the actual human algorithm directly lets you predict things like how humans will behave when they look at things that look like trees to them.
    More generally, one possible superhuman AI configuration I can imagine is one where the bulk of the circuits are used to predict its best-guess for what will happen in the world. There may also be a set of circuits that operate in a more humanlike ontology used specifically for predicting humans, or it may be that the best-guess circuits are capable enough that this is not necessary (and if we scale up our reporter we eventually get a human simulator inside the reporter).
    The optimistic case here is if the "actually a tree" abstraction happens to be a thing that is useful for (or is very easily mapped from) the weird alien ontology, possibly because some abstractions are more universal. In this ...
    9 min
  • LW - Want to predict/explain/control the output of GPT-4? Then learn about the world, not about transformers. by Cleo Nardo
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Want to predict/explain/control the output of GPT-4? Then learn about the world, not about transformers., published by Cleo Nardo on March 16, 2023 on LessWrong.
    Introduction
    Consider Act II Scene II of William Shakespeare's Julius Caesar.
    In this scene, Caesar is at home with his wife Calphurnia, who has just had a bad dream and is pleading with him not to go to the Senate. Caesar initially agrees to stay home but changes his mind after being convinced by Decius Brutus that the dream was misinterpreted and that the Senate needs him to address important matters.
    CAESAR: The cause is in my will: I will not come; That is enough to satisfy the senate. [...]
    DECIUS BRUTUS: [...] If Caesar hide himself, shall they not whisper 'Lo, Caesar is afraid'? Pardon me, Caesar; for my dear dear love To our proceeding bids me tell you this; And reason to my love is liable.
    CAESAR: How foolish do your fears seem now, Calphurnia! I am ashamed I did yield to them. Give me my robe, for I will go.
    This was the morning of the Ides of March, 15 March 44 BC, which is the date today coincidentally. Caesar was assassinated during the Senate meeting.
    Suppose I change Caesar's final line to
    CAESAR: My mind is firm, Decius. I'll stay within these walls, And not tempt Fortune on this cursed day. Worry me not, for I will stay.
    and I feed this modified scene into GPT-4, what would be the output?
    I don't know.
    But how might I determine the answer?
    The claim
    You might think that if you want to predict the logits layer of a large autoregressive transformer, then the best thing would be to learn about transformers. Maybe you should read Neel Nanda's blogposts on mechanistic interpretability. Or maybe you should read the Arxiv papers on the GPT models.
    But this probably won't help you predict the logits layer for this prompt.
    Instead, if your goal is to predict the logits layer, then you should probably learn about Shakespearean dramas, Early Modern English, and the politics of the Late Roman Republic.
    And maybe someone has already run GPT-4 on this prompt — if your goal is to explain the logits layer, then you should probably learn about Shakespearean dramas, Early Modern English, and the politics of the Late Roman Republic.
    This is also true if you're trying to construct a prompt which will make GPT-4 output a particular target continuation — if your goal is to control the logits layer, then you should probably learn about Shakespearean dramas, Early Modern English, and the politics of the Late Roman Republic.
    Dataset vs architecture
    The output of a neural network is determined by two things:
    The architecture and training algorithm (e.g. transformers, SGD, cross-entropy)
    The training dataset (e.g. internet corpus, literature, GitHub code)
    As a rough rule-of-thumb, if you want to predict/explain the output of GPT-4, then it's far more useful to know about the training dataset than to know about the architecture and training algorithm.
    In other words,
    If you want to predict and explain the output of GPT-4 on Haskell code, you need to know Haskell.
    If you want to predict and explain the output of GPT-4 on Shakespearean dialogue, you need to know Shakespeare.
    If you want to predict and explain the output of GPT-4 on Esperanto, you need to know Esperanto.
    If you want to predict and explain the output of GPT-4 on the MMLU benchmark, you need to know the particular facts in the benchmark.
    I think alignment researchers (and AI researchers more generally) underestimate the extent to which knowledge of the training dataset is currently far more useful for prediction/explanation than knowledge of the architecture and training algorithm.
    Recall that as the cross-entropy loss of LLM steadily decreases, then the logits of the LLM will asymptotically approach the ground-truth distribution which generated the dataset...
    9 min
  • LW - GPT-4: What we (I) know about it by Robert AIZI
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: GPT-4: What we (I) know about it, published by Robert AIZI on March 15, 2023 on LessWrong.
    OpenAI released a press release, research statement, and system card about GPT-4 approximately one eternity (24 hours) ago. The general public can’t use it yet, but it’s in the process of being rolled out to paid subscribers of ChatGPT, and via a waitlist to the API. We also got confirmation that the Bing AI (also currently rolling out via waitlist) is based on GPT-4.
    Here I’ll try to summarize the news and boil down what we (I) know about GPT-4. Many points lifted from the discussion at lesswrong.
    My main takeaways:
    Capabilities progress is continuing without slowing.
    OpenAI spent a lot of time on RLHF/fine-tuning to prevent unethical use (facilitating crime, generating hate speech, etc), and they behave as if this is sufficient to solve alignment.
    OpenAI is no longer so open - we know almost nothing about GPT-4’s architecture.
    Previously from OpenAI.
    (Just recapping the progress of the GPT series of models, feel free to skip.)
    AIs advance very quickly. The most impressive AI these days are large language models, including the GPT series, and they are all based on the transformer, an architecture introduced in 2017.
    In 2018 OpenAI released the Generative Pre-Trained Transformer (GPT), which approached natural language tasks by predicting the next token. It was especially evaluated on narrow tasks (e.g. “Is the sentiment of this user review positive or negative? [user review]. The sentiment is.”). A key technique for GPT (and all its successors) was the eponymous “pre-training”, where the AI is trained not on any particular task, but just to predict the next token in a text. This gives you access you a huge volume of training data (literally all text), while building general understanding of the world - answering factual questions is a form of token completion, so the AI needs to be able to answer those questions, etc. This pre-training built a general knowledge base, and then GPT was “fine-tuned” to individual tasks with additional training on those datasets.
    We know from the GPT-4 press release that OpenAI trained GPT-3.5 “a year ago”, using the same architecture as GPT-3 but with a custom-designed supercomputer and a better “deep learning stack”. While I’m not aware of publicly available comparisons of GPT-3 and 3.5, some users reported that 3.5 felt smarter, and I’m inclined to believe them.
    During this time, OpenAI also became interested in Reinforcement Learning on Human Feedback (RLHF). In RLHF, a human evaluates the output of the AI, and rates it on some objectives (such as “helpful and honest”), and this is used to train the AI. An RLHF'd version of GPT 3.5 was released in November 2022 under the name ChatGPT, which became somewhat popular.
    GPT-4 Timeline
    According to the research statement, GPT-4 “finished training” in August of 2022. It’s not entirely clear what they mean by this, because they say they’ve been “iteratively improving” it since then - was this RLHF, fine-tuning, or something else? If they mean it finished pre-training, why didn’t they use that term?
    Capabilities Improvements
    GPT-4 continues to improve capabilities over GPT-4 and GPT-3.5. The raw numbers are available in the paper, but I think in the long run what matters is what GPT is being evaluated on. Now, in addition to AI benchmarks like “MMLU” and “HellaSwag”, GPT-4 is being evaluated on exams that humans take.
    GPT-4 scored a 1410/1600 on the SAT and a 4/5 or 5/5 on the AP Art History, Biology, Calculus BC, Chemistry, Environmental Sciences, Macroeconomics, Microeconomics, Physics 2, Psychology, Statistics, US Government, US History, and US World History exams (a 3/5 is passing. GPT-4 scored only a 2/5 on {English Language and Composition} and {English Literature and Composition}). We’re now in ...
    22 min
  • LW - How well did Manifold predict GPT-4? by David Chee
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How well did Manifold predict GPT-4?, published by David Chee on March 15, 2023 on LessWrong.
    Chat GPT-4 is already here!! Who could have seen that coming. oh wait Manifold (kinda) did?
    I thought I’d write a short piece on how Manifold Markets was used to predict the launch of GPT-4 and its attributes. Both its successes and its failures. Disclaimer I work at Manifold.
    How well did we predict the launch date?
    Throughout the end of last year, people were bullish on a quick release, which began to decline as we entered the start of this year.
    The first spike in February corresponds to the release of Bing’s chatbot which people speculated was Chat CPT-4. Turns out it actually was! Although Open AI did a fantastic job at concealing this with our market on it hovering at a stubborn 50-60%.
    There was a lot of uncertainty on if GPT-4 would be released before March. However, on the 9th of March Microsoft Germany CTO Andreas Braun mentioned at an AI kickoff event that its release was imminent which caused the market to jump.
    Although the market graphs are a beautiful representation of hundreds of traders’ predictions, did they actually give us any meaningful information? One thing that stands out about these graphs in particular is the strong bets away from the baseline towards YES throughout February. Is this just noise, or is something more going on?
    Insider Trading
    Being the socialite I am, I go to a whopping one (1) social gathering a month!! At 100% of these, the SF Manifold Markets party and Nathan Young’s Thursday dinner, I spoke to someone who claimed they were trading on the Chat GPT-4 markets based on privileged insider information.
    One of them got burnt as allegedly there were delays from the planned launch and they had gone all-in on the GPT-4 being released by a certain date.
    I love knowing people with privileged information are able to safely contribute to public forecasts which wouldn’t be possible without a site like Manifold Markets. As they were trading from anonymous accounts I have no way of knowing whether they are the ones responsible for the large YES bets, but I suspect some of them are. That said, someone with insider knowledge would be better off placing a large limit order to buy YES just above the current baseline which would exert strong pressure to hold the market at/slightly above its current probability. Placing a large market order which causes the spikes gives them less profit than they otherwise could have earned.
    What else are people predicting about GPT-4?
    Jacy Reese Anthis, an American social scientist of the Sentience Institute, created a market on if credible individuals with expertise in the space will claim GPT-4 is sentient. 16% seems surprisingly high to me, but the market has only just been created and needs more traders. Go now and place your bets!
    One of our most popular markets, which failed in spectacular fashion, was whether it would get the Monty Fall problem correct (note - this is not the same as the Monty Call problem, click through to the market description for an explanation).
    This might be the single most consistent upward-trending market I have ever seen on our site. I wonder if GPT-4 hadn’t been released yet how much further it would have continued to trend upwards before plateauing.
    Part of the confidence came from Bing’s success in answering correctly when set to precise mode. Many speculated GPT-4 was going to be even more powerful than Bing, even though they turned out to be the same. I’m not exactly sure what the difference is using the “precise” setting, if anyone knows let me know!
    Markets you can still predict on
    Here are some more open markets for you to go trade-in. It’s free and uses play money!
    Thanks for reading! Hope it was interesting to see the trends on Manifold, even if not a particularly in-depth an...
    4 min