The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • EA - Hiding in Plain Sight: Mexico’s Octopus Farm/Research Facade by Tessa @ ALI
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Hiding in Plain Sight: Mexico’s Octopus Farm/Research Facade, published by Tessa @ ALI on April 19, 2023 on The Effective Altruism Forum.
    Recap:
    In November 2022, Aquatic Life Institute (ALI) implemented a global campaign that aims to increase public and legislative pressure on countries/regions where octopus farms are being considered to achieve a regulatory ban, and reduce future chances of these farms being created elsewhere. We started Banding Together to Ban Octopus Farming, and have witnessed both promising and pessimistic developments.
    Good News First:
    2023 has, thus far, demonstrated just how universal the looming issue of octopus farming is. Concerns originated from the establishment of a potential farm in Spain, however, grievances quickly traveled.
    From the United States; Hawaii's Division of Aquatic Resources issued a cease and desist letter to Kanaloa Octopus Farm for operating without the required permits. And house bill 1153: "Prohibiting Octopus Farming" was proposed in Washington state. HB 1153 passed the first committee vote with overwhelming support from policymakers and members of the public across the globe. The bill received 9 "yes" votes and 2 "nays" from committee members. Despite overwhelming, bipartisan support, the bill now sits in the Agriculture and Natural Resources Committee until 2024. However, this truly represents a historic moment for the movement to #BanOctopusFarming. During the voting session, policymakers spoke about octopus sentience and intelligence, and how this is a great opportunity to protect an incredible species from somber futures in factory farms. ALI will ensure that this bill is a priority topic again during the next legislative session.
    Across the pond, RSPCA calls for halt to plans for world's first octopus farm. This announcement is the first public objection raising "concerns over the commercial farming of octopuses and the negative welfare impact it could have on this complex animal" from a seafood certification body. Aquatic Life Institute works closely with RSPCA Assured (RSPCA’s farm animal welfare assurance scheme), to ensure that all seafood production prioritizes high-welfare practices. They are a valuable stakeholder in ALI's Certifier Campaign, and we commend them for taking the lead on such a timely issue.
    The Not So Good News:
    The small town of Sisal, Yucatan is now the location of Mexico’s first octopus farm, appearing extensively in national and international media outlets as a groundbreaking industry for the region.
    The Sisal unit of the Universidad Autonoma de Mexico (UNAM), the country’s largest and most prestigious university, initiated the research project to study the physiology of the most common regional species: Octopus Maya. UNAM’s research center created an agreement with local families to establish Moluscos del Mayab, the commercial branch of the facility.
    UNAM research facility obtains pregnant females from the surrounding ecosystem. Still relying on wild populations for replenishing their broodstock, they are currently conducting research related to the development of a self-sustaining reproductive unit. Once females lay eggs to be kept in a patented incubator, they are then slaughtered and commercialized. The eggs are artificially incubated for 50 days, producing around 20,000 eggs per month.
    The Sisal unit has around 6-8 recirculating tanks for adults and 12-15 tanks for juveniles. Pre-growth tanks hold around 25 larvae per square meter. In total, this tank can fit approximately 707 larvae at any given time. The second growth tank can hold around 288 juveniles, reducing the density at this stage due to aggressive juvenile cannibalism. Up to 191 octopus are held in each grow-out tank. With a 52% mortality rate, averaging around 5% per week, researchers note around 30% is directly relat...
    7 min
  • LW - Capabilities and alignment of LLM cognitive architectures by Seth Herd
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Capabilities and alignment of LLM cognitive architectures, published by Seth Herd on April 18, 2023 on LessWrong.
    Epistemic status: Hoping for help working through all of these new ideas.
    TLDR:
    Scaffolded, "agentized" LLMs that combine and extend the approaches in AutoGPT, HuggingGPT, Reflexion, and BabyAGI seem likely to be a focus of near-term AI development. LLMs by themselves are like a human with great automatic language processing, but no goal-directed agency, executive function, episodic memory, or sensory processing. Recent work has added all of these to LLMs, making language model cognitive architectures (LMCAs). These implementations are currently limited but will improve.
    Cognitive capacities interact synergistically in human cognition. In addition, this new direction of development will allow individuals and small businesses to contribute to progress on AGI. These new factors of compounding progress may speed progress in this direction. LMCAs might well become intelligent enough to create X-risk before other forms of AGI. I expect LMCAs to enhance the effective intelligence of LLMs by performing extensive, iterative, goal-directed "thinking" that incorporates topic-relevant web searches.
    The possible shortening of timelines-to-AGI is a downside, but the upside may be even larger. LMCAs pursue goals and do much of their “thinking” in natural language, enabling a natural language alignment (NLA) approach. They reason about and balance ethical goals much as humans do. This approach to AGI and alignment has large potential benefits relative to existing approaches to AGI and alignment.
    Overview
    I still think it's likely that agentized LLMs will change the alignment landscape for the better, although I've tempered my optimism a bit since writing that. A big piece of the logic for that hypothesis is why I expect this approach to become very useful, and possibly become the de-facto standard for AGI progress. The other piece was the potential positive impacts on alignment work. Both of those pieces of logic were compressed in that post. I expand on them here.
    Beginning with a caveat may be appropriate since much of the below sounds both speculative and optimistic. I describe many potential improvements and positive-sum synergies between different capabilities. There will surely be difficulties and many things that don’t work as well or as easily as they might, for deep reasons that will slow development. It’s quite possible that there are enough of those things that this direction will be eclipsed by continued development of large models, and that progress in integrating cognitive capacities will take a different route. In particular, this approach relies heavily on calls to large language models (LLMs). Calling cutting-edge LLMs will continue to have nontrivial costs in both time and money, as they require substantial computing resources. These may hamper this direction, or drive progress in substantially less human-like (e.g., parallel) or interpretable (e.g., a move to non-natural language core processing) directions.
    With these caveats in mind, I think the potentials for capabilities and alignment are enough to merit serious consideration from the alignment community, even this early in the game.
    I think AutoGPT, HuggingGPT, and similar script wrappers and tool extensions for LLMs are just the beginning, and there are low-hanging fruit and synergies that will add capability to LLMs, effectively enhancing their intelligence and usefulness. This approach makes an LLM the natural language cognitive engine at the center of a cognitive architecture. Cognitive architectures are computational models of human brain function, including separate cognitive capacities that work synergistically.
    Cognitive architectures are a longstanding field of research at the conjunctio...
    39 min
  • EA - Organizing a debate with experts and MPs to raise AI xrisk awareness: a possible blueprint by Otto
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Organizing a debate with experts and MPs to raise AI xrisk awareness: a possible blueprint, published by Otto on April 19, 2023 on The Effective Altruism Forum.
    Thanks to Leon Lang from University of Amsterdam for proposing this post and reviewing the draft. Any views expressed in this post are not necessarily his.We (Existential Risk Observatory) have organized a public debate (recording here, photos here) in Amsterdam, the Netherlands with the purpose of creating more awareness of AI existential risk among policymakers and other leading voices of the societal debate. We want to share our experience in this post, because we think others might be able to follow a similar approach, and because we expect this to have significant xrisk-reducing net effects.
    Goals
    Our high-level goal is to reduce existential risk, especially from AGI, by informing the public debate. We think more awareness will be net positive because of increased support for risk-reducing regulation and increased attention for AI safety work (more talent, more funding, more institutes working on the topic, more diverse actors, and more priority).
    For this debate, we focused on:
    Informing leaders of the societal debate (such as journalists, opinion makers, scientists, artists) about AI existential risk. This should help to widen the Overton window, increase the chance that this group of people will inform others, and therefore raise AI existential risk awareness in society.
    Informing politicians about AI existential risk, to increase the chance that risk-reducing policies will get enacted.
    Strategy
    We used a debate setup with AI existential risk authorities and Members of Parliament (MPs). The strategy was that the MPs would get influenced by the authorities in this setting. We already contacted MPs and MP assistants before and had meetings with several. In order to be invited to meetings with MPs and/or MP assistants, we think that having published in mainstream media, having a good network, and having good policy proposals, are all helpful. If budget is available, hiring a lobbyist and a PR person is both helpful as well (we have a freelance senior PR person and a medior volunteer lobbyist).
    Two Dutch MPs attended our debate. For them, advantages might include getting (media) attention and informing themselves. Stuart Russell agreed to be our keynote speaker (remote), and several other experts who all had good AI existential risk expertise attended as well. Additionally, we found a moderator with existing AI existential risk knowledge, which was important in making sure the debate went well. Finally, we found a leading debate center in Amsterdam (Pakhuis de Zwijger) willing to host the debate. We promoted our debate on our own social media, through the venue, we were mentioned in the prominent Dutch Future Affairs newsletter, and we promoted our event in EA whatsapp groups. This was sufficient to sell out the largest debate hall in Amsterdam (320 seats, tickets were free).
    We organized this event in the Netherlands mainly because this is where we are most active. However, since the event was partially focused on getting policy implemented, it should be considered to organize events such as this debate especially where policy is most urgently needed (DC, but perhaps also Beijing, Brussels, or London).
    Program
    We started with an introductory talk, after which we played some documentary fragments. After this introduction, our main guest Stuart Russell gave a talk and audience Q&A on AI existential risk. We closed with the 5-person panel, consisting of Queeny Rajkowski (MP for VVD, the largest governing party, center-right, with a portfolio including Digitization and Cybersecurity), Lammert van Raan (MP for the Party for the Animals, a medium-sized left party focusing on animal rights and climate), Mark Br...
    8 min
  • AF - Davidad's Bold Plan for Alignment: An In-Depth Explanation by Charbel-Raphael Segerie
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Davidad's Bold Plan for Alignment: An In-Depth Explanation, published by Charbel-Raphael Segerie on April 19, 2023 on The AI Alignment Forum.
    Gabin Kolly and Charbel-Raphaël Segerie contributed equally to this post. Davidad proofread this post.
    Thanks to Vanessa Kosoy, Siméon Campos, Jérémy Andréoletti, Guillaume Corlouer, Jeanne S., Vladimir I. and Clément Dumas for useful comments.
    Context
    Davidad has proposed an intricate architecture aimed at addressing the alignment problem, which necessitates extensive knowledge to comprehend fully. We believe that there are currently insufficient public explanations of this ambitious plan. The following is our understanding of the plan, gleaned from discussions with Davidad.
    This document adopts an informal tone. The initial sections offer a simplified overview, while the latter sections delve into questions and relatively technical subjects. This plan may seem extremely ambitious, but the appendix provides further elaboration on certain sub-steps and potential internship topics, which would enable us to test some ideas relatively quickly.
    Davidad’s plan is an entirely new paradigmatic approach to address the hard part of alignment: The Open Agency Architecture aims at “building an AI system that helps us ethically end the acute risk period without creating its own catastrophic risks that would be worse than the status quo”.
    This plan is motivated on the assumption that current paradigms for model alignment won’t be successful:
    LLMs won't be able to be aligned just with RLHF or a variation of this technique.
    Scalable oversight will be too difficult.
    Interpretability used to retro-engineer an arbitrary model will not be feasible. Instead, it would be easier to iteratively construct an understandable world model.
    Unlike OpenAI's plan, which is a meta-level plan that delegates the task of finding a solution for alignment with future AI, davidad's plan is an object-level plan that takes drastic measures to prevent future problems. It is also based on rather testable assumptions that can be relatively quickly tested (see the annex).
    Plan’s tldr: Utilize near-AGIs to build a detailed world simulation, train and formally verify within it that the AI adheres to coarse preferences and avoids catastrophic outcomes.
    How to read this post?
    This post is much easier to read than the original post. But we are aware that it still contains a significant amount of technicality. Here's a way to read this post gradually:
    Start with the Bird's-eye view (5 minutes)
    Contemplate the Bird's-eye view diagram (5 minutes), without spending time understanding the mathematical notations in the diagram.
    Fine Grained Scheme: Focus on the starred sections. Skip the non-starred sections. Don’t spend too much time on difficulties (10 minutes)
    From "Hypothesis discussion" onwards, the rest of the post should be easy to read (10 minutes)
    For more technical details, you can read:
    The starred sections of the Fine Grained Scheme
    The Annex.
    Bird's-eye view
    The plan comprises four key steps:
    Understand the problem: This entails formalizing the problem, similar to deciphering the rules of an unfamiliar game like chess. In this case, the focus is on understanding reality and human preferences.
    World Modeling: Develop a comprehensive and intelligent model of the world capable of being used for model-checking. This could be akin to an ultra-realistic video game built on the finest existing scientific models. Achieving a sufficient model falls under the Scientific Sufficiency Hypothesis (a discussion of those hypotheses can be found later on) and would be one of the most ambitious scientific projects in human history.
    Specification Modeling: Generate a list of moral desiderata, such as a model that safeguards humans from catastrophes. The Deontic Sufficiency Hypothesis pos...
    37 min
  • EA - Who were The Righteous Among the Nations? by Carole Bibas-Barkan
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Who were The Righteous Among the Nations?, published by Carole Bibas-Barkan on April 18, 2023 on The Effective Altruism Forum.
    On the occasion of the Holocaust Remembrance Day happening today in Israel, I wanted to explore connections between the Holocaust and EA. My first thoughts were toward the importance of remembrance: can collective memory prevent future generations from repeating past mistakes? But then, I recalled my trip to Poland and the concentration camps years ago. I remembered our guide mentioning a study identifying four characteristics of The Righteous Among the Nations. I think that identifying the characteristics of those who chose "good" during one of the darkest times in History can be of particular interest to the EA community. Unfortunately, I didn't find this study, but here is an extract from the Yad Vashem Museum article that explores the subject:"Most rescuers were ordinary people. Some acted out of political, ideological or religious convictions; others were not idealists, but merely human beings who cared about the people around them.
    In many cases they never planned to become rescuers and were totally unprepared for the moment in which they had to make such a far-reaching decision. They were ordinary human beings, and it is precisely their humanity that touches us and should serve as a model. The Righteous are Christians from all denominations and churches, Muslims and agnostics; men and women of all ages; they come from all walks of life; highly educated people as well as illiterate peasants; public figures as well as people from society's margins; city dwellers and farmers from the remotest corners of Europe; university professors, teachers, physicians, clergy, nuns, diplomats, simple workers, servants, resistance fighters, policemen, peasants, fishermen, a zoo director, a circus owner, and many more.
    Scholars have attempted to trace the characteristics that these Righteous share and to identify who was more likely to extend help to the Jews or to a persecuted person. Some claim that the Righteous are a diverse group and the only common denominator are the humanity and courage they displayed by standing up for their moral principles. Samuel P. Oliner and Pearl M. Oliner defined the altruistic personality. By comparing and contrasting rescuers and bystanders during the Holocaust, they pointed out that those who intervened were distinguished by characteristics such as empathy and a sense of connection to others. Nehama Tec who also studied many cases of Righteous, found a cluster of shared characteristics and conditions of separateness, individuality or marginality. The rescuers’ independence enabled them to act against the accepted conventions and beliefs.Bystanders were the rule, rescuers were the exception.
    However difficult and frightening, the fact that some found the courage to become rescuers demonstrates that some freedom of choice existed, and that saving Jews was not beyond the capacity of ordinary people throughout occupied Europe. The Righteous Among the Nations teach us that every person can make a difference."
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    3 min
  • EA - Ghana has approved the use of a malaria vaccine with >70% efficacy by Henry Howard
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Ghana has approved the use of a malaria vaccine with >70% efficacy, published by Henry Howard on April 19, 2023 on The Effective Altruism Forum.
    The R21/Matrix-M malaria vaccine showed 71%-80% efficacy in preventing cases of malaria in a randomised controlled phase 2 trial published at the end of last year. A phase 3 trial is ongoing.
    67 (51%) of 132 children who received R21/Matrix-M with low-dose adjuvant, 54 (39%) of 137 children who received R21/Matrix-M with high-dose adjuvant, and 121 (86%) of 140 children who received the rabies vaccine developed clinical malaria by 12 months
    (the rabies vaccine was the control)
    The next best thing is the RTS,S/AS01 vaccine, which WHO started rolling out in some pilot programs in 2016 after trials showed its reduced hospital admissions from severe malaria by around 30%, less impressive than R21/Matrix-M.
    A few days ago, Ghana's food and drugs administration announced that they've approved the R21/Matrix-M for children aged 5 months to 36 months. It seems like there will be more steps before the vaccines actually start rolling out (they might need to wait for WHO approval and/or the results of the phase 3 trial). In any case, very exciting news.
    I found out about this because it is on the In the news section of the front page of Wikipedia.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min
  • EA - Follow-up on "Institutions for Future Generations" by tylermjohn
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Follow-up on "Institutions for Future Generations", published by tylermjohn on April 19, 2023 on The Effective Altruism Forum.
    3.5 years ago I wrote a post called "Institutions for Future Generations" indicating that I had started a project to figure out what institutions and policies could help better represent future generations' interests in government and soliciting help coming up with ideas.
    I finished the research for this project in 2020, and had expected to spend more time wrapping it up and making it pretty for public consumption, but I've never come back to the project and don't expect to. It's possible that some of the research will be useful to people thinking about policy and institutional reform, so I wanted to at least share the (messy!) Google doc with my work in it for consumption on the EA Forum.
    Here's the Google doc.
    I pulled out many of the most useful things I learned and published them in a few stand-alone papers, which are probably more helpful than the Google doc overall:
    Longtermist Institutional Reform, with William MacAskill
    A general paper on what gives governments short time horizons and some of the more promising things we can do about this
    Securing Political Accountability to Future Generations with Retrospective Accountability (with help from Charlotte Siegmann)
    A mechanism design paper proposing one way we might incentivise current generations to promote the interests of future generations, by making policymakers financially dependent on the decisions future generations make about how successful they were at promoting long-term interests
    Empowering Future People by Empowering the Young?
    A short paper, the most important part of which is that empowering the young doesn't seem to do all that much for future generations, though there's some positive evidence from the finding that age is a significant determinant of pro-environmentalist attitudes, to the point that a 20-year-old voter is 10 per cent more likely to vote favourably to the environment than an 80-year-old voter
    Want politics to be better? Focus on future generations, with William MacAskill (and lots of help from Fin Moorhouse)
    Popular-audience piece on the Japanese Future Design movement, extinction risk, integrating forecasting and technology expertise into government, and representing future generations in government
    My takeaways from this research:
    Overall, I've come to think that this research is less valuable than I thought it was when I started the project. There's very little strong literature on building future-oriented institutions and policy, I didn't find a lot of great success stories of policymakers doing this, and it actually just seems pretty hard to get right. I think there are a few great things people can do to generally make governments more future-oriented, like lowering the discount rate and infusing better technology expertise into governance, but most of these things are technocratic, moderate, and look less like "representing future generations" than I initially expected. I also increasingly find abstract institutional design work less relevant than policy work.
    I'm still optimistic that someone could find ideas better than the ones that I found, and I would still be excited if a huge number of economists and political scientists spent much more time thinking about these mechanism design issues, but I think marginal resources are better invested in existential risk policy, especially AI policy.
    I'll try to answer any questions, but it's been a few years since I've thought about these things much and I'm unsure how much of the doc I still endorse today!
    I make no guarantee that the scoring system used in the doc is accurate, internally consistent, or informative. I meant to spend more time developing a useful scoring system, but because I didn't manage...
    4 min
  • LW - Notes on Teaching in Prison by jsd
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Notes on Teaching in Prison, published by jsd on April 19, 2023 on LessWrong.
    Note: this is a post I wrote in 2018, so some of the numbers may be out of date. I've also changed my mind on lots of things since 2018, and there are many parts that I would write differently. Until now, the post was only accessible from the Wayback Machine; I'm sharing it here with some light edits because I think some LWers might find it interesting.
    Between September 2017 and April 2018, I worked full-time teaching in a French jail as part of my first year at École polytechnique. All of what I'll be talking about happened in France. The jail I taught in was fairly typical of French Maisons d’arrêt, but even these vary greatly. Some of the pages I link to below are only available in French. I don't teach in prison anymore. Since most of the differences are lost in translation anyway, I’ll use inmate, detainee, prisoner, and convict interchangeably. Same with jail and prison, as well as prison officer and prison guard.
    Context
    As of December 2018, there are 188 prisons in France and about 80,000 inmates. They are handled by the Penitentiary Administration (Administration Pénitentiaire), which comprises prison guards (28,000) and wardens as well as probation officers (5,000), who are in charge of a total of 250,000 people (the remaining 170,000 are either on probation or serving other sentences, such as electronic monitoring or community service) and work with an annual budget of 2.79 billion euros. The prison I taught in was a Maison d’Arrêt. These are the most common kind of facility (there are 130 of them), in which you find detainees incarcerated for a short amount of time (in theory, less than two years) and prisoners accused of more serious offenses who are still awaiting their final verdict, at which point they will be transferred to another jail. All convicts were over 18. The jail where I taught had 1,000 inmates, 200 of whom were women. In France, only 3.8% of inmates are women.
    All the figures come from this page (link in French).
    Maisons d’arrêt are where prison overpopulation is most serious: our prison had a density of around 140%, meaning there were 1.4 as many inmates as beds theoretically available. This is average for Maisons d’arrêt, with maximal densities reaching 200%. It’s also in Maisons d’arrêt that the detainee’s freedom is most restricted: normally, they literally stay in their cell all the time, where they can only interact with their cellmate (or by passing stuff through the window gratings, though this is theoretically forbidden), except for planned activities (school, manual labor, sports, the occasional movie screening or cultural performance), and recess.
    Inmates are allowed a TV (except in disciplinary quarters), and there are extremely rarely more than 2 convicts per cell (who sleep on a bunk bed). Material conditions vary a lot between jails: mine was fairly recent, whereas a detention center I visited had alleys still dating from the 1960s. These sorts of disparities are surprising at first. Where you commit an offense has a huge impact on what your jail time (if any) will be like. Breaking the law in Paris, where prisons are flooded, or in Bordeaux, which is calmer, can lead to very different outcomes: you might get incarcerated only in the second situation. Even within a region where you offend also has an influence: you can end up in a big (say 1,000 inmates), impersonal but fairly modern jail, or in a very small (50 inmates), crummy yet familial prison. Prisons have reputations, and judges will often assign inmates accordingly.
    This being said, an inmate can request to be transferred to a jail that is closer to their family, although (a recurring pattern) the administrative handling of the request may take a long time.
    People are either very surprise...
    20 min
  • LW - AI Alignment Research Engineer Accelerator (ARENA): call for applicants by TheMcDouglas
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI Alignment Research Engineer Accelerator (ARENA): call for applicants, published by TheMcDouglas on April 17, 2023 on LessWrong.
    TL;DR
    Apply here for the second iteration of ARENA!
    Introduction
    We are excited to announce the second iteration of ARENA (Alignment Research Engineer Accelerator), a 6-week ML bootcamp with a focus on AI safety. Our mission is to prepare participants for full-time careers as research engineers in AI safety, e.g. at leading organizations or as independent researchers.
    The program will commence on May 22nd, 2023, and will be held at the Moorgate WeWork offices in London. This will overlap with SERI MATS, who are also using these offices. We expect this to bring several benefits, e.g. facilitating productive discussions about AI safety & different agendas, and allowing participants to form a better picture of what working on AI safety can look like in practice.
    ARENA offers a unique opportunity for those interested in AI safety to learn valuable technical skills, engage in their own projects, and make open-source contributions to AI safety-related libraries. The program is comparable to MLAB or WMLB, but extends over a longer period to facilitate deeper dives into the content, and more open-ended project work with supervision.
    For more information, see our website.
    Outline of Content
    The 6-week program will be structured as follows:
    Chapter 0 - Fundamentals
    Before getting into more advanced topics, we first cover the basics of deep learning, including basic machine learning terminology, what neural networks are, and how to train them. We will also cover some subjects we expect to be useful going forwards, e.g. using GPT-3 and 4 to streamline your learning, good coding practices, and version control.
    Topics include:
    PyTorch basics
    CNNs, Residual Neural Networks
    Optimization
    Backpropagation
    Hyperparameter search with Weights and Biases
    Model training & PyTorch Lightning
    Duration: 5 days
    Chapter 1 - Transformers & Mechanistic Interpretability
    In this chapter, you will learn all about transformers, and build and train your own. You'll also learn about Mechanistic Interpretability of transformers, a field which has been advanced by Anthropic’s Transformer Circuits sequence, and open-source work by Neel Nanda.
    Topics include:
    GPT models (building your own GPT-2)
    Training and sampling from transformers
    TransformerLens
    In-context Learning and Induction Heads
    Indirect Object Identification
    Superposition
    Duration: 9 days
    Chapter 2 - Reinforcement Learning
    In this chapter, you will learn about some of the fundamentals of RL, and work with OpenAI’s Gym environment to run their own experiments.
    Topics include:
    Fundamentals of RL
    Vanilla Policy Gradient
    PPO
    Deep Q-learning
    RLHF
    Gym & Gymnasium environments
    Duration: 6 days
    Chapter 3 - Training at Scale
    There are a number of techniques that are helpful for training large-scale models efficiently. Here, you will learn more about these techniques and how to use them. The focus is on hands-on learning, rather than just a theoretical understanding.
    Topics include:
    GPUs
    Distributed computing
    Data/tensor/pipeline parallelism
    Finetuning LLMs
    Duration: 4 days
    Chapter 4 - Capstone Projects
    We will conclude this program with capstone projects, where participants get to dig into something related to the course. This should draw on much of the skills and knowledge participants will have accumulated over the last 5 weeks.
    Duration: 6 days
    Below is a diagram of the curriculum as a whole, and the dependencies between sections.
    Here is some sample material from the course, which you will be able to full understand once you reach that point in the course. This notebook is on Indirect Object Identification (from the chapter on Transformers & Mechanistic Interpretability), it will represent one of a set of optional 2-day mi...
    11 min
  • LW - Encouraging New Users To Bet On Their Beliefs by YafahEdelman
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Encouraging New Users To Bet On Their Beliefs, published by YafahEdelman on April 18, 2023 on LessWrong.
    [Epistemic Status: I think this post gestures in the right direction and the ideas in it are positive on the margin. However I am uncertain what I would write if I put substantially more time into thinking on the subject and think it plausible that discussion on this post will cause me to change my mind.]
    tl;dr: I think LessWrong would benefit from new users betting on predictions at the top of their posts/comments. This might be difficult to do, so in order to encourage it I am offering to subsidize markets created by new LessWrong users on Manifold - a site for betting with play money for free.
    Sometimes it is important to be able to discuss things without exposing them to harsh analytical criticism or being forced into objective stances. This probably applies even more often to people new to LessWrong who may not be used to the particular epistemic standards here. However, if you think you are comfortable submitting your ideas to more rigorous scrutiny, one thing you can do is bet on them.
    Betting on your beliefs is a major part of the culture of LessWrong. Beliefs that someone is willing to bet on are taken more seriously and can help ground discussion. Recently, the site Manifold.markets has provided a way to do this for free, using play money called mana to bet on user created (and resolved) markets. They also provide a subsidy feature, where users can subsidize markets to encourage betting on them.
    I think a lot of recent content by new users on LessWrong gestures at concepts sort of vaguely or proposes a theory without communicating the writer's level of confidence, which isn't unreasonable - communicating in a novel way is hard! But this has the unfortunate result of making it difficult to have productive conversations, at least for me, as I often end up feeling like I might disagree with the writer's underlying beliefs but those beliefs aren't made explicitly enough for me to contradict.
    In order to encourage betting by new users, I am offering to subsidize manifold markets new users include within their posts or replies. I intend do this for allsuch markets I see mentioned by new users that I see (see footnotes for qualifications/details). I recommend putting the markets in the top of your comment/reply to maximize the chance that I notice them. This should provide an incentive (in mana) to make such markets, as well as increasing the accuracy of those markets by incentivizing betting on them.
    Some proposals for markets you can create, ideally with expiry dates in the near future (next few months) with a probability that is not too close to 0% or 100% in your point of view.
    Whether a newsworthy event will happen. It doesn’t need to perfectly capture all of what you are arguing, just “will a major newspaper report in the next year that X happened” works.
    Whether a decision you are advocating for will be made.
    Whether there will be a substantial amount of interest in a few months about a topic you think is promising.
    Whether a project you are proposing will be successful, or continue to exist.
    Whether a prominent researcher will publicly agree with something.
    I like this post which discusses that.
    This does imply that new users to LessWrong who may lack experience betting are at a disadvantage when betting against users who have been around awhile and have practice betting. This is why I'm offering to subsidize bets users make rather than just offering to bet against them.
    How large the subsidy is may vary but I'll start off by defaulting to 100 mana for markets in posts and 20 mana for markets in replies.
    All markets within reason, if someone tries to abuse this I do not intend to subsidize them.
    Where new user includes anyone with a sprout icon next to ...
    4 min