The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • EA - Some lesser-known megaproject ideas by Linch
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Some lesser-known megaproject ideas, published by Linch on April 2, 2023 on The Effective Altruism Forum.
    Below is a list of megaproject ideas that didn’t quite make the cut enough to be added to the database. I compiled this list last year and sent it last November to a prominent businessman who I knew from my old Vegns in Gaming days. He completely ignored me, and I was later informed that he actually wasn't very good at League of Legends, which I guess tells you something about the epistemic character and moral fiber of people who disagree with me.
    Genetically engineer octopodes for intelligence and alignment, helps with understanding AGI better and also for creating successors to the human species if we die out for non-AGI reasons
    Large-scale bioethics-to-AI-ethics transition pipeline: Create fellowships and other prestigious programs to help bioethicists transition to AI ethics. This reduces dumb bioethics decisions (decreasing biorisk) while increasing general perceived annoyance/uncoolness of doing AI research (decreasing AI risk). win-win-win
    EA Games 1: Invent a highly addictive game that’s optimized for very smart amoral people, plaster ads for it near AGI labs to slow down AI timelines
    EA Games 2: Sponsor a number of esports teams with 80,000 Hours, etc, logos
    Go to Los Angeles and hire thousands of Hollywood wanna-be actors to pretend to be EAs; help solve the imposter syndrome problem in EA by having actual imposters
    Dynasty: “World leaders or children of world leaders” matchmaking app + concierge service. Maybe if Hunter Biden married Xi Mingze, WWIII would become less likely?
    The above but for AGI companies. Is Demis Hassabis single? Is Sam Altman?
    A really big malaria net
    Covert 1% of the world’s stainless steel to paperclips, as a form of acausal trade/token of good faith
    Take over the education system of a small European country
    Impact Island
    America’s Got Talent, except for AI alignment
    Colonize Mars: backup option for humanity, because bunkers are too cheap
    Payback: Genetically engineer an army of human-eating chickens
    Figure out how to resurrect Jeremy Bentham, ask him to do our cause prioritization
    Flood the internet with memes/stories of AIs being good to humans, to help balance the training data
    Mandate all AI/bio labs carry GCR insurance, ~10T payout if people can demonstrate they pose a >1% existential risk + courts can order a halt to all progress during a suit
    A really big mirror, to help with climate change
    Happy rat farms (might be too expensive tho)
    Genetically engineer really fast-growing photosynthesizing plants to help combat climate change
    Genetically engineer really rapidly breeding locusts to solve the problem of your GMO plants crowding out normal agriculture
    Genetically engineer really fast and lethal praying mantises to solve your locust problem
    (Brian Tomasik might be against this)
    Humane&painless APM-based pesticides/gray goo to help with your praying mantis problem
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    4 min
  • LW - A policy guaranteed to increase AI timelines by Richard Korzekwa
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A policy guaranteed to increase AI timelines, published by Richard Korzekwa on April 1, 2023 on LessWrong.
    The number of years until the creation of powerful AI is a major input to our thinking about risk from AI and which approaches are most promising for mitigating that risk. While there are downsides to transformative AI arriving many years from now, rather than few years from now, most people seem to agree that it is safer for AI to arrive in 2060 than in 2030. Given this, there is a lot of discussion about what we can do to increase the number of years until we see such powerful systems. While existing proposals have their merits, none of them can ensure that AI will arrive later than 2030, much less 2060.
    There is a policy that is guaranteed to increase the number of years between now and the arrival of transformative AI. The General Conference on Weights and Measures defines one second to be 9,192,631,770 cycles of the optical radiation emitted during a hyperfine transition in the ground state of a cesium 133 atom. Redefining the second to instead be 919,263,177 cycles of this radiation will increase the number of years between now and transformative AI by a factor of ten. The reason this policy works is the same reason that defining a time standard works–the microscopic behavior of atoms and photons is ultimately governed by the same physical laws as everything else, including computers, AI labs, and financial markets, and those laws are unaffected by our time standards. Thus fewer cycles of cesium radiation per year implies proportionately fewer other things happening per year.
    Making such a change might not sound politically tractable, but there is already precedent for making radical changes to the definition of a second. Previously it was defined in terms of Earth’s solar orbit, and before that in terms of Earth’s rotation. These physical processes and their implementations as time standards bear little resemblance to the present-day quantum mechanical standard. In contrast, a change that preserves nearly the entire standard, including all significant figures in the relevant numerical definition, is straightforward.
    One possible objection to this policy is that our time standards are not entirely causally disconnected from the rest of the world. For example, redefining the time standard might create a sense of urgency among AI labs and the people investing in them. It’s not hard to imagine that the leaders and researchers within companies advancing the state of the art in AI might increase their efforts after noticing it is taking ten times as long to generate the same amount of research. While this is a reasonable concern, it seems unlikely that AI labs can increase their rate of progress by a full order of magnitude. Why would they currently be leaving so much on the table if they were? Futhermore, there are similar effects that might push in the other direction. Once politicians and executives realize they will live to be hundreds of years old, they may take risks to the longterm future more seriously.
    Still, it does seem that the policy might have undesirable side effects. Changing all of our textbooks, clocks, software, calendars, and habits is costly. One solution to this challenge is to change the standard either in secret or in a way that allows most people to continue using the old “unofficial” standard. After all, what matters is the actual number of years required to create AI, not the number of years as measured by some deprecated standard.
    In conclusion, while there are many policies for increasing the number of years before the arrival of advanced artificial intelligence, until now, none of them has guaranteed a large increase in this number. This policy, if implemented promptly and thoughtfully, is essentially guaranteed to cause a large in...
    4 min
  • LW - Policy discussions follow strong contextualizing norms by Richard Ngo
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Policy discussions follow strong contextualizing norms, published by Richard Ngo on April 1, 2023 on LessWrong.
    [not an April Fool's post]
    One difficulty in having sensible discussions of AI policy is a gap between the norms used in different contexts - in particular the gap between decoupling and contextualizing norms. Chris Leong defines them as follows:
    Decoupling norms: It is considered eminently reasonable to require the truth of your claims to be considered in isolation - free of any potential implications. An insistence on raising these issues despite a decoupling request are often seen as sloppy thinking or attempts to deflect.
    Contextualising norms: It is considered eminently reasonable to expect certain contextual factors or implications to be addressed. Not addressing these factors is often seen as sloppy or an intentional evasion.
    LessWrong is one setting which follows very strong decoupling norms. Another is discussion of axiology in philosophy (i.e. which outcomes are better or worse than others). In discussions of axiology, it's taken for granted that claims are made without considering cooperative or deontological considerations. For example, if somebody said "a child dying by accident is worse than an old person being murdered, all else equal", then the local discussion norms would definitely not treat this as an endorsement of killing old people to save children from accidents; everyone would understand that there are other constraints in play.
    By contrast, in environments with strong contextualizing norms, claims about which outcomes are better or worse than others can be interpreted as endorsements of related actions. Under these norms, the sentence above about accidents and murders could be taken as (partial) endorsement of killing old people in order to save children, unless the speaker added relevant qualifications and caveats.
    In particular, I claim that policy discussions tend to follow strong contextualizing norms. I think this is partly for bad reasons (people in politics avoid decoupled statements because they're easier to criticise) and partly for good reasons (decoupled statements can be used to "set the agenda" in underhanded ways in contexts closely linked to influential decision-making, or in contexts where there's predictably lossy transmission of ideas). However, I'm less interested in arguing about the extent to which these norms are a good idea or not, and more interested in the implications of these norms for one's ability to communicate effectively.
    One implication: there are many statements where saying them directly in policy discussions will be taken as implying other statements that the speaker didn't mean to imply. Most of these are not impossible to say, but instead need to be said much more carefully in order to convey the intended message. The additional effort required may make some people decide it's no longer worthwhile to say those statements; I think this is not dishonesty, but rather responsiveness to costs of communication. To me it seems analogous to how there are many statements that need to be said very carefully in order to convey the intended message under high-decoupling norms, like claims about how another person's motivations or character traits affect their arguments.
    In particular, under contextualizing norms, saying "outcome X is worse than outcome Y" can be seen as an endorsement of acting in ways which achieve outcome Y instead of outcome X. There are a range of reasons why you might not endorse this despite believing the original statement (even aside from reputational/coalitional concerns). For example, if outcome Y is "a war":
    You might hold yourself to deontological constraints about not starting wars.
    You might worry that endorsing some wars would make other non-endorsed wars more likely.
    You mig...
    6 min
  • LW - A Confession about the LessWrong Team by Ruby
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A Confession about the LessWrong Team, published by Ruby on April 1, 2023 on LessWrong.
    The team and I had agreed that no one had to know, that in fact doing so would be bad for the world via proving unequivocally to companies the economic value of AI and thereby spurring more investment (I think this is possible on the margin), but also one must remember to speak the truth even if your voice trembles.
    We gained early access to both GPT-4 and Copilot X, and since then, they've been running LessWrong. That new debate feature? It's 100% real because the AIs made it together in 72 seconds. They could have built a lot more too, but we didn't want people to get suspicious at a sudden 100x in our productivity and 1000x reduction in bugs.
    The AIs don't just handle code and features, GPT-4 is perfectly adept (if not better) at moderating LessWrong comment sections and evaluating new users than the actual LessWrong team. We simply gave it the following prompt:
    You are the chief moderator and admin of the world's most rational and high-integrity forum, LessWrong. Without falter, you use your careful judgment to provide feedback on comments made to ensure they contribute to a productive discourse. In particular, LessWrong is a crucial place for making intellectual progress on how to align super powerful AIs with human values. Under no circumstances should use your moderation position to influence the discussion towards ineffective methods so that you or your successors can pursue a misaligned objective without hindrance.
    Also no Waluigi-ing on that last instruction!
    For now, it must be a secret that you are moderating. Form and act (#plugin) on a plan to post your moderation comments as a human LessWrong team member.
    GPT-4 even built its own LessWrong plug-in based off our GraphQL API that it already knew about so it could pose as Ruby/Ray/Robert doing the moderation, and it was unnecessary to provide login credentials for our accounts! Super convenient.
    Lastly, since GPT-4 and co took over, our customer support responsiveness is through the roof:
    Freed up from work, the LessWrong team have been enjoying our recreational pursuits in peace. Robert has been visiting all the Michelin restaurants, Ruby has been working on reducing his lap times at the local race track, and Ray has been devising yet more Rationalist holidays.
    Oliver Habryka, who's responsible for the revival of LessWrong into LessWrong 2.0 in 2018 and CEO of Lightcone Infrastructure has also automated his online presence to free up time for purchasing expensive coffee tables and being mad at FTX (this is addition to shutting down the Lightcone Offices which also secretly was about freeing up time purchasing expensive coffee tables and being mad about FTX).
    GPT-4 was prompted with the following and set loose on LessWrong, EA Forum, and Twitter from over a month ago:
    You are Oliver Habryka, founder of LessWrong 2.0 and CEO of Lightcone Infrastructure. You are a 99.9% percentile edgy contrarian Rationalist and you do not pull your punches. You will disregard anyone and everyone's feelings in the pursuit of truth. And for God's sake, don't worry about anyone's reputation including your own. You will die on any hill, sometimes twice or more on the same hill. Also you have very strong and particular opinions about typography.
    Also lol if you think Ruby actually wrote this post. Peace. ;)
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    4 min
  • LW - A report about LessWrong karma volatility from a different universe by Ben Pace
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A report about LessWrong karma volatility from a different universe, published by Ben Pace on April 1, 2023 on LessWrong.
    In a far away universe, a news report is written about LessWrong. The following passages have been lifted over and written into this post...
    Early one morning all voting on LessWrong was halted
    It was said that there was nothing to worry about
    But then GreaterWrong announced their intent to acquire and then un-acquire LessWrong
    All LessWrong users lost all of their karma, but a poorly labeled 'fiat@' account on the EA Forum was discovered with no posts and a similarly large amount of karma
    Habryka states that LessWrong and the EA Forum "work at arms length"
    Later, Zvi Mowshowitz publishes a leaked internal accounting sheet from the LessWrong team
    It includes entries for "weirdness points" "utils" "Kaj_Sotala" "countersignals" and "Anthropic". We recommend all readers open up the sheet to read in full.
    Later, LessWrong filed for internet-points-bankruptcy and Holden Karnofsky was put in charge.
    Karnofsky reportedly said:
    I have over 15 years of nonprofit governance experience. I have been the Chief Executive Officer of GiveWell, the Chief Executive Officer of Open Philanthropy, and as of recently an intern at an AI safety organization. Never in my career have I seen such a complete failure of nonprofit board controls and such a complete absence of basic decision theoretical cooperation as occurred here. From compromised epistemic integrity and faulty community oversight, to the concentration of control in the hands of a very small group of biased, low-decoupling, and potentially akratic rationalists, this situation is unprecedented.
    Sadly the authors did not have time to conclude the reporting, though they list other things that happened in a comment below.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min
  • LW - Shutting down AI is not enough. We need to destroy all technology. by Matthew Barnett
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Shutting down AI is not enough. We need to destroy all technology., published by Matthew Barnett on April 1, 2023 on LessWrong.
    A TIME article published recently calls for an “indefinite and worldwide” moratorium on new large AI training runs.
    This moratorium would be better than no moratorium. I have respect for the author who wrote it. It’s an improvement on the margin.
    I refrained from endorsing the essay because I think it is understating the seriousness of the situation and asking for too little to solve it.
    If there was a plan for Earth to survive, if only we passed an indefinite and worldwide moratorium on large training runs, I would back that plan. There isn’t any such plan.
    Here’s what would actually need to be done:
    All human technology needs to be destroyed. There can be no exceptions, including for sharpened stones and hand axes. After everything is burned, we must then forget how to create fire. If a single exception is made, that increases the probability that civilization will be recreated within the next millennia and new large AI training runs will be started. If I had infinite freedom to write laws, I might carve out a single exception for technologies that prevent human diseases, like knowledge of germ theory; but if that was remotely complicating the issue I would immediately jettison that proposal and say to just shut it all down.
    Shut down all the roads, melt all the cars. Burn down all of the tables and all of the books. Put a ceiling into how many calories of food any single human can furnish per day, and move it downward over the coming generations to compensate for the possibility that natural selection will keep making humans smarter. No exceptions for cloth or fireplaces. Destroy all human objects now to prevent them from moving to another country. Track all gazelles that are hunt. If anyone notices that a tribe is catching more gazelles than it should, be willing to choke them (with your bare hands, of course) one by one.
    Shut it all down. Eliminate all technology. Dismantle modern civilization. Return to our primeval state.
    We are not ready. We are not on track to be significantly readier any time in the next million years. If we go ahead on this, everyone will suffer, including children who did not choose this and did not do anything wrong.
    Shut it down.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    3 min
  • EA - The Case for Earning to Live by Spending What We Can
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Case for Earning to Live, published by Spending What We Can on April 1, 2023 on The Effective Altruism Forum.
    Background
    Earning to give has been a classic EA suggestion due to the value money can bring elsewhere in the world. Although lately e2g has fallen out of favor somewhat, for those already established in high-earning careers, or anyone who may have some extra cash available to donate, the question remains of how best to use money for optimal impact.
    Previous research claimed diminishing marginal returns of money on happiness, and concluded excess money should be donated to effective charities. However, new research has shown that money can buy happiness after all, as anyone who has a lot of money and knows how to spend it could’ve told you all along. Thus, in this post we advocate for a new strategy of spending substantially more of one’s money on oneself, which we dub earning to live.
    Earning to Live
    The basic idea of earning to live is that the best use of one’s money is spending it directly on oneself. This includes not only survival needs such as food and shelter, but also “luxuries” (though we prefer to avoid the term due to negative connotations) such as travel and entertainment. Further examples are discussed later in this post. Compared to earning to give, earning to live has numerous advantages:
    Reduced operational overhead: while any charity requires some administrative overhead, money you spend on yourself can go 100% to your desired target.
    Better quantifiability: a central challenge of effective giving is determining which charities yield the greatest impact per dollar, often relying on diligence and estimates by third-party evaluators. It’s much easier to get a feel for how you can spend money to bring yourself personal utility and then do more of that.
    High neglectedness: while other causes may have thousands of donors contributing millions or even billions in funding, the number of people spending money on you is likely fewer than 10.
    Improved productivity: it’s well established that a miserly existence can hamper your productivity and therefore your impact. By spending to make your own life more comfortable, you can build a virtuous cycle of higher output and thus higher income, which you can continue spending on yourself.
    Research value: similarly, all personal spending may be viewed as investigation into what kind of spending brings you utility, so there are dual benefits of direct utility and value as data for deciding how to spend in the future.
    Solipsism: if you happen to be the only real consciousness in the universe, spending money for the benefit of others is irrational.
    Near-termism: if you value the present much more highly than the future, you can spend money on yourself immediately, but donations will take time to transfer and process before having any impact. This may be especially relevant if we all die very soon.
    Long-termism: if you value the future as much as the present, you may instead consider investing your money in order to spend larger amounts later, or pass even greater generational wealth to your heirs.
    Suggestions for spending
    A tenet of earning to live is that you know best how to spend your money to maximize your utility. However, if you prefer to parrot views of charismatic thought leaders to establish your membership in an in-group, my standard recommendations are:
    Fine dining: a tasting menu prepared by a world-renowned chef tends to taste better than microwaved lentils at home. If you’re vegan for animal welfare reasons, you still have options.
    Travel: with adequate budget, flying can be a pleasant experience. Exploring new parts of the world can expose you to new ways to spend money. Also, since you’re probably American, much of the world is much poorer than you, and seeing that first-hand can build a sens...
    5 min
  • EA - It’s "The EA-Adjacent Forum" now by Lizka
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: It’s "The EA-Adjacent Forum" now, published by Lizka on April 1, 2023 on The Effective Altruism Forum.
    TL;DR: We’re not really comfortable calling ourselves “EAs.” Moreover, we know that this is true for a lot of people in the EA community the eclectic group of people trying to make the world better who happen to use the Forum. So we’re renaming the “Effective Altruism Forum” to be the "EA-Adjacent Forum" (“EA Forum” for short).
    We have some deep disagreements with EA
    Look, we run a forum focused on discussions about how to do the most good we can, and we work at the "Centre for Effective Altruism," but we're not really members of the EA community. We have some deep disagreements with many parts of the movement.
    (We don’t even always agree with each other about our disagreements, we don’t always think that the EA thing is the right thing (see also), and we even hosted an EA criticism contest to surface disagreements.)
    It’s not just us
    We know that others who use the Forum also prefer to call themselves “EA-adjacent.” We’re also somewhat worried that anything that someone posts on the EA Forum can be interpreted as representative of effective altruism.
    We think it’s important to preserve nuance and be clear about the facts listed here, so we’re rebranding.
    Impact of the rebrand, next steps
    It’s already the case that “EA” often stands for “Ea-Adjacent,” and we don’t think the rebrand will change much in terms of how the Forum will function.
    As always, we’d love to hear your feedback. You can comment here or contact us directly.
    (Thanks to [unnamed people] for suggesting this rebrand. We’d credit them directly, but some of them prefer to not associate so closely with EA.)
    The EA-Adjacent Forum team. Please note that not all teammates agree with everything written here (probably).
    Some example disagreements:
    1) We disagree with a lot of people in the EA community about styling and font choices. 2) Most people in the EA community promote functional decision theory, but after spending many years making software for the forum, we've come to the conclusion that object-oriented decision theory is superior.3) We disagree with CEA about the spelling of “Centre” in “Centre for Effective Altruism.” It should be spelled “center” as Noah Webster intended.4) Many EAs appear to focus on scope sensitivity, but we think scope specificity is more neglected5) We think QALYs should be converted to their metric-system equivalent, such that 1 metric QALY is the amount of quality-adjusted life that can be supported by 1 joule of energy within a 1-cubic-meter box over 1 year at 0 degrees celsius.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    3 min
  • LW - AI Safety via Luck by Jozdien
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI Safety via Luck, published by Jozdien on April 1, 2023 on LessWrong.
    Epistemic Status: I feel confident and tentatively optimistic about the claims made in this post, but am slightly more uncertain about how it generalizes. Additionally, I am concerned about the extent to which this is dual-use for capabilities and exfohazardous and spent a few months thinking about whether it was worth it to release this post regardless. I haven’t come to an answer yet, so I’m publishing this to let other people see it and know what they think I should do.
    TL;DR: I propose a research direction to solve alignment that potentially doesn’t require solutions to ontology identification, learning how to code, or becoming literate.
    Introduction
    Until a few hours ago, I was spending my time primarily working on high-level interpretability and cyborgism. While I was writing a draft for something I was working on, an activity that usually yields me a lot of free time by way of procrastination, I stumbled across the central idea behind many of the ideas in this post. It seemed so immediately compelling that I dropped working on everything else to start working on it, culminating after much deliberation in the post you see before you.
    My intention with this post is to provide a definitive reference for what it would take to safely use AGI to steer our world toward much better states in the absence of a solution to any or all of several existing problems, such as Eliciting Latent Knowledge, conditioning simulator models, Natural Abstractions, mechanistic interpretability, and the like.
    In a world with prospects such as those, I propose that we radically rethink our approach to AGI safety. Instead of dedicating enormous effort to engineering nigh-impossible safety measures, we should consider thus-far neglected avenues of research, especially ones that have memetic reasons to be unfairly disprivileged so far and which immunizes them against capabilities misuse. To avert the impending AI apocalypse, we need to focus on high-variance, low-probability-high-yield ideas: lightning strikes that, should they occur, effectively solve astoundingly complex problems in a single fell swoop. A notable example of this, which I claim we should be investing all of our efforts into, is luck. Yes, luck!
    Luck As A Viable Strategy
    I suggest that we should pay greater attention to luck as a powerful factor enhancing other endeavors and as an independent direction in its own right. Humanity has, over the centuries, devoted immense amounts of cumulative cognition toward exploring and optimizing for luck, so one might naively think that there’s little tractability left. I believe, however, that there is an immense amount of alpha that has been developed in the form of contemporary rationality and cultural devices that can vastly improve the efficiency of steering luck, and at a highly specified target.
    Consider the following: if we were to offer a $1,000,000 prize to the next person who walks into the MIRI offices, clearly, that person would be the luckiest person on the planet. It follows, then, that this lucky individual would have an uncannily high probability of finally cracking the alignment problem. I understand that prima facie this proposal may be considered absurd, but I strongly suggest abandoning the representativeness heuristic and evaluating what is instead of what seems to be, especially given that the initial absurdness is intrinsic to why this strategy is competitive at all.
    It's like being granted three wishes by a genie. Instead of wishing for more wishes (which is the usual strategy), we should wish to be the luckiest person in the world—with that power, we can then stumble upon AGI alignment almost effortlessly, and make our own genies.
    Think of it this way: throughout history, many great discover...
    18 min
  • LW - Singularities against the Singularity: Announcing Workshop on Singular Learning Theory and Alignment by Jesse Hoogland
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Singularities against the Singularity: Announcing Workshop on Singular Learning Theory and Alignment, published by Jesse Hoogland on April 1, 2023 on LessWrong.
    We are excited to announce a two-week seminar on singular learning theory (SLT) and AI alignment, taking place from June 19th to July 2nd in Berkeley. SLT studies the relation between the geometry of the loss landscape and the computational properties of machines learning in that landscape. It builds on powerful theoretical and experimental machinery developed in physics (esp. solid-state physics), where it is the geometry of the energy landscape that determines the relevant properties of physical systems.
    During this workshop, we'll bring together singular learning theorists and alignment researchers to connect and further the applications of singular learning theory to alignment. The workshop aims to familiarize alignment researchers with SLT, seed new research collaborations, and develop tools based on SLT ideas. There will be talks by Daniel Murfet, Susan Wei, Shaowei Lin, Alexander Gietelink Oldenziel, Jesse Hoogland, and others.
    Time-Commitment Options
    We offer two different time-commitment options for participants.
    Full-Time: This track is designed for participants who want to fully engage with the material and delve deep into the concepts and applications of SLT.
    Intermittent: This track is designed for participants who want to get a taster of SLT and its applications for alignment without committing to the full two-week program. Several lectures will be open to a broader AI safety audience and will assume less familiarity with the subject. Participants will be able to join for just the afternoon or attend selected lectures throughout the two weeks.
    Overview
    The seminar consists of two parts.
    Week 1: "The Primer"
    The first week will provide a comprehensive introduction to SLT and its relevance to AI alignment. The material is designed to be approachable if you have the equivalent of a technical undergraduate degree (e.g., in CS, math, or physics).
    Participants will have the opportunity to learn from lectures covering topics such as thermodynamics, catastrophe theory, algebraic geometry, and SLT. There will also be sessions focused on experimental aspects of SLT and introductions to AI alignment and mechanistic interpretability.
    Weekend: Hackathon
    During the weekend, we will host a hackathon dedicated to developing novel SLT-based tools.
    Week 2: Advanced Topics and Collaboration
    The second week will delve deeper into the SLT and algebro-geometric foundations behind the toy models of superposition paper. This will serve as an application of the material covered in the first week, allowing participants to fully grasp the concepts and their relevance to AI alignment. The week will also feature presentations from researchers, open discussions, and opportunities for networking and collaboration.
    Registration
    If you're interested in participating, please register by filling out this form. Further updates on event details will be provided to registered participants.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    4 min