The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • EA - Speak the truth, even if your voice trembles by RobertM
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Speak the truth, even if your voice trembles, published by RobertM on January 14, 2023 on The Effective Altruism Forum.
    Epistemic status: Motivated by the feeling that there's something like a missing mood in the EA sphere. Informed by my personal experience, not by rigorous survey. Probably a bit scattershot, but it's already more than a month after I wanted to publish this. (Minus this parenthetical, this post was entirely written before the Bostrom thing. I just kept forgetting to post it.)
    The last half year - the time since I moved to Berkeley to work on LessWrong, and consequently found myself embedded in the broader Bay Area rationality & EA communities - have been surprisingly normal.
    The weeks following the FTX collapse, admittedly, a little less so.
    One thing has kept coming up, though. I keep hearing that people are reluctant to voice disagreements, criticisms, or concerns they have, and each time I do a double-take. (My consistent surprise is part of what prompted me to write this post: both those generating the surprise, and those who are surprised like me, might benefit from this perspective.)
    The type of issue where one person has an unpleasant interaction with another person is difficult to navigate. The current solution of discussing those things with the CEA Community Health team at least tries to balance both concerns of reducing false positive and false negatives; earlier and more public discussion of those concerns is not a Pareto-improvement.
    But most of them are other fears: that you will annoy an important funder, by criticizing ideas that they support, or by raising concerns about their honesty, given publicly-available evidence, or something similar. And the degree to which these fears have shaped the epistemic landscape makes me feel like I took a wrong turn somewhere and ended up in a mirror universe.
    Having these fears - probably common! Discussing those fears in public - not crazy! Acting on those fears? (I keep running face-first into the fact that not everybody has read The Sequences, that not everybody who has read them has internalized them, and that not everybody who has internalized them has externalized that understanding through their actions.)
    My take is that acting on those fears, by not publishing that criticism, or raising those concerns, with receipts attached, is harmful. For simplicity's sake, let's consider the cartesian product of the options:
    to publicize a criticism, or not
    the criticism being accurate, or not
    the funder deciding to fund your work, or not
    The set of possible outcomes:
    you publicize a criticism; the criticism is accurate; the funder funds your work
    you publicize a criticism; the criticism is accurate; the funder doesn't fund your work
    you publicize a criticism; the criticism is inaccurate; the funder funds your work
    you publicize a criticism; the criticism is inaccurate; the funder doesn't fund your work
    you don't publicize a criticism; the criticism is accurate; the funder funds your work
    you don't publicize a criticism; the criticism is accurate; the funder doesn't fund your work
    you don't publicize a criticism; the criticism is inaccurate; the funds your work
    you don't publicize a criticism; the criticism is inaccurate; the funder doesn't fund your work
    What predicted outcomes are motivating these fears? 2 and 4 are the obvious candidates.
    I won't pretend that these are impossible, or that you would necessarily see another funder step in if such a thing happened. You could very well pay costs for saying things in public. I do think that people overestimate how likely those outcomes are, or how high the costs will be, and underestimate the damage that staying silent causes to community epistemics.
    But I will bite the bullet: assuming the worst, you should pay those costs. In the long run, you do not a...
    9 min
  • LW - Beware safety-washing by Lizka
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Beware safety-washing, published by Lizka on January 13, 2023 on LessWrong.
    Tl;dr: don’t be fooled into thinking that some groups working on AI are taking “safety” concerns seriously (enough).
    Outline
    Two non-AI examples
    Greenwashing
    Humanewashing
    Definition of safety-washing
    What are the harms?
    What can (and should) we do about this?
    Note: I’m posting this in my personal capacity. All views expressed here are my own. I am also not (at all) an expert on the topic. [Also, I originally wrote this for the EA Forum and have not changed it for the cross-post. I don't know what that changes, but it might change something.]
    Two non-AI examples
    Greenwashing
    Companies “greenwash” when they mislead people into incorrectly thinking that their products or practices are climate and environment-friendly (or that the company focuses on climate-friendly work).
    Investopedia explains:
    Greenwashing is an attempt to capitalize on the growing demand for environmentally sound products.
    The term originated in the 1960s, when the hotel industry devised one of the most blatant examples of greenwashing. They placed notices in hotel rooms asking guests to reuse their towels to save the environment. The hotels enjoyed the benefit of lower laundry costs.
    Wikipedia: “[Jay Westerveld, the originator of the term] concluded that often the real objective was increased profit, and labeled this and other profitable-but-ineffective ‘environmentally-conscientious’ acts as greenwashing.” (Wikipedia also provides a long list of examples of the practice.)
    I enjoy some of the parody/art (responding to things like this) that comes out of noticing the hypocrisy of the practice.
    Humanewashing
    A similar phenomenon is the “humanewashing” of animal products. There’s a Vox article that explains this phenomenon (as it happens in the US):
    A carton of “all natural” eggs might bear an illustration of a rustic farm; packages of chicken meat are touted as “humanely raised."
    In a few cases, these sunny depictions are accurate. But far too often they mask the industrial conditions under which these animals were raised and slaughtered.
    Animal welfare and consumer protection advocates have a name for such misleading labeling: “humanewashing.” And research suggests it’s having precisely the effect that meat producers intend it to. A recent national survey by C.O.nxt, a food marketing firm, found that animal welfare and “natural” claims on meat, dairy, and egg packaging increased the intent to purchase for over half of consumers.
    ...rather than engaging in the costly endeavor of actually changing their farming practices, far too many major meat producers are attempting to assuage consumer concerns by merely changing their packaging and advertising with claims of sustainable farms and humane treatment. These efforts mislead consumers, and undermine the small sliver of farmers who have put in the hard work to actually improve animal treatment.
    If you want a resource on what food labels actually mean, here are some: one, two, three (these are most useful in the US). (If you know of a better one, please let me know. I’d especially love a resource that lists the estimated relative value of things like “free-range” vs. “cage-free,” etc., according to cited and reasonable sources.)
    Definition of safety-washing
    In brief, “safety-washing” is misleading people into thinking that some products or practices are “safe” or that safety is a big priority for a given company, when this is not the case.
    An increasing number of people believe that developing powerful AI systems is very dangerous, so companies might want to show that they are being “safe” in their work on AI.
    Being safe with AI is hard and potentially costly, so if you’re a company working on AI capabilities, you might want to overstate the extent to which you focus on “safety.”
    ...
    10 min
  • EA - A general comment on discussions of genetic group differences by anonymous8101
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A general comment on discussions of genetic group differences, published by anonymous8101 on January 14, 2023 on The Effective Altruism Forum.
    (content warning: discussion of racially motivated violence and coercion)
    I wanted to share that I think it's not bad to think about the object level question of whether there are group differences in intelligence rooted in genetic differences. This is an empirical claim, and can be true or false.
    My moral beliefs are pretty rooted in egalitarianism. I think as a matter of policy, but also as a matter of moral character, it is good and important to treat the experience of strangers as equally valuable, regardless of their class or race. I do not think more intelligent people are more worthy of moral consideration than less intelligent people. I think it can be complicated at the extremes, especially when considering digital people, animals, etc., but that this has little bearing on public policy when concerning existing humans.
    I don't think genetic group differences in intelligence are likely to be that relevant given I have short AI timelines. If we assume longer timelines, I believe the most likely places they would be important in terms of policy would be in education and reproductive technology. Whether or not there are such differences between groups now, there could easily come to be large differences through the application of embryo selection techniques or other intelligence enhancing technologies. From an egalitarian moral framework, I suspect it would be important to subsidize this technology for disadvantaged groups or individuals so that they have the same options and opportunities as everyone else. Even if genes turn out to not be a major cause of inegalitarian outcomes today, they can definitely become a major cause in the future, if we don't exercise wisdom and thoughtfulness in how we wield these technologies.
    However, as I said, I don't expect this to be very significant in practice given short AI timelines.
    Most importantly, from my perspective, it's important to be able to think about questions like this clearly, and so I want to encourage people to not feel constrained to avoid the question because of fear of social censure for merely thinking about them. For a reasonably well researched (not necessarily correct) discussion of the object level, see this post:
    [link deleted at the author's request]
    I think it's important context to keep in view that some of the worst human behaviors have involved the enslavement and subjugation of whole groups of people, or attempts to murder entire groups—racial groups, national groups, cultural groups, religious groups. The eugenics movement in the United States and elsewhere attempted to significantly curtail the reproductive freedom of many people through extremely coercive means in the not-so-distant past. Between 1907 and 1963, over 64,000 individuals were forcibly sterilized under eugenic legislation in the United States, and minority groups were especially targeted. Presently in China, tens of thousands of Uighurs are being sterilized, and while we don't have a great deal of information about it, I would predict that there is a major element of government coercion in these sterilizations.
    Coercive policies like this are extremely wrong, and plainly so. I oppose and condemn them. I am aware that the advocates of these policies sometimes used genetic group differences in abilities as justification for their coercion. This does not cause me to think that I should avoid the whole subject of genetic group differences in ability. Making this subject taboo, and sanctioning anyone who speaks of it, seems like a sure way to prevent people from actually understanding the underlying problems disadvantaged groups or individuals face. This seems likely to inhibit rather than pro...
    6 min
  • LW - A general comment on discussions of genetic group differences by anonymous8101
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A general comment on discussions of genetic group differences, published by anonymous8101 on January 14, 2023 on LessWrong.
    (content warning: discussion of racially motivated violence and coercion)
    I wanted to share that I think it's not bad to think about the object level question of whether there are group differences in intelligence rooted in genetic differences. This is an empirical claim, and can be true or false.
    My moral beliefs are pretty rooted in egalitarianism. I think as a matter of policy, but also as a matter of moral character, it is good and important to treat the experience of strangers as equally valuable, regardless of their class or race. I do not think more intelligent people are more worthy of moral consideration than less intelligent people. I think it can be complicated at the extremes, especially when considering digital people, animals, etc., but that this has little bearing on public policy when concerning existing humans.
    I don't think genetic group differences in intelligence are likely to be that relevant given I have short AI timelines. If we assume longer timelines, I believe the most likely places they would be important in terms of policy would be in education and reproductive technology. Whether or not there are such differences between groups now, there could easily come to be large differences through the application of embryo selection techniques or other intelligence enhancing technologies. From an egalitarian moral framework, I suspect it would be important to subsidize this technology for disadvantaged groups or individuals so that they have the same options and opportunities as everyone else. Even if genes turn out to not be a major cause of inegalitarian outcomes today, they can definitely become a major cause in the future, if we don't exercise wisdom and thoughtfulness in how we wield these technologies.
    However, as I said, I don't expect this to be very significant in practice given short AI timelines.
    Most importantly, from my perspective, it's important to be able to think about questions like this clearly, and so I want to encourage people to not feel constrained to avoid the question because of fear of social censure for merely thinking about them. For a reasonably well researched (not necessarily correct) discussion of the object level, see this post:
    [link deleted at the author's request; see also AnonymousCommentator's note about the racial IQ gap]
    I think it's important context to keep in view that some of the worst human behaviors have involved the enslavement and subjugation of whole groups of people, or attempts to murder entire groups—racial groups, national groups, cultural groups, religious groups. The eugenics movement in the United States and elsewhere attempted to significantly curtail the reproductive freedom of many people through extremely coercive means in the not-so-distant past. Between 1907 and 1963, over 64,000 individuals were forcibly sterilized under eugenic legislation in the United States, and minority groups were especially targeted. Presently in China, tens of thousands of Uighurs are being sterilized, and while we don't have a great deal of information about it, I would predict that there is a major element of government coercion in these sterilizations.
    Coercive policies like this are extremely wrong, and plainly so. I oppose and condemn them. I am aware that the advocates of these policies sometimes used genetic group differences in abilities as justification for their coercion. This does not cause me to think that I should avoid the whole subject of genetic group differences in ability. Making this subject taboo, and sanctioning anyone who speaks of it, seems like a sure way to prevent people from actually understanding the underlying problems disadvantaged groups or individuals face. T...
    6 min
  • LW - On Cooking With Gas by Zvi
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: On Cooking With Gas, published by Zvi on January 13, 2023 on LessWrong.
    Research meta-analysis says gas stoves give off unsafe N02 levels and increase risk of childhood asthma by 34% and are responsible for 12.7% of childhood asthma.
    None of this is new. The way safety panic works is that you focus on a particular risk this week and an attempt is made to make everyone’s life worse in its name.
    Emily Oster takes a deeper dive on the original studies. She is not impressed.
    For the most part, the other papers in the meta-analysis have similar features. They tend to show positive correlations between asthma and gas stove exposure, but the effects are a bit noisy and they do not necessarily paint a consistent picture. A 2013 paper finds, for example, a link with asthma for girls but not boys, and no links with other respiratory issues.
    What to take from this? We know that gas stoves emit nitrogen oxides and that, in general, those are not good. We know that air pollution, in general, is bad for respiratory symptoms, including asthma. So it seems very plausible that there is some link here. However: the magnitude is likely small. In most of the estimates, it’s small. And, beyond that, we do not see the kind of smoking gun in any of these data that would suggest a really consistent link.
    Another way to put this is that there are clearly many, many factors other than gas stoves that explain asthma. Some of these may also be environmental. But I’m skeptical that gas stoves play a huge role, as would be suggested by the new paper.
    The link on the state-wide level runs the other way.
    I am confident gas stoves do not prevent asthma because physics. One still notes that if this graph had been reversed it would be part of the case to ban such stoves.
    One explanation for the reversed correlation is that gas stoves cost more and positively correlate with income. This also suggests they are a superior good. Rich people choose to cook with gas. The spike at the top of the income distribution supports this. As does 80% of restaurants, who care the most about quality, choosing gas stoves, versus 40% of households. Alternatively, some of this could be because richer people live in older and more expensive buildings that happen to have gas stoves. If new construction is restricted in the most expensive areas, and old construction has more gas stoves, that could create this effect as well.
    Emily’s practical advice seems solid, if one is concerned:
    Can I do anything now?
    Assuming you are not replacing your gas stove now, is there anything you can do?
    Yes.
    First, run the hood fan on your stove when you use it. It’s not clear how much this matters, but on the margin it should help.
    Second, use a HEPA filter with an activated carbon filter. At least one study showed that installing such a filter in the kitchen lowered nitrogen dioxide levels by half as much as replacing the stove. This is a lot more effective in the data than the hood fan.
    As for me, even if the full effect in the meta-analysis was real? I. Do. Not. Care.
    There is a reason the expression ‘now you’re cooking with gas’ stuck. Electric stoves are not good. Gas stoves are a Nice Thing.
    You can attempt to pry my nice thing from my future no longer functioning asthmatic hands. I am so done with taking the Nice Things in life and ruining them in the name of marginally better health. They also increase resiliency in a blackout.
    We had an electric stove in Warwick. We adapted, but it was clearly worse. Neither me nor my wife ever fully adjusted, and she grew up with an electric stove. I have been in otherwise quite nice Airbnbs with electric stoves so slow and terrible that they made me not want to cook breakfast. I have yet to see a good one.
    There are those who disagree with my cooking preference for gas stoves. They say that no, you...
    10 min
  • EA - Iron deficiencies are very bad and you should treat them by Elizabeth
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Iron deficiencies are very bad and you should treat them, published by Elizabeth on January 13, 2023 on The Effective Altruism Forum.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    1 min
  • AF - Concrete Reasons for Hope about AI by Zac Hatfield-Dodds
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Concrete Reasons for Hope about AI, published by Zac Hatfield-Dodds on January 14, 2023 on The AI Alignment Forum.
    Recent advances in machine learning—in reinforcement learning, language modeling, image and video generation, translation and transcription models, etc.—without similarly striking safety results, have rather dampened the mood in many AI Safety circles. If I was any less concerned by extinction risks from AI, I would have finished my PhD[1] as planned before moving from Australia to SF to work at Anthropic; I believe that the situation is both urgent and important.[2]
    On the other hand, despair is neither instrumentally nor terminally valuable.[3] This essay therefore lays out some concrete reasons for hope, which might help rebalance the emotional scales and offer some directions to move in.
    Background: a little about Anthropic
    I must emphasize here that this essay represents only my own views, and not those of my employer. I’ll try to make this clear by restricting we to actions, and using I for opinions to avoid attributing my own views to my colleagues. Please forgive any lapses of style or substance.
    Anthropic’s raison d’etre is AI Safety. It was founded in early 2021, as a public benefit corporation,[4] and focuses on empirical research with advanced ML systems. I see our work as having four key pillars:
    Training near-SOTA models. This ensures that our safety work will in fact be relevant to cutting-edge systems, and we’ve found that many alignment techniques only work at large scales.[5] Understanding how capabilities emerge over model scale and training-time seems vital for safety, as a basis to proceed with care or as a source of evidence that continuing to scale capabilities would be immediately risky.
    Direct alignment research. There are many proposals for how advanced AI systems might be aligned, many of which can be tested empirically in near-SOTA (but not smaller) models today. We regularly produce the safest model we can with current techniques,[6] and characterize how it fails in order to inform research and policy. With RLHF as a solid baseline and building block, we're investigating more complicated but robust schemes such as constitutional AI, scalable supervision, and model-assisted evaluations.
    Interpretability research. Fully understanding models could let us rule out learned optimizers, deceptive misalignment, and more. Even limited insights would be incredibly valuable as an independent check on other alignment efforts, and might offer a second chance if they fail.
    Policy and communications. I expect AI capabilities will continue to advance, with fast-growing impacts on employment, the economy, and cybersecurity. Having high-trust relationships between labs and governments, and more generally ensuring policy-makers are well-informed, seems robustly positive.
    If you want to know more about what we’re up to, the best place to check is anthropic.com for all our published research. We’ll be posting more information about Anthropic throughout this year, as well as fleshing out the website.
    Concrete reasons for hope
    My views on alignment are similar to (my understanding of) Nate Soares’. I think the key differences are because I don’t think there’s enough evidence to confidently predict the difficulty of future problems, and I do think it’s possible for careful labs to avoid active commission of catastrophe. We also seem to have different views on how labs should respond to the situation, which this essay does not discuss.
    Language model interventions work pretty well
    I wasn’t expecting this, but our helpful/harmless/honest research is in fact going pretty well! The models are far from perfect, but we’ve made far more progress than I would have expected a year ago, and no signs of slowing down yet. HHH omits several vital pieces of the f...
    10 min
  • EA - On Living Without Idols by Rockwell
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: On Living Without Idols, published by Rockwell on January 13, 2023 on The Effective Altruism Forum.
    For many years, I've actively lived in avoidance of idolizing behavior and in pursuit of a nuanced view of even those I respect most deeply. I think this has helped me in numerous ways and has been of particular help in weathering the past few months within the EA community. Below, I discuss how I think about the act of idolizing behavior, some of my personal experiences, and how this mentality can be of use to others.
    Note: I want more people to post on the EA Forum and have their ideas taken seriously regardless of whether they conform to Forum stylistic norms. I'm perfectly capable of writing a version of this post in the style typical to the Forum, but this post is written the way I actually like to write. If this style doesn’t work for you, you might want to read the first section “Anarchists have no idols” and then skip ahead to the section “Living without idols, Pt. 1” toward the end. You’ll lose some of the insights contained in my anecdotes, but still get most of the core ideas I want to convey here.
    Anarchists have no idols.
    I wrote a Facebook post in July 2019 following a blowup in one of my communities:
    "Anarchists have no idols."Years ago, I heard this expression (that weirdly doesn't seem to exist in Google) and it really stuck with me. I think about it often. It's something I try to live by and it feels extremely timely. Whether you agree with anarchism or not, I think this is a philosophy everyone might benefit from.What this means to me: Never put someone on a pedestal. Never believe anyone is incapable of doing wrong. Always create mechanisms for accountability, even if you don't anticipate ever needing to use them. Allow people to be multifaceted. Exist in nuance. Operate with an understanding of that nuance. Cherish the good while recognizing it doesn't mean there is no bad. Remember not to hero worship. Remember your fave is probably problematic. Remember no one is too big to fail, too big for flaws. Remember that when you idolize someone, it depersonalizes the idolized and erodes your autonomy. Hold on to your autonomy. Cultivate a culture of liberty. Idolize no one.
    Idolize no one. Idolize no one.
    My mentor, Pt. 1.
    When I was in college, I had a boss I considered my mentor. She was intelligent, ethical, and skilled. She shared her expertise with me and I eagerly learned from her. She gave me responsibility and trusted me to use it well. She oversaw me without micromanaging me, and used a gentle hand to correct my course and steer my development. She saw my potential and helped me to see it, too.She also lied to me. Directly to my face. She violated an ethical principle she had previously imparted to me, involved me in the violation, and then lied to me about it. I was made an unwitting participant in something I deeply morally opposed and I experienced a major, life-shattering breach of trust from someone I deeply respected. She was my boss and my friend, but in a sense, she was also my idol. And since then, I have refused to have another.
    Abusive people do not exist.
    A month after my mentor ceased to be my mentor, I took a semester-long course, "Domestic Violence". It stands as one of the most formative experiences in my way of thinking about the world. There's a lot I could write about it, but I want to share one small tidbit here, that I wrote about a few years after the course concluded:
    More and more people are promoting a shift in our language away from talking about “abusive relationships” and toward relationships with “abusive people.” This is a small but powerful way to locate where culpability lies. It is not the relationship that is to blame, but one individual in it. I suggest taking this a step further and selectively avoiding use of...
    10 min
  • LW - How we could stumble into AI catastrophe by HoldenKarnofsky
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: How we could stumble into AI catastrophe, published by HoldenKarnofsky on January 13, 2023 on LessWrong.
    This post will lay out a couple of stylized stories about how, if transformative AI is developed relatively soon, this could result in global catastrophe. (By “transformative AI,” I mean AI powerful and capable enough to bring about the sort of world-changing consequences I write about in my most important century series.)
    This piece is more about visualizing possibilities than about providing arguments. For the latter, I recommend the rest of this series.
    In the stories I’ll be telling, the world doesn't do much advance preparation or careful consideration of risks I’ve discussed previously, especially re: misaligned AI (AI forming dangerous goals of its own).
    People do try to “test” AI systems for safety, and they do need to achieve some level of “safety” to commercialize. When early problems arise, they react to these problems.
    But this isn’t enough, because of some unique challenges of measuring whether an AI system is “safe,” and because of the strong incentives to race forward with scaling up and deploying AI systems as fast as possible.
    So we end up with a world run by misaligned AI - or, even if we’re lucky enough to avoid that outcome, other catastrophes are possible.
    After laying these catastrophic possibilities, I’ll briefly note a few key ways we could do better, mostly as a reminder (these topics were covered in previous posts). Future pieces will get more specific about what we can be doing today to prepare.
    Backdrop
    This piece takes a lot of previous writing I’ve done as backdrop. Two key important assumptions (click to expand) are below; for more, see the rest of this series.
    How we could stumble into catastrophe from misaligned AI
    This is my basic default picture for how I imagine things going, if people pay little attention to the sorts of issues discussed previously. I’ve deliberately written it to be concrete and visualizable, which means that it’s very unlikely that the details will match the future - but hopefully it gives a picture of some of the key dynamics I worry about.
    Throughout this hypothetical scenario (up until “END OF HYPOTHETICAL SCENARIO”), I use the present tense (“AIs do X”) for simplicity, even though I’m talking about a hypothetical possible future.
    Early commercial applications. A few years before transformative AI is developed, AI systems are being increasingly used for a number of lucrative, useful, but not dramatically world-changing things.
    I think it’s very hard to predict what these will be (harder in some ways than predicting longer-run consequences, in my view),2 so I’ll mostly work with the simple example of automating customer service.
    In this early stage, AI systems often have pretty narrow capabilities, such that the idea of them forming ambitious aims and trying to defeat humanity seems (and actually is) silly. For example, customer service AIs are mostly language models that are trained to mimic patterns in past successful customer service transcripts, and are further improved by customers giving satisfaction ratings in real interactions. The dynamics I described in an earlier piece, in which AIs are given increasingly ambitious goals and challenged to find increasingly creative ways to achieve them, don’t necessarily apply.
    Early safety/alignment problems. Even with these relatively limited AIs, there are problems and challenges that could be called “safety issues” or “alignment issues.” To continue with the example of customer service AIs, these AIs might:
    Give false information about the products they’re providing support for. (Example of reminiscent behavior)
    Give customers advice (when asked) on how to do unsafe or illegal things. (Example)
    Refuse to answer valid questions. (This could result from compani...
    27 min
  • EA - Forecasting could use more gender diversity by Tegan
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Forecasting could use more gender diversity, published by Tegan on January 13, 2023 on The Effective Altruism Forum.
    The new org I’m part of, the Forecasting Research Institute, recently began a hiring round, and we noticed that around 80% of research analyst applicants were male. This isn’t terribly surprising—a large percentage of forecasting tournament participants have been male historically, so we expected familiarity and interest in forecasting to be male-skewed currently. But, as a woman in forecasting myself, it does bug me.
    It bugs me on several levels. One is puzzlement: many of the fields most relevant to forecasting and forecasting research—like international relations, history, social science, and, in the case of forecasting research, behavioral science—are not short of women. (And in any case, forecasting is so interdisciplinary that most backgrounds have some value to add.) There’s also nothing about the broad spectrum of potential uses of forecasting, in policy, public discourse, institutional decision-making, philanthropy, etc., that suggests its impact is limited to particularly male domains.
    Which brings me to the next point. If the field of forecasting is systematically failing to attract roughly half of the population, containing roughly half of the talent, good ideas, valuable skills and knowledge, it’s made poorer and will likely face an uphill climb toward relevance and real-world impact. Which would be a shame, given its great potential.
    So if you’re a woman reading this (or another sort of person who feels underrepresented in forecasting) and you think there’s even a chance you might be interested in doing forecasting research, here’s my case for why you should apply to FRI’s research analyst position:
    You don’t need to be an experienced forecaster, or like doing forecasting. You don’t need to be super quantitative. You don’t need to be intimately familiar with forecasting research—for the right candidate, as long as you’re curious, motivated, and can get excited about unsolved problems in forecasting, FRI is willing to train you up.
    Forecasting is an early-stage field with high impact potential, and the opportunity to be part of building such a field is kind of rare.
    This also means there are tons of interesting open questions we need to answer, and lots of room to be creative in finding the answers.
    In terms of subject matter, the work is extremely varied—so if you’re, say, interested in bio and AI and public policy, you might get to touch on all of them at the same time, or in quick succession. Some of our projects will also allow you to interact with leading domain experts or prominent organizations doing cool work when we partner to test forecasting tools.
    If you want to use it this way, forecasting research can be a great facilitator for working on your own epistemics, and can provide a framework for interpreting and filtering the many important things one can have views about in a relatively independent way. (This is one of the primary motivations of my colleague Josh.)
    Not only is everyone on our team enthusiastic about improving gender diversity in our org, and in the forecasting space generally, they’re also just extremely nice, interesting, competent and thoughtful as people.
    If you’re still not sure if you could get sufficiently interested in forecasting for applying to be worth it, I’d suggest checking out our research page to see the projects we’ll be working on in the near future. But to give you a sense of how diverse the traits of people who’ve gotten into forecasting research are (all on the FRI team):
    I don’t have a “formal” background of any sort: I dropped out of college, and subsequently hopped around doing stuff like writing and generalist research. I then got into “caring about AI” via compelling arguments from friend...
    5 min