The Nonlinear Library

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The Cognitive-Theoretic Model of the Universe: A Partial Summary and Review, published by jessicata on March 27, 2024 on LessWrong.

About 15 years ago, I read Malcolm Gladwell's Outliers. He profiled Chris Langan, an extremely high-IQ person, claiming that he had only mediocre accomplishments despite his high IQ. Chris Langan's theory of everything, the Cognitive Theoretic Model of the Universe, was mentioned. I considered that it might be worth checking out someday.

Well, someday has happened, and I looked into CTMU, prompted by Alex Zhu (who also paid me for reviewing the work). The main CTMU paper is "The Cognitive-Theoretic Model of the Universe: A New Kind of Reality Theory".

CTMU has a high-IQ mystique about it: if you don't get it, maybe it's because your IQ is too low. The paper itself is dense with insights, especially the first part. It uses quite a lot of nonstandard terminology (partially because the author is outside the normal academic system), having few citations relative to most academic works. The work is incredibly ambitious, attempting to rebase philosophical metaphysics on a new unified foundation.

As a short work, it can't fully deliver on this ambition; it can provide a "seed" of a philosophical research program aimed at understanding the world, but few implications are drawn out.

In reading the work, there is a repeated sense of "what?", staring and looking at terms, and then "ohhh" as something clicks. These insights may actually be the main value of the work; at the end I still don't quite see how everything fits together in a coherent system, but there were a lot of clicks along the way nonetheless.

Many of the ideas are similar to other intellectual ideas such as "anthropics" and "acausal interaction", but with less apparent mathematical precision, such that it's harder to see exactly what is being said, and easier to round off to something imprecise and implausible.

There is repeated discussion of "intelligent design", and Langan claims that CTMU proves the existence of God (albeit with a very different conceptualization than traditional religions). From the perspective of someone who witnessed the evolution / intelligent design debate of the 90s-00s, siding with the "intelligent design" branch seems erroneous, although the version presented here differs quite a lot from more standard intelligent design argumentation.

On the other hand, the "evolutionists" have gone on to develop complex and underspecified theories of anthropics, multiverses, and simulations, which bring some amount of fundamental or nearly-fundamental mind and agency back into the picture.

I didn't finish summarizing and reviewing the full work, but what I have written might be useful to some people. Note that this is a very long post.

Abstract

Perception is a kind of model of reality. Information about reality includes information about the information processor ("one's self"), which is called reflexivity. The theory identifies mental and physical reality, in common with idealism. CTMU is described as a "supertautological reality-theoretic extension of logic"; logic deals in tautologies, and CTMU somehow deals in meta-tautologies. It is based in part on computational language theory (e.g. the work of Chomsky, and type theory).

Central to CTMU is the Self-Configuring Self-Processing Language (SCSPL), a language that can reflect on itself and configure its own execution, perhaps analogous to a self-modifying program. SCSPL encodes a form of "dual-aspect monism" consisting of "infocognition", integrated information and cognition.

CTMU states that the universe comes from "unbounded telesis" (UBT), a "primordial realm of infocognitive potential free of informational constraint"; this may be similar to a language in which the physical universe could be "specified", or perhaps...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Summary: Against the Singularity Hypothesis (David Thorstad), published by Nicholas Kruus on March 27, 2024 on The Effective Altruism Forum.

This post summarizes "Against the Singularity Hypothesis," a Global Priorities Institute Working Paper by David Thorstad. This post is part of my sequence of GPI Working Paper summaries. For more, Thorstad's blog, Reflective Altruism, has a three-part series on this paper.

Introduction

The effective altruism community has allocated substantial resources to catastrophic risks from AI, partly motivated by the singularity hypothesis about AI's rapid advancement. While many[1] AI experts and philosophers have defended the singularity hypothesis, Thorstad argues the case for it is surprisingly thin.

Thorstad describes the singularity hypothesis in (roughly) the following three parts:[2]

Self-Improvement: Artificial agents will become able to increase their own quantity of general intelligence.

Intelligence Explosion: For a sustained period, their general intelligence will grow at an accelerating rate, creating exponential or hyperbolic growth that causes them to quickly surpass human intelligence by orders of magnitude.

Singularity: This will produce a discontinuity in human history, after which humanity's fate - living in a digital form, extinct, or powerless - depends largely on our interactions with artificial agents.

Growth

Thorstad offers five reasons to doubt the intelligence growth rate proposed by the singularity hypothesis.

Extraordinary claims require extraordinary evidence: Proposing that exponential or hyperbolic growth will occur for a prolonged period,[3] is an extraordinary claim that requires many excellent reasons to suspect it's correct. Until this high burden of evidence is met, it's appropriate to place very low credence on the singularity hypothesis.

Good ideas become harder to find: Idea-generating becomes increasingly difficult as low-hanging fruit is picked. For example, spending on drug and agricultural research has seen rapidly diminishing returns.[4] AI will likely be no exception, as hardware improvement (e.g. Moore's law) is slowing. Even if the rate of diminishing research productivity is small, its effects become substantial as it compounds over many cycles of self-improvement.[5]

Bottlenecks: No algorithm can run quicker than its slowest component, so, unless every component can be sped up at once, bottlenecks may arise. Even a single bottleneck would halt an intelligence explosion, and we should expect them to emerge because…

There is limited room for improvement in certain processes (e.g., search algorithms)

There are physical resource constraints (we shouldn't expect supply chains' output to increase a thousandfold or more very quickly)

Physical constraints: Regardless of path, improving AI will eventually face intractable limitations from resource constraints and laws of physics, likely slowing intelligence growth. Consider Moore's law's demise:

Circuits' energy requirements have massively increased - increasing costs and overheating.[6]

Capital is drying up, as semiconductor plant prices have skyrocketed.[7]

Our best transistors' diameter is now that of ten atoms, making manufacturing increasingly difficult and soon subject to quantum uncertainties.[8]

Sublinearity: Technological capabilities[9] have been rapidly improving, meaning, if intelligence grows proportionally to them, then continuing current trends would create exponential intelligence growth. But intelligence grows sublinearly to these capabilities, not proportionally.

Consider almost any performance metric plausibly correlated with intelligence - e.g., Chess, Go, protein folding, weather and oil reserve prediction - historically, exponential increases in the quantity of computation power yield merely linear gains.[10] If these performa...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AIM Animal Initiatives, published by Joey on March 27, 2024 on The Effective Altruism Forum.

TLDR: If you're an EA-minded animal funder donating $200K/year or more, we'd love to connect with you about several exciting initiatives that AIM is launching over the next several months.

AIM (formerly Charity Entrepreneurship) has a history of incubating and supporting highly effective charities across various cause areas. We have also launched a variety of additional programs aiming at other impactful sectors, from philanthropy to research to local effective giving.

We have noticed through engaging on these different levels of impact that animal welfare seems particularly impactful and particularly neglected, even amongst a crop of already impactful and neglected cause areas.We believe that there are several opportunities to meaningfully impact animal welfare through donor collaboration and programming. To that end, we're launching a few exciting initiatives over the coming months.Specifically, we are excited about two projects that are launching soon:

An animal-focused Foundation Program round, where we'll be supporting a cohort of ambitious founders as they develop their philanthropic strategy. This cohort begins April 15.

An animal-focused funding circle, bringing funders together to strategically deploy capital to the most promising animal charities. This will likely launch mid-summer.

We believe these initiatives will offer ambitious funders unique opportunities for increased impact. If you're an EA-minded animal funder who donates $200K or more per year, please don't hesitate to reach out.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Open Philanthropy: Our Progress in 2023 and Plans for 2024, published by Alexander Berger on March 27, 2024 on The Effective Altruism Forum.

Like many organizations, Open Philanthropy has had multiple founding moments. Depending on

how

you

count, we will be either seven, ten, or thirteen years old this year. Regardless of when you start the clock, it's possible that we've changed more in the last two years than over our full prior history. We've more than doubled the size of our team (to ~110), nearly doubled our annual giving (to >$750M), and

added

five

new

program

areas.

As our track record and volume of giving have grown, we are seeing more of our impact in the world. Across our focus areas, our funding played a (sometimes modest) role in some of 2023's most important developments:

We were among the supporters of the

clinical trials that led to the World Health Organization (WHO)

officially recommending the R21 malaria vaccine. This is the second malaria vaccine recommended by WHO, which expects it to enable "sufficient vaccine supply to benefit all children living in areas where malaria is a public health risk." Although the late-stage clinical trial funding was Open Philanthropy's first involvement with R21 research, that isn't the case for our new global health R&D program officer,

Katharine Collins, who invented R21 as a grad student.

Our early commitment to AI safety has contributed to increased awareness of the associated risks and to early steps to reduce them. The Center for AI Safety,

one of our AI grantees, made

headlines

across the

globe with its

statement calling for AI extinction risk to be a "global priority alongside other societal-scale risks," signed by many of the world's leading AI researchers and experts. Other grantees contributed to many of the year's other big AI policy events, including the UK's

AI Safety Summit, the US

executive order on AI, and the first

International Dialogue on AI Safety, which brought together scientists from the US and China to lay the foundations for future cooperation on AI risk (à la the

Pugwash Conferences in support of nuclear disarmament).

The US Supreme Court upheld California's Proposition 12, the nation's strongest farm animal welfare law. We were

major supporters of the original initiative and

helped

fund its successful

legal defense.

Our grantees in the YIMBY ("yes in my backyard") movement - which works to increase the supply of housing in order to lower prices and rents - helped drive major

middle housing reforms in Washington state and California's

legislation streamlining the production of affordable and mixed-income housing. We've been the largest national funder of the YIMBY movement since 2015.

We've also encountered some notable challenges over the last couple of years. Our available assets fell by half and then recovered half their losses. The FTX Future Fund, a large funder in several of our focus areas, including pandemic prevention and AI risks, collapsed suddenly and left a sizable funding gap in those areas. And Holden Karnofsky - my friend, co-founder, and our former CEO

stepped down to work full-time on AI safety.

Throughout these changes, we've remained devoted to our mission of helping others as much as we can with the resources available to us. But it's a good time to step back and reflect.

The rest of this post covers:

Brief updates on grantmaking from each of our 12 programs.

Our leadership changes over the past year.

Our chaotic macro environment over the last couple of years.

How that led us to revise our priorities, and specifically to expand our work to reduce global catastrophic risks.

Other lessons we learned over the past year.

Our plans for the rest of 2024.

Because it feels like we have more to share this year, this post is longer and aims to share more than I have the last

two...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Daniel Kahneman has died, published by DanielFilan on March 27, 2024 on LessWrong.

He was 90 years old.

His death was confirmed by his stepdaughter Deborah Treisman, the fiction editor for the New Yorker. She did not say where or how he died.

The obituary also describes an episode from his life that I had not previously heard (but others may have):

Daniel Kahneman was born in Tel Aviv on March 5, 1934, while his mother was visiting relatives in what was then the British mandate of Palestine. The Kahnemans made their home in France, and young Daniel was raised in Paris, where his mother was a homemaker and his father was the chief of research for a cosmetics firm.

During World War II, he was forced to wear a Star of David after Nazi German forces occupied the city in 1940. One night in 1941 or '42, he later recalled, he stayed out past the German-imposed curfew for Jews while visiting a friend, and he turned his sweater inside out to hide the star while he walked a few blocks home. He then crossed paths with a soldier in the SS, who called Daniel over, picked him up - and hugged him.

"I was terrified that he would notice the star inside my sweater," Dr. Kahneman noted in a biographical essay for the Nobel Prize ceremonies. But the German pulled out his wallet, showed him a photo of a boy, gave him some money and sent him on his way. "I went home more certain than ever that my mother was right," Dr. Kahneman said in the essay. "People were endlessly complicated and interesting."

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Nick Bostrom's new book, "Deep Utopia", is out today, published by peterhartree on March 27, 2024 on The Effective Altruism Forum.

Description:

A greyhound catching the mechanical lure - what would he actually do with it? Has he given this any thought?

Bostrom's previous book, Superintelligence: Paths, Dangers, Strategies changed the global conversation on AI and became a New York Times bestseller. It focused on what might happen if AI development goes wrong. But what if things go right?

Suppose that we develop superintelligence safely, govern it well, and make good use of the cornucopian wealth and near magical technological powers that this technology can unlock. If this transition to the machine intelligence era goes well, human labor becomes obsolete. We would thus enter a condition of "post-instrumentality", in which our efforts are not needed for any practical purpose. Furthermore, at technological maturity, human nature becomes entirely malleable.

Here we confront a challenge that is not technological but philosophical and spiritual. In such a solved world, what is the point of human existence? What gives meaning to life? What do we do all day?

Deep Utopia shines new light on these old questions, and gives us glimpses of a different kind of existence, which might be ours in the future.

Links to purchase:

Amazon

Barnes & Noble

There's a table of contents on the book's web page.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AE Studio @ SXSW: We need more AI consciousness research (and further resources), published by AE Studio on March 27, 2024 on LessWrong.

Quick update from

AE Studio: last week, Judd (AE's CEO) hosted a

panel at SXSW with Anil Seth,

Allison Duettmann, and Michael Graziano, entitled "The Path to Conscious AI" (discussion summary here[1]).

We're also making available an unedited

Otter transcript/recording for those who might want to read along or increase the speed of the playback.

Why AI consciousness research seems critical to us

With the release of each new

frontier

model seems to follow a cascade of questions probing whether or not the model is conscious in training and/or deployment. We suspect that these questions will only grow in number and volume as these models exhibit increasingly sophisticated cognition.

If consciousness is indeed sufficient for

moral patienthood, then the stakes seem remarkably high from a utilitarian perspective that we do not commit the Type II error of behaving as if these and future systems are not conscious in a world where they are in fact conscious.

Because the ground truth here (i.e., how consciousness works mechanistically) is still poorly understood, it is extremely challenging to reliably estimate the probability that we are in any of the four quadrants above - which seems to us like a very alarming status quo. Different people have different default intuitions about this question, but the stakes here

seem too high for default intuitions to be governing our collective behavior.

In an ideal world, we'd have understood far more about consciousness and human cognition before getting near AGI. For this reason, we suspect that there is likely substantial work that ought to be done at a smaller scale first to better understand consciousness and its implications for alignment.

Doing this work now seems far preferable to a counterfactual world where we build frontier models that end up being conscious while we still lack a reasonable model for the correlates or implications of building sentient AI systems.

Accordingly, we are genuinely excited about rollouts of

consciousness evals at large labs, though the earlier caveat still applies: our currently-limited understanding of how consciousness actually works may engender a (potentially dangerous) false sense of confidence in these metrics.

Additionally, we believe testing and developing an empirical model of consciousness will enable us to better understand humans, our values, and any future conscious models. We also suspect that consciousness may be an

essential cognitive component of human prosociality and may have additional broader implications for solutions to alignment. To this end, we are currently collaborating with panelist Michael Graziano in pursuing a more mechanistic model of consciousness by operationalizing

attention schema theory.

Ultimately, we believe that immediately devoting time, resources, and attention towards better understanding the computational underpinnings of consciousness may be one of the most important

neglected approaches that can be pursued in the short term. Better models of consciousness could likely (1) cause us to dramatically reconsider how we interact with and deploy our current AI systems, and (2) yield insights related to prosociality/human values that lead to promising novel alignment directions.

Resources related to AI consciousness

Of course, this is but a small part of a larger, accelerating conversation that has been ongoing on

LW and the

EAF for some time. We thought it might be useful to aggregate some of the articles we've been reading here, including panelists Michael Graziano's book, "

Rethinking Consciousness" (and article,

Without Consciousness, AIs Will Be Sociopaths) as well as Anil Seth's book, "

Being You".

There's also

Propositions...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: I'm a Harvard freshman, and I'm trying to gently nudge my peers toward EA., published by chanden on March 27, 2024 on The Effective Altruism Forum.

Recently I got published an op-ed in The Crimson advocating, sort of, for an Earning to Give strategy.

The Crimson is widely read among Harvard students, and its content runs through many circles - not just those who care about student journalism.

I thought the piece was important to write.

I've noticed a recurring trend in conversations about careers here at Harvard: people want to do good, but have no idea how. So either - they give up and "sell out" to a comfy lifestyle, or they follow their passions/work at an NGO/etc. without even considering Earning to Give as a legitimate option.

I'm aware that orgs like 80,000 Hours have moved away from their (original) primary focus on Earning to Give as a career strategy.

But I think, based on folks I've talked to at Harvard, it's still one of the most compelling ways to at least get people on board - it doesn't require sacrifice of a well-paid lifestyle, but more importantly, it doesn't require sacrifice of a prestigious career (which is what so many here care about).

80,000 hours also has a set of bulletpoints intended to determine whether you'd be a good fit: https://80000hours.org/articles/earning-to-give/

They ask four questions:

Do you have high earning potential? (Yes. As I note in the article, Harvard students are lucky enough to be recruited by some of the highest-paying firms in the world.)

Do you want to gain skills and career capital in a higher-earning option? (Yes as well. Harvard kids want to preserve optionality.)

Are you uncertain about which problems are most pressing? (Resounding yes. I commonly hear things like "I want to do good for the world, I just don't know how.")

Do you want to contribute to an area that is funding-constrained? (This is fuzzier, I think, seeing as the answer to this question would probably have to come after the last one.)

Anyway, I would appreciate if you gave my article a read. Feedback appreciated!

https://www.thecrimson.com/article/2024/3/26/climaco-harvard-sell-out/

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Failures in Kindness, published by silentbob on March 27, 2024 on LessWrong.

There's a particular kind of widespread human behavior that is kind on the surface, but upon closer inspection reveals quite the opposite. This post is about four such patterns.

Computational Kindness

One of the most useful ideas I got out of Algorithms to Live By is that of computational kindness. I was quite surprised to only find a single mention of the term on lesswrong. So now there's two.

Computational kindness is the antidote to a common situation: imagine a friend from a different country is visiting and will stay with you for a while. You're exchanging some text messages beforehand in order to figure out how to spend your time together. You want to show your friend the city, and you want to be very accommodating and make sure all their preferences will be met.

So you simply ask them: "What do you want to do"? And maybe you add "I'm completely fine with anything!" to ensure you're really introducing no constraints whatsoever and you two can do exactly what your friend desires.

People often act like this, and they tend to assume they're doing the other person a favor by being so open and flexible. After all, this way the other person will have to make no trade-offs and can spend their time exactly as they please. The problem with this however is that it's computationally unkind: it offloads all the effort of coming up with ideas and making decisions to the other person.

So while it is kind on one level (respecting their object level preferences), it's unkind on another (effort, and respecting their possible meta level preferences about the planning process). And particularly if the friend's preferences about what exactly to do are not that strong, it now gives them a difficult and uncertain task for very little payoff.

So what's the computationally kind way of approaching this situation? You could name a (not too long) list of concrete proposals of how you could spend your time. If you know the person really well, you could suggest a full-fledged plan. If you don't know them that well, you could ask a few clarifying questions about their general preferences and then come up with a plan.

And on top of this (rather than instead of it) you can make sure to point out that you're open to anything and are happy to change plans in any way. This way, the other person can decide themselves how much cognitive effort to invest. They can just say "yes" to your proposal, or can suggest some adjustments, or even come up with an entirely new plan if they really want to go that far.

Responsibility Offloading[1]

A somewhat similar pattern to computational kindness is that of offloading responsibility. Imagine Alice and Bob, two friends who are just getting to know each other better, are hanging out at Alice's place. It's getting late, but they're having a fun time. Bob is unsure about whether and when Alice wants him to leave, but he's fine with staying much longer.

So he playfully says "By the way - feel free to throw me out any time! I've got tomorrow off, so am flexible, but just let me know when you've had enough of me".

Sometimes this is indeed a good move. Particularly when Bob knows that Alice is an assertive person who doesn't shy away from stating her preferences. But there are cases where this puts a big burden on Alice. Imagine Alice is generally rather insecure and indecisive. She now has to feel solely responsible for terminating the hangout. This is now something on her plate that she has to think about and decide, and communicate to Bob eventually in a non-offensive way.

There are Alices out there who would be rather stressed out by this, and who would prefer Bob to carry that responsibility, or to have the two of them figure it out together. And there are Bobs out there who have no ide...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Modern Transformers are AGI, and Human-Level, published by abramdemski on March 26, 2024 on LessWrong.

This is my personal opinion, and in particular, does not represent anything like a MIRI consensus; I've gotten push-back from almost everyone I've spoken with about this, although in most cases I believe I eventually convinced them of the narrow terminological point I'm making.

In the AI x-risk community, I think there is a tendency to ask people to estimate "time to AGI" when what is meant is really something more like "time to doom" (or, better, point-of-no-return). For about a year, I've been answering this question "zero" when asked.

This strikes some people as absurd or at best misleading. I disagree.

The term "Artificial General Intelligence" (AGI) was coined in the early 00s, to contrast with the prevalent paradigm of Narrow AI.

I was getting my undergraduate computer science education in the 00s; I experienced a deeply-held conviction in my professors that the correct response to any talk of "intelligence" was "intelligence for what task?" -- to pursue intelligence in any kind of generality was unscientific, whereas trying to play chess really well or automatically detect cancer in medical scans was OK.

I think this was a reaction to the AI winter of the 1990s. The grand ambitions of the AI field, to create intelligent machines, had been discredited. Automating narrow tasks still seemed promising. "AGI" was a fringe movement.

As such, I do not think it is legitimate for the AI risk community to use the term AGI to mean 'the scary thing' -- the term AGI belongs to the AGI community, who use it specifically to contrast with narrow AI.

Modern Transformers[1] are definitely not narrow AI.

It may have still been plausible in, say, 2019. You might then have argued: "Language models are only language models! They're OK at writing, but you can't use them for anything else." It had been argued for many years that language was an AI complete task; if you can solve natural-language processing (NLP) sufficiently well, you can solve anything. However, in 2019 it might still be possible to dismiss this.

Basically any narrow-AI subfield had people who will argue that that specific subfield is the best route to AGI, or the best benchmark for AGI.

The NLP people turned out to be correct. Modern NLP systems can do most things you would want an AI to do, at some basic level of competence. Critically, if you come up with a new task[2], one which the model has never been trained on, then odds are still good that it will display at least middling competence. What more could you reasonably ask for, to demonstrate 'general intelligence' rather than 'narrow'?

Generative pre-training is AGI technology: it creates a model with mediocre competence at basically everything.

Furthermore, when we measure that competence, it usually falls somewhere within the human range of performance. So, as a result, it seems sensible to call them human-level as well. It seems to me like people who protest this conclusion are engaging in goalpost-moving.

More specifically, it seems to me like complaints that modern AI systems are "dumb as rocks" are comparing AI-generated responses to human experts. A quote from the dumb-as-rocks essay:

GenAI also can't tell you how to make money. One man asked GPT-4 what to do with $100 to maximize his earnings in the shortest time possible. The program had him buy a domain name, build a niche affiliate website, feature some sustainable products, and optimize for social media and search engines. Two months later, our entrepreneur had a moribund website with one comment and no sales. So genAI is bad at business.

That's a bit of a weak-man argument (I specifically searched for "generative ai is dumb as rocks what are we doing"). But it does demonstrate a pattern I've enco...