Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Effective Altruism and Eastern Ethics, published by EA Lifestyles on July 7, 2023 on The Effective Altruism Forum.

This post was inspired by a phone call I made a few days ago to my mother where I informed her I was turning vegetarian - I knew it would be rough but it turned out significantly worse than I expected still. The phone call quickly turned sour as she believed I was making a grave mistake and that I was being unjust. This was not unique - I talked to two other individuals raised in East Asian households which elicited similar if not stronger responses. One friend told me that coming out vegan elicited a stronger response than coming out as gay.

Effective Altruism's moral roots are weird. Really weird to those raised on a western ethical system and absurdly weird to those raised on an eastern ethical system.

This is because following the virtues of effective altruism can often lead to conflict within Eastern ethical systems. Being virtuous requires us to stick to a moral code, even if that means clashing with personal relationships. Otherwise, virtue would demand that we keep doing things that aren't morally right just to keep the peace in our circles. This dilemma can be difficult for everyone, but it's particularly challenging for those of us brought up in Asian households.

Take my mother's moral compass, for instance. It closely reflects the Chinese ethos, which in turn is heavily influenced by Confucian principles. In the Confucian view of virtue, the focus is on fulfilling one's duties within a socially hierarchical and communal framework. This is articulated through key concepts such as "ren" or benevolence, often expressed through the cultural concept of "mianzi" or face, which is deeply embedded in social relationships. A core expectation within the family is to fulfill one's duties, especially towards one's parents.

In the eyes of Confucian virtue, my vegetarianism disrupts this order. The sharing of food, often including meat, is a significant social ritual in Chinese culture, promoting family cohesion. My deviation from this norm is seen as a violation of the principle of "Li", disrupting harmony and causing displeasure at the family table. My mother’s emotionally charged conversations with me reflects this deep-rooted cultural clash.

In many Asian cultures, there are two important ideas. The first is "guanxi," or the idea that everyone is connected and relies on each other. The second are the unbreakable responsibilities to your parents, called "filial piety." These ideas are a big part of the way people think and act in these cultures, and they create a complicated set of rules and expectations for how people should behave.

This can make it hard for someone to do something different, because it might upset this delicate balance. Often, what's seen as most important is what's best for the family or community, even if it means sacrificing your own personal beliefs.

So, for someone from these cultures to take on the role of an Effective Altruist - someone who tries to do the most good possible - it could be seen as going against their culture and disappointing those around them. It's like they're rejecting their roots and the values their community holds in high regard.

While pursuing an Effective Altruist moral framework may be hard for all, it is certainty harder for those of from these Asian backgrounds. The communal ties and expectations so core to eastern philosophy does not appear in the west.

Vegetarianism itself was made all the more challenging by the meat-heavy diets reguarly seen on dinner tables in Asian circles today. The rapid development of Asia has led to an increase in meat consumption, as the rise in living standards often associates meat with luxury and nutritional completeness. For many families who could only enjoy meat sparin...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Apparently, of the 195 Million the DoD allocated in University Research Funding Awards in 2022, more than half of them concerned AI or compute hardware research, published by mako yass on July 7, 2023 on LessWrong.

Suggests some really interesting focus, so I thought I should mention it.

The awards concerned "pursuing basic research spanning multiple scientific disciplines", which makes it pretty interesting that three fifths of them are either directly AI and compute hardware research, or have applications to compute hardware research.

From a merit-based review of approximately 340 proposals received, a panel of experts narrowed the proposals to a subset, from which the 28 final awards were selected

(Only 26 were listed in the table. I wonder if the other two were a strategic omission.)

Take a look at the table of project topics. Bold means probably directly AI/compute, italic means probably primarily applicable to AI/compute.

Social Network-Transcendent Behavioral Dynamics Learning Dynamics and Detecting Causal Pathways in Coupled Online- Offline Systems

Microelectronic Test Science Exploiting Latent Energy and Electromagnetic Radiation Comprehensive Minimally/non- invasive Multifaceted Assessment of Nano-/ microelectronic Devices (CoMMAND)

Cavity Molecular Polaritons MURI: Polariton Chemistry

Effects of Radiation Damage on Performance of Wide-Bandgap Electronics REDESIGN: Radiation Effects preDiction through Experimentally validated Simulation of Gallium Nitride

Understanding Neural Systems Integration for Competent Autonomy in Decision and Control FLAP: Fast, Lexicographic Agile Integrates Decision and Control in a Spike-Resolved, Sensorimotor Program

Nonlinear Optical Material Design with Extreme Interband Nonlinearities Atomic-scale Manipulations of Interband Optical Nonlinearities - (AtoMs➝IONs)

Synthetic Quantum Matter Reimagining Atoms and Photons in SYnthetic, DYnamical, and INteracting Quantum matter

Composability of Synthetic Biological Circuits Rules of Composition in Synthetic Biology Across Scales of Complexity: Theory and Tools

Bio-architected Responsive Materials with 3D Nanoscale Order 3D Nanomaterial Architectures with Programmable Reconfiguration States and Functions

Topological Seeds of Complex Response in Materials Programming Multistable Origami and Kirigami Structures via Topological Design

Connectivity and Transport in Disordered Hyperuniform Networks Transport in Disordered Hyperuniform Systems and Networks

Uncovering the Underlying Neurobiological Mechanisms of Cognitive Fatigue Understanding and Predicting Cognitive Fatigue across Multiple Timescales, Distinct Aspects of Cognition, and Different Individuals with Multiscale Whole Cortex Models

Gut-Neuronal Signaling Through Polymeric Mucin via Chemical Probes and Imaging Mucin-mimetic Interventions to Modulate the Gut-Brain Axis

ELECTROBIOLOGY: Electronic Control of Biological Communication Faster, More Efficient, and Hybrid Computation in Microbial Bioelectronic Systems ARO Rice University

Novel Routes to Majorana Qubits for Topologically- Protected Quantum Information Epitaxial phase-biased Josephson junctions

Molecular Doping of Organic Electronic Materials Next Generation Molecular Dopants for Organic Electronics: From Fundamentals to New Device Concepts

Learning from Hearing Neurobehavioral, Physiological, and Computational Processes of Auditory Object Learning in Mammals

Hydrodynamics of Fish Schooling Revealing the Hydrodynamic Principles of Three-dimensional Fish Schools; From Biology to Schooling Robotics

Self-learning for Real-world Perception Intentional multi-modal self-learning to perceive and understand the real world

Fundamental Non-equilibrium Processes in Weakly Ionized Hypersonic Flows Development of Validated Hypersonic Plasma Kinetics Models Including Atomic Excitation

I...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Introducing bayescalc.io, published by Adele Lopez on July 7, 2023 on LessWrong.

I made a simple online calculator for doing elementary hypothesis testing!

I was disappointed that an intuitive and easy-to-use app for using bayes' theorem apparently did not exist, so I decided to make it. My goal was to make something that:

Helped people correctly and quickly evaluate the effect of evidence while comparing hypotheses.

Was easy enough for someone who didn't know math to use.

And which also helped show what the math was doing in an intuitive way, so that you didn't have to trust math you didn't understand.

Felt good enough to use that it would actually be used.

Could be used to share simple models of things in a way that would help people have more productive discussions, and promote shared model building.

Hopefully I've at least made substantial progress on these goals, and I'd really appreciate feedback on ways in which it falls short! This includes even minor interface or design issues. You can leave feedback as a comment here, or on the issues page.I'd also be really happy to see people share examples they've made in the comments!

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Great power conflict - problem profile (summary and highlights), published by Stephen Clare on July 7, 2023 on The Effective Altruism Forum.

I've spent quite a bit of time over the last few years trying to answer two questions:

Will World War III break out this century?

If it does, could it be so devastating that it causes an existential catastrophe, threatening humanity’s long-term future?

It's been fun.

Last week I published an in-depth problem profile for 80,000 Hours that sums up what I’ve found so far: that World War III is perhaps surprisingly likely and we can’t rule out the possibility of a catastrophic escalation. It also discusses some ideas for how you might be able to help solve this problem.

This post includes the top-line summary of the profile. Following that, I draw out the highlights that seem particularly relevant for the EA community.

Profile Summary

Economic growth and technological progress have bolstered the arsenals of the world’s most powerful countries. That means the next war between them could be far worse than World War II, the deadliest conflict humanity has yet experienced.

Could such a war actually occur? We can’t rule out the possibility. Technical accidents or diplomatic misunderstandings could spark a conflict that quickly escalates. Or international tension could cause leaders to decide they’re better off fighting than negotiating.

It seems hard to make progress on this problem. It’s also less neglected than some of the problems that we think are most pressing. There are certain issues, like making nuclear weapons or military artificial intelligence systems safer, which seem promising - although it may be more impactful to work on reducing risks from AI, bioweapons or nuclear weapons directly. You might also be able to reduce the chances of misunderstandings and miscalculations by developing expertise in one of the most important bilateral relationships (such as that between the United States and China).

Finally, by making conflict less likely, reducing competitive pressures on the development of dangerous technology, and improving international cooperation, you might be helping to reduce other risks, like the chance of future pandemics.

Overall view

Recommended

Working on this issue seems to be among the best ways of improving the long-term future we know of, but all else equal, we think it’s less pressing than our highest priority areas (primarily because it seems less neglected and harder to solve).

Importance:

There's a significant chance that a new great power war occurs this century.

Although the world's most powerful countries haven't fought directly since World War II, war has been a constant throughout human history. There have been numerous close calls, and several issues could cause diplomatic disputes in the years to come.

These considerations, along with forecasts and statistical models, lead me to think there's about a one-in-three chance that a new great power war breaks out in roughly the next 30 years.

Few wars cause more than a million casualties and the next great power war would probably be smaller than that. However, there's some chance it could escalate massively. Today the great powers have much larger economies, more powerful weapons, and bigger military budgets than they did in the past. An all-out war could kill far more people than even World War II, the worst war we've yet experienced.

Could it become an existentially threatening war - one that could cause human extinction or significantly damage the prospects of the long-term future? It's very difficult to say. But my best current guess is that the chance of an existential catastrophe due to war in the next century is somewhere between 0.05% and 2%.

Neglectedness:

War is a lot less neglected than some of our other top problems. There are thousands of peo...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: What are the best non-LW places to read on alignment progress?, published by Raemon on July 7, 2023 on LessWrong.

I've lately been thinking I should prioritize a bit more keeping up with alignment-relevant progress outside of LessWrong/Alignment Forum.

I'm curious if people have recommendations that stand out as reliably valuable, and/or have tips for finding "the good stuff" on places where the signal/noise ratio isn't very good. (Seems fine to also apply this to LW/AF)

Some places I've looked into somewhat (though not made major habits around so far) include:

Blogs of OpenAI/Deepmind/Anthropic

reddit.com/r/ControlProblem

Stampy Discord

EleutherAI Discord

I generally struggle with figuring out how much to keep up with stuff - it seems like there's more than one full-time-job's worth of stuff to keep up with, and it's potentially overanchoring to think about "the stuff people have worked on" as opposed to "stuff that hasn't been worked on yet."

I'm personally coming at this from a lens of "understand the field well enough to think about how to make useful infrastructural advances", but I'm interested in hearing thoughts about various ways people keep-up-with-stuff and how they gain value from it.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Understanding the two most common mental health problems in the world, published by spencerg on July 7, 2023 on The Effective Altruism Forum.

Co-authored with Amanda Metskas

This is a linkpost from ClearerThinking.org. We've included some excerpts of the article below, but you can read the full post here.

Chances are, your life has been impacted by anxiety disorders or depression, either through your direct experience or through the impact they have had on your loved ones. Anxiety and depression are the two most common mental health conditions in the world, but they are frequently misunderstood.

Previous EA Mental Health surveys (2023, 2021, and 2018) have also highlighted the importance of these topics to people in the community who take the surveys.

In this data-based essay, we aim to help you better understand anxiety and depression, as well as the hidden links between them. By improving your understanding of these disorders, you may find it easier to recognize anxiety and depression in yourself and be more effective at supporting people in your life who experience these conditions. This infographic summarizes some of our interesting findings about the differences and similarities between anxiety and depression (click here to see the infographic at full size).

The scale of both anxiety and depression is vast: the World Health Organization estimates that 301 million people worldwide suffer from an anxiety disorder, and 280 million people worldwide suffer from depression. Worldwide, depression ranks as the second largest cause of disability, and anxiety ranks eighth, according to analyses of the most recent Global Burden of Disease study. And yet, despite their prevalence and severe impacts, humanity’s scientific understanding has a substantial way to go to fully understand and highly reliably treat these conditions. Improved treatment and management techniques could make a huge difference in the quality of life of hundreds of millions of people around the world. These astounding statistics have motivated us to run our own studies investigating how these conditions work and how they relate to each other. This article will explain what we found!

Overlapping Disorders

A major obstacle to understanding anxiety and depression is that they often go together - many people who experience one also experience the other. Approximately 45% of people who experience a depressive disorder in their lifetime also experience an anxiety disorder, and these often occur during the same timeframe. Among people with Generalized Anxiety Disorder, about 43% of them will also experience depression in their lifetime. In one of our own studies, we found that commonly used measures of anxiety and depression (the GAD7 and PHQ9 scales) had shockingly high correlations (r=0.82). These strong links between anxiety and depression can make it more difficult to disentangle how each of these disorders works and make it more difficult for a person with anxiety and depression to effectively manage their conditions. Some people even think they have an anxiety disorder when it's more accurate to say they have a depressive disorder or the reverse.

The co-occurrence of anxiety and depression is a bit puzzling because they almost seem like opposites when experienced in the moment. A high level of anxiety often feels like being “wound up” - muscle tension, rapid heart rate, and chest tightness are among the most common physical symptoms. People experiencing anxiety may have a nervous energy that makes it difficult for them to relax, even if there is nothing they can practically do to address whatever is making them anxious. Depression, on the other hand, often feels like struggling to muster energy or motivation to care about things enough to take any action at all. Doing things, including things that a perso...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Jesse Hoogland on Developmental Interpretability and Singular Learning Theory, published by Michaël Trazzi on July 6, 2023 on LessWrong.

Jesse Hoogland is a research assistant at David Krueger's lab in Cambridge studying AI Safety who has recently been publishing on LessWrong about how to apply Singular Learning Theory to Alignment, and even organized some workshop in Berkeley last week around this.

I thought it made sense to interview him to have some high-level overview of Singular Learning Theory (and other more general approaches like developmental interpretability).

Below are some highlighted quotes from our conversation (available on Youtube, Spotify, Google Podcast, Apple Podcast). For the full context for each of these quotes, you can find the accompanying transcript.

Interpreting Neural Networks: The Phase Transition View

Studying Phase Transitions Could Help Detect Deception

"We want to be able to know when these dangerous capabilities are first acquired because it might be too late. They might become sort of stuck and crystallized and hard to get rid of. And so we want to understand how dangerous capabilities, how misaligned values develop over the course of training. Phase transitions seem particularly relevant for that because they represent kind of the most important structural changes, the qualitative changes in the shape of these models internals.

Now, beyond that, another reason we’re interested in phase transitions is that phase transitions in physics are understood to be a kind of point of contact between the microscopic world and the macroscopic world. So it’s a point where you have more control over the behavior of a system than you normally do. That seems relevant to us from a safety engineering perspective. Why do you have more control in a physical system during phase transitions?" (context)

A Concrete Example of Phase Transition In Physics and an analogous example inside of neural networks

"Jesse: If you heat a magnet to a high enough temperature, then it’s no longer a magnet. It no longer has an overall magnetization. And so if you bring another magnet to it, they won’t stick. But if you cool it down, at some point it reaches this Curie temperature. If you push it lower, then it will become magnetized. So the entire thing will all of a sudden get a direction. It’ll have a north pole and a south pole. So the thing is though, like, which direction will that north pole or south pole be? And so it turns out that you only need an infinitesimally small perturbation to that system in order to point it in a certain direction. And so that’s the kind of sensitivity you see, where the microscopic structure becomes very sensitive to tiny external perturbations.

Michaël: And so if we bring this back to neural networks, if the weights are slightly different, the overall model could be deceptive or not. Is it something similar?

Jesse: This is speculative. There are more concrete examples. So there are these toy models of superposition studied by Anthropic. And that’s a case where you can see that it’s learning some embedding and unembeddings. So it’s trying to compress data. You can see that the way it compresses data involves this kind of symmetry breaking, this sensitivity, where it selects one solution at a phase transition. So that’s a very concrete example of this." (context)

Developmental Interpretability

"Suppose it’s possible to understand what’s going on inside of neural networks, largely understand them. First assumption. Well then, it’s still going to be very difficult to do that at one specific moment in time. I think intractable. The only way you’re actually going to build up an exhaustive idea of what structure the model has internally, is to look at how it forms over the course of training. You want to look at each moment, where you learn sp...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A Defense of Work on Mathematical AI Safety, published by Davidmanheim on July 6, 2023 on The Effective Altruism Forum.

AI Safety was, a decade ago, nearly synonymous with obscure mathematical investigations of hypothetical agentic systems. Fortunately or unfortunately, this has largely been overtaken by events; the successes of machine learning and the promise, or threat, of large language models has pushed thoughts of mathematics aside for many in the “AI Safety” community. The once pre-eminent advocate of this class of “agent foundations” research for AI safety, Eliezer Yudkowsky, has more recently said that timelines are too short to allow this agenda to have a significant impact. This conclusion seems at best premature.

Foundational research is useful for prosaic alignment

First, the value of foundational and mathematical research can be synergistic with both technical progress on safety, and with insight into how and where safety is critical. Many machine learning research agendas for safety are investigating issues identified years earlier by foundational research, and are at least partly informed by that research. Current mathematical research could play a similar role in the coming years, as more funding and research are increasingly available for safety. We have also repeatedly seen the importance of foundational research arguments in discussions of policy, from Bostrom’s book to policy discussions at OpenAI, Anthropic, and DeepMind. These connections may be more conceptual than direct, but they are still relevant.

Long timelines are possible

Second, timelines are uncertain. If timelines based on technical progress are short, many claim that we have years not decades until safety must be solved. But this assumes that policy and governance approaches fail, and that we therefore need a full technical solution in the short term. It also seems likely that short timelines make all approaches less likely to succeed. On the other hand, if timelines for technical progress are longer, fundamental advances in understanding, such as those provided by more foundational research, are even more likely to assist in finding or building more technical routes toward safer systems.

Aligning AGI ≠ aligning ASI

Third, even if safety research is successful at “aligning” AGI systems, both via policy and technical solutions, the challenges of ASI (Artificial SuperIntelligence) still loom large. One critical claim of AI-risk skeptics is that recursive self-improvement is speculative, so we do not need to worry about ASI, at least yet. They also often assume that policy and prosaic alignment is sufficient, or that approximate alignment of near-AGI systems will allow them to approximately align more powerful systems. Given any of those assumptions, they imagine a world where humans and AGI will coexist, so that even if AGI captures an increasing fraction of economic value, it won’t be fundamentally uncontrollable. And even according to so-called Doomers, in that scenario, for some period of time it is likely policy changes, governance, limited AGI deployment, and human-in-the-loop and similar oversight methods to limit or detect misalignment will be enough to keep AGI in check.

This provides a stop-gap solution, optimistically for a decade or even two - a critical period - but is insufficient later. And despite OpenAI’s recent announcement that they plan to solve Superalignment, there are strong arguments that control of strongly superhuman AI systems will not be amenable to prosaic alignment, and policy-centric approaches will not allow control.

Resource Allocation

Given the above claims, a final objection is based on resource allocation, in two parts. First, if language model safety was still strongly funding constrained, those areas would be higher leverage, and avenues of foundat...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Agency begets agency (the world is malleable), published by Richard Ngo on July 6, 2023 on LessWrong.

The second way of cultivating excitement-based motivation I'll discuss is cultivating agency, by which I mean the ability to try to shape the world. The key idea I want to convey here is that the feeling of agency is in many cases a self-fulfilling prophecy, because the hardest part of many things is mustering the energy and initiative to get started. If you think of yourself as agentic, then you’ll continually be looking out for opportunities, and approach the world with a “why not?” attitude. That’ll give you more chances to do well, which will give you more evidence that you’re the sort of person who can make stuff happen, and reinforce your sense of agency.

Why don’t people have that sense by default? One reason is that most people have experienced a huge amount of social pressure towards conformity, especially as children. Doing unusual or novel things often leads to criticism or ridicule, which ingrains a sense of learned helplessness about defying consensus. This is especially strong for teenagers, most of whom spend a huge amount of effort trying to be liked and fit in. And it’s not just enough to be willing to stand out: your success might make others look bad, which creates incentives for them to ”cut you down to size”, or deter you from trying in the first place (e.g. by mocking anyone who cares enough about something to try hard to achieve it).

A more intellectual justification for lacking agency comes from epistemic modesty: the view that you should pay relatively little attention to your own view on most topics, and defer instead to other people. One intuition that often motivates epistemic modesty is the idea that markets are often efficient, in the sense that you can’t consistently beat them. However, people often apply epistemic modesty to conclude that they shouldn’t trust their inside view even in domains that are very different from markets: founding a startup, or doing research, or reasoning about politics, or even finding a partner. In those domains, epistemic modesty often involves thinking that you’re not so special: if others disagree, then there’s little reason to think you’re mistaken rather than them; and if others haven’t taken an opportunity, then there’s probably some reason you shouldn’t (or can’t) either.

Certainly there are some people who should pay more attention to epistemic modesty. However, there are at least three clusters of arguments for why epistemic modesty is strongly overrated: epistemic, normative, and social. Many important arguments in the first cluster are laid out by Yudkowsky in his book Inadequate Equilibria. One of his core points is that efficient market arguments only apply when inefficiencies are exploitable - i.e. you can benefit from noticing the inefficiency. This is often not true, e.g. in cases where the decision-makers behind the inefficiency don’t have skin in the game. More generally, decisions made by large bureaucracies are often dumb even from the perspectives of the people in those bureaucracies, since their optimization for local incentives can easily make the bureaucracy as a whole dysfunctional.

One of the few upsides of covid is that it’s provided us many straightforward examples of repeated, lethal societal incompetence, in terms of our gain-of-function policies and vaccine development policies and mask policies and lockdown policies and vaccine deployment policies and many others.

A second cluster of arguments against epistemic modesty relates to what you should actually do. Even in the cases where epistemic modesty is the right mindset for making predictions, it’s often a very bad mindset for choosing actions - because the biggest rewards often come from defying consensus. For example, spotti...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: AI #19: Hofstadter, Sutskever, Leike, published by Zvi on July 6, 2023 on LessWrong.

The big news of the week is OpenAI introducing the Superalignment Taskforce. OpenAI is pledging 20% of compute secured to date towards a four year effort to solve the core technical challenges of superintelligence alignment. They plan to share the results broadly. Co-founder and OpenAI Chief Scientist Ilya Sutskever will lead the effort alongside Head of Alignment Jan Leike, and they are hiring to fill out the team.

That is serious firepower. The devil, as always, is in the details. Will this be an alignment effort that can work, an alignment effort that cannot possibly work, an initial doomed effort capable of pivoting, or a capabilities push in alignment clothing?

It is hard to know. I will be covering these questions in a post soon, and want to take the time to process and get it right. In the meantime, the weekly post covers the many other things that happened in the past seven days.

Table of Contents

Introduction.

Table of Contents.

Language Models Offer Mundane Utility. Commercial LLMs keep the edge.

Language Models Don’t Offer Mundane Utility. Can you?

Fun With Image Generation. Valve says: Keep all that fun away from Steam.

They Took Our Jobs. Then they came for America’s finest news source.

Introducing. If only it were as easy to forget.

Reinforcement Learning By Humans From Feedback. United we grok.

Costs are not Benefits. Easy mistake to make. Many such cases.

Quiet Speculations. Questions that are not so infrequently asked.

The Quest for Sane Regulation. Some small good signs.

Another Open Letter. EU CEOs call on EU to not act like the EU.

The Week in Audio. Odd Lots offers AI fun.

No One Would Ever Be So Stupid As To. Is that what you think?

Safely Aligning a Smarter than Human AI is Difficult. Formal verification?

Rhetorical Innovation. The latest crop of explanations and resources.

People Are Worried About AI Killing Everyone. Including Douglas Hofstadter.

The Lighter Side. Actual progress.

Language Models Offer Mundane Utility

Open source models often do well on open source benchmarks. So do foreign efforts like Falcon or Ernie. When it comes to actual mundane utility, or tests that were not anticipated in advance, the answer seems to come back somewhat differently.

Lmsys.org: Quick note - we’ve transitioned from the deprecated vicuna benchmark to a more advanced MT-bench, including more challenging tasks and addressing biases/limitations in gpt4 eval. We find OpenChat’s performance on MT-bench is similar to wizardlm-13b. That’s said, there remains a significant gap between open models and GPT-3.5, which is exactly what we aim to emphasize with MT-bench - to highlight this discrepancy. Though not flawless, it’s one step towards a better chatbot evaluation. Please check out our paper/blog for more technical details and leaderboard for complete rankings.

Jim Fan: For most of the “in the wild” trials, GPT-3.5 just feels much better than open-source models that claim good performance metrics. Such “vibe gap” is typically caused by inadequate benchmarking. Don’t get excited by numbers too quickly. Beware of over-claims.

Links: Blog, Leaderboard, Paper.

Falcon-40B is down at 5.17.

Note that reasoning is the place where GPT-4 has the largest edge.

Will they offer all that mundane utility for free? David Chapman thinks that without a moat no one will make any money off of LLMs. Other than Nvidia, of course.

Will Manidis: the core innovation of Foundation Model providers is not technical it’s allowing VCs to deploy $500m into a deal with basically zero underwriting that’s $20m in fees, $100m in carry at a 2x for like . 10 days of memo writing and no customers to reference.

David Chapman: Regardless of how useful GPTs turn out to be, I’m skeptical anyone makes much money off of...