Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Should we call them something other than retreats?, published by Rockwell Schwartz on May 18, 2022 on The Effective Altruism Forum.

Retreats—described by Leilani Bellamy of the new EA org Canopy Retreats as "a gathering of people with a common interest or goal, all sharing space over the course of a weekend or a week"—are increasingly in popularity within university groups, local groups, and EA orgs. They typically involve presentations, group and 1-1 discussions, and bonding activities, and they sometimes produce tangible outputs.I worry that the term "retreat" also contributes to optics issues because it (1) implies opulence (e.g. "wellness retreats"), (2) harkens to religious retreats and may increase "cult" vibes, and (3) doesn't actually capture the work that occurs (i.e. when I've said I'm going on a retreat, friends have assumed I'm on vacation).Thoughts on alternative terms?

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Gato's Generalisation: Predictions and Experiments I'd Like to See, published by Oliver Sourbut on May 18, 2022 on The AI Alignment Forum.

I'm deliberately inhabiting a devil's advocate mindset because that perspective seems to be missing from the conversations I've witnessed. My actual fully-reflective median takeaway might differ.

My covid has made writing difficult at the moment, and I haven't had the energy to gather citations or fully explain the detail for some of the assertions in this post.

I believe (for various reasons not detailed here) that qualitative advances in general agentic performance from artificial systems are likely this decade and next - I just don't think Gato represents progress in that direction. I'm not particularly surprised by anything in the Gato paper[1]. Naturally, then, I'm against hyperbole around the capabilities demonstrated.

There is not enough information in the paper to say either way, but it may be the case that Gato represents a novel and cutting-edge distillation technique and if so, I would prefer if it were framed that way than as an increment towards generalisation ability of artificial agents!

The authors do a reasonable job of reporting their experiments (though I sense some motivated reasoning in the presentation of some results[2]), but the PR version of the story (at least as I have encountered it) departs from the mostly-honest paper itself[3].

I focus here almost entirely on sequential control, since that is the aspect of the discussion around the Gato paper which seems most clouded, while being at the centre of the purported generalisation, and in which I have the most interest and expertise.

Techniques and phenomena

We've known for many years that attention mechanisms (early 2010s) allow parameter sharing over sequences and over inputs of variable size/shape, that behavioural cloning (early 1990s?) often works with enough expert examples, that representation transfer (embeddings, early 2010s?) works, and that fine-tuning parameters (2000s? 2010s?) is faster than learning from scratch.

What should the key empirical takeaway be?

In my reading, if you subtract the already-known phenomena, and pass over the mostly-ignored distillation aspect, there is an important empirical finding in the Gato paper. I'm not sure if it's novel per se, but it probably is at this scale in this domain.

Namely, in our current systems, fine-tuning is typically fast when pretraining includes closely related tasks (e.g. DM Control Suite and Meta-World), slow when pretraining does not include closely related tasks (e.g. every control task without domain-relevant pretraining, including RGB stacking), and actively worse when pretraining includes distracting unrelated tasks (e.g. the single held-out Atari task, Atari boxing, is distracted by any pretraining in the reported experiments!).

Fine-tuning progress demonstrating benefit of closely related pretraining, and neutrality or harm of unrelated pretraining

Fine-tuning progress demonstrating benefit of closely related pretraining, and neutrality of unrelated pretraining

There is the additional novelty of goal-conditioning by context-prompting with a completed task, both during training and deployment (section 2 of the paper). Goal-conditioning is an interesting and relatively recent area of research. As with all such techniques I am aware of, this conditioning technique requires explicit presentation of a goal state which is structurally very similar to those seen in training. As briefly discussed in the paper, this could be applicable in the context of perceptual variation for closed-environment control (like some industrial robotics tasks) but this is fundamentally separate from general agency.

Comparison to other results regarding sequential control

Let's consider a comparison to the DQN pap...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Some potential lessons from Carrick’s Congressional bid, published by Daniel Eth on May 18, 2022 on The Effective Altruism Forum.

The primary for Oregon’s 6th congressional district is now over, and it’s looking like Carrick is going to lose, coming in second out of nine candidates. That Carrick lost is a real shame. My sense is if Carrick had won, he could have done a lot of good – in particular, advancing pandemic prevention (e.g., via participating in bill markups), with an outside chance of getting Biden’s pandemic prevention plan enacted. As a silver lining, I think the value of information from this race was quite high, and I wouldn’t be surprised if the information this race provides will be valuable enough to EA that it outweighs the costs associated with the race.

First off, I want to applaud Carrick for taking the plunge and running; as a general rule, I think EAs should be more ambitious, especially in novel situations where the value of information is high, and it’s nice to see someone really following through with this. Carrick, his wife Kathryn, and their whole team put it in an enormous amount of work, faced many obstacles outside of their control, and withstood unfair public criticism, all in the service of improving the world.

Below are what I believe to be some of the more important lessons that we should take away from this race. For transparency sake, I started jotting down notes and sketching part of a draft of this piece before the day of the race – I didn’t write the piece entirely today. I also got feedback on this post from a few people in my network, and I strongly recommend others who plan on writing further posts on this topic similarly get feedback from other EAs before posting, in order to ensure that the post embodies community ideals, both to those reading it from within our community and those who may be looking in from the outside, especially right now as more external attention is being directed at EA. Personally, some of the feedback I got from others led me to rephrase certain things in ways I hadn’t considered and for which I think was helpful for reducing misunderstandings of EA.

Lesson 1: EAs can seriously compete for open congressional seats, and preparing to run well in advance of a race probably would significantly further improve the odds of winning

Yes, Carrick lost. But he came in second out of nine, despite several factors pushing pretty strongly against him. Had things shaken out differently on a few key factors, he could have won.

Here’s a partial list of factors that were pushing against Carrick:

lack of local political connections, which led local political actors to strongly fight against him

relative lack of practice/training in talking like a politician, which understandably led to politically costly misstatements

running in a state that he had very recently moved back to (he moved back after COVID hit), which appears to have led to some skepticism

a combination of the above factors led to unfavorable media coverage

Salinas (the winner of the primary) benefitting from $1M of Super PAC money herself from BOLD PAC.

Most of the items on that list seem avoidable by any future EA candidate who preps for a political career far ahead of time instead of first beginning to think about running when a seat opens up – even the adversarial Super PAC spending (the least controllable item on the list) may be reduced by having better long-term relationships with the political community. For instance, if Carrick had held local office before running for the House, I think he would have been in a much stronger position for this race (I think he probably would have won). To be clear, this point is not a criticism of Carrick running for the House directly – I actually agree with the decision, given that Congressional seats only open u...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Popular education in Sweden: much more than you wanted to know, published by Henrik Karlsson on May 17, 2022 on LessWrong.

Growing up on the Swedish seaside, I had a five-minute walk to four open learning facilities – not counting the library and the youth center. It was very Christopher Alexander.

One of the premises was an abandoned church that my friends and I used as a recording studio; we'd renovated it ourselves with funding from a study association. There we played distorted pop. In another, I learned French from an émigré of Montpellier. We arranged public lectures – once, to our great surprise, we even managed to book then general secretary of the United Nations Ban-Ki Moon for a lecture in Uppsala. I analyzed Soviet cinema with a group of whom an unsettling number sang Sång för Stalin before the screenings.

Since leaving Sweden, I have realized that not everyone grows up like this. And I miss it. In fact, if the whole of Sweden was about to burn down and I could only save one thing, I might grab just folkbildningsrörelsen.

Folkbildningsrörelsen: that is the name we have for this movement of self-organized study groups, resource centers, maker spaces, public lectures, and free retreats for personal development.

These types of things exist in other countries too – but not at the same scale. Or even close.

To get a sense of how comprehensive folkbildningsrörelsen is, it helps to remember that Sweden has a population roughly comparable to New York City. If NYC had as many free resource centers per inhabitant as the municipality where I grew up, Manhattan would look like this:

I was going to do all of New York but my hand started hurting from making all these dots, so I only managed the tip of Manhattan.

At every other intersection, there would be a few rooms where you could go in and get some money to buy literature or access tools you needed. (In practice, the resource centers in cities tend to be lumped together in larger units, but the map still captures a lived reality for the 7.5 percent of Sweden's population who regularly take part in study associations.)

Experientially, the spaces I have been part of have felt more like niche internet forums than schools. There were plenty of trolls, witches, and freaks – but we were also able to sustain a depth of conversation which was out of scope at school. When I entered university, seminars often felt like play-acting in comparison. In our often quite dilapidated buildings (as in internet communities), we hadn’t thought about what we were doing as learning.

We were just obsessing about things.

How did this all come about?

In the 19th century, when these houses and the financing that enables them began to be built out, the main impetus came from the German Bildung tradition.

Bildung etymologically refers to shaping yourself in the image (das Bild) of God. God in this context should be imagined as a highly self-possessed spectral being – in control of its emotions, with mind and heart in harmony, and willing to take individual moral responsibility. Think Bertrand Russell but less atheist, and sitting on a cloud.

This is the look.

In its original formulation, Bildung had a somewhat bourgeois flavor. It smelled of tweed and leather elbow patches. But in the early 1800s, thinkers such as Johann Heinrich Pestalozzi, N.F.S. Grundtvig, and Johann Friedrich Herbart figured out how to sell Bildung to farmers and day laborers – a folk Bildung, or folkbildning in Swedish. This was the tradition that took root in Sweden: the popular movement to shape yourself in the image of Bertrand Russell.

The English language version of folkbildning’s Wikipedia page refers to it as popular education. This translation is not entirely correct. The term "popular education" has a strong political connotation – the Wikipedia page talks about "c...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: EA analyst/intern wanted, published by alsilverback on May 17, 2022 on The Effective Altruism Forum.

I am looking for an analyst (remote is ok) that can help with 'custom cause/action prioritization' .

There are certain cause areas I am interested to invest time and capital in (for ex. Global Governance, Energy Revolution, Intelligence Revolution) . However, within each I would like to run a quick INT (importance, neglectedness, traction) or similar prioritization frameworks, and take the subsets that stand out and run another cross-analysis with my personal character nuances, corporate and network resources, and my interests/goal dynamics to see which a) fits most the aforementioned criteria, b) most actionable with my resources.

I have an engineering background, so a quantitative analysis would be preferred. I assume my input would be the initial broad causes of interest, followed by a discussion on why I am interested in those, the analyst would then run an INT analysis on the subsets of these causes, followed by a discussion on where I and my resources can fit and why. Short written reports (1-2 pages) should be a deliverable after each stage.

About me: I am an ex-deep tech entrepreneur (robotics/automation) , have since exited and looking to direct my new investment vehicle towards the most fulfilling purpose. Based in Sweden.

This can be paid of course. Let me know if you're interested, a bit about yourself and your hourly rate.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: On saving one's world, published by Rob Bensinger on May 17, 2022 on LessWrong.

If the world is likeliest to be saved by sober scholarship, then let us be sober scholars in the face of danger.

If the world is likeliest to be saved by playful intellectual exploration, then let us be playful in the face of danger.

Strategic, certainly; aware of our situation, of course; but let us not throw away the one mental mode that can actually save us, if that's in fact our situation.

If the world is likeliest to be saved by honest, trustworthy, and high-integrity groups, who by virtue of their trustworthiness can much more effectively collaborate and much more quickly share updates; then let us be trustworthy. What is the path to good outcomes otherwise?

CFAR has a notion of "flailing". Alone on a desert island, if you injure yourself, you're likelier to think fast about how to solve the problem. Whereas injuring yourself around friends, you're more likely to "flail": lean into things that demonstrate your pain/trouble to others.

To my eye, a lot of proposals that we set aside sober scholarship, or playful intellectual exploration, or ethical integrity, look like flailing. I don't see an argument that this setting-aside actually chains forward into good outcomes; it seems performative to me, like hoping that if our reaction "feels extreme" enough, some authority somewhere will take notice and come to the rescue.

Who is that authority?

If you have a coherent model of this, we can talk about it and figure out if that's really the best strategy for eliciting their aid.

But if no one comes to mind, consider the possibility that you're executing a social instinct that's adaptive to threats like tigers and broken legs, but maladaptive to threats like Unfriendly AI.

If you feel scared about something, I generally think it's good to be honest about that fact and discuss it soberly, rather than hiding it. I don't think this is incompatible with rigorous scholarship or intellectual play.

But I would clearly distinguish "being honest about your world-models and feelings, because honesty is legitimately a good idea" from "making it your main strategy to do whatever action sequence feels emotionally resonant with the problem".

An "extreme" key doesn't necessarily open an "extreme" lock. A dire-sounding key doesn't necessarily open a dire-feeling lock. A fearful or angry key doesn't necessarily open a lock that makes you want to express fear or anger.

Rather, the lock's exact physical properties determine which exact key (or set of keys) opens it, and we need to investigate the physical world in order to find the right key.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: EAs Should Consider Applying to Join U.S. Diplomacy, published by abiolvera on May 17, 2022 on The Effective Altruism Forum.

The U.S. Foreign Service – the United States’ diplomatic corps – is a highly sought after job in Washington - and is likely a promising path for U.S. citizens aiming to work on issues of great power competition, cybersecurity, artificial intelligence governance, biosecurity, global health, and global poverty.

The two of us served as diplomats; the knowledge and experience we gained was invaluable. Here, we reflect on our careers at the State Department in the context of effective altruism.

Is this career for me?

The Foreign Service, the diplomatic corps of the United States, offers a rare opportunity to get experience in different long-termist/EA topics, including nonproliferation, public health, biosecurity, AI, cybersecurity, and U.S.-great power relations, even if you come in without experience or expertise in such topics. Similar to a military officer, the career requires you to rotate your job assignment every few years, providing the opportunity to get meaningful experiences across a range of issue areas, anywhere on the globe.

Employment as a Foreign Service Officer grants you entry into the federal government system, a security clearance, and substantive foreign policy experience in varying subjects. Foreign Service Officers gain career experience as policy “generalists,” able to switch in and out of important topics. This skillset and diverse set of experiences allows Foreign Service Officers who depart to easily enter other policy careers and/or specialize in important topics.

The hiring process is different from most other federal and private sector careers. All that is required is that you pass a couple standardized tests and resume screening – no strict education/work experience requirements exist, nor does the hiring process strongly favor people with prior policy experience. EAs with limited or no policy experience who are looking to get into the U.S. government or policy should consider applying despite the 2-3% pass rate. While the Foreign Service is often a second career (average age of entry is 32), there are those people who enter the Foreign Service in their early 20s as their first career via applying directly or a fellowship program. Abi entered at 23 right out of her masters program and Dan joined at 26. (Note: You must be a U.S. citizen to apply.)

The Foreign Service hiring process is largely immune to the power of networking because each of the four phases of review involves multiple independent judges and scoring systems, which are mostly disconnected from each other. Recommendations carry no weight in the process. An American citizen can take the first portion of the test almost anywhere in the world (embassies offer the test to U.S. citizens abroad), so one of the judges knowing a candidate is rare and would likely lead to a judge recusing themselves. The application process isn’t perfect by any means, but issues stem from the difficulty of assessing the 13 desired skills rather than nepotism.

The testing process seems to moderately favor extroverts and those with good writing/grammar skills, knowledge of global affairs, and familiarity with embassies/Foreign Service general structures. Eligibility for both the medical and security clearances could be impacted by mental health issues, though the government only denied 11 security clearances for psychological reasons in 2021, or because of recent drug use. Marijuana use, for instance, is federally illegal even if used in a state that has legalized it.

To evaluate whether the career is right for you, check out the State Department’s website and take its quiz on whether you’re a good fit. Most relevantly, Foreign Service Officers move to an entirely new country every f...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [Link post] The Esoteric Social Movement Behind This Cycle’s Most Expensive House Race, published by abierohrig on May 17, 2022 on The Effective Altruism Forum.

In case you missed, there was a Politico article last week about effective altruism’s increasing role in politics. The piece isn’t totally perfect, but I think it’s high-quality journalistic engagement — it’s very well-researched, and it gives an impressively clear characterization of what effective altruism is and why effective altruism principles are motivating greater political involvement. If you’re ever looking for an article or primer about what EA has to do with politics both in theory and practice, I think this one is the best so far.

Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Formula for Dying Babies, published by Zvi on May 17, 2022 on LessWrong.

Note: This was posted originally on Thursday, May 12 as part of the weekly Covid post. It is being separated out into its own post for future reference, and in case some people are no longer reading Covid posts.

There’s a shortage of specialty infant formula. Half of all types are unavailable. Some parents are panicking, without a plan for how to feed a baby that can’t use regular formula.

An infant formula plant shutdown triggered by two infant deaths has created a new nightmare for some parents: There’s now a dangerous shortage of specialized formulas that are the only thing keeping many children and adults alive.

The Abbott Nutrition plant in Sturgis, Mich., was not just one of the biggest suppliers of infant formula nationally, but it was also the major supplier of several lesser-known specialty formulas that are a lifeline for thousands of people with rare medical conditions, including metabolic, allergic and gastrointestinal disorders, which can make eating regular foods impossible or even dangerous. The situation has not only rattled parents and medical professionals, but has raised questions about whether the federal government should do more to ensure critical, life-sustaining supply chains don’t break down.

“If this doesn’t get fixed soon, I don’t know how my son will survive,” said Phoebe Carter, whose 5-year old son John — a nature-lover and “paleontologist in training” — has a severe form of Eosinophilic Esophagitis, a rare digestive and immune system disease driven by a dysfunctional immune response to food antigens. “I just can’t stress that enough.”

One of my good friends looked into this a bit and isn’t buying that the shortage could be caused by shutting down this one plant. This article’s primary contribution is that the supply chains were already strained before the shutdown due to demand fluctuations on top of supply issues in the wake of the pandemic. It’s certainly a large contributor, and it’s possible without the shutdown we wouldn’t have a problem.

This one points out that before the recent recall, we were on the edge of disaster to start, through a combination of the usual supply chain disruptions and the usual tariffs. We went out of our way to ensure that the supply of baby formula couldn’t compete against American dairy farmers, and, well, whoops. This quote from that post (the quote is originally from here) is very on the nose:

Canada agreed that, in the first year after the agreement takes hold, it can export a maximum 13,333 tonnes of formula without penalty. In USMCA’s second year, that threshold rises to 40,000 tonnes, and increases only 1.2 per cent annually after that. Each kilogram of product Canada exports beyond those limits gets hit with an export charge of $4.25, significantly increasing product costs..

Canada wanted to attract investment for a baby formula facility because it uses skim milk from cows as an ingredient. Healthy consumer appetites for butter leave provincial milk marketing boards with a surplus of skim. Baby formula looked like a smart use for it, and Canada didn’t have any significant infant formula production before Feihe arrived.

Expanding this plant, or building a second infant formula plant somewhere else in Canada, look like less attractive business propositions under this new trade deal.

I don’t get why even people who generally sound even more infuriated about this than I do, like Scott Lincicome, still produce sentences like this first one:

These regulatory barriers are probably well-intentioned, but that doesn’t make them any less misguided—especially for places like Europe, Canada, or New Zealand that have large dairy industries and strict food regulations. Indeed, as the New York Times noted about “illegal” European formula in ...

Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: The "Adults in the Room", published by David Udell on May 17, 2022 on LessWrong.

Significant spoilers for mad investor chaos and the woman of asmodeus.

Dath ilan has put significant effort into protecting what it has to protect. To that end, it's thought quite a bit about how to think effectively. Dath ilan knows how to train people to preempt reality, to concentrate their thoughts on modeling the tiny slivers of possibility space that reality is on track to wander into and to effectively respond.

However, they don't disseminate this art to all their citizens to the greatest possible extent. They teach it fully to just a select few, and teach everyone else a good chunk. Their reason for holding out on making most people into the best rationalists they can be . is that, as an unfortunate quirk of human psychology . becoming the best rationalist you can as quickly as you can is not the most fun path you can chart through your life.

A few dath ilani specialize in becoming Keepers -- full time, professional, regularly tested and proven masters of Bayescraft. Keepers in dath ilan are the "adults in the room." Their job is to make sure that everyone else can life a fun-theoretically optimized life without sacrificing their civilization's ability to seriously reason about its future. A purely Keeper civilization would be missing out on something worth protecting; a civilization without enough adults around would be helpless in the face of capricious reality.

There is a commonly held wisdom, in dath ilan, about the way a human mind is put together, that it is a thing made of little subtle tensions and balances and internal compromises. The human mind being the limited thing that it is, these balances form around your current level of ability to see into yourself and see the implications of what you already know - or not see them, as the case may be. The reason why not everybody runs off to learn all they can from Keepers, the reason why not everyone asks a Keeper to tell them all the answers about themselves, is that this would bring parts of themselves into conflict that were previously living in a more agreeable truce of ignorance. You might not survive as yourself, if you could see yourself.

Those who say "That which can be destroyed by the truth should be" may continue to walk the Path from there. But not uncommonly, even somebody who sets out along that Path, turns back at some point, and well short of becoming a Keeper. It's not a trivial price, higher for some than others, and there is varying willingness to pay. A lot of the reason why Keepers exist as what they are, is that the people who have large comparative advantages there - in how little they'll be hurt by knowing themselves, or how much they really internally want to keep going anyways - are conceived of by larger society as being paid to throw themselves on that grenade, so others don't have to. And if, to some Keepers, it doesn't feel like much of a grenade at all, they understand that their case is not typical, and are grateful for winning the comparative-advantage lottery.

Going up by two local standard deviations, in whatever it is that Owl's Wisdom enhances, is something that the current structures of Keltham's personality were never built to withstand. He knows, from up here, because he couldn't stop himself from glancing in that direction, that in dath ilan he would never have had his 144 children. He would have tried to be special and failed and been sad and then maybe gotten an ordinary +0.8sd job and either paid for a child out of that or decided he was too strange and unhappy to have one.

It's not considered necessary for somebody Keltham's age to go and pay a Keeper to tell them exactly what the probabilities are, about something like that. It's not so much that people are encouraged to lie to th...