The Nonlinear Library

The Nonlinear Library

By The Nonlinear FundEducation
Download on the App Store

All The Nonlinear Library episodes

  • LW - Reframing inner alignment by davidad
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Reframing inner alignment, published by davidad on December 11, 2022 on LessWrong.
    The standard frame (Evan Hubinger, 2021) is:
    Outer alignment refers to the problem of finding a loss/reward function such that the training goal of “a model that optimizes for that loss/reward function” would be desirable.
    Inner alignment refers to the problem of constructing a training rationale that results in a model that optimizes for the loss/reward function it was trained on.
    Here’s the reframe (I believe the credit for this breakdown is due to John Wentworth, although I haven't found anything online to link to for it):
    Reward Specification: Finding a policy-scoring function J(π) such that (nearly–)optimal policies for that scoring function are desirable.
    "Are you optimising for the right thing?"
    Adequate Policy Learning: Finding a policy that’s actually (nearly–)optimal for that scoring function.
    "Did you optimise for it enough?"
    As Paul Christiano points out (in an excerpt recently highlighted by Alex Turner to make a similar point), factoring out Reward Specification represents only one "particular kind of alignment strategy, that's like a two step plan" for how one might try to conclude that a learned policy is desirable, where we first align a scoring function J with what we actually want, and then align a training process with our scoring function. Under this kind of plan, the proof tree for overall existential safety would conclude by applying an implication like this:
    Mesa-optimisers
    At this point you might be wondering: what happened to the concept that π might contain a mesa-optimiser for something different from J?
    The proper role of the mesa-optimiser concept is in an explanation of why Adequate Policy Learning is insidiously difficult:
    In machine learning, one typically approximates (the gradient of) the reward function by aggregating a set of evaluations on only a finite sample (the empirical “training distribution”), whereas the true J also depends on what π does globally, even in rare states that may never be sampled during training.
    If the trajectories observed in deployment have a state distribution which diverges from the training distribution, then the policy that was optimal on the training set will likely diverge from the optimal policy. This is standard out-of-distribution generalisation failure.
    If the policy that was optimal on the training set includes a good understanding of the real world and consequentialist reasoning (which is likely, if the training set is sufficiently rich that you can get a policy that's transformative in the real world), then it contains a capable mesa-optimiser, and it's likely that it's still transformative in the real world, even while experiencing generalisation failure: capabilities generalise better than alignment.
    The bottom line is, once we get beyond some level of capabilities, it becomes likely that the gap between the empirical training distribution and the intended reward function suddenly gets exploited in a pretty worst-case-like way (i.e. optimising for some arbitrary thing, instead of merely being bad at optimising the outer objective). I think this phenomenon is also what the concept of a sharp left turn refers to, so this post could equally also be titled "Reframing the sharp left turn".
    Inner alignment
    The standard "inner alignment" answer to "what do we do about this?" is, to quote Paul again,
    let’s build a system that’s internalized that objective in some sense, or is not doing any other optimization beyond pursuit of that objective.
    In other words: let's make sure that the policy π contains a mesa-optimiser for the outer objective J, or at least, contains no other mesa-optimisers.
    If we assume outer alignment and Reward Specification are solved, the above notion of inner alignment is neither necessary nor su...
    9 min
  • LW - A brainteaser for language models by Adam Scherlis
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: A brainteaser for language models, published by Adam Scherlis on December 12, 2022 on LessWrong.
    I came up with the following puzzle the other day:
    A: x =
    The intended answer is in the form of a number.
    text-davinci-003 guesses my intended answer at 11.8% probability, which is the second-highest probability for any answer.
    (This is somewhat cherry-picked; small changes to the phrasing give worse results. ChatGPT gave the intended answer the third time I asked it, but this appears to have been dumb luck. The true rate for ChatGPT is probably below 10%, and maybe below 5%.)
    So far, friends have found it fairly difficult. About two dozen people made at least one guess, and at least six spent a while on it. So far, two people have figured it out, in both cases after being told that GPT-3.5 could do it.
    You may want to try to solve it yourself before reading on.
    Here are some progressively-stronger hints:
    The answer is a string of ordinary decimal digits, it's not "NaN" or "/" or "63; 65536".
    The fact that 63 = 2^6 - 1 is a red herring and has no relevance. Sorry.
    Number bases other than decimal are not relevant.
    The answer depends on the digit strings in base ten as well as the value of the numbers.
    The answer depends on an accidental feature of some of GPT's training data.
    The accidental feature is a formatting bug.
    Here's the answer without explanation:
    x = 216
    And here's the explanation:
    63=216 and 216=65536
    Why can GPT-3.5 solve this? My guess:
    Superscripts are often lost when rich text is converted to plain text, producing sentences like "the Sun's mass is about 1030 kilograms". This problem affects a lot of GPT-3's training data.It appears that GPT is somewhat convinced, on some level, that "63 = 216" and "216 = 65536" are plausible facts.
    Some successful attempts to double-check this understanding:
    Replacing 63 with nearby numbers causes GPT to give much lower probabilities for the answer "216". (Except for 62, which gives similar results to 63...)
    Numbers near 216 get dramatically (20x-1000x) lower probabilities than 216 does.
    Even without "= 65536", 216 is favored ~100x over nearby numbers.
    Some wrinkles / contrary evidence:
    The top result, above "216", is the string "2^16 = 65536" -- but this is still the top output after changing 63 to other numbers, so it doesn't seem to be inspired by the number 216.
    GPT gives the token "216" in the string "63 = 216" a very low probability, just as low as "215" or "217".
    Replacing "63" with "62" in the prompt still gives "216" as an output with ~10% probability.
    As mentioned above, this doesn't work as well if you tweak the prompt in irrelevant ways, or ask a different model.
    Thanks to everyone who tried to solve my puzzle, congrats to Anton and Eli for solving it, and thanks to Georgia Ray for making the joke that inspired the puzzle.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    4 min
  • LW - Will Manifold Markets/Metaculus have built-in support for reflective latent variables by 2025? by tailcalled
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Will Manifold Markets/Metaculus have built-in support for reflective latent variables by 2025?, published by tailcalled on December 10, 2022 on LessWrong.
    Prediction markets and similar systems are currently nice for soliciting predictions for outcomes where there is a clear, unambiguous objective resolution criterion. However, many phenomena in the real world are hard to directly observe, but tend to have multiple indirect indicators. A familiar example might be aging/senescence, where you have indirect indicators like muscle weakness, gray hair, etc. that someone is aging, but you do not have a directly observable Essence Of Aging.
    There exists a type of math which can be used to statistically model such variables, called reflective latent variables. There are a number of specific implementations for specific contexts (factor analysis, latent class models, item response theory), but they are all mostly based on the notion of having several indicator variables which are supposed to be independent conditional on the latent variable.
    Essentially, a prediction market could implement this by allowing people to create questions with multiple resolution criteria, and allowing people to make correlated predictions over those resolution criteria. Then people could be scored based on their overall accuracy across these resolution criteria. If sufficiently many correlated predictions have been made, people might not even need to have specific opinions on the resolution criteria, but might just be able to bet on the probabilities of the abstract latent variables, and have the market infer what the corresponding bets on the resolution criteria would look like.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min
  • EA - Observations of community building in Asia, a 🧵 by Vaidehi Agarwalla
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Observations of community building in Asia, a 🧵, published by Vaidehi Agarwalla on December 11, 2022 on The Effective Altruism Forum.
    This is a lightly edited twitter thread I wrote after EAGxSingapore. I figured it might be useful to post to the Forum as well.
    Caveats
    these are my takeaways (with some interpretation) not just problems people told me.
    summarized a bunch to fit into the 280 character limit (flag emoji FTW).
    good > perfect
    Asian CB's please correct me + enhance!
    Everyone - please ask clarification questions!
    What do we do with people? This isn’t unique to Asian CB’s, but there are strictly fewer and less attractive opportunities available. I might do a separate thread or add to this thought later.
    Some countries have less traction with English & worry about EA being presented as Western concept (e.g. Japan & Iran). Translation of key texts seems important, and could be a way to engage newer EAs with a concrete project (see a test from Estonia).
    Countries with lots of internal issues may have difficulty gaining EA traction, but it may be a matter of perspective and approach. A Turkey CB mentioned that the fact that another group from Iran was able to get traction was inspiring, since they perceived Iran as having worse problems
    Multiple CB’s suggested that after talking to other EAs they started thinking more about city / uni group building rather than trying to build national groups to start with. For large countries (India) there are many choices, ...
    ..But some countries (Nepal, Pakistan, Vietnam) have 1 or 2 (Kathmandu, Karachi, Ho Chi Minh & Hanoi) major cities that are likely to be viable for EA groups. Kathmandu has all of Nepal’s top uni’s and best talent pool so in practice EA Nepal ~= EA Kathmandu.
    A few CBs want to/are starting local groups in liberal arts unis which they feel are more EA aligned. A challenge in Turkey is that vegans are abolitionist and against welfarism, and was concerned about discussing farmed animal welfare within EA.
    In Japan (+ others?), many students study abroad. There may be an opportunity to get those students interested in EA before they go (and connect them to local EA groups in the West), and catch them again after they return.
    E.g. 1 uni group struggled with reading group retention. It seemed plausible they could focus on their existing ~8 engaged members, or do a “trial week” for their reading group to help attendees evaluate fit early on.
    There is uncertainty over what messaging works best and non-existent testing. People mostly rely on their insights. More testing seems good, e.g. how much do you need to incorporate native philosophy vs. localizing examples and stories. bad e.g. "Asking someone who grew up or has spent a lot of time in LMICs to imagine a child drowning in a lake is not a hypothetical - it's something they might see and ask themselves every single day. This thought experiment loses a lot of its power. What are some alternatives?"
    Asking someone who grew up or has spent a lot of time in LMICs to imagine a child drowning in a lake is not a hypothetical - it's something they might see and ask themselves every single day. This thought experiment loses a lot of its power. What are some alternatives?
    Vietnam doesn’t have a big book-reading culture, so EA books could be less likely to be a way in. Perhaps focusing on blogs or podcasts or other formats is more promising?
    Many of us (including myself) learnt a lot from Israel on volunteer management and early stage group priorities. I believe a lot of value I provide CB's is expanding the option space and generating lots of specific examples. I wish more CB’s from around the world had attended.
    Early stage groups (which is most Asia groups today) could benefit from The Lean Startup model of validated learning - trying to optimize early activities to l...
    5 min
  • EA - Reflections on the PIBBSS Fellowship 2022 by nora
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Reflections on the PIBBSS Fellowship 2022, published by nora on December 11, 2022 on The Effective Altruism Forum.
    Cross-posted from LessWrong and the Alignment Forum
    Last summer, we ran the first iteration of the PIBBSS Summer Research Fellowship. In this post, we share some reflections on how the program went.
    Note that this post deals mostly with high-level reflections and isn’t maximally comprehensive. It primarily focusses on information we think might be relevant for other people and initiatives in this space. We also do not go into specific research outputs produced by fellows within the scope of this post. Further, there are some details that we may not cover in this post for privacy reasons.
    How to navigate this post:
    If you know what PIBBSS is and want to directly jump to our reflections, go to sections "Overview of main updates" and "Main successes and failures".
    If you want a bit more context first, check out "About PIBBSS" for a brief description of PIBBSS’s overall mission; and "Some key facts about the fellowship", if you want a quick overview of the program design.
    The appendix contains a more detailed discussion of the portfolio of research projects hosted by the fellowship program (appendix 1), and a summary of the research retreats (appendix 2).
    About PIBBSS
    PIBBSS (Principles of Intelligent Behavior in Biological and Social Systems) aims to facilitate research studying parallels between intelligent behavior in natural and artificial systems, and to leverage these insights towards the goal of building safe and aligned AI.
    To this purpose, we organized a 3-month Summer Research Fellowship bringing together scholars with graduate-level research experience (or equivalent) from a wide range of relevant disciplines to work on research projects under the mentorship of experienced AI alignment researchers. The disciplines of interest included fields as diverse as the brain sciences; evolutionary biology, systems biology and ecology; statistical mechanics and complex systems studies; economic, legal and political theory; philosophy of science; and more.
    This approach broadly -- the PIBBSS bet -- is something we think is a valuable frontier for expanding the scientific and philosophical enquiry on AI risk and the alignment problem. In particular, this aspires to bring in more empirical and conceptual grounding to thinking about advanced AI systems. It can do so by drawing on understanding that different disciplines already possess about intelligent and complex behavior, while also remaining vigilant about the disanalogies that might exist between natural systems and candidate AI designs.
    Furthermore, bringing diverse epistemic competencies to bear upon the problem also puts us in a better position to identify neglected challenges and opportunities in alignment research. While we certainly recognize that familiarity with ML research is an important part of being able to make significant progress in the field, we also think that familiarity with a large variety of intelligent systems and models of intelligent behavior constitutes an underserved epistemic resource. It can provide novel research surface area, help assess current research frontiers, de- (and re-)construct the AI risk problem, help conceive of novel alternatives in the design space, etc.
    This makes interdisciplinary and transdisciplinary research endeavors valuable, especially given how they otherwise are likely to be neglected due to inferential and disciplinary distances. That said, we are skeptical of “interdisciplinary for the sake of it”, but consider it exciting insofar it explores specific research bets or has specific generative motivations for why X is interesting.
    For more information on PIBBSS, see this introduction post, this discussion of the epistemic bet, our research map (currentl...
    35 min
  • LW - Formalization as suspension of intuition by adamShimi
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Formalization as suspension of intuition, published by adamShimi on December 11, 2022 on LessWrong.
    While reading Bachelard (one of the greatest philosophers of science of all time), I fell upon this fascinating passage:1
    From now on an axiomatic accompanies the scientific process. We have written the accompaniment after the melody, yet the mathematician plays with both hands. And it’s a completely new way of playing; it requires multiple plans of consciousness, a subconscious affected yet acting. It is far too simple to constantly repeat that the mathematician doesn’t know what he manipulates; actually, he pretends not to know; he must manipulate the objects as if he didn’t know them; he represses his intuition; he sublimates his experience.
    Le nouvel esprit scientifique p 52, 1934, Gaston Bachelard
    Here Bachelard is analyzing the development of non-euclidean geometries. His point is that the biggest hurdle to discover these new geometries was psychological: euclidean geometry is such a natural fit with our immediate experience that we intuitively give it the essence of geometry. It’s no more a tool or a concept engineered for practical applications, but a real ontological property of the physical world.
    Faced with such a psychologically entrenched concept, what recourse do we have? Formalization, answers Bachelard.
    For formalization explicitly refuses to acknowledge our intuitions of things, the rich experience we always integrate into our concepts. Formalization and axiomatization play a key role here not because we don’t know what our concepts mean, but because we know it too well.
    It’s formalization as suspension of intuition.
    What this suspension gives us is a place to explore the underlying relationships and properties without the tyranny of immediate experience. Thus delivered from the “obvious”, we can unearth new patterns and structures that in turn alter our intuitions themselves!
    A bit earlier in the book, Bachelard presents this process more concretely, highlighting the difference between Lobatchevsky’s exploration of non-euclidean geometries and proofs by contradictions:
    Indeed, we not only realize that no contradiction emerges, but we even quickly feel in front of an open deduction. Whereas a problem attacked through proof by contradiction moves quickly towards a contradiction where the absurd comes about, the deductive creation of the Lobatchevskian dialectic anchors itself more and more concretely in the reader’s mind.
    Le nouvel esprit scientifique p 46-47, 1934, Gaston Bachelard
    Of course, formalization has many other uses. But I still find this Bachelardian function enlightening. It not only points at the constructivist nature of our models of the world; it also gives us a concrete tool to realize the perpetual rectification which Bachelard sees as the core process of scientific discovery and progress.
    This post is part of the work done at Conjecture.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    3 min
  • AF - Reflections on the PIBBSS Fellowship 2022 by Nora Ammann
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Reflections on the PIBBSS Fellowship 2022, published by Nora Ammann on December 11, 2022 on The AI Alignment Forum.
    Last summer, we ran the first iteration of the PIBBSS Summer Research Fellowship. In this post, we share some reflections on how the program went.
    Note that this post deals mostly with high-level reflections and isn’t maximally comprehensive. It primarily focusses on information we think might be relevant for other people and initiatives in this space. We also do not discuss specific research outputs produced by fellows within the scope of this post. Further, there are some details that we may not covered in this post for privacy reasons.
    How to navigate this post:
    If you know what PIBBSS is and want to directly jump to our reflections, go to sections "Overview of main updates" and "Main successes and failures".
    If you want a bit more context first, check out "About PIBBSS" for a brief description of PIBBSS’s overall mission; and "Some key facts about the fellowship", if you want a quick overview of the program design.
    The appendix contains a more detailed discussion of the portfolio of research projects hosted by the fellowship program (appendix 1), and a summary of the research retreats (appendix 2).
    About PIBBSS
    PIBBSS (Principles of Intelligent Behavior in Biological and Social Systems) aims to facilitate research studying parallels between intelligent behavior in natural and artificial systems, and to leverage these insights towards the goal of building safe and aligned AI.
    To this purpose, we organized a 3-month Summer Research Fellowship bringing together scholars with graduate-level research experience (or equivalent) from a wide range of relevant disciplines to work on research projects under the mentorship of experienced AI alignment researchers. The disciplines of interest included fields as diverse as the brain sciences; evolutionary biology, systems biology and ecology; statistical mechanics and complex systems studies; economic, legal and political theory; philosophy of science; and more.
    This approach broadly -- the PIBBSS bet -- is something we think is a valuable frontier for expanding the scientific and philosophical enquiry on AI risk and the alignment problem. In particular, this aspires to bring in more empirical and conceptual grounding to thinking about advanced AI systems. It can do so by drawing on understanding that different disciplines already possess about intelligent and complex behavior, while also remaining vigilant about the disanalogies that might exist between natural systems and candidate AI designs.
    Furthermore, bringing diverse epistemic competencies to bear upon the problem also puts us in a better position to identify neglected challenges and opportunities in alignment research. While we certainly recognize that familiarity with ML research is an important part of being able to make significant progress in the field, we also think that familiarity with a large variety of intelligent systems and models of intelligent behavior constitutes an underserved epistemic resource. It can provide novel research surface area, help assess current research frontiers, de- (and re-)construct the AI risk problem, help conceive of novel alternatives in the design space, etc.
    This makes interdisciplinary and transdisciplinary research endeavors valuable, especially given how they otherwise are likely to be neglected due to inferential and disciplinary distances. That said, we are skeptical of “interdisciplinary for the sake of it”, but consider it exciting insofar it explores specific research bets or has specific generative motivations for why X is interesting.
    For more information on PIBBSS, see this introduction post, this discussion of the epistemic bet, our research map (currently undergoing a significant update, to be released...
    35 min
  • EA - Kurzgesagt's most recent video promoting the introducing of wild life to other planets is unethical and irresponsible by David van Beveren
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Kurzgesagt's most recent video promoting the introducing of wild life to other planets is unethical and irresponsible, published by David van Beveren on December 11, 2022 on The Effective Altruism Forum.
    Kurzgesagt, in their recent video "How To Terraform Mars - WITH LASERS" (just came out a few hours ago as of writing this post), promotes the idea of seeding wildlife on other planets without considering the immense suffering it would cause for the animals on it. Instead of putting thought into the ethical implications of these actions, the video (as par for the course) focuses solely on the potential benefits for humans.
    Sadly this problem isn't an isolated incident either, the pattern of ignoring the real risk for immense wild animal suffering is common in almost all major plans and discussions involving the terraforming of planets or space colonisation.
    Sure, a new green planet with lots of nature sounds cool in theory, but it would very likely mean subjecting countless animals to a lifetime of suffering. These animals would be forced to adapt to potentially hostile and unfamiliar environments, and face countless challenges without any choice in the matter. There's no way around it that I can see.
    You might argue that in these proposed worlds, we'd create an environment for wild animals where there wouldn't be food scarcity, predators, disease, or even anthropogenic harms. Setting aside the immense improbability of such an world (imagine convincing a rhinoceros not to fight to the death for their territory against a wild boar or elephant), none of the terraformed videos or articles I've read have even hinted at wild animal suffering as a potential issue to be concerned about.
    Also setting aside the conversation of whether or not we should extend human life into other planets and galaxies (for those who don't particularly follow longtermism, or the staunch antinatalists that might be reading this), wouldn't we be far better off just seeding these terraformed planets with plant life instead?
    If the key decision makers of the future decide they have to bring animals to other planets (and we can't convince them otherwise), then just introducing herbivores would be preferred, at the very least. I'd still be staunchly against this unless we could somehow guarantee that the lives of every individual animal would be net-positive, but sadly— we're not even close to getting people to include this kind of consideration into these types of conversations. At least, not that I know of.
    Don't get me wrong, Kurzgesagt has always been one of my favorite educational channels to watch. I'll continue to stay subscribed because I think they spread a lot of good, but their promotion of seeding wild life on other planets, without any consideration of the consequences, is unethical, and irresponsible.
    Instead of blindly pursuing our own interests and trying to populate every inch of the galaxy with life, we should consider the impact of our actions on other future beings and strive to minimize suffering whenever possible— or in this case, preventing it from happening at all.
    Thanks for reading.
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    3 min
  • LW - [ASoT] Natural abstractions and AlphaZero by Ulisse Mini
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: [ASoT] Natural abstractions and AlphaZero, published by Ulisse Mini on December 10, 2022 on LessWrong.
    I just read Acquisition of Chess Knowledge in AlphaZero and it's both really cool and has interesting implications for the Natural Abstractions Hypothesis. AZ was trained with no human data and yet it settles on relatively interpretable abstractions for chess.
    They trained sparse linear probes on hidden layers to predict Stockfish evaluation features. Here are some of the graphs
    Analyzing the largest differences between the predicted Stockfish score total_t_ph and the actual. There's a clustering effect due to the hardcoded Stockfish eval not taking into account "piece captures in one move" (as usually, the search would handle) - leading to large disagreements on value when a Queen is hanging.
    Although the degree of structure shown here is surprising, this example is not cherry-picked; it was the first concept/layer/checkpoint combination we tried, and the positions presented are simply those whose residuals are past a cutoff that.
    Not cherry-picked!
    Training a linear regression model to predict the value function, given human interpretable features like piece value weights and Stockfish features like king safety: results in approximately recovering the chess 9-5-3-3-1 piece values!
    They also did various experiments with unsupervised probing, here's a visualization of some of those features. (I didn't look into these results as much as the others, so not much to say here.)
    In conclusion: I'd recommend checking out the paper, or any from Neel's list. Interpretability is fascinating!
    This isn't that surprising, chess is somewhat of a toy example after all. The concrete experiments still provide bits though!
    Thanks for listening. To help us out with The Nonlinear Library or to learn more, please visit nonlinear.org.
    2 min
  • LW - Finite Factored Sets in Pictures by Magdalena Wache
    Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Finite Factored Sets in Pictures, published by Magdalena Wache on December 11, 2022 on LessWrong.
    Finite factored sets are a new paradigm for talking about causality. You can use them to do some cool things you can’t do with Pearl’s causal graphs, for example inferring a causal arrow between two binary variables.
    Also, finite factored sets are a really neat mathematical structure: they are a way of taking a set and expressing it as a product of some factors. Set factorizations are analogous to integer factorizations, in the same way that set partitions are analogous to integer partitions.
    So, here is my current understanding of finite factored sets, in pictures.
    1. What are Set Factorizations?
    What do these “factored sets” look like? Let’s start with a set S and factor it.
    The first concept we need is a partition of a set S. A partition is a way of chopping up S into subsets (called parts). Here are a few examples of partitions:
    We usually call the partitions X,Y,Z,U,V, or W, and their parts xi,yi,. like this:
    U is called the trivial partition. It only has one part.
    We can think of partitions as properties, or variables over our set. For example, consider a set like this:
    and compare it to the partitions X,Y and Z from above:
    Then
    The partition X is the property “color”, with x1 = blue and x2 = orange.
    The partition Y is the property “form” with y1 = square, and y2 = circle.
    The partition Z is the property “filled” with z1 = yes, and z2 = no.
    Exercise
    Consider these two partitions X and Y on the set S. What would it look like to represent them as properties (e.g. X = shape, Y = color) instead?
    Spoiler space
    It could look something like this:
    Here X = shape = {x1,x2,x3} = {star, circle, square}, and Y = color ={y1,y2} = {green, orange}.
    I hope you can see how partitions and properties are basically the same thing. In the rest of this post, I will use “partitions” and “properties” interchangeably. Sometimes I will use the ring-style visualization of partitions, and sometimes the property style, depending on what I find more intuitive in any given example.
    Now we can define set factorizations:
    A factorization B of our set S looks like this:
    A factorization is a set B={b1,b2,.,bn} of partitions (called factors). In this case B={b1,b2}={X,Y}={{x1,x2},{y1,y2}}.
    But it can’t just be any set of partitions. In the following sections, I will explain the two conditions that B needs to fulfill in order to count as a factorization:
    There is a unique element for all combinations of properties
    No factor is trivial
    1. There is a unique element for all combinations of properties
    Let’s look at our partitions in terms of properties again:
    What we need in order for B to be a factorization, is that for all combinations(xi,yj) of properties (for example (x1,y2), which is (blue, circle)), there is a unique element with these properties.
    We can see that this is the case here: We have exactly one blue square, exactly one orange square, exactly one blue circle, and exactly one orange circle.
    To express it more mathematically: For B={X1,X2,.,Xn} to be a factorization, we need that for all x1∈X1,x2∈X2,.xn∈Xn, it holds that the intersection of {x1,x2,.,xn} contains exactly one element. This means the cartesian product of our factors is bijective to the set S, which justifies that we say we can “express S as the product of our factors”.
    2. No factor is trivial
    Here is an example of a non-factorization:
    B = {U, V} is not a factorization here, because U is trivial and factors aren’t allowed to be trivial.
    This is analogous to integer factorization, where we don’t count 1 as a factor. For example, for the integer 6 we say the factorizations are {6} and {2,3}, and don’t mention {6,1} and {6,1,1} and {6,1,1,1} and so on.
    Exercise
    What about this? Is B = {X, Y} a factorization here? (take a moment to t...
    22 min