Academic research is under serious fire right now. The suspects fueling a replication crisis include the peer-review system, academic journals, and the system of evaluating faculty for tenure. The questions are also not new. The challenges are structural, baked into the underlying incentives. There are no easy answers it seems to the challenges.

My guest for this episode is Mathïs Fédérico, founder of the startup company Bycelium, which aims to rethink science with Bayesianism. Mathïs shared his personal journey through the traditional research pipeline and explained how the emphasis on publication count and narrative crafting distorts scientific progress. Our conversation explored Bycelium’s approach to measuring the credibility and impact of scientific hypotheses in real time by incentivizing the sharing of data and negative results and encouraging honest debate rather than just novel publications.

As Mathis told me, “Science is never perfect. Science will never tell you that something is true or false. Science will just nudge the credibility of things thanks to evidence.”

Will Bycellium work? It’s too early to say, but I find the ideas behind it intriguing and illuminating.

Challenges in academic research trends

Michael Horn

Alrighty. Welcome to the Future of Education. I’m delighted because several months ago Mathïs Fédérico reached out to me somewhat on a whim, I think because he had seen something I had posted about the research challenges in academia and higher education. And my hypothesizing that actually, you know, a lot of my research and writing is about how we need to reinvent the teaching and learning model itself is broken. And I said, you know, there’s this whole other thing that’s also broken, which is the research model itself. And then since then he reached out and we’ll talk about why he did in a moment. But one of the things that’s happened since then is a lot more people are very dialed into the challenges that the research process has. We’ve had this Nature article coming out that said, you know, 3,900 studies published in 62 journals and half of them could not be reproduced.

We’ve seen a lot of people realize, hey, actually a lot of the Nobel Prize winners are not coming from traditional higher education pathways. As of late. We have DeepMind, Google, pharma companies, a lot of researchers that are the most impressive breakthroughs aren’t coming from the universities we expect to produce the research. We have a lot of claims of not just reproducibility challenges or replication, but outright falsifying of research and the like. And then you have this other backdrop, which we know well, is that people, when they publish in research journals, increasingly to get published, not only do you have to have something that looks statistically interesting, you also have to have something that is unique. So by definition almost not replication. And that has caused many faculty to look at narrower and narrower and narrower questions that have less and less relevance to other people. And as a result I don’t have the stats in front of me.

But very few peer reviewed journals, articles that get published are ever, ever read. Very few get more than say two or three citations ever out there. And so you have a bunch of challenges hitting all at once. And so my guest Mathïs Fédérico, he has started an entity called Biselium, which we’re going to talk more about to actually solve the root causes of this challenge. But first, Mathïs, welcome. Thanks for reaching out. I’ve been like, we had this conversation on the phone and I’ve been buzzing ever since and saying, I think we have someone who might have an answer actually. So first, welcome and thanks for being here.

Mathïs Fédérico

Well, thanks for having me. I’m very glad to be here, I guess. Yeah. All that you’ve said, which seems very diverse in a way, and seems like a lot of things are happening independently, but all linked to the same source, which is, we are not pushing for the right thing. The thing that academia is optimizing for is not the good objective. And so whenever you try to maximize an objective like this, well, what’s happening is you maximize the wrong thing. You have a lot of the wrong things. And for us the wrong thing is the number of publications.

That’s what we maximize today. The number of publications is what we use to evaluate. It’s what we use to quantify breakthroughs, to quantify how good a researcher is. But it’s not what matters. It’s not what matters at all. What does matter though, is how much did you change the minds of others, what are the evidence that were provided and how much did you change what we think of reality, what is credible and what is not? And so that’s exactly what kind of data we want to measure now with biselium, the general idea is to say, okay, could we reimagine a system where what we measure is not publication and citations, but what’s the credibility of things, how credible are hypotheses? So if we could measure this in real time and know that, oh this, this changed my mind and like this evidence changed my mind, then you literally have the impact, the scientific impact that you had on others minds. And so it has a lot of ripple effects around this.

And so that’s the main idea.

Michael Horn

Yeah, well, I want to get into all of them because. So let’s talk about your personal story before we even get into what you’ve been building and the root cause, because I think you just actually nailed it, that this emphasis on publication number for its own sake has sort of become its own ill, if you will. It’s the law of unintended consequences. Right. When you pick one metric that is not the actual metric you care about, but a proxy for it becomes a bad proxy pretty quickly, Campbell’s Law. But you had a personal set of interactions in your own educational journey that sort of laid this bare where you were doing research, as I understand it, in a lab. So maybe talk about that origin just to ground us about your own personal journey to this.

Mathïs Fédérico

So my journey started as I want to pursue science. So I discovered science in higher education in France, I discovered like preparatory classes, how you can reason. So the thing that fascinated me with science is that not only you can prove others wrong, but you can prove yourself wrong with tools.

Michael Horn

And that’s actually a good thing.

Mathïs Fédérico

Yeah, yes. Because that’s how you learn, right? That’s how you progress. So I love this. And so very quickly, when I was 16, I was extremely passionate about epistemology, which is, how do you like, what’s the science of learning? What’s the science of us discovering knowledge altogether? Right. And so being into this, I continued my studies, went into engineering school at CentraleSupélec in France. But after that I reached research and my goal was like most young people that go in the system. I want to do a PhD, I want to pursue research, I want to be a researcher, I want to push the boundaries of knowledge. That’s the classical route you’re told is the good route to do science. Right.

But then when I tried to do research at some point I actually had the advice being given to me that in order to have a scientific career, I had to spend less time on my experiments so I could write. And I should spend time on writing a nice story about it so I could get published in high chair journals so I could get cited because this is how you make a scientific career. And I refused it. I refused it because for me this was not a good way to do science. So I looked for other ways. And so this led me to entrepreneurship, this led me to other paths, even like research and development in industry, hoping that it would be different, but it’s not.

Michael Horn

Yeah. And I want to, I want to come there in a second because there’s something else when we talked that you said was, if I’m remembering correctly, which was not only the focus on crafting the narrative, right. Sell it in essence, but you actually were very interested in like, how do I make my data vary. So others can understand it and others can look at it and really present it neatly. And like you were very obsessed with that and they said, why are you doing that? Stop doing that. Say a little bit more about that. Yeah.

Mathïs Fédérico

It’s not just making it understandable. So I guess I was leaning into software engineering a bit already at the time. And I just wanted to make my code to make my experiment very reusable. I wanted what I was doing to be packaged in a way that anyone could build on top of it very quickly. So not only they could understand it, but they could tinker around with it and they could build on top of it. So that was one of my main interests. And so what people told me, as you said, is stop spending time on doing that because you don’t really care, what matters is that you have the paper, you ship the code with it, but people that will build on top, that’s a problem. Right.

And so this was crazy to me that you have to optimize for this short term paper publication instead of the long term science for it. That’s obviously that target for me. But somehow this is how the system works and this is what the system pushes you to do.

Michael Horn

Right, and so let’s dig in then. So then you said you started to look toward entrepreneurship. You sort of rejected this system. Tell us about your pathway then, because it starts to wind into Bycelium, but maybe before that, like what are the root causes you discovered that are distorting this system where someone would say to you, ah, you know, the data, whatever, making it accessible and whatever, like, you know, just focus on selling it. We just need you to get published regardless of the truth underlying it, regardless of how much we-

Mathïs Fédérico

I will make the argument against it. Yeah, I would make the argument against the fact that no, no one will push you to ignore the truth. That’s not the case. Right. It’s more subtle than this. It’s really more subtle and that’s why it would be much more visible if it was as bland as this and people just pushing you not to.

Michael Horn

So then we’d all see it and do something about it.

Mathïs Fédérico

Exactly. So it’s more subtle than this. Right. It’s a few things here and there that tell you, oh, well, you know, spend less time on this, spend more time on this. It’s like this small orientation and these small pushes that you have in the direction of writing and being a good writer of science and being able to tell this nice story about the ideas and basically trying to convince the reviewers more than trying to convince yourself at some point. And so that’s, I think, what I didn’t like at all within that system. And that triggered some alarms in my instinct that was telling me, well, I don’t want to be part of that system because of those small signals that were accumulating. So it’s much more subtle than someone telling you you should lie.

That never happens. It’s really more, there’s the ground coming up and we have to have released some papers, we have to have done some public. So let’s rush for the deadline, let’s publish. Oh, we have the conference deadline coming up, so let’s get some results quickly before the conference so we can publish something. It’s more like those kinds of things which are not bad and wrong in a way it doesn’t seem like it, but it accumulates. And at the end of the day you’re not happy about what you’ve published.

And so I think that’s one of the most important signals that we have is, that imagine for authors of books, actual books, and you’re one yourself, right?

Michael Horn

Yeah. For better or worse.

Mathïs Fédérico

So imagine that we would force you to publish fast and so you wouldn’t be proud. So as much that you wouldn’t be proud of the thing that you publish when you do, because they have been rushed and they have been pushed because they needed to be there. And there is studies that shows that it’s very little scientists that actually are proud of the thing that they have published. So it could be also that maybe we want to publish the perfect thing and so on. And that’s also a bad thing. And so that’s why I’m arguing against publication oriented science altogether, because I think it’s a very bad practice. Science is not and should not be about publication at all. Publication is something that happens always, all the time.

We are in the 21st century. We are not writing things in journal and communicating by letters anymore. The information is everywhere and we should share it as much as it’s produced. For me, the freedom of science that we could have back is that you’re just doing things, you’re doing your experiments and you’re publishing as they go, as they are done. Because you don’t need to have the approval of people to see that something is relevant or not. It comes afterwards anyway.

For me, the problem that we have with the current scientific system is that we mixed publication and evaluation. They are not the same thing at all. And they should not be mixed. Evaluation, which is how much impact will have, is it true or does it hold or does it not hold? That’s something that evaluates over time after it has been shared. Because as you mentioned, it needs replication, it needs to be checked in other contexts, it needs all of this. And if you don’t have this, you don’t have this before publication. So when do we have it right? So how could you try to do this filter before publication? For me, it makes no sense. And so it’s not necessary and it’s not good to have this filter of before we publish.

We should try to make sure it’s perfect or it’s good. Science is never perfect. Science will never tell you that something is true or false. Science will just nudge the credibility of things thanks to evidence. Evidence will always nudge and nudge the credibility until the credibility is so far on one side or another that it’s nearly certain. But it’s never completely certain. Because if it were completely certain, it also would mean that any data that you would gain in the future would never change your mind, which is weird. That’s not really a very scientific mindset to have a mindset where anything in the future would not change your mind.

So, yeah, I guess that’s what.

Michael Horn

That’s an important insight, right, that you just landed on, which is my understanding and you can correct me, but is that sometimes people will exclude certain data points because it does not contribute to the narrative or it looks like unclean data or things of that nature.

Building a new research system

Michael Horn

Right as opposed to saying, wow, that’s an anomaly. I don’t know how to explain it. Let me just sort of be honest with that and then let’s see what that means in terms of where the preponderance of evidence falls. Or maybe there’s a different circumstance and under one this is true, and under another it’s not. Who knows? Let’s try to work this through. But I guess a lot of that thinking though is still under the notion of publication is the referee, if you will. Right. And you’re now creating Bycelium to say, actually there’s a different system altogether where other scientists can look at people’s research to sort of say, hey, do we believe this hypothesis? Do we not? Where’s the preponderance of evidence and belief at any given point? Talk about what you’re building.

Mathïs Fédérico

So it’s not just me, to be clear. It’s actually an ecosystem that’s starting to be built. So not my part and my job within that ecosystem is exactly the bit that you mentioned, which is for me, it’s about this idea of hypothesis credibility and attributing the evidence, attributing the change of credibility to some evidence. That’s the part I want to contribute with Bycelium. But that’s not like the only. We are not the only actors, actually. There is the Open Exchange Architecture, the Continuous Science Foundation that are like making things that are. On this idea of continuous science, we should publish things as we go.

And there are a lot of institutes that are existing. There is the Astera Institute, for example, in Arcadia Science. There is a lot of people that are trying to do other ways of doing science where there is a lot of more transparency and honesty and openness from the get go. So it’s not publication oriented. Publications are always happening in real time as you do experiments and you just share everything that you do when you do it. And afterwards you can look back and try to do a summary, try to do this kind of summary of concise journal, journal and concise bits of information where you summarize everything that has happened. But when you do the thing, you publish it on the go. And on the go, those little anomalies, sometimes they accumulate and they actually shine light on something deeper that has happened a lot already in history.

Those things we were considering anomalies became entire different fields. That’s the case of quantum physics. Quantum physics in the early of the 19th century was, you know, it was just when Planck was trying to work. I think there is this anecdote of his professor telling him that there’s nothing left to do in physics so he should choose another field. Right? Because there’s nothing to discover in physics. Right. And so he leaned into like a, Planck leaned into the Wien’s constant of like the black body radiations, right. And so these laws that were here and that has some discrete jumps. And so this is the, the thing that was the entrance and this little anomaly of this little weird data point and weird jumps are the thing that led afterwards to quantum physics.

Right. And so we have had anomalies like this that accumulate into very important things. So that’s exactly your point. The fact that us wanting to tell a perfect and nice narrative to make believe for the reviewers is, I think, detrimental to actual science and honesty and transparency about what’s happening and why, and also to the reviewers and also to everything. Because if you’re optimizing not for publication and the acceptance by others, but you’re optimizing instead for the long term impact, which is what we should do long term, it will change the mind of others on those subjects. Then it changes everything. So that’s exactly what we’re trying to orient with Bycelium. But Bycelium is really a simple idea actually, right.

Scoring and incentivizing contributions

Mathïs Fédérico

It’s really just let’s measure something that we don’t. So it’s just instead of doing this whole review process and the citations and using all of that, let’s just use instead, I changed my mind because of X. And so the question of course lies on if you have this data, how do you derive a score out of it? How do you score people depending on the contributions that they have made? And so how do you score them in a way that you drive the right incentives for them? So how can you establish a game if you want, in a way that maximizing the score of that game. So maximizing prestige if you want, right would lead you to do science, would lead you to be honest about the credibility of hypotheses that you’re putting on the like. So it’s very similar to predictability prediction markets in a way. Right.

Where if you have a prediction market, your internal belief is the best action that you can do on a prediction market. And so we kind of want to do exactly the same kind of mechanisms. Except unlike prediction markets, where you’re trying to predict an event so there will be a resolution at some point that tells you, oh, well, this was the truth. Right. For science, not really the same. It’s not like someone will tell you at some point, oh, the hypothesis was correct. No, nobody would know.

Michael Horn

An election doesn’t. Right. So just to make it clear to the audience, right. In a prediction market, there’s a game, there’s election, there’s some sort of thing, we all had made bets on it and now it’s proved true or false relative to how we bet. Science, this prediction market, if you will, will actually be ongoing for any item or piece of scientific question that we’re sort of, or hypothesis maybe is the better way to say about it that we’re, that we’re looking into. Is that the right way to think about it?

Mathïs Fédérico

Exactly. So it’s the idea that you have this never ending update of what do humanity think is credible? And so if you provide evidence that changes what humanity thinks is credible on how reality works, you’ve done science. That’s as simple as this, right. And so this is the simple equation. We’re trying to put the infrastructure, the platform and everything around and the framework and everything around for it. And we are trying to integrate, as I mentioned, through this new ecosystem of how to do this continuous science and how to do the science in this more transparent and better way and more, without this drawback of having journals and needing reviewers before because it seems weird, but for, for people, it seems that if two people have said like two reviewers have accepted a paper, that’s, it becomes science, you know, because before that it was not science and because it’s accepted by two people now it’s science.

Michael Horn

A silly low threshold there, if you will.

Mathïs Fédérico

Yes, it’s critical. And so I don’t think it’s a good truth now.

A walk-through example of Bycelium’s system

Michael Horn

Yeah, yeah. So, let’s dig in. Like take us through a micro example, right? Say Planck’s constant or Heisenberg or someone comes in today right. We don’t know, we, we don’t have the, that, you know, that they, that we now accumulated around these questions. How would this play out? Right, in Bycelium, what would. They would put forth a hypothesis, they would back it up with data. People would start to make bets of some sort, I guess, of whether they thought this made sense or not, or what would this actually look like on a micro example.

Mathïs Fédérico

Yeah. So let’s take an example like this. Do you want a concrete scientific example or do you.

Michael Horn

Sure, you get to choose. You get to choose.

Mathïs Fédérico

I guess let’s take the example of the Aharonov–Bohm hypothesis. It’s a very nice hypothesis in physics because it has spanned a lot of back and forth of people changing their mind over 30 years. So that’s why I like it. So let’s imagine that you’re Bohm or Aharonov at that time initially. So before the data, before the experiment, and you’re having this idea that actually when I’m looking at Schrodinger’s equation, so on the hypothesis itself, what it is, is to say that potentials can act on matter and can act on particles and not just fields, because usually in physics only forces can act on things. Only forces will change the behaviors of matter because you have to have a force somewhere. So it might be electromagnetic force, it might be gravity as a force, but there needs to be a field that contains that force. Right? And so what this hypothesis was that actually not always there is space, there is cases where just the potential can change the gravity.

So imagine, like if you take the physical analogy for this, imagine that if you were to be higher on top of a mountain, right, Just because you had a higher potential, then you would change, something would change in the way you fall because you’re higher. And so that’s something that’s very weird and very surprising in physics. And so everyone was against this when the hypothesis was of course initially published, right? So how it would play out in Bycelium would be that Aharonov and Bohm could open the hypothesis, which is an absurd hypothesis for most, and bet for it, right. And have this position where they say, actually we think it’s pretty credible they might not be like 100% certain. So they don’t, you can’t anyway say on Bycelium that you are 100% certain or 100 or 0%.

Michael Horn

But they put a percent confidence essentially with their claim, if you will.

Mathïs Fédérico

They might even be a conservative themselves and say, oh, maybe there’s like 60% chance or something. Like that it is, or maybe even 20% chance. Right. Maybe they say that there’s 20% chance that this is actually,

Michael Horn

I think we found some evidence that suggests this could be so we’re going to put forth the claim with our level of confidence against it.

Mathïs Fédérico

Okay? Exactly. And so they do that, which is we could be just, instead of being negligible, like 0.1% now it becomes like 1% or 2%. Right. So that could already be more than most scientists. So they do this, they do that bet if you want, and then they release evidence. And so that evidence might change the mind of others. So the others would initially bet, like in the negligible, like below 1%, below 0.1% Probability that this is true. And then maybe some would change their minds.

For example, Richard Feynman changed his mind very quickly when there was the first experimental experiments around it. Although there were imperfections in the system. Richard Feynman looked at the equation and said, at first it looks horrendous, but then it looks obvious that this is the case. And so he changed his mind completely. And he actually started to teach this at that time. And at the same time you had people that were completely against this and were saying, ah, but the experiments are flawed because there is some leakage of field. And so there is actually some existing field that exists. And so we would.

Michael Horn

So it doesn’t in fact change the principle.

Mathïs Fédérico

Yes, exactly. So it is evidence, but it’s not enough evidence. And so there are people that completely try to gather evidence to go against this. And so, you know, there was this battle, and so you could see on Bycelium, this battle, because people would change their mind and update their position depending on, oh, actually, this new thing that came out changed my mind. No, I’m thinking more this. Oh, no, I’m thinking, I’m thinking a bit more this. And you do as such as more as you, you gather evidence.

And the more you have evidence and the more you would gather while you would converge to the truth is what we expect of science. Right.

Michael Horn

You’re essentially incentivizing others then to do experiments, to replicate, to seek. Because they can change people’s mind as well.

Mathïs Fédérico

Exactly.

Michael Horn

Let me, let me ask this question. What’s the trigger that gets me to care about the question in the first place or to participate and say, I actually, you know, you put forth this crazy idea that I’d never thought about before because it seems so crazy now I’m looking, why do I even pay adherence to it and participate?

Mathïs Fédérico

I think that’s a very good question. I think the question of what makes an hypothesis relevant is actually a good question already today. What makes an hypothesis relevant for someone to explore it in the first place? And it’s very hard to know and to pinpoint exactly what would drive people. The easy answer to this. Let me start by this, probably, the easy answer to this. This is just because you are a domain expert and you want to prove that you are an expert in a domain. Whenever there is a hypothesis that pops in your domain, you try to gain prestige because if you’re right on that hypothesis, then you would get prestige on it.

So that’s the game, mechanical explanation.

Michael Horn

So in essence, I can actually get credit not just for, quote unquote, doing the experiment or presenting the data, but actually arbitrating it or weighing in on the discussion in some sense. So universities presumably could actually give credit for me being someone that is digging in on these important hypotheses.

Understanding foresight and hypotheses

Mathïs Fédérico

Okay, yeah, because you were right on how people were going to change their minds later on. And so you had like, you were right before the others, these kinds of things. So you had this good foresight on your own domain. So within a hypothesis of your domain, you’ve been able to foresight what was going to be the credibility of the hypothesis in the future. And so that’s kind of one of the easy incentives that you would have for an hypothesis. But yeah, as I mentioned, there is also a deeper question on what makes a good hypothesis. And actually there is a very nice suggestion of an economist that thought a lot about academia and how to change this, which is Robin Hanson. And he thought about this idea that I think is very relevant to the question here, which is we don’t know now what hypotheses are relevant.

Right. But we will know in 50 years or we will know in 100 years. Right. We will know in the future what were relevant hypotheses or not, right?

So what we could do is we could have a prediction market basically, or a future market more generally that will make us bet on what are the importance of those hypotheses. And so it would be our best estimations of that future, knowing that in 50 years we will ask some historians or scientists to look back and say what were important hypotheses. And so you could try to have this measure of the importance of hypothesis or even contributors themselves. Right. And try to have this like, oriented towards the future incentives system of incentives as well. So this is exactly the kind of subjects we are thinking about,

Michael Horn

Sort of two layers Then right one is, is it an interesting hypothesis? Do we want to engage into it? Because there’s lots of hypotheses that we could come up with. And then the second one, and you could imagine vectors, I guess, based on plausibility, impact, influence, et cetera. And then the second one is, is it true or false in our estimation or how likely. True or false is a little binary.

Mathïs Fédérico

I would prefer to say is it credible or not? Because credible or not. Because true or false is like zero or one.

Michael Horn

Thank you for correcting my language. Yeah, no, that’s better. Right, so is it credible or not as a hypothesis is another set of debates. And then I guess the next question is, right, how do we get people. You can imagine that there’s a sort of a piling in or lobbying external sort of set of incentives that could occur outside of the market.

Mathïs Fédérico

Yes, of course.

Michael Horn

How do you get it to impact present day behaviors the way we want it to, not just a hundred years from now. We figured out, wow, Mathïs was really onto something and we should have recognized it because that’s the thing. It has to really supplant or disrupt the publication peer review model to really, I would imagine, take off in some ways. So how does that process work in your mind? I know you’re building it right now so you don’t have to have every answer, but I’m sort of curious your current thinking.

Mathïs Fédérico

So there’s good news and bad news about this. So I’m going to start with the bad news. The bad news is that time is unforgiving and the way causality works prevents you from forecasting actually breakthroughs. By definition, breakthroughs are things you can’t see coming. So by definition you cannot say, oh, this hypothesis on frogs that no one cared about actually changed the course of humanity 100 years from now. That may be true, but you have no way to know it beforehand. Right. And so you don’t really have a way to forecast faster than the future comes.

So that’s the bad part and the sad part about this. So we have to have a system that does its best at, but it will never be able to have this like actual future impact, knowing this actual future,

Michael Horn

Maybe that’s, maybe that’s okay because you actually developed a real conversation. Right? Okay, so I’ll let you go. Go ahead, tell us.

Mathïs Fédérico

No, no, go on, go on.

Michael Horn

But that’s, I was going to say it occurs. It occurs to me though that that’s okay because we want to have robust discussions about what are the relevant hypotheses? And you could imagine two archetypes, right? One is we have a very robust set of debates around a limited set of hypotheses that seem really important and that solves the problem of us doing arcane things that aren’t relevant. And then you could imagine that there’s a couple rogue scientists, right? Or people in their field and they’re like, I am so convinced this is important. I don’t care what you all think about, I’m going to, over, you know, the course of my 50 year career, persuade you that this is a question of real import.

Right and maybe that’s actually healthy.

Challenges in scientific reproducibility

Mathïs Fédérico

And I think that’s healthy. I think it’s always what science has been. I think I started by this like science. I liked science because it allowed me to prove others wrong and myself wrong, but also others wrong.

And so proving others wrong is, I think, a feeling that is underrated. And it’s not a bad thing. It’s not a bad feeling to have. I think it’s actually quite healthy that you do your best to prove others wrong and you do your best to prove yourself wrong. As long as you’re not trying to prove anyone’s right, that’s fine. I would say, because otherwise you would fall into your own confirmation biases and so on, right? So as long as you’re trying to prove someone is wrong and you’re trying every way to prove that something is wrong, I think it’s a very healthy way to debate. And so that’s exactly what we’re trying to push for, which is not the case in the current journals where you can’t really say and publish negative results. Right? You can’t really say, oh, this actually doesn’t work.

And so that’s exactly what we want to have with Bycelium. We want to have a place like this.

Michael Horn

Well, that was going to be my next question. Right, so you’re now actually creating an incentive for people to publish sort of the null result, if you will.

Mathïs Fédérico

Exactly.

Michael Horn

Okay, say more about that because that seems like a very important piece that’s missing right now where you, you gave the example earlier. There’s the time pressure of grants, but there’s also the pressure of just like getting more grants. And I want to be in a lab seen as having lots of publications and proof points and positive results. Right. Because therefore my center is going to get more research grants than your center. And that’s its own problem. Right.

Mathïs Fédérico

And so for that you need to have high chair journals. And in order to have high chair journals, the number one criterion that they all have is novelty because they are literally journals in the sense of they want to be read by people. And so their incentive is to publish shiny good new things, not to publish the hard, oh, actually we were wrong or you know, that kind of thing.

Michael Horn

Well, so make this. I mean, I think this is clearest in social science research, even more so than. Right. Where you know, sort of the posturing and all these things like had magical effects. And then you look into it and it’s not quite as big as you first thought, but to your point, huge publication value to be out with that first right.

Mathïs Fédérico

Yes. And so there is that and there is a. But you’re mentioning social sciences. I would also say that even in condensed matter physics you have this problem. So it’s also a problem of hard, hard sciences too. And it’s. We are clearly not immune to that. We have the same system.

It’s quite universal. So it’s maybe more visible, I think, in social sciences because it’s maybe more approachable and more reproducible in some way than the condensed matter physics experiments that you need a lot more material to be able to try it again. Right. But it exists too and there is actually a very good documentary about this if you are interested online, you can look it up on condensed matter physics and the crisis of unpredictability within condensed matter physics. There’s also those problems and it’s not immune. And math is the same. Like there is a lot of wrong math proof out there. Yeah, yeah.

It’s not just social sciences, just to put the emphasis on it and. Yes. So to come back to the idea of having those incentives to push for negative results. That’s exactly what we want. We want to be able to have someone, to have a platform where if you push for negative results and you say actually this thing that people think is working, it doesn’t. And I’m going to show you, I’m going to show it. I want this to be evaluated. Right.

By the system. That’s one of the main interests of the system. And so the good thing for us is that, as I mentioned, what we want to reward is people changing their mind. But either way we don’t care if they change their mind positively or negatively. Right. On a hypothesis, as long as you change the mind of others, that’s valuable. That makes success. That’s matter.

Yeah, that’s what matters. Right. So for us, we don’t have a bias towards positive things. We actually have no bias in any direction as long as you do amplitude. And amplitude means you actually convinced others with strong arguments. It could be just arguments. Even a position proper like the words themselves can be considered as evidence. If they change your mind on how to view something, that’s fine.

It’s a lesser proof, but it’s still a proof. It’s still proof of something that could work and something that is becoming more credible in your mind. And that’s fine. It doesn’t have to be always data based.

Michael Horn

That’s good, right? Because you create room for theoretical and experimental both in this system. I was going to ask a curiosity question which is, is there a danger that if people get really confident about a particular hypothesis, people don’t want to weigh in because they’re like, yeah, that’s probably true and I won’t be able to change someone’s mind. How do you get people to still put sort of their whole bets forward? Yeah, talk about.

Mathïs Fédérico

The whole idea of designing the game around it. Because that’s why it’s so interesting for us to be able to design the algorithm that will link those input data to the prestige. Because now the question is, how do you make this system so that it has the properties that you just said, that the more people are agreeing on something that is wrong and the higher you will gain if you make them change their mind with strong evidence. And so it just means that you might have to look for stronger evidence than if they were uncertain. Of course. But the good thing is currently in our current first design of the system, the faster you move your mind compared to others and the best it will be for you. So if you think one evidence has came up, and it will change the mind of everyone, you are incentivized to react as fast as possible.

Because if you are the first one to move, you will benefit for all the others that will move after you on your position. Right. Or towards your position.

Michael Horn

So you get more currency, in other words, for being okay, gotcha,

Mathïs Fédérico

or more prestige or. Yeah, yeah.

Michael Horn

And so that currency is the prestige. Yeah, okay.

Betting against popular market beliefs

Mathïs Fédérico

Yeah, yeah. So that’s exactly the idea that if you move early and then so having a lot of people at the same place on the market or on the hypothesis more. If you have a lot of people at the same place on the hypothesis very positively and you have a strong evidence, something that you think is a strong evidence towards the negatives, then you are very incentivized to bet yourself the other way or where you think you will convince others and then reveal your evidence, and then reveal your evidence and try to convince the others your way. Because if you do so, you gain twofold once because your position will gain in value if people go towards your position. So you will have the first forecasting prestige that we talked about. Right. And on top of that, you will also have the contribution prestige because you provided evidence that changed the mind of others. So, yeah, that’s exactly the kind of.

That’s exactly what we’re trying to see is how can we design a system that have those kind of properties that we want?

Michael Horn

Okay, so let’s say there’s a hypothesis out there. There’s a fair amount of consensus. I want to jump in there and disprove it or, you know, try to convince other people, right. That they should switch their minds. I go through it and I’m like, oh, man. I think the preponderance of evidence suggests that I probably would put the bet that confirms the hypothesis. What’s the incentive for me to make that bet and sort of increase, you know, just be one more voice, if you will, on the bandwagon.

Who’s agreeing now with. With the prevalent view. If. Does that make sense?

Mathïs Fédérico

Yeah, yeah. So one of the ways we’ve done this for now, so this is a more technical question on how do we measure things?

Michael Horn

Yeah, yeah, yeah, yeah.

Mathïs Fédérico

One of the ways we have done that is that whenever you arrive at a position, the people already there will gain from your arrival. So, for example, if you can precisely predict where people will enter the market, you are actually gaining on that. So it looks a lot like a Ponzi scheme, actually, but it’s a Ponzi scheme that will shift on evidence. So it uses kind of the mechanisms and incentives mechanism of a Ponzi scheme in some way, which is fun because the Ponzi scheme itself, of course, is not a good thing. But I guess if you can use a Ponzi scheme to find science and to actually change their mind, to actually change the way we view the scientific incentives, that’s fine by me.

Michael Horn

I was going to say you’ll have made two contributions. One, to improve science and second, to show the one useful place for a Ponzi scheme. I suppose if you.

Mathïs Fédérico

Yeah, I mean, that’s fine by me. Right. If that’s the case. And so it’s a bit like a Ponzi scheme in the sense of if you’re early on a place that you think a lot of people will agree with you, then you will gain from that.

Michael Horn

Okay, last question as we start to wrap up here, which is talk about the entrepreneurial journey. Where are you in developing Bycelium.? I know you’ve been in some startup camps of some sort, incubators of some sort, I don’t know exactly what, but trying to get it out there. So talk to us about what that process is like. Where are you in development? When might people start being able to use this in your best imagination?

Mathïs Fédérico

Yeah. Okay. So for now we have made a lot of focus on building initial draft and initial platform and trying to partner up with foundations because we think that’s where it will start. Like the actual prestige comes also from the money. And so people that are giving you money. So funders of science, I think, are one of the most interested in having the science for their money. Right. And so that’s where we’re starting right now.

What’s next with Bycelium

Mathïs Fédérico

And so our main interest for us is to be doing pilots with foundations right now. So that’s what we are doing, like small control pilots where we have those kinds of competitions where we make scientists try to maximize their prestige. And by doing so we test those incentive designs and it allows us to have those testing bears and those experiments to see do we actually drive the right incentive, do we have the good metrics? Should we tinker the system one way or another? So that’s what we are doing at our stage right now. But of course the goal is then to open up and have as much and to grow the size of each of the experiments in order to afterwards be able to open it worldwide and have something that everyone can hop in. You mentioned the startup program. Yeah, I’m part of freight right now. So actually in Helsinki we are having a three months condensed work on the subject. So this allows me to build the initial platform, all the initial technical details, the first model that we just talked about a bit, which is how exactly the system and the instances.

Because before the streamers it was just an idea, now it works in practice. And so, yeah, we’re getting there. And so things are building, things are getting there. And so now what we need is mostly, well, people to join us. So smart people that are able to build fast and good and then the scientific funders that are willing to take the adventure with us and see how we can measure science and how we can measure the impact of science in a different way. And so our hope is that we will gain more and more credibility ourselves so we can probably have an hypothesis on the credibility of our own system.

Michael Horn

Well, I was about to say. Yeah, yeah, yeah, yeah. If the hypothesis works for it, we’ll just see what the revised hypothesis is if it doesn’t work out. But that’s the call to action is those who want to see better science and are funders or connected to funders, get in touch. How should they get in touch with you? How should they learn more about Bycelium?

Mathïs Fédérico

They can go on the website. They can actually book a meeting with us on the website directly. They can contact us by contact Bycelium..com for the email. And so they can. Yeah, there are a lot of ways to contact us, mostly directly, like via website or by direct email. That’s fine.

Michael Horn

Well, the tagline I think that you’re using is rethinking science at Bycelium.. So Mathïs Fédérico, everyone get in touch. Let’s work on this. Because this is another part of higher education, a part of our society, a part of progress that needs a desperate injection, I think of reinvention compared to where we’ve been. So I really appreciate the work you’re doing and hopefully you’ll come back, couple hypotheses down the road or as it starts to play out in the real world and give us an update. I really appreciate it.

Mathïs Fédérico

Thank you so much for the invite, Michael. It’s been a pleasure to be here today and talk about this. We have been a lot of time spending on our computer by ourselves. It’s nice to get the word out there and to have some feedback. So if you have feedback or just if you want to support us, you can also follow us on LinkedIn and try to reach out to just say that you like the initiative. That would also make our day much happier. So, yeah

Michael Horn

Good call to actions. Yeah. So everyone check it out. We’ll be back next time on the Future of Education.

The Future of Education is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit michaelbhorn.substack.com/subscribe