Discuss this episode in the Muse community
Follow @MuseAppHQ on Twitter
Show notes
00:00:00 - Speaker 1: Infinite canvases are essentially like a different
document format. The screen represents a camera that’s floating above a
surface, and there are things on that surface, and those things can be
anything. You can move them, you can duplicate them or resize them, and
that each one of these types of thing on the canvas also has its own
rules about how it can be changed.
00:00:26 - Speaker 2: Hello and welcome to Meta Muse. Us as a tool for
deep work on iPad and Mac, but this podcast isn’t about muse product,
it’s about the small team and the big ideas behind it. I’m Adam
Wiggins here with my colleague Mark McCrannigan. Hey, Adam. And our
guest today is Steve Ruiz of TL Draw. Hello. And Steve, I understand
you’re a little bit ahead of me on the fatherhood journey. You’ve got
a 315 year old. What do I have to look forward to in the coming couple
00:00:55 - Speaker 1: Oh man, a kid is what, like 1.5?
00:00:58 - Speaker 2: That’s right.
00:00:59 - Speaker 1: Eventually they will start drawing, they’ll start
playing with words, and they will grow ever more interested in iPads,
and also, everything will be an iPad, every computer, every device,
everything with the screen will be an iPad. That’s my prediction, yep.
00:01:17 - Speaker 2: Yeah, that makes sense. Yeah, already is the case
that trying to keep screen time limited is a challenge. Now, of course,
we can also look forward to having built in beta testers for our
software. You got a chance to do that yet?
00:01:29 - Speaker 1: Yeah, actually, my daughter has probably spent
more time with tealra than anyone else, and I’ve learned quite a lot by
watching her kind of poke around and try and draw, try and make
different things. She really likes arrows, but the touch targets are too
small and mobile. That’s one thing that I need to work on.
00:01:46 - Speaker 2: I’m reminded of a scene in one of these Steve
Jobs biography movies where I believe it’s his daughter comes in and
tries out Mac Paint, and they sort of show this maybe hypothetical idea
that he was sort of inspired to make software and a computer generally
that was easy enough that a young person could be creative with it.
00:02:04 - Speaker 1: Yeah, I mean, if you have, uh, they’re kind of
hard to schedule or bring into the office. I don’t think the agencies
do 3 year olds, but toddlers are excellent beta testers. They won’t
tell you what’s wrong, but you’ll definitely see it.
00:02:19 - Speaker 2: So you’re working on TLDraw. Tell us about that
project and your background that brought you there.
00:02:24 - Speaker 1: Yeah, so Tal Draw it started out as like an open
source project, I guess it still is an open source project. It grew out
of kind of earlier open source work that I did with a library called
Perfect Freehand, which is an algorithm for creating like digital ink,
so pressure sensitive or variable width lines, takes a whole bunch of
input points, and comes up with a whole bunch of output points that kind
of describe a shape that surrounds the input points. So, perfect
freehand, I kind of started working on it pretty publicly on Twitter. So
posting all these in progress GIFs and talking about the different ways
that I was trying to solve this problem of doing pressure sensitive
slash variable width lines.
You can’t really do that in the browser, like it doesn’t have any
primitives for like lines that change their width. So, hence the whole
like journey into trying to figure out how to do that kind of
It works, it works super good. I would recommend it if you have anything
to do with the browser that needs to have a kind of a cool ink. And
it’s being used all over the place. It’s used in Draw.io, it’s used
in Excali Draw, it’s used on, I think NextGS Live uses it in their
It was one of these situations where The status quo for like a pencil
tool or a pen tool was like so poor on the internet or on the browser
especially. That any improvement sort of would have been well timed and
perfect freehand just happens to be an improvement that is pretty good.
You should use it. That’s what I say.
So, I’d been working on perfect freehand, I’d been doing some
integrations with like other diagramming tools like cala Draw and I
guess in along the way of making perfect freehand, I built a couple of
these. Canvas playgrounds almost, places where I could test this thing
out or try out the different algorithms, see where it was going right or
wrong. And I’ve done that a couple of times for Perfect Freehand, also
for like a kind of an offshoot of this project called Globs. But anyway,
I just made enough of these sort of infinite canvas editors that once
Perfect Freehand was pretty much done, I started working on that kind of
Originally it was gonna be like a programmable design tool that I could
use both together with direct manipulation, but also with like, you
know, programming, in order to help me figure out problems like perfect
forehand or problems like cool arrows or something. And once I started
posting that, On Twitter, saying like, oh cool, I’m looking, I’m kind
of making almost like a figma that you can program, or a figma that you
can put anything on the canvas that you want, and I’m doing it in
React. That story suddenly got pretty popular. Like I had folks reaching
out to me and saying like, hey, we’re building something that kind of
needs this kind of canvas. Can you open source this or can you share
this or can I hire you? So based on that attention, I was like, all
right, well, maybe this is worth proceeding with. I took some time off
between jobs. I was gonna start a job actually at Adobe, and I thought,
OK, I’ll take some time off in between and I’m just gonna work on this
project, make it open source, spend 6 months on it, and then, yeah, see
what happens. Worst case scenario is that I’ve made something really
cool open source, worked on something interesting in this space, and I
should say there’s really very, very few open source kind of canvas UI
type of tools out there. And none of them that were using this kind of
00:06:04 - Speaker 2: It’s interesting that you got that initial
demand, you might say a product manager lingo might be it’s sort of
market validation, you know, don’t build it until you already know
people want it. Even before you got into the project itself, did you
have a sense of why people wanted it or had they tried to build it
themselves and discovered it was hard? Had they not built it and they
just hoped someone else would do it for them? It just they liked your
other work and they just thought if you did a good job on this other
library, maybe you do a good job on this library too. What was the core
of their demand, I guess.
00:06:36 - Speaker 1: Yeah, my theory about this or my guess about this
is that this was 2021, so not too long ago, 18 months ago maybe. We were
still middle of the pandemic, seemed like everyone had started using
these apps like Miro or Mural or I think Fig Jam had just been released
and The place for online collaboration was starting to just be the sort
of the whiteboard, you know, abstractly like this 2D kind of surface
that you could move around on and interact with people and co-create
And the tools in that space were pretty mature, like again mural fig jam
was new, but it was already, you know, built on good bones of FIMA
itself. And so, As normally happens with like successful general
products, like you start seeing different teams wanna carve off
verticals for that, say, like, OK, cool, Mirro is great. Mirro is for
everyone, Miro is for everything. What if it was just Mirro for project
managers? What would that look like? Can we steal a billion dollars off
of Canva? Can we, you know, take a little bit of that giant market from
FIMA or something? The issue there is that all of these tools, pretty
much anything. That wants to use that kind of canvas UI, it’s a tough
engineering problem. And there’s like a ton of uh functionality maybe
of like a canvas like that that is like, it’s almost like a text editor
that it just has to be there in order for it to feel complete. And if
it’s not complete, it’ll feel broken. But if it is complete, if it has
those like table sticks features and all that, then congratulations, no
one will notice because they expected them to be there from the
beginning. And maybe you ran into some of this with Muse as well like.
Selection should just work, like Undo reader should just work, the cloud
stuff should just work. And some like really gnarly, like logic puzzles
involved in these type of things that we take for granted in something
like Mirro or fig jam. Yeah, like they’re they’re just gnarly
problems. Never mind the whole like, how do I render this thing. So when
I started saying like, look, this is something that you can use as like
a starting point, where all those problems are gonna be solved. Now
it’s just about picking like, what do you want to put on the canvas,
how do you want these things to interact with each other. I think that
story of giving someone like good open source starting place, that was
like the compelling story. It’s just like the same with maybe like
prose mirror or a code mirror. Text editors are super hard, no one wants
to build one themselves. I mean, I don’t want to build a text editor
myself ever. But I do want to make apps that include them, and I do want
to do stuff with them, and if I can do that on open source work, then
all the better. So, I think that’s where the validation started
clicking in, or like pre-validation. I didn’t have an idea really, but
I did notice that there was a demand for this kind of thing, because,
yeah, it was becoming more of like what software looks like, especially
around collaboration, like it looks like a canvas.
00:09:30 - Speaker 2: Fun little parallel story there from the Hiroki
days, which is a very early version of Hiroki that was mid-2000s, 2007,
2008, had a text editor in it, but yeah, we had to implement that from
scratch in the browser and you know, pretty quickly you exactly as you
said, all these table stakes things you just expect to work,
particularly in programming editors, you know, we had to implement it
and that’s not the differentiated or wasn’t the differentiated part of
But it wasn’t long after that that the AC editor, I believe it was, was
kind of the first really solid open source in the browser code editor,
and that seemed to unlock a kind of explosion of people seeing that.
I mean, I know GitHub used it in the early days for some of their stuff,
but lots of other projects did as well. Suddenly people saw, oh,
there’s a really good code editor that covers that table stakes stuff.
Now I can build this weird project idea that I had that I haven’t been
doing because I don’t want to have to build a whole, you know, fully
functional text editor first. That’s just too hard and isn’t the core
of the idea. So maybe there’s, if your hypothesis that the canvas is a
foundational type of some kind, as you said, how software is just
starting to be, then maybe there’s a similar explosion that could be
unlocked with the right canvas tool kits.
00:10:52 - Speaker 1: I hope so. I released this thing in November of
2021, so after, I think I started working on it full time in like July,
so not too much time. It got pretty popular, the initial usage.
00:11:07 - Speaker 2: You talking about the open source or because I
know, I guess I should describe that.
00:11:09 - Speaker 1: While I was developing this, I was posting about
it on Twitter a lot. You’re gonna hear me say Twitter a lot, probably
00:11:19 - Speaker 2: I think we could have done a build in public
episode with you if we already did it with the maker of Canopio Club.
Yes, so it’s a similar kind of concept which is showing your work in
progress as you go and obviously it’s a very visual domain and yeah,
people like that. They like those little bite sized pieces and they like
00:11:39 - Speaker 1: Yeah, I’d started thinking like, this was right
when GitHub sponsors had come up, and I guess the media that I was
consuming was largely like sponsor driven. Podcasts, YouTube channels,
etc. And I was really interested to see if that kind of model could work
for programming. Programming does not lend itself well to YouTube
streams or videos, even like the educational stuff, I think it can work,
but it’s not like a sponsor model that kills, it’s like a course model
00:12:10 - Speaker 2: Yeah, from what I’ve seen that typically people
who have programming YouTube channels, then you have an upsell into buy
my course, so they put the more basic stuff online, you find it, you
watch it, you get to the end and you like the teacher, quote unquote,
and then you think, well, I want more, I want the intermediate level
stuff, then you go buy their course that’s behind some paywall.
00:12:29 - Speaker 1: Yeah, which is great, and I’ve certainly bought,
you know, courses and books as well, and especially if you have an
education budget, you know, consider spreading that around, don’t let
it go to waste. But I was kind of thinking about the type of content
that I could post every day almost, or the or the type of content that
required like a very low level of engagement with, like, much lower than
by my course, and even like a level of engagement where just clicking
like was like the correct.
Appreciation for this type of content.
You know, I’ve seen folks who are making like really amazing involved
educational material, not just in programming, but in other stuff too,
and it always feels bad where like all I can do is just click a little
heart, you know, it’s like, oh man, like this is worth way more than
that, kind of feels awkward. So anyway, I kind of made user content,
educational content when I worked at a company called Framer. So that
was all very involved as well, very kind of produced.
Took a lot of time to make, and I didn’t want to do that anymore.
So, the kind of the place where I settled in terms of like the kind of
content that I would use to drive interest around Tealro before I had
released it, like, while it was under development, was just like these
eight second gifts. These gifts that maybe had like, you know, were at
150% speed, didn’t take very long to make at all, and certainly didn’t
take very long to consume and just be done with, and, you know, just
clicking that little like button was, what more could you do? Like, it
And so I was posting this a lot as I was working on Tealraw and at the
same time, I had made the like TealDraw.com a kind of sponsorware. So
the only way to access this was to be a GitHub sponsor. And there really
wasn’t any like floor for that. You could give me a $1 and, you know,
have full access. It wasn’t even like a $1 a month. It was just like,
give me something and you can come in. And that worked pretty well. I
got like a couple 100 sponsors, which certainly wasn’t enough for like
a tech salary or anything like that. But it did show that people were
interested in this and it was like a good thing to come back to as I was
developing it. I was also a good motivator for making that content.
00:14:46 - Speaker 2: And to come back to that product manager style
validation, here’s the next step. People parting with money any amount,
the number doesn’t even matter that much, is just such a hugely higher
bar and saying they like it, clicking a like button. Telling you they
think it’s cool, even using it. Parting with your hard earned cash is
just the ultimate measure of validation. So I don’t know if this was
intentional, but it seemed you’re following the product management
playbook kind of market discovery in this journey.
00:15:18 - Speaker 1: It was not very intentional. Again, I was kind of
just playing with this before going to take a job at Adobe, but, you
know, I liked making the content, I liked being able to share this
before it was ready and get that like feedback from people. And I liked
that those people were a little bit motivated as sponsors, and somehow
it just really like ended up in a kind of warm place. It felt like there
were a lot of people on Tal Draw’s side as I was making this and like
wanted to see it do well and wanted to You know, be a part of that. I
spent a lot of time asking folks on Twitter like, how should this
feature work, you know, this, here’s how it works in sketch, here’s
how it works in FIMA, here’s how it works in Miro. What should happen
when you try and resize a group of shapes where one of them is rotated?
I don’t know, like, does it skew? Does it smash? Does it lock the
access or uh aspect ratio and then resize that way, so.
There was a lot of like kind of audience involvement in a sort of a
And yeah, that carried right into the release in November, where I made
everything open source, like overnight. It had previously been closed
source. I’ve removed all the kind of the sponsor walls around
Tealra.com. And just said, like, there you go, release, you know, there
was no product hunt launch. I didn’t post it on hacker news or
anything, but other people did. Enough that I had to ask like product
I’m like, can you just stop posting these? Like, I would love to do a
launch sometime in the future, but I don’t want to do that. And then it
was like at the top of Hacker News for a while, like, for like a
majority of the day, it started. Getting a ton of attention on GitHub,
like in terms of stars and interest from people there.
My Discord started exploding. It was, it was like a really interesting
week as this thing made its way around the internet. And just as like a
free diagramming drawing tool that had I poured a lot of like, I guess,
attention into the microinteractions to sort of add up to a canvas tool.
Yeah, it was fun, and bunch of cool contributions started coming in, and
I was thinking, I don’t know, maybe I don’t go to Adobe, maybe this is
like a good Maybe I can make this into a little microsas product for
And then what happened was, since it was open source like folks started
building with it and I started seeing a lot of interesting projects that
started being used or built around Tealra and getting some amazing
contributions from people who are kind of using it. And noticing like
where it needed to grow and and how it needed to change, and a ton of
feature requests and suggestions and will it do this, could you make it
do this? Can we contract you to come over and integrate it here or
there? And also interest from like investors and other people who are
kind of working on similar projects, but there weren’t many. It was a
Eventually, the interest was enough. And I had some conversations with,
I guess people whose advice I could take that this might have legs, this
idea, kind of like this bet that we’re living in a moment where, for
better or for worse, there’s a lot more remote collaboration. People
want to do interaction or like collaboration through software more and
more that needs to happen somewhere.
The canvas is probably the place where that’s gonna happen.
My bet was these apps, there’s just gonna need to be more of them. It
will be impractical for every team that wants to make Miro for project
managers or fig jam for doctors who wanna collaborate on annotating
images, like, it’ll be impractical for every one of those companies to
also build Miro or also build fig jam, which at the moment is sort of,
there are no primitives, so, yeah, good luck. Like if you want to do
that type of software, like your work cut out for you. If you want to do
that kind of software then. Yeah, there’s really nowhere to start
except from scratch. So, there’s Tal Draw the startup, that’s just
Teal Draw the company Teal Draw Inc and TealDraw GB now also.
00:19:28 - Speaker 2: Well, congratulations, and at the moment is this,
the story you told up till that point, it sounded like it was very much
a solo effort doing kind of everything yourself, one man show. Is there
a team now or is that still to come?
00:19:42 - Speaker 1: Yeah, it was pretty much just me, probably like
99% of the code in the project was just written by me over those couple
of months between July and November. I have had amazing contributions
from a lot of people on the GitHub repository. And those continue to
come in, we have a nice little community growing. But yeah, now I have a
team. I’m up to, I just hired my first employee, and I have two other
contractors who are taking up different parts of the project. We’re
00:20:13 - Speaker 2: Oh, congratulations on your new job as a manager,
00:20:17 - Speaker 1: Yeah, exactly. I have no experience, but so far so
00:20:22 - Speaker 3: So when you open sourced TLDraw, were you at that
point specifically positioning it as a platform for making verticalized
canvas apps, or was it more open sourcing the app and then people
discovered that you can mod it basically and do their own thing?
00:20:41 - Speaker 1: Yeah, so I had picked a kind of a point to chop
the app into two, in a way. I was saying, OK, here’s a rendering layer.
Here’s a, basically like, put react components on the canvas. I call
that the kind of the core layer. And then on top of that, I built a kind
of an application that would use that core layer, which implemented
things like selection or erasing or different types of shapes. And my
guess was that other teams would want to build on that core layer and
say, OK, well, you know, Tealro’s cool, but we’re gonna make a
different app, so we’re just gonna use the same render, like react
Canvas type of render. Um, I was completely wrong. I think there’s
maybe one team that ended up doing that. Everyone else just forked Teal
I picked the wrong place to chop, essentially, that what I thought was
unique to the app, to TealDraw. Ended up being more general than I
And so things like the selection tool where you, you can whatever hold
shift and click things and now they’re all selected or draw a box in
order to collide things and select them that way. Turns out that’s the
same for every canvas and I just gotten it right and followed all the
conventions that I saw in other apps who had also reimplemented the same
sort of logic and whatever an eraser is an eraser is an eraser like.
So the part of the task, once I did decide to keep going on this was to
basically start over, like within 6 weeks of releasing, I was like, all
right, let’s start over, let’s make this for real. And let’s make
that abstraction point that like, where can I chop this? What can I say
is general versus what is specific to til draw and move that way up so
that it really is, you could get more for free without having to fork
00:22:31 - Speaker 3: That’s interesting and that’s super surprising
to me. I feel like for these complex problems, it’s basically
impossible to design a framework de novo, and people try this all the
time, but it very rarely works. Instead, what happens is you have an
application that works well, and then basically you copy and paste that
2 or 3 times, and then you look at the discs and the things that aren’t
the disks become the framework. And that sounds like it’s basically
what you ended up doing with this.
00:22:55 - Speaker 1: Yeah.
00:22:56 - Speaker 2: I think that was a story with rails as well, where
there was a bit of they kept writing these kind of crud apps with a
model view controller framework, and once they’d done it a few times,
it became pretty clear what should be extracted from that and be common
to all of them and what would be different.
00:23:12 - Speaker 1: Right. Yeah. Well, even though I did pick the
wrong place to chop or the wrong abstraction, it was part of the kind of
the DNA of this project from the beginning to be something that other
And so, once it was evident that like folks just were using Tera the
component, which I should say that we did distribute the complete app as
something you could put inside of any website. So it was like also
something that you might like drop into your video chat app, even if you
didn’t want to change how it looked or worked, but you just wanted to
have it somewhere other than TDraw.com.
But yeah, once it became evident that folks wanted to say like, Teal jaw
is perfect. We also want to add like a shape that represents a person.
You know, and that like looks like an avatar and has like that type of
data attached to it. How do we do it? And my only answer was like, smash
that fork button and then own this thing forever, because like, I just,
it’s not built for that, or start from scratch essentially again, like,
maybe not zero, but still very close to it.
So yeah, version 2 is much more kind of Anticipates those kind of
stories, and I had no idea folks would want to do that, but looking
back, I guess, yeah, now it’s obvious.
00:24:30 - Speaker 2: I see that’s where you are today. I’d love to
hear a little bit of the backstory there. You mentioned briefly working
at Framer, which is a very interesting product. It’s been through a lot
of iterations and itself has its own canvas aspect, and I know you’re
actually even relatively new to the programming field, so tell us a
little bit about the journey that brought you to Teal Draw.
00:24:48 - Speaker 1: Yeah, I guess, you know, it seems like everyone I
talked to has a non-traditional story about how they got into this kind
For me, I studied art. I studied, uh, you know, painting for undergrad
and then grad at University of Chicago. So I had a tiny bit of like
technical experience building like portfolio websites, but that was
pretty much it. It wasn’t until I had turned 30, I was now living in
Kind of broke, realizing I should probably make some money in my life
and have a career that has a little bit more speed to it. So, yeah, I
kind of shut down my studio out out near Cambridge and started looking
at different ways of using that same creativity and I guess industry, I
So I started out in design, and then I quickly learned about like
prototyping as a place where the tiny little seed kernel of technical
skills that I had had from all those WordPress sites that I’d in the
Bush administration could be brought over and applied.
This was also like 2017, the idea of like designer who codes was like,
even for someone just crashing into the tech scene like was very
evidently like hot and so that essentially became my brand or my story
as I came in. I was an extremely active user of Framer’s first product,
the Framer, now Classic. So typing out coffee script code and making
things spin around when you click at them.
And my first couple of jobs here in London were essentially, yeah, like
prototyping. Like I was brought in because one way or another, they
wanted to be able to build something before they actually brought
engineers on to build it. And so this Kind of approach to design and
approach to programming of like discovery mode of like, well, we’re not
even sure what this needs to be, but let’s start hacking something
together. Yeah, that was essentially what I have been doing my whole
career and still what I’m doing now.
Ended up working for Framer doing their education, which was an
interesting break from actually like shipping designs or shipping
prototypes and instead trying to figure out. How do I talk to people
about this, or how do I show how to do this type of work? And instead,
like, how do I present this to other people? How do I communicate, like,
what makes a good prototype? How do you go about that and specifically
how do you do that with Framer.
I wasn’t so good at that job. And so afterwards I went back into design
and prototyping for a company called Play who’s making a design tool,
another design tool for iOS, which surprisingly like, you wouldn’t
think that a design tool fits on a phone or on an iPad, but it does.
It’s a pretty cool one to check out if you haven’t.
That was a fun opportunity to kind of rethink all of the creative
software experience. As a designer, you kind of can’t help but think
about the tools that you use because of their software too, and you’re
designing software. Play was like an extreme example of being able to
rethink features like a layers list or an objects list that haven’t
changed since. I don’t know, Adobe Illustrator, and say like, OK, well,
how do you do that on the phone? How do you do a properties panel when
there’s, you know, the size of an iPhone to work with? That was a blast
for me, and it was during that contract when I started doing the open
source work around arrows and around a perfect freehand, and then
eventually healra too. So, that’s sort of the abridged version of my
kind of path from Design and technical design, or increasingly kind of
technical aspects of design until now I’m, I guess, not too far away,
but they definitely do more programming than design and now managing
00:28:49 - Speaker 2: Not to mention it seems like there’s an evolution
there from prototypes which are by nature or even by definition
throwaway, and then going to something that’s an open source library,
other people are building on it, you need API stability. There’s going
to be, I mean, so far the project’s pretty new, but you need long term,
it’s quite the opposite of that prototyping, so I imagine you’re
growing new skills there too as well.
00:29:12 - Speaker 1: Yeah, to be honest, this is also a place where I
think the team that I’m building is gonna Do a lot of the heavy lifting
because my relationship to software is still very like disposable, kind
of like crumple up and and throw in the trash, just do lots and lots and
lots of sketches until it resembles what you want. Not to say that Tilra
isn’t like solidly built and, you know, we’ve fixed the bugs and it it
has good abstractions and solid API and all that. But yeah, definitely
there are certain problems that you can’t quite as easily start over
with. So, I will be slightly hands off on the on those problems just
because different skill set for sure.
But the community side of things, I mean, managing not only the open
source, like an engineering project, but also as a community project has
been really interesting too. That’s probably closer to the work that I
was doing at Framer in terms of education and doing a lot of work with
the community and a lot of work on Twitter and uh chats and such of kind
of unblocking people as they’ve been working with at that time, Framer
now and with Tealro turned out to be decent training for those sort of
Open source relations manager, open source project, open source
maintainer, culture or role. Luckily we haven’t had any drama with
Tera, but it’s still a little bit chaotic and fun.
00:30:36 - Speaker 2: So our topic today is infinite canvas or perhaps
So I think we’ve explored this a little bit already in talking about in
this new world of kind of, yeah, the whiteboard brought into a digital
space and made collaborative and that is increasingly feels like a
foundational piece of many pieces of software.
So yeah, I’d be curious to, you know, kind of go to the fundamentals
here, which is certainly how you Steve or Mark how you define. What is
an infinite canvas, and then we should probably also talk about the
name, cause that’s an interesting thing. But let’s start with
definition. Steve, what do you makes one of these canvases what it is
and different from other types of software?
00:31:15 - Speaker 1: Yeah, well, especially on the web, we are used to
a kind of a document metaphor that, well, the web is just built for,
which is kind of a vertically scrolling infinite page. Infinite canvases
are essentially like a different document format. This idea that there
are two dimensions or kind of almost 3, you know, you can move left, you
can move right, you can move up, down, you can zoom in and out, and that
the screen, what you’re looking at kind of represents a camera that’s
kind of floating above a surface, and there are things on that surface.
And those things can be anything. And in a drawing app, those things
might be little lines that you’ve drawn in a visual note taking app or
canvas-based note taking app, those would be different notes or
different headers or different flags or in a whiteboard, they might
include arrows and texts and sticky notes and all that.
The actual things on the canvas don’t matter. It’s mostly about Stuff
on the canvas that you can view, or if you’re able to, if you have the
correct permissions, then you’re able to directly manipulate.
You can drag things around, you can move them, you can duplicate them or
resize them, and that each one of these types of thing on the canvas
also sort of has its own rules about how it can be changed. So maybe
it’s a video, maybe the aspect ratio of that video can’t change, even
though you can’t change the size. Whereas maybe it’s like a rectangle
and you can change that aspect ratio or the sizes.
And whatever you wanna call these things on the canvas like shapes or
primitives or elements, the thing that you do with an infinite canvas
is, you know, to manage these and read them or arrange them or put the
canvas into a state where it represents something.
So, obviously, again, like whiteboards are pretty clear, like you wanna
do a retro and you’re moving sticky notes around, and maybe voting by
stamping things. And the important thing also is that, unlike a document
where you’re only seeing a very small part of infinitely scrolling
The canvas works really, really well for collaboration. The idea of
having multiple people working on the same surface is pretty natural.
You just represent them by their cursor, wherever the cursor is, that’s
where they are. Um, you can do things like follow people around. I
suppose you could do that on a vertically scrolling page, but it might
not be as fun. Yeah, so that’s why I think it’s been picked up so
readily by, again, like whiteboarding or diagramming or or places where
you need to have more than one person editing the same document. It’s
not like something with text where, as I’m editing text at the top of
the document, all the other text is being pushed down the document.
00:34:00 - Speaker 2: The elements are relatively independent of each
00:34:03 - Speaker 1: Yeah, exactly. You know, you can have independent
experiences on the same document that are just in different places.
And I suppose the idea of place is also a big part of this, and
potentially a big part of this.
The idea of near and far, of close and distance, or like it quickly
drifts into like video game territory of your cursor is your avatar, or
maybe you have an avatar and you’re moving that avatar around this top
down view of an office or a space and jumping into video calls with
people close to you. So yeah, that infinite canvas, that’s kind of a
long definition of how I see it. It’s just a found an infinite 2D plane
and you are a little camera represented by a cursor. Moving around,
moving over that surface. Mark, what do you think?
00:34:52 - Speaker 3: Yeah, I have a pretty similar take. I’ve often
called these multimedia canvases, cause I think there are two dimensions
So one is the canvasness, which is at least 2, sometimes 3 dimensions
and the freedom and flexibility to place content items wherever you
want, like you were saying with relative independence.
The second axis is multimedia, so Which types of media the digital
When you lay it out like this, it’s interesting because this actually
captures the whole universe of digital document editors.
So for example, a classic plain text editor is at the sort of bottom
left of this graph where it’s very limited to one content type and
it’s very linear, a spreadsheet in contrast. Has pretty limited content
types. It’s basically text and numbers, maybe a little bit of color,
but it’s quite high on the canvasness.
You have this flexibility and freedom to place things and by the way, as
we’ve talked about in this podcast, that’s something people love about
spreadsheets. It’s not just a calculator, it’s just a place where you
can put stuff, people like that.
Then you can go all the way out to the top right where I would put like
muse and TL draw where you have a full 2D canvas and it’s highly
multimedia. You have handwriting, text, images, videos, links,
Or you could kind of go back down and say we want lots of different
media types, but we don’t have the full flexibility. So I would put
notion in this category, for example, of you basically have whatever
media type you want, and there’s, there’s a little bit of flexibility,
but it’s pretty much a linearized document.
So in this view, the multimedia canvas is simply the fully generalized
final form of a digital document.
00:36:38 - Speaker 1: It’s good to be here at the end, yeah.
00:36:40 - Speaker 3: And all the others are sort of specializations of
00:36:45 - Speaker 2: One element you touched on there, Steve, which
also I think fits in with the multimedia side as well as you talked
about the elements, you know, we call them cards and news just because I
think that works for us visually and particularly with the touch screen.
It feels like an index card moving around on a desk or something, but
yeah, the elements have a certain sameness and I think this does go back
to Illustrator, which in many ways was the original Infinite canvas to
my mind, but you know, maybe sketch and. FigMA and Framer kind of
modernized that a little bit, but when you lay down a bit of text or add
an image or add a rectangle in, for example, sketch, you can click on
and select each one of them. You resize them the same way as you said
there might be slightly different rules around resizing.
There may be snapping or other things, but in the end, You can do the
same things with all of them, and I think that’s really important and
it’s actually something that to me comes actually from file systems.
This is what’s great about files. It doesn’t matter what’s in the
file. You can always delete it the same way. You can copy it the same
way. You can put it in a folder the same way, you know, inspected size,
that sort of thing. There’s this uniform container. And then over the
years, files have gotten more capable and contained more and more
different types of things, including, you know, things like video that
the original creators of the file system probably couldn’t have even
pictured being possible on a computer, but because it’s this general
purpose container and as a user, I feel that gives me a lot of agency
and power because even if I don’t know the specific type, I know
exactly how to manipulate it.
00:38:16 - Speaker 3: This is reminding me that there’s potentially
some secondary axis here. So one is collaboration, which we’ve
mentioned, and we almost take that for granted now when we think about
modern software, but you know that is a separable axis. The other is
this notion of inline editing, which I think is actually pretty
essential to what we think of as a modern multimedia canvas app, in
contrast, you know, with the typical file manager like Finder, that’s
actually very high on multimedia and quite high on. 2D flexibility,
especially on your desktop, but you don’t have the inline editing, so
it it feels like a totally different experience.
00:38:50 - Speaker 1: Yeah, this ability to not only move, delete,
organize, copy, hold alt to clone, to directly manipulate content, but
also It’s not only about making choices about where a thing is, or its
relationship to other things, but it’s the place also to make stuff and
And yeah, one of the bigger kind of challenges with tealra was deciding
how to allow interactions within shapes versus interactions with
So, for example, you have a video, you can click on the video and drag
it around. But you might also want to like pause the video or change the
time of the video, and thinking about, OK, well, how do you transition
from a shape that is, I call them shapes, whatever tail drops I’m gonna
keep using that word, too hard to deprogram a shape that is Acting like
a shape versus in a shape that is acting like a video, and that you can
interact with with like a video.
And yeah, it turns out that there’s a pretty good rules around that,
like how to make it consistent, which works just as well for like text
as it does for videos, as it does for, I think I have a code sandbox,
you know, running on the canvas and teal draw in one of my example
projects. The joy of using a kind of a web-based canvas of rendering
stuff using the web, although that sounds like such a bad idea to render
things in HTML and CSS, but it really does give you the ability to just
put anything that can be in a browser on the canvas and interact with
00:40:29 - Speaker 3: And this also brings us to the why now and why is
this hard. Like if this is the fully generalized form of digital
documents, why didn’t software start this way? I think a big part of it
is just that it’s hard. Like we talked about how it’s hard to write a
text editor, but what if you now need to write a text editor, an image
editor, a video editor, and audio editor, and they all need to be in the
same thing, you know, it’s, it’s quite difficult.
And then on top of that, you got to actually render all the stuff and
make it manipulable and fast and responsive. It’s just quite difficult.
I think that’s a big reason why.
We haven’t seen it until relatively recently.
I also think there is an element of people just, they don’t fully
realize the expansive possibilities of the software, perhaps when
they’re starting from a very limited. World.
But OK, I constantly bring up this analogy of like a woodworking shop.
Could you imagine if you had like a woodworking district and you had a
shop just for like your chisel work and then a shop for your saw work
and then a shop for your standing work and like you had to take this,
you know, quote unquote file of a project and bringing it across to
these different buildings. That’s kind of the world that we live in
now, and you can’t even look at two things at the same time, like
you’re standing and your chiseling work. That’s kind of the world that
we’ve been living in with respect to software for a long time. And
yeah, it takes some work to bring all those things into one. Workshop
and to learn all the tools and to keep them all maintained and stuff,
but that’s what you really want as a creator.
00:41:43 - Speaker 1: Yeah, especially in the browser, it’s like. Every
10 years, I’m told that you just have to forget what you knew about
like what you can do and what you can’t do just because it just gets
Certainly when I was Poking around with WordPress websites in college,
the web was not a place where you could make a kind of a dumb driven
canvas and make it like fast and good and perform it and do all the
things that you need to do with pressure and multi-touch and all that.
But it is possible today, and I think that when a platform is mature
enough, you know, for you to have like an anything app, kind of a place
where you can just put anything. OK, cool. Videos, yeah, put it there.
Text, we’ll put it there. And that’s always sort of been the promise
of the browser. It’s just been really constrained to this document
format that is increasingly showing its limits.
I think one of the newer APIs that I was looking at from Google, you
know, involved like transitions, like kind of iOS style transitions
between things, you know, that absolutely explodes the notion of the web
page as a page and hopefully Tera also kind of demonstrates that this
technology could be used.
By the way, like, the idea of a rack driven canvas, not to keep coming
back to framer, but That’s where I saw that this was possible because
their, their canvas is driven by react and is driven by the dumb and
like surprised me with the fact that it worked and that it could be made
secure and all that. So, I don’t think I would have come up with this
project without having seen it been done by people more capable than me
than considered that as a possibility. It’s like, hey man, you could
put anything on here. Like, why aren’t we?
00:43:26 - Speaker 3: This was an important data point for me in
believing 4 or 5 years ago that these multimedia canvases were going to
be really important because anywhere you had anything that was like
multimedia canvas, if you squint, people loved it and they were using it
for all kinds of stuff. I mentioned the example of spreadsheets. Another
example is using tools like PowerPoint or drawing programs to do like
whiteboarding basically. We’ve also seen this more recently with FigMA.
FIMA is obviously a design tool, but people will use it for a personal
note taking just because they really like the ability to put different
00:43:56 - Speaker 1: Yeah, I think one of my first creative software
experiences was with Hypercard, way back in the day. I think it was
still Hypercard, like Apple Hypercard or not macromedia or Flash or
anything like that, but yeah, there’s this idea of like, that is not an
infinite canvas, that’s just a glorified slideshow.
But it’s still like I’m like, man, you can make games with this, you
can make presentations for this, you can make You know, animations, if
you just click next fast enough.
Yeah, um. A little bit of Like giving users a little bit of room to run,
you know, it’s always like the best, just because, yeah, suddenly you
have Doom running in Microsoft Excel.
Yeah, and I think with TLDraw, I mean, folks have made a slideshow app,
like a slide editor app with TLDraw, which apparently didn’t take too
much work cause it already had like pages and you’re just have a UI for
moving between pages and I’ve seen it used to do video annotation, like
you pause a video at a certain time and just overlay like teal draw and
be able to edit, like, you know, this guy’s gonna run over here or
whatever, and it’s been used for like wikis.
One of my favorite stories about this actually is that there are two
products that compete with each other. They’re both for Dungeons and
Dragons game masters or Dungeon masters. And when you’re telling or
running like a story campaign or a story world game, you have a bunch of
locations and people and, you know, items and all this stuff and they
might have relationships to each other, etc. And so there are two pieces
of software, one is called WorldAnvil, one is called Legend Keeper, and
they both are essentially wikis, custom wikis for people who are running
these games. And Legend Keeper shipped a whiteboard view of their like
wiki, essentially. Based on TalDraw, like 6 weeks after I shipped
TealDraw, like open source it, like immediately, they seemed to be like
immediately, oh cool, this thing that we wanted to do for like 2 years,
like, let’s just do it. Like we could just nail this, and they did. And
then about 6 months later, World Envil released their canvas, their
whiteboard view of their wiki, which was also based on Tealra. And I had
been, I’d been telling people I’m like, someday there’s gonna be a
product. Or two products that are competing with each other, similar
features, and the thing that they’re not gonna be competing over is
like how well the eraser works or how well the select tool works.
They’re gonna be competing at that higher level of, yeah, like the
features that they built that are unique to their products because both
of them are gonna be using TealDraw. And then it happened 6 months in,
it happened that I was like able to point to both of them and I love
that. They’re both wonderful products and then their communities are
always posting. Like I kind of lurk on their Discord channels just to
see people’s like whiteboards, and yeah, it’s like 200 individual
characters, you know, in a big family tree structure or like alliances
between things. Yeah, definitely not what I expected when I was working
on Tealdrop, but that’s the whole point, right? It’s like to see where
00:47:06 - Speaker 2: That’s great and I like the Dungeon master and
world building aspect. We have some new customers who do a similar thing
for their sort of dungeon mastering, but it is in a way the purest form
Yeah, you have this complex world, you need to keep track of stories,
characters, yes, it’s all made up.
There’s a version of this for sort of fictional worlds as well, right?
Like, yeah, if you work on some franchise like Star Trek or whatever. to
keep some shared knowledge base of all the canonical everything that has
ever happened, timeline and characters and rules for the universe and
things like that, but yes, it is a mix of obviously textual, kind of
more linked wiki knowledge graph style stuff.
But obviously diagrams, maps, drawings, all of that can and should be a
part of it, but traditionally the digital tools that we use tend much,
much more towards those that text and that linear top to bottom
document, and it’s not even top to bottom, it’s really about the
inline. It’s about the text that flows left to right and it wraps when
it hits the edge of the screen and it keeps going down until you get to
the bottom and any multi.
The media you add tends to kind of float awkwardly as sort of a big
character or something, but I think when you look at something like the
Dungeon Master use case which so purely and in this made up realm
captures the complexity of what you can do with computers in terms of
tracking knowledge and what you might want to do, and quickly you see
that text is a big part of the story, but it’s not the only part.
00:48:38 - Speaker 3: Yeah, and I think related to this idea of
constrictive media types and layouts is the notion of premeditated
workflows versus totally in the control of the user workflows.
So typically with software, you have workflows that are Designed by a
product manager or whatever, and they write down the software and
that’s that. So you can imagine for world building, you have a database
table that’s like characters and database table that’s equipment and
you know, if you have something that doesn’t have a common database,
well too bad, you know, email the product manager maybe able to add it
But people really like the ability to craft the software to their use
cases and motivations. I think this is a big draw of The multimedia
canvas, you can just do whatever you want with it.
You know, if you have a different way that you think about equipment or
characters or whatever, you just put it on the canvas and and do
And I think that’s an important aspect.
But the other thing here is you can go down a layer and change the
actual programming, which is one of the things that sounds like it is
really exciting with TLDraw. And again, people, when they just give them
a little kernel of power and capability and you meet that. With some
motivation, people do all kinds of stuff. I’m reminded of the Half-Life
game engine. I don’t know if you guys know the story, but the game
itself was great. It was very successful. But this game engine spawned
off all kinds of stuff, you know, that critically, the original creators
did not anticipate, design, approve, or even know about, right? Just
someone else, they took this kernel of power and made something new. I
think that flow was so great.
00:50:05 - Speaker 1: Yeah, absolutely. One of the opportunities of
having like a multimedia canvas is the ability to extend it
semantically, almost, and you see some of this discourse around like the
idea of notion-based editors, block-based editors, right? Is that like
text is a primitive, images are primitive, but you start to kind of wrap
these things up into more specific domain specific or just project
specific and more meaningful blocks, right? And those things can
interact in ways that maybe have to do with like what they are. So in
the kind of the Dungeon Master example with something like Legend
Keeper, the things on the canvas that they added were parts of your wiki
that you can put on the canvas. And so the character on the canvas, you
know, you can drag out a character from your wiki and put it on the
canvas, is not just an image of that character, but it’s sort of is a
representation of it. And It could be that dragging a character onto
another character might produce a different outcome, might suggest a
different user intent than dragging a character onto a location, right?
Now suddenly you’re able to extend this surface metaphor into something
where those same basic interactions, those same basic direct
manipulation actions can be like meaningful in a way and produce
different outcomes based on what you’re working with. Um, it’s not
just text, it’s not just images. Now, we have people, we have places,
we have things, and they might have their own rules around them.
00:51:34 - Speaker 3: Yeah, I think this idea of extending the content
types in the canvas is super cool. We explored this a little bit back in
the day with the prototypes that led to Muse in the Ink & Switch
research lab, and I remember a couple that were really fun.
One was a person. Which is represented as like an avatar and a name, but
it was so useful to be able to have someone on the board and put them
next to tasks or put people in a group, or, you know, put your current
work under your picture. It’s like a great primitive, in other words,
maps and locations, something that actually people use all the time for
But I think there’s an important sell to here around extensibility. So
there’s one thing you could do, which is draw out the notion of a box
and say, we as the creators of this program could put different things
in this box, you know, notion could put different things in their block
abstraction. And just that is useful because then you can manipulate
these in standard ways and link to them and so on. But then users being
able to extend it is so fun and powerful. I remember with our
prototypes, because this was back when we were doing the prototypes in
JavaScript and people could just kind of write the stuff like a react
component. And it was people were so excited to be able to write their
own things, you know, things that were too specific and niche for the
central, you know, controllers of the framework to ever worry about or
even think of, but people felt so empowered to write their own
components. This brings me to a problem that we’ve been noodling on for
several years and and to my mind it’s still a critical open question.
So when you’re trying to build these end user extensible digital
document systems, there’s a few deerrata that you want. OK, you want it
to be very fast, you want it to be safe in the sense of end users
aren’t gonna be injecting wild stuff, you know, and other users' data
or something. You want it to be approachable cause you want end users to
be able to actually use this thing. You don’t want to give them some
like, you know, assemble or guide and say go have fun. And ideally, I
think you want the extensions to not feel like a totally different
world, like some limited, slower, neutered, you know, subset of the
platform. You really want to feel like the extensions and the platform
itself are like written in the same way. And in fact, if an extension
does well, it can be promoted into the platform. Or if a piece of the
platform is, you know, not finding a lot of use that can be sliced off
as an extension. And I just think it’s very hard to get all these
things at the same time and it’s not clear to me how you do it. So for
example, C++ can be very fast and It can be safe. Maybe Rust would be a
better example of something that’s very fast and safe. But then telling
end users to write their components in Rust, that’s a little bit harsh.
And JavaScript in our experience can be very approachable, it’s very
flexible, but it can be very hard to make fast, especially at the 120
FPS level that we’re targeting from use. So I’m just kind of curious
if they that problem statement kind of resonates with you, and if so,
how you’ve been approaching that.
00:54:25 - Speaker 1: Yeah, I think that’s a really good summary of the
Um, which some of that is gonna be true for any products, right? Like
it’s true for a text editor as for a video conferencing infrastructure.
In the case of Tealra. I’ve addressed this in kind of three design
decisions. One is to use React for the canvas again, because I think the
development story there makes up for the performance story. And the
performance story is pretty good. It’s never gonna be as good as
something like custom webGL canvas that was written in a systems
language, right? That’s not gonna be possible. But on the other hand,
uh, you can put things in there that you just can’t put in anything
else. You can put a mapbox map or you could put a, you know, YouTube
video, like as we’ve talked about, and you can do that, you can offer
those things fairly easily. So, in the case of Tealra, all the shapes
that you and I’m kind of referencing this next version that’s gonna be
launching soon, all the shapes that you see. When you look at teardrop
when it loads up, those are essentially extensions. Those are all
custom. We wrote the user facing app kind of at that same level as you
would write your custom plug-ins and your custom things. And I was
pretty good about not cheating, and of course it’s easy cause we can
change that lower level, you know, that core level in order to
facilitate things that don’t work yet, but um all the shapes are custom
shapes or plug-ins. And then the shapes themselves, these sort of like
plug-in shapes. are all based on like uh primitives that the lower
levels do share. So, for example, you have something like a box, right?
A box rotates in a certain way, it resizes in a certain way, it hit
tests against like points and lines in a certain way, and all of those
things are different than if it was like a pencil shape, like a drawn
line. All those answers are different. However, they’re all the same.
Every box is gonna hit test pretty much the same way as every other box.
Every line is gonna hit test the same way as every other line. So when
you’re creating these custom shapes, when you’re creating these
extensions, you’re able to just sort of inherit all that functionality
from kind of the base, right? You say this is a box, all right, job
done. This is its model. It has a width and height just like a normal
box, but it also has a latitude, longitude and a camera zoom. And then
the next question is like, well, what does it look like? And you say,
well, it looks like a map box component, and it uses that data from the
model, and that’s pretty much all the code you have to write, because
all the other parts of that box in terms of its behavior and such are
just their default, right? If you wanted to make something crazy and
unique or if you wanted to make something that was very different than
all of the primitives that we do share, things like whatever boxes,
lines, polygons. Then you can also access those same primitives, those
like lower level primitives to say, OK, well, here’s my squiggle. It
has 3 ups and downs. Here’s it’s like outline that you can use for hit
testing, and here’s how it resizes and all that. But most of the time,
the custom shapes that we’ve written are just boxes, mostly just boxes
with react components inside of them. So that’s the authoring
experience, or the like developing story there, which itself is a
developing story as we try and push like what you can do with these type
of shapes. Then separately, it’s like, how do you do that, especially
in a multiplayer situation, how do you allow multiple people to be using
their own custom shapes, especially if they’re sort of end user
credable. At the moment, the answer is, well, you don’t. The way that
I’m thinking about this is deferring to sort of the implementer level,
is that TealDraw as a SAS product might someday have its own extension
markets like you might find in Figma where you can download stuff. But
in the short term, this is primarily infrastructure tool for other teams
to create apps with. Those folks would be the ones defining, you know,
OK, well, here’s my person place thing shape. Those folks would be also
guaranteeing that everyone who’s using the app has access to those same
shapes. And so it wouldn’t really be like end user developer who’s
00:58:38 - Speaker 3: Yeah, so each instance of the TLDraw derived app
chooses its extension ecosystem and curates those and makes sure that
they’re safe, and it’s not every individual Joe, you know, injecting
JavaScript into everyone else’s computer.
00:58:52 - Speaker 1: Yeah, exactly. There might be a future where we
want to solve those types of problems or like create those types of
marketplaces. And there are a ton of interesting questions about like,
well, how do you distribute those shapes in like a multiplayer situation
with people who haven’t installed the plug-in. What happens if you copy
your squiggle shape from one project into a different project that
doesn’t have that squiggle shape? Those are all questions for later
down the road. I think the primary thing that I’m going for with this
version is just giving other developers the tools that they need in
order to build the experience that they want for their users to have
that involves this sort of canvas.
00:59:27 - Speaker 3: Yeah, and I think that’s a very reasonable and
practical set of trade-offs, you know, react, curated by the instance,
makes a lot of sense. It is fun to think about the fully general world.
00:59:37 - Speaker 1: Oh, totally, yeah.
00:59:39 - Speaker 3: Of arbitrary extensions.
00:59:40 - Speaker 1: I’ll get there eventually. There’ll be some cool
streaming JavaScript modules, you know, that are pulled in at runtime in
order to, you know, we’ll get there, don’t worry. happy not to be
00:59:52 - Speaker 2: I think of the time, certainly.
00:59:54 - Speaker 3: And your experience with React is that it’s fast
enough. My recollection, so this was a few years ago, but my
recollection was that React or React like systems on the web for this
canvas use case was Not fast enough.
01:00:09 - Speaker 3: More like was very close. It was right on the line
in the sense that if you kind of did anything weird or made a little
performance mistake, it’s very easy to throw the flags, not meeting
even 60 FPS. I don’t think we ever could try to get to 120 FPS because
the browsers on the iPads don’t support 120 FPS as far as I know.
01:00:34 - Speaker 1: Yeah, that’s true, that’s true, but it’s like
01:00:37 - Speaker 1: Yeah, I think Chrome goes up to 120, and I mean
you could try out on Teal Draw, even though the current Tealraw.com is
sort of the first version, and that has only like a little bit of the
optimizations that we’ve since done with the next version. It’s
primarily a memory management issue because those big garbage collects
is what gets you. And it’s also about having a lot of control over what
is rendering, of course, what should react, recognized as a render. And
Paint is a big thing with canvases also, which the browser can do a lot
on its own, and there are some CSS optimizations that you can really
push the rendering engine to do as little as possible and to only do it
01:01:21 - Speaker 3: The memories are starting to come back. I’m
recalling, if you use certain CSS incantations, it invalidates a whole
suite of repaint optimizations, so you get totally hosed, so you got
like, you know, the specific CSS, you know, magic words to say and to
01:01:36 - Speaker 1: Yes. And they’re not all the same between
browsers either. Those magic words, for example, like uh just as a quick
example, I think in Chrome. They perform their transform operations,
they scale, and then they translate or something. They scale last,
essentially. In Safari, they scale first, or maybe it’s the other way
around. No, no, Chrome, they scale first, Safari they scale last. So
essentially that if you do some of these optimizations that sort of skip
the paint and say like, don’t repaint that, you’ve already repainted
it. Suddenly everything’s blurry if you’re zoomed in because they’ve
01:02:14 - Speaker 3: Another way to articulate this might be the react
model in the abstract. If it correctly maps down to GPU operations, is,
it’s great and it’s totally suitable.
And the challenge is Getting that mapping to happen correctly through
your use of react and CSS incantations and so on. It’s a little bit
subtle because that’s a little bit of a black box, you know, there’s
several layers there, and like I said, if you make a mistake, you end up
repainting the whole screen and you’re out of luck.
I also think just taking a step back from the very down in the weeds
implementation details, there is a challenge with canvases and the react
style cause it’s less obvious, I think, what needs to be recalculated.
So in a traditional document. You have just take a very simple case,
like you have a sequence of rectangles that gets rendered, and when you
change, you know, a div, for example, you need to re-render the div and
then put that back in the tree, but you kind of know that the rest of
the stuff hasn’t changed, at least in a simple case.
Right, because it’s a series of non-overlapping rectangles.
But when you have cards on a canvas, for example, there’s all these
implications around, you know, intersections and hit testing and
overlapping, and that’s just for rectangular cards, you know, add ink
and erasers and it’s a whole other thing. So, I’m just kind of
recalling a few years ago, the challenges that we faced with using the
reactor model on a canvas. It’s not easy.
01:03:31 - Speaker 1: It’s not easy. Having done it a couple of times
and having kind of rebuilt that engine, that rendering layer a couple of
One of the tricks was to keep it as flat as possible. So, for example,
HTML allows elements to be placed inside of other elements, and they’re
all can be positioned relative to one another.
Don’t do that. If you’re if you’re building a canvas, you want to be
managing those positions yourself, and you want it to not be pushing
that type of change through React, right? So, if a shape has two shapes
as its children, even though those shapes are being positioned relative
to the parents, and if you move the parent around, that the children
move around too, in the react model, those are siblings. They have
transforms that are being recalculated automatically based on their
dependent data, which is like the parents transform.
And so that takes a little bit of the lifting off of React, to say like,
don’t worry, the thing that you are gonna take care of, we will take
care of, and then to use a ton of essentially observables in order to
kind of Take the wheel in terms of the data. So, You know, nothing has
props, like, OK, that’s too much work for React to do. We’ll take care
of it using observables and we’ll update things when they need to
update. So, there is a lot going on in Tealraw where like, I think we
had someone on the Discord being like, hey, I’m kind of new to React,
and this looks like a really fun project, and I’m like, this is the
wrong project for someone who’s new to react because we’re actively
short circuiting like a bunch of the things that React normally does in
order to be fast. In this type of an app instead.
01:05:11 - Speaker 3: And I would imagine that the programming within a
component looks more standard. It’s like a sort of regular, but then
what you basically want is you want react to render each card, and then
to take all of those and give them to the GPU to composite because that
can be done super fast. So as long as you don’t ask, you know, react to
the equivalent to like recalculate and re-render this whole tree instead
of just making individual cards you can posit down, it should be pretty
01:05:39 - Speaker 1: Yeah, and that’s pretty much a summary of like
what how we do it is that if nothing’s changing with relation to with a
card shape element, we almost want it to be treated as a texture.
In fact, we do want it to be treated as a texture. Don’t think about
it, don’t re-render it, don’t repaint it. But then, you know, you have
something like someone zooms in and suddenly you have to repaint
everything and have to, but at least you don’t have to re-render it.
You just have to, you know, repaint it. And there’s a lot of these days
you can use CSS to avoid a certain type of data flow that otherwise
would have had to be pushed into all those components. So, for example,
maybe it’s the width of the Indicator, when you’re like hovering over
something, right? You want that width of that indicator to be based on
the zoom, so that it’s always exactly one pixel on the screen. And if
you’re zooming in, then you’re actually scaling this canvas using CSS,
but you don’t want that line to also scale. You want it to like counter
scale it against that, right? In the past, you would have had to push
that zoom level down into that box component and say like, OK, well, now
the zoom has changed this, you need to adjust the width of your
whatever. No longer the case, you can use CSS variables and calculations
and such in order to get around that. And so now suddenly, you are still
pushing the data around, but it’s happening in CSS rather than in
JavaScript land, and so that’s a whole class of things that no longer
has to Deal with, did my data change or whatever. And so, especially the
things that happen like moment to moment, like dragging interactions,
like selection and hovering and all that, I try really really hard to at
least keep those things isolated, if I can’t keep it out of JavaScript
completely. Yeah, I’ll probably end up having to hire like a X Mozilla
browser CSS Wizard to optimize all the all the browser stuff because it
is like, we’re not writing a render, we’re not writing a systems level
like WebGL render. Instead, the browser is our render and so we need to
have as much knowledge about how that thing works as we would if we were
working with a lower level architecture. So, if you’re interested in
that, My email address is [email protected] and uh get at me. I’d love
01:08:04 - Speaker 2: If I can transition us out of the technical deep
dive, one thing I’d be curious to get both your take on before we start
winding to a close is the name infinite canvas, whether that’s a
category or a component, and it’s one that I have some mixed feelings
about, because on one hand, Mark uses the term multimedia canvas more
commonly we called Muse a spatial canvas and kind of our 1.0
And I also like the term open canvas, we actually do use that a bit on
our website talking about kind of the interaction model, I guess, or the
idea that it’s more about the flexibility and the fluidity rather than
kind of infinite sounds more like how big it is.
But one thing that I did learn is that that name actually already has a
lot of traction, if that’s the right word for it.
We get lots of reviews on the App Store, people tend to just organically
reach for that term they have since the beginning, and furthermore, that
even experimenting with App Store search ads, I discovered that actually
infinite canvas is a very frequently searched term.
I’ve learned over the years in trying to define new spaces for software
and technology that some name that just seems to stick with people,
whether it’s Tool for Thought or Infinite canvas or something else,
it’s sort of like not worth your while to fight it, so I’ve got my
quibbles with it. I’m not sure it’s the best name or the one that I
would pick, but it does seem to be the one that has some legs. I’m
curious how you both see that.
01:09:34 - Speaker 1: Yeah, not to mention that in the browser, we have
something called Canvas, which is a specific technology which is images,
offering images through JavaScript and like tealras as we talked about,
it’s like emphatically not that. And so I’m always presenting it as
like, it’s an infinite canvas that’s not using Canvas like and it’s
immediately I’ve, I’ve gone overboard.
01:09:55 - Speaker 2: I mean, canvas is just a lovely word, right?
01:09:57 - Speaker 1: So, yeah, of course. And I like the potential for
infinite to not only imply.
01:10:10 - Speaker 2: How big it is, but also like what you can do with
it, or like, that it has that other sort of depth, but yeah,
possibilities, which definitely that interpretation is less, how big is
it or can you zoom in an infinite amount, but more about that it has a
lot of possibilities, you can put anything on it, you have a lot of
freedom in how you manipulate it.
01:10:24 - Speaker 3: Yeah, I think about it very similarly where,
again, to me, the important parts are the freedom and the flexibility
and openness, so I might use freeform canvas, but also to me canvas kind
of implies free form, you know, it’s a blank cheat and do whatever you
want it’s kind of the whole point. So that’s why I often use
multimedia Canvas, but I also agree that it’s probably not worth
fighting too hard, you know, if everyone’s already using those words,
just say it back at them.
01:10:50 - Speaker 1: Yeah, I would be curious to look at that data
about what other, what other terms those people are looking for, because
the other risk with canvas is, and again, not made any easier by the
fact that I picked a name that has draw in it, but that like canvas is
associated with like a drawing or painting kind of like context. And so
I’d be curious if folks that you’ve seen were searching for like
infinite canvas, plus art, plus drawing, plus like Apple pencil. Rather
than infant canvas plus productivity plus whatever, collaboration, etc.
01:11:24 - Speaker 3: This does bring up the very interesting question
of what is the appropriate physical analog, and like you’re saying,
it’s not exactly the canvas because canvas, while it is free and open,
it does connote more drawing and art. To me, the analogy was always a
desk, it’s a place where you put down different content and work on it,
but I don’t think anyone is searching in the app store for Infinite
01:11:43 - Speaker 2: desktop as a metaphor has already been used.
01:11:46 - Speaker 3: Yeah, I got a little bit more traction with
whiteboard because people do use a whiteboard in that way or a pin board
cork board, but it is an interesting point that there isn’t quite an
obvious physical analogy that people tend to reach for.
01:11:59 - Speaker 1: Yeah, Pinboard seems to lack the direct
manipulation aspect of whatever this type of thing is now that we’re
calling it. I like whiteboard too, although whiteboard also It seems
like a fairly narrow subset. For example, like Scalira uses a virtual
whiteboard as it’s kind of like description of itself, and that’s
really good, but it also, you don’t expect to have like what you’re
describing of like multimedia types on a virtual whiteboard. So an
infinite white open pin board is really what we’re going for here, just
doesn’t roll off the tongue.
01:12:37 - Speaker 2: A whiteboard is an interesting term. Digital
whiteboard is one that I’ve seen Miro and others use to describe
themselves, and I think it’s not bad.
It feels a little trivializing or it seems to miss the grandeur and the
potential that I think this space has because a physical whiteboard is
pretty limited not only in size, but really what can do with it, which
is just draw in very limited ways in a few colors, whereas again I think
the file system is more of a source of inspiration to me, something that
is extremely flexible, powerful, searchable, has a lot of depth, has a
lot of extensibility, has programmability.
So I think whiteboard isn’t bad, but maybe an infinite canvas indeed is
01:13:19 - Speaker 1: I was curious, when you were using the kind of
term spatial, like, I use spatial canvas, I think when I was first
presenting this idea to investors, and that seemed to click with that
crowd, because they probably heard it from other people who are pitching
these type of apps. But yeah, does that have legs? Did people understand
what Muse was when you were calling it like a spatial app or a special
01:13:43 - Speaker 2: Yeah, I think so. I think it wasn’t very
accessible, so I think folks who are, you know, read the Humane
interface by Jeff Raskin or know about Zoomy UIs or I don’t know, just
have a very deep interest in the topic or maybe that’s more likely for
them to understand, but for a more Not even casual user, but just a
regular professional user that is looking for a tool to solve their
problem. I don’t think it meant a lot, it didn’t seem to resonate,
didn’t seem to strike a chord necessarily. I like it, and I’ve even
seen others use it as well, but it feels like that’s one’s less likely
to really get the traction.
01:14:19 - Speaker 1: I guess zooy won’t take off either, for the same
01:14:22 - Speaker 2: Probably not, yeah. ZUI, yeah.
01:14:26 - Speaker 3: This is very interesting to me that there’s two
ways we’re defining fitness of the name. One is, for example, if
you’ve used uses a lot, does it seem to accurately describe what it
does, but the other is, what are people typing in the app store when
they want something like this, and these things are actually quite
different, and I think the latter is harder.
01:14:47 - Speaker 1: Yeah, I mean, you always want to try and meet
people halfway, but it also, I don’t know, it’s been fun to watch what
people’s expectations are with Tealraw, like when they come to it, and
like, due to some poor user interface choices on my side, like, it’s
not obvious that this is a multiplayer app and that you can like, use it
with other people, and so I’ve definitely been like on Twitter and
people are like, man, this is so good, but I wish that it was
multiplayer, and I’m like, oh no, it is, it is, it is, and I know that
I’m only meeting like the one out of the 100 people or 1000 maybe who
decided to actually tweet about it, rather than just being like, oh man,
whatever, it’s not multiplayer like Miro is. So I think getting some of
the naming can also work to kind of signal those type of things, like,
what can you do with this? What is it for? And Yeah, like if it was
called a multiplayer whiteboard, like right off the bat, you know,
chances are people would look a little harder for that button to say
like, OK, open a new multiplayer page.
01:15:45 - Speaker 2: Notably, Mirro was originally called Real Time
board, which I think sort of captures, maybe not the most elegant name,
but it does capture that feeling, right? And then later they rebranded
once maybe they were big enough that they didn’t need the product’s
name to be so descriptive, so. I don’t know, but we look forward to an
upcoming rebrand of TL draw to TL collabra or TL I use realtime board
01:16:07 - Speaker 1: There you go, waiting, waiting for it. I, I did
always feel bad for Mural, M U R A L, which is another direct competitor
of Miro, and then they had that name first and then Miro changed their
name to Miro, which is almost like identical pronunciation. Yeah,
surprised it worked, but yeah, I hope no one else does it with Seal
draw, CL draw or something like that.
01:16:36 - Speaker 3: Well, regardless of what we end up calling this, I
think the good news is it’s a very powerful category of software,
especially over the past 5 years. I think we’ve really seen it come to
the forefront with products like Sigma, and now with tools like TLDraw,
we could see a whole another blossoming of verticalized instances of
this pattern. So I think it’s really cool to be a part of that
01:17:03 - Speaker 1: Yeah, for sure. I really just can’t wait to see a
bit like what you were describing with text editors, you know, like,
once we had a good text editor on the web, something that you could use
to build other products, you know, we saw a ton of really interesting
projects that just wouldn’t have been built, because like the same team
that wanted to build code sandbox might not have. Wanted to build
Monaco, right? Or might not have also wanted to build the text editor
that you would need to build code sandbox. Whatever else happens with
Tera, I really, really hope that it lowers kind of the barrier of entry
to making this type of app, that if there are bad ideas that involve the
canvas, that suddenly we’ll see bad ideas that involve the canvas
because there’s not 2 years of engineering time in order to get that
bad idea out into the world. And I’m more than happy to do that 2 years
of engineering time cause I Like this stuff, but I’m looking forward to
seeing what people do with it.
01:18:01 - Speaker 2: Well, let’s wrap it there. Thanks everyone for
listening. If you have feedback, write us on Twitter at museAppHQ or via
email, hello at museApp.com. And Steve, I want to thank you for helping
inspire and push forward the infant canvas revolution that hopefully
we’re all going to be a part of.
01:18:20 - Speaker 1: Thank you. I should say, if you want to kind of
follow some of my journey here, you can follow me on Twitter. I’m at
Steve Rui OK. And yeah, if you want to try out TealDraw itself, it’s at
tealdraw.com. It is multiplayer. Don’t let the UI fool you.
