Off The Main Thread Podcast

The Big Build Bool Bonanaza II

In this episode, Jake investigates whether the existence of build tools is a symptom of the web being underpowered, or if they are part of the solution to make the web better. Surma shares his experience of learning about Bazel and how it can be used to build web projects.

Resources:

Transcript
  1. Surma:All right, Jake, we're back. This is episode two of The Main Thread, which will surprise
  2. Jake:All right.
  3. Jake:Yes!
  4. Surma:people because it's been a month. Yeah.
  5. Jake:Rather than a year or two. Yeah, this is a... we're back on track. It's like the
  6. Jake:old times, like, during lockdown, where we literally had nothing better to do than this.
  7. Surma:Yeah. Let's see how quickly we can disappoint people again.
  8. Jake:Did you think that's what we did last time? We were quite rusty. We should tell people that we,
  9. Jake:the previous podcast people heard,
  10. Jake:was the second recording, because the first one was just too rubbish.
  11. Surma:Yeah, a bit boring, right? We're just a bit out of practice. And still, I think the last podcast, well, it was actually, I have to say, it was really, really lovely
  12. Surma:seeing both the excitement leading up to the release and then the positive response after
  13. Surma:release to our very first new episode. That really was quite energizing, which is probably
  14. Surma:part of the reason why we were like, yeah, let's just lock in the next day to immediately
  15. Surma:record one. But yeah, like listening back to it, still a bit rusty, still a bit of synergy.
  16. Jake:Mmm, a lot of rust. Yeah. But in this episode, I want to talk about
  17. Jake:build tools and whether they're a symptom of a problem or whether they're part of a solution.
  18. Surma:Well, I'm going to talk about build tools. It is. Yeah, I'm gonna I'm gonna go a bit
  19. Jake:And what are you going to talk about later on the show, Sam?
  20. Jake:Oh, it's a very build tool heavy episode. That's exciting. Oh.
  21. Surma:into a build tool that isn't popular on the web, I'd say, which in general probably isn't.
  22. Surma:And we're going to talk about Bazel, the thing that Google open sourced. No, but it's nice
  23. Jake:Excellent. Oh, okay. That's exciting. And we should say,
  24. Jake:well, I am going to say that this podcast is sponsored by Shopify,
  25. Jake:by which they let us do this and pay for the hosting and editing. And it feels like I have
  26. Jake:to say that. They're not asking me to say that, but I'm just so happy that they're paying for it.
  27. Surma:to acknowledge that, you know, they said, you go do your podcast while you work for
  28. Surma:us as well. So that's nice. Indeed, I have ever told you that I find the phrase moving
  29. Jake:So I'm in a new location because I have moved house since we last spoke.
  30. Surma:house very weird as an ESL because not moving the house with the contents. I mean, unless
  31. Jake:Oh, that is true. Huh.
  32. Surma:you actually did move your house, which would be deeply impressive.
  33. Jake:Oh, God, no. It was complicated enough, what we did. And our intention was to be in a different
  34. Jake:building at the end. So, yeah, the buildings remained where they were, thankfully, throughout
  35. Jake:the process. But the process of, I know, it's like such a privileged whinge. But the amount
  36. Surma:But anyway, I'm going to talk about Bazel, the thing that Google open sourced. No, but
  37. Jake:of people you just sort of hemorrhage money to that I don't understand what they do.
  38. Jake:I got an email from our solicitor and it said, this was all part of the agreeing the contracts.
  39. Surma:it's nice to acknowledge that, you know, they said, you go do your podcast while you work
  40. Surma:for us as well. So that's nice. Indeed, I have ever told you that I find the phrase
  41. Jake:And this email said, I have taken a release from the buyer solicitor and have given a release
  42. Surma:moving house very weird as an ESL because not moving the house with the contents. I
  43. Jake:to your seller's solicitor. And maybe it's because I have the humor and wit of a 13 year old boy.
  44. Surma:Yeah, strong release.
  45. Jake:But that sounds like farts to me. Does it sound? Yeah. And so I've just got this vision of the way
  46. Jake:that whole process of moving works is it's a load of solicitors in a room
  47. Jake:cupping their hand, farting into it, and then kind of throwing it at each other in a very
  48. Jake:theatrical kind of Harry Potter wand battle kind of way, but throwing farts.
  49. Surma:I imagine it a bit to be like the touching cloth in terms of atmosphere, the touching cloth sketch.
  50. Jake:Oh, we will put a link to the touching cloth sketch because it is
  51. Jake:an absolute, it's one of my favorite sketches and features. Yeah. Award-winning Olivia Coleman
  52. Surma:Especially because it's Olivia Colman. And you just.
  53. Jake:talking about poo. I must have told you this before, but I, to one of the directors at Google,
  54. Jake:I told, I was talking to her about a thing I'm working on and I was like, I'll just send you
  55. Jake:a link to it. And I pasted the link, but it wasn't the link I intended. It was just whatever
  56. Jake:else was on my clipboard at the time. And it was that sketch.
  57. Surma:Enjoy. Oh, yeah, that stuff you've sent me that I think would have been probably many
  58. Jake:And when that happened, I was like, well, that's embarrassing, but do you know what?
  59. Jake:Could have been worse. Could have been much worse.
  60. Surma:HR violations, so. She has a sense of humor while also being extremely important, TM.
  61. Jake:Yeah. Although it was, it was to Parisa, who is head of Crow Knees. I can't remember her title,
  62. Jake:but she is a good person and definitely saw the funny side of it. Absolutely.
  63. Jake:Yes. Important and competent. And that sense of humor. What a mix. What a mix.
  64. Jake:Anyway, what have you been up to recently?
  65. Surma:What have I been up to? Oh, you know, I flew to to Austria with my partner because they
  66. Surma:were giving a talk there and I was just, you know, along for the ride. And what really
  67. Surma:stayed with me for some reason, you know, sometimes you witness an interaction and you
  68. Jake:Ah. Yep, yep, yep.
  69. Surma:just keep thinking about it. And this seems like really mundane. But for some reason,
  70. Surma:I watched two passengers on the airplane squeeze past each other, kind of like the typical
  71. Surma:British, like, oh, apologies. Thank you. Thank you. I'm so sorry. And then saying,
  72. Jake:No one enjoys flights.
  73. Surma:enjoy your flight to each other. And I keep dissecting this three word sentence, enjoy
  74. Surma:your flight. When they're both on the same plane and we all know it's not going to be
  75. Surma:that enjoyable. But what a weird thing to say, I thought. I mean, you know how.
  76. Jake:It's true. Yeah. Well, I mean, if one of them's in first class and one of them is in
  77. Jake:economy, then there might be relatively different
  78. Jake:enjoyments. But as you say, it's not going to be.
  79. Surma:Inter-Europe flights, the business class is the same seats, but the middle seat has a
  80. Jake:Oh, of course.
  81. Surma:table duct tape to it. It's the same.
  82. Jake:Yes, that is, yes, that is true. I've only, I got upgraded to one of those ones and it was
  83. Jake:a massive disappointment. Although I was, I was sat next to, well, not next to, because obviously
  84. Jake:the table is blocked out. Michael Portillo, who is a politician who now, instead of doing politics,
  85. Jake:does documentaries about trains. And my, my friend who I was originally going to be sat next
  86. Surma:Are you not massively into trains?
  87. Jake:to, who was now a few rows back, is massively into his trains. And so there was, it was a huge
  88. Jake:point of jealousy because I didn't, I didn't, I didn't bother Mr. Portillo about anything,
  89. Jake:at least of all trains. My friend was absolutely furious. I, do you know what? I have spent years
  90. Jake:as a commuter, which has really soured trains for me. Like, yeah, just horrible means to ends.
  91. Surma:Yeah. In a way, yes. Although I've recently learned about the Emirates cabin class, which
  92. Jake:Much like planes.
  93. Jake:Oh, absolutely. And I, and I still might, I mean, maybe that will be different, but my,
  94. Surma:is higher than first class. We have a whole cabin, like a closeable little cubicle with
  95. Surma:a door. And yeah, I don't know. I didn't look up how much it costs, but it's like, that
  96. Surma:looks nice. It's probably also going to be inaffordable to anyone.
  97. Jake:my theory on air travel is that it's, no matter what class you look or pay your way into,
  98. Jake:it's, it's still worse than, like, or you'd hope it's worse than your destination, right? Like,
  99. Jake:it's still, you know, it's like, oh, this, this, I'm in a seat that, you know, for 10 hours,
  100. Surma:Yeah, it's kind of like you're just offsetting yourself from the worst case within this flying
  101. Jake:that costs thousands of pounds or does at normal price. And it's just not as nice as my sofa at
  102. Jake:home or as nice as my desk chair. You know, it's, it's, you kind of, it's a feeling of relative
  103. Jake:luxury, but not actual luxury. Absolutely. No, when I'm paying for flights myself, my,
  104. Surma:metal tube. I agree. Fully agree. Let's talk about the web.
  105. Jake:my attitude is, I don't, I don't know if I could pay less and just be strapped to the wing of the
  106. Jake:plane, I would, because I will, I, it just feels like such a waste of money compared to like
  107. Jake:spending the same amount at the destination on a nice meal or something. Should we talk about the
  108. Jake:web? I genuinely didn't know that you were going to talk about, and I knew you were going to talk
  109. Jake:about Bazel at some point. I didn't know you were doing that in this episode, but that's kind of
  110. Surma:There's our title.
  111. Jake:great because it makes it the big build tools bonanza, which I feel we might have, have we
  112. Jake:done that before in another podcast? Anyway, let's, let's, let's assume, let's assume we're not
  113. Jake:retreading entirely old ground. But yeah, I kind of wanted to talk about where build tools sit
  114. Jake:in, in the way we build websites and whether they're a good thing or not. And if we come to
  115. Jake:the conclusion, no, then it's going to make your section very redundant. But here's the thing,
  116. Surma:Okay.
  117. Jake:we're not. So a bit of a spoiler alert there. When I first used the web, I was on a 14K modem, which
  118. Jake:was a little old even for the, for the time, but just browsing around the web, like the whole
  119. Jake:communication aspect of it, I found absolutely amazing. And I was looking at sites like,
  120. Jake:I don't know, Lisa's photos of disused underground stations or like Terry's ring pole museum,
  121. Jake:like these proper sort of niche weird sites. But what struck me is like, until this moment,
  122. Surma:Dun, dun, dun.
  123. Jake:I was used to getting information from large publishing houses or like broadcasting
  124. Jake:corporations, but like, that's not what Terry and Lisa were part of. They weren't backed by
  125. Jake:these companies. So how were they doing it? And I found a little menu item in the browser I was
  126. Jake:using at the time, which I think was probably Netscape. And it was view source. I know. Yeah,
  127. Surma:Oh, for sure.
  128. Jake:but it blew my mind because I was now like, I, like, you know, I'd been using computers a long
  129. Jake:time before this, like playing games and stuff. And I'd always wondered how they worked, but you
  130. Jake:can't in a, when you're playing a game, you can't just right click and go to view source. But, but
  131. Jake:here I was looking at how the site was put together. And I could, I could start to map
  132. Jake:bits of the code I was seeing to the site I was seeing. And I was like, it was how I got into,
  133. Jake:into web development.
  134. Surma:My origin story isn't on the web, but I had a similar driving force because for me, what
  135. Surma:I want to find out is how do you make .exe files? Because it was a mystery to me. I looked
  136. Surma:at them on my computer, I drag and drop them into notepad and that was gibberish. And I
  137. Surma:sometimes sound individual like sentences and there are words and I tried changing them
  138. Surma:and then everything broke because I didn't know. Because I didn't know it had to be the
  139. Jake:Broke. Yeah, I've done that too.
  140. Jake:Yeah.
  141. Surma:same length. And that's what I literally remember asking like, how do you make .exe files?
  142. Surma:Which is such a funny question to ask. But yeah, the web is then what I did actually
  143. Surma:took my very first steps and it's like I write it in an editor and you open in the browser
  144. Surma:and it does a thing. That's kind of amazing.
  145. Jake:And you make a great point because I had done the same exe thing. And but then I did it with
  146. Jake:a webpage. I saved the, you know, did save as, and I had a HTML file on disk and I went in and
  147. Jake:changed it thinking because of the whole exe file thing, it's like, well, this isn't going to work.
  148. Surma:Yes.
  149. Jake:And then I dragged it into a browser. I was like, what? Like I have modified,
  150. Jake:successfully modified this website. But I saw a talk recently that did like the same
  151. Jake:view source thing with a modern website. And the code was
  152. Jake:illegible. Like, particularly like those, those big class name lists, like that were
  153. Surma:Oh, I have so many opinions and feelings.
  154. Jake:illegible, like a couple of characters each, lots of different class names in a, in the HTML.
  155. Jake:It was, it was probably tailwind, but minified. And the question that was being put is like, you
  156. Jake:know, the, the web has sort of lost its way. And, you know, have, have we,
  157. Jake:have we lost something from, from those early days where we both played around with, with view source?
  158. Surma:But I guess you also have some. So.
  159. Jake:I do. I do. Do you want to know them?
  160. Surma:Yes. Tell me Jake.
  161. Jake:Right. Okay. So I would consider myself part of the old school web crowd. Like there's been people
  162. Jake:who have been around a lot longer than I have. But I feel like I share more of their ideals
  163. Jake:than perhaps some of the folks who started on the web at the same time as I did.
  164. Jake:A few years into my career, probably five, six, seven years into my career,
  165. Surma:What year are we talking?
  166. Jake:like the front end framework started becoming a big thing. And I was, oh yeah. I was worried
  167. Jake:you might ask that. So in, like, I'm talking like Angular, Ember.
  168. Surma:Yeah, I was going to say it strikes me as like a I would have liked hearing that statement.
  169. Surma:I would have said, you know, 2010, 2012.
  170. Jake:Yeah. Between 2010 and 2013, based on a blog post I wrote around the time.
  171. Jake:That's the only hook I've got. But, you know, I was against these sites because they were
  172. Jake:serving like an empty body tag and then doing everything else with JavaScript.
  173. Jake:And that resulted in a much slower and worse user experience than the patterns I was using already.
  174. Surma:I remember trying Angular one like at that time or having to because one of the companies
  175. Jake:I felt like these things came with a steep learning curve, in my opinion,
  176. Jake:and a lack of escape hatches. We talked, I think we talked about this before.
  177. Surma:I was working for used it and it did not click for me. I like the idea because I think Angular
  178. Jake:Yes.
  179. Surma:was one of the first ones where you had like you wrote your own implement your own HTML
  180. Surma:tags almost like visually in terms of the syntax. But then you had what was it like
  181. Surma:directives and services and like I was probably just not understanding the pattern, but it
  182. Surma:did not click when I should be using what and I did not enjoy.
  183. Jake:Yeah, I couldn't get over I couldn't get up the learning curve either. And even when I was
  184. Jake:had something working, I would think, oh, if I just do it this slightly different way, I can
  185. Jake:make it twice as fast. But the framework was like, no, this is not your project. Now it's
  186. Jake:my project. And you may not do those things because I don't know those things yet.
  187. Surma:Yeah.
  188. Jake:And I thought that sucked. I hated that as a pattern. Like there were benefits in terms of
  189. Jake:the DX, but I didn't feel they outweighed the cons, even in terms of DX. And so yeah, this is
  190. Jake:how I kind of have the date in my head because I wrote articles about these issues like 10 years
  191. Jake:ago. So back in 2013, about like, you know, here's the benefits of progressive enhancement
  192. Jake:and these frameworks throw it away and it's not good. And like one camp, the old school web people
  193. Surma:And that distinction still lingers around today. Right.
  194. Jake:were very supportive of these articles. But like everyone who I was trying to reach out to at the
  195. Jake:time, the framework folks were just like plain, just dismissive, saying it was old-fashioned
  196. Jake:thinking. Or that the patterns that I was talking about were not for, well, they were for websites,
  197. Jake:not web apps, you know.
  198. Surma:Like, I mean, even I sometimes make a distinction where the web in itself has started to be
  199. Surma:a lot more accommodating to what you would consider an app.
  200. Surma:But still the kind of like the documents base type still bleeds through and you have to
  201. Surma:work against it, especially at the start.
  202. Surma:You know, in a native app, you think in display sized views and you can build that on the
  203. Jake:Yes.
  204. Surma:web, but it's not the default.
  205. Surma:And I'm not saying it should be, but that's definitely one of the things we're always
  206. Surma:like, oh, I have to do this thing again where if I want to think in apps, I have to build
  207. Surma:this kind of like a view model in there somehow.
  208. Surma:And I yeah, I guess it's both a benefit and a drawback of the web and it is backwards
  209. Surma:compatible eternally.
  210. Jake:Yeah, that definition of sites and apps is a really tricky one because I sort of get it and I
  211. Jake:do refer to things, I will use the term website and web app, but there's no real definition of
  212. Surma:Yeah.
  213. Jake:it, especially it's like you can get the BBC News native app, but people will say that's
  214. Jake:definitely a website when you're looking at it in a browser. But then you've got things like,
  215. Jake:well, you use Wikipedia, but if you're editing an article, are you now, is it an app now rather
  216. Jake:than a site? Like it's, I do, there are vague feelings whereas, oh, that feels like a site,
  217. Jake:that feels like an app, but I think it's difficult to, hmm.
  218. Surma:I think I think it's just much more of a spectrum.
  219. Surma:I don't think there is a point even in defining a hard line.
  220. Jake:I think that's, I think that's reasonable. I think that's reasonable. But as years went by,
  221. Surma:Right.
  222. Surma:Like, what do we gain from having a very specific line between you are a site and the other
  223. Surma:ones are an app?
  224. Surma:What decisions do you derive off of that?
  225. Surma:For me, it's much more about, you know, offline caching.
  226. Surma:Both benefit, although probably traditionally more associated with being an app.
  227. Jake:I think newer frameworks had better escape hatches. I think React actually does a really
  228. Surma:Yeah.
  229. Jake:good job of this right now, although I worry they're going to make it harder in future.
  230. Jake:But they had ways of generating markup as well. So you could serve HTML from a server or from a,
  231. Jake:you know, static build or something like that. Although I think it even, it took a number of
  232. Jake:years from that point before we had tools that were making good use of it. And we see that now
  233. Jake:with like Remix, with Next.js, that kind of thing. And so, yeah, this is modern day now.
  234. Surma:Yeah, there's no bitterness in your voice at all.
  235. Jake:And some of the people who were dismissive about things like progressive enhancements,
  236. Jake:like 10 years ago, are now championing it, even building tools around it,
  237. Jake:even promoting courses around it. And I'm mostly happy about it, so it's fine.
  238. Jake:No, not at all. But it's, hey, look, it is a good thing. I wish it didn't take us 10 years
  239. Jake:to get there, but whatever. But okay, we still use build tools, like more so than ever. Like,
  240. Surma:I'm biting my tongue so hard right now.
  241. Jake:whereas back when I started on the web, you'd write the code, send that code to the browser,
  242. Jake:and it would render on the page.
  243. Surma:It's such a trigger to make me go on an opinion rant.
  244. Surma:Please continue, Jake.
  245. Jake:Are build tools a symptom of a problem that we should be solving,
  246. Jake:or should we be leaning into them more? And like I said at the start, because, you know, whatever,
  247. Jake:like, my take is that build tools are part of the solution, not part of the problem.
  248. Surma:Thank you.
  249. Jake:And I think, but I do feel this is where I start to split a little bit from
  250. Jake:the more old school web crowd that I have kind of considered peers and friends. I still do,
  251. Jake:but like, I've kind of shared their thinking on a lot of stuff, but this is an area where I really
  252. Jake:don't. And I think the old school folks really dislike the upfront complexity of frameworks and
  253. Jake:build tools. But like, I see the point, but things like Terry's Ringpool Museum,
  254. Surma:Thank you.
  255. Jake:that you can still build that in the same way it was built back then. It still runs today.
  256. Jake:You can still build in that simple way. It's not that simple things have got harder,
  257. Jake:it's that the harder things have become possible, but you need tooling and like other, like,
  258. Jake:newer patterns in order to make that happen. Yeah, I hated earlier frameworks because they
  259. Jake:took a mallet to the user experience over what I saw as a fairly modest net gain in developer
  260. Surma:For sure.
  261. Jake:experience, but that changed. And now these newer frameworks and newer tooling, they close that gap.
  262. Jake:And I think it's possible now to use a framework and get out the other side,
  263. Jake:better user experience and a better developer experience than you would without those tools.
  264. Jake:And I do think that is mostly down to the build tool.
  265. Surma:There's always this thing of like make better developer experience.
  266. Surma:So you free up developers to create better user experiences.
  267. Surma:And I think generally, I don't think that is universally true.
  268. Jake:So,
  269. Surma:If anything, you free up the developer to spend time on other things.
  270. Surma:I don't know.
  271. Surma:It's a cynical thing.
  272. Surma:But every language, at least when it comes to compiled languages, has a build step.
  273. Surma:And that is there because programs are a lot better at squeezing performance and doing
  274. Jake:hmm.
  275. Surma:optimizations and being smart about the code base as a whole in one go than humans are.
  276. Surma:I mean, the simple example is nobody wants to write minified JavaScript.
  277. Jake:So,
  278. Surma:Nobody wants to read code where variables have one letter and figure out what the code does.
  279. Surma:You want to write readable code so that other humans can work with that readable code.
  280. Surma:But you do want to ship minified code to your web server because less bytes over the wire equals good.
  281. Jake:yeah, and you're right, like decades before the web, we had, you know, languages that had these
  282. Jake:build steps and it meant that, yeah, what you wrote was different to what the machine ran
  283. Jake:because what the machine ran was illegible. Whereas on the web in the olden days, like,
  284. Jake:you would have to pick a target audience for the code. Like, is it the developers working on the
  285. Jake:code base or the browser that loads and interprets the code? And yes, so these frameworks that I
  286. Surma:Yeah.
  287. Jake:hated were leaning towards the DX side and, you know, making compromises on the UX.
  288. Jake:But in my own projects, like, it wasn't perfect. Like, I was making DX compromises
  289. Jake:in order to make it fast and smooth. But like you're saying,
  290. Jake:build tools mean you don't have to make that tradeoff.
  291. Surma:I think there was a phase where build tools, I guess like frameworks, would do things to enable good experiences that turned into worse user experience,
  292. Jake:So,
  293. Surma:both in terms of performance, in terms of accessibility, in terms of bundle size.
  294. Jake:hmm.
  295. Surma:But that is all kind of like caught up a little.
  296. Surma:And I don't know if this is right to credit TypeScript.
  297. Surma:But I do think TypeScript, to an extent, has just made the build step, having a build step, much more normal.
  298. Jake:So,
  299. Surma:And once you have it anyway, it becomes a lot easier to add other tools to it as well that do some of the other optimizations,
  300. Surma:minifying your HTML, processing your CSS, and just having a bundler set up in general.
  301. Surma:I think we started exploring what other work that is usually deferred to runtime can be pushed into compile time so that the site is smaller or faster and things like that.
  302. Surma:And, you know, I guess the extreme versions of this is stuff like Svelte, where they have almost like a pseudo language,
  303. Jake:yeah.
  304. Surma:which is all JavaScript with slight additions, but just much more stuff being done on the developer's machine at build time,
  305. Surma:so that it doesn't have to be running on the user's machine or phone when they load the page.
  306. Jake:Exactly. And another example, this is
  307. Jake:Facel folks. They, you know, were showing off their code, which appeared to have an SQL statement
  308. Jake:in part of the front end code. And they were baiting people when they did this. They knew
  309. Jake:what they were doing. But a lot of people got very angry about separation of concerns.
  310. Jake:But I think a lot of developers are too quick to draw that line down things like
  311. Jake:languages. Like, we've got CSS, we've got HTML, there's a separation concern. JavaScript,
  312. Jake:there's an extra separation of concern. Front end, back end, there's a separation concern.
  313. Jake:I don't think it's, I don't, I think that's too simplistic a view of that. I think,
  314. Jake:similar to that, yeah, front end and back end, it's not the right separation concern.
  315. Surma:Hmm.
  316. Jake:Component models are a better separation of concern, I think. But also, a separation of
  317. Surma:Hmm.
  318. Surma:Hmm.
  319. Jake:authored code versus served code is a valid separation. And that's one that build tools
  320. Surma:Hmm.
  321. Surma:Hmm.
  322. Jake:let us make. And that's why I didn't, I think you didn't agree either. I didn't agree with
  323. Surma:Hmm.
  324. Surma:Hmm.
  325. Jake:this talk that showed like the ugly source code of some website. And it was like, oh,
  326. Surma:Hmm.
  327. Surma:Hmm.
  328. Surma:Hmm.
  329. Jake:this is horrible and bad. Because like, once the code hits the browser,
  330. Jake:all I care about is, does that work well for users?
  331. Surma:Hmm.
  332. Surma:Yeah, and I guess there is like a sentiment that I understand in that it was really nice back in the days to be inspecting other people's markup and website and learn from it.
  333. Surma:But I don't think that that is worth enforcing or like worth keeping at the cost of the users.
  334. Surma:In the end, you know, like some pages are open source and you can learn from that.
  335. Surma:And by using build tools, we haven't prohibited the learning experience that we had at the learning approach where you start writing just an HTML file in notepad and drag and drop into the browser.
  336. Surma:That still works.
  337. Surma:You can still do that.
  338. Surma:And I think that's the more important part that there is not the binary format that you have to put into browsers, right?
  339. Jake:So, yeah, I agree with that.
  340. Surma:Like you want to be able to continue just writing stuff, have no build tool and get started understanding the fundamentals, the primitive languages, how they interact with each other.
  341. Surma:But a build tool is just doing optimizations that otherwise would not happen because they're too hard to do for a human by hand.
  342. Surma:And then the users would suffer.
  343. Jake:Agreed.
  344. Jake:Yeah.
  345. Surma:Well, not suffer.
  346. Surma:That's how it's pretty hard, right?
  347. Surma:But, you know, they could have had a much better experience.
  348. Surma:There's things that you would get for free.
  349. Jake:Yeah. And although, in some ways, view source is dead, like, long live DevTools,
  350. Jake:because they'll give you a tree of elements. It doesn't matter if all the white space is
  351. Jake:missing in the actual text. It will show you the cascade of styles. It'll prettify minified
  352. Surma:Yeah.
  353. Surma:I mean, if people care about, you know, keeping ViewSource alive for their production websites, deploy source maps.
  354. Jake:code to some extent or the source maps, that kind of thing. So, yeah, although view source
  355. Surma:Yeah.
  356. Jake:might look ugly, I think we've got tooling there now, which is much, much better at doing that job.
  357. Surma:I mean, if people care about, you know, keeping ViewSource alive for their production websites, deploy source maps.
  358. Jake:Absolutely. Absolutely. And, yeah, so, we can use build tools to let us write beautiful,
  359. Jake:maintainable, readable code, source maps so people can inspect it as well. And we don't
  360. Jake:have to compromise on performance and usability or, you know, that kind of user experience stuff
  361. Surma:Yeah.
  362. Jake:once it hits the browser. That isn't to say that all build tools and build setups are good.
  363. Jake:We've talked about it on the show before. I really struggle with Webpack. I could never
  364. Jake:understand Webpack configs and I could never get my head around Webpack plugins. It has been years
  365. Jake:since I tried. It might be better. The docs might be better. The actual implementation might be
  366. Jake:better. But I have had a much better experience with Rollup. And I still consider it a superpower
  367. Jake:to go and read that. The Rollup docs are not long. You could just spend a couple of hours
  368. Surma:I mean, if people care about, you know, keeping ViewSource alive for their production websites, deploy source maps.
  369. Jake:and read the lot and then you know how to write plugins and it is like a superpower,
  370. Jake:the things you can do with that sort of stuff.
  371. Surma:It is.
  372. Jake:Hmm. Yes.
  373. Surma:We gave a whole talk on this, which we should probably link to.
  374. Surma:For me personally, like Vite has kind of replaced Rollup as my standard starting point.
  375. Surma:On the one hand, because the plugins are, the plugin interface is the same.
  376. Surma:So the superpower mostly remains intact.
  377. Surma:I say mostly because there is some minor differences.
  378. Surma:Like, for example, in Vite, while the development server is running, you cannot emit additional files.
  379. Surma:Because it doesn't actually run Rollup.
  380. Jake:Hmm.
  381. Surma:It runs ESBuild and its own implementation.
  382. Surma:In an actual build, it runs actual Rollup.
  383. Surma:So there it would fully work.
  384. Surma:That's, it's in general, I think, something that you always point out.
  385. Surma:Having a different setup between development and production is always a bit of a concern.
  386. Surma:But so far, Vite has done it right.
  387. Surma:And I mostly use Vite because it does the one thing out of the box that I found quite hard to teach to Rollup,
  388. Surma:which is take an HTML file as my entry point, not a JavaScript file,
  389. Jake:Yes.
  390. Surma:where it actually figures out which CSS files are pulled in, which JavaScript files are pulled in.
  391. Surma:It does all the right things.
  392. Surma:It has support for CSS, CSS modules, JavaScript, TypeScript, JSX.
  393. Surma:All of that is just set up by default with good default configuration,
  394. Surma:but also always with the escape hatches to change the configuration or write your own plugins.
  395. Surma:And so for the vast majority of little things that I end up writing, Vite has been absolutely brilliant.
  396. Jake:It's it's I've played a little bit with it. And it's every time I start a new project,
  397. Jake:I'm like this is the one where I should, you know, lean heavily into Vite. But I'm
  398. Jake:I'm so wanting to get to the rest of the project that I don't and I just use some
  399. Surma:Oh, that's completely fair.
  400. Jake:Rollup setup that I've already. But I need to find the excuse to to use it.
  401. Jake:So, OK, build tools are good, but I also don't think we should stop advancing the platform
  402. Jake:because particular things are covered by build tools. And I really hope we can get to a point
  403. Jake:where bundling is less necessary. So because right now we bundle into just a few JavaScript files,
  404. Jake:meaning if you make a small code change, you're invalidating the cache of a big chunk of code.
  405. Jake:We had this whole promise of HTTP2 was going to solve this, right? But it didn't really pan out.
  406. Surma:Hmm.
  407. Jake:But even in that world, even if it.
  408. Surma:Well, mostly because if you don't bundle the files, like if they're separate files,
  409. Surma:to get the similar loading behavior to bundled files, you would have to utilize H2 push.
  410. Surma:Because when the first file gets requested,
  411. Jake:Yes.
  412. Surma:you don't want to wait for the browser to discover what are the other dependencies.
  413. Jake:Yes.
  414. Surma:The server would like to go, and here's all the other stuff as well.
  415. Surma:But push turned out to be so hard to get right that it's been, I think it's basically been deprecated.
  416. Surma:Like it's not recommended by anyone at this point, as far as I understand.
  417. Jake:Yeah, I'm pretty sure browsers pulled it out of their implementations as well.
  418. Jake:But it'd still be nice to get that back in some way that actually works. But even if it did,
  419. Jake:I would still want a build step. I still want minification. I still want tree shaking. I still
  420. Jake:want hashing, right? Like they're still important things that you don't just get by authoring files
  421. Surma:I think we have.
  422. Jake:and throwing them onto an FTP server or whatever. But one feature that I do think is going to make
  423. Jake:a big difference is import attributes, which have recently landed in Safari and recently gained some
  424. Jake:some TypeScript support. And we have talked about this before on an episode, a video episode of
  425. Jake:HTTP203. Do you remember the episode, Simon? It's... Yeah, it was. Well, do you know what?
  426. Surma:Wasn't that type assertions?
  427. Jake:I don't even think it was back then. I don't think it even become type assertions.
  428. Jake:But we were talking about the reasons some feature like that was needed because of
  429. Jake:bugs in importing JSON. And this was back in 2020. So three years ago.
  430. Jake:Just to catch folks up who haven't seen the episode, which I imagine is most people,
  431. Jake:a spec landed saying that you can import JSON now. Go have fun. So it'd just be like import data from
  432. Surma:Yeah.
  433. Jake:like foo.json. There you go. Landed in the spec until someone pointed out, this is really bad
  434. Jake:because the person who decides if that's JSON or not is the server. Because it's not done by
  435. Jake:file extension. It's done by the MIME type, the content type that's sent as an HTTP header.
  436. Jake:So you could import JSON, which you know is super safe, like really safe to import
  437. Surma:Yeah.
  438. Jake:because it doesn't execute arbitrary code. You're only just going to get an object back.
  439. Jake:But later, someone, the other person, the owner of that site can just change it and start executing
  440. Jake:JavaScript on your site. So people decided that that's a bad idea.
  441. Jake:The thing that should control whether it wants JSON or not should be the page,
  442. Jake:like the script, the thing that is, the place it is going to be executed or parsed.
  443. Surma:Yeah.
  444. Jake:So as you say, we got a feature called import assertions, which was import data from foo.json
  445. Jake:with assert JSON. So you're saying like, go and import this, but only if it's in a JSON format.
  446. Jake:And that was a decision that was made, therefore, both by you as the importer,
  447. Jake:but also by the site. It still had to use the correct content type.
  448. Jake:And it shipped in Chrome ages ago. And it was in stage three in TC39. But they changed the syntax
  449. Surma:Oh, okay.
  450. Jake:from, you know, it stopped being import assertions and became import attributes.
  451. Jake:So it was import data from foo.json with type JSON. And I think it was just that rename
  452. Jake:was the difference because it still requires the correct MIME type to be coming from the server.
  453. Jake:So it's, yes.
  454. Surma:And is there a list of like supported MIME types?
  455. Surma:Because where is defined what that variable that I'm importing from that module, what it contains?
  456. Jake:Well, it's down to the host by which, a host is the browser. And so the HTML spec defines JSON and
  457. Surma:Right.
  458. Jake:CSS as words that can be used. And Safari has shipped this, they shipped it for JSON, not CSS
  459. Surma:Okay.
  460. Jake:yet. But that works today in Safari. It works with the old syntax in Chrome. But yes, you could
  461. Surma:That's quite interesting for bundlers.
  462. Jake:have things like node or dno or bun, like defining their own. It's potentially a place where you
  463. Jake:could end up with sort of namespace collision issues, but hopefully people are just sensible
  464. Jake:and work with standards groups. Because that's how it always works.
  465. Surma:Because I know that in our projects, we have always prefixed our import paths with a magic word and a colon, for example,
  466. Surma:both to signify to readers that this is special.
  467. Jake:Yes.
  468. Surma:This is magic.
  469. Surma:This will be processed by a build tool because it's clearly a non-standard path.
  470. Surma:But also that then when we write the plugin, we could just check for the prefix to know should this plugin handle that.
  471. Jake:I know. And that's exactly the thing I'm excited about. So, yeah, yeah. In our projects, I've used
  472. Surma:That's actually quite nice to potentially detach those two concerns, separations of concerns, Jake.
  473. Jake:like fake URL schemes to signify that this import is doing something special. I kind of like that
  474. Jake:pattern. The pattern that typically happens is that a file is .CSS. So, therefore, the build
  475. Jake:tool is going to come and do something special with it. And, like, I hate that pattern. Because
  476. Jake:you look at it, and it's it just looks like a normal JavaScript import, except it's a CSS file
  477. Jake:or an image or whatever else. Because I'm left with, like, well, what does it get out the other
  478. Jake:side? What does it mean to import CSS? Does it get a style sheet object back? Does it get a string?
  479. Jake:Does it get a CSS module kind of thing? Like, if I import an image, am I getting an image bitmap,
  480. Jake:an object, a URL, base64 URL, a human take array? Yeah. And the way you find it out is by going and
  481. Surma:Yeah.
  482. Jake:looking through the webpack config or whatever. Because this thing is defined with regex. So,
  483. Jake:it's not even easy to find. You've got to crawl through and find it.
  484. Jake:And I actually think that that pattern recreates some of the bug with JSON imports. Like,
  485. Jake:because the script is requesting it. But the script doesn't decide what it gets back out
  486. Jake:of the other side. It's the well, not the server in this case, but the build tool that chucks out.
  487. Surma:I mean, yeah, they're in a privileged role to begin with.
  488. Jake:It's not a security problem, because the build tool and the source hopefully kind of trust each
  489. Jake:other. They have to, really. Exactly. But it's the same DX issue. You don't really know what's
  490. Jake:going to be spat out the other side. And as you know, there's another pattern which I hate even
  491. Jake:more in build tools. And that's ones that look at the AST and go, ooh, this looks like new worker
  492. Jake:and then a string. So, I'm going to take that string and I'm going to assume it's a module.
  493. Surma:Yep.
  494. Jake:I'm going to import it as a JavaScript entry point that can be used as a worker. Because
  495. Jake:every time I use a plugin like that, or a project that uses a plugin like that, I will modify the
  496. Jake:code in a totally legitimate way to do something I need to do. And I just pop off the rails of what
  497. Jake:that plugin understands as a worker. And okay. But as you say, these import attributes give us a
  498. Surma:Yeah.
  499. Jake:standard way of dealing with this. Now I can say, like, import URL from cat.avif with type roll up
  500. Jake:URL. Or, like, roll up image URL with sizes. If I wanted to give me a URL for the image, but also
  501. Surma:Yeah.
  502. Jake:the width and height of the image as numbers that I can do stuff with. Or import JavaScript as a
  503. Jake:worker. Like, give me the URL to that that I can put in new worker or service worker or a worklet.
  504. Jake:All of that kind of stuff.
  505. Surma:I actually like the idea, the future that where bundler plugins, unless really otherwise necessary, just are listed as a mapping for which import types are they responsible for.
  506. Surma:And you know exactly that they don't then basically wouldn't be allowed to transform the AST of other files, except the ones that are imported with their specific identifier in the import statements.
  507. Jake:Finding municipality looks like a lot of things.
  508. Surma:And so that would make it very easy to find in your bundler's config.
  509. Surma:You see a roll up image URL with sizes identifier and you go to the list of plugins and you see a mapping from roll up image URL with sizes to a specific JavaScript function.
  510. Jake:Let me have a look.
  511. Surma:You can then switch it out or look how it works.
  512. Jake:Exactly!
  513. Surma:And it makes it so nice and clean to look at.
  514. Surma:I really like that idea.
  515. Jake:That's the future I want. That's the future I want.
  516. Jake:Unfortunately we're not quite there yet and the reason is like I said that TypeScript has support
  517. Jake:for this and it does but what it doesn't have support for is changing what's returned based
  518. Surma:That surprises me, but I mean, should I say, oh, that seems like an easy fix.
  519. Jake:on the type which sucks.
  520. Jake:Well look they know it's a missing feature and there are discussions on it and like how
  521. Jake:because right now you end up with like one d.ts file mapped to one file but now you're
  522. Jake:going to have a system where like one d.ts file is going to map to one file and the different
  523. Surma:Yeah.
  524. Jake:types it can be imported as but you know they just need to go and solve that problem and then
  525. Jake:everything's good.
  526. Surma:I mean, it's already a bit like, you know, like when we use our previous approach.
  527. Jake:Mmm.
  528. Surma:With, you know, like a special marker string at the start of the import path.
  529. Surma:We had to solve this problem with what TypeScript gave us, which is you just declare module, like a user, you know, missing types DTS ambient module.
  530. Surma:And you do declare module and do identifier colon star.
  531. Jake:Yes.
  532. Surma:This also works with other types.
  533. Surma:Often you will find this in other projects where they have star dot CSS to define what will be returned when you import a CSS file in JavaScript.
  534. Jake:Yes.
  535. Surma:TypeScript can already do that.
  536. Surma:I guess they will have to maybe potentially add new syntax for declare module potentially, but.
  537. Jake:Well, I want them to go a step further than that. Like, I want them to do that,
  538. Jake:because yes, for things like import an image as URL with sizes, it's like, right, okay,
  539. Jake:whenever that's used, it's going to give you a URL and width and height numbers.
  540. Jake:But I would like to get to a point where that can be done on a per file level.
  541. Jake:So, like, if you've got a CSS file that's imported as CSS module, that it doesn't have to talk about
  542. Jake:it in general terms. It could say, like, this specific file exports these properties when it's
  543. Jake:loaded as, you know, type CSS module or whatever. Whereas right now you can just say it's like,
  544. Surma:Yeah, I agree.
  545. Jake:oh, it's an object with string keys and string values, you know. So,
  546. Jake:being able to do it in individual d.ts files I think would be really valuable as well.
  547. Surma:Well, let's, should we like, can we, can we link to a GitHub issue so people can comment plus one?
  548. Jake:Absolutely. I can go and find that again. But yeah, I think we're on a good path. I
  549. Surma:Yeah.
  550. Jake:do think the web has changed in fundamental ways. I recently posted a silly 3D view transition
  551. Jake:and a silly little demo and more than one person replied saying, can I get the code for this?
  552. Jake:And I was a bit like, my friends, you have the code for this because you have the link to it.
  553. Jake:You know, it was not minified. It was really simple. But I guess we have reached a stage
  554. Jake:now where people assume that the source of a website is not going to be useful to them in
  555. Surma:Yeah.
  556. Jake:any way and they need the unbundled, unbuiltooled version. But I think where we are is much better
  557. Jake:than where we were. And so I say that let's keep on build tooling.
  558. Surma:Yes, on that note, Jason Langstorff, formerly Datlify, now does a really good YouTube channel called Learn with Jason.
  559. Surma:Has started a new series called four web devs, one app idea where he has four web developers and all get the same concept given and they build it and then send it to each other and compare and critique.
  560. Surma:And I haven't fully watched the video, but Cassidy Williams is the first one in this video.
  561. Jake:Ooh.
  562. Surma:And she built an app with no build tools.
  563. Surma:She just hand wrote HTML, JavaScript and CSS.
  564. Jake:Ooh.
  565. Surma:And I was like, you go.
  566. Surma:That's.
  567. Surma:Really cool to see.
  568. Surma:Also, it was really funny to see both the comments and reactions on Twitter where people were really surprised that almost said that's possible.
  569. Surma:And I, it made me realize that it is, has become, I guess, a niche approach.
  570. Jake:Ooh.
  571. Surma:I do this all the time where I just, you know, grab object.
  572. Surma:I give HTML elements and ID and grab them all off of document dot all just like quickly string things together.
  573. Surma:But obviously, most, I guess, most starter kits, most people who start with document start a couple layers higher with a framework and a build tool in the middle.
  574. Jake:Yeah.
  575. Jake:Yeah.
  576. Surma:So, yeah, has often Cassidy to showing off that that is very much a valid and completely capable approach to writing a web app.
  577. Jake:Oh, absolutely. And like I say, you're considering, you know, when the code hits the browser,
  578. Jake:you're considering the user experience the most. But if your users are developers who are using
  579. Jake:that source to learn how to do stuff, then yeah, that's a great reason. And that's what I did with
  580. Surma:Yeah, well, do you want to hear how I have probably offended the royal family?
  581. Jake:the view transition demos and that's why I didn't minify them because, you know, it's a developer
  582. Jake:audience. I want them to view source and I want them to have a good experience doing so.
  583. Surma:And I'm not sure we can, we can leave this in, but I'm going to tell the story anyway.
  584. Jake:Do you know what? Yes, I do.
  585. Surma:And we'll see.
  586. Jake:Okay.
  587. Surma:We drove to London for friend's birthday.
  588. Jake:Yep.
  589. Surma:Dropped my partner off.
  590. Surma:And I dropped them off.
  591. Surma:At Green Park.
  592. Surma:Cube station.
  593. Jake:Oh, yeah. Very close to the Royals.
  594. Surma:Very close to.
  595. Surma:Yeah, Buckingham Palace.
  596. Jake:Oh.
  597. Surma:I had my dog in the back of the car.
  598. Surma:I had lots of other stuff in the car.
  599. Surma:And anyone who knows London knows parking in that area is virtually impossible.
  600. Jake:I'm amazed you took a car there. You tend to only make that mistake once.
  601. Surma:Well, we had reasons.
  602. Surma:Well, I was just to drop my partner off.
  603. Surma:I was then continuing on and was going to meet them somewhere else.
  604. Jake:Ah, yeah.
  605. Surma:But we were short for time.
  606. Surma:That was the way to make sure they would make their appointment in time.
  607. Jake:Mm.
  608. Surma:So it was.
  609. Surma:However, because we were late, we did a nonstop drive from Bristol early in the morning.
  610. Jake:Yeesh.
  611. Surma:And I had to wee.
  612. Jake:Uh-huh. Yep. And there are famously, do you know what? Toilets at Green Park Station.
  613. Surma:Yeah, but I'm in a car.
  614. Surma:And I have a dog in the car.
  615. Jake:Yep. Okay.
  616. Surma:And there's nowhere to park.
  617. Jake:No. Fair.
  618. Surma:And I can't really leave.
  619. Surma:My dog alone in the car, even if I had found a place to.
  620. Surma:I was like driving around, trying to find like a spot or even just like an alley where.
  621. Surma:You know, I could maybe put the hazards on and jump into a cafe or something.
  622. Jake:Sure.
  623. Surma:And you know, like and after driving around pretty much aimlessly, because this was getting very urgent, just like twice because I was even closer to Buckingham Palace and I did what any.
  624. Jake:Yep.
  625. Surma:Brave trucker would do.
  626. Jake:I'm looking forward to this. Go on, then.
  627. Surma:I, I, I peed in a bottle in my car right next to Buckingham Palace.
  628. Jake:Yep. Mm-hmm.
  629. Jake:Yay. And then you threw it into the big fountain while shouting, God save the Queen.
  630. Surma:And.
  631. Surma:And kill this one.
  632. Surma:Maybe I should have done that.
  633. Surma:But I.
  634. Surma:That was the first in my life in general.
  635. Surma:It's like I did see that.
  636. Surma:And there was a CCTV camera.
  637. Surma:And now I'm wondering if anybody has this tape.
  638. Jake:Oh.
  639. Surma:But I, I don't know.
  640. Surma:I feel like is this is this a rite of passage as a British citizen?
  641. Jake:What, to have a book pal peepee?
  642. Surma:Yeah.
  643. Jake:I've not done it myself because I have gone at the station, though, so I guess it counts as my
  644. Jake:book pal peepee. Well, your joining pack is in the post. You can now be, I think, racist towards
  645. Surma:Yeah, well, let it count, because it's obviously a thing now.
  646. Jake:people coming to the country. I think that's what happens. You get your right of passage,
  647. Surma:Ah.
  648. Jake:and now you have to be like, close the door behind you and become one of those people.
  649. Surma:I have been looking for that privilege for many, many times.
  650. Jake:I've been looking forward to you talking about this topic for a long time, so yes, I am ready.
  651. Surma:Should we rather talk about.
  652. Surma:Well, I was going to say the web, but actually, this is not web, at least not super web.
  653. Surma:Do you want to hear about Basel? Do you want to learn a little bit?
  654. Surma:In fact, you have been staying away from a work meeting where I talked about Basel.
  655. Jake:I know.
  656. Surma:So you can actually not know what is coming.
  657. Surma:It's, yeah, it's, it's, it's been a really interesting journey.
  658. Surma:So I guess I should really make sure that people know what I'm talking about, because it is not a mainstream build tool, I'd say.
  659. Surma:It is very much in use only at big companies with big products of like Snapchat uses it.
  660. Surma:Uber uses it.
  661. Jake:Is that right? His origins were in more of a typical C++ tool chain. Is that where it comes
  662. Surma:Stripe, like all like fairly big names.
  663. Surma:And I'm sure there's also smaller companies.
  664. Surma:But when you look around, you don't hear about it a lot.
  665. Surma:Naturally, especially in web development, because it was built for more projects.
  666. Surma:But building, compiling is part of the process of Java, C++, now Rust and similar things.
  667. Jake:from?
  668. Surma:I think it was built at Google or like its predecessor is built at Google and still in use at Google called Blaze.
  669. Surma:And I don't know exactly where the differences between Blaze and Basel are.
  670. Jake:Mm.
  671. Surma:But they're conceptually, and I think even syntactically, mostly the same.
  672. Surma:And yeah, like, so it was built to do the things that Google needs to do.
  673. Surma:Build huge Java code bases, build huge C++ code bases, and later on build huge Go code bases.
  674. Jake:Mm.
  675. Surma:So it has all of those things out of the box.
  676. Surma:And very much you can, you can smell that it was made by Google, but you can extend it.
  677. Surma:And it has been extended, which is where it actually becomes usable and interesting for large web code bases.
  678. Surma:And so that's, that's the angle I want to talk about a little bit.
  679. Surma:Like, because that's what this podcast is about, web stuff.
  680. Surma:But I think it's first, it's important to note that Basel is not like Vite or not like Turbo Repo,
  681. Jake:Mm.
  682. Surma:what Vercel has published, which is also like big monorepos, because those are all JavaScript specific.
  683. Surma:Basel is more like Make in that it's a build tool orchestrator.
  684. Surma:So Basel itself doesn't do building.
  685. Surma:It orchestrates other tools that do the building.
  686. Surma:And in that sense, like Make, it invokes other tools and takes our outputs and does that multiple times recursively
  687. Jake:Okay.
  688. Surma:until the thing you want to have out the other end comes out the other end.
  689. Surma:But yeah, so I'm going to talk about it in the context of web apps, because we now have lots of build steps in normal production web apps,
  690. Jake:Mm.
  691. Surma:which starts with, you know, you just have to take your TypeScript code and transform it into JavaScript code,
  692. Surma:remove the type annotations and stuff.
  693. Surma:But also, you know, SAS or CSS modules, all of those things are build steps.
  694. Surma:This bundling is a build step and as such can be modeled in Basel.
  695. Surma:The thing that a tool like Basel brings to the table is skipping work when it doesn't need to be redone,
  696. Jake:Mm.
  697. Surma:which sounds very simple, but it's actually quite hard to get right.
  698. Jake:Yeah, this is something that I've not seen many web bundlers do
  699. Jake:really well. It tends to be something you'll have in make rather than
  700. Surma:Yeah, well, I think we're kind of getting there.
  701. Surma:For example, you know, TypeScript, the TSC compiler introduced incremental builds,
  702. Surma:which is that thing where it doesn't type check a file if it's already done it previously and it hasn't been changed.
  703. Jake:empty.
  704. Surma:Vite now has a cache as well, and especially tools like even Webpack has a cache.
  705. Surma:Vite does as well, Rollup as well, but only while they're running in what is usually called watch mode.
  706. Jake:Mm.
  707. Surma:So they do everything, produce the output and they watch your source tree.
  708. Jake:Yes.
  709. Surma:And if a file gets changed, they are smarter about what work needs to be redone.
  710. Surma:But if you kill the server and you restart it, usually it does everything all over again.
  711. Surma:Even there, I think Vite now is starting to get better.
  712. Jake:Mm.
  713. Surma:But wouldn't it be interesting if not every tool had to reinvent these mechanisms?
  714. Surma:Because it becomes a bottomless pit because then the SAS tool chain needs its own cache.
  715. Surma:And then the bundling tool chain and the CSS model, each of those should start having their own cache.
  716. Surma:That seems like wasted effort.
  717. Surma:And so tools like Make take the approach like they have.
  718. Surma:It's a bit like actually old school Unix philosophy where write a tool that does one thing and one thing well.
  719. Jake:Mm.
  720. Surma:And makes one thing is effectively knowing how to invoke build tools and knowing when it's not necessary.
  721. Surma:And I guess I should maybe explain how without going into like how do you write a Make file,
  722. Surma:but Make does this by annotating every command in your build script, so to speak,
  723. Surma:with the input files that it needs and the output file that builds that generates.
  724. Surma:And it then checks if any of the input files has a newer date, like a modification date, than the output file.
  725. Jake:Mm.
  726. Surma:And if that's the case, that means an input file has changed since it was last built.
  727. Jake:Mm.
  728. Surma:I need to rerun this build step.
  729. Surma:And that's pretty much all the magic behind Make.
  730. Surma:Of course, there's environment variables and all.
  731. Surma:It is quite the complex system, but at a conceptual level, that's all Make is doing.
  732. Jake:Mm.
  733. Surma:Well, with Make, even you do at some point encounter it.
  734. Jake:It's such a simple system, it starts to make me wonder why the web's build tools haven't done this, right?
  735. Jake:Mm.
  736. Surma:You run into a couple of problems.
  737. Surma:For example, it doesn't really know when you upgrade your compiler.
  738. Surma:Like the tool that you invoke, it doesn't know whether it's been updated since the last build
  739. Jake:Mm.
  740. Surma:or it can't really check that easily.
  741. Surma:So if I, for example, were to update from if I used a Make file,
  742. Surma:and I were to update from SAS version 19 to SAS version 20,
  743. Surma:a Make file would not rebuild because it thinks the build is still completely out.
  744. Surma:The input files haven't changed, but the tool has changed.
  745. Jake:Right.
  746. Surma:When it comes to actual compilation...
  747. Jake:The output would be different, but it doesn't know that it needs to make it happen.
  748. Surma:Yeah.
  749. Surma:So there's a couple of things where Bazel goes a step further on this front.
  750. Surma:On the first thing, it doesn't use dates.
  751. Surma:While it seems to work quite well for Make, it always feels a bit fragile to me.
  752. Jake:Yes.
  753. Surma:I don't trust the dates on my files, on the file system.
  754. Surma:Like if you unpack an archive, you can put arbitrary dates on them.
  755. Surma:Bazel instead uses content hashes.
  756. Jake:Hashes? Yeah, there we go.
  757. Surma:Yeah.
  758. Surma:So it's based on the content and it says basically, well,
  759. Surma:if the hashes of the input files haven't changed, therefore, the output file wouldn't change.
  760. Surma:Of course, it has to store that somewhere and there is your local Bazel cache,
  761. Surma:but that's just a detail.
  762. Surma:But in the sentence is an interesting assumption,
  763. Surma:namely that if the input files haven't changed, the output will not change,
  764. Surma:which means that Bazel assumes and enforces that every one of your build steps is,
  765. Jake:Mm.
  766. Surma:I could say, pure.
  767. Surma:Like same inputs to the function must give the same output.
  768. Surma:If that isn't true, Bazel will not be able to cope with it.
  769. Surma:That's a very good question, Jake.
  770. Surma:Basically, Bazel says your build steps need to be hermetic
  771. Jake:Mm.
  772. Surma:because what they want to achieve is cacheability and reproducibility.
  773. Surma:They want to be able to roll back and rebuild the exact same version of your build,
  774. Jake:Mm.
  775. Surma:even from years ago.
  776. Surma:So everything in your build setup needs to be explicitly specified and hermetic.
  777. Surma:So in Bazel, not only are your files encapsulated in your build files,
  778. Jake:Mm.
  779. Surma:but also the tool chains, and that is stuff like Node.
  780. Surma:So in Bazel, you specify which exact version of Node you want to use to do this build.
  781. Surma:In that sense, now the tooling, your compiler or whatever, is part of the cache key.
  782. Jake:Mm.
  783. Surma:So it all becomes obviously files are hashed,
  784. Surma:but not only are the files together used for the cache key,
  785. Surma:but also what is your processor architecture,
  786. Surma:which for JavaScript is technically not relevant,
  787. Surma:but for compiled languages, it very much is.
  788. Surma:But also, which exact tool are you invoking?
  789. Jake:Mm.
  790. Surma:What is the version of the tool? Where do I get it from?
  791. Surma:Because Bazel also takes care of setting that up.
  792. Surma:So if I clone a fresh Bazel project and I say, build it,
  793. Surma:it downloads the tools.
  794. Surma:Even if I have Node installed on my system,
  795. Surma:it downloads the exact version to make sure the build remains hermetic and reproducible.
  796. Surma:And of course, yes.
  797. Jake:Mm.
  798. Surma:Actually, that parallel is a good point because it goes further.
  799. Surma:They also do these builds in a sandbox.
  800. Surma:This is not like a security sandbox.
  801. Jake:Nice.
  802. Surma:It is more a sandbox where each build step is executed in a sandbox
  803. Surma:where only the files and the tools that you have declared as inputs are there.
  804. Jake:Mm.
  805. Surma:So while your actual GitHub repository has loads of files,
  806. Surma:if my build step is only about the CSS file
  807. Surma:and the tool that turns that CSS file into a CSS module,
  808. Surma:then while that script executes in the sandbox,
  809. Surma:only that one CSS file will be visible.
  810. Surma:It's as if the file system had nothing else but those files.
  811. Surma:And that way, Bazel helps you discover if you rely on more than what you have specified.
  812. Jake:Nice.
  813. Surma:So really, in a way, it helps you be hermetic
  814. Surma:and make sure that what you have is reproducible.
  815. Surma:I should be clear that Bazel itself out of the box
  816. Surma:doesn't know how to do JavaScript and bundling or any of that.
  817. Surma:There is, of course, you know, it invokes those other tools.
  818. Surma:But because they have to be hermetic and Bazel needs to know,
  819. Surma:like, where do I download this tool?
  820. Jake:Mm.
  821. Surma:There's an ecosystem mostly by the people from a company called Aspect
  822. Surma:which have written rules for, you know, JavaScript,
  823. Surma:which does Node for TypeScript, which has a TypeScript compiler,
  824. Jake:Mm.
  825. Surma:ESBuild, Rollup, SWC, Jest.
  826. Surma:There's a good chunk of off-the-shelf...
  827. Surma:I'm going to call them plugins because it is kind of that,
  828. Jake:Mm.
  829. Surma:but in Bazel, they're called rules.
  830. Surma:So they take care of all the heavy lifting.
  831. Surma:What's really interesting is that they also take care of your dependencies.
  832. Surma:So they understand the package.json,
  833. Surma:which is actually correct. They don't look at the package.json.
  834. Surma:They, in fact, ignore the package.json.
  835. Surma:They require you to use the package lock.json
  836. Surma:or the yarn lock or whatever.
  837. Jake:Mm.
  838. Surma:And the hash, and they actually...
  839. Surma:Yeah, and the hash, they verify as well
  840. Surma:to protect you from supply chain attacks, for example.
  841. Surma:They make sure that what you're putting is exactly what is expected.
  842. Jake:Mm.
  843. Surma:Again, like, reproducibility is one of the big things of Bazel.
  844. Surma:Right, so I won't be able to give a narrator tutorial on Bazel.
  845. Jake:Mm.
  846. Surma:I might actually see if I can write up a little example repository
  847. Surma:where people can look at what it looks like to work with Bazel instead.
  848. Surma:But what you do, like when you use make,
  849. Jake:Mm.
  850. Surma:you write a makefile, which is effectively
  851. Surma:a series of command line invocations
  852. Surma:with those extra annotations of inputs and outputs.
  853. Surma:In Bazel, it's a bit more complex due to the nature of it.
  854. Surma:And you write it in a language called Starlog,
  855. Surma:which really is just Python.
  856. Surma:They just remove... Yeah, it is really just Python.
  857. Jake:Mm.
  858. Surma:They just removed the abilities to have side effects.
  859. Surma:So from within Starlog itself, you cannot access the file system.
  860. Surma:You only can almost work declaratively.
  861. Surma:Again, this is for the reason that Bazel wants to be able to analyze
  862. Surma:what your build looks like before actually having to run it.
  863. Surma:And it helps it yet again to be hermetic, to be pure,
  864. Surma:without side effects.
  865. Jake:Mm.
  866. Surma:Under the hood, all what you're writing in Starlog
  867. Surma:is usually macros, which are invocations of more macros,
  868. Surma:which in the end boil down to rules.
  869. Jake:Mm.
  870. Surma:And rules just emit series of actions.
  871. Surma:So everything in Bazel boils down to actions,
  872. Surma:which are very simplistic, low-level activities like
  873. Jake:Mm.
  874. Surma:copy a file, run a command, take a file as output.
  875. Surma:And you just chain those together arbitrarily complex.
  876. Surma:What is interesting though, because they enforce this rule
  877. Surma:of having to specify the tool and the input file
  878. Jake:Mm.
  879. Surma:and the expected outputs explicitly every single time,
  880. Surma:Bazel gets implicitly for free this entire deep dependency graph.
  881. Jake:Mm.
  882. Surma:And when one single file changes,
  883. Surma:it can immediately tell you which actions need to be rerun.
  884. Surma:If a file didn't change, the copy doesn't need to be done.
  885. Surma:It's already done. But if a file did change,
  886. Surma:it knows exactly which path to the graph it basically has to take
  887. Jake:Mm.
  888. Surma:to get the refreshed output at the end.
  889. Surma:And that's how Bazel gets so good at rebuilding very fast
  890. Surma:because it is able at a very, very granular level
  891. Surma:to skip work that doesn't need to be done.
  892. Surma:And I will say this one thing, with Bazel,
  893. Surma:the caching works.
  894. Surma:Not once in my entire journey so far did I have to
  895. Surma:wipe the local cache or any cache to fix a build.
  896. Surma:It was always I who was wrong.
  897. Jake:Mm.
  898. Surma:While with Make, I definitely had the other experience
  899. Surma:where I typed in Make, it said,
  900. Surma:oh no, your build is up to date. I deleted the cache,
  901. Surma:I ran Make, and I got a completely different output file.
  902. Jake:Yes.
  903. Surma:I think many people have encountered it, and I think with most build tools
  904. Jake:Mm.
  905. Surma:we have encountered that. I remember with Parcel as well,
  906. Surma:which is a quite sophisticated tool chain
  907. Surma:and does a lot of caching, but there I had it a couple of times
  908. Surma:where I had to wipe the cache to fix my build.
  909. Jake:Mm.
  910. Jake:Mm.
  911. Surma:So I should be clear, Bazel isn't perfect.
  912. Surma:You can make them not hermetic, and sometimes it's necessary,
  913. Surma:sometimes it's just what it is.
  914. Surma:Especially, I think, in C++ land, which I haven't done with Bazel,
  915. Surma:I have heard that there is just no way around
  916. Surma:the fact that you need to link against operating system libraries
  917. Surma:that are on the system, on the machine.
  918. Jake:Mm.
  919. Surma:And that is, of course, technically not hermetic.
  920. Surma:I think, for example, on macOS
  921. Surma:you need to link against one of the operating system libraries
  922. Surma:for the kernel to even allow you to run it.
  923. Surma:But to the most extent, they try to guarantee
  924. Jake:Mm.
  925. Surma:that these build tools are hermetic.
  926. Surma:So I guess we should finally talk about web
  927. Surma:and what it looks like building web stuff with Bazel.
  928. Surma:If you take the rules for JavaScript,
  929. Surma:you could just invoke stuff from NPM.
  930. Surma:So I could just write a single rule and say,
  931. Surma:And that would work, and that would probably...
  932. Jake:Mm.
  933. Surma:Exactly, you would...
  934. Surma:Exactly, you would throw away all the benefits
  935. Surma:that you might be getting.
  936. Surma:But let's think about what a default setup with Vite
  937. Surma:or our default setup with Rollup would do.
  938. Surma:It turns all your TypeScript into JavaScript.
  939. Surma:It bundles all your JavaScript.
  940. Surma:CSS modules, which under the hood is,
  941. Surma:on the one hand, mangling the class names
  942. Surma:and then outputting a JSON file that has a mapping
  943. Jake:Mm.
  944. Surma:from the original class name to the new CSS model class name
  945. Surma:and, of course, all the special CSS module directives that exist.
  946. Surma:Maybe you have other stuff like you can import images
  947. Surma:to get a path to the production URL of the image
  948. Surma:or even the sizes, as you mentioned earlier.
  949. Surma:You could even have stuff like if you import an SVG,
  950. Surma:please run the SVG optimizer
  951. Surma:before you add it to my production build.
  952. Jake:Mm.
  953. Surma:All these things are possible,
  954. Surma:and most of these things end up being
  955. Surma:plugins in modern build systems.
  956. Surma:With Bazel, at the start, it feels a bit awkward
  957. Surma:because of that sandboxing and the requirement
  958. Surma:to having to list every single file.
  959. Surma:It feels a bit awkward to just have to write that all down.
  960. Surma:But it's definitely a bit different because it's even stuff like
  961. Surma:your node modules folder won't be there
  962. Surma:unless you specify it as an input.
  963. Jake:Mm.
  964. Surma:And now you can think about do you want the whole modules folder
  965. Surma:or only parts of it, like specific packages?
  966. Surma:Because, for example,
  967. Jake:Mm.
  968. Surma:if you update your test dependency, like if you update Jest,
  969. Surma:that doesn't affect your build.
  970. Surma:So really, ideally, you don't want
  971. Jake:Mm.
  972. Surma:to rebuild your project just because you updated your version of Jest.
  973. Surma:You want your tests to rerun, but not the build.
  974. Surma:And by default, the rules for JavaScript do the right thing.
  975. Surma:But it's quite interesting to discover that
  976. Jake:Mm.
  977. Surma:these things actually are optimization angles
  978. Surma:that you can benefit from. And on bigger projects,
  979. Surma:all these things can add up to a huge difference.
  980. Jake:Mm.
  981. Surma:In Bazel, I actually ended up doing it a bit different.
  982. Surma:So we are currently looking at Bazel at Shopify,
  983. Surma:but instead of putting it all
  984. Jake:Mm.
  985. Surma:on one build tool and loading plugins,
  986. Surma:we have tried to move most of the work
  987. Surma:into either off-the-shelf CLI tools
  988. Surma:from NPM or otherwise,
  989. Jake:Mm.
  990. Surma:or little one-off node scripts.
  991. Surma:So, for example, instead of leaving it up to the bundler
  992. Surma:to consume CSS and invoking a plugin on it,
  993. Jake:Mm.
  994. Surma:we have instead written a small script that invokes
  995. Surma:PostCSS with a CSS modules plugin.
  996. Surma:It takes a single CSS file as input
  997. Surma:and outputs a single CSS file and a JSON file
  998. Surma:with that mapping. And we run the script
  999. Surma:individually on each CSS file, which sounds
  1000. Jake:Mm.
  1001. Surma:incredibly slow. And technically it is, because every time
  1002. Surma:you spin up Node, you load all the packages,
  1003. Surma:or actually, you know, you create the sandbox
  1004. Surma:with all the files, you spin up Node, you load the script,
  1005. Surma:you load all the required packages, you read the file,
  1006. Surma:you process it, and you do the output. But you only do that
  1007. Surma:once. And then the new output
  1008. Surma:files are now, together with your actual JavaScript
  1009. Surma:code, given to your bundler of choice.
  1010. Surma:In our case, it's esbuild, but you could use rollup. Later, maybe
  1011. Jake:Mm.
  1012. Surma:we can use rolldown when that becomes a thing.
  1013. Surma:And what is really interesting to me about that is...
  1014. Surma:Yes, that's the Rust rewrite that they're
  1015. Surma:working on. Yeah, but what's really interesting
  1016. Jake:Hang on, so just roll down, is that the Rust re-implementation of?
  1017. Surma:for me was the realization that we have
  1018. Jake:Mm.
  1019. Surma:decoupled our tools that do the preprocessing
  1020. Surma:from our choice of bundler.
  1021. Surma:Because if we wanted to migrate from, you know,
  1022. Surma:Vite or esbuild to rolldown, we would
  1023. Surma:have to make sure that the plugin exists for
  1024. Surma:CSS modules, but actually also that it's the same version of
  1025. Jake:Mm.
  1026. Surma:CSS modules behave the same way. Because otherwise, maybe
  1027. Surma:it's a corner case and now suddenly our app breaks. With this,
  1028. Surma:the CSS module processing is a separate tool.
  1029. Surma:And as far as the bundler is concerned, it's only consuming
  1030. Surma:JavaScript, nothing else. And that's actually also a performance
  1031. Surma:benefit, at least in our case, because esbuild,
  1032. Surma:being written in Go, does not
  1033. Surma:invoke Node or a JavaScript
  1034. Surma:engine unless absolutely necessary. So your plugins
  1035. Surma:usually... Exactly!
  1036. Jake:Oh, it stays in go land.
  1037. Surma:So it's parsing in Go, it's doing the bundling in Go, but if you have a
  1038. Surma:plugin, you have to register it with a regex
  1039. Surma:to say, only files that match
  1040. Surma:this regex should be handled by my plugin. And only then
  1041. Surma:does esbuild effectively shell out to Node,
  1042. Surma:which is, of course, full of overhead.
  1043. Jake:Yeah.
  1044. Surma:And so the one benefit is
  1045. Surma:esbuild gets to stay almost all the time
  1046. Surma:in Goland and be fast. But also
  1047. Surma:we have decoupled our
  1048. Surma:plugins from our choice of bundler.
  1049. Surma:And usually in esbuild, but also
  1050. Surma:rollup and other tools, when you restart the tool,
  1051. Surma:it would do all the CSS work and other plugin work
  1052. Surma:all over again. With Bazel now providing
  1053. Surma:a generic caching smartness
  1054. Jake:Mm.
  1055. Surma:engine almost,
  1056. Surma:it just reuses the previous
  1057. Surma:output. And so if you change
  1058. Surma:your JavaScript code, only the bundler gets invoked, which is now
  1059. Surma:very fast. None of the CSS stuff. And we've really seen some
  1060. Surma:very interesting numbers where that gets lots
  1061. Surma:of benefits in terms of
  1062. Surma:build speed, which is exactly
  1063. Surma:what we were after.
  1064. Jake:Mm.
  1065. Surma:Yeah.
  1066. Jake:Mm.
  1067. Surma:Yeah.
  1068. Jake:Mm.
  1069. Surma:Yeah.
  1070. Surma:There is a couple of things that are not as simple, which
  1071. Surma:I'm going to get to in a bit, or that have slight drawbacks, of course,
  1072. Surma:as well. But there is a huge angle for DX here
  1073. Surma:as well, because Bazel takes care of downloading
  1074. Surma:the toolchains and has all of that kind of codified.
  1075. Jake:Mm.
  1076. Surma:As I said, getting started on a project
  1077. Surma:that uses Bazel is as simple as just running
  1078. Surma:that Bazel command. And Bazel is published to
  1079. Surma:NPM. So basically, NPM run dev
  1080. Surma:can still work and it just runs Bazel. So as far as the developer
  1081. Surma:is concerned, they wouldn't even have to necessarily know that Bazel
  1082. Jake:Mm.
  1083. Surma:is being used. And because Bazel is agnostic, it actually
  1084. Surma:gets really interesting if you, for example, are a WebAssembly
  1085. Surma:user and you have multiple programming languages
  1086. Surma:at work. So I've been actually
  1087. Surma:thinking about for Squoosh, where we have
  1088. Surma:parts of it on Rust, part of it on C++ for a time we even had
  1089. Surma:a bit of AssemblyScript. Then the main app is TypeScript.
  1090. Surma:Actually, also almost like three TypeScript projects for workers,
  1091. Surma:for the service worker and for the main app,
  1092. Surma:built with Rollup. In Bazel, it is very easy to
  1093. Jake:Mm.
  1094. Surma:express that an output of the Rust project
  1095. Surma:should be an input for the Web project,
  1096. Surma:which currently is a bit shoehorned in.
  1097. Surma:And the main thing
  1098. Surma:where you can, well, at Shopify we're trying to save
  1099. Surma:time and money, but I think where even some open source projects could
  1100. Surma:benefit from is that the caching
  1101. Surma:while it happens on your disk can also
  1102. Surma:be stored remotely. So Bazel out of the box
  1103. Surma:is able to just put all these
  1104. Jake:Mm.
  1105. Surma:artifacts onto an
  1106. Surma:S3 or a Google Cloud Storage bucket.
  1107. Surma:And so if you make that bucket world readable
  1108. Jake:Mm.
  1109. Surma:or you give credentials to everyone,
  1110. Surma:Bazel will check whether a build step needs to
  1111. Surma:run. It will first check your local cache,
  1112. Jake:Mm.
  1113. Surma:but then it will check that remote cache and rather than actually building it,
  1114. Surma:it will just download the output, which
  1115. Surma:for longer builds, bigger builds like
  1116. Surma:Rust WebAssembly can be really beneficial
  1117. Jake:Mm.
  1118. Surma:or if you want to reduce
  1119. Surma:your minutes spent on GitHub Actions because it will just
  1120. Surma:download the artifact rather than building everything
  1121. Jake:Mm.
  1122. Surma:because maybe your PR is just a quick change
  1123. Surma:to one CSS file, which means you don't want to rebuild all the
  1124. Surma:bundling. There's a lot of things here where you can really
  1125. Jake:Mm.
  1126. Surma:squeeze
  1127. Surma:build time and performance out of
  1128. Surma:many things in your build pipeline by Bazel just like skipping work.
  1129. Surma:And I really like that they just allow you to just put in
  1130. Jake:Mm.
  1131. Surma:an S3 bucket and that is your remote cache now.
  1132. Surma:And again, it still works. Bazel's caching
  1133. Surma:factors in your operating system and architecture so that
  1134. Surma:you don't rely on artifacts that don't work for you.
  1135. Surma:I've been just really impressed with how reliable
  1136. Surma:the caching is, but usually caching is one of the hard problems
  1137. Surma:in computer science, isn't it?
  1138. Jake:Mm.
  1139. Surma:...
  1140. Surma:...
  1141. Surma:...
  1142. Surma:...
  1143. Surma:...
  1144. Surma:...
  1145. Surma:Yeah.
  1146. Surma:It is and I really
  1147. Surma:like that architecture as just like a mental
  1148. Jake:Mm.
  1149. Surma:model. But I guess I should then also talk about the things I
  1150. Surma:dislike about Bazel because it is
  1151. Surma:I definitely don't think it should in its current
  1152. Jake:Mm.
  1153. Surma:incarnation. It's not for everyone. It's not for every project.
  1154. Surma:You know, it's Bazel itself
  1155. Surma:is written in Java, then runs this Python like language
  1156. Jake:Mm.
  1157. Surma:and invokes all the other tools. It definitely has a
  1158. Surma:steep, steep learning curve and the documentation is
  1159. Surma:a bit meh. Like it's there. It's good.
  1160. Surma:But it definitely was a steep
  1161. Jake:Mm.
  1162. Surma:hill for me to climb. And the more
  1163. Surma:I look how specifically
  1164. Surma:the rules for JavaScript work under the hood
  1165. Surma:the more I feel a bit weird about it.
  1166. Surma:It obviously works and it works correctly and I think
  1167. Surma:they are the way they are for good reason.
  1168. Surma:But one of the things that they use in
  1169. Surma:the sandbox is that
  1170. Surma:to be faster, I suppose, they don't copy
  1171. Surma:all the files and tools you need into the sandbox.
  1172. Jake:Mm.
  1173. Surma:They use symlinks and for JavaScript that
  1174. Surma:can actually turn a bit into a problem
  1175. Surma:because if you imagine you're processing a file
  1176. Surma:as a bundler that is a symlink itself
  1177. Surma:and you have a relative import
  1178. Surma:like import a component from dot dot slash
  1179. Jake:Mm.
  1180. Surma:something. You are the bundler.
  1181. Surma:What do you do? Do you resolve that import to where
  1182. Surma:the symlink is or to where the actual file is?
  1183. Surma:And what if there's multiple levels
  1184. Jake:Oh, interesting.
  1185. Surma:of symlinks? And this has caused
  1186. Surma:many headaches for me to
  1187. Surma:figure out how to do the right thing.
  1188. Surma:Even Node itself because it actually executes
  1189. Surma:that line of JavaScript. You can tell
  1190. Surma:Node via command line flags to not
  1191. Surma:resolve symlinks or to resolve symlinks.
  1192. Surma:And both versions have caused problems in different scenarios.
  1193. Jake:I was going to say, resolving sounds like the right thing, in the same way that, oh, do you know what?
  1194. Surma:So what rule...
  1195. Jake:I was going to say, in the same way that, on the web, redirects are followed.
  1196. Jake:And now I'm losing confidence in that statement.
  1197. Jake:But I am 70% sure that redirects are followed and it's the final destination that is the module.
  1198. Jake:Oh, God, I don't know anymore.
  1199. Surma:Yeah, I mean if you want to dive
  1200. Surma:a little bit deeper for one minute. So as I said
  1201. Surma:you can specify that your input is the entire Node modules
  1202. Surma:folder or just sub-packages of it. So the Node modules
  1203. Surma:folder is a real folder but everything
  1204. Surma:inside is a symlink. Now if you
  1205. Jake:Mm-hmm.
  1206. Surma:written code without thinking about Bazel
  1207. Surma:a package could do something like
  1208. Surma:import dot dot slash other package name
  1209. Surma:because that would work in production.
  1210. Jake:Mm-hmm.
  1211. Surma:But now if you know you're in Node modules
  1212. Surma:your Node dot dot slash other package name will give you
  1213. Surma:the other package. Now in Bazel land
  1214. Surma:where you are actually somewhere completely different
  1215. Surma:on disk, if you don't resolve
  1216. Surma:the symlinks, then that still works. If you
  1217. Surma:do, you're importing nothing
  1218. Surma:or something different. And so sometimes
  1219. Surma:if your code base is written for Bazel
  1220. Surma:that's fine. If it wasn't for Bazel and you're migrating to it
  1221. Jake:Mm.
  1222. Surma:you will find lots of interesting
  1223. Surma:things breaking because of the assumption that
  1224. Surma:everything is actual files on the file system
  1225. Surma:and we just had lots of that. So that's definitely something
  1226. Jake:Mm.
  1227. Surma:and when I look at how these rules for JavaScript are implemented
  1228. Surma:they mostly make that work by
  1229. Surma:kinda injecting custom loaders
  1230. Surma:into Node that deal with all of that.
  1231. Surma:And I'm sure they did it right. I just don't like it
  1232. Surma:because it's not Node anymore. It is Node
  1233. Surma:with monkey patches. And again, I get it, but
  1234. Surma:I usually don't want that. And
  1235. Surma:it just feels a bit weird
  1236. Surma:to be honest. But yeah, I'm
  1237. Jake:Mm.
  1238. Surma:trying not to question it because the benefits just outweigh my
  1239. Surma:desire for purity, I guess.
  1240. Jake:There's a sign that it has not been built for the other tools that we use on the web right now.
  1241. Surma:Yeah, maybe. And I think
  1242. Surma:I was kinda thinking
  1243. Jake:But, you know, maybe that could change in future.
  1244. Surma:couldn't they add a flag where I say
  1245. Surma:for this build step, don't use symlinks, actually copy them
  1246. Surma:or use heartlinks or
  1247. Surma:I don't know, use a custom file system driver where you can actually
  1248. Surma:layer multiple things.
  1249. Surma:The other gripe I have
  1250. Surma:but it's not really a gripe, it's just something that has
  1251. Surma:caused frustration that in Bazel
  1252. Surma:the input file cannot be overwritten
  1253. Surma:by an output file.
  1254. Jake:Mm.
  1255. Surma:So every file name globally
  1256. Surma:in your build pipeline is pretty much
  1257. Surma:unique or at least can't be used at the same time.
  1258. Jake:Mm.
  1259. Surma:And that doesn't sound too bad, but I have had it where
  1260. Surma:for example, CSS modules where
  1261. Surma:I wanted the output CSS file to have the same name as the input
  1262. Jake:Mm.
  1263. Surma:CSS file, but Bazel would error me saying, sorry, your input
  1264. Surma:and output have the same name and that doesn't work. So I had to do
  1265. Surma:filename.process.css
  1266. Surma:and then figure out how to rewrite the import
  1267. Jake:Mm.
  1268. Surma:to now point to processed or to point to the JSON.
  1269. Jake:Oh, interesting.
  1270. Surma:So that's just minor nuisances and
  1271. Surma:you can totally deal with them. Overall
  1272. Surma:I find Bazel really interesting to think about
  1273. Surma:really interesting to work with and I'm genuinely considering maybe
  1274. Surma:not to merge, but just to try it out if I can migrate
  1275. Surma:Squoosh to it because I want to see
  1276. Surma:what it's like if people wouldn't have to
  1277. Surma:worry about having Docker or anything installed and could contribute
  1278. Surma:to both the codecs or just the web app with the
  1279. Jake:Nice.
  1280. Surma:same build tool. Mostly maybe as an experience report and then
  1281. Surma:potentially writing a blog post just to contribute to
  1282. Surma:one more piece of documentation for Bazel
  1283. Surma:out there. There's obviously a lot more I didn't
  1284. Surma:talk about. For example, Bazel can model tests
  1285. Surma:and applies the same smartness to tests.
  1286. Surma:As far as Bazel is concerned, a test is just a build.
  1287. Surma:So for example, the rules for Jest break your tests
  1288. Surma:apart into all the files and it will only rerun
  1289. Surma:the suite from a single file if any of the
  1290. Jake:Oh, nice.
  1291. Surma:inputs files have changed, which is really
  1292. Surma:smart. So it really reruns the tests that have been affected
  1293. Surma:by your code changes, which is really
  1294. Surma:good for stuff like CI where you potentially want to
  1295. Jake:Mm.
  1296. Surma:skip unnecessary work, unnecessary tests,
  1297. Surma:and still get a valid green checkmark.
  1298. Surma:So that's really interesting.
  1299. Jake:Mm.
  1300. Surma:It has, as I mentioned only in passing, this really
  1301. Surma:detailed dependency graph and that is actually
  1302. Surma:inspectable. You can write queries
  1303. Jake:Mm.
  1304. Surma:almost like a SQL query. It's a different language, but you can
  1305. Surma:write queries to figure out is there
  1306. Surma:some way where this package is imported by
  1307. Surma:that package and if so, how? Because for example, a problem we've
  1308. Surma:had is where the lazy loaded part of the app
  1309. Surma:was accidentally statically imported by the
  1310. Surma:initial part of the app. Bazel would help you
  1311. Jake:Mm.
  1312. Surma:write a rule that basically could fail CI when it
  1313. Surma:finds a static import edge from the
  1314. Surma:initial part to the lazy part and could say you did a bad.
  1315. Jake:Mm.
  1316. Surma:Don't do that. You could just express that as a
  1317. Surma:query. You could also visualize it as a graph.
  1318. Surma:You can find if you have multiple versions of the
  1319. Surma:same dependency from NPM in your app, all these
  1320. Jake:Mm.
  1321. Surma:things become quite easy to express.
  1322. Surma:Bazel can do remote execution because it does
  1323. Surma:this whole hermeticity thing where everything that is
  1324. Surma:needed, including the tool, is encapsulated declaratively.
  1325. Surma:Technically, you can just throw that at a different machine
  1326. Surma:and run the build there. So if you have a very build heavy
  1327. Surma:app, you can scale it out across a network,
  1328. Surma:across multiple machines and get a very easy build cluster.
  1329. Surma:And lastly, there's a...
  1330. Jake:This is what happens with Chrome, right?
  1331. Surma:Yes.
  1332. Jake:This is when they're building browsers that's a similar sort of pattern.
  1333. Surma:I guess lastly, there's iBazel, which is
  1334. Surma:the file watcher for Bazel. Again, because
  1335. Surma:it can inspect this dependency tree and
  1336. Surma:knows every single file and where it is used.
  1337. Surma:It is very good at knowing what steps to rerun
  1338. Surma:when you make a change. And iBazel just does this
  1339. Surma:automatically the second you save a file.
  1340. Surma:And that integrates really nicely within a dev server like Vite,
  1341. Surma:which just picks up the output and serves it to your browser.
  1342. Jake:Mm.
  1343. Surma:That is my current state of Bazel.
  1344. Surma:I probably should have said at the start, I'm still
  1345. Surma:in my journey of learning. I'm still very much a
  1346. Surma:beginner, so this should all be taken with a huge pinch of salt.
  1347. Surma:But I had fun. I have fun thinking about it and
  1348. Surma:using it. And I'm glad I was given
  1349. Surma:a reason to learn it.
  1350. Jake:Mm.
  1351. Surma:Yes.
  1352. Surma:Well,
  1353. Surma:I wanted to point out
  1354. Jake:Mm.
  1355. Surma:that we have managed
  1356. Surma:to record more than last time.
  1357. Surma:Look at us.
  1358. Jake:Mm.
  1359. Surma:We set this at 10 minutes,
  1360. Surma:and then we are roughly at the same time
  1361. Jake:Mm.
  1362. Surma:in the chat.
  1363. Jake:Mm.
  1364. Surma:And I'm glad we did.
  1365. Surma:We're not any
  1366. Jake:Mm.
  1367. Surma:further along on
  1368. Jake:Mm.
  1369. Jake:Mm.
  1370. Surma:a new sign-off catchphrase, are we?
  1371. Jake:Mm.
  1372. Jake:Mm.
  1373. Jake:Mm.
  1374. Surma:That was good, wasn't it?
  1375. Jake:Mm.
  1376. Jake:Mm.
  1377. Surma:Should we just do that?
  1378. Jake:Mm.
  1379. Surma:Because I enjoyed it too.
  1380. Jake:Oh, at some point I'm going to suggest going back to the old one, because I miss it.
  1381. Surma:Happy next time!
  1382. Jake:See you next time. Bye-bye!
  1383. Surma:Oh, are you kidding me?
  1384. Surma:I'm back.
  1385. Jake:End of the podcast sorted.
  1386. Jake:Maybe just every episode can be you going to the door and Watson barking, so.
  1387. Surma:It's how we roll.
  1388. Surma:Ooh, HMRC. That's not...
  1389. Surma:I'm gonna open this live on podcast now.
  1390. Jake:Oh, someone's done a fraud.
  1391. Surma:A self-assessment statement?
  1392. Surma:This is just... I did my job, isn't it?
  1393. Surma:Yeah, I did. I did my thing. All good.
  1394. Jake:Oh yeah, I...
  1395. Surma:I have not pulled any criminal activities into my taxes.
  1396. Surma:Exactly.
  1397. Jake:Not that they've caught you for, anyway, so.
  1. Putting React In The Browser

    Putting React In The Browser

  2. Canvas-based Web Apps

    Canvas-based Web Apps

  3. The Apple PWA Ban

    The Apple PWA Ban

  4. TC39 Roundup and Bevy’s ECS

    TC39 Roundup and Bevy’s ECS

  5. The Big Build Bool Bonanaza II

    The Big Build Bool Bonanaza II

  6. WebGPU and Browser Ideologies

    WebGPU and Browser Ideologies

  7. From the Archive — Changing jobs, Deno, and optimizing animations

    From the Archive — Changing jobs, Deno, and optimizing animations