Transcript
Transcript prepared by
Adám Brudzewsky, Bob Therriault, and Sanjay Cherian
00:00:00 [Miguel Raz]
If you see someone, I'm preaching to the choir here, but like for some of my Julia friends that are maybe listening to this in the future, if you've never seen an APL-er or a person who is proficient in BQNs or J start solving a problem on an array language with their REPL, it is mind-boggling. They'll just start blazing through, crunching the numbers, instant feedback. They're like, "Oh, no, this algorithm is wrong." And then like, you know, eight, nine characters later, the problem is solved.
00:00:36 [Conor Hoekstra]
Welcome to episode 103 of ArrayCast. I'm your host, Conor. And today with us, we have two panelists, plus a first-time special guest who I am very excited to talk to. But first, we're going to start with brief introductions. We'll start with Bob, and then we'll go to Rich.
00:00:51 [Bob Therriault]
I'm Bob Therriault. I'm a J enthusiast, and I have to take the hit for this one. I probably screwed up on the timing, which is why Marshall might not be here.
00:00:58 [Richard Park]
My name is Richard Park. I teach APL at Dyalog Limited, among other things.
00:01:06 [BT]
Oh, and we have Marshall.
00:01:08 [Marshall Lochbaum]
I'm Marshall Lochbaum. I am the creator of BQN, and I've programmed in a few other array languages, too.
00:01:14 [CH]
And as mentioned before, my name is Conor, array language enthusiast at large, excited to talk about an array language that hasn't gotten much airtime in the first 102 episodes of this podcast. So, excited to get to that conversation. But first, we have two announcements. Technically three, but two you're going to hear from Bob. Then I'm going to do a short introduction of our guest, and we're going to hear a third surprise announcement after that.
00:01:34 [BT]
My announcement, my two announcements. [01] One is that J9.7 is in beta. I'll say beta, although it's almost alpha. It just launched. So, if you wanted to play around with the new J9.7. And the usual process for this is for the next year, we'll be working on it. And then sometime around December, January, it'll launch as the new version. So, the working version now is J9.6, and J9.7 is in testing. And the second announcement, if you've had a chance to look at the array portal, and it's very much in alpha beta as well. But it's a search engine for, at this point, APL and J, and it's really quite amazing. As I said, sometimes you go there and it's not working, and other times it's just working brilliantly because Ed's working on it. Anyway, the array portal is running, and if you wanted to test it out, if you're interested in learning more about APL and J, it's great. And the plan is to add future languages such as BQN, Uiua, and others.
00:02:41 [CH]
Which brings us to introducing our guest for today. He goes by Raz, and he is now going to be, I think, the third representative of one of the, what I would consider, and I think the panel considers, non-Iversonian array languages. So, back on episode 79, we had the trio of folks, led by Steve Wilcockson, talking about the MATLAB language. And I think Julia actually got talked about a little bit there, because one of the three folks is sort of moved into the Julia space. Then on episode 86, we had on Paul Teetor to talk about the R language. And now, we're bringing on Raz, who is, he goes on some of, I mean, I think he's on all three of the Twitters, Blue Skies, and Mastodons. I don't think consistently across all three you go by this, but at least on one of them, you go by JuliaREPLStan. REPL standing for read eval print loop. I assume 98% of our listeners are familiar with that. And you've been on a couple other podcasts. I was just doing a little bit of light research before this. I saw that you were on episode 15 of Julia Talk. And the title of that episode was "The Julia REPL is Amazing." So, my guess is that the REPL is going to come up, but obviously, we're going to be talking about Julia. And we'll throw it over to you for your announcement, but then maybe after that, we can get into your background and your path to the Julia language and why the Julia REPL is so amazing.
00:04:00 [MR]
Yeah. Thank you. Thank you all for having me, because many of you have answered my silly questions on programming forums. So, I'm very delighted to be here. The announcement is that Julia now has static compilation. You can distribute binaries now. And they're trimmed, so they're not like 187 megabytes. They don't include the LLVM library to compile everything. I believe it's going to be on Julia 1.12 as stable. It could be on 1.11. But you should be able to do it out of the box, so to speak. Yeah, it's not that you couldn't before. You just had to go through a lot of hoops to do it. Now, it's just like a single compiler flag, and you get your binary out. Native code generation has always existed in Julia. It's just distributing a binary was a pain. Now, it's just a command line away.
00:04:50 [CH]
No. See, I think this is unfair. I heard this first from Jeff Bezansen, one of the creators of Julia. People have been harping on the Julia devs for basically a decade. "Hey, when do I get standalone binaries?" And they're like, "My dude, you do Docker distributions for your Python setups." "Why is this a thing against Julia?" I guess because it plays in the space of usurping some C++ power that people expect to be able to be deploying it everywhere. So people get used to developing a package or a crate or a... I don't know if it's called a Python wheel, but it's like a standalone Julia code is called a package. It's very easy to turn your script into a package in a sense. And then that you just upload to the registry, and people will download and run it. And since there's a very good package manager story, it's really easy to replicate your code on somebody else's computer.
00:05:56 [ML]
But that's just distributing as source code, right?
00:05:59 [MR]
Yeah. And then once people will run it for the first time, it will get pre-compiled and all future invocations will run as they should with fast code. So yeah.
00:06:09 [CH]
Awesome. We'll pause on the technical talk and we'll make sure to put a link to... I assume that there's an announcement on the Julia site at some point. But we'll pause the technical. I mean, pause the technical. Maybe it's going to be technical from the get-go. We're going to say rewind until whatever age you want to rewind to. And tell us your story into programming, computer science, Julia. Maybe it was right from birth you were handed a computer. And so it is technical. We're not pausing the technical.
00:06:37 [MR]
I don't know. I think my first recollection of playing with a computer was playing with this game called Busytown on my parents' computer. It was like a Windows 95 thingy. It's like a learn-to-read game and you got to bake some cakes or knead some dough around this cool place. I didn't do much programming until basically college. My first experience, I remember reading... What's this guy's name? Learn Python the Hard Way? Over a Christmas break and then dropping it. Then I took a computational physics class. And then I had one of my professors, David Sanders, tell me, "You know, Julia is the future. There's no reason to keep using Fortran 77 as all the rest of your physics professors." And I went and I said, "Absolutely not." And so I went and took a summer course in Fortran. But afterwards, I managed to just go onto the Julia Slack. This was a completely new thing for me. And I'd never been on an internet forum where you could just start asking questions and people would answer. So if you come at this from a perspective of someone who studies physics, if you want to meet the people who are great and build stuff and have very fancy last names that everybody knows about, you have to work for, I don't know, 10 years to publish something. And maybe you'll get a glimpse of someone famous at a conference or maybe somebody visiting your university or something. Whereas with programming, I remember going onto the Julia Slack feeling like a hacker, like, "Oh, my God, I made it. I'm part of the secret cabal of Uber lead hackers." And I remember asking a question that was not more complicated than, like, "How do I do 2+2?" I was literally just futzing around with the first thing in the REPL for Julia. I'd never really done any serious coding except for turning in assignments. And this really weird person-- or this really kind person with a liking hat answered my question like, "Hey, welcome to Julia. Just click here. This is-- yep, 2+2. This is how you do it. All right, see you again." And I'm like, "What a nice man and what a silly hat." Well, that man with a silly hat was Stefan Karpinski, co-creator of the Julia language, and he was just answering newbie questions. And this is a regular occurrence. I've had questions answered here by Marshall on the BQN Discord and by other people, and it's astounding to me how when it comes to contributing to open source or computer science projects, you can just go onto an internet board, and the person who built it's there answering questions lots of times. So it was like this disconnect between like, "Oh, you can start contributing. You can fix a typo." That's a contribution, right? Versus like 10 years of not even getting a PhD finished. So yeah, that's how I basically got started into asking lots of silly questions. I didn't know what a string was. I didn't know-- I still don't know a lot of things about strings, but--
00:09:33 [ML]
Does anybody know what a string is? Contact@arArrayCast.com if you have the answer.
00:09:40 [MR]
Yeah, let us know. But I was very fortunate that there were people who would just hang around in chat and answer my questions, and I'd try and solve problems. And they'd come back like, "All right, yeah, you did it, but what about if you did it this way or this other way?" So I had a very-- what's it called? Apprenticeship sort of way into it. I've been lucky enough that strangers on the internet have taught me almost everything I know about programming, and I like to give it back where possible.
00:10:07 [CH]
You said that initially you had that professor that said, "Julia is the future," but you rebutted and went and took a Fortran course, correct? Yeah. So at what point did you decide that maybe, "Julia, I should take a look at this"?
00:10:22 [MR]
I took a Fortran course because there's a very famous physics professor here in Mexico called Miguel Alcubierre, the famous one from the Alcubierre Drive. If you happen to be that kind of nerd, then it lets you travel faster than light or whatever. So incredible scientist, very cool human person, very kind, did that paper as a sh1tpost and will not stop living down its fame. He gets stopped at conferences due to-- he's a very serious physicist, and also, the other respect, he was just watching Star Trek, and he was like, "Well, how would I make the warp drive work?" And he came up with something ludicrous like, you know, there has to be negative mass and all that stuff. He's the one who said like, "Hey, I do numerical relativity. I like to solve for black holes on a computer. That takes years and years of specialization, and if you want to get into my branch of physics, you have to learn Linux, English, and Fortran." And I said, "All right, I know English. I know how to do CD on Linux. I have to go and learn Fortran." And that's why I started with that. And now that I see the state of Fortran codes and what you can do in Julia and be faster, I have no need to ever go back to Fortran.
00:11:28 [CH]
I mean, I think this is the-- we won't bash on Fortran anymore.
00:11:32 [MR]
No, no, no.
00:11:36 [CH]
Because Lord knows we'll probably have some-- or do you want to take a couple seconds to--
00:11:38 [MR]
No, so like many of my gripes with Fortran would go away if they had the modern package management that they now have, right? Or like the FPM. So like it's a completely different experience. Tooling and ergonomics for an ecosystem is such a difference when you're handed down some inscrutable master student's code from seven years ago that worked on some laptop versus like, "Oh, okay, here's a proper package. This is how I started. Tests can run. This is how you know things didn't break, and this is how you add other people's dependencies, and it's all well and done." And you're like, "Oh, it's not-- it could be worse," right? I mean, Fortran will still be Fortran, but it doesn't have to be the worst version of Fortran possible.
00:12:21 [CH]
Yeah, modern Fortran is-- it's a far way away from modern languages, but it's also a very far away from Fortran 77, the way you used to have to do things. I've only written a tiny bit of Fortran, but I had some folks once again on the internet that were like, "Oh, you can actually do this, this, and this," and it still didn't look amazing at the end, but it looked a lot better than what the average Stack Overflow post was telling me to do. But so, I mean, I think now we've been chatting and announcing things for, you know, 10 plus minutes or so, but the reason we're having you on is because you're a massive Julia fan, and I can't remember whether it was on one of the Twitters or Blue Skies, or it was via email or something, but at some point I recall you saying, "I feel like-- I can't remember if it was me specifically or the ArrayCast panel in general-- hasn't given Julia a fair shot in that." I'm sure if they just heard what I had to say, they would love it.
00:13:18 [MR]
I've been thinking that ever since I heard about this podcast. I feel so vindicated to be here, and I hope I do it justice.
00:13:27 [CH]
Yeah, so what is the Julia pitch? What are we missing? Convert us all. I'm ready to be converted. I'm looking for a bandwagon to jump on at every opportunity.
00:13:35 [MR]
So, it's not a personal dig, but if you know anything about Conor, he has a lot of really cool videos on YouTube [02] where he's like, "Hey, let's do this lead code problem. Let's do it in Haskell, Python, C++, there's a chance for Rust, and then the APL way." And I'm like, "Great! That's awesome." But there is this trade-off of how succinctly can you express an idea, and how fast it is, and how quickly you can iterate on it. And basically at the core of Julia, you have that choice. People came from MATLAB, and Fortran, and C++ to solve hard data-crunching, numeric scientific problems, and they said, "We can either start baking our DSLs for incredibly specialized problems that come back to a fancy language, or come back to a compiled language, or we can just start playing around in Python." And so your code is either going to be fast, or it's going to be easy to program in, or it's going to be compiled, or it's going to be interpreted. And the Julia people basically looked at this state of affairs of closed-source, non-extendable DSLs for very specific scientific niche problems that would never generalize. So part of the problem with many of these different-- So there's specific languages that got developed for solving problems in astronomy, and in genomic sequencing analysis. All of them always ended up just sinking down towards their niche because they never had the tools of a general programming language. You never had a, "Oh, I have to download something," or "I have to process some files." MATLAB got strings not so long ago. It's really painful when you have a full stack of something that resembled a data science application, and then you have your front-end JavaScript and then your NumPy code, and then you have to do some string processing in Perl, and then something else. I think Stefan Karpinski worked on Etsy for a while, and to do some of the data crossings, he called it a Rube Goldberg of all the different numerical dependencies you could have. And so it's not that they came up with the idea of one language should rule them all, but rather, we deserve better. There's a sort of shared ethos with the Rust crowd, where they just looked at 40 years of C++ people ignoring safety concerns and bad tooling and ergonomics and just saying, "We can do better than this." And they really went for the greedy approach. They went with, "We want it to be fast. We want it to be interactive. We want it to be extendable and generic. We want to have our codes run anywhere, and we want it to be a general programming language. We don't want it to just stay stuck in a niche where it's only doing, I don't know, statistics and only that. We want it to be full-fledged. You can do anything you want to."
00:16:39 [BT]
And that's why it would be really strong in the package management, right? Because it's got to be able to handle that kind of stuff.
00:16:44 [MR]
Yeah. I don't think I can do Julia justice to the extent to which you can compose with other people's code. But suffice it to say, it is a regular happening that people will-- So you can have different sets of developers where one writes a differential equations ecosystem, one of them writes a plotting library, one of them writes a library to handle intervals for uncertainties, and one of them writes a statistics package. And every single one of those packages you can compose to have your differential equation also use uncertainty intervals and then plot that and have it work accordingly and "have it just work." And none of those developers had to talk to each other for those interfaces to work, in a sense. So the amount of code that I get to reuse when I'm working with Julia is bonkers, in the best case. I'm not going to say it's free. Sometimes you do run into issues when you're a library writer of like, "Crap, I don't know that I can assume that this interface is written out this way." I don't know that people have figured out the problem of how do I have fast generic code that isn't tied down at compile time and yet have some guarantees about how much you can extend the interface. We can get into that later if you want. But Julia falls much more on the side of, "Let's let people extend however much they want." And still get fast code.
00:18:14 [BT]
And how do you find the packages? So in other words, if you've got a really easy way to get the packages out there, they work with everybody else, are there a lot of packages? Do you have to sift through a lot of things to get them?
00:18:34 [MR]
So I don't know about other people, but I happen to be at the point where I happen to know a lot of people in the ecosystem like, "Oh, this person is good for... This is the guy who wrote the DataFrames package. This is the person who knows about plotting. These people know about differential equations. These people know about DNA sequencing." So I'll just go up there like, "Hey, who do you know is doing this?" If it fails. Otherwise, if someone's like usually... I don't know. Somebody at some point was like, "Oh, by the way, do you have an algorithm to calculate the Voronoi?" Whatever it is. And I'll just go, "I don't know." And I'll just Google Julia Voronoi blah, blah, blah GitHub. And then up it comes. And I'm like, "All right." And then I'll see it. I'll scope it out. I'll see that it's like a serious package. It has proper tests and documentation. I'm like, "Yep, this looks like exactly what I wanted." We're nowhere near the point, obviously, of the Python ecosystem where you can just expect to Google and copy the first three lines from the most voted Stack Overflow post and have that work. You did need a little bit more of an adventuring side to get started with Julia when I started. But part of the pitch is that you can have this tradeoff between you can be flexible and have native code if you're willing to pay a price every time that you compile a function for the first time. I think the cooler term for it is that it's a just ahead of time compiled language, not a JIT compiled language. But it's also not a JIT in that most people know a JIT like JavaScripts. Because those are profile-based JITs. If you have a really hot piece of code, there's a bookkeeping that you can do to say, "Oh, these lines in my interpreter get called more than 10,000 times per second or something. So I'm going to go ahead, compile those, re-inline it, and then that's going to streamline this particular code path." Julia gets compiled and cached on type signatures. So it's a data flow analysis. There's no extra--there's no runtime bookkeeping in that regard.
00:20:33 [ML]
Yeah, I mean, so if I can clarify how I understand it, it's basically like you call a function-- so every--Julia semantically is dynamically typed, right? I mean, you have the types in the source code, but those are just constraints. So it's basically a dynamically typed approach. And so you can call functions with all sorts of types. And then when you call the function, it checks the type, says, "Do I have a function? Do I have a compiled version?"
00:21:01 [MR]
Yeah, is it in cache? Is it not? Let me go compile it.
00:21:04 [ML]
I don't see what the issue is with describing that as just in time. That makes perfect sense to me. Oh, now it's time to call this function on these types. Well, I'll compile it just in time. Yeah, yeah. I get that historically it's been used differently.
00:21:17 [MR]
It's more like the characteristics of what a cold start and a hot start is for that particular JIT is very different from what people expect. Because most of the--people mostly understand JITs as coming from the profile-based tradition.
00:21:32 [ML]
Yeah, well, I mean, particularly with Java, the advantage of JIT was sold as being able to get runtime feedback, which is--it's unclear how much that advantage even exists. Yeah. But yeah, so then you have a dynamic compiler that doesn't get runtime feedback. Now what do you call it?
00:21:47 [MR]
And so that would be like your first day of Julia. Once you get to the third or fourth day, or maybe the second day, people will be like, "All right, we're not going to pay the compilation cost on every invocation. Every time we restart the program, we are going to look at--like if you format your code into a nice package, then we can introspect and say, "All right, all of these function calls, we can just go ahead and precompile ahead of time." So there's a whole set of tools to understand how your code and how much of your code gets precompiled. And yeah, there's a growing body of work on speeding up your compilation times, on not doing--on not getting your cache dirty, in a sense. We call it invalidations. So if you have new methods that overwrite the old ones, you have to go and backtrack and redo a bunch of that work. And there's sloppy ways that you can code in Julia that will just make everybody that depends on your packages cry because you cost their compile times to skyrocket.
00:22:50 [CH]
So the pitch here, essentially, is that Julia is a language where regardless of the way you want to program, we can accommodate that. There's something for everyone, correct?
00:23:02 [MR]
Up to a point. I mean, if you want to do like, you know, embedded--like everyone has their niche, right? If you wanted to write the super most concurrent system, go with Erlang, you know, for that particular type of concurrency. If you wanted to write something incredible for embedded systems, and you want to have a lot of safety guarantees, go with like, I don't know, like Ada or Rust, like stuff that's similar to that. For Julia, I would say, if you are crunching problems that--where you think-- where you think your laptop can be faster than somebody else's cluster, then go with that, right? The elevator pitch I have for it is, looks like Python runs like C, and then if you really want to be a hipster, it feels like Lisp. Some people call it like a Lisp with square brackets, but-- Coming back to your question of like, why do you feel the need to include-- or to talk to the rest of the Iversonian kids, you know, in this space? I think what really drew me to the Array space is that-- so I'm on a bunch of social media as the REPL stan. I basically had to choose the REPL stan because I wanted to like, you know, kind of like get my foot in the Julia ecosystem, but it's very hard to like call yourself an expert on something when everybody else is like incredibly talented and smart. And I was like, well, you can just like something a lot, so I'll just like the REPL a lot. And so you end up--like the facility with which I can get my problems done in Julia is remarkable. And what really drew me to Array language is I think--I think it was one of your videos, or maybe Aaron Hsu's, [03] where they were like, all right, we're just going to start crunching some numbers here, and we're going to do a graph, like whatever. And they started typing away at the REPL, and they were solving problems 10 times faster than what I was being able to solve. Like, if you see someone who--like this is--I'm preaching to the choir here, but like for some of my Julia friends that are maybe listening to this in the future, if you've never seen an APL-er or a person who is proficient in BQNs or J start solving a problem on like an Array language with their REPL, it is mind-boggling. They'll just like start blazing through, crunching the numbers, instant feedback. They're like, oh, no, this algorithm is wrong, and then like, you know, eight, nine characters later, the problem is solved, whereas I'm looking up the name of the freaking function. So I now have this inclination to think of when someone's like two times better than something at me, I'm like, well, you know, they're better. When someone's like 10 or 100 times better, I'm like, they have a better system or a better tool, right? I can learn that. And so that's what really drew me to Array languages in a sense. I also feel that like, because Julia has this like really deep interest in linear algebra, that it's also really trying to push the envelope on how much bang you can get out of your buck on just knowing how to use matrices well. Like not a lot of languages ship with like multi-dimensional array support as like the first class concern when it comes to like generic program language. I mean, Rust only has vectors, right? And there are creme de la creme in many other respects. But, you know, linear algebra is freaking hard. You need a lot of math people to really come together and get the abstractions right. There's a lot of very thorny issues that are not at all easy to figure out at first, or even after like many years of discussion. There's a very famous GitHub issue, I think it's called, it's 4774 in Julia. And it's about how we represent row vectors versus column vectors versus row vectors, right? And if you, I mean, there was even a paper that came out of this, I think, which is if you try to multiply a row vector and a column vector, you're supposed to get out a scalar at the end. And in MATLAB, you can only get a one by one matrix, which is not the same as a scalar, but which does get coerced. Well, in MATLAB it kind of is. And so the thing is, there is no representation for a scalar in that type system, I believe.
00:27:28 [ML]
Yeah, I mean, MATLAB's concept is just, we'll only use matrices. And consequently, it works if you only use matrices. Right?
00:27:39 [MR]
This problem ended up being even traced back to householder notation itself for linear algebra. How do you actually denote these different types? And how do you know that when you're manipulating them, you're actually getting out the mathematical types that you expect from these operations? And you just end up really special casing a lot of different hairy logics if you're not very, very careful with choosing your semantics.
00:28:03 [ML]
Yeah, and I guess the approach we take to that in APL, of course, we don't have the row and column vector difference. But we just say both of those are just lists. And so matrix multiplication, if you put the list on one side of the matrix, that'll still multiply right. But if you have two lists, you need to decide whether you want to do a dot product or an outer product when you apply them. So it's less like the notation is no longer. And this was Iverson's whole idea to get the ideas of mathematical notation without being tied to this specific notation we use. So he said, yeah, it's fine if you have to change the way you write things, because then you'll be able to use a better notation and get the benefits of understanding it or whatever.
00:28:52 [MR]
I really appreciated that from array languages like the, hey, you want to write the hamming distance? That's just three characters. When I first saw that code, I was like, this is mind boggling. There's a close-ish equivalent to doing the map reduce of not equals and then doing the sum reduction in Julia. And you get really, really fast code. I really love to do the demo of hand coding the hamming distance as you would write it in Python on your first day. Then with iterators, then as a one-liner, and then just by calling the map reduce on the not equals plus and then on two collections. And you still get out really fast code. To come back to something that Conor asked at the beginning of what really drew me into Julia, when I kept coming back to the boards and talk with people or the Julia Slack, people would always be -- there's even a channel now called Performance Help Desk. And so people would be like, hey, I have these strings and I have to count how many characters, blah, blah, blah. Or I have this matrix and I have to figure out how fast this goes. I can only do it in 20 milliseconds. Can someone do it better? And someone will just be like, yeah, I got it down to five milliseconds, right? And they'll share their code. And to benchmark your code in Julia is also really trivial, like less than a line of code. But also to look at the generator assembly is also really trivial, like less than a line of code. So getting -- there's like a niche community of like perf hackers that Julia really attracted. Like the speed junkies really were drawn to it. And I really like that part about just someone shares a problem, then everyone tries their hand at it on different ways. And then you see which one's the fastest. I've seen very much of the spirit also in the APL and the BQM discords. It's like very much alive in that tradition. I guess I only first experienced it when I came to Julia. And I've not really seen that in other programming language discords as far as my experience goes. Like you really have to be invested into, all right, I'm going to do like the 30 lines of setup to benchmark my code and see. Or even like, I don't know, five or ten in Rust and still be inclined to do that.
00:31:14 [BT]
Yeah, one of the things I've always been impressed with with the array languages is there is always a strong sense of community. And I've always kind of thought it's because if you come to the array languages from other sort of more procedural languages, it's a very different world. And most people have at some point, I mean, Adám Brudzewsky excepted, have not grown up in the array languages. So at some point they've had to learn how to think differently. And so everybody knows what that feels like. And there's a lot of coaching and there's a lot of helping on the early stages because you know you've been there and you felt that way.
00:31:48 [MR]
And I guess in that sense, like Julia really, I'm weird in the way that like Julia was like the first real programming language that I got invested into. And so now everything that I look at, I'm like, I think of it in terms of Julia. Like the reigning paradigm in Julia is this idea of multiple dispatch. Like, hey, if you can just compile fast code for any type signature, why not dispatch on that too? Right? And code the different behaviors into the type system and then just have the compiler figure out which one to call accordingly. So that means that you can write the same code like, hey, accept this collection, then process it, and then at the end do some cleanup. You don't need to rewrite that first base code. You can just write the generic functions that call like on the collection and all that stuff. And then afterwards you can define new type signatures for those functions that will just work on your special new types. So that's, once you start to see, there's a really cool talk by Stefan Karpinski called "The Unreasonable Effectiveness of Multiple Dispatch". And it's like one of those quotes where, you know, every large enough code base has an ad hoc, sloppy, informally specified reimplementation of half of common Lisp. We have some of that. We have the same thing, but for multiple dispatch at some point. Right? It's just someone figures out we want to do the same thing. We want to do the same behavior with different types at some point. And it's not that Julia's, it's not that other languages don't have dispatch. From my investigations, common Lisp had multiple dispatch in the 80s. Like there's a talk, there's this famous course by Sussman and Abelson on Lisp from MIT. Right? They talk about multiple dispatch, but they spent, the system that did the bookkeeping was very inefficient. So it didn't keep up with the times. That's kind of like a happy marriage of LLVM's compiler and JIT technologies had to come together for this multiple dispatch system to be worth it. Also, Jeff Bezanson had to kind of write a super new cool algorithm to figure out, oh, you call this with a type. I don't have that type. I have a more specific one. Is that the one you meant? So that's called the subtyping algorithm that goes and figures out which specific method you have to call. And it is highly non-trivial to understand or implement. So much so that it got published and that's his thesis, amongst other things.
00:34:31 [ML]
But so, I mean, you're talking about just purely the performance implications here, right? Because, I mean, like these are all dynamically typed languages, as I said. Like the view that we have of this in APL is, I mean, all right, you're writing a function that's going to take a list, maybe takes two lists and zips them together or something like that, interleaves the elements. Well, I mean, you write the function, your understanding is, well, the arguments are lists. So what's the type of the arguments? Well, I mean, it's a list of something, but you just don't care. It's a list and each element is dynamic. And, I mean, then if the language happens to have a particular type in the list, it's the language's job to optimize that. But, I mean, that's really not visible to the, like the programmer doesn't have to care until they start optimizing their program.
00:35:24 [MR]
Yeah, yeah, yeah. And it's funny because I think I remember seeing this talk by Aaron Hsu on the interpreted advantage for APL.
00:35:31 [ML]
Ohh no that was me.
00:35:33 [MR]
Oh, there you go. You. It rhymed a lot with what I knew from Julia, like, oh, you don't have all the information to specialize in a specific algorithm ahead of time sometimes. If you can look at it just before you run it, then you can take smarter decisions. Well. If I recall correctly.
00:35:54 [ML]
I mean, you could get the advantage that way. But, I mean, what we do in APL [04] is that we have a small enough number of types that, yes, we can actually just compile all our primitives for every type. And that's the interpreter. Yeah. So, and I mean, the big thing about that is that not only. So, with just in time, you've got a system that automatically specializes on type. When we're manually specializing on type, we can do things like say, all right, the user asked us to sort a one byte type. Well, I happen to know that the best way to sort something that's one byte long is to just do a do a counting sort instead of doing a comparison thing. Just have count up all the the number of each value there are and then write them out in order. So, because we know all the possibilities and I know that Julia actually does have this sort of sorting optimization. But basically, because we can do this for every primitive because we know every primitive you could use and every type you could use them on. So, in that sense, I mean, an automatic system can also handle everything. But it doesn't necessarily handle everything in the best way if the space of what it could cover is just too large for to be able to investigate all the possibilities.
00:37:13 [MR]
Right. And to give BQN even and APL a lot more respect in that sense. I really wish that Julia people would. So, basically, how I would sell array languages to Julia people would be you would not believe the speed with which they manipulate bit matrices. And bit vector algorithms are just optimized out the wazoo. And we don't have that inclination in Julia yet. Like we haven't, Marshall, I don't think we can nerd snipe you into contributing those algorithms into Julia or someone you know. But if we got someone to do that, it would be so, so cool. So, I remember trying to encode at one point the NOR scan, right, for a specific algorithm or some problem. And it turns out that the scan implementation will only give you back the return type of the initial collection. You can't say, oh, there are 64 elements to start with. Please give me back a 64 bit vector. At least not with the ones in the standard library.
00:38:23 [ML]
Wait. So, are you trying to scan on input types that aren't Boolean?
00:38:29 [MR]
So, I want the result to be a bit. If I want to use those.
00:38:38 [ML]
So, you take in like a vector of bits, but that's not of the bit vector type. Yeah. Yeah, okay.
00:38:44 [MR]
It's not that you couldn't in Julia, it's just like there hasn't been someone that has gone through and said.
00:38:49 [ML]
Said yeah. Well, I mean, this is one of the nice things about.
00:38:52 [ML]
Yeah, well, I mean, this is one of the nice things about saying, you know, there's all we've got are numbers and you can put them in a list if you want. There's just no distinction in APL because, you know, a list of numbers, if the numbers are Booleans, then representing it as a bit vector is valid. And so, yeah, you can choose to do that whenever you want.
00:39:09 [MR]
So, Julia has an interesting problem space as a design because you want to stay generic through all of these layers of your standard library and you can rely on the JIT compiler in the end. So, if you were in Python, you could do a bunch of code like, hey, I'm going to put these things into a dictionary and then I'm going to call the C subroutine just to add them up. Right. That's a well-known code path. Whereas in Julia, you'd have like, you know, maybe 17 different indirect calls for like the specific type of generic container on which you are operating on and then do the addition at the end. So, if you do not, this is important because it makes debugging Julia a very, very slow experience because you're going through all of these intermediate mechanisms that, you know, get optimized out in the final version of the code, but not in the debug or interpretive version, so to speak. So, when you're trying to read like some base library Julia code, there's a lot more jumping back and forth than you would expect at first to handle all of these different constraints.
00:40:24 [BT]
So, does Julia use the 1-0 true/false for like the Iverson languages do?
00:40:30 [MR]
No, only true and false. There's no truthiness in Julia.
00:40:34 [RP]
I mean, maybe that goes partly to explaining why it's not so heavy on the manipulation of bit vectors. That's bread and butter APL.
00:40:43 [MR]
So, the cool part is that you could just define your, I mean, you could overload the bit vectors of the bit matrices and have a lot of operations on that.
00:40:54 [ML]
Wait, so I'm not sure if, so are you saying there is a separate Boolean type or there's not? There is a separate Boolean type. And the things inside a bit vector are Booleans?
00:41:01 [MR]
The bits inside of bit vector are, no, no, no, they're zeros and ones, yes.
00:41:11 [ML]
Okay, so they are numbers. Yes. Wait, but there are tons of numeric types, which is, can they?
00:41:17 [MR]
So, maybe we have like conflicting definitions here because like, so when it's called, so the specific name of the type is called a bit matrix and so that will use for 64 bits, like a single 64-bit integer, right? Yeah. But if you want to use a Boolean matrix, then it will use a byte for each one of those entries and just use the true or false.
00:41:42 [ML]
Yeah. Okay. So the bit matrix, like strictly speaking, it's, I mean, I guess you can interact with it as a collection, but like the, it doesn't have a particular type for its elements, does it?
00:41:55 [MR]
It doesn't have a particular type for its elements. I don't think so. No.
00:42:01 [ML]
Okay. Yeah. So it's a, it's only a collection type. There's no, I mean, if you, you can get a bit, but you can get the bit in whatever type you want to get it in, I guess. Yeah.
00:42:12 [MR]
I think that is correct. At the end of the day, I just see a lot of, so I remember seeing, I think one of your talks where it was like, hey, we're going to do a bit matrix transposition. And then we're going to start going block by blocks. When I saw that talk, and then I went and saw, I went into the Julia-based library to see if we had any of these like super specific, I don't know that super specific, but like more specialized algorithms for bit matrices.
00:42:40 [ML]
These days I called them absolute.
00:42:41 [MR]
Obsolete. Oh, interest ing. What would you use?
00:42:45 [ML]
I would use the modular permutation now. I have this really cool diagram that.
00:42:50 [MR]
Oh, I'd be very interested to hear that.
00:42:53 [ML]
It is harder to write.
00:42:55 [MR]
So my takeaway is we need to catch up. Like, I'm very much of the idea of like, if you go and see what the people with the best tools are building and how fast they're going, you should steal all the cool ideas, like no matter where it's from. And I really think that at least for a bit matrix manipulations and bit vector manipulations, we should really start stealing everything from the array languages.
00:43:21 [ML]
Well, so the thing on bit matrices is that actually in BQN right now, our one byte transposes so fast that we just convert to one byte and do a SIMD transpose on that and convert back. So what I'm referring to is that I've found better methods to do pretty much everything that I presented in my talk.
00:43:44 [CH]
When are you giving the updated talk then, Marshall? We're all living in the past of your discarded ideas because you haven't given the 2.0 talk.
00:43:52 [ML]
Now I do implementation notes pages, which are far drier and less conversational.
00:44:00 [MR]
Yeah, I hadn't seen this part of the BQN docs, actually.
00:44:04 [ML]
Yeah, so this is within the implementation notes that are kind of-- I mean, it's basically where I describe the ideas that go into CBQN either before or after we do them. So in kind of terms that it's pretty condensed.
00:44:23 [MR]
All right, I'll gladly read them later on.
00:44:27 [BT]
Is anybody else using those techniques, Marshall? Or is that something your BQN and CBQN are using?
00:44:34 [ML]
Well, the thing that I linked, [05] which is for transposes with one dimension is short. I mean, I came up with that a year or two ago. So I think that's just us. It's very rare to even do kernel transposes, which is kind of strange. So the idea of a kernel transpose is that you have-- you don't want to just pick out one element at a time from the argument and put them in the result because that's-- like, loading a single element is a full instruction, and that's pretty slow. So what you do instead is load them into vector registers in groups of-- so you'll take, like, a slice of each row from the argument and then transpose that somehow in registers, which can get pretty complicated. And then you write a slice of each of those to the result. And the fact that you can do this is pretty well known. There's even, like, in Intel's headers for their SIMD intrinsics, there's this really weird macro that's, like, for specifically transposing a 4 by 4 matrix of 32-bit floats in SSE registers. Only in the 16-byte registers because they didn't update it for the 32-byte ones because they presumably figured out it was, like, really strange to just provide this one exact pattern. But basically, the idea of how you do this is pretty well known. But it's-- like, I think the Fourier-- fastest Fourier transform in the West, FFTW, uses this method. I saw it somewhere else. But, yeah, I mean, it's fairly rare. Like, I don't know any programming languages that give you access to this directly.
00:46:23 [MR]
So not to flex or anything, but Julia has been very fortunate that FFTW author Stephen G. Johnson is an MIT professor. And at some point in the Julia early days, they came up to him and was like, "Hey, this is Julia. It's a cool project." And he took it for a spin. And he came back and was like, "All right, your language is doomed to fail unless you get going with this problem and this problem and this problem." And then he started working on that. And what a trooper. Stephen Johnson wakes up just about every Saturday morning on the Julia boards and answers everyone's questions, no matter how, you know, complicated or easy they are. He's, like, one of the best-- he's one of the good ones. So he's consistently been one of the people who answer questions on numerics and linear algebra and optimization. And he's been working on Julia for a long, long while. We've been very lucky to have people like Stephen Johnson giving away his time and expertise to the Julia project.
00:47:26 [ML]
So, yeah, I mean, I haven't checked Julia's transpose. Maybe they do have it. But I think one of the issues you might run into in Julia is that it wouldn't-- transpose, like, wouldn't necessarily be written as just a call to the built-in. Like, you might write a for loop in a different order or something. So at that point, the language has to recognize that it's a transpose, pull out the transpose, and say, "Well, this part needs to be handled using a different loop."
00:47:53 [MR]
Right. But see, once again, if someone has faster arrays in Julia, you just need to say that on some of the boards. And then, like, the nerd pride will kick in. Someone will come in and say, like, "No, we can't have this for longer. We need to do something about it." Nerd pride is an unending fuel source for Julia development.
00:48:12 [ML]
Yeah, well, so they might have-- even if they have good transposes on matrices where both the side links are large. See if they have good transposes when one side link is short, because the latest version of CBQN has those. And that's a pretty tricky method, so I'd be surprised if anybody else has it.
00:48:33 [MR]
I'll start from-- I won't claim that Julia is the fastest on any one case. I will claim it can catch up.
00:48:42 [ML]
Oh, yeah. Yeah, yeah.
00:48:44 [MR]
And they'd be happy to take a pull request in that case.
00:48:48 [ML]
Although one thing is it does depend on how the programmer writes their code, right? So, I mean, if the programmer writes the code where the transpose is recognizable-- like, if the programmer knows, understands the language and is able to use what the language provides, then they get it. But, you know, in a language like Julia that supports completely general scalar stuff, if you write, like, a weird loop, it's much easier to write things that the language doesn't understand when you have all that flexibility.
00:49:20 [MR]
And I will say, like, there's such a boom to-- I didn't know that I was sold on it before. Like, how did-- someone phrased it like, "A good programming language will teach you new ways to think," but also--
00:49:33 [BT]
Oh, Perlis.
00:49:38 [ML]
Well, yeah, Perlis said the contrapositive or whatever. If a programming language doesn't teach you new ways to think, it's not worth learning.
00:49:46 [MR]
There you go. And that's kind of been my approach. So, I started with Julia, then I saw Rust as a good complement, and then I saw, you know, the craziness that was going on in array language. And I was like, "I have to know what they're up to." But there's so many places where this-- you are so constrained into only using arrays that you have to reformulate your problems into amenable solving techniques. It's so cool that many of these techniques are making a comeback to SIMD processing, to GPU processing. I keep seeing talks now, I'm like, "Hey, this is data-driven design now." And I'm just like, "Those are just arrays. You are thinking with arrays." I'm glad. I keep trying to-- I want to port Aaron Hsu's PhD thesis of tree manipulation into Julia.
00:50:42 [ML]
Not the specific code, but all the techniques.
00:50:45 [MR]
Yeah, yeah, yeah. It's really hard to pin down a sensible workshop where you get the code and the examples running. As someone who's not that familiar with APL yet, that Aaron Hsu's worked on. Because I think it's Andrew Kelly from Zig that's also like, "Hey, we had this very different parsing infrastructure." We did like-- we would be traversing a tree and have all these point and directions versus we just linearized our data structures. It's now all just basically arrays, and it takes so much less time and so much less memory. And I'm like, "Oh, yeah." And then there's the Parias people, I think. It's a Futhark-inspired variant that does parsing-- it does compilation fully on the GPU, I think.
00:51:35 [ML]
Well, it's implemented in Futhark.
00:51:36 [MR]
Yeah. And see, that's another paper where I'm just like, "Someone's doing compilation on a GPU." Clearly, they have a different tool to approach this problem. Otherwise, it'd be intractable.
00:51:49 [ML]
Well, they had to simplify the language, too.
00:51:53 [MR]
Yeah, yeah. Still, as a concept, I'm amazed by it.
00:51:59 [CH]
All right. Well, I was going to say I can rewind the stack because I've been just listening, but have had a question for a while. That goes all the way back to multiple dispatch, of which I probably know one of my good friends is going to just DM me once he hears this. And he'll be like, "Here's an article. This will explain it." I don't think I've actually ever fully understood multiple dispatch. I think the topic of polymorphism is, in general, widely undertaught. You go to school and they teach you polymorphism in some course that teaches Java. And they say, "Oh, it's just inheritance." And they show you a diamond. But there is a double-digit number of types of polymorphism. The two main categories are dynamic and static, like runtime, compile time. But even within those, there's a ton of different variants. Ranked polymorphism is one that gets mentioned in our community, but not in other communities. And when I hear multiple dispatch, it seems most closely related to parametric polymorphism. I always have to bring up the Philip Wadler paper, which I actually did bring up earlier, which now I have too many tabs open. Am I going to be able to find it? But there's parametric and ad hoc polymorphism. This is from the "How to Make Ad Hoc Polymorphism Less Ad Hoc" by Philip Wadler in 1988. I think this is actually the paper that they introduced type classes in, which is why it's one of Wadler's most well-known. But it says, and I actually brought this up in a meetup the other night, "Ad hoc polymorphism occurs when a function is defined over several different types acting in a different way for each type." And then the example it gives is multiplication across integer types and floating point types. And then parametric polymorphism occurs when a function is defined over a range of types acting in the same way for each type. For example, length works on arrays, strings, et cetera, and gives you the same thing. So where does multiple dispatch fit into the world of -- from what I understand, I know it's at runtime, so it's not -- it may be superficially similar to things like function overloading or compile time, dispatch, and C++. But it happens at runtime. But every time -- which admittedly it's only been like once or twice that I've tried to fully understand, because I've heard that multiple dispatch, similarly to how you mentioned that talk that is widely known in the Julia community, it seems to me that multiple dispatch is one of the killer features of Julia. Yet I have failed to fully comprehend.
00:54:24 [MR]
And I failed to sell it, too. I'll take the blame on that.
00:54:33 [CH]
It's not you, though. I've looked at a couple different -- and I guess actually the last time was before the advent of LLM, so maybe I should just go sit down with my friend Claude and ask it to explain, and I'll end up understanding. But why is it considered this killer feature that is better than different types of polymorphism across other languages?
00:54:53 [MR]
So here's what I do know. In object-oriented programming, [06] you look at the world in terms of types and which operations belong to them. So if I have the type foo, and then I -- as part of this class, I give it the plus, then I can do a plus between two different foo types. But if I have a foo and then a bar type, I don't necessarily know who it belongs to. There's a really cool phrase by -- it's Harold Abelson, I think, on this same MIT course, that once you have functions as part of types, you have to really worry about identity. Like, you have to worry about who owns the operation. And in the most simple sense, the operation plus does not belong to the type on the left or the type on the right. If I ask you to add two integer numbers or to add two fractionals or to add two matrices, there is a sense of what that function does in each of those cases. So the idea of multiple dispatch is that you have this notion of there is a plus operation, but depending on the types of things that you feed it, it's going to do different things. And what's really killer is that you can end up writing your code as in, "Oh, we're going to do the plus of these two different types." And then whenever you run into a case where your algorithm runs into new types, you can just write that new dispatch for that type signature and then have the old code work again and have it be fast. So your ability to -- if you write the right code at first, then you don't need to rewrite it or you get to reuse it a lot. And there's a couple of different starting tools that you learn on your first days of Julia. So, for example, if you just wanted to -- if I give you a vector and you want to add it up, right? So if we were writing this in Python, we'd start with a zero, then we'd go over the collection and add it up and then return that result. There's a specific function in Julia called the zero function that will give you back a zero of the actual type that you gave it. So if you give it a floating point 32-bit type, it'll give you a zero that's 32 bits. If you give it a floating point of 64 bits, it'll give you a 64-bit floating point, right? Or if you give it an integer of size 16, it'll give you a zero of size 16. Instead of hard coding your zero to be the integer 32 type in the Julia code, you would call it -- you would define that zero with that function. And so that would make your code generic on the collection type. And so if you write the plus so that it's also generic on the collection type, you can have that same addition start to work on your 2D point, on your 3D point, on your whatever other collection you want. So you wouldn't -- if you get down the generics right, you don't have to come back and rewrite it.
00:58:09 [RP]
Does that mean there are a set of primitives that already work across all the types that you could give it? Or do you have to think a priori like, "Oh, I'm going to be tackling these sets of types, so I need to enumerate through all those and --"
00:58:27 [MR]
So you can constrain the generosity in a sense. You can say, "For all types T that are a subtype of numbers," right? Or, "For all types T that fulfill -- that they're -- you know, that are floating point types, then do this specific -- then use this specific method." So once you start seeing this tool, you start to see it like everywhere. Like in the sense that like the Rust people will be like, "Oh, all of these different issues we can just look at through the ownership system." A lot of many different issues when it comes to design problems we start just looking at in terms of multiple dispatch. So one of the classic ones is printing. Printing a type is an incredibly polymorphic endeavor. You are going to have all different kinds of structs and information inside of it. And for many of them, you could just say, "Oh, if it happens to be a struct with just this kind of shape, just do this expected printing." And even if you define new types, you can have them hook into the same abstract types, super types, and have your function be defined so that it works on those. So we -- I could say, "Hey, this is a -- for the abstract case of -- for a super type of an abstract number, have my function foo always do this." And then all new structs that you could -- that you write for afterwards could subtype that abstract class or subtype -- yeah, subtype that. And so your function will work with any one of those subtypes, even if you haven't seen them before.
01:00:06 [CH]
I mean, this is where I need someone to give like a polymorphism, like a 1001 course (not the 101 course). I need the advanced course that like explicitly delineates X, Y, and Z.
01:00:22 [ML]
I've got this really cool feature that I'm working on where you dispatch on the type of polymorphism that's currently in use [everyone laughs].
01:00:30 [CH]
I wouldn't be surprised if there's some kind of metaprogramming system where you can make that happen. Probably a Lisp.
01:00:35 [MR]
Metaprogramming Julia is fun. Generic algorithms and metaprogramming gets you really far. There's a way to hoist a lot of your computations onto compile time.
01:00:49 [CH]
We'll definitely get those links in the show notes. But that's actually what I was going to say. When you mention that there's this zero type in Julia that is like probably dissimilar in some ways but very similar to the same concept (I don't know if it has a name) in C++, but it happens at compile time. Probably the most common use of it is there's a limits library in the standard library. When you want the maximum or minimum or lowest of some type, you spell std::numeric-limits, then angle brackets for the type that you want. If you want to hard code it to int because you know that's what you want (like an int32) you do that. Or if you're in the midst of programming a generic function, you can just take your generic type T, put that in your angle brackets. And then technically it's monomorphization, when the compiler happens (so that's where you get into like the detailed differences) but the point being is that like at some point there's going to be code running through this. For the type that you are currently using, it's going to get the int32 max or the int64 max, et cetera.
01:01:59 [MR]
Yeah, exactly. We'd call those the type min or the type max. And if you wanted the max int float for float 32, you could call it on the type itself.
01:02:07 [CH]
Yeah, so I feel like there's some similarity, but correct me if I'm wrong, by default all this stuff happens at runtime? And you said that you have to go to the meta programming in order to get compile time dispatch stuff?
01:02:22 [MR]
No, no, no, no, no, no, no. So this is all just compiler machinery that's going to happen behind the scenes the first time you call the function.
01:02:30 [CH]
Oh, right. So it's JIT or just ahead of time compiled.
01:02:35 [ML]
[chuckles] The compile time happens in between various runtimes.
01:02:39 [MR]
Yes! The flow chart for when things happen in Julia can have a lot more back edges, if you want.
01:02:51 [CH]
I think that's probably where the killer featureness comes from, is that the fact that it's JITed [means] there are certain situations where it is extremely difficult to do something at compile time because if you have some data-dependent type information, how do you compile that? You need your data first, which is why you need to do some kind of JIT stuff. And in C++, you just can't do that. Or, I know there's hash embed and stuff. The stuff that you can do at compile time is ever-increasing in C++. But it sounds like in Julia, you don't need to worry about doing flips at compile time. It's going to have way more information at the point in time it needs this function that it can just JIT the version of it that it needs.
01:03:36 [MR]
So, I'm not the person to have the credentials to talk to you about how much of dependent ... [sentence left incomplete]
01:03:41 [CH]
I mean, neither am I; neither am I [laughs].
01:03:43 [MR]
Like, how much dependent types you could have in Julia, in a sense. Because I've heard that from Jeff and other people. They're like: "oh, we have a bunch of stuff that would make dependent types people go crazy", in a good sense. But there is a very accessible facility for saying: "oh, this number, I want to put it into the type system so that the compiler can access it later and then specialize the unroll factor on that." For example, there are a lot of facilities for communicating runtime information into types and then have the compiler specialize on that type information.
01:04:23 [CH]
That's really interesting.
01:04:25 [MR]
So, I have something called a Savitsky-Golay filter, which doesn't really matter, but it has to calculate some coefficients and it only depends on two coefficients, n and m. And you can know them ahead of time. So, in C++ land, or Rust land, you can use a const generic functions now. You can define them as type parameters and then monomorphize later. You can have that same deal in Julia, in a sense. But because it can happen at JIT time, you get some cool flexibility there.
01:05:02 [CH]
Yeah, it's something definitely I need to look into. So, you were mentioning talks, too. We'll make sure we get them after the fact. But do you want to, for the listener right now?
01:05:10 [MR]
There is a very cool one by Stephen G. Johnson of FFDW fame called something like "You don't need to be doing metaprogramming", or something. [07] And it's a really good talk about metaprogramming and it's a keynote on, I think, JuliaCon Baltimore. It's about [for] how most of your functions, you can just write functions. If you're asking for a macro, if you're asking for problems. Stop writing macros in your first six months of Julia. You probably don't need it. If you really do need it, then he goes into the tips and tricks and what are the sort of problems that you can really tackle. There's a lot of functionality for writing n-dimensional code and array manipulations with macros. So you can write a bunch of array indexing operations that happen in n dimensions with some macro facilities. And those are pretty sweet.
01:06:03 [ML]
You're trying to tell the APLers that writing functions that work across all different ranks is really something.
01:06:10 [MR]
Oh, yeah. Oh, yeah [Conor laughs]. When I keep hearing stuff on this podcast and I'm just like: "man, we have the same problems" [laughs]. Yeah, it felt at home in many cases. I don't know that Julia could also flex in comparison to any of the ... [sentence left incomplete]
01:06:34 [ML]
Yeah I mean, kind of what I'm getting at is that writing things with macros versus just functional programming, the way you write them is different. I think the way you write them with functional programming is pretty much always clearer and easier to motivate why you would implement it with functional programming. What macros do is kind of let you control ... [sentence left incomplete]. I mean, they let you say: "this happens at a different level than the regular code". And I kind of think that's the deficit in the language; it's issues with the performance of being able to handle functional code or with not having enough features be first class in the language, rather than really macros being very special.
01:07:31 [MR]
Yeah. I spoke at the very beginning of this on how there were a lot of DSLs that people sort of built, in order to tackle specific scientific problems. I think IDL is an astronomy [Marshall agrees] programming language. But there's a lot of other languages. I think Stan is also for statistics processing. Because you have this access to macros in Julia, you end up writing what looks like sometimes very domain-specific code: I want to write my statistical distribution that's distributed in such and such a way. Or my differential equation, I want to write as it would look on my textbook and then have all of the JIT compiler magic come in later and give you a well-optimized model.
01:08:20 [ML]
Yeah, I mean, so working with syntax, if you want your programming language to be flexible in that way, macros are the way to do it.
01:08:26 [MR]
For sure. And some very robust Julia libraries have grown around defining a shared interface in syntax to model the problem, and then being incredibly generic over the different backends. So if it's a numerical optimization problem, there's 50,000 different types of optimizers. Some are free, some are not. Some are open source, some are not. But having the shared syntactic interface for that is incredibly useful.
01:09:00 [CH]
I was going to say, you've got multiple dispatch; you've got metaprogramming. The only thing you don't have, and this is my second biggest gripe with Julia, is the longest keyword for defining a function [laughs]. I can't get past, seven, eight letters: "f u n c t i o n".
01:09:19 [MR]
If it's a one-liner, you can just do f(x) equals x. There's an inline version for short functions.
01:09:29 [CH]
Oh, okay. So similar to JavaScript? The one that also has that? I don't know. I don't do enough JavaScript programming.
01:09:37 [RP]
I think of the arrow functions, that bit.
01:09:37 [CH]
Yeah, the arrow functions, it sounds like.
01:09:39 [MR]
Yeah, we have arrow functions for lambdas, too.
01:09:43 [CH]
But yeah, you're not appealing to the array programmer.
01:09:48 [MR]
Yeah, I know. But Rust had the right idea. Some languages, it used to be the type, back in the day. You'd put the type before the function name, then people started getting inspired by mathematics.
01:09:59 [ML]
Well [chuckles], that's not really the distinguishing thing, because you put the type before the variable name, too.
01:10:06 [RP]
It's clear that this JIT compilation, combined with the REPL and the multiple dispatch, is fairly very powerful for people doing scientific and numerical computing, who, at first, just want to get some stuff done, but they will hit performance issues that they need to find a way around. But all this talk about writing the same code, or code that does the same stuff for multiple types, does not massively appeal to my particular sensibilities. It sounds like the kind of work I'm trying to avoid by having an interpreter figure out all the stuff for me. I'll couch that by saying, you've sold the ecosystem and the ability to find other people's code very easy. So that probably also appeals massively to those types of programmers.
01:10:57 [CH]
I mean, I think part of (and correct me if I'm wrong, Raz) is the pitch for Julia that he was making, is that compared to other languages in the same (I don't know what you call it) Algol descendant space, Julia gives you this ability to write generic programs so that you don't actually need to re-implement algorithms across types, similar to what generic programming is in C++ if done right. In other languages, you actually have to ... [sentence left incomplete]. I know actually Go got parametric polymorphism in like 1.18. So the state of Go today, I think you can do this. But back in the day, you would have to re-implement the max. That was my biggest [gripe]. I probably told it like three or four times on this podcast or across my podcast universe: that only used to have a max function for floating points. So if you wanted to take the max of integers, you'd have to cast a floating point, then take the max, and then convert back, which from any language like C++ or Julia where you have some kind of facility for generics or parametric polymorphism it is like: "how could you?". Or even Python. Python doesn't have that. It's got what, row polymorphism via duck typing? But even Python, you can still just like write it and it works and it's fine. But there are some languages where you actually have to like: "oh, here's a new type; I'll go implement this function". "Here's another type; I've got to go implement the same function and just change the types around". Correct me if I'm wrong. That's the pitch, right?
01:12:19 [MR]
Right. And so every time that I see someone saying something like: "I'm writing my service in Rust, and I do my data scraping in Python, and I'm going to see how I interface them", I just think for a lot of those people, just use Julia. I mean, if there was such a magic wand where there were not many real-world constraints and your boss isn't screaming at you to get this deployed into production and to blah, blah, blah. I buy Hillel Wayne's argument [that] [08] you probably should just use whatever doesn't get you fired [chuckles] right? I wish a lot more people would give it a shot. I think that if such a thing as technical merit should help out (good ideas win out) I think Julia should be a good candidate to test that. Yeah. There's all sorts of quips and interviews. Was it one of the big PyTorch developers or one of the big TensorFlow developers where they were like: "yeah, we really looked into Julia; we really liked it; it should have been done in Julia, but not enough people know it so, well, we went with Python". And it was like crushing.
01:13:32 [CH]
Maybe the last, unless if folks have other questions, I was going to go subscribe to this Talk Julia podcast, but then, according to my podcast app that I'm staring at on my phone right now, there hasn't been a new episode in two and a half years? What is the state of Julia podcasts?
01:13:52 [MR]
I think it's literally called the State of Julia.
01:13:55 [ML]
I mean, subscribing to it won't hurt you at all!
01:13:58 [CH]
Yeah [laughs], it won't add to my queue, but the goal of subscribing is that I'll get to listen to some Julia content.
01:14:04 [MR]
There's one called The Julia Dispatch with Chris Rackauckas and a lot of other cool folks. That's the one that's been mostly picking up steam. Chris Rackauckas is like one of the Julia rock stars, founded most of the differential equations ecosystem.
01:14:25 [CH]
And I guess maybe in general too (now that we're, I guess, closing towards the end) if you've got resources in general ... I know that JuliaCon is a pretty big event every year.
01:14:35 [MR]
JuliaCon is big. There's a lot of really good videos that get done for JuliaCon. There's some smaller Julia meetups that are happening now and people are uploading. There was just a Rust and Julia meetup that I helped organize in Netherlands. Unfortunately, not recorded. But yeah.
01:14:54 [RP]
There was a talk at a JuliaCon (it says 2016) [where] someone has done an APL implementation in Julia of some description. I need to look into it. Intriguing.
01:15:05 [MR]
Yeah, that's Shashi Gouda, I believe. So one Alan Edelman is the academic grandfather of the Julia project. He's the guy who got the grad students to start working on Julia seriously. He was a grad student when his advisor (it might have been Gilbert Strang or one of these super legendary professors) got the 001 MATLAB license. And they were a huge APL fan when they were a grad student. And Alan Edelman could not get Stefan or Jeff or Viral (any of the three co-creators) to care about APL [chuckles]. They just saw it as the line noise and they were like: "it's not going to happen." But they took a bunch of good ideas out of it. I mean, you can see it. You can smell it. Julia is not as present in the Twitter's and BlueSky's as many other programming languages are. I think I see Julia people as more involved in the building their own code than liking to talk about them. End of the spectrum. It's not like either/or.
01:16:20 [ML]
Yeah, well, they're not as invested in the language itself. There are many, many users who aren't at all.
01:16:26 [MR]
Obviously, I would be the person who would be sharing this kind of story. But it is the case that we keep hearing of more and more people going to conferences that are not Julia-centric and people coming up to them like: "oh, our whole code base is in Julia." I know some of the US Federal Reserve does their forecasting in Julia. The Pfizer vaccine was designed in Julia, at least the molecular dynamics side. The WhatsApp video codec was designed in Julia as well. There's a whole bunch of use cases if you go to julialang.org and like: "who actually cares about this and is willing and able to invest in it?"
01:17:12 [CH]
That's awesome. We'll make sure to leave all the links to past podcasts, current podcasts, JuliaCon. I mean, I'm probably not going to compile all your local meetups, but maybe there's a Julia ... [sentence left incomplete]. I know that there's one for C++. Maybe there's a Julia page that lists global meetups.
01:17:32 [MR]
Yeah, there's the big meetup that happens. It's JuliaCon once a year.
01:17:35 [ML]
Compiling all the meetups at once would not be the Julia approach.
01:17:40 [MR]
No, no [everyone laughs]. That's where a lot of learning, or at least a lot of knowledge sharing has happened: the JuliaCon YouTube channel (is it a YouTube channel?). I think it's maybe just Julialang.
01:17:55 [CH]
We'll find it and put it in the show notes. And I know that they were doing it online for a while, but I think it's backed in person [TODO: Did Conor mean to say "It's back to being in person ..."?], correct?
01:18:01 [MR]
Yeah, yeah, yeah. I helped organize the online versions. It was very hectic to try and tame chat during an online COVID conference.
01:18:09 [CH]
I mean, that's the problem: it's trying to tame it. You just let it go wild.
01:18:13 [MR]
So, during COVID there were a lot of very not safe for work spam bots in the YouTube comments going, during the live talks. And I was just trying to play whack-a-mole.
01:18:25 [CH]
My trick is you just promote five or six people to be mods and then you've got multiple people, whatever, whacking moles instead of just one.
01:18:35 [MR]
I was one of those.
01:18:37 [CH]
All right. Well, thank you so much for coming on, Raz. This has been very informative and I will have to take a closer look. It definitely seems to be the case that there should be some more cross-pollination [Raz agrees] of the APL (and friends languages and company) and Julia. Because Julia is definitely in the array language bubble for sure. It's around the line of the Iversonian one, but seems like there could be some opportunities. Especially we've always (what's the word?) ruminated about a future ArrayCon where it's an annual conference for all array languages. And Julia would definitely be welcome there. If that ever happens, we'll make sure that we connect. But for folks that have been listening, if you have thoughts, comments, questions, either for us or for Raz about the Julia language, you can reach us at ...
01:19:34 [BT]
contact@arraycast.com. And last episode, you had used the word "swathe" and then wondered whether or not it was actually a word. And now you used "ruminated". So, I'm really impressed we've got multiple syllables here.
01:19:46 [CH]
I think "rumination" is definitely a word that I learned from my father, who's a journalist. "Swathe" was just something I pulled from the back of my brain. I don't think I was ever taught that [everyone laughs].
01:19:57 [MR]
Going off on the multi-syllabics, just to flex on them.
01:20:02 [BT]
Anyway, contact@arraycast.com if you want to get your questions into us. We've had numerous questions. We've had a few questions on the Dan Bricklin episode about different things. And so, we've replied to those and lots of positive response on that one. Well, all our shows get fairly positive response. But yeah, contact@arraycast.com.
01:20:21 [CH]
And once again, thank you for coming on, Raz. Hopefully, we'll bump into each other in person at some point at a future event.
01:20:28 [MR]
Yeah, yeah, yeah, I hope so too. We'll see where it happens.
01:20:33 [CH]
All right. And with that, we'll say, happy array programming!
01:20:36 [All]
Happy array programming!
01:20:39 [music]