Transcript

Transcript prepared by Bob Therriault, Adám Brudzewsky, Igor Kim and Sanjay Cherian.
[ ] reference numbers refer to Show Notes

00:00:00 [Stephen Taylor]

So Michal, you've done a lot here to demystify k to show that it's not a cult, you don't need to belong to a race of semi divine.

00:00:10 [Michal Wallace]

We do pray to Arthur every every Thursday.

00:00:12 [Laughter]

00:00:15 [MUSIC]

00:00:25 [Conor Hoekstra]

Welcome to another episode of ArrayCast. Today, we have a special guest who we will get to introducing in a couple minutes, but before we do that, we're going to go around and do brief introductions from our panelists, and then we'll have a few announcements and one follow up. So we'll go to Stephen and then to Bob then to Marshall and then to Adám.

00:00:43 [ST]

I'm Stephen Taylor. I'm an APL and q programmer.

00:00:46 [BT]

I'm Bob Therriault. I am a J enthusiast.

00:00:49 [Marshall Lochbaum]

I'm Marshall Lochbaum.

00:00:50 [ML]

I'm a former J programmer, Dyalog developer and now BQN developer.

00:00:54 [Adám Brudzewsky]

I'm Adám Brudzewsky, full time APL programmer and teacher.

00:00:58 [CH]

And my name is Conor. I'm professionally a research scientist, and I'll just start saying that I'm a polyglot from now on and I know multiple languages. So I think we have one follow up that we will throw it to Marshall for and then after that we've got a couple announcements, one from Stephen and then a couple from Adám so let's start with the follow-up from Marshall.

00:01:26 [ML]

Yeah, so last episode we talked about you know what makes, uh, what puts something in the APL family, the Iversonian family of languages and one thing I think it was Adám brought up after the episode that we didn't think of during the episode was that all these other Iversonian languages have no precedence between what in APL are functions or in what in like C or another language would be operators. So, if you have like plus minus times, divide all of those are on the same level. They apply. Uhm, there's no like you don't have to know that multiplication happens before addition. So that would put all the classics, APL, J, k, BQN *** actually, Nial as well in the Iversonian or closer to the Iversonian category and we would throw out a lot of other families like Futhark, like Wolfram language. So that's an interesting criterion that we didn't bring up.

00:02:20 [CH]

Yeah, maybe we should have a whole episode at some point on. Maybe we can tie that in 'cause at some point we have slotted a discussion about overloading and I feel like that might go sort of hand in hand 'cause there's other languages. Interestingly, Smalltalk doesn't have actually. I think they do to a certain extent, but like the same way that when you have plus times like a mathematical expression, it just evaluates left to right, whereas APL is right to left. But yeah, we will stay tuned listeners and we may talk about this in the future. So let's go to Stephen for our announcements now.

00:02:52 [ST]

Since the last episode we've held a couple of q Vector dojos. In these, in these dojos, we're concentrating on the shift from working in loops and scalar that scalar languages kind of teach to thinking in, to moving to vector solutions, getting fat ***, getting fluent with the techniques, and being able to see the vector solutions rather than what Joel beautifully described on this podcast as the one potato, two potato approach.

00:03:22 [CH]

That was awesome.

00:03:23 [ST]

Uh, we for the time being we're keeping the dojo small so they're like tutorial groups. There's me and no more than three other people. We work for an hour and a half hour or so on problems that suit all of us. It's highly informal. It's very interactive. Nobody gets left out. Everybody gets to participate on the solution and it's been a lot of fun. The next one will be in the next week or so. It will suit the India time zone as well. We had a lot of interest from people in India and we need to pick a time zone that suits them. If you're not already plugged into this, the conversation is in the Iverson College workspace on Slack. If you'd like to get in right to me SJT AT 5jt DOT com. [01]

00:04:15 [CH]

Awesome and I think yeah links will be in the notes and I think Adám you've got a couple more announcements.

00:04:20 [AB]

Yeah, so Richard Park and I have released a new episode of our APL Notation As A Tool Of Thought podcast, this time on notational awkwardness is that should be fun.[02]

00:04:36 [CH]

I have to interrupt I have to interrupt and I said I would do this before. Is it really a podcast yet? Because it does not exist in the podcast Topia and I have checked, I wouldn't say every uh, but I've checked a few times 'cause I know it's on YouTube and there is an MP3 file on a website or like a playable thing, but it is for those that are listening and being like I haven't found it on my app. I think that's because it's not on any of the apps yet is this correct?

00:05:07 [AB]

Yeah, I have this weird thing about not wanting to pay for external services, or indeed use any libraries or anything, so I'm trying to implement the whole RSS feed from scratch myself and I'm.

00:05:20 [CH]

Oh, that explains it. That explains it.

00:05:23 [AB]

Until I figure out how to do this right and it's not there. I think I have some some leads.

00:05:29 [ML]

What makes an Internet broadcast a podcast must have an episode on this.

00:05:35 [AB]

Oh no meta! If somebody out there wants to help me and the repository you can link to it, you can see what it looks like and I'll be happy with it, some help to get it right.

00:05:45 [BT]

When he gets it right we'll all be living in Adám's world.

00:05:48 [AB]

OK then and there was recently this Dyalog user meeting. And little by little Dyalog is releasing these videos of people's presentations, and so there's a whole set again that was released on some technical stuff and some user experience. People doing actual things with APL. So go and check that out. Yeah, and especially I'd like to to point out there's something that hasn't happened for many, many years is that we put out a special edition of the interpreter called the conference edition, and it's a special build of the interpreter where lots of experimental features are there. It might not be stable. It can give affirm errors and stuff when you use it, but there's some really cool stuff that some of which might go into version 19 and it's available after the use meeting. Anybody can go and download it and play with it and we'd really like feedback, both positive and negative feedback on that. This is crazy, or that's awesome or you could do it like that. Some things have just been done in a certain way because that's the idea John Daintree got and might want to do it differently.

00:07:06 [CH]

Two short questions. Are there sort primitives and is there the BQN before AKA the Sigma Combinator? Any of those?

00:07:15 [AB]

No, it doesn't have any new primitives. It's just part of. It's just the system basically that's different, however uhm, there will. It hasn't been released yet, but I did a presentation on 3 new primitives that we're thinking of adding and it has been met with applause both internally and at the user meeting with the audience there so, it's pretty certain that we are getting those. One of which is indeed that backwards compose or you want to call it behind something here.

00:07:44 [CH]

Ooh, one or two so stay tuned to it's.

00:07:48 [AB]

It's it's not in the conference edition built, but the yeah.

00:07:51 [CH]

It's coming soon.

00:07:54 [AB]

And then we got to sort. I'm not even proposing that we add sort primitives, but rather adding a select or from function which makes together with the behind or before a combinator then sorting becomes really simple. It's 3 characters and it's not a train, so you don't need to parenthesize and it works both for sort and sort by. So that should be fine. You can already do this in BQN of course.

00:08:24 [CH]

All right, well, once it's in the interpreter, we'll talk about it on a future episode. Is that all the announcements we have?

00:08:31 [AB]

I think so.

00:08:32 [CH]

I just realized while Stephen was speaking that I have one announcement and then I realize there's another thing I can now. So the thing that I thought of while Stephen was speaking was that there is actually another and you can correct me if I'm wrong Stephen, k, or actually not k but q announcement in that they changed their free license policy, so it used to be. I think that the license if you downloaded the experimental license or free license, it was good for a month, but now it's good for a year, which is great because now you don't have to and actually.[03] Does it expire after a year? Maybe it doesn't expire at all. I can't remember the details but, you can download it and it's free for a lot longer and you don't need to update things if you were running into that problem and my last announcement is that there is a video, what was it called? It's called "Stop Writing Dead Programs" from the StrangeLoop 2022 Conference, which is a conference that covers a variety of programming languages. I think Clojure and Lisps are very popular there, but usually they have some esoteric topics and this talk which was just given a couple months ago is now the fifth most viewed talk on that YouTube channel with close to 200,000 views, which is massively impressive. 'cause like the number 3 most viewed talk, was by Joe Armstrong from like 2014 and at the beginning of this talk the presenter, Jack Rusher, [04] basically goes from assembly to C to a couple other languages and boils down to what he thinks is like the highest abstraction, you know the most beautiful thing, and it's APL, so should all go watch that code. APL gets a shout out, and it doesn't say much else about APL, but the talk in general is just fantastic. Talks about a plethora of things and how we could be doing a lot better.

00:10:25 [CH]

Anyways, our guest has been patiently waiting because we've had so many introductions and announcements and skipping around. Our guest today, probably you know this by the title of this episode is Michal Wallace, who goes by, I believe, on YouTube, Twitch and Twitter. Maybe a couple other platforms as Tangentstorm and super excited to talk to Michal today because he works at 1010data currently but has had a long and storied career, and from looking at his LinkedIn profile, has coded in a plethora of languages, Python, Java, I could go on and on and so it's going to be interesting to hear his story from going through all these different languages and ending up at, I believe it's k that they use at 1010data and then J which is what you typically promote and I've seen some live streams where you've been building up, you know, editors and lots of cool things, and I've also I saw what was it? A YouTube video that was really really great in showing the power and expressivity of J that was sort of building up a little deck of cards sort of program which will link that in the show notes.[05] It's a great example similar to sort of some of Bob's videos of highlighting what you can do with J. Anyways, I'll stop there. Throw it over to Michal and he can tell us the story. Go back as far as you want to when you were born, when you started programming and bring us to how you got to, you know, working at 1010data and sort of living in this array language world.[06]

00:11:43 [MW]

Wow, well, thanks for having me guys. Uhm yeah, so I've been programming for ages as far as far back as I can remember. I think first or second grade some lady came to our school, enrolled in like the one computer in the city for the school district through our to our class and showed us hey, this is this is a thing called the computer and it has this thing called BASIC on it and you can tell it what to do with this language called BASIC and I thought that was cool. I was hooked on it. I didn't. I don't think I even touched the thing, I was just, you know, like I just thought it was neat. And so I , for years any time I happened to go in a library or book fair and there was something about programming I would take that book or check it out. Or you know, just try and read it the same way you might read about, you know going to outer space or dinosaurs or something. I just thought it was a cool thing, so probably by the time I was in, I think even in middle school I think I took like one little little course and like a college for kids kind of thing during the summer where we got to spend a couple weeks at the local community college and I had a, I have a floppy disk from it, but I don't know. I have no idea what happened in that class or what they taught me, but I must I I must have been there and uhm, I think by the time I was in middle school like I knew like I had the ideas in my head about like you know, what programming was and kind of like general and then I think I was finally an 8th grader so I was a teachers assistant and my reading teacher had four computers. Four Tandy 1000's [07]in the back of her class and had probably never touched them or had any idea what to do with them and she gave me let me use it so that that's what I did for half of half a school year was just sit in a room for one period a day and I had one one little floppy disk and a copy of the GW BASIC manual. No Internet, nothing like that. Just the keyboard and a floppy disk and the manual and I started making little screen savers, and I'd make a, you know, what was some some primitive attempt at a game or something like that and learned how to use this thing and eventually my parents got me a computer, so I used, I learned a lot of GW BASIC and you could sit there. You could turn on your computer, fire up, type in GW BASIC at the DOS prompt and you would get the didn't have an editor or anything. You just type you wanted to edit line ten of your your program. You type the number 10 space print Hello world enter you always started with 10 right, because you couldn't delete lines, you might want to put a line in between.

00:14:06 [MW]

I moved on to Turbo Pascal pretty quickly.[08] I thought that was neat. That was my my go to language for many years. Just fooling around, teaching myself so I feel like that's how I learned that program, not not professionally or anything. I did eventually get a job as a. In fact, by the time I graduated from from high school, I had no plans to be a computer programmer. I didn't know that was a thing. Like I said, I said I compared it to dinosaurs earlier. Like I, I really thought that programming was something that like maybe two or three guys at NASA did or something like I just I didn't realize how much a demand there was for this skill. And so I thought I was going to be some kind of writer, eventually found out that this paid better, but I did take a job as a QA guy, it was it was a my dad's company like they hired the children of the people that worked there every summer and so I had an internship there and they they put me on this QA team they were forming. They'd hired an outside consulting company to build the software for them and they were all embedded there, but so they hired, or they had they brought one of their guys to be the QA tzar. And so I started in QA and we did like a lot of like it was called Visual Test which was like a dialect of Visual Basic. We just write tests about about this, you know about this program. Like you know there's an insurance insurance and they sell like they do family planning for military families. So I made up this little family and they were. They were Fred and Wanda Tempy. I use them ever since then, I've used them as my test people for years and years now, but they have a little family and they did all these. They bought all this insurance and you put him through the wringer and that was what I did also.

00:15:48 [CH]

Put them through the wringer!

00:15:51 [MW]

Put the software through the wringer.

00:15:54 [CH]

Poor Fred and Wanda, just for years they've been getting.

00:15:57 [MW]

They have a website.[09] If you go to tempy tantrum Dot com you can see that. You can see Fred Tempy then. That was the first time I had like a professional programmer like you know, critiquing my work and telling you, you know, hey, you could do this better and some guy but but again I I still thought I was going to be like a a novelist. So I took after high school. I took some time off and tried to write a book. I wrote a very short book. It was it was an OK first attempt for a kid and I tried to write another one and it just basically. I spent the year after high school, just just kind of being depressed 'cause all my friends were off at school and I was all about my home trying to write stuff. Trying to write this book and somebody a friend of mine I met online invited me and said hey, maybe you know you should do some teaching and so I got involved teaching little kids to read and or first first just helping after school with the homework and that kind of thing and eventually started working professionally as a uh, through AmericaCorps,[10] a thing Bill Clinton set up kind of like a domestic version of the Peace Corps that they give you money for college. A little stipend every month so that that was my that was my career path for a while then was to to teach children, you know, I wasn't at any time really trying to be a computer programmer. In college I was an art and performance major again thinking I'd do some kind of writing stuff. I had a minor in called cognitive Science, so it was like half it was their their closest thing to the University of Texas at Dallas they had to AI degrees so they had psychology and computer science. And so I took a C Class and assembly class, really didn't really find my calling. Eventually I wanted dropping out and moved across country for a girl and just was looking for something to do I wound up working just that in a in a plastic surgery center. They were kind of disorganized in some ways, like I had this software for everybody called receivables like so you people have come in and they got in their surgery and you're trying to build their insurance company, but maybe some of the selective *** and so you're you're building the person directly. And so they had to call like all these people and it turned out in the software they were using. There were like 6 screens that they had to go to look at, just gather the information to call one person. Uh, and so I said that's that's kind of dumb and let me let me take a look. And I dug in and they're actually just this software was just an Access database under the hood, so I wrote him a little report that you could print out and they would have everything they needed for each person and all the people they needed to call in one little report and I gave it to him and they looked up, hey whatever, thanks and then the next day they're like you're my hero, you know. So, eventually I called the company who made that software and like they were a local company in Atlanta. I said, you know, hey, I'm fixing your software and come hire me and so they did and I went to work for that company for a while. I kind of split it between teaching people how to use their software. Sometimes I would sit down and with like at somebody's medical practice and I'd sit down with the, you know, the lady they'd hired outside and said, you know this is a mouse, let's play some solitaire so you can learn how to use this mouse. And then we flew all around the country doing that kind of stuff teaching just all kinds of different people.

00:17:39 [MW]

Eventually, I left that and started working at a consulting company and we did just more Microsoft Access stuff and then all of a sudden this thing called the web started popping up, then we kind of sat the the boss down one day and said hey look, we just stop with this access stuff and start doing, you know, Internet and then we made all kinds of stuff for all kinds of Fortune 500 companies and eventually I decided I wanted to start my own company and actually, I had been teaching a friend of mine to program and we decided to start a company and he said, well, let's make something for a company. So we made a little shopping cart system in PHP and that turned into an attempt to build a software company. It was just me and him and we didn't. We don't know the first thing about actually like running the business, so let's let's do something simple and let's just get a server and put it out there and start charging people for web host, right? Just to learn the process of like how do you take a credit card? How do you do this? And so we did that I did that within a few years that was my source of income. I was running a web hosting company and he went on to do something else and we never built a software company, but I made a web hosting company had and I did that for like 10 years, every once in a while I take a a little side job, but I mostly just and when I say I ran a company it was like me, right? I was a support person that had a couple 100 customers. I built a control panel and a billing system and all this stuff in Python. At the time, there weren't, you know, today you go download Flask and Jinja templates or something like that and or Django if you want to. But none of that existed, and so I had to write roll my own web framework, but any case I ran that company for many years. I did a lot I had gave me a lot of time to just learn about other languages that you know had a lot of freedom and flexibility. You know I was, I was tied to my my laptop to answer support emails, but I had the time to you know, work on whatever I wanted in my business and a lot of time that meant like automating whatever I want. So I was a lot, spent a lot of time just making stuff in Python for that business, and a lot of time like looking at other languages and like trying out Smalltalk and Haskell [11] and just, you know, learning it kind of finishing my education by being self-taught that things that I didn't learn in you know about computer science, like how to write parsers and how to write virtual machines and things like that kind of kind of fills in the gaps on my own. And then at some point that that you know the most of my clients were bloggers and when the 2008 financial crisis hit a lot of people were strapped for cash and they were like I don't need to pay 5 bucks a month, 10 bucks a month for a blog anymore. And so I started losing business very rapidly. Started focusing more on like client work and but, I was spending so much time outside of the company that it just like my tech support got even worse and worse and worse.

00:22:05 [MW]

So I just had to, you know, stop that and needed a job and actually you know I was trying to save the company for awhile and so like cut costs I like move back in with my parents for a while and, but you know, I was living with my brother for a while and it's like, you know, I can't mooch on my my family forever, so I gotta I gotta, I gotta work. And so I decided to go back to work. I worked at a publishing company that made yearbooks. We were doing like Angular and like an actual modern Python back in I wasn't terribly happy. It's actually an hour long commute every morning and they had meetings at 7:00 o'clock in the morning. My boss was the early bird, so I was not not too thrilled with that situation, and so I started looking for another job and by this time I had learned J and I had spent a lot of time like on the Internet, especially when I when I wasn't working I spent a lot of time on the Internet hanging out in a channel that called learn programming on an IRC channel and we had little bots and there's one day I went to like the Haskell Channel say 'can we get the Haskell bot in here' so you could you could you could type Haskell expression and we got like we collected a Perl bot in that in a we had just like a bunch of finally went into this J Software channel [12] and I said can we get this J bot? And so I said sure and or they said sure and we we had a we had the J bot in our in our channel. So I learned J like just using it as a calculator for a long time in that in that Channel and we just had it there and everyone. So I'd say, oh, you can make this little pattern. It was nice to have it as like a demonstration tool for if somebody had a question about an algorithm, you could often like type it into the to the the J prompt and it would print out. You know you could very easily generate like a little bit map of you know characters on the screen and like show something step by step and I would show just show it off and so that's kind of how I learned J was just as a kind of a parlour trick for this IRC channel for a little while. And then one day I thought, you know, it had been so long since I had been a beginner, I thought you know what I'm gonna try and do something that's just really outside my comfort zone. I said I'm gonna learn this J language. I'm gonna learn it like actually make programs in J. And so I read the J for C programmers [13] and the primer and all these things. And I worked through them and I don't know. I came out of there with it with the new language, so that and I made a video about it about one thing I thought was cool. This was Ulam's spiral. So Ulam was a mathematician and one day he was doodling and he had this piece of I guess graph paper and you you're the number one and started counting but he wrote the numbers in the spiral. Right? And then he marked out the primes and primes made some little pattern. He's like, oh that's neat. And so it's called Ulam's spiral.

00:24:42 [MW]

And so we were somehow that came up in the channel and I was like Oh my God, I could do this and it turned out to be like you know I found a way that I thought was kind of clever to do it in J in like one line of code. And so I made a video about that one line of code. And that's now like this, that was the start of my channel and it became or it's certainly the most popular video I've ever uploaded, even today.

00:25:03 [MW]

While I was job searching, I was looking for a J job. There are no such things. I think Eric Iverson, when he was on here,[14] said people don't admit to using J and they don't even tell their boss they're using J when they use J. But there are a lot of k jobs, and I think I even tweeted that, Kx like you know, why is everybody in New York? Are there any remote jobs? Anyway, I was looking for a job and I thought it would be neat to work in an array language, so I had that on my radar. I'd actually almost applied to 1010, but something, I don't know, some links didn't work on the website or whatever and I'm like: "I just have to move on". Yeah, I'm not really looking to move to New York anyway, but our chief scientist posted to the subreddit one day and he said: "Not really a job advertisement; we're always looking for people who are interested in in k" and so I wound up talking to him on Reddit for a little bit and one thing turned to another and I wound up coming out here to New York to interview. And I actually met Joel Kaplan and Stevan Apter,[15] both of whom have been on this podcast and they both interviewed me that day and they didn't kick me out, so they gave me an offer and I think New York. And that's where I've been ever since. I think I'm done rambling.

00:26:25 [Conor Hoekstra]

I said it's pretty interesting because it sounds like. I'm not sure if I'm going to enumerate every single one, but from QW Basic to ASP to Python to PHP, there's a few others you mentioned, and then you were just happening to be on IRC. Maybe I'm not sure if you're the only outlier that we've interviewed, but many of our guests that we've interviewed were immediately taken by the language and they, you know it was love at first sight for one reason or another, the notation, the brevity, et cetera. And it sounds like you were exposed to it because someone asked for a J bot. And you implemented that and then you used it for a while, but only ended up, sort of ... Or maybe I'm sort of inferring this, and it might not even be true that you have a a deep passion for the language. It's more, you were fascinated with it and you find it cool to make YouTube videos. Maybe you're not; maybe you're dissimilar and it's a language or a paradigm you use because 1010data uses it now, but it's interesting that you sort of ended up becoming proficient in it or going and learning it as like a challenge to yourself, not because you were, you know, in love with the digraphs or something about the language spoke to you.Is there something behind that? Or is anything you want to add to contextualize that sort of difference.

00:27:50 [MW]

So when I was running the web hosting company, I had a lot of time to just, you know, follow my interests. I talked to a lawyer once like: "is it possible to get investments in this company?" He said "No, This is a lifestyle business, right?" And it was true, this was me, I didn't know what I was doing. Had I been a better business person, I would have weathered the storm and you know, grown the company, right? But no, it's just a little boutique. And so I had a lot of time, you know, because I was spending a lot of time just following my interests rather than working on the business. We would dabble in lots of different languages. Every time I would start a new project, I always had the choice of: do I try this new language and actually make something in it, or ...

00:28:37 [MW]

And you have to fight through the learning curve, and you learn yet another way to do the same thing I've learned how to do 100 times before in other languages. Or do I just stick with Python and actually make it. So usually what it turned out to be was, I'd play with Smalltalk for a couple of days and go learn what was cool about it. I'd play with Pascal, same thing, and in fact I have a GitHub repository called Tangentlab [16] and there's a directory of, maybe 75 different languages and there's little snippets of different things I've done in all those languages. Most are trivial. They're just going through tutorials and stuff; but I did a lot of that.

00:29:18 [MW]

There was also: one of the big programming communities that I was involved with at the time was c2.com wiki. They were the people at the Ward Cunningham wiki, that they had actually created. (He's the guy that created the term wiki and invented the concept of a wiki) It was about whatever people wanted to write about, but they attracted quite a few programmers and especially people from Smalltalk. But they also would have pages; there was a page called "J language" and you know, they had little snippets posted and like, you look at this in awe. So it wasn't like I had never heard of the language before, but I was aware that J was a thing and there's a paper that I talk about, or that I think about a lot, that Paul Graham wrote, called the Python Paradox. [17] Basically he was making the argument that if you use Python, which is this obscure language that nobody uses (at the time) then you're going to attract [pause] You are going to think that you won't be able to find anybody that uses this obscure language, but in fact, the people that use it are going to be people who are really interested in languages and really excited about programming, and vice versa. You know, if you find a Python job, then you know, maybe that's a cool place to work. For that reason, when I see the company like 1010data uses k: oh that's interesting. I didn't know k at the time. I knew it was something similar in the ballpark of J.

00:30:45 [MW]

And so I sent him the video that is part of my introduction that I had made about the Ulam spiral. They don't expect to hire a k programmer, for that same reason, but they're saying, well, you can learn. So if you have an interest in stuff, you learn. And so I think (I know I'm rambling) the actual question was, whether I thought J was just a challenge to myself. I think the answer is that I like different languages and I like to try and puzzle out at what it means to express an idea in a particular language. And if you throw different languages at me, I wouldn't say I'm in love with a particular language, but I do like J; J is definitely one of my favorite languages. And was it an instant? Uhhh, no. It was not love at first sight. It was just this is interesting at first sight.

00:31:35 [BT]

So if I've got this right, Michal, you didn't know k when you were interviewed by Joel and Stevan for 1010data.

00:31:45 [MW]

Correct, but I think I had downloaded Kona [18] and then Adam, the guy that hired me, said no, you want to download one from Kx. It's a little closer to what we use in [indistinct]. And so I'd play with it a little, and I tried to make one or two lines of k, but yeah. I was not a k programmer, but I knew J and that was my credentials.

00:32:11 [BT]

So I think Stephen Taylor has talked about this: a lot of learning of k goes on within the shop. Is that sort of your experience when you've picked it up. Were you working closely with other people to pick it up?

00:32:21 [MW]

That was not my experience because I was moving across the country to get to New York. I actually took some time and like learned k so I didn't know it when I was interviewing but by the time I got there, I knew enough. Also at 1010, we have, it's a database product and it has a query language that is kind of an XML language. For the first few years that I was there, we had an intern program. And we hired a bunch of people. We went to (John Ernest and I) went up to Stony Brook (college around here, university around here) and we would go talk to the people and we told them about k, just as a recruiting tool. That was me doing one thing.

But other people, did like massive amounts of effort to recruit people. And we found the best and brightest interns we could find.

00:33:15 [MW]

And every year we did a course in k so we had a first week. Lots of different people from the company would give presentations about k and give them homework and never had a problem with it. Never had a problem with people not learning k or would not. People liked it. It's a nice language. After that week of training they were productive enough to build things that went into our system, over the course of the summer. And so you know, I think it's not an obstacle to hire, to find, k developers or to find people that are willing to be k developers and try it out. I think some of our best hires have been people who, like the Python paradox, were people who are already interested in k.

00:33:57 [MW]

Like I mentioned John Ernest. Joel Kaplan also mentioned him; he said you should talk to him. I think you should too, but he and I started right around the same time. Both responded to that ad and he's written a JavaScript interpreter for k called ok.js. [19] So he was ultra into k at that time and and also, one of the best programmers I ever met, so a lot of our best hires have been people who are just really interested in k, for whatever reason. So I think, just that enthusiasm. One of the guys I hired, I was talking to him on the phone one day when we were interviewing. He had asked me about something and somehow the topic of that IRC channel came up. He's like: oh, I've talked to you. I know who Tangentstorm is on IRC, and you helped me with some question. Just because of that, I consider that a very good sign, that he was somebody who was interested enough in languages to, you know, to seek us out because of k, you know?

00:35:03 [MW]

So I've had that argument with people, with managers that have been hired since (you know, since we got acquired; Joel retired and the culture changed a little bit) and I remember talking to a new manager who's like: "we can't hire k developers, it's not possible". And I said: "you haven't tried". Because if you try, you will find that you can find some amazing developers that either know this and are enthusiastic about it, or or see it and want to learn more. I posted on LinkedIn once when I was hiring, saying (this is not a job that's available now), but I posted a job advertisements to my LinkedIn group [20] and it was called "want to work it with k at 1010data?" and there's at the very top, a little screenshot of the Mandlebrot Set, rendered in ASCII. Then next to it is the k code that I used to generate that. I don't know, somebody told me that was the best job. Somebody at the bottom said something like, it was the best job advertisement they'd ever seen. But we actually wound up not hiring anybody for this role. It's not hard to find people who have an interest in interesting languages.

00:36:12 [BT]

Do you find that people who might have a more traditional background like you, (you were talking about how you came from Python); is that an obstacle to learning one of the array languages, or do you find that you really just look for an interest in the array language first?

00:36:26 [MW]

To be clear, I don't think interest in the array languages is a requirement. Our different recruiters do different things. The guy that ran our intern program, he's no longer with the company. The candidates that he brought were somehow way better than many of the people that we would get through the traditional recruiting tools that had been in the industry for many years and had all this stuff on their resume and looked great on paper. But you know it's always hard to judge someone in a 30 minute interview or an hour interview it, you know. We do the the leet coding thing. or give them some kind of puzzle and try and grill them and see how they do, but you know, for me, it's all about do they have the ability to take an idea, and it's not even like, yeah, I guess there's two parts, right? You got the the problem solving part, figuring out what to do when you're solving an interview question or whatever. And then the: can you take that idea and express it to a computer?

00:37:24 [MW]

So, the last video I did that Conor mentioned earlier about dealing cards; there's a part in it where I talk about an interview question I get: Which is just give me every permutation of an array; of the numbers one through X, right? So if you say give me all the permutations of 1 of X. First you say: "you know what a permutation is?" Make sure you know the right words. Then, now do it. Do it by hand. Show me that you understand this concept. So they write out (1,2,3), (1,3,2) and you know all that. So yeah, we agree that those are the permutations of this. There's six of them. Now, how do you get a computer to do that and what I found is that some people who have been programming forever, like no matter how many hints you give them, they just never get there. They cannot write this program. I always test interviews. Like I went to John Ernest and I went to some of the brighter people that I saw in the company and junior developers that I thought were good and I said, can you do this? And they passed the test so I said: yeah, that's what I'm looking for. And some people just sit there and they don't know what recursion is [21]. I don't know. And some people struggle for a bit and then they figure it out. Usually it's not something you just know how to do. You shouldn't memorize this stuff, but if you can figure it out in 10-15 minutes or I know that you're getting there, then that's the person I want to hire. I don't care what your resume says; I don't even care if you can figure it out. I'm saying if we can talk through the solution because you're not solving puzzles most of the time. Sometimes you are, but mostly it's like: ok, we're agreeing on doing something. Can you take what we've agreed to do and actually express it in code? And if you can do that, then I don't think it matters what language you give the person or whatever. If they can express it to a computer and get the computer to understand it, they can learn any language. That's my philosophy.

00:39:12 [BT]

Do you find k as a language that once you have somebody who thinks that way, who solves problems, works their way through problems, who can work with somebody else to work their way through problems; do you find that k is a language that makes that easier?

00:39:26 [MW]

I think it certainly gives you the tools to do that. Well, it's kind of like J, even more so; you have so many more primitives. k is a much smaller subset. k is like just a traditional scripting language with a lot of shortcuts and a nice syntax for doing that common thing. It's compressed. k is like a compressed traditional language in my point of view, or at least you can treat it that way, and I think a lot of people (do). I guess what I'm saying is k definitely feels to me like you can use it as a traditional language. If you are used to Python or JavaScript or anything like that, you can write code that is similar to what you're used to, right? I don't think that's the case in J, but k feels like a traditional language, just a little more compressed.

00:40:11 [MW]

And then if you want to, as you get used to it, you can start seeing: oh, I can do this even faster; and you know, I can use this little trick and I can pick up the idioms and I can start using some of the array concepts, the vector concepts, to do that. We have wildly differing styles of code in our code base; we tend to have one or two people owning a particular thing, and if you look at different parts of the code, they look different. It's a web platform, right? So there's a whole database component, and then you know that's a massive thing that's all about distributed computation. We've got a thing about managing the users, and all the things that you've uploaded, and where your data is, and what's it called and who has permission to, and all that stuff.

00:40:58 [MW]

And that's a different thing. And then, like the stuff I work on; I work on the user interface stuff. So like we have this on top of our language, we have this ability to to build components and to draw them on the screen and compose. It's like a RAD tool; you can compose little widgets and you can say "I want a drop-down here", and it's bound to the database, so you can say "I want a drop-down that has everything from this query" or "I have a bar chart" or whatever it is, it's all based on queries against your data. I work on that system that helps you build applications on top of our system. And so externally, that's like you're doing this XML code. But what I do is I provide chunks of little JavaScript and k that talk to the system and it's, you know, a lot of string manipulation and building up things: it's not cryptic. On that job listing I show that really cryptic (what I would consider cryptic if I were looking at it as a newbie code) for the Mandelbrot set, but then you scroll down, well, it says something about like: "don't worry, not all of our code looks like this because we don't have 1500 Arthur Whitneys [laughs] or even 100 Arthur Whitneys". You look at Stevan Apter's code and it's very clear and mathematical and one line at a time, but he's an outlier; most of our code is not. I consider Stevan's code extremely well crafted, but he's writing a parser and writing that one thing and it's, you know, it's a module that fits into the system, but other parts of the system are just much, much closer to traditional work because that's the nature of what's being done. We're sticking strings together, making HTML things, and we're, reading a form and we're dealing with dictionaries.

00:42:50 [Stephen Taylor]

So Michal, you've done a lot here to demystify k to show that it's not a cult. You don't need to belong to a race of semi-gifted [not-completed].

00:43:01 [MW]

We do pray to Arthur every every Thursday, so.

00:43:06 [Stephen Taylor]

Well, I was about to say you don't need to belong to a race of semi divine beings in order to use it. You could, it's enough to be curious with a k and to get in and learn how to write good application code in k.

So the question that leads me to is: Would you go back? Would you. Can you imagine leaving doing development in k to go work in? Oh, I don't know Java, C++.

00:43:39 [Michal Wallace]

Yeah, I don't. I don't think I'm married to like you know, even now. I'm not like a pure k developer, so I'd say. I'd say I do about 25% K. Yeah, probably it's probably evenly distributed at this point because I, in addition to the GUI, I also work on the the our API so that that same technology for building applications you can use to build like dynamic endpoints that. Like feed your query as a certain kind of JSON object. So to write clients for the API. I do a lot of work in Python to like provide demos for people. We also. I do a lot of testing like a uh. I write test code in Python, so it's about 25% Python. Which which is new. That used used to be very rare. But I started doing a lot of it.

And then I'm writing a lot of HTML and JavaScript for the UI components, and then I spent a lot of time writing. Uh, code in our macro lang so it doesn't have a name. We just called it 1010 macro language. But it's in a lot of ways.

It's a it's a, you know. It's so. So when you query our system you can write SQL, but you can also. Maybe I should back up. Our system was designed so that you would upload data. And you would have a web interface. You'd see your table and you would pick from a menu. I want to. I want to sort this table. I want to select this table. I, you know, I want to tabulate on this column and you give me a summary or whatever I want to do and you did it all through the GUI.

So there was, there was no language, there was no typing. There was no query language, it was just just select things and now save this. This collection of something. So somewhere along the line we created. A notation for an XML notation for just storing all the, you know, the work that we did. It had previously just been stored. As like k objects. So that your query was just, uh, k object. That was a list of steps you did to to make this thing happen, and then, what we call, macro code grew out of that. And so now we have, it eventually became like the internal. Like like the. The actual representation is now this this XML thing so, but it's a series of steps. And they're all in a way very similar to the k primitives. I think I think Joel said in his interview. If he didn't say it there, he said it to me numerous times, that: He couldn't have built this company without catering. It was you know. The k primitives mapped very closely to operations you do on a database. Select and transpose and things you would want to do to columns and then you know even you know the grade operators. Why there's no sort in K. There's a grade operation. In case, so you have a column that says. You get to graded that column on what that gives you is the indices that tell you what order. I know Marshall can give me the actual definition 'cause I stumble over it, but. But you know, it tells you what it gives you: an array that tells you what order this. It tells you the order of these things should be in what order to put these things in.

And then you can apply that to all the columns.

00:46:31 [Marshall Lochbaum]

Yeah, how you would reorder them in order to sort them.

00:46:34 [MW]

Yeah, and so like that's the thing that you can apply to. You inspect one column and now you can apply it to all your columns. All the vectors. So you as a user of the system, you don't see that internal vector. But there's one operation called,you know, sort, or whatever. You tell you what you wanted to sort by and so unlike or in addition to the normal SQL operations that you can do. We have a SQL operations, SQL parser that parses SQL and turns it into these operations. But you can also just write them directly. And so then you can tell it step by step what you wanted to do in your own in your own macro codes. Things that would be you would have done by following along in system. And then on top of that we started saying, well OK. Now let's add if-then and loops and conditionals and things like that. So, you can generate this macro code using an extra. There's like a another subset of languages called. We call, block code, which is the. These blocks of things that give you procedures for generating procedures and conditionals and loops and variables for generating the the query that you eventually want to query. And then on top of that we added this this user interface stuff. So there's a tag called Widget now, or dynamic and then inside of that you have widgets. Since you lay out a user interface that can then call all these dynamic and dynamically generate queries for you, or I don't think any of our customers have ever done this. But if they wanted to, they could write logic in K3, or they could write logic in Python now. That's something we're adding to that. We've just started adding to this: Python and R. But I do a lot of work in macro and I would say it's unfortunate that I can't. Like you know it's a proprietary system, but I would say and you know. I would say macro code is one of the the nicer languages that I've ever used in my life because it does all that data binding for you and so you can whip up a user interface and just. We call them quick apps. You can just like: couple lines of code and now you have an application that queries and gives you interaction on your data and so, and yeah. I would say the combination of macro code and k and then I still like Python and I like web stuff, so I like doing all those things and I liked a variety of it and somebody doubled my salary. Sure I could. Go do COBOL you. I don't. I don't know how to answer your question about The ... If I would go back.

00:48:51 [ST]

No, that answers it. You have to be paid a lot too to leave it.

00:48:56 [MW]

Yeah, I would say so. I'm very happy where I am now.

00:48:59 [ST]

But we've also learned that you can. You can learn k and work with it and work with it for years, and you don't wind up in the. Forest baying at the full moon.

00:49:06 [MW]

You know the one of the very first questions I asked. 'cause I think hey, I asked Adam like I our chief scientist. Some are you guys functional like: Can you read each other's code? I mean can you really do this at scale and part of the answer to that is that. You know, I think Eric Iverson said this too. Is it like a, uh J team is like one person? A k team is usually one person you can.

You can manage a pretty big chunk of the system. With just one person because you have so little code to do and so part of it is ownership. But you know, I've looked at many, many other k developers code, and it's just. It's not. It's not intimidating. There's occasionally. There are people who use the primitives differently than I do. Actually, I'd say like you know, again, we have different styles like you know. Steven Apter like knows how to use every single primitive and like you know his code is sometimes intimidating to look at. But then when you look at it like oh, this makes perfect sense. I would be more than happy to maintain some of this code going forward. For the most part like we don't find ourselves. Like you know, going in and like dealing with. I'm not gonna say there's no bad code like, so it's not that there's for the most part like it's not a pain to like pick up someone else's k code. I think the troubles, that we had, and, is that, like sometimes when you've got a piece of the system that's been handed from person to person to person to person you you get like and this happens anywhere. Like any language you can get like code that is way too long. Like you know, a function that has grown to 36 pages when you print it out. This is not, uh, not fiction.

00:50:44 [Conor Hoekstra]

In k code? That can't be possible.

00:50:46 [MW]

Yeah, and I mean it's this complicated system. Yeah, this is. This is not. This is not this is I would think not specific to any particular.. Like this is a function of organizations writing code, right? You get to a point where once a piece of code gets bigger than like a screen full. You know you can't. You people use their. Their screen is like working memory like your visual field is part of your working memory. You can't keep it all in your head, and so the more that you can keep on the screen, the more working memory. Yeah, and so as soon as it scrolls off the screen. That's more that has to go back in your head to keep track of. Everything unless you have tools like compress it and you know give you whatever, so. Or you know there's some logic to it that you can understand like this is. Part is isolated. But like I'm always scared that like, can I even introduce a variable that might have been, you know, declared your way up here somewhere. Yeah, I don't know 'cause I can't see it on the screen. I think Joel said this: was it that one of the benefit I know Arthur. The reason Arthur doesn't put any whitespace is because he wants to see everything, and that is a tremendous benefit of these languages, and that runs into a conflict with the the natural tendency of an organization to sometimes when you're handing things back and forth and sometimes just. Yeah, technical debt like you always have technical debt that and something. This is 1 particular thing where I think it can get out of hand. And you lose that benefit of k when you look when you let that happen. Course, the answer is to factor things into smaller pieces, and I'm not blaming anyone like my own code is that. It's like if I have a particular thing, sometimes the logic. There's more more clauses in the in the if, else or whatever. You just. You gotta stop and refactor sometimes. Sometimes that doesn't get done, so I picked up racket of recently and specifically to write up K3 parts for it to try and make a refactoring tool for that problem. And it's only like in one or two places in the code, but. Rather than do it by hand, I would just wanted to make a tool. So didn't get very far. I got I made a parser in an editor and it highlights the code and then it parses our codebase, but I didn't get to the refactoring part yet.

00:52:43 [CH]

Maybe in the future I would be remiss if before we ended this conversation, I didn't ask a little bit more about. We mentioned it at the beginning and a couple of times throughout your YouTube Channel, but also I don't think we've mentioned since the beginning that you stream sort of live coding sessions on Twitch. So on top of. Sort of plugging those, and maybe giving a pitch for if people are interested what they can expect both out of your YouTube channel and your Twitch stream. Yeah, feel free to comment on that and then also if you want to plug other stuff your twitters or if your other places 'cause it sounds like you know a part of your career story has been taking the role of a teacher and just sort of as part of your job. But also, just as you know, you spend time on on different forums answering questions. So yeah, where can people find you and what can they expect on your different sort of platforms?

00:53:35 [MW]

Uhm yeah, I'm on Twitter for now at least. Just as long as it's still in business I'm on. Second, in Tangentstorm you know GitHub as a bunch of stuff. You can find me on. Yeah, I haven't streamed on YouTube and Twitch as Tangentstorm and also on YouTube. I started the second channel called Tangentstream where I'm putting some. So I had a, uh, a daughter 15. My second daughter was born 15 months ago, so she's. So about that time I was streaming everyday because I was getting up ridiculously early in the morning against the ridiculously early J morning show. And just watching her for awhile I've been I've been. I've been taking the night shift previously and then we got to the point where she was sleeping in the morning, so I had for about a month I was I was streaming every single day. Or every weekday just working on a tool for it was called jJPrez [22] the tool for. That I'm using too. Record interactions with the J REPL you know, record and play it back in. The video so I didn't want to. I hate typing live in a video clip while I'm trying to talk 'cause I you know, I make so many typos. And just it's just a waste of people's time to add I much rather have it go smoothly in the video and have the. Yeah, you know, be able to synchronize. You know, the the cadence of each little keystroke. Yeah, 'cause you can. You can pause in the middle of it. What were you doing so I filmed? I think 23 episodes or I published 23 episodes of me working on this program. Fairly large day program and just, you know, talking about what I'm doing and explaining things as they go along. So that's probably the bulk of my videos, but I also try to do. Like the presentations that I've scripted and edited and like you. Know polished a. Little bit, and in fact, that probably the biggest headache I had when I was the streaming was like I I hate just uploading a live stream every day and not being able to edit it. The editing just thinking creating so I decided to post a chance. And I haven't. I haven't done a lot of live streaming because. I was in a tiny house with the baby that stopped sleeping at at 7:00 o'clock in the morning and he was, you know, screaming. We were in a tiny two-bedroom apartments and we finally moved out to a house and thought I was going to have some time to to start doing streaming again and then we've had. There's a fixer upper, so we have contractors here all the time, but sometime in the near future you know when things settle down a little bit. I hope to get back to streaming that's I don't know. It probably won't be early in the morning, but at some some point in the day I plan to do some regular streaming. I gotta find a project to work on it and then periodically I have a bunch of talks planned. Or, uh, you know I made this tool to make videos and I made the they dealing cards in J video which is all about permutations and I made a. A little short video about. Yeah, there's a YouTube short about some idiom that I saw on the J wiki that I didn't recognize and turned out that was. A It's now a primitive and that's why I never but he couldn't. I didn't record. But uhm. But I made those tool I made those videos with this tool that. I made and so like. There's more stuff I want to do and you keep working with so um. I guess that's it then yeah, come watch me on YouTube if you if you feel like listening. To me, ramble and.

00:56:56 [Adám Brudzewsky]

But I don't understand the thing. If you had a a baby that was screaming early in the morning, how did you stream and not have background noise?

00:57:03 [MW]

Oh no, I mean. I mean, she was sorry. I meant sleeping in early in the morning.Oh, she was she she was, she was.

00:57:09 [AB]

You scream all night and then you're up. And then you might as well record OK.

00:57:13 [MW]

Yeah, she she she was. She was taking her nap right around that time in the morning and I had gotten used to getting up that early in the morning anyway, so so I started taking the morning shift while everyone else in the house was asleep and then. But then towards the end of it, like. She started waking up earlier and. Earlier and just yeah. There I just I couldn't find a quiet time to record anything.

00:57:35 [CH]

She found out you were doing J and she just she couldn't sleep. She couldn't nap.Too exciting, you know.

00:57:43 [CH]

Alright, well we will. We'll put. Links to all of those sort of yeah YouTube Twitch, Twitter, GitHub, all that stuff. So folks want to go follow you and whenever, yeah, the construction ends and the you know things settle down more.Whether that's seen in the near term for future or the intermediate to long term, future people can, yeah. Check check out the. Videos and live streams there. And yeah, I'll. Yes, thank you for coming on and I think there's a bunch of this will. Be interesting for the listeners. I think that you know, I I definitely I have to go read that Python paradox paper which is. It's paradoxical that Python is the language chosen, 'cause it's now like the number one most popular language in the world, but at the time, like I would much more of his have expected, like a language like. Haskell, which I've heard the exact same argument at Haskell meetups that you know it's not actually. People think that there's a problem. In finding Haskell developers, but if you really put yourself out there, there are a ton of people looking for Haskell work, but programming in some other language because they think that there are no Haskell jobs. So really, there's these two sets of people that are perfect for each other, and if you. Let's go put the company out there like then they're looking for Haskell. You do that self selection thing that there are people that are very passionate about that technology. So not only are they people that know Haskell, but they're people that you know, really, really know Haskell. And would you know, even trade, you know, a certain amount of you know whether whatever Google is paying them to code in golden language X to. To be able to develop in that. And yeah, I'll have to go read that paper. Interesting, very funny that at the time it was written they chose Python, 'cause now it's gone on to become the exact opposite of what that paper is.

00:59:19 [MW]

He he he mentions a couple other languages there. One of them is J. At the at the bottom of that paper you. Mentioned state and puts it in the same category.

00:59:25 [CH]

Interesting, interesting yeah.

00:59:27 [MW]

It just it just you know and they had a thing called CP3 [23] computer programming for everybody. They it wasn't just luck, right? It was a little bit of a this is, you know, palatable. And looks clean and what? They've made an effort to do this right. I haven't. I haven't watched the episode so you guys did. On the popularizing the array languages episodes or listen to.

It, But I'm eager to listen to that 'cause I think. I like I, I don't think it's out. I don't think it's crazy to think that.That that that you know J and APL could. Be far more popular than they are. I mean I have like 5 books on my shelf that were like mass market books that people wrote that are all about, you know.Statistics in APL, Algebra in APL, you know, like this was. This was at one time a market leader in in program in the World of programming I Mean it's it could be again.

01:00:26 [CH]

Yeah, it's a very I love there. I will add this to the show notes as well, but there's a video by Richard Feldman called. Well, I think we've mentioned it before on this podcast. Why isn't functional programming the norm? [24] And it's a talk about what makes a programming language popular, and how come there's no sort of functional languages in the on average across the different programming language rankings that there are, you know, 10 different ones. Typically it's you know procedural or imperative languages like you know Java, C++, Python, etc. And the talk talks about, you know: What is it that makes a language become super, super popular? And anyways I'll for those that haven't seen at all, I'll leave that as a exercise if you want to go watch it. But yeah, those kinds of discussions, 'cause it's. I think it's a lot less. It's like not black and white. You know what? What actually makes a language quote, unquote, successful. And also defining what success is. You know it's not just necessarily always the number of people using the language, but. Anyways, yeah thank you for coming on. I think the listeners are definitely enjoy this and who knows, maybe in the future if there's ever such a k posting that 1010 data is looking for, we can come back and in retrospectively add to the show notes. In case it's you know, two years from now. It's 2022 right now, but for for all we know. So you know, there's someone in 2024 or something when there's a new posting and you know. Check the show notes 'cause there might be a a new posting in the future that has has yet to come up yet. But yeah, once again, thanks for coming on. And with that we will say: Happy array programming!

01:02:02 [ALL]

Happy Array Programming!