Yelling at Roadshow Edition

JR Reynolds: Hey, hey, I'm a clapper. Man, you l let me worry about that, okay? Okay. All right. Hey, hey, hey, folks, it's Yelling at Robots, Roadshow Edition. I am in Spain. Ian is in ⁓ New York and America. They let him back across the border. And we are both in hotel rooms. He seems to be in a fairly normal one. I'm in some of no no no no no no no. I'm in my brother-in-law's apartment. ⁓ nice. I can afford a hotel room for three weeks in New York. I don't know, maybe the company pays for it. You know, you got that big you got that Ellis, you got that AI money feeding in there, pal. Well, I am in a hotel room and it is called the Radison Red, which seems to be some sort of combination of dystopian satanic ⁓ generation Z. Is that the one? Like this is the picture I have above my bed. It's very, very disturbing. And it's all red. Like the whole all the different things are. It's red and black. Yeah. Nice. Amazing. Yeah, well here we are. and ⁓ it is Yelling at Robots episode sixteen. It's crazy. The number just keeps going. Look at the hand. ⁓ my That's my hand. It's still big. Action! Get ready. It's time for world famous Yelling and Robots. Starring Tech Legends, Joshua Raphael Reynolds, and Ian Thalonius Bosberry. The show starts now. This is look at those f<unk>ing games. Look at those monsters. ⁓ You know what? It's fine. This is Roacho Edition. This is a learning experience. I I went to ⁓ a local media market and I picked up one of these this is a you know, this is for like from a TikTok influencer. I can bring this microphone with me anywhere I go. It looks a lot lighter than this. This is this is I'm gonna have some carpal tunnel issues after the test. ⁓ you didn't you didn't bring your arm to Jesus Christ. So, folks, first off, welcome to the road show. when we're on the road, we don't really do much preparation or even thinking or talking about stuff. I mean Well, the first time we did, the first time was our first episode where we agreed we will put in work ahead of time. And I had notes, I was well prepared. I was but this time no. This time no. Do remember that? That was the ⁓ the original. I do, I do. I remember I remember you were in yep, you were in Poland. Yeah, European edition. Well that was European edition. I was still at home. ⁓ right, you were still at home. Yeah. Yeah. well what else we're gonna talk about? ⁓ Microsoft's fun co-pilot little ⁓ adventure. ⁓ did you hear about this at all? I read it, ⁓ I read it briefly. ⁓ I don't fully remember everything about it, but I do remember Feeling well, I wasn't really that scared because I don't use I don't use the three sixty-five stuff at work that much. I guess I was more scared of like our company or many companies might be really at risk right now because some people are really using Copilot. Deep in. Yeah. For those of you who don't know, this was I don't know if they observed this exploit in the wild, but it was a pretty ⁓ straight straightforward-ish exploit of basically getting it was ⁓ the way that ⁓ the model loads prompts. ⁓ f now I sh we do need to do more. Amazing. essentially what it did was it used a specially crafted URL. It was so the copilot actually executes so when it's retrieving text from the web. It actually does put it inside of some sort of ⁓ sandbox and protects it for or does it it doesn't execute any or retrieve any URLs while it's parsing and loading the text, except if it's a trusted URL for like Bing or an internal Microsoft service. And so what they did was they crafted the URL so that it was a URL for Bing, but added guess what? A query parameter ⁓ man, yeah. And re to redirect or pull from a malicious URL that then and was a prompt injection instructions to the bot or something. And I don't remember how they got the two FA codes out of that. I think it was to actually not sure how they got got to the two FA codes. Are you are you loading it up right now and reading it? I'm looking at it right now. Yeah, yeah, yeah. I I remember the they were using the query parameter in the URL. So like if for those of you who don't know Query parameter. Usually, like I think it even Google works the same way, but it's kind of a convention. If you have a ⁓ full sec full text search query like what is temperature in New York City to Google, you'll see it reload on the next page. And in your URL, you'll have a question mark and a Q equals and you'll see what you typed in ⁓ in probably in quotations. And that's what a query parameter is that ⁓ Josh is referring to. So you can put anything in there. And it's just a little bit you stick at the end of a URL to make someone do stuff. Yeah. So the sequence looks like Copilot starts streaming its response, which includes an image tag. The browser sees an image, renders it, fires off an HTT request to the source URL, Copilot finishes generating, the guardrail wraps everything in a code block. That's right. It was that it was that the request was getting fired before the guardrails were being evaluated. That was Yeah, yeah, yeah. Exactly. What's the beef here? What is interesting about this? What's the beef? What's our takeaway? as good if you have experience as a software engineer, historically, in the last 15, 20 years, you tend to write automated testing for your software. So unit tests, integration tests, end-to-end tests, which we've talked about in the show before. Yes. And I think we've talked about this as well. Is that the Problem with any of those tests when I write my software is I can only test things I can think of. As opposed to say property tests, which is more you define properties around your your system that have to hold true, and then you can use a property bas a property testing framework to generate examples that try to prove you wrong. Mm-hmm. I would expect that. Especially if you're using ⁓ if you're more AI, write the code for the AI, do more AI code. We don't read the code as much. It's AI, AI. That the AI is more close these it's more if you if you look at that as a spectrum, AI is more towards the property based side of things where the AI does think of things that you didn't think of, dumbass. But it just seems that like That never happens. And then the people writing it didn't obviously think of this sort of thing, or didn't think of the ordering of like when to apply guardrails for sequence of events. And yeah, I don't know. So this is just di another situation where it's it does feel like this was probably because somebody used AI and more AI to write the code for the thing the same way that Meta did. And didn't think through like never or not even think through, it didn't look at what was happening and think through what could happen. I don't know. Those are my thoughts. ⁓ this is just new technology. So kind of like in and you know, this in some way, this is like a race condition that we never would have even conceived of because of the chaos magic implementation of an a you know, so like that's fair. You know, ⁓ Microsoft has had an extremely aggressive rollout of copilot. We're just gonna shove this thing into every goddamn system inside of Microsoft. And it's like for better or worse, this is the world's business runs on Microsoft. And it's like, so that's that I guess that's part of it for me too, is I ⁓ I mean sh happens, you know. I I like I don't know that this one d in the article did it say if like ⁓ this was an active exploit or this was just ⁓ zero day that some security researchers discovered. ⁓ to bring it with my understanding was that it was security from Veroneus devised an exploit. No, this was a security form, I think. Yeah. So it's it's a bit of confirmation bias or just a reminder that, you know, these systems can't be trusted and use them accordingly, just but with with caution. Use with caution. Yeah, that's it. ⁓ we're at 32 minutes. yeah. Let's ⁓ let's start there. ⁓ so tired. Ian. What time is it there? It is almost ten o'clock. ⁓ wow, that's late for Josh. Yeah. That's right. I've I'm well past my bedtime, folks. I'm gonna get up and ⁓ hit the hit the hit the the weird red gym. I went to the gym, I checked it out. It's like there's like it's it is All red light. So you go in the gym and it's I know. It's really it's gonna be interesting. We'll see. well I guess that's a show, folks. That's it. ⁓ don't forget ⁓ did we get any ⁓ submissions in our in our contest, by the way? I don't even know where we'd look for that. On the video itself, maybe? On the video, yeah. In the comments on the video. Just throw your big neat hand in the video. Yeah, let's take a look. ⁓ Good. ⁓ well Ian's pulling it up, folks. ⁓ zero butt zero comments on the on the ⁓ on the video, but was it the was it the right was it the right one? We did get a comment on one of our videos though. I don't know if you probably want to market meta says yes, right? Yeah. Meta says yes. ⁓ it was it was this one, I do believe. I actually I'm pretty sure it was because I still haven't had a chance to ⁓ add chapters and or ⁓ smaller videos. So maybe it'll ⁓ maybe we keep it running. Or do I just can I make my entry? What was my entry? I don't remember. I texted you. ⁓ the Zuccubus. The Zucubus. Look at you look at you. Like that so much. I've thought I've thought about it not at all. I've had a lot of lot of sh cooking. Fair. All right, folks. That's your episode for the day. Road Show edition. I'm gonna be here for the next three weeks, so if we do another one. Are you serious? ⁓ yeah. Yeah. Madre. Well, next next episode, let's prepare. All right, folks, you heard it here first. Next episode, we will be ready, we will be prepared, we will get together, we will discuss, we will have a plan, and it will be good. Until then, enjoy the succubus. getting zucked off by the succubus. Over to you. It's a wrap. You just survived another episode of Yelling at Robots. Better luck next time.