# MediaChunks

chunk_text:
  these issues. If you want to know more about that, check out our website. There has been great progress in understanding these issues and in mitigating the issues. But there's still a lot of work to be done for asset owners. And in the broadest sense, it seems that these kind of mission-critical technologies are just not being assessed enough. And with that, I conclude this presentation, and I'd love to take any questions if there are any. Yes, thank you very much. Are there any questions in the audience? One question. Tetra is here to stay for a little while before we go over to a new system. Would it be feasible to tunnel some other protocol on top of it, which is encrypted, to overcome these hurdles? Yes, definitely. Etsy themselves, they envision a multi-decade future for Tetra. At the very least, the networks that are out there are not leaving anytime soon. So indeed, it's possible to use something like a VPN before you start sending your IP traffic over Tetra, which would be a very good idea. For existing networks, however, especially in the industrial context, due to the strictness of the acceptance testing and the safety engineering involved, it's very hard to make these upgrades or modifications because you introduce new failure cases. You have to do a lot of work, again, proving that the safety of the plant cannot be compromised by adding security. So it's a great idea. It should be done, but in practice, especially for existing plants, it's not that simple to actually roll out. Thank you. Well, that concludes the time we have for this talk. Thank you very much, and I would like to offer you with a token of our appreciation, of course, like everyone else. Thanks. Thank you very much. My pleasure.


chunk_text:
  safety engineering involved, it's very hard to make these upgrades or modifications because you introduce new failure cases. You have to do a lot of work, again, proving that the safety of the plant cannot be compromised by adding security. So it's a great idea. It should be done, but in practice, especially for existing plants, it's not that simple to actually roll out. Thank you. Well, that concludes the time we have for this talk. Thank you very much, and I would like to offer you with a token of our appreciation, of course, like everyone else. Thanks. Thank you very much. My pleasure.


chunk_text:
  { "webpage_url": "https://www.youtube.com/watch?v=vBpUaIPUOHY", "title": "How To Crack Seven Billion Passwords? - Jeroen Van Beek", "description": "", "channel_url": "https://www.youtube.com/channel/UCgJoGNj3B2Mt2F3TUblcoOA", "duration": 1863, "channel": "OrangeCon", "uploader": "OrangeCon", "upload_date": "20240909" } This text was transcribed using whisper model: large-v2 Thank you. My name is Jeroen van Beek. This is a summary of my professional history. Not that important right now. This is Taylor Swift. She was recently in the news, linked to an IT security incident. What happened? Someone else? No one? There's a hint on the screen. Taylor Swift ticketing is handled by Ticketmaster. Ticketmaster is a huge company. Ticketmaster uses a number of IT providers. One of them is Snowflake. And Snowflake got hacked. There are some reports out there covering the incident response from Maniant. And there are a couple of interesting findings. One of them is that there was no mandatory multi-factor authentication for administrators. And at least about 80% of the accounts leveraged by the attackers were already known. And were already exposed before the breach. So it's a classic example of credential stuffing. And not taking even basic measures. And as a result of that, with a username and a password, you could breach the account data of over half a billion of people. So this is a really huge leak. It's not only Ticketmaster. There are over a hundred other companies affected. But probably the threat actors will continue exploiting those in the coming couple of weeks or months. Another one. Uber. There are a lot of incidents with Uber involved. This one is about a settlement for $148 million. And it was IT security related as well. Anyone? What happened? No one? Uber uses GitHub for source code storage. Again, there was no mandatory multi-factor authentication. In this case for the developers. And the developer reused their passwords. So they used the password for other services as well. And one of those other services, or at least one of those other services, got hacked as well. The password of the developer was cracked. The threat actor figured out that it also worked on Uber's GitHub and got access to their Uber GitHub. And looked into the source code. One of the things he or she found were storage credentials from AWS S3 bucket. And with that, they could recover almost 60 million Uber user account details and 600,000 driver data files. Bonus. Uber did not report


chunk_text:
  over half a billion of people. So this is a really huge leak. It's not only Ticketmaster. There are over a hundred other companies affected. But probably the threat actors will continue exploiting those in the coming couple of weeks or months. Another one. Uber. There are a lot of incidents with Uber involved. This one is about a settlement for $148 million. And it was IT security related as well. Anyone? What happened? No one? Uber uses GitHub for source code storage. Again, there was no mandatory multi-factor authentication. In this case for the developers. And the developer reused their passwords. So they used the password for other services as well. And one of those other services, or at least one of those other services, got hacked as well. The password of the developer was cracked. The threat actor figured out that it also worked on Uber's GitHub and got access to their Uber GitHub. And looked into the source code. One of the things he or she found were storage credentials from AWS S3 bucket. And with that, they could recover almost 60 million Uber user account details and 600,000 driver data files. Bonus. Uber did not report this and paid the hackers $100,000 to keep quiet. Again, classic example of credential stuffing. You only need to use a name and a password and the impact is enormous. So you might ask yourself, are account takeovers the new zero days? If I look back in my experience as over 20 years professional pen tester, it all started mainly with exploiting operating system issues. So the Windows and Solaris flaws. Later on, there were software flaws, backup agents, databases, those type of things. Followed by application code flaws like SQL injection. But nowadays, if the basics are covered by a company, typically in my experience, the easiest way in is account takeover. I'm not the only one saying this. For example, Verizon publishes an interesting report on a yearly basis and they say credential abuse is the big thing to focus on. One of the reasons, I think they say this, is that we see a trend. And the trend is that there's a huge increase in the amount of available breach data and there's a huge decrease in the cost and the complexity for an attacker. So a smart teenager can do this and can do this successfully.


chunk_text:
  this and paid the hackers $100,000 to keep quiet. Again, classic example of credential stuffing. You only need to use a name and a password and the impact is enormous. So you might ask yourself, are account takeovers the new zero days? If I look back in my experience as over 20 years professional pen tester, it all started mainly with exploiting operating system issues. So the Windows and Solaris flaws. Later on, there were software flaws, backup agents, databases, those type of things. Followed by application code flaws like SQL injection. But nowadays, if the basics are covered by a company, typically in my experience, the easiest way in is account takeover. I'm not the only one saying this. For example, Verizon publishes an interesting report on a yearly basis and they say credential abuse is the big thing to focus on. One of the reasons, I think they say this, is that we see a trend. And the trend is that there's a huge increase in the amount of available breach data and there's a huge decrease in the cost and the complexity for an attacker. So a smart teenager can do this and can do this successfully. So what do cyber criminals do? The most important part is the input. So they got breach data from websites and from hacked computers. On the other end, we got the output and that's money. So with extortion or ransomware or stealing money directly, the criminals will get your money. Well, what do we do at Scattered Secrets? We provide some services in two categories. One is we provide companies with the breached credentials of their company, the domains that the company owns. And the other ones is we add an extra password control. Probably we all know those controls. So you check for a password that's 8 long, 10 long. And some complexity is applied to the passwords, uppercase, lowercase, letters, numbers, et cetera. We provide an additional check and that is the password reused. And if it is reused, you should not use it. Well, if you want to run a service like that, you need a large quality data data set. And the question might be how to get it. Well, first of all, you need raw data, a lot of raw data. That might sound difficult, but actually it's really, really easy. Perhaps some people are


chunk_text:
  So what do cyber criminals do? The most important part is the input. So they got breach data from websites and from hacked computers. On the other end, we got the output and that's money. So with extortion or ransomware or stealing money directly, the criminals will get your money. Well, what do we do at Scattered Secrets? We provide some services in two categories. One is we provide companies with the breached credentials of their company, the domains that the company owns. And the other ones is we add an extra password control. Probably we all know those controls. So you check for a password that's 8 long, 10 long. And some complexity is applied to the passwords, uppercase, lowercase, letters, numbers, et cetera. We provide an additional check and that is the password reused. And if it is reused, you should not use it. Well, if you want to run a service like that, you need a large quality data data set. And the question might be how to get it. Well, first of all, you need raw data, a lot of raw data. That might sound difficult, but actually it's really, really easy. Perhaps some people are familiar with this website, Breach Forums. There are a couple of them, or actually quite a lot of them. It's not dark web. It's not expensive. You can download a lot of data. This is sorted by the number of accounts. You see 800 million accounts. You can download it in most cases for free. If you're in a hurry, if you pay one or two euros per data set, you're good. So it's really cheap and really easy to get a lot of data. Nowadays, the more interesting channels are file sharing services and Telegram, especially the Telegram channels. There's a lot of data over there. If you got the raw data, you need to extract credentials. So in our case, we extract the emails and the password hashes. And if you did that, you're at step three. That's the most interesting one. You can do some analysis and cracking, actually enriching the data and getting valuable information out of the raw data. But what kind of data will you see if you collect and extract data from data breaches? First of all, started in the early 60s, we had plain passwords in the first time sharing machines. We


chunk_text:
  familiar with this website, Breach Forums. There are a couple of them, or actually quite a lot of them. It's not dark web. It's not expensive. You can download a lot of data. This is sorted by the number of accounts. You see 800 million accounts. You can download it in most cases for free. If you're in a hurry, if you pay one or two euros per data set, you're good. So it's really cheap and really easy to get a lot of data. Nowadays, the more interesting channels are file sharing services and Telegram, especially the Telegram channels. There's a lot of data over there. If you got the raw data, you need to extract credentials. So in our case, we extract the emails and the password hashes. And if you did that, you're at step three. That's the most interesting one. You can do some analysis and cracking, actually enriching the data and getting valuable information out of the raw data. But what kind of data will you see if you collect and extract data from data breaches? First of all, started in the early 60s, we had plain passwords in the first time sharing machines. We understand now that that's not a good idea because if someone breaches your password file, all passwords are breached instantly. But plain storage is still used nowadays. So I think on average, we still see one or two raw files containing plain password storage. Don't ask me why, but it's still used. There are a couple of simple fixes or not fixes. One was obfuscation. The second one was the introduction of Descript back in 1974, which improved the situation. Other types of hashes you see are basic hashes. So for example, the MD and SHA family, MD4, MD5, SHA1, SHA2. They were introduced in the early 90s, but there are some issues. One of the issues is that the same password will result in the same hash. So you can create a big lookup table. So if you can get your hands on a hash, you can just look up what the password is. And then the second thing is dedicated password cracking was introduced. It sounds pretty normal now, but at some point in time, there were no dedicated password crackers. Familiar ones for some people are Crack and Jack, 1991-93. One that's probably still very familiar for


chunk_text:
  understand now that that's not a good idea because if someone breaches your password file, all passwords are breached instantly. But plain storage is still used nowadays. So I think on average, we still see one or two raw files containing plain password storage. Don't ask me why, but it's still used. There are a couple of simple fixes or not fixes. One was obfuscation. The second one was the introduction of Descript back in 1974, which improved the situation. Other types of hashes you see are basic hashes. So for example, the MD and SHA family, MD4, MD5, SHA1, SHA2. They were introduced in the early 90s, but there are some issues. One of the issues is that the same password will result in the same hash. So you can create a big lookup table. So if you can get your hands on a hash, you can just look up what the password is. And then the second thing is dedicated password cracking was introduced. It sounds pretty normal now, but at some point in time, there were no dedicated password crackers. Familiar ones for some people are Crack and Jack, 1991-93. One that's probably still very familiar for most people is John the Ripper, introduced in 1996 and still actively is supported nowadays. Well, what was the fix? The fix was salting. So you're not storing the hash, but you're storing the salt and salted hash. And the result is if you got, for example, a thousand times more hashes, it takes a thousand times more computing power to achieve the same result. So that might sound good. But it's not that good. Again, some issues. It introduces herd immunity only. So if you got a thousand or ten thousand or a million accounts, it works. But it doesn't help against targeted attacks. So if you want to crack a single salted account, it's just as fast as it previously was. So it's good for protecting your whole population, but it's not that good for protecting one or two admin accounts. Then we got Moore's Law. Moore's Law, there are different interpretations. But in this case, I mean that back in the days, the CPU power doubled about every two years because we were cracking on CPUs back in the days. And the algorithms are unchanged. So after two years, twice as much computing power. After four years,


chunk_text:
  most people is John the Ripper, introduced in 1996 and still actively is supported nowadays. Well, what was the fix? The fix was salting. So you're not storing the hash, but you're storing the salt and salted hash. And the result is if you got, for example, a thousand times more hashes, it takes a thousand times more computing power to achieve the same result. So that might sound good. But it's not that good. Again, some issues. It introduces herd immunity only. So if you got a thousand or ten thousand or a million accounts, it works. But it doesn't help against targeted attacks. So if you want to crack a single salted account, it's just as fast as it previously was. So it's good for protecting your whole population, but it's not that good for protecting one or two admin accounts. Then we got Moore's Law. Moore's Law, there are different interpretations. But in this case, I mean that back in the days, the CPU power doubled about every two years because we were cracking on CPUs back in the days. And the algorithms are unchanged. So after two years, twice as much computing power. After four years, four times as much computing power. But the algorithms are unchanged. So the security level of the algorithm goes down over time. So it becomes easier to crack passwords. Again, there was a fix. In this case, key stretching. That's what I call it. And what it basically means is that you run multiple rounds of an algorithm. The most well-known one still used today in some cases is MD5 script. Might sound familiar to at least some of you. Introduced in 1994. And what they do, they just run 1,000 MD5s, which is about 1,000 times slower. Later versions include SHA family of script. So we got still, we see SHA 512 script a lot in operating system security, etc. This worked pretty well for a long time. But again, there were some new issues introduced. Especially the general purpose programming platforms for general processing units for GPUs, video cards. So the video cards were out there already for a long time. But with the introduction of the platforms with CLI code, it was, you were able to unleash the power, the raw computing power of a GPU. That meant back in the days that the speed up was


chunk_text:
  four times as much computing power. But the algorithms are unchanged. So the security level of the algorithm goes down over time. So it becomes easier to crack passwords. Again, there was a fix. In this case, key stretching. That's what I call it. And what it basically means is that you run multiple rounds of an algorithm. The most well-known one still used today in some cases is MD5 script. Might sound familiar to at least some of you. Introduced in 1994. And what they do, they just run 1,000 MD5s, which is about 1,000 times slower. Later versions include SHA family of script. So we got still, we see SHA 512 script a lot in operating system security, etc. This worked pretty well for a long time. But again, there were some new issues introduced. Especially the general purpose programming platforms for general processing units for GPUs, video cards. So the video cards were out there already for a long time. But with the introduction of the platforms with CLI code, it was, you were able to unleash the power, the raw computing power of a GPU. That meant back in the days that the speed up was about 50 to 100 times for most algorithms. So that's a lot. Like two orders of magnitude if you're lucky. Then Moore's law, of course, still works and also works for GPUs. So we got the same issue all over again. The algorithms are unchanged. So it becomes easier to crack passwords that are protected with key stretching over time. The new fix, and actually the first really good fix, we saw that in 1999 with introduction of bcrypt. And unlike the previous algorithms that are general purpose hashing algorithms, bcrypt is the first well-known password hashing algorithm. So it's designed with a couple of design criteria. One is to be configurable. So you can configure the number of iterations. And another very important one is that there are some hardware limitations built in. So it's slow on specific hardware. Back in the days, 25 years ago, GPUs were not a thing for password cracking. But even today in 2024, bcrypt is a really crappy algorithm to crack on your GPU because it takes more memory than is available, fast memory on your GPU. So with a very good design, it's still GPU resistant 25 years later. So this was


chunk_text:
  about 50 to 100 times for most algorithms. So that's a lot. Like two orders of magnitude if you're lucky. Then Moore's law, of course, still works and also works for GPUs. So we got the same issue all over again. The algorithms are unchanged. So it becomes easier to crack passwords that are protected with key stretching over time. The new fix, and actually the first really good fix, we saw that in 1999 with introduction of bcrypt. And unlike the previous algorithms that are general purpose hashing algorithms, bcrypt is the first well-known password hashing algorithm. So it's designed with a couple of design criteria. One is to be configurable. So you can configure the number of iterations. And another very important one is that there are some hardware limitations built in. So it's slow on specific hardware. Back in the days, 25 years ago, GPUs were not a thing for password cracking. But even today in 2024, bcrypt is a really crappy algorithm to crack on your GPU because it takes more memory than is available, fast memory on your GPU. So with a very good design, it's still GPU resistant 25 years later. So this was a very, very big step back in the days. And bcrypt, obviously, if you're a developer, is still a well-known algorithm and is used a lot. Again, history repeated itself. New technology was used for password cracking, in this case FPGAs. So there's a limitation for cracking, for example, bcrypt, but you can overcome the limitation to make a hardware device that's specifically tailored for cracking, in this case, bcrypt hashes. Support is pretty new. It was introduced in John the Ripper in 2019 for a very specific type of FPGA and for a limited number of algorithms. When it was introduced, the speed-up was again 50 to 100 times based on the hardware you were using. So that's, again, a lot. Furthermore, Moore's Law is still working for this technology and for earlier technology, and especially for GPUs, this is pretty interesting because cracking speed for GPUs is now low, for example, for bcrypt, or relatively low, but we're at the point that the tech will overcome the hardware limitations. So somewhere in the near future, probably the hardware limitations will not work for the latest generation of GPUs, and you can crack at full speed. The latest fix,


chunk_text:
  a very, very big step back in the days. And bcrypt, obviously, if you're a developer, is still a well-known algorithm and is used a lot. Again, history repeated itself. New technology was used for password cracking, in this case FPGAs. So there's a limitation for cracking, for example, bcrypt, but you can overcome the limitation to make a hardware device that's specifically tailored for cracking, in this case, bcrypt hashes. Support is pretty new. It was introduced in John the Ripper in 2019 for a very specific type of FPGA and for a limited number of algorithms. When it was introduced, the speed-up was again 50 to 100 times based on the hardware you were using. So that's, again, a lot. Furthermore, Moore's Law is still working for this technology and for earlier technology, and especially for GPUs, this is pretty interesting because cracking speed for GPUs is now low, for example, for bcrypt, or relatively low, but we're at the point that the tech will overcome the hardware limitations. So somewhere in the near future, probably the hardware limitations will not work for the latest generation of GPUs, and you can crack at full speed. The latest fix, current fix, is the introduction of fully configurable algorithms, sCrypt and Argon2. If you're a developer, that's one of the two you want to use. And besides the configurable number of iterations, there's also configurable resource usage. So let's say a new tech platform pops up tomorrow with specific characteristics, you change the parameters of your sCrypt or Argon2 algorithm to be just a little bit more, drastically slowing down the cracking power of the new platform. So this is good, and this will probably be there for the next couple of decades at least. Besides that, you need effective and efficient cracking. Password cracking is an exponential problem, and to give you an idea about what it looks like, if you want to brute force 10 million hashes, and you've got basic hashes like MD5 or SHA-1, SHA-2, you can crack about brute force 8 to 9 length. So that's pretty good. Probably you will find most or at least a lot of passwords. If it's salted, so still old stuff, but salted for 10 million hashes, you can only crack brute force length of 5, which is not a lot. So 1, 2, 3, 4, 5, 6, if


chunk_text:
  current fix, is the introduction of fully configurable algorithms, sCrypt and Argon2. If you're a developer, that's one of the two you want to use. And besides the configurable number of iterations, there's also configurable resource usage. So let's say a new tech platform pops up tomorrow with specific characteristics, you change the parameters of your sCrypt or Argon2 algorithm to be just a little bit more, drastically slowing down the cracking power of the new platform. So this is good, and this will probably be there for the next couple of decades at least. Besides that, you need effective and efficient cracking. Password cracking is an exponential problem, and to give you an idea about what it looks like, if you want to brute force 10 million hashes, and you've got basic hashes like MD5 or SHA-1, SHA-2, you can crack about brute force 8 to 9 length. So that's pretty good. Probably you will find most or at least a lot of passwords. If it's salted, so still old stuff, but salted for 10 million hashes, you can only crack brute force length of 5, which is not a lot. So 1, 2, 3, 4, 5, 6, if you try to crack it brute force, you won't find it. If it's keystretch as well, you can only brute force 3 to 4, and if it's a good algorithm, so bcrypt or later, then it's 1 to 2. So that means that's about 100 to 10,000 passwords. So if you want to try the password top 100, it will take you months or years. So it's pretty good protection if you want to attack the herd. You might think, well, I buy some extra computing power and I'm good. Well, that's not the way it works for an exponential problem. If you want n extra length, you need 100 times to the power of n more computing power. So if you want to increase this number from 5 to 6, you need 100 times more computers. If you want to increase it from 5 to 7, you need 10,000 more computers. So that's not working. It's not a practical solution. So you need to think different, and thinking different means, in our case, using tricks, using the best platforms for a very specific job, and perform data analysis to increase your cracking efficiency over time. This is an example


chunk_text:
  you try to crack it brute force, you won't find it. If it's keystretch as well, you can only brute force 3 to 4, and if it's a good algorithm, so bcrypt or later, then it's 1 to 2. So that means that's about 100 to 10,000 passwords. So if you want to try the password top 100, it will take you months or years. So it's pretty good protection if you want to attack the herd. You might think, well, I buy some extra computing power and I'm good. Well, that's not the way it works for an exponential problem. If you want n extra length, you need 100 times to the power of n more computing power. So if you want to increase this number from 5 to 6, you need 100 times more computers. If you want to increase it from 5 to 7, you need 10,000 more computers. So that's not working. It's not a practical solution. So you need to think different, and thinking different means, in our case, using tricks, using the best platforms for a very specific job, and perform data analysis to increase your cracking efficiency over time. This is an example of a trick. This is a trick on the data from Ashley Madison. It's a US-based dating company that was breached a number of years ago, and they upgraded from MD5 to bcrypt with the work factor 12. So that's from extremely fast for an attacker to extremely slow. What you can do, if you're lucky, the application code is also leaked. That was the case here. And the database still uses some algorithms based on MD5, so for upgrade parts and also for reset procedures. If you lost your password, that was not a bcrypt password, or at least not the token that was used, but it was still MD5-based. So what you can do is you crack the legacy fast hashes first with 8-character brute force, for example, and you use the results as a dictionary to crack the slow results, and that's, of course, a dictionary with a very high hit rate, a hit rate near 100%. What's the result? Great success. The attack speeds up with about 100 billion times. So that's testing 100 passwords in months. That was the case. If you didn't use this trick, if you're using this trick, you can try to


chunk_text:
  of a trick. This is a trick on the data from Ashley Madison. It's a US-based dating company that was breached a number of years ago, and they upgraded from MD5 to bcrypt with the work factor 12. So that's from extremely fast for an attacker to extremely slow. What you can do, if you're lucky, the application code is also leaked. That was the case here. And the database still uses some algorithms based on MD5, so for upgrade parts and also for reset procedures. If you lost your password, that was not a bcrypt password, or at least not the token that was used, but it was still MD5-based. So what you can do is you crack the legacy fast hashes first with 8-character brute force, for example, and you use the results as a dictionary to crack the slow results, and that's, of course, a dictionary with a very high hit rate, a hit rate near 100%. What's the result? Great success. The attack speeds up with about 100 billion times. So that's testing 100 passwords in months. That was the case. If you didn't use this trick, if you're using this trick, you can try to brute force passwords up to 8 or 9 characters. So buying hardware doesn't work. Using smart tricks does. Then you should choose the best cracking platform. Typically, that's a GPU, but if you've got a huge dataset or if you're using very complex rules, GPUs haven't got enough memory, so the attacks cannot run in a single run. It takes way more time, and in that case, it might be beneficial to use a CPU. If this is the case, you can simply calculate this in your spreadsheet program, but if this is the case, use high core count CPUs with AVX-12, extend the factor extensions, and you're pretty good. If you've got hashes with hardware-specific slowdowns, use FPGAs, but there's an issue. You cannot buy them. There are no off-the-shelf FPGAs crackers, and the amount of hardware models that support it is very limited, but it is the fastest way. You get more speed for less power, which is good. So that's what we did. This is one example. This is the actual advertisement picture from eBay from the first batch we bought. So I don't know why he's got a hat and sunglasses, but that's the way the


chunk_text:
  brute force passwords up to 8 or 9 characters. So buying hardware doesn't work. Using smart tricks does. Then you should choose the best cracking platform. Typically, that's a GPU, but if you've got a huge dataset or if you're using very complex rules, GPUs haven't got enough memory, so the attacks cannot run in a single run. It takes way more time, and in that case, it might be beneficial to use a CPU. If this is the case, you can simply calculate this in your spreadsheet program, but if this is the case, use high core count CPUs with AVX-12, extend the factor extensions, and you're pretty good. If you've got hashes with hardware-specific slowdowns, use FPGAs, but there's an issue. You cannot buy them. There are no off-the-shelf FPGAs crackers, and the amount of hardware models that support it is very limited, but it is the fastest way. You get more speed for less power, which is good. So that's what we did. This is one example. This is the actual advertisement picture from eBay from the first batch we bought. So I don't know why he's got a hat and sunglasses, but that's the way the guy used the devices for Bitcoin mining, because they were very popular for Bitcoin mining at the time, and what we do, we build them in rack mount services with proper cooling, proper power supplies, et cetera, so they can run 24-7 at the max speed to crack a lot of unique content. So what are the results and observations? We recovered over 7 billion email and password pairs, so in my opinion, that's quite a lot. Typically, we see one or two million new passwords a day come in. There's a lot of fake password news out there and disinformation. I've got some examples later. And within a typical organization, if you've got more than, let's say, 1,000 accounts, you're in, and so if you're the attacker, I'm a pentester, professional pentester as well, if you've got more than 1,000 accounts, there's always at least one person that's reusing passwords and allows you to get in and get some foothold and getting the party started. At scale, so at scale, I mean millions of accounts, for example, clients with millions of customer accounts, we can typically breach about in between 5% and 15% of the accounts that are enrolled.


chunk_text:
  guy used the devices for Bitcoin mining, because they were very popular for Bitcoin mining at the time, and what we do, we build them in rack mount services with proper cooling, proper power supplies, et cetera, so they can run 24-7 at the max speed to crack a lot of unique content. So what are the results and observations? We recovered over 7 billion email and password pairs, so in my opinion, that's quite a lot. Typically, we see one or two million new passwords a day come in. There's a lot of fake password news out there and disinformation. I've got some examples later. And within a typical organization, if you've got more than, let's say, 1,000 accounts, you're in, and so if you're the attacker, I'm a pentester, professional pentester as well, if you've got more than 1,000 accounts, there's always at least one person that's reusing passwords and allows you to get in and get some foothold and getting the party started. At scale, so at scale, I mean millions of accounts, for example, clients with millions of customer accounts, we can typically breach about in between 5% and 15% of the accounts that are enrolled. So again, that's up to you, but in my opinion, that's quite a lot. Very interesting, typically, a typical company doesn't have effective protection against credential stuffing at all. Then, all passwords together. So I guess most people are familiar with the LinkedIn breach from 2012. Typically, people say, ah, but it's very old. It's not relevant anymore, and old data, old passwords. True, but we still recover new passwords in 2024 from the LinkedIn breach. Not a lot, but still tens of thousands each month. And typically, those are not the password 01s and the welcome 01s, but those are the really complex passwords. And the complex of the database, the higher the chance that it's still used by people nowadays still in 2024. More examples included later as well. Then the grass is greener on the other side, so if you got even basic measures in place, that will help. So cyber criminals, they typically perform some test runs, and if it doesn't look that good, if they cannot breach any code, well, go to your competitor and try to credential stuff your competitor. Well, that's good. Mission accomplished. So even basic measures will help you. An interesting


chunk_text:
  So again, that's up to you, but in my opinion, that's quite a lot. Very interesting, typically, a typical company doesn't have effective protection against credential stuffing at all. Then, all passwords together. So I guess most people are familiar with the LinkedIn breach from 2012. Typically, people say, ah, but it's very old. It's not relevant anymore, and old data, old passwords. True, but we still recover new passwords in 2024 from the LinkedIn breach. Not a lot, but still tens of thousands each month. And typically, those are not the password 01s and the welcome 01s, but those are the really complex passwords. And the complex of the database, the higher the chance that it's still used by people nowadays still in 2024. More examples included later as well. Then the grass is greener on the other side, so if you got even basic measures in place, that will help. So cyber criminals, they typically perform some test runs, and if it doesn't look that good, if they cannot breach any code, well, go to your competitor and try to credential stuff your competitor. Well, that's good. Mission accomplished. So even basic measures will help you. An interesting one we saw at the pen test of the client, a good password policy can actually work counterproductive. So this client used 12-character, 12-length password policy with all kinds of fancy things you don't see at most organizations. Credential stuffing was successful, and the client was surprised. So they asked us, hey, can you show what data did you use, and why did it work? Because we got a really good password policy. Well, in the end, it's very simple. The number of candidates that were leaked goes down, because you already know for sure that 1, 2, 3, 4, 5, 6 will not work. It's not long enough. So the number goes down, but the quality of the candidates that are remaining goes drastically up. So you end up with a more efficient and more stealthier attack. So this is a bit counterintuitive, but this is the way it seems to work in real life. Now, some fake news. Who's a hookers.nl user? We see one. If you're not familiar, for our other folks that are not familiar with hookers.nl, it's a website where you can review sex workers. So if you're a regular customer, then you can say,


chunk_text:
  one we saw at the pen test of the client, a good password policy can actually work counterproductive. So this client used 12-character, 12-length password policy with all kinds of fancy things you don't see at most organizations. Credential stuffing was successful, and the client was surprised. So they asked us, hey, can you show what data did you use, and why did it work? Because we got a really good password policy. Well, in the end, it's very simple. The number of candidates that were leaked goes down, because you already know for sure that 1, 2, 3, 4, 5, 6 will not work. It's not long enough. So the number goes down, but the quality of the candidates that are remaining goes drastically up. So you end up with a more efficient and more stealthier attack. So this is a bit counterintuitive, but this is the way it seems to work in real life. Now, some fake news. Who's a hookers.nl user? We see one. If you're not familiar, for our other folks that are not familiar with hookers.nl, it's a website where you can review sex workers. So if you're a regular customer, then you can say, hey, well, this lady or this guy is really good, or don't go there. They got hacked, and a lot of people were worried. So there was a statement from the operators from the website, and they said, well, no passwords stolen. There were no passwords stolen, so nothing's going on. We already had the data for quite some time, so we saw that it was not true, and we were actually doing some burning tests for some new hardware, and in a long weekend, we cracked 57% of the passwords. So I think it's good not to trust a victim. Typically, victims, they might have a different agenda, which is the case in many documented cases we've seen before. Then, typically, there's no basic protection. So if you're hacked because of missing basic cyber hygiene, you can just say, well, it was a very sophisticated attack. We've never seen this before, or it was a state-sponsored threat actor. Typically, no critical questions are asked, and not even by the media, and typically, it's not true. So in reality, if you've got a school kid with five euros or five dollars, they can buy billions of accounts on the forum,


chunk_text:
  hey, well, this lady or this guy is really good, or don't go there. They got hacked, and a lot of people were worried. So there was a statement from the operators from the website, and they said, well, no passwords stolen. There were no passwords stolen, so nothing's going on. We already had the data for quite some time, so we saw that it was not true, and we were actually doing some burning tests for some new hardware, and in a long weekend, we cracked 57% of the passwords. So I think it's good not to trust a victim. Typically, victims, they might have a different agenda, which is the case in many documented cases we've seen before. Then, typically, there's no basic protection. So if you're hacked because of missing basic cyber hygiene, you can just say, well, it was a very sophisticated attack. We've never seen this before, or it was a state-sponsored threat actor. Typically, no critical questions are asked, and not even by the media, and typically, it's not true. So in reality, if you've got a school kid with five euros or five dollars, they can buy billions of accounts on the forum, or they can get it for free on the Telegram channels, and if they've got a gaming PC, they can launch a successful attack. So if you're familiar with PC gaming, and you know Call of Duty, for example, and the rule of thumb is if you can run Call of Duty, you can brute force an eight-position hash within a day. So that's the amount of computing power that's available to school kids nowadays. Then, what we see, that incident response sometimes at clients, is that sometimes the guys get caught, and they are actually teenagers doing this. After the Snowflake breach, and the fact that the Taylor Swift tickets, we looked up our data on Snowflake, this is part of it, and you need to trust me if I tell you that redacting the data took more time than looking the data up. It's a lot. So this is the dashboard for Snowflake users. So these are actual enrolled production environments that are used by Snowflake users within the second column, the username, and the third column, their passwords. I'm always surprised, so I got a little quiz question. I'm always surprised by the eagle eye view of


chunk_text:
  or they can get it for free on the Telegram channels, and if they've got a gaming PC, they can launch a successful attack. So if you're familiar with PC gaming, and you know Call of Duty, for example, and the rule of thumb is if you can run Call of Duty, you can brute force an eight-position hash within a day. So that's the amount of computing power that's available to school kids nowadays. Then, what we see, that incident response sometimes at clients, is that sometimes the guys get caught, and they are actually teenagers doing this. After the Snowflake breach, and the fact that the Taylor Swift tickets, we looked up our data on Snowflake, this is part of it, and you need to trust me if I tell you that redacting the data took more time than looking the data up. It's a lot. So this is the dashboard for Snowflake users. So these are actual enrolled production environments that are used by Snowflake users within the second column, the username, and the third column, their passwords. I'm always surprised, so I got a little quiz question. I'm always surprised by the eagle eye view of some of the attendees. What do you see? Do you see something out of the ordinary in the password column? Sorry? Yeah? But that's not what I'm looking for? Very good. So one of the passwords, and this was actually internal Snowflake accounts, so this was operated by Snowflake themselves, Snowflake 23, explanation mark. So not even the basic password controls were in place at the time of that company. And this was data we had six or nine months before the actual breach. Then the last item, all passwords matter. We were contacted by a tech journalist who wrote a book about passwords, and he was promoting his book at Ava Yenix talk show, and he contacted us. He said, hey, can you crack some passwords from Ava? That would be fun. Sure, so that's what we did. So we cracked 2012 Dropbox password, so again, very old password, same age as the LinkedIn breach. And it was still used by Ava for all her passwords for all important services. So it was a live show, and actually the journalist had to help her change all her passwords before going live, because they were afraid that otherwise things went


chunk_text:
  some of the attendees. What do you see? Do you see something out of the ordinary in the password column? Sorry? Yeah? But that's not what I'm looking for? Very good. So one of the passwords, and this was actually internal Snowflake accounts, so this was operated by Snowflake themselves, Snowflake 23, explanation mark. So not even the basic password controls were in place at the time of that company. And this was data we had six or nine months before the actual breach. Then the last item, all passwords matter. We were contacted by a tech journalist who wrote a book about passwords, and he was promoting his book at Ava Yenix talk show, and he contacted us. He said, hey, can you crack some passwords from Ava? That would be fun. Sure, so that's what we did. So we cracked 2012 Dropbox password, so again, very old password, same age as the LinkedIn breach. And it was still used by Ava for all her passwords for all important services. So it was a live show, and actually the journalist had to help her change all her passwords before going live, because they were afraid that otherwise things went wrong. So probably you guys, you changed your passwords, you got quality passwords, et cetera, but most normal people, they reuse passwords. That's the way it works in real life. All passwords can also help you to follow a person over time, because historical passwords can help you to unravel the mechanism that a lot of people are using for creating passwords. So if you got 10, and they use some kind of mechanism that you can understand, then probably you can guess number 11 and number 12. So that's a very useful resource for guessing future passwords. The other thing is that you can find personal accounts based on professional accounts and the other way around. So if the password is quite unique, that allows you to search on a password and find other email addresses. Well, it's always fun to do that with politicians, at least that's what I think. Most politicians use really crappy passwords, like most other people, but some of them use high quality passwords, that high quality, that they're unique worldwide. So we only see that password 10 times in 7 billion passwords that we've breached. Using this, you can identify their personal accounts.


chunk_text:
  wrong. So probably you guys, you changed your passwords, you got quality passwords, et cetera, but most normal people, they reuse passwords. That's the way it works in real life. All passwords can also help you to follow a person over time, because historical passwords can help you to unravel the mechanism that a lot of people are using for creating passwords. So if you got 10, and they use some kind of mechanism that you can understand, then probably you can guess number 11 and number 12. So that's a very useful resource for guessing future passwords. The other thing is that you can find personal accounts based on professional accounts and the other way around. So if the password is quite unique, that allows you to search on a password and find other email addresses. Well, it's always fun to do that with politicians, at least that's what I think. Most politicians use really crappy passwords, like most other people, but some of them use high quality passwords, that high quality, that they're unique worldwide. So we only see that password 10 times in 7 billion passwords that we've breached. Using this, you can identify their personal accounts. So let's say this is John Doe at this official government agency. You also find John Dee at Gmail, Jay Doe at Hotmail, and some burner addresses. The interesting thing is you can use those addresses again to look if they were used in other data breaches and see what the folks were doing with their personal accounts. Well, one of them was a member of the House of Parliament, or an ex-member of the House of Parliament. And another quiz question, what did we find? Really close, really close. Which obviously makes it possible for damaging the reputation, that's one, but might also be used, for example, for blackmailing. So it's very easy to make a mistake. The internet will never forget. And if you get more and more data, and more and more computing power, it's really difficult to stay on the radar. That's it for now. I don't know if we've got time for questions, but for sure you can check if your accounts were breached at our website. We've got free community accounts. And if you've got questions later on, please find me somewhere at orange.com. Or have we got time for one question? I don't


chunk_text:
  So let's say this is John Doe at this official government agency. You also find John Dee at Gmail, Jay Doe at Hotmail, and some burner addresses. The interesting thing is you can use those addresses again to look if they were used in other data breaches and see what the folks were doing with their personal accounts. Well, one of them was a member of the House of Parliament, or an ex-member of the House of Parliament. And another quiz question, what did we find? Really close, really close. Which obviously makes it possible for damaging the reputation, that's one, but might also be used, for example, for blackmailing. So it's very easy to make a mistake. The internet will never forget. And if you get more and more data, and more and more computing power, it's really difficult to stay on the radar. That's it for now. I don't know if we've got time for questions, but for sure you can check if your accounts were breached at our website. We've got free community accounts. And if you've got questions later on, please find me somewhere at orange.com. Or have we got time for one question? I don't know. Yeah, I think we have time for one question. What are the legal implications of keeping the found password data online, and doing the hashing stuff? Well, I'm not a legal guy, but obviously we consulted some pro-legal guys, and they tell us that it shouldn't be a problem, because there are some exceptions in GDPR. And one of the exceptions is protecting your, I don't know what the exact words are, but protecting computer networks and automated systems. That's one. The other thing, if you look at the cyber crime laws, they say it's an issue if you publish non-public data. But this is debatable. But this is my statement. This is public data. Well, and with that, I would like to ask you all for a huge round of applause for Jeroen van Beek.


chunk_text:
  know. Yeah, I think we have time for one question. What are the legal implications of keeping the found password data online, and doing the hashing stuff? Well, I'm not a legal guy, but obviously we consulted some pro-legal guys, and they tell us that it shouldn't be a problem, because there are some exceptions in GDPR. And one of the exceptions is protecting your, I don't know what the exact words are, but protecting computer networks and automated systems. That's one. The other thing, if you look at the cyber crime laws, they say it's an issue if you publish non-public data. But this is debatable. But this is my statement. This is public data. Well, and with that, I would like to ask you all for a huge round of applause for Jeroen van Beek.


chunk_text:
  { "webpage_url": "https://www.youtube.com/watch?v=Ev3GeR-oRNk", "title": "Graph API Mastery: Logs To Real World Attacks - Shiva P & Parthiban R", "description": "", "channel_url": "https://www.youtube.com/channel/UCgJoGNj3B2Mt2F3TUblcoOA", "duration": 1796, "channel": "OrangeCon", "uploader": "OrangeCon", "upload_date": "20240909" } This text was transcribed using whisper model: large-v2 Thanks everyone for willing to spend the next 30 minutes with us on our talk. Today we wanted to talk to you about a newly introduced log source from Microsoft, which is Microsoft Graph API Activity Logs. The goal of the presentation is to make sure that you get familiarized with the log source and then so that you can go on and perform threat hunting using the Graph API Activity Logs. So there's a little bit about us. So I'm Parthivan. I've been in the security for over 10 years now, currently working as a Senior Intelligence Analyst at Atlassian. I previously worked with my friend Shiva for the past at Symantec and Microsoft as an anomaly. And outside of work, food is something that I really enjoy and my friends. And yeah, hi, I'm Shiva. So I also have like around 10 years of experience in security. Right now I work for Dart and I'm a Senior Security Researcher, previously worked in Symantec and Anaplan as well. And I've been around multiple different security roles. And yeah, outside of cybersecurity, it's more of gaming. Cool. So in today's agenda, I wanted to go through a little bit of fundamentals about what is a Microsoft Graph API and how to get these logs as it's brand new. And I wanted to go over through a couple of key fields which you can make use of during your threat hunting or investigation and also how you can effectively utilize Graph API logs with other data sources. And then we'll also go through a couple of KQL functions that you might be, you might find it useful during investigation. And then my friend Shiva will walk you through a couple of real world attack scenarios that he worked during his time at Dart. And then he'll finish off the session with how to perform baselining and auditing in Graph API logs. So what is a Graph API? Graph API is not something brand new. It's been around for a while now, but the logging is something that was enabled recently. So using Graph API, one can interact with a wide variety of


chunk_text:
  previously worked in Symantec and Anaplan as well. And I've been around multiple different security roles. And yeah, outside of cybersecurity, it's more of gaming. Cool. So in today's agenda, I wanted to go through a little bit of fundamentals about what is a Microsoft Graph API and how to get these logs as it's brand new. And I wanted to go over through a couple of key fields which you can make use of during your threat hunting or investigation and also how you can effectively utilize Graph API logs with other data sources. And then we'll also go through a couple of KQL functions that you might be, you might find it useful during investigation. And then my friend Shiva will walk you through a couple of real world attack scenarios that he worked during his time at Dart. And then he'll finish off the session with how to perform baselining and auditing in Graph API logs. So what is a Graph API? Graph API is not something brand new. It's been around for a while now, but the logging is something that was enabled recently. So using Graph API, one can interact with a wide variety of M365 services, which means you can use a single endpoint to access applications such as OneNote, OneDrive, Calendar. There are so many endless applications that you can do. A simple example would be you can use the API to get the latest emails from your Outlook or you can also use it to list all your files in OneDrive. So due to the extensive features and ease of use, in the wrong hands, it can be abused by threat actors to perform malicious activities. So as a defender, we should be in a position to detect and respond to threats that are happening using Graph API. So this screenshot is just a simple example of what you can do with the Graph API. This is a web application from Microsoft. You can just go to akms.ge that will take you to this page. You can try to explore what different endpoints that you can query and what type of data that it will get back to you. So if you look on the left side, you can see you can get the user profile. Similarly you can also get, if you have enough permissions, you can also get information about


chunk_text:
  M365 services, which means you can use a single endpoint to access applications such as OneNote, OneDrive, Calendar. There are so many endless applications that you can do. A simple example would be you can use the API to get the latest emails from your Outlook or you can also use it to list all your files in OneDrive. So due to the extensive features and ease of use, in the wrong hands, it can be abused by threat actors to perform malicious activities. So as a defender, we should be in a position to detect and respond to threats that are happening using Graph API. So this screenshot is just a simple example of what you can do with the Graph API. This is a web application from Microsoft. You can just go to akms.ge that will take you to this page. You can try to explore what different endpoints that you can query and what type of data that it will get back to you. So if you look on the left side, you can see you can get the user profile. Similarly you can also get, if you have enough permissions, you can also get information about other users that can be used for enumeration. Similarly you can also get your emails, like I mentioned, your items from your OneDrive. You can schedule calendar appointments, et cetera, lots of applications. Before we move on to how to get these logs, I want to give you another couple of basics on how authentication and permission schemes work in Graph API. In order to access the Graph API, one needs to have an application that needs to be registered in your Azure portal, and that's one of the fundamentals. And then the authentication is done using OAuth protocol, which means you need to be having a client ID and access token to access the API. So once you have these two done, the next step is to assign permissions to your applications. So in the Graph API world, there are two permission schemes. The first one is delegated access. In this delegated access mode, the user needs to be present and signed in. So let's say if the user is trying to access the latest emails from there using an application, there will be a sign-in pop-up that will ask them to sign in to access the resource. And


chunk_text:
  other users that can be used for enumeration. Similarly you can also get your emails, like I mentioned, your items from your OneDrive. You can schedule calendar appointments, et cetera, lots of applications. Before we move on to how to get these logs, I want to give you another couple of basics on how authentication and permission schemes work in Graph API. In order to access the Graph API, one needs to have an application that needs to be registered in your Azure portal, and that's one of the fundamentals. And then the authentication is done using OAuth protocol, which means you need to be having a client ID and access token to access the API. So once you have these two done, the next step is to assign permissions to your applications. So in the Graph API world, there are two permission schemes. The first one is delegated access. In this delegated access mode, the user needs to be present and signed in. So let's say if the user is trying to access the latest emails from there using an application, there will be a sign-in pop-up that will ask them to sign in to access the resource. And in terms of app-only access, it will basically act as its own. So you can consider it as a bot that performs activity in the backend using the permissions that are assigned at the time of creation. So this diagram will summarize how the two different permission schemes work. So in terms of delegated access, when someone is trying to access the resource, the resource, let's say in this case, similarly, we can have OneDrive. So the OneDrive resource will ask a couple of questions to the application. The first one is, is the user authorized to perform this activity? If it's yes, then the next question would be, is the app authorized to perform this activity on behalf of the user? So if both conditions met, then the application will be able to access the contents from the OneDrive. In terms of app-only access, the only question that will be asked by the resource is, is the app authorized to perform this activity? Then if the admin has given consent to this application, then the app can access the resource. Now we'll move on to how to get these logs. Firstly, the Graph API activity logs are not collected


chunk_text:
  in terms of app-only access, it will basically act as its own. So you can consider it as a bot that performs activity in the backend using the permissions that are assigned at the time of creation. So this diagram will summarize how the two different permission schemes work. So in terms of delegated access, when someone is trying to access the resource, the resource, let's say in this case, similarly, we can have OneDrive. So the OneDrive resource will ask a couple of questions to the application. The first one is, is the user authorized to perform this activity? If it's yes, then the next question would be, is the app authorized to perform this activity on behalf of the user? So if both conditions met, then the application will be able to access the contents from the OneDrive. In terms of app-only access, the only question that will be asked by the resource is, is the app authorized to perform this activity? Then if the admin has given consent to this application, then the app can access the resource. Now we'll move on to how to get these logs. Firstly, the Graph API activity logs are not collected by default. So you need to enable them, so your administrator need to enable them to collect these logs. The second point here is, the logs are only available to EntroID P1 or P2 license holders. Any other license, you wouldn't be able to get access to these logs. And another important thing here is, before you start collecting these logs, you need to perform a cost estimation of how much this is going to cost you, because if you have a larger cloud presence and having lots and lots of assets, then you'd be racking up so much bill. So please be mindful about before enabling these logs. And to collect these logs, you can simply go to the Azure portal and then go to EntroID. And under the EntroID, you will have the diagnostic settings, and like the one in the screenshot right side, you can see under the diagnostic settings, there will be a number of different log sources. Under that, you can enable the Graph API activity logs. And then once you enable it, you can either send it, you can send it to your preferred data lake, or you can send it to, in this


chunk_text:
  by default. So you need to enable them, so your administrator need to enable them to collect these logs. The second point here is, the logs are only available to EntroID P1 or P2 license holders. Any other license, you wouldn't be able to get access to these logs. And another important thing here is, before you start collecting these logs, you need to perform a cost estimation of how much this is going to cost you, because if you have a larger cloud presence and having lots and lots of assets, then you'd be racking up so much bill. So please be mindful about before enabling these logs. And to collect these logs, you can simply go to the Azure portal and then go to EntroID. And under the EntroID, you will have the diagnostic settings, and like the one in the screenshot right side, you can see under the diagnostic settings, there will be a number of different log sources. Under that, you can enable the Graph API activity logs. And then once you enable it, you can either send it, you can send it to your preferred data lake, or you can send it to, in this case, we are sending the logs to Log Analytics in Azure. And now, once you have these logs, the next step is to understand what fields are present in the logs. Here are some of the key fields that are present in the API logs. The first important field is the application ID. So this is a unique identifier used for applications, so this is going to act as a central pivot point during your investigation. And then the IP address is your source IP address. And then the request method is another important field here, because that's how you will know what type of activity is happening. So for example, if it's a GET request, the app is trying to get data from there, so you can assume it as in some sort of enumeration activity in terms of malicious stuff. So let's say the app can list all the items from your OneDrive, so that's an enumeration. So in terms of post activity, that's basically your app is trying to interact with the resource, so you can consider it as something like an app is trying to send an email. So in this case, your data will be


chunk_text:
  case, we are sending the logs to Log Analytics in Azure. And now, once you have these logs, the next step is to understand what fields are present in the logs. Here are some of the key fields that are present in the API logs. The first important field is the application ID. So this is a unique identifier used for applications, so this is going to act as a central pivot point during your investigation. And then the IP address is your source IP address. And then the request method is another important field here, because that's how you will know what type of activity is happening. So for example, if it's a GET request, the app is trying to get data from there, so you can assume it as in some sort of enumeration activity in terms of malicious stuff. So let's say the app can list all the items from your OneDrive, so that's an enumeration. So in terms of post activity, that's basically your app is trying to interact with the resource, so you can consider it as something like an app is trying to send an email. So in this case, your data will be sent to the resource, so it's a post request. And similarly, the response status code is important, so that's how you will be able to differentiate whether this particular activity was successful or not. If it's 200 series, then it's a successful activity, and if it's 400, then there's some sort of error happening. And finally, the request URI, another important field, because that's how you will be able to know which endpoint the app is trying to hit. So let's say if the endpoint is trying to query items from OneDrive, then you would be able to see the corresponding endpoint in the request URI. So that's how you'll be able to perform investigation on what activities are happening. And we'll also walk you through a couple of useful tips. So on its own, Graph API logs, it's useful, but when you combine it with other data sources, you will have more context, so that will be super helpful during investigation. So the first one is, there is a user ID field in Graph API, but that's just a unique identifier that won't give you any extra context. So if you join it with the identity info table, you


chunk_text:
  sent to the resource, so it's a post request. And similarly, the response status code is important, so that's how you will be able to differentiate whether this particular activity was successful or not. If it's 200 series, then it's a successful activity, and if it's 400, then there's some sort of error happening. And finally, the request URI, another important field, because that's how you will be able to know which endpoint the app is trying to hit. So let's say if the endpoint is trying to query items from OneDrive, then you would be able to see the corresponding endpoint in the request URI. So that's how you'll be able to perform investigation on what activities are happening. And we'll also walk you through a couple of useful tips. So on its own, Graph API logs, it's useful, but when you combine it with other data sources, you will have more context, so that will be super helpful during investigation. So the first one is, there is a user ID field in Graph API, but that's just a unique identifier that won't give you any extra context. So if you join it with the identity info table, you can get other context around email address, names, SPNs, etc. So those will be super helpful. And similarly, you can also combine it with the Active Directory RISC user, that will provide further context around what type of RISC is associated with the user ID. That might help you to type of RISC, like logging in from a different location, suddenly those sort of extra information you'll be able to find. There's another table called Cloud App Event Logs that can be used when you're, let's say, you're trying to investigate an activity where the user is trying to download a list of files, but in the Graph API logs, you'll only see the file identifier, but you won't be able to see what's the file name. So if you combine it with the Cloud App Event Logs, you'll be able to get that as well. So that's super helpful. And a couple of useful KQL functions. The first one is parse URL. So if you parse the request URI, the field that we just talked about, the function will parse the URL into multiple fields. So it will split the URL into domain, subdomain, ports, and it also has


chunk_text:
  can get other context around email address, names, SPNs, etc. So those will be super helpful. And similarly, you can also combine it with the Active Directory RISC user, that will provide further context around what type of RISC is associated with the user ID. That might help you to type of RISC, like logging in from a different location, suddenly those sort of extra information you'll be able to find. There's another table called Cloud App Event Logs that can be used when you're, let's say, you're trying to investigate an activity where the user is trying to download a list of files, but in the Graph API logs, you'll only see the file identifier, but you won't be able to see what's the file name. So if you combine it with the Cloud App Event Logs, you'll be able to get that as well. So that's super helpful. And a couple of useful KQL functions. The first one is parse URL. So if you parse the request URI, the field that we just talked about, the function will parse the URL into multiple fields. So it will split the URL into domain, subdomain, ports, and it also has the ability to extract the parameters that you're passing in the API. So that will be super helpful to differentiate what type of activities are happening. And finally, the Geo IP information. So it's self-explanatory. If you parse the source IP address information in there, that will help you to get context around geolocation information, such as city, country information, like that. Now I'm going to pass the mic to my friend Shiva, who's going to walk you through real-world attack scenarios. Yep. Thanks, Parthiban. So before we jump into the real-world attack scenario, I just wanted to tell you that all of these scenarios that we're going to see here, it's basically derived from the engagements that we've had in Dart. Dart basically provides instant response as a service to multiple external customers. With that being said, though, all the screenshots that you're going to see here, it's completely a recreated attack from our own test environment, so which means you will not see any sort of customer data anywhere in any of the screenshots. So with the disclaimer out, let's jump in. Basically, we've split the attack scenarios into miter matrix so that it's pretty clear for you


chunk_text:
  the ability to extract the parameters that you're passing in the API. So that will be super helpful to differentiate what type of activities are happening. And finally, the Geo IP information. So it's self-explanatory. If you parse the source IP address information in there, that will help you to get context around geolocation information, such as city, country information, like that. Now I'm going to pass the mic to my friend Shiva, who's going to walk you through real-world attack scenarios. Yep. Thanks, Parthiban. So before we jump into the real-world attack scenario, I just wanted to tell you that all of these scenarios that we're going to see here, it's basically derived from the engagements that we've had in Dart. Dart basically provides instant response as a service to multiple external customers. With that being said, though, all the screenshots that you're going to see here, it's completely a recreated attack from our own test environment, so which means you will not see any sort of customer data anywhere in any of the screenshots. So with the disclaimer out, let's jump in. Basically, we've split the attack scenarios into miter matrix so that it's pretty clear for you to understand what's really happening here. And yeah, also the queries here, it's basically an excerpt from the actual queries. We wanted to sort of save the real estate of these slides. So if you want basically the queries, we've added a QR code at the very end of the slide, so feel free to use that, and you'll be directed to an article which has all the queries that is presented here. Yep, so this is one of the most seen tactics. We've observed it in all of our, of course, all of the IR engagements. So actors love gathering data. They love elevating privileges. They love trying to figure out where they are and where they want to go. So one of the things that we did here is that, to get this query, is basically we took a look into all of the engagements that we've been on. We've isolated all sort of reconnaissance that's happened using Graph API. We pulled all the APIs, which is related to reconnaissance, and along with that, we also took a look into all the different tools that is available in the market. Shout out to Graphrunner and Azure Hound. They


chunk_text:
  to understand what's really happening here. And yeah, also the queries here, it's basically an excerpt from the actual queries. We wanted to sort of save the real estate of these slides. So if you want basically the queries, we've added a QR code at the very end of the slide, so feel free to use that, and you'll be directed to an article which has all the queries that is presented here. Yep, so this is one of the most seen tactics. We've observed it in all of our, of course, all of the IR engagements. So actors love gathering data. They love elevating privileges. They love trying to figure out where they are and where they want to go. So one of the things that we did here is that, to get this query, is basically we took a look into all of the engagements that we've been on. We've isolated all sort of reconnaissance that's happened using Graph API. We pulled all the APIs, which is related to reconnaissance, and along with that, we also took a look into all the different tools that is available in the market. Shout out to Graphrunner and Azure Hound. They use Graph API to do a lot of reconnaissance. So we used all of these to basically come up with a list of, a dynamic list basically, to add all the API endpoints. And basically using that, we've also added a response, I wish I had a laser, but we also added a response status code here, which is equal to 200, which means it's a successful response. And along with that, what we did was we grouped the whole thing into a time bin of two minutes, and we are then looking for sort of a burst activity, which means if there's going to be reconnaissance, it's going to be happening in a short span of time. So it is going to be like 100 seconds for this query alone. And finally, what we did was we are counting three or more unique calls, and total call of around 40 or more, or if the total calls is more than 100, then this is going to create a hit. And again, this is more of completely based on the test scenarios that we did based on our tenants, which means you will have to adjust your filter attributes for


chunk_text:
  use Graph API to do a lot of reconnaissance. So we used all of these to basically come up with a list of, a dynamic list basically, to add all the API endpoints. And basically using that, we've also added a response, I wish I had a laser, but we also added a response status code here, which is equal to 200, which means it's a successful response. And along with that, what we did was we grouped the whole thing into a time bin of two minutes, and we are then looking for sort of a burst activity, which means if there's going to be reconnaissance, it's going to be happening in a short span of time. So it is going to be like 100 seconds for this query alone. And finally, what we did was we are counting three or more unique calls, and total call of around 40 or more, or if the total calls is more than 100, then this is going to create a hit. And again, this is more of completely based on the test scenarios that we did based on our tenants, which means you will have to adjust your filter attributes for your own tenants for your environment. And of course, since Microsoft Graph API is constantly evolving, you might need to add other endpoints or calls that is made to the dynamic list. Oh, there's another scenario where we notice a service principle of an application that was compromised. So basically, this application had this role management or read, write, dot directory role. So what this role basically does is it gives this application an ability to assign any sort of role to any sort of user. That is exactly what it does. And in this scenario, or in the attack that we noticed as part of the Dart engagement, basically, the actor provided multiple different users with multiple different roles to of course, proceed with their action on objectives. And we recreated, of course, this scenario. You will see on the right, there is like multiple screenshots. The first screenshot, you see the account name hacker, they have not assigned any sort of roles there. And post which, what we did was we abused this Graph API application, and we provided global administrator, which is on the second screenshot. And this is basically how the logs would look like. You


chunk_text:
  your own tenants for your environment. And of course, since Microsoft Graph API is constantly evolving, you might need to add other endpoints or calls that is made to the dynamic list. Oh, there's another scenario where we notice a service principle of an application that was compromised. So basically, this application had this role management or read, write, dot directory role. So what this role basically does is it gives this application an ability to assign any sort of role to any sort of user. That is exactly what it does. And in this scenario, or in the attack that we noticed as part of the Dart engagement, basically, the actor provided multiple different users with multiple different roles to of course, proceed with their action on objectives. And we recreated, of course, this scenario. You will see on the right, there is like multiple screenshots. The first screenshot, you see the account name hacker, they have not assigned any sort of roles there. And post which, what we did was we abused this Graph API application, and we provided global administrator, which is on the second screenshot. And this is basically how the logs would look like. You would see IP, I mean, what is important here is the IP address, the request URI, the response status codes, and user agent and app IDs. So using this log and all other logs that we found in the research, we were able to come up with something of a simple query, where you look into the, where it figures this request, the request made is equal to post, and for the response code is 204, you would find a hit. One of the things that you have to note here is that, the role. If you see the role there, it doesn't really scream to an investigator that it is a global administrator role. That is like one of the biggest cons, I would say, of Graph API logs, Graph activity logs. It doesn't give you all the information right to an investigator. And just like how Parthaman illustrated in the previous slides, you'll have to correlate this log with other logs, such as in this case, it's going to be audit logs, to understand what sort of role this is, and what sort of, what the user was basically assigned, what sort of role was assigned to the


chunk_text:
  would see IP, I mean, what is important here is the IP address, the request URI, the response status codes, and user agent and app IDs. So using this log and all other logs that we found in the research, we were able to come up with something of a simple query, where you look into the, where it figures this request, the request made is equal to post, and for the response code is 204, you would find a hit. One of the things that you have to note here is that, the role. If you see the role there, it doesn't really scream to an investigator that it is a global administrator role. That is like one of the biggest cons, I would say, of Graph API logs, Graph activity logs. It doesn't give you all the information right to an investigator. And just like how Parthaman illustrated in the previous slides, you'll have to correlate this log with other logs, such as in this case, it's going to be audit logs, to understand what sort of role this is, and what sort of, what the user was basically assigned, what sort of role was assigned to the specific user. This was another interesting attempt. So this was an internal lateral movement. And what basically happened here is, the actor compromised a user token through a phishing attempt and used it with a delegated permission app. So what basically happens here is, the actor basically sends a phishing email, the user clicks it, they give their credentials out, and the user is presented with a consent screen. The consent basically asks for simple access, such as sending an email out. And as soon as the user clicks yes, which an user will do, the app receives this token, the actor uses this token to interact with the Graph API, and sends multiple different emails to, yeah, multiple different phishing emails internally to the user. And what we've done is, the first query basically finds all sort of slash send mail, which is the request URI there. It is one of the API functions which is used to send an email out. And similarly, the second query is pretty interesting. It establishes a baseline of all the apps using this function. So this gives you a complete eagle eye view of what is exactly happening in your environment, especially


chunk_text:
  specific user. This was another interesting attempt. So this was an internal lateral movement. And what basically happened here is, the actor compromised a user token through a phishing attempt and used it with a delegated permission app. So what basically happens here is, the actor basically sends a phishing email, the user clicks it, they give their credentials out, and the user is presented with a consent screen. The consent basically asks for simple access, such as sending an email out. And as soon as the user clicks yes, which an user will do, the app receives this token, the actor uses this token to interact with the Graph API, and sends multiple different emails to, yeah, multiple different phishing emails internally to the user. And what we've done is, the first query basically finds all sort of slash send mail, which is the request URI there. It is one of the API functions which is used to send an email out. And similarly, the second query is pretty interesting. It establishes a baseline of all the apps using this function. So this gives you a complete eagle eye view of what is exactly happening in your environment, especially with the slash send mail. And this is another similar to the previous slide. So we have seen multiple engagements where actors access mailboxes either by abusing Graph API's delegated or application permission. So let's jump into a test scenario. The threat actor abuses an app with an application permission mail.read.all. And what this does is, the mail.read.all, it basically gives an application the ability to read all sort of emails across the entire organization. So if the actor has access to this app and its token and this specific permission in the app, they would be able to read the CEO's email or any sort of emails and any sort of user in your organization. So what we did was, again, another query here. It basically creates a stat of all the applications used for reading emails. And it also provides you a time frame of sorts between a specific time to a specific time when this was read. And it gives you all the targets or basically all the mailboxes that were touched, which means all sort of emails that were read from different people's mailboxes. And of course, account. So what happens here is that you'll get


chunk_text:
  with the slash send mail. And this is another similar to the previous slide. So we have seen multiple engagements where actors access mailboxes either by abusing Graph API's delegated or application permission. So let's jump into a test scenario. The threat actor abuses an app with an application permission mail.read.all. And what this does is, the mail.read.all, it basically gives an application the ability to read all sort of emails across the entire organization. So if the actor has access to this app and its token and this specific permission in the app, they would be able to read the CEO's email or any sort of emails and any sort of user in your organization. So what we did was, again, another query here. It basically creates a stat of all the applications used for reading emails. And it also provides you a time frame of sorts between a specific time to a specific time when this was read. And it gives you all the targets or basically all the mailboxes that were touched, which means all sort of emails that were read from different people's mailboxes. And of course, account. So what happens here is that you'll get a clear picture as to different email inboxes that were touched and by different app IDs. So this gives you clarity about different applications which is using this sort of feature from Graph API. And yeah, as an investigator, your first thing is to baseline and understand if this is something that's supposed to happen. If it does, then all good. If it does not, then yeah, you'll have to, of course, spin up an incident. Okay, this is another place where we noticed another incident where we noticed compromised identities accessing files on SharePoint and OneDrive via Graph API. So the permissions used here are files read write and sites read write. And yeah, what I mean, the query, what it does is it's like a pretty simple one. It's a great starting point for investigating Graph API calls related to download activities. So if you take a look into the below image, you will see that this is the log that would be a resultant of such a read or such a download of a file from any sort of users. One of the pain points again of this log is that the one where I have marked


chunk_text:
  a clear picture as to different email inboxes that were touched and by different app IDs. So this gives you clarity about different applications which is using this sort of feature from Graph API. And yeah, as an investigator, your first thing is to baseline and understand if this is something that's supposed to happen. If it does, then all good. If it does not, then yeah, you'll have to, of course, spin up an incident. Okay, this is another place where we noticed another incident where we noticed compromised identities accessing files on SharePoint and OneDrive via Graph API. So the permissions used here are files read write and sites read write. And yeah, what I mean, the query, what it does is it's like a pretty simple one. It's a great starting point for investigating Graph API calls related to download activities. So if you take a look into the below image, you will see that this is the log that would be a resultant of such a read or such a download of a file from any sort of users. One of the pain points again of this log is that the one where I have marked it in red, that is the item ID. Basically, that is the item ID resolves to a file name. You do not see that here in the graph. It doesn't tell you which specific file was downloaded. So which means you'll have to correlate this with over here, it is going to be Cloud App Events to really understand what sort of file was touched and what was downloaded. I think that was pretty much the end of real life incidents. There were like multiple complex incidents, of course, we have seen as a part of experience and that. But of course, the focus here is to show you guys how powerful Graph API can basically be and to stress how important it is for you to collect these logs. So what we recommend here is to audit regularly all the intra-applications that use Graph API. This sort of reveals you all the excessive permissions or unexpected accesses that happen. And of course, when you see something of this sort, it could possibly mean that a service principle has been compromised. And of course, another thing is basically when you create auditing or baselining, it creates a safe list of


chunk_text:
  it in red, that is the item ID. Basically, that is the item ID resolves to a file name. You do not see that here in the graph. It doesn't tell you which specific file was downloaded. So which means you'll have to correlate this with over here, it is going to be Cloud App Events to really understand what sort of file was touched and what was downloaded. I think that was pretty much the end of real life incidents. There were like multiple complex incidents, of course, we have seen as a part of experience and that. But of course, the focus here is to show you guys how powerful Graph API can basically be and to stress how important it is for you to collect these logs. So what we recommend here is to audit regularly all the intra-applications that use Graph API. This sort of reveals you all the excessive permissions or unexpected accesses that happen. And of course, when you see something of this sort, it could possibly mean that a service principle has been compromised. And of course, another thing is basically when you create auditing or baselining, it creates a safe list of applications that is being used in your environment. So with this list, you can continuously monitor to detect the new application that pops up in your environment. And what we have done here in this query is that we have created a data table and we have sort of listed all sort of permissions and the privileges that comes along with it. And the reason why we feel that it could be a sensitive one. And we used all of this to basically quantify this with an app ID and their respective permissions. So this gives you sort of an eagle eye view of all the different app IDs that is being used and all sensitive roles that is being used for those specific app IDs. And then with this, if you sort of negate all the first party applications, what happens is you'll get a clear clarity about all different applications which is being used in your environment. So from there on, what you're going to do is trying to understand if those app IDs first is supposed to have these sort of roles. If it is, all good. If it is not supposed to have it, then of


chunk_text:
  applications that is being used in your environment. So with this list, you can continuously monitor to detect the new application that pops up in your environment. And what we have done here in this query is that we have created a data table and we have sort of listed all sort of permissions and the privileges that comes along with it. And the reason why we feel that it could be a sensitive one. And we used all of this to basically quantify this with an app ID and their respective permissions. So this gives you sort of an eagle eye view of all the different app IDs that is being used and all sensitive roles that is being used for those specific app IDs. And then with this, if you sort of negate all the first party applications, what happens is you'll get a clear clarity about all different applications which is being used in your environment. So from there on, what you're going to do is trying to understand if those app IDs first is supposed to have these sort of roles. If it is, all good. If it is not supposed to have it, then of course another investigation needs to be spinned up. After which, you constantly keep on monitoring such app IDs coming up with new permissions. And each time there's going to be a new permission, it's something that you have to jump on and understand if it is supposed to be there and if it is supposed to have those sort of roles. Okay, quick recap. So we saw what a graph API is. It is basically a service that connects multiple M365 applications. So we saw how to obtain these logs. One thing that you have to remind is that it's going to be super expensive, especially if you have a massive foothold in cloud. So please make sure to estimate the cost before you start collecting it. So we went over multiple key fields. We looked into a few correlatable tables such as cloud app events, audit logs, and then AAD risk users. And then we also looked into a few useful functions which is used during investigations. And then we went over multiple real world attack scenarios and of course a few test scenarios to illustrate all of these to understand better. And we looked into how baselining these


chunk_text:
  course another investigation needs to be spinned up. After which, you constantly keep on monitoring such app IDs coming up with new permissions. And each time there's going to be a new permission, it's something that you have to jump on and understand if it is supposed to be there and if it is supposed to have those sort of roles. Okay, quick recap. So we saw what a graph API is. It is basically a service that connects multiple M365 applications. So we saw how to obtain these logs. One thing that you have to remind is that it's going to be super expensive, especially if you have a massive foothold in cloud. So please make sure to estimate the cost before you start collecting it. So we went over multiple key fields. We looked into a few correlatable tables such as cloud app events, audit logs, and then AAD risk users. And then we also looked into a few useful functions which is used during investigations. And then we went over multiple real world attack scenarios and of course a few test scenarios to illustrate all of these to understand better. And we looked into how baselining these applications based on permissions and privileges help us. So if you did stick until now and felt that these graph API logs could be interesting, something that your organization doesn't collect or maybe they collect it and are not really using it, we would also highly recommend you to go over different contributions and articles from other researchers as well. These are pretty cool ones. There is one from Cloud Brothers where they have amazing advanced queries in it. Well, this is it. Thank you for having us. And if you folks have any questions about graph API or if you want to know how cool it is to work in a creation or Dart, feel free to come say hi. We'll be around here in this room. And I'm leaving you guys a QR code to scan that will take you to the article and the respective queries that we were speaking about. And yeah, of course, a shout out to DebugPrivilege, of course, he helped us mentor this whole thing. Thank you. Are there any questions in the audience? No further questions? Ah, there we are. Hi, guys. Thanks. I see this most likely a potential tool very


chunk_text:
  applications based on permissions and privileges help us. So if you did stick until now and felt that these graph API logs could be interesting, something that your organization doesn't collect or maybe they collect it and are not really using it, we would also highly recommend you to go over different contributions and articles from other researchers as well. These are pretty cool ones. There is one from Cloud Brothers where they have amazing advanced queries in it. Well, this is it. Thank you for having us. And if you folks have any questions about graph API or if you want to know how cool it is to work in a creation or Dart, feel free to come say hi. We'll be around here in this room. And I'm leaving you guys a QR code to scan that will take you to the article and the respective queries that we were speaking about. And yeah, of course, a shout out to DebugPrivilege, of course, he helped us mentor this whole thing. Thank you. Are there any questions in the audience? No further questions? Ah, there we are. Hi, guys. Thanks. I see this most likely a potential tool very attractive for hackers to sneak into organization, read as much as they can, reconnaissance collection and so on. So I believe that we have to see who has access to Microsoft Graph API to be able to protect that account even more to avoid this from happening. Have you ever seen an attack in which the hacker is exploiting the knowledge that he fetches from there so far? Because it's quite a new product, but I believe it's very attractive. So all of the scenarios that we spoke about, all of those were real life incidents. So everywhere we were able to see an actor performing one of these attacks that we spoke about. It was all from different engagements for different customers. But basically the point here is that they did access Graph API to do everything that I just said. Yep. There's one there. Thank you for the talk. At the end, or towards the end, you mentioned baselining applications to know what permissions they normally use and if there are any new applications in the environment. Has there been any thought to productizing that in something that's easier for customers to consume? Because, I mean, what you


chunk_text:
  attractive for hackers to sneak into organization, read as much as they can, reconnaissance collection and so on. So I believe that we have to see who has access to Microsoft Graph API to be able to protect that account even more to avoid this from happening. Have you ever seen an attack in which the hacker is exploiting the knowledge that he fetches from there so far? Because it's quite a new product, but I believe it's very attractive. So all of the scenarios that we spoke about, all of those were real life incidents. So everywhere we were able to see an actor performing one of these attacks that we spoke about. It was all from different engagements for different customers. But basically the point here is that they did access Graph API to do everything that I just said. Yep. There's one there. Thank you for the talk. At the end, or towards the end, you mentioned baselining applications to know what permissions they normally use and if there are any new applications in the environment. Has there been any thought to productizing that in something that's easier for customers to consume? Because, I mean, what you described here, you have to enable the logging, watch out for any additional costs that might incur, tune the query for your environment to know what applications or what permissions you're using. Given that this is such a useful tool for an organizational administrator, has there been any thought to just making this easier for someone to see what applications are in my environment, what permissions are they using, what's changed recently? So right now, there are a few audit logs that appears on the cloud app and a few other defender products where you can actually see what sort of applications are existing on your environment and with high privileges and basically stuff like that. Of course, for that, you'll have to know really which one, where to go look for it, which report to go click and really analyze it. And this is more of a shortcut, especially for a security investigator, where you can really run a query and then understand, okay, these are the applications, this is what I have to concentrate on, this is where I have to go look at it, and these are the applications that I know that is existing in my


chunk_text:
  described here, you have to enable the logging, watch out for any additional costs that might incur, tune the query for your environment to know what applications or what permissions you're using. Given that this is such a useful tool for an organizational administrator, has there been any thought to just making this easier for someone to see what applications are in my environment, what permissions are they using, what's changed recently? So right now, there are a few audit logs that appears on the cloud app and a few other defender products where you can actually see what sort of applications are existing on your environment and with high privileges and basically stuff like that. Of course, for that, you'll have to know really which one, where to go look for it, which report to go click and really analyze it. And this is more of a shortcut, especially for a security investigator, where you can really run a query and then understand, okay, these are the applications, this is what I have to concentrate on, this is where I have to go look at it, and these are the applications that I know that is existing in my environment with the highest privilege, something that I have to keep an eye on, and anything beyond this, anything suddenly pops up beyond what I already know, that is something that I might have to investigate on. So this is sort of like, let us say, a quick fix, especially for someone who has access to log analytics and who constantly bugs on these sort of things. Thank you. Yep. Thank you very much. Sorry, on top of what my friend said, to make your life a bit easier, there is a webpage in Microsoft Documents where they will list all of their first party applications. So basically, the reason why we didn't include that is because you can't simply whitelist all of the first party applications, because there is a way people can use those applications to perform malicious activities. One of the examples is like, so you can use a device auth token, so which means that can be used for phishing, and that would be a first party Microsoft application. So if we whitelist everything, then we might be missing that activity. So like Shiva mentioned, it has to be manual baselining activity, just once, and then


chunk_text:
  environment with the highest privilege, something that I have to keep an eye on, and anything beyond this, anything suddenly pops up beyond what I already know, that is something that I might have to investigate on. So this is sort of like, let us say, a quick fix, especially for someone who has access to log analytics and who constantly bugs on these sort of things. Thank you. Yep. Thank you very much. Sorry, on top of what my friend said, to make your life a bit easier, there is a webpage in Microsoft Documents where they will list all of their first party applications. So basically, the reason why we didn't include that is because you can't simply whitelist all of the first party applications, because there is a way people can use those applications to perform malicious activities. One of the examples is like, so you can use a device auth token, so which means that can be used for phishing, and that would be a first party Microsoft application. So if we whitelist everything, then we might be missing that activity. So like Shiva mentioned, it has to be manual baselining activity, just once, and then you can, once you finish it, then it might be a bit easier for you to perform investigation. Yeah. Thank you. Thank you. Any other questions? All right, then I would like to thank you very much.


chunk_text:
  you can, once you finish it, then it might be a bit easier for you to perform investigation. Yeah. Thank you. Thank you. Any other questions? All right, then I would like to thank you very much.


chunk_text:
  { "webpage_url": "https://www.youtube.com/watch?v=gkxOmbT2Jls", "title": "Louis Barrett - Small Language Models for Application Security - Beyond ChatGPT", "description": "Small language models such as LLaMa and Alpaca will revolutionize the way applications are designed and secured. This talk will provide an overview of how to use small, open source language models to enhance application security workflows and help us shift even further left.\n\n\nhttps://lascon2023.sched.com/event/1RE5A/small-language-models-for-application-security-beyond-chatgpt", "channel_url": "https://www.youtube.com/channel/UCDHsOiMPS-SLppAOAJRD37Q", "duration": 3634, "channel": "LASCON", "uploader": "LASCON", "upload_date": "20231118" } This text was transcribed using whisper model: large-v2 You You You Afternoon last con welcome to small language models for application security beyond chat GPT My name is Louie Barrett I'll be your guide today through the journey on the future of application security I'm a full stack security researcher with a passion for building security tools and Solutions that increase the efficiency and happiness of my teams. I currently lead product security at scale AI where we focus on implementing security solutions for the MLOps lifecycle and ultimately improving the security of AI backed products Prior to this I led detection engineering and instant response teams at both segment Twilio and these roles I gained my first experience implementing machine learning algorithms and approaches to detect and prioritize security threats ever since then I've been experimenting with solutions at the intersection of artificial intelligence and security and The goal of this presentation today is to enable security teams to unlock cutting-edge capabilities and apply them to security challenges To accomplish this we'll cover some basics of language models their recent history and current capabilities Next we'll look at some of the common application design patterns Emerging our LLMs and categorize them to better understand how language models are built Next we'll look at what models under 30 billion parameters can do for us These are small language models and take a look at some common language of model apps break down their functional composition and Then we'll show you how these can map to app sec responsibilities Finally I'll share a couple demos and lessons learned around these use cases if time remains I'll take some questions Before I get started a quick poll Who here is responsible for threat modeling security reviews or addressing vulnerabilities? Awesome that makes things easy who here has heard of a large language models Okay, how about small language models? That's what I thought have any of you used open AI


chunk_text:
  prioritize security threats ever since then I've been experimenting with solutions at the intersection of artificial intelligence and security and The goal of this presentation today is to enable security teams to unlock cutting-edge capabilities and apply them to security challenges To accomplish this we'll cover some basics of language models their recent history and current capabilities Next we'll look at some of the common application design patterns Emerging our LLMs and categorize them to better understand how language models are built Next we'll look at what models under 30 billion parameters can do for us These are small language models and take a look at some common language of model apps break down their functional composition and Then we'll show you how these can map to app sec responsibilities Finally I'll share a couple demos and lessons learned around these use cases if time remains I'll take some questions Before I get started a quick poll Who here is responsible for threat modeling security reviews or addressing vulnerabilities? Awesome that makes things easy who here has heard of a large language models Okay, how about small language models? That's what I thought have any of you used open AI prior to chat GPT? Interesting and how many of you are familiar with hugging face? cool and Finally who has attempted to deploy your own language models internally? Interesting okay cool great. Thanks for that So for anyone who wasn't familiar Large language models are more commonly large language models are powerful AI algorithms that can analyze text and language as name implies These models are trained on massive a massive corpus of data and tasks allowing to produce incredible results when applied to challenges typically handled by humans Over the past three years or so and more recently the last 12 months Development of an evolution of these models has basically exploded It's what we're kind of terming the AI summer Prior to 2020 practical and production use of artificial intelligence within security teams Was basically science fiction for all but the most well-funded organizations And even then those who did have access to artificial intelligence often find it unwieldy and challenging to deploy The language models of this era often required an absurd amount of computing resources Performing the simplest natural language processing tasks and to make things even more challenging Deploying them generally required a data science


chunk_text:
  prior to chat GPT? Interesting and how many of you are familiar with hugging face? cool and Finally who has attempted to deploy your own language models internally? Interesting okay cool great. Thanks for that So for anyone who wasn't familiar Large language models are more commonly large language models are powerful AI algorithms that can analyze text and language as name implies These models are trained on massive a massive corpus of data and tasks allowing to produce incredible results when applied to challenges typically handled by humans Over the past three years or so and more recently the last 12 months Development of an evolution of these models has basically exploded It's what we're kind of terming the AI summer Prior to 2020 practical and production use of artificial intelligence within security teams Was basically science fiction for all but the most well-funded organizations And even then those who did have access to artificial intelligence often find it unwieldy and challenging to deploy The language models of this era often required an absurd amount of computing resources Performing the simplest natural language processing tasks and to make things even more challenging Deploying them generally required a data science background. I don't know about you guys But I don't have that background and I'm not a big math guy So that was really a challenging period for me getting into this stuff And leads to say the security solutions that are built on top of these models at the time were far from reliable And that's all that's like partially because the people who are building them don't understand our use cases and also partially because They're just so large and unwieldy in the first place But in June of 2020 this all really changed With the release of open the AIS chat GPT and while we had transformer architecture models before this point It wasn't until the public and other engineers really got Aware of this technology that it really started to boom because we started applying our own AI For those weren't aware GPT-3 is a decoder only transformer model of deep neural networks Which uses attention in place of previous recurrence like older generation models did and this allows the model to selectively Focus on segments of its input and predict, which is the most relevant to the actual user Development this new transformer model jump-started


chunk_text:
  background. I don't know about you guys But I don't have that background and I'm not a big math guy So that was really a challenging period for me getting into this stuff And leads to say the security solutions that are built on top of these models at the time were far from reliable And that's all that's like partially because the people who are building them don't understand our use cases and also partially because They're just so large and unwieldy in the first place But in June of 2020 this all really changed With the release of open the AIS chat GPT and while we had transformer architecture models before this point It wasn't until the public and other engineers really got Aware of this technology that it really started to boom because we started applying our own AI For those weren't aware GPT-3 is a decoder only transformer model of deep neural networks Which uses attention in place of previous recurrence like older generation models did and this allows the model to selectively Focus on segments of its input and predict, which is the most relevant to the actual user Development this new transformer model jump-started the development of similar models using the architecture and The midst of a global pandemic where many of us are trapped inside and the unexpected availability of GPUs do their major cryptocurrency platform or Moving away from GPU based mining see theory emerge the next three years would see hundreds of models being developed which brings us to 2023 This year has been especially interesting to the release of open the eyes GPT-4 and chat GPT-4 This year has been especially interesting to the release of open the eyes GPT-4 and chat GPT As well as the beginning of the open-source revolution led by the accidental leaking of llamas, I'm sorry met as llama model Since then hundreds of models based on this transformer architecture and met as llama have been released allowing researchers and organizations the ability experience artificial intelligence for themselves and even more recently Mosaic ML and Mistral AI Respectively released the first open-source models which could be used commercially. This is a game-changer You can now take these models train them a little bit and actually sell them back to your customers for your security company. For example these new open-source models have been found to


chunk_text:
  the development of similar models using the architecture and The midst of a global pandemic where many of us are trapped inside and the unexpected availability of GPUs do their major cryptocurrency platform or Moving away from GPU based mining see theory emerge the next three years would see hundreds of models being developed which brings us to 2023 This year has been especially interesting to the release of open the eyes GPT-4 and chat GPT-4 This year has been especially interesting to the release of open the eyes GPT-4 and chat GPT As well as the beginning of the open-source revolution led by the accidental leaking of llamas, I'm sorry met as llama model Since then hundreds of models based on this transformer architecture and met as llama have been released allowing researchers and organizations the ability experience artificial intelligence for themselves and even more recently Mosaic ML and Mistral AI Respectively released the first open-source models which could be used commercially. This is a game-changer You can now take these models train them a little bit and actually sell them back to your customers for your security company. For example these new open-source models have been found to perform comparably to the original GPT-3 and considering GPT-3 took millions of dollars to train and occupied 800 gigabytes of memory and utilizes 264 billion parameters. A model of only 7 billion parameters performing nearly equally is nothing short of an engineering miracle In the next few slides I'll go over what these models are able to do for us and get to some of the common types of language model apps So what can language models do for us? Language models of today depending on how they were trained are able to perform a number of different types of tasks You can see the different types listed here But first up is text generation, which we're I think all familiar with at this point It represents the model's ability to use internal neural networks to generate coherent human language Next we have text summarization Where models are able to ingest large corpuses of text extract key points and during a more concise result So compressing some text down to the more concise version of itself Question answering where the model is able to predict I'm sorry we're able to provide answers to questions asked based on the data in


chunk_text:
  perform comparably to the original GPT-3 and considering GPT-3 took millions of dollars to train and occupied 800 gigabytes of memory and utilizes 264 billion parameters. A model of only 7 billion parameters performing nearly equally is nothing short of an engineering miracle In the next few slides I'll go over what these models are able to do for us and get to some of the common types of language model apps So what can language models do for us? Language models of today depending on how they were trained are able to perform a number of different types of tasks You can see the different types listed here But first up is text generation, which we're I think all familiar with at this point It represents the model's ability to use internal neural networks to generate coherent human language Next we have text summarization Where models are able to ingest large corpuses of text extract key points and during a more concise result So compressing some text down to the more concise version of itself Question answering where the model is able to predict I'm sorry we're able to provide answers to questions asked based on the data in central network and occasionally user input and finally classification Models are able to determine if the provided data by the user falls in the one category or the other Say for example, is this Twitter post offensive right things like this By combining the language model capabilities on the previous slide with traditional software engineering principles a number of design patterns have emerged These design patterns typically take the form of A play adapting AI to human roles in some capacity and some popular examples include Conversation apps such as chat bots and personal assistants the most popular of which you know is arguably chat GPT Next we have our writing apps Next we have our writing apps Leveraging the journey of a summer of capabilities of the models along end-users to prove create creative writing and professional content One such example is copy the AI where you can make business copy email copy marketing copy using these methods and Lately, we're seeing the development of a new pattern where you have these process magic and analysis support applications And all those applications are amazing they have one thing in common in my opinion that essentially is an Achilles heel


chunk_text:
  central network and occasionally user input and finally classification Models are able to determine if the provided data by the user falls in the one category or the other Say for example, is this Twitter post offensive right things like this By combining the language model capabilities on the previous slide with traditional software engineering principles a number of design patterns have emerged These design patterns typically take the form of A play adapting AI to human roles in some capacity and some popular examples include Conversation apps such as chat bots and personal assistants the most popular of which you know is arguably chat GPT Next we have our writing apps Next we have our writing apps Leveraging the journey of a summer of capabilities of the models along end-users to prove create creative writing and professional content One such example is copy the AI where you can make business copy email copy marketing copy using these methods and Lately, we're seeing the development of a new pattern where you have these process magic and analysis support applications And all those applications are amazing they have one thing in common in my opinion that essentially is an Achilles heel Nearly all of them rely on commercial large language models And while this seems like an advantage at first There are numerous issues related data privacy data sovereignty and information security when chat GPT So when GPT 3 and later chat GPT became popular many organizations quickly had their data leaked almost overnight due to overeager employees using the models but that being said let's dive into understanding how these applications are composed by using the LMAP maturity scale and Hopefully should allow us to build safer more private alternatives to some of these application design patterns And ultimately leverage them for sensitive use cases such as security So the LMAP rating scale is a Scale we use internally to basically evaluate how complex an application is And what we should look at in terms of either securing it or deploying it The scale is currently broken down into five levels level one being basic Which is basically a one-to-one interaction with the model you have a basically no short term memory so the model doesn't remember the last interaction you had with it is basically prompt in response out and The entire context past the model is used for


chunk_text:
  Nearly all of them rely on commercial large language models And while this seems like an advantage at first There are numerous issues related data privacy data sovereignty and information security when chat GPT So when GPT 3 and later chat GPT became popular many organizations quickly had their data leaked almost overnight due to overeager employees using the models but that being said let's dive into understanding how these applications are composed by using the LMAP maturity scale and Hopefully should allow us to build safer more private alternatives to some of these application design patterns And ultimately leverage them for sensitive use cases such as security So the LMAP rating scale is a Scale we use internally to basically evaluate how complex an application is And what we should look at in terms of either securing it or deploying it The scale is currently broken down into five levels level one being basic Which is basically a one-to-one interaction with the model you have a basically no short term memory so the model doesn't remember the last interaction you had with it is basically prompt in response out and The entire context past the model is used for that one-to-one interaction At level two Models gain extended context and we see the chat design parents are emerging You have this limited short-term memory which allows the model to remember its last few interactions up to a point This is usually based on the maximum context window available to the model Additionally allows the model to remain grounded because you have some previous context in your discussion So it's less likely to hallucinate if you're nudging it along. We'll touch on this a bit more later and the actors in these LMAP systems tend to be now a human user a System user or a system prompt to give it some guidance and of course language models internal neural network The data sources in this case are the user input Of course the models internal neural networks training data and the act interactions topology And the act interaction style becomes a multi-turn so you're going back and forth with the model until it's able to give you the type of response you need And as mentioned before chat is the most common example of this chat design pattern At level three things get really interesting and this is where


chunk_text:
  that one-to-one interaction At level two Models gain extended context and we see the chat design parents are emerging You have this limited short-term memory which allows the model to remember its last few interactions up to a point This is usually based on the maximum context window available to the model Additionally allows the model to remain grounded because you have some previous context in your discussion So it's less likely to hallucinate if you're nudging it along. We'll touch on this a bit more later and the actors in these LMAP systems tend to be now a human user a System user or a system prompt to give it some guidance and of course language models internal neural network The data sources in this case are the user input Of course the models internal neural networks training data and the act interactions topology And the act interaction style becomes a multi-turn so you're going back and forth with the model until it's able to give you the type of response you need And as mentioned before chat is the most common example of this chat design pattern At level three things get really interesting and this is where I see a lot of the current design trending to within LMAPs where we're now using indexes to do a pattern called rag or retrieval augmented generation and It allows the language model to no longer have to rely on the data. It was trained on Do the addition of retrieval indexes in the form of vector databases or traditional search clusters? Models are able to augment their capabilities immensely and almost completely eliminate hallucinations so you can imagine passing in a complete corpus of What the model should use as rules and will always reference those rules and it's like ladder generations So This leads to models having responses that are more truthful more correct and ideally safer for an end-user to actually interact with And due to our retrieval works It's a multi-phase process You can't pass an entire database worth of content to a model did this limited input window Basically chunk in the smaller pieces and that's what kicks this multi-phase type of interaction in the model behind the scenes Even though this is typically invisible to the end-user Now I mentioned there are five levels to this right and so far we've touched on three


chunk_text:
  I see a lot of the current design trending to within LMAPs where we're now using indexes to do a pattern called rag or retrieval augmented generation and It allows the language model to no longer have to rely on the data. It was trained on Do the addition of retrieval indexes in the form of vector databases or traditional search clusters? Models are able to augment their capabilities immensely and almost completely eliminate hallucinations so you can imagine passing in a complete corpus of What the model should use as rules and will always reference those rules and it's like ladder generations So This leads to models having responses that are more truthful more correct and ideally safer for an end-user to actually interact with And due to our retrieval works It's a multi-phase process You can't pass an entire database worth of content to a model did this limited input window Basically chunk in the smaller pieces and that's what kicks this multi-phase type of interaction in the model behind the scenes Even though this is typically invisible to the end-user Now I mentioned there are five levels to this right and so far we've touched on three and I'd like to mention that For the most part a lot of organizations right now are just scratching the surface of level three You don't really see it well commercialized and level four is almost unheard of in production But I want to just touch on one notable example of a level three application that we've developed at scale It's a retrieval platform called Donovan that allows you to basically connect either Things like confluence Google Drive your Google Docs Whatever you want to your model as a retrieval index such you can ask questions in that corpus We use this a lot internally on the security team to drop in all of our policy documentation Give this to our like sales people such that they want to come to security and ask questions It's all already in the docs and the model is basically following our best practices But that being said let's actually take a look at what level three Entails because things get really interesting at this tier because it's really experimental and since there aren't too many commercial products It kind of diverges in two directions. Whereas the previous levels have been directly iterative So


chunk_text:
  and I'd like to mention that For the most part a lot of organizations right now are just scratching the surface of level three You don't really see it well commercialized and level four is almost unheard of in production But I want to just touch on one notable example of a level three application that we've developed at scale It's a retrieval platform called Donovan that allows you to basically connect either Things like confluence Google Drive your Google Docs Whatever you want to your model as a retrieval index such you can ask questions in that corpus We use this a lot internally on the security team to drop in all of our policy documentation Give this to our like sales people such that they want to come to security and ask questions It's all already in the docs and the model is basically following our best practices But that being said let's actually take a look at what level three Entails because things get really interesting at this tier because it's really experimental and since there aren't too many commercial products It kind of diverges in two directions. Whereas the previous levels have been directly iterative So it's again I mentioned it's split into two levels we have a little 4a and a little 4b Level 4a brings about language model based agents, which we're just starting to scratch the surface on Most of these aren't in a true production Environment right now and as models can interact on their own to some degree and solve problems for their user So this introduces the addition of a goals manager or goal framework behind the scenes and that basically breaks your single query down into either steps or Instructions for the model that generates sub prompts that either the core model or additional sub models follow So the actors in the system become the human user our existing system prompts for guidance, which were basically always there from level three and up Our goals manager or goals framework and obviously the language models internal neural network As with the previous level we also have retrieval indexes here So not only is the model able to go out or the application able to go out and act on behalf of the user It's also able to reference content that users provided ahead of time As guidance or a guidebook,


chunk_text:
  it's again I mentioned it's split into two levels we have a little 4a and a little 4b Level 4a brings about language model based agents, which we're just starting to scratch the surface on Most of these aren't in a true production Environment right now and as models can interact on their own to some degree and solve problems for their user So this introduces the addition of a goals manager or goal framework behind the scenes and that basically breaks your single query down into either steps or Instructions for the model that generates sub prompts that either the core model or additional sub models follow So the actors in the system become the human user our existing system prompts for guidance, which were basically always there from level three and up Our goals manager or goals framework and obviously the language models internal neural network As with the previous level we also have retrieval indexes here So not only is the model able to go out or the application able to go out and act on behalf of the user It's also able to reference content that users provided ahead of time As guidance or a guidebook, right? and As with the previous level the interaction style is multi-turn and multi-phase so you can chat back and forth this type of a model and you can also have it occur in phases because we're using retrieval and also the goal framework stepping through a list of Goals until it reaches the end, right? So that becomes multi-phase And We've recently seen the emergence of a little 4a and chat GPT, but it's not as direct as people would really think They add the recent feature called search with Bing. It's very experimental. Lots could go wrong there. I'll touch on that later in the talk but the models Systematically does Bing searches until it finds the answer to the thing you were looking for and behind the scenes it's treating Bing as a retrieval index and Treating your prompt as the source for that original goal breakdown and when it finally breaks things out to goals It's taking all those goals and searching them on Bing and then summarizing everything and bringing it back to the end-user So right now, I think chat GPT is the only Commercially or public accessible platform that I've seen do this


chunk_text:
  right? and As with the previous level the interaction style is multi-turn and multi-phase so you can chat back and forth this type of a model and you can also have it occur in phases because we're using retrieval and also the goal framework stepping through a list of Goals until it reaches the end, right? So that becomes multi-phase And We've recently seen the emergence of a little 4a and chat GPT, but it's not as direct as people would really think They add the recent feature called search with Bing. It's very experimental. Lots could go wrong there. I'll touch on that later in the talk but the models Systematically does Bing searches until it finds the answer to the thing you were looking for and behind the scenes it's treating Bing as a retrieval index and Treating your prompt as the source for that original goal breakdown and when it finally breaks things out to goals It's taking all those goals and searching them on Bing and then summarizing everything and bringing it back to the end-user So right now, I think chat GPT is the only Commercially or public accessible platform that I've seen do this at least a little bit But we're definitely gonna see more of this as time progresses But with that said let's jump into a little 4b since again, I mentioned things are kind of diverging a bit here Level 4b introduces multi-model consensus where you have multiple models kind of acting at together Here you need an extensive short-term memory because the number of interactions you have even in a single conversation Can be pretty extensive Requires a large input size so models that support more than the standard 2,000 tokens and just to kind of break down what tokens are for language models One token is about three quarters of a word, right? So you can imagine you can get pages and pages of data into these things if you have a large input window We have a new item getting added to our data sources in the form of an orchestrator the goal of this orchestrator is to basically give work to all the sub models in this system and Tell them how to actually operate and once this orchestrator has Gotten the results back from all the sub models it rolls those results back up. It says


chunk_text:
  at least a little bit But we're definitely gonna see more of this as time progresses But with that said let's jump into a little 4b since again, I mentioned things are kind of diverging a bit here Level 4b introduces multi-model consensus where you have multiple models kind of acting at together Here you need an extensive short-term memory because the number of interactions you have even in a single conversation Can be pretty extensive Requires a large input size so models that support more than the standard 2,000 tokens and just to kind of break down what tokens are for language models One token is about three quarters of a word, right? So you can imagine you can get pages and pages of data into these things if you have a large input window We have a new item getting added to our data sources in the form of an orchestrator the goal of this orchestrator is to basically give work to all the sub models in this system and Tell them how to actually operate and once this orchestrator has Gotten the results back from all the sub models it rolls those results back up. It says hey user. Here's your answer So you can imagine this being used if you arrange models into teams or virtual teams We have one being the manager one being the engineer potentially and one being the designer, right? And they all have different goals. They relate to the larger project You can kick a prompt off this model such that it Rolls up what the entire project plan would look like because the sub models have their own individual goals being kicked off by the orchestrator Chat GPT actually does this behind the scenes and this is for GPT-4 specifically Lots of people think GPT-4 is a single model But it's actually many models behind the scenes or like, you know, three models in a trench coat essentially is what you're really getting Level five again, this is like unheard of this doesn't really exist in a commercial sense yet But models get long-term memory and start acting true cohort systems So taking that concept from level 4b where you have small pods of actors and scaling that out, right? so horizontally scaling that concept the input size has to be Immense because you're basically taking a huge chunk of


chunk_text:
  hey user. Here's your answer So you can imagine this being used if you arrange models into teams or virtual teams We have one being the manager one being the engineer potentially and one being the designer, right? And they all have different goals. They relate to the larger project You can kick a prompt off this model such that it Rolls up what the entire project plan would look like because the sub models have their own individual goals being kicked off by the orchestrator Chat GPT actually does this behind the scenes and this is for GPT-4 specifically Lots of people think GPT-4 is a single model But it's actually many models behind the scenes or like, you know, three models in a trench coat essentially is what you're really getting Level five again, this is like unheard of this doesn't really exist in a commercial sense yet But models get long-term memory and start acting true cohort systems So taking that concept from level 4b where you have small pods of actors and scaling that out, right? so horizontally scaling that concept the input size has to be Immense because you're basically taking a huge chunk of data and dividing that across a number of different models We have our orchestrator from the previous level Along with internal indexes and we have this multi-turn multi-phase interaction But the difference here is now you have pods of models able to communicate with each other So you have an orchestrator in each pod that talks to another pod says, hey, did you finish? You know either Summarizing or whatever the goal of that other pod was and then there's a grand orchestra that rolls everything back up to the top so you get the ability to basically coordinate models at a high level, but the resource usage and Requirements of the system like this typically make it something that's out of the realm of consumer usage and I'll kind of leave that there All right That All right, so those of you are following along Congrats, we've reached a practical portion of the talk and I'll try to slow down if I'm going too fast, please. Let me know we'll get into real-world use cases for the model capabilities and design parents that And map the design parents to this middle zero world cases Using the rating skill we


chunk_text:
  data and dividing that across a number of different models We have our orchestrator from the previous level Along with internal indexes and we have this multi-turn multi-phase interaction But the difference here is now you have pods of models able to communicate with each other So you have an orchestrator in each pod that talks to another pod says, hey, did you finish? You know either Summarizing or whatever the goal of that other pod was and then there's a grand orchestra that rolls everything back up to the top so you get the ability to basically coordinate models at a high level, but the resource usage and Requirements of the system like this typically make it something that's out of the realm of consumer usage and I'll kind of leave that there All right That All right, so those of you are following along Congrats, we've reached a practical portion of the talk and I'll try to slow down if I'm going too fast, please. Let me know we'll get into real-world use cases for the model capabilities and design parents that And map the design parents to this middle zero world cases Using the rating skill we covered earlier So I just want to share this quote from William Gibson real quick because I think it really touches on How things are currently moving in the AI space and that's the future is already here. It's just not very evenly distributed and The goal of this talk is to more evenly distribute that future and allow us to greatly improve application security, right? So to me application security has always been about securing the software development lifecycle So, let's see how we can apply some of the concepts I discussed earlier to the actual software development lifecycle So AppSec teams typically have a challenge of juggling security design reviews, threat modeling, code reviews, and vulnerability management Each of these responsibilities requires us to read lengthy documentation, draw conclusions, follow up with developers, and provide recommendations and So for those who've been paying attention you'll notice these responsibilities basically map fairly well with the task of abilities of language models which are a summarization, closed QA, classification, and instruction following and The natural alignment between our responsibilities and the capabilities of these models is why I first began this research In the following slides, we'll see how we


chunk_text:
  covered earlier So I just want to share this quote from William Gibson real quick because I think it really touches on How things are currently moving in the AI space and that's the future is already here. It's just not very evenly distributed and The goal of this talk is to more evenly distribute that future and allow us to greatly improve application security, right? So to me application security has always been about securing the software development lifecycle So, let's see how we can apply some of the concepts I discussed earlier to the actual software development lifecycle So AppSec teams typically have a challenge of juggling security design reviews, threat modeling, code reviews, and vulnerability management Each of these responsibilities requires us to read lengthy documentation, draw conclusions, follow up with developers, and provide recommendations and So for those who've been paying attention you'll notice these responsibilities basically map fairly well with the task of abilities of language models which are a summarization, closed QA, classification, and instruction following and The natural alignment between our responsibilities and the capabilities of these models is why I first began this research In the following slides, we'll see how we can leverage the capabilities of these models mainly small language models To improve the speed and accuracy of our security team But please keep in mind that the goal is not to fully automate your jobs That simply cannot be a thing at this point But rather to assist you in completing them and reduce the time you spend doing, you know, painstaking tasks So everything I'll describe going forward requires some level of human interaction or human-in-the-loop somewhere in the process and that's both to avoid the LM top ten vulnerability number nine, which is over reliance on the models output Because we do have to review and make sure these things are correct. Otherwise Terrible things can happen. You don't want dev shipping something that's vulnerable that you had the model sign off on without any kind of review Cool, so let's touch on how we can use models for security design review Typically, hopefully you guys can see this, a development team provides some initial documentation Such as a software design document or product requirement document The team then reviews the document signs off with a rating of pass, fail, or review A pass allows developers to


chunk_text:
  can leverage the capabilities of these models mainly small language models To improve the speed and accuracy of our security team But please keep in mind that the goal is not to fully automate your jobs That simply cannot be a thing at this point But rather to assist you in completing them and reduce the time you spend doing, you know, painstaking tasks So everything I'll describe going forward requires some level of human interaction or human-in-the-loop somewhere in the process and that's both to avoid the LM top ten vulnerability number nine, which is over reliance on the models output Because we do have to review and make sure these things are correct. Otherwise Terrible things can happen. You don't want dev shipping something that's vulnerable that you had the model sign off on without any kind of review Cool, so let's touch on how we can use models for security design review Typically, hopefully you guys can see this, a development team provides some initial documentation Such as a software design document or product requirement document The team then reviews the document signs off with a rating of pass, fail, or review A pass allows developers to continue to the next iteration of the design Until a significant change occurs However in scenarios where they need to or doesn't satisfy design requirements We fail or say it's a review and then security team follows up and says developer change thing go back to review The review process ideally shouldn't take up much time, but when you consider how outnumbered most security teams are These reviews can quickly pile up and clog up the works Considering these challenges Let's think about what security design review really is and how we can plug in a language model and make the process more scalable So whenever I'm thinking about should I use, whether or not I should use a language model I'm asked, you can ask yourself this, who is prompting me and Should they be prompting a model instead, right? And we've applied it, if we apply this mapping back to the sequence diagram We should be able to see how a language model can help here In this case the model is able to take over reviewing the user submitted documents by reading this and summarizing the content Following the instructions provided in a review standard and answering


chunk_text:
  continue to the next iteration of the design Until a significant change occurs However in scenarios where they need to or doesn't satisfy design requirements We fail or say it's a review and then security team follows up and says developer change thing go back to review The review process ideally shouldn't take up much time, but when you consider how outnumbered most security teams are These reviews can quickly pile up and clog up the works Considering these challenges Let's think about what security design review really is and how we can plug in a language model and make the process more scalable So whenever I'm thinking about should I use, whether or not I should use a language model I'm asked, you can ask yourself this, who is prompting me and Should they be prompting a model instead, right? And we've applied it, if we apply this mapping back to the sequence diagram We should be able to see how a language model can help here In this case the model is able to take over reviewing the user submitted documents by reading this and summarizing the content Following the instructions provided in a review standard and answering a few basic questions defined in the same review It then provides a roll-up of the results to members of the security team for review And although we only eliminated one step from this workflow Those of you do lots of code review will understand this is the most time-consuming step Design documents may be many pages long highly complex contain references to systems or concept that more junior engineers might not be able to understand And generally coupled with the volume of reviews that come in it can make design review like a pretty heavy task for any security team And if you're trying to shift left It becomes almost impossible given the scale of it, right? And this is a relatively simple LM app the use case We're able to quickly analyze documents notify the developers of gaps And we're able to do a lot of other things in the same way So we're able to do a lot of things in the same way We're able to quickly analyze documents notify the developers of gaps and ideally while reducing the burden on your security team So this is a really simple example, but I want you to


chunk_text:
  a few basic questions defined in the same review It then provides a roll-up of the results to members of the security team for review And although we only eliminated one step from this workflow Those of you do lots of code review will understand this is the most time-consuming step Design documents may be many pages long highly complex contain references to systems or concept that more junior engineers might not be able to understand And generally coupled with the volume of reviews that come in it can make design review like a pretty heavy task for any security team And if you're trying to shift left It becomes almost impossible given the scale of it, right? And this is a relatively simple LM app the use case We're able to quickly analyze documents notify the developers of gaps And we're able to do a lot of other things in the same way So we're able to do a lot of things in the same way We're able to quickly analyze documents notify the developers of gaps and ideally while reducing the burden on your security team So this is a really simple example, but I want you to keep these Benefits in mind as we move on to the next example, which is basically a larger version of a security design review the threat modeling session Threat modeling can be considered more advanced form of design review like I mentioned This is typically a collaborative process between developers members of the security team and occasionally other stakeholders The threat models can be conducted synchronously or asynchronously with teams providing guidelines and documentation ahead of time or bringing them to the meeting But regardless of the approach there is a heavy analysis burden placed in this Documents when they're provided Fine okay, awesome. So additional design documents when provided by this these development teams They they might contain Content of the Siri team like I mentioned before wasn't familiar with and we kind of have to go through those and reason with them Is this system in line with our design standards, right? and do the complexity of the process and the number of people involved it typically requires a significant investment both from development teams themselves and our security team Dev teams are often on tight schedules. They want to face them as quickly as possible but security


chunk_text:
  keep these Benefits in mind as we move on to the next example, which is basically a larger version of a security design review the threat modeling session Threat modeling can be considered more advanced form of design review like I mentioned This is typically a collaborative process between developers members of the security team and occasionally other stakeholders The threat models can be conducted synchronously or asynchronously with teams providing guidelines and documentation ahead of time or bringing them to the meeting But regardless of the approach there is a heavy analysis burden placed in this Documents when they're provided Fine okay, awesome. So additional design documents when provided by this these development teams They they might contain Content of the Siri team like I mentioned before wasn't familiar with and we kind of have to go through those and reason with them Is this system in line with our design standards, right? and do the complexity of the process and the number of people involved it typically requires a significant investment both from development teams themselves and our security team Dev teams are often on tight schedules. They want to face them as quickly as possible but security teams want things to ideally be secure. We're security teams and Address all the vulnerabilities up front that way they don't reach production the disconnect typically results in either short light threat models that leave gaps or lengthy threat models that block the pace of development So I can write balance between speed and quality requires parties involved to be on the same page and Collectively understand the risks within this threat model Do the differences in levels and point of view this this process like design reviews? I just touched on could be extremely difficult to scale But let's look at from a high-level perspective for how we can actually apply some of the concepts we saw earlier to the threat modeling process So The existing process requires quite a bit of back-and-forth interaction between development teams and security and a solid understanding of the implications of both design and mitigation decisions This can be especially difficult in resource constrained organizations, I don't know about you guys, but we don't have a one-to-one relationship between our engineers and our security team members and Look, you know, it's a situation where you can be swamped even if you are proficient


chunk_text:
  teams want things to ideally be secure. We're security teams and Address all the vulnerabilities up front that way they don't reach production the disconnect typically results in either short light threat models that leave gaps or lengthy threat models that block the pace of development So I can write balance between speed and quality requires parties involved to be on the same page and Collectively understand the risks within this threat model Do the differences in levels and point of view this this process like design reviews? I just touched on could be extremely difficult to scale But let's look at from a high-level perspective for how we can actually apply some of the concepts we saw earlier to the threat modeling process So The existing process requires quite a bit of back-and-forth interaction between development teams and security and a solid understanding of the implications of both design and mitigation decisions This can be especially difficult in resource constrained organizations, I don't know about you guys, but we don't have a one-to-one relationship between our engineers and our security team members and Look, you know, it's a situation where you can be swamped even if you are proficient in threat modeling and design review But luckily we produce a number of interactions required by injecting a language model into the process This way we can use the same task capabilities from the previous designers review example to delegate work to a language model The model analyze provide documentation as before Read our threat modeling guidelines, and you'll hear me mention guidelines a lot here If you produce a good documentation internally today, you can easily benefit from using language models So that's another reason to like have your junior folks produce documentation because you can feed that back into a model It makes everyone's life easier on the line, right? But yeah, I mentioned that it goes to the guidelines It will highlight the risks in the proposed system and come up with some mitigations before submitting them to the security team for review And in this example, the model actually takes care of most of the heavy lifting I'm reducing the burden of your security team by order of magnitude But to take things a step further we can automate the process a bit more By providing the model Guidelines this is like a retrieval type of


chunk_text:
  in threat modeling and design review But luckily we produce a number of interactions required by injecting a language model into the process This way we can use the same task capabilities from the previous designers review example to delegate work to a language model The model analyze provide documentation as before Read our threat modeling guidelines, and you'll hear me mention guidelines a lot here If you produce a good documentation internally today, you can easily benefit from using language models So that's another reason to like have your junior folks produce documentation because you can feed that back into a model It makes everyone's life easier on the line, right? But yeah, I mentioned that it goes to the guidelines It will highlight the risks in the proposed system and come up with some mitigations before submitting them to the security team for review And in this example, the model actually takes care of most of the heavy lifting I'm reducing the burden of your security team by order of magnitude But to take things a step further we can automate the process a bit more By providing the model Guidelines this is like a retrieval type of setup The model will use this documentation to follow your best practices and rely on less indirect feedback from your security team So you're going from a few interactions with your security in the model to one or two While still remaining accurate Summarize the use case here We can reduce number of interactions between teams automate the majority of document diagram analysis. That's a big one for some people Just a note on diagram analysis If you currently do threat models with just like images of your system You might want to stop doing that use mermaid JS markdown to describe your document because then the model can treat your design as text and still understand the flow of Design making it a lot easier to actually slot these things in so again I'll harp on this a lot But good data in gives you good results out and if you don't do that data work up front You're gonna have a world of hurt, right? All right So we've got some mitigations our model here, but let's touch on another aspect of AppSec responsibility the dreaded code review. I have to do a number of these per week


chunk_text:
  setup The model will use this documentation to follow your best practices and rely on less indirect feedback from your security team So you're going from a few interactions with your security in the model to one or two While still remaining accurate Summarize the use case here We can reduce number of interactions between teams automate the majority of document diagram analysis. That's a big one for some people Just a note on diagram analysis If you currently do threat models with just like images of your system You might want to stop doing that use mermaid JS markdown to describe your document because then the model can treat your design as text and still understand the flow of Design making it a lot easier to actually slot these things in so again I'll harp on this a lot But good data in gives you good results out and if you don't do that data work up front You're gonna have a world of hurt, right? All right So we've got some mitigations our model here, but let's touch on another aspect of AppSec responsibility the dreaded code review. I have to do a number of these per week and Honestly, this is the whole reason I put this together We you know We have developers who won't see you for weeks and then all of a sudden you have a PR with thousands of file changes, right? It's on a code review Like I mentioned this one isn't too crazy, but You know merge after Wednesday. We get this on a Tuesday or something and 24 files have changed 758 lines of code have been added. I don't know about you, but I'm not gonna read that dude. I'm just gonna be like hey Looks good to me stamp it let the phone Management team deal with it. Let the security operations team deal with it, but obviously not the best way to actually design software, right? Yeah Just like previous examples look at the existing way we do code review and then see how we can slide a model to help us out So today Yeah, we're gonna assume where the developer first creates a new branch repository this isn't always the case They could also take over for another developer which introduces other issues But next they commit some code possibly hundreds of times and eventually


chunk_text:
  and Honestly, this is the whole reason I put this together We you know We have developers who won't see you for weeks and then all of a sudden you have a PR with thousands of file changes, right? It's on a code review Like I mentioned this one isn't too crazy, but You know merge after Wednesday. We get this on a Tuesday or something and 24 files have changed 758 lines of code have been added. I don't know about you, but I'm not gonna read that dude. I'm just gonna be like hey Looks good to me stamp it let the phone Management team deal with it. Let the security operations team deal with it, but obviously not the best way to actually design software, right? Yeah Just like previous examples look at the existing way we do code review and then see how we can slide a model to help us out So today Yeah, we're gonna assume where the developer first creates a new branch repository this isn't always the case They could also take over for another developer which introduces other issues But next they commit some code possibly hundreds of times and eventually open the pull request Either as a draft or when that's ready for review Some CI checks occur stack analysis workflows running against the PR the dev team notifies some reviewers usually within their own team and Sometimes include the security team and I just want to know the development teams might have a good idea of when to include security and a lot of that's actually our Fault because when we put security checks directly in repos they assume that hey security checks ran on my my code I shouldn't need security to come review it But as we all know Security checks on your repo aren't enough. We actually give through manual code review But let's continue the assumption that they did ask us to review and by this point We're often taken by surprise feels kind of like a pop. It's like hey surprise. You have a huge PR to review Do you know what to do here right like if you've been seeing this code base a lot of us within security aren't actually Developers and that disconnect is like one of the issues that causes friction during code review or missed items I'm needless to


chunk_text:
  open the pull request Either as a draft or when that's ready for review Some CI checks occur stack analysis workflows running against the PR the dev team notifies some reviewers usually within their own team and Sometimes include the security team and I just want to know the development teams might have a good idea of when to include security and a lot of that's actually our Fault because when we put security checks directly in repos they assume that hey security checks ran on my my code I shouldn't need security to come review it But as we all know Security checks on your repo aren't enough. We actually give through manual code review But let's continue the assumption that they did ask us to review and by this point We're often taken by surprise feels kind of like a pop. It's like hey surprise. You have a huge PR to review Do you know what to do here right like if you've been seeing this code base a lot of us within security aren't actually Developers and that disconnect is like one of the issues that causes friction during code review or missed items I'm needless to say the huge number of changes can put a massive strain on the team So let's look at how we can do code review in an AI assisted way You'll notice from the flows here without going to too much detail So I'm sure you can't see the actual wording The majority of the interactions now occurring between the developer and the language model at a high level and security is almost uninvolved at this point, right and You know do the fact that these recent language models are excellent both generating and evaluating code coupled with the basic capabilities of generation summarization Is now possible to improve the speed and accuracy of code review by providing a model context from development process such as commit messages File diffs a PR description. So again another reason to have good data Force your devs to put descriptions on PRS So a model can read the description and you don't have to go back and say what does this PR actually do? We can potentially Reduce the evaluation time to seconds instead of minutes or days and actually keep pace with developers which makes them happier It makes us happier In the


chunk_text:
  say the huge number of changes can put a massive strain on the team So let's look at how we can do code review in an AI assisted way You'll notice from the flows here without going to too much detail So I'm sure you can't see the actual wording The majority of the interactions now occurring between the developer and the language model at a high level and security is almost uninvolved at this point, right and You know do the fact that these recent language models are excellent both generating and evaluating code coupled with the basic capabilities of generation summarization Is now possible to improve the speed and accuracy of code review by providing a model context from development process such as commit messages File diffs a PR description. So again another reason to have good data Force your devs to put descriptions on PRS So a model can read the description and you don't have to go back and say what does this PR actually do? We can potentially Reduce the evaluation time to seconds instead of minutes or days and actually keep pace with developers which makes them happier It makes us happier In the new AI assisted version of code review Like I mentioned the language model took care of most of the work going back and forth reading all the docs commenting on things And also keeping track of the commit messages diffs and more All right So the more context you have in your PRS the higher quality of data is get the better result at the other end Just to summarize rapidly analyze massive amounts of code they were designed to do this they were trained to do this Preside provided consistent view quality like if you have a junior engineer review a PR you have a senior engineer You typically have different results so this gives you a consistent baseline that you can also use to improve the quality of your more junior engineers and Hold your more senior engineers accountable since we all slip every now and then the other nice thing here is you can scale to handle the load of developers just spitting out PRS right like we've all been at the end of the year whenever we have to launch new software and You know teams are asking for reviews. We don't want to block their


chunk_text:
  new AI assisted version of code review Like I mentioned the language model took care of most of the work going back and forth reading all the docs commenting on things And also keeping track of the commit messages diffs and more All right So the more context you have in your PRS the higher quality of data is get the better result at the other end Just to summarize rapidly analyze massive amounts of code they were designed to do this they were trained to do this Preside provided consistent view quality like if you have a junior engineer review a PR you have a senior engineer You typically have different results so this gives you a consistent baseline that you can also use to improve the quality of your more junior engineers and Hold your more senior engineers accountable since we all slip every now and then the other nice thing here is you can scale to handle the load of developers just spitting out PRS right like we've all been at the end of the year whenever we have to launch new software and You know teams are asking for reviews. We don't want to block their actual development process. We never want to be blockers and security We'll get that bad reputation. So this allows us to scale the process without compromising on quality And you know up until now I haven't touched on any of the nuance of actually building LM apps But the shed a little bit more light on this I'll break down the code review app a little further and provide a recipe for building a similar solution The next few slides will cover our builder requirements which model to use for certain tasks Prerequisites and a recipe for applying the design to your own environment So first let's start with design requirements We want this to integrate into existing development workflows We don't want to take developers out of what they're used to doing and make them do something completely different, right? They're either not gonna do it or not gonna enjoy We also want to make sure that produces a result at least as quickly as a human reviewer if the process takes longer Why are we doing it? Right? So a lot of people will slot in language models and Not really understand why they're sliding them in other


chunk_text:
  actual development process. We never want to be blockers and security We'll get that bad reputation. So this allows us to scale the process without compromising on quality And you know up until now I haven't touched on any of the nuance of actually building LM apps But the shed a little bit more light on this I'll break down the code review app a little further and provide a recipe for building a similar solution The next few slides will cover our builder requirements which model to use for certain tasks Prerequisites and a recipe for applying the design to your own environment So first let's start with design requirements We want this to integrate into existing development workflows We don't want to take developers out of what they're used to doing and make them do something completely different, right? They're either not gonna do it or not gonna enjoy We also want to make sure that produces a result at least as quickly as a human reviewer if the process takes longer Why are we doing it? Right? So a lot of people will slot in language models and Not really understand why they're sliding them in other to other than just use AI for the sake of AI We don't want to do that here. We want to make sure we're getting an actual benefit out of this integration Additionally, we want to maintain data privacy and security by using open source models Such that we're not leaking our code to third-party model providers such as open AI and others and We want to also obviously for security team's sake be able to audit the decisions made by our models So keeping some log of the interaction that occurred between the model and the connected systems as well as looking at For example, if we have a review that came through the human go back and that result or is it unchecked, right? So we know where the fault is So all of these requirements are actually sourced from both development team and the security team like kind of actively in In the ecosystem right now You can kind of tell which items are developers first versus ones are security But we need both that we need all of these requirements to actually make it a workable solution So next let's get to level mapping our code


chunk_text:
  to other than just use AI for the sake of AI We don't want to do that here. We want to make sure we're getting an actual benefit out of this integration Additionally, we want to maintain data privacy and security by using open source models Such that we're not leaking our code to third-party model providers such as open AI and others and We want to also obviously for security team's sake be able to audit the decisions made by our models So keeping some log of the interaction that occurred between the model and the connected systems as well as looking at For example, if we have a review that came through the human go back and that result or is it unchecked, right? So we know where the fault is So all of these requirements are actually sourced from both development team and the security team like kind of actively in In the ecosystem right now You can kind of tell which items are developers first versus ones are security But we need both that we need all of these requirements to actually make it a workable solution So next let's get to level mapping our code review application to see exactly what we did So next let's get to level mapping our code review application to see exactly what we need to put together for it to work For Due to the fact that it takes in, you know file diffs commit messages and status checks. We actually need extensive short-term memory The extensive short-term memory and large input size are needed for within our application to deal again with the diffs But this necessitates also an internal index to store The diffs from these files the commit messages and everything else in one record So when we pull it into our model, it's able to reason over the entire PR and have to do several retrievals behind the scenes Right, but that's only possible when you have a large enough context window to actually take in all that content Additionally the fact that the PR commits can occur asynchronously they don't all happen at once it can be over many days Um, but the order doesn't matter we need to treat these as a event stream or conversation So that maps very well to the original level two design where we have a chat type


chunk_text:
  review application to see exactly what we did So next let's get to level mapping our code review application to see exactly what we need to put together for it to work For Due to the fact that it takes in, you know file diffs commit messages and status checks. We actually need extensive short-term memory The extensive short-term memory and large input size are needed for within our application to deal again with the diffs But this necessitates also an internal index to store The diffs from these files the commit messages and everything else in one record So when we pull it into our model, it's able to reason over the entire PR and have to do several retrievals behind the scenes Right, but that's only possible when you have a large enough context window to actually take in all that content Additionally the fact that the PR commits can occur asynchronously they don't all happen at once it can be over many days Um, but the order doesn't matter we need to treat these as a event stream or conversation So that maps very well to the original level two design where we have a chat type application But in this case the chat isn't user facing. It's system facing basically the the chat is between the model and your github or github or Gitlab or whatever, right? So that event stream becomes your chat Building on top of that and given the requirements Um, we basically need a level three model at a level three app at minimum because we have a retrieval section we need to do and we also need a section for Interacting the model back and forth and maintaining that that session stream that conversation stream In this case, you can consider the session id The repo and the PR number so this is how you're going to break up What conversation is occurring and be able to manage a number of these at once, right? So we can scale with that Um Next one again to how we choose a model um Since we know our high level requirements and that require a sizable input window And we're dealing primarily with code. I had basically two options for language models to use for this Mosaic mls mpt7b with an 8k context Or hugging faces zephyr 7b, which is based on the


chunk_text:
  application But in this case the chat isn't user facing. It's system facing basically the the chat is between the model and your github or github or Gitlab or whatever, right? So that event stream becomes your chat Building on top of that and given the requirements Um, we basically need a level three model at a level three app at minimum because we have a retrieval section we need to do and we also need a section for Interacting the model back and forth and maintaining that that session stream that conversation stream In this case, you can consider the session id The repo and the PR number so this is how you're going to break up What conversation is occurring and be able to manage a number of these at once, right? So we can scale with that Um Next one again to how we choose a model um Since we know our high level requirements and that require a sizable input window And we're dealing primarily with code. I had basically two options for language models to use for this Mosaic mls mpt7b with an 8k context Or hugging faces zephyr 7b, which is based on the recently released mistral 7b model I just want to note like i've had to rewrite this talk like 13 times do the pace of changes In the ai industry, but this is currently my top two for actually deploying this stuff internally So our chosen model should be somewhat obvious I hope Was mpt7b And this was released back in march and the mistral model was released this month but what I found through testing was that the mosaic model was Generally better all around and excelled at code even though it was only okay at retrieval tasks the 8k context window made it Amazing for giving it a prompt tuned Instructions in this case prompt tuning is in between Your prompt engineering and your classic fine-tuning where you have to fully retrain the model instead you give it all the instructions you can Using up a good portion of that 8k context to say do this thing That way you don't have to go back and retrain the model and that allows you to dynamically update how your models behave without spending hundreds of thousands of dollars retraining or you know hours and hours and Getting the resources to


chunk_text:
  recently released mistral 7b model I just want to note like i've had to rewrite this talk like 13 times do the pace of changes In the ai industry, but this is currently my top two for actually deploying this stuff internally So our chosen model should be somewhat obvious I hope Was mpt7b And this was released back in march and the mistral model was released this month but what I found through testing was that the mosaic model was Generally better all around and excelled at code even though it was only okay at retrieval tasks the 8k context window made it Amazing for giving it a prompt tuned Instructions in this case prompt tuning is in between Your prompt engineering and your classic fine-tuning where you have to fully retrain the model instead you give it all the instructions you can Using up a good portion of that 8k context to say do this thing That way you don't have to go back and retrain the model and that allows you to dynamically update how your models behave without spending hundreds of thousands of dollars retraining or you know hours and hours and Getting the resources to do this I just want to note that in terms of model sizing to train a model that takes up 13 gigs of video memory usually requires an order of magnitude higher than that and I don't know about you, but I can't find a video card that has 140 gigabytes of memory at least not in my budget, right so Using ones with large context despite their small size can give you a nice middle ground to actually achieve a good results All right, so before I actually hop over to the recipe here or the ingredients I should say One more thing I want to mention about These two models. These are the ones that are commercially available for people to use outside of just hobby type usage So if you want to build a product about you actually can question A great question, how do I measure them? So there are benchmarks have been released But a lot of these benchmarks are subjective because they weren't based on security use cases They're based on general language use cases. So I measure these through actually using them over the past say like nine or ten months internally, right


chunk_text:
  do this I just want to note that in terms of model sizing to train a model that takes up 13 gigs of video memory usually requires an order of magnitude higher than that and I don't know about you, but I can't find a video card that has 140 gigabytes of memory at least not in my budget, right so Using ones with large context despite their small size can give you a nice middle ground to actually achieve a good results All right, so before I actually hop over to the recipe here or the ingredients I should say One more thing I want to mention about These two models. These are the ones that are commercially available for people to use outside of just hobby type usage So if you want to build a product about you actually can question A great question, how do I measure them? So there are benchmarks have been released But a lot of these benchmarks are subjective because they weren't based on security use cases They're based on general language use cases. So I measure these through actually using them over the past say like nine or ten months internally, right and so how often do they give us the result that we didn't want in a review versus the one that actually our security team would Have used and we take those examples back out and put into that large system prompt Like I mentioned such a nudge in the right direction. Thanks for the question But yeah moving on to the ingredients the other parts of the system We will need other than the model since you consider the model the core, but we're gonna build a level three application around it the ingredients are again our small language model a server to host this a CI webhook host this can be serverless anyone who's done any kind of GitHub actions or CI automation should be familiar with that So I won't really dive in there I'm an orchestration host it's like a workflow manager such as AWS step functions and 8n or temporal a Retrieval index host you can go a couple different directions here Either using a vector database, which is a bit more complex or Elasticsearch, which I really like to use in this case. It's simple to deploy and performs amazingly well when paired with


chunk_text:
  and so how often do they give us the result that we didn't want in a review versus the one that actually our security team would Have used and we take those examples back out and put into that large system prompt Like I mentioned such a nudge in the right direction. Thanks for the question But yeah moving on to the ingredients the other parts of the system We will need other than the model since you consider the model the core, but we're gonna build a level three application around it the ingredients are again our small language model a server to host this a CI webhook host this can be serverless anyone who's done any kind of GitHub actions or CI automation should be familiar with that So I won't really dive in there I'm an orchestration host it's like a workflow manager such as AWS step functions and 8n or temporal a Retrieval index host you can go a couple different directions here Either using a vector database, which is a bit more complex or Elasticsearch, which I really like to use in this case. It's simple to deploy and performs amazingly well when paired with the model but here you can also use things like Big query and other available databases in its place next we have Our system prompts templates that we use to do basic code review filed it for a file diff guidelines and status check guidelines So the rules we give models when interacting with certain parts of our actual system Next we have our model configuration, so how do you want to set this model up to actually work within your application? Like I mentioned before the NPT 7 model is an 8k context But I don't always use all 8k of that context and the reason is when you set it that high The model has to always consider that volume of information and actually performance more slowly So the smaller you can get that context window set you'll have faster tokens per second or faster response rate back from the model I Typically allocate the token capacity in like a 4 to 2 ratio So like half of the total context is for system prompts a quarter of that is for the user input And then the last quarter is for some Usage specific additional context so if


chunk_text:
  the model but here you can also use things like Big query and other available databases in its place next we have Our system prompts templates that we use to do basic code review filed it for a file diff guidelines and status check guidelines So the rules we give models when interacting with certain parts of our actual system Next we have our model configuration, so how do you want to set this model up to actually work within your application? Like I mentioned before the NPT 7 model is an 8k context But I don't always use all 8k of that context and the reason is when you set it that high The model has to always consider that volume of information and actually performance more slowly So the smaller you can get that context window set you'll have faster tokens per second or faster response rate back from the model I Typically allocate the token capacity in like a 4 to 2 ratio So like half of the total context is for system prompts a quarter of that is for the user input And then the last quarter is for some Usage specific additional context so if you're doing a filed if you might say this is a file diff And you know we have this criteria kind of going along with it We're in the main system context you have you are a bot that does Threat modeling or code review or whatever else like the cornerstone being that the the half of the total context Next I want to touch on temperature. This is a setting within these models that basically adjusts. How variable the output will be and Ideally should be set as low as possible for production use cases because you don't want it going off and like writing you poetry, right? You want this thing to be as on the rails as possible if we're going to use it in production So set temp to 0.1. This works really well the NPT 7 model not as well with certain others because they do have a Finer grade of what that temperature looks like Time okay All right, so they get the best results of the model we chose we need to design our inference system to actually can handle these requirements, right? The NPT 7 models and 7 billion parameters in general


chunk_text:
  you're doing a filed if you might say this is a file diff And you know we have this criteria kind of going along with it We're in the main system context you have you are a bot that does Threat modeling or code review or whatever else like the cornerstone being that the the half of the total context Next I want to touch on temperature. This is a setting within these models that basically adjusts. How variable the output will be and Ideally should be set as low as possible for production use cases because you don't want it going off and like writing you poetry, right? You want this thing to be as on the rails as possible if we're going to use it in production So set temp to 0.1. This works really well the NPT 7 model not as well with certain others because they do have a Finer grade of what that temperature looks like Time okay All right, so they get the best results of the model we chose we need to design our inference system to actually can handle these requirements, right? The NPT 7 models and 7 billion parameters in general consume 13 gigabytes on disk 13 gigabytes of system memory and up to 20 gigabytes of GPU memory inference time and Do the way these models are loaded first being copied it to disk and then to system memory and then into GPU memory It's actually necessary to have a server that has more System memory that does GPU memory This is like a major pitfall of a lot of these designs people try to roll this out Because most of the cloud offerings don't give you that split because they were focused on a previous generation of machine learning where you didn't need this massive amount of system memory to do that initial load, right and just for some additional guidance here like reference a 30 billion parameter model takes up 60 gigs a 70 billion parameter model takes up 140 gigs and That's just on disk and in memory when they're loaded when they're running they take up about a third more than that Right. So again, one of the reasons I trended towards smaller language models There literally aren't systems that we can use to actually run them. Otherwise, right once you go above say 30 billion


chunk_text:
  consume 13 gigabytes on disk 13 gigabytes of system memory and up to 20 gigabytes of GPU memory inference time and Do the way these models are loaded first being copied it to disk and then to system memory and then into GPU memory It's actually necessary to have a server that has more System memory that does GPU memory This is like a major pitfall of a lot of these designs people try to roll this out Because most of the cloud offerings don't give you that split because they were focused on a previous generation of machine learning where you didn't need this massive amount of system memory to do that initial load, right and just for some additional guidance here like reference a 30 billion parameter model takes up 60 gigs a 70 billion parameter model takes up 140 gigs and That's just on disk and in memory when they're loaded when they're running they take up about a third more than that Right. So again, one of the reasons I trended towards smaller language models There literally aren't systems that we can use to actually run them. Otherwise, right once you go above say 30 billion parameters Okay, so I think I want to touch on here is the retrieval index system this is elastic search you typically have these in clusters Here you don't need too much too many resources because you're really just dealing with text It's not like a logging or scene platform where you have gigs and gigs of text But you still need enough headroom such that's performant and here I use basically a smallest offering for cores 8 gigabytes and 40 gigs of storage per node because actually we don't need to retain any of the file diffs or PR context for a long period of time They life cycle out Cool so gave some core components of the actual design. We have our interface layer, which is a Gradio This is an experiment interface Now owned by the team at hug base and basically allows you to wrap any language model in Three line in three lines of code and actually serve that as both a web app with a GUI and an API Which is really convenient Next we have some core libraries. You're gonna want to be familiar with PyTorch being the most Necessary here. I just


chunk_text:
  parameters Okay, so I think I want to touch on here is the retrieval index system this is elastic search you typically have these in clusters Here you don't need too much too many resources because you're really just dealing with text It's not like a logging or scene platform where you have gigs and gigs of text But you still need enough headroom such that's performant and here I use basically a smallest offering for cores 8 gigabytes and 40 gigs of storage per node because actually we don't need to retain any of the file diffs or PR context for a long period of time They life cycle out Cool so gave some core components of the actual design. We have our interface layer, which is a Gradio This is an experiment interface Now owned by the team at hug base and basically allows you to wrap any language model in Three line in three lines of code and actually serve that as both a web app with a GUI and an API Which is really convenient Next we have some core libraries. You're gonna want to be familiar with PyTorch being the most Necessary here. I just want to mention that there are two current branches of PyTorch version 1 and version 2 Version 2 it has been heavily optimized, but most projects are pinned at version 1 Leveraging version 2 typically gives you a 33% speed up Without using any more memory or anything like that and also allows you to in certain circumstances directly load Models from disk into GPU memory bypassing the system memory, right? but this is only true when you couple it with safe tensors, which are an optimized version of the Models Core files like they're exported different formats such. There's no executable code No risk of having malware run in your models actual File and just keeping in mind that these models are actually pickled Python code so you can have Model malware. It's a thing You save tensors Next here is hugging face transformers a few of you are familiar with this It's probably de facto great thing with my natural language processing and Also, it can give you you know a model interface in three lines of code really really easy implement Finalist one touches some general system things Python projects have been around forever. We a lot


chunk_text:
  want to mention that there are two current branches of PyTorch version 1 and version 2 Version 2 it has been heavily optimized, but most projects are pinned at version 1 Leveraging version 2 typically gives you a 33% speed up Without using any more memory or anything like that and also allows you to in certain circumstances directly load Models from disk into GPU memory bypassing the system memory, right? but this is only true when you couple it with safe tensors, which are an optimized version of the Models Core files like they're exported different formats such. There's no executable code No risk of having malware run in your models actual File and just keeping in mind that these models are actually pickled Python code so you can have Model malware. It's a thing You save tensors Next here is hugging face transformers a few of you are familiar with this It's probably de facto great thing with my natural language processing and Also, it can give you you know a model interface in three lines of code really really easy implement Finalist one touches some general system things Python projects have been around forever. We a lot of people use virtual environments. I really like how compostable poetry makes design these projects I mean you get to include every system libraries Pin versions on model specific things and basically have something that's repeatable as you can run the container or as a Microservice if you want to go that route As the operating system that actually runs the app. I recommend lambda stack They basically roll in all of your dependencies around and video libraries Drivers everything into one clean nice package and it keeps itself updated over time so you don't to go back and say oh my pytorch implementation broke because my Driver version is now different than the version required by the actual library, right? So highly recommend that if you don't want headaches Yeah, let's all go on to the recipe real quick here make sure I'm leaving enough time for questions Yes, we install a lambda sack starting a poetry Python project We create inference pipeline using hugging face again. This is three lines of code like literally three lines of code We add llama index which I haven't really touched on just yet that allows you to chunk Documents into sizable


chunk_text:
  of people use virtual environments. I really like how compostable poetry makes design these projects I mean you get to include every system libraries Pin versions on model specific things and basically have something that's repeatable as you can run the container or as a Microservice if you want to go that route As the operating system that actually runs the app. I recommend lambda stack They basically roll in all of your dependencies around and video libraries Drivers everything into one clean nice package and it keeps itself updated over time so you don't to go back and say oh my pytorch implementation broke because my Driver version is now different than the version required by the actual library, right? So highly recommend that if you don't want headaches Yeah, let's all go on to the recipe real quick here make sure I'm leaving enough time for questions Yes, we install a lambda sack starting a poetry Python project We create inference pipeline using hugging face again. This is three lines of code like literally three lines of code We add llama index which I haven't really touched on just yet that allows you to chunk Documents into sizable pieces that model can digest and this is what powers a retrieval system Next we wrap the model in a radio interface to easily provide an API and a web GUI for either administrative purposes or You know troubleshooting But just a note on radio since this is something that's used pretty much ubiquitously And it does offer the predict an API endpoints a GUI and the API are in the same route So I really recommend putting a reverse proxy in front of that to only serve one or the other Otherwise, you can get devs basically playing with your models back end and you don't typically want that Finally we deploy a webhook your classic CI webhook to call this radio API when the new PR is ready Yeah, so again, like I mentioned this is a pretty simple API Yeah, so again, like I mentioned things have been changing pretty rapidly so next week I'll be providing a One code sample of this actual app, but more detailed descriptions around the actually building this recipe out and share the repo But that being said Let's dive into a little demo, right? Actually first here. Let me show


chunk_text:
  pieces that model can digest and this is what powers a retrieval system Next we wrap the model in a radio interface to easily provide an API and a web GUI for either administrative purposes or You know troubleshooting But just a note on radio since this is something that's used pretty much ubiquitously And it does offer the predict an API endpoints a GUI and the API are in the same route So I really recommend putting a reverse proxy in front of that to only serve one or the other Otherwise, you can get devs basically playing with your models back end and you don't typically want that Finally we deploy a webhook your classic CI webhook to call this radio API when the new PR is ready Yeah, so again, like I mentioned this is a pretty simple API Yeah, so again, like I mentioned things have been changing pretty rapidly so next week I'll be providing a One code sample of this actual app, but more detailed descriptions around the actually building this recipe out and share the repo But that being said Let's dive into a little demo, right? Actually first here. Let me show you guys this diagram. Hopefully this loads Come on Will it load? Yes, okay. So here's our new flow with everything kind of built in They're level 3 app. So I'll start I'll start from where the CI webhook kicks off so you guys can see the flow of that But the security webhook upside the CI webhook receives a call from the PR the code review begins based on the branch and the repo name Again repo name branch name Confirms it back to the app and then the app is what actually goes and retrieves any supporting documentation from in this case elasticsearch And at the same time sends over one of our system prompts With the criteria and guidelines as the the head of that actual prompt and says hey go review this PR the search is executed based on the PR number since we're actually taking in some of the Details around the PR. So our file diffs, commit messages, etc in our retrieval index These diffs are chunked. So the results are chunked back to the LLM app using LLAMA index and checked against neural network and Then that's returned back to the actual application


chunk_text:
  you guys this diagram. Hopefully this loads Come on Will it load? Yes, okay. So here's our new flow with everything kind of built in They're level 3 app. So I'll start I'll start from where the CI webhook kicks off so you guys can see the flow of that But the security webhook upside the CI webhook receives a call from the PR the code review begins based on the branch and the repo name Again repo name branch name Confirms it back to the app and then the app is what actually goes and retrieves any supporting documentation from in this case elasticsearch And at the same time sends over one of our system prompts With the criteria and guidelines as the the head of that actual prompt and says hey go review this PR the search is executed based on the PR number since we're actually taking in some of the Details around the PR. So our file diffs, commit messages, etc in our retrieval index These diffs are chunked. So the results are chunked back to the LLM app using LLAMA index and checked against neural network and Then that's returned back to the actual application as a short summary and verdict which then bumps it back to our security team for review So pretty straightforward like I mentioned I'll share a repo with the actual code for this as well as like a proper architecture diagram. It's a little easier to read And let me switch over here to show you guys the actual Front-end or administrative interface of this app. I'm kind of using that retrieval pattern Okay Same diagram there. So this is not an interactive example. It shows my prompt here I always want to tell it the source of the data because a lot of the times when that you look at the training data that Went to these models. They're structured like this. So if you're ever curious how to use a model properly Actually go read it's either the training code or see the data sets it was trained on because the examples are really guide the types of prompts you should make and use with this model and Additionally you can also know terms you shouldn't use in your prompts that would throw the model off base So I was looking at the tokens file for the model


chunk_text:
  as a short summary and verdict which then bumps it back to our security team for review So pretty straightforward like I mentioned I'll share a repo with the actual code for this as well as like a proper architecture diagram. It's a little easier to read And let me switch over here to show you guys the actual Front-end or administrative interface of this app. I'm kind of using that retrieval pattern Okay Same diagram there. So this is not an interactive example. It shows my prompt here I always want to tell it the source of the data because a lot of the times when that you look at the training data that Went to these models. They're structured like this. So if you're ever curious how to use a model properly Actually go read it's either the training code or see the data sets it was trained on because the examples are really guide the types of prompts you should make and use with this model and Additionally you can also know terms you shouldn't use in your prompts that would throw the model off base So I was looking at the tokens file for the model to see exactly What things are stop tokens special keywords, etc that you should avoid and those tend to come up in code naturally So it's something you want to know ahead of time So here we ask it to provide a separate of the actions taking this pull request and review for security concerns We have some guidelines the security team is only concerned with changes that impact authentication authorization or data security and then the output format should be a short summary with verdict pass fail a review which would get sent to your security team and As you see the data source here is one of my repos You can basically put that in on this interface and should go to the repo and actually pull back some context So I'm talking about a high level. I think I have time to show you the Semi interactive or that actually yeah, this is a real PR now internally that we had to review something goes across The public repo, so it's totally fine to show you guys this this is open source But there are some security concerns here and the model will go through and actually


chunk_text:
  to see exactly What things are stop tokens special keywords, etc that you should avoid and those tend to come up in code naturally So it's something you want to know ahead of time So here we ask it to provide a separate of the actions taking this pull request and review for security concerns We have some guidelines the security team is only concerned with changes that impact authentication authorization or data security and then the output format should be a short summary with verdict pass fail a review which would get sent to your security team and As you see the data source here is one of my repos You can basically put that in on this interface and should go to the repo and actually pull back some context So I'm talking about a high level. I think I have time to show you the Semi interactive or that actually yeah, this is a real PR now internally that we had to review something goes across The public repo, so it's totally fine to show you guys this this is open source But there are some security concerns here and the model will go through and actually use this PR Review if you look at how long this is actually taking I don't know about you, but it's faster than I am I've been doing this a while We'll actually go through and do the review summarize everything in the PR and give your result back So like I mentioned radio has a GUI and an API I'm showing you the GUI here to basically illustrate this But this would happen behind the scenes between the web the user and like the post back to your security team wherever they are with A slack or you know directly on the PR or wherever else Cool and that brings me to questions happy to show you guys an interactive live demo But I think I'll take a few questions since time is now running short Awesome You in the back there I Love how this is always the first question over there's like AI generated content. This is dolly free in this case. Yeah other questions Okay, so there are two ways to approach this right Your spend on open AI per year is one thing to look at I've seen people spend hundreds of thousand dollars a


chunk_text:
  use this PR Review if you look at how long this is actually taking I don't know about you, but it's faster than I am I've been doing this a while We'll actually go through and do the review summarize everything in the PR and give your result back So like I mentioned radio has a GUI and an API I'm showing you the GUI here to basically illustrate this But this would happen behind the scenes between the web the user and like the post back to your security team wherever they are with A slack or you know directly on the PR or wherever else Cool and that brings me to questions happy to show you guys an interactive live demo But I think I'll take a few questions since time is now running short Awesome You in the back there I Love how this is always the first question over there's like AI generated content. This is dolly free in this case. Yeah other questions Okay, so there are two ways to approach this right Your spend on open AI per year is one thing to look at I've seen people spend hundreds of thousand dollars a month to to run models or an inference through open AI, right? Cost is a big one so if you're when I mentioned the difference between The previous generation of cloud-based AI systems and the current jet and the current generation You find that you're unable to get enough system memory to satisfy your needs with the GPU memory Almost ever without paying about at minimum like 700 bucks a month, right something crazy But you spend 1,500 or 2,000 once and you have a system that can do inference for you at this scale For the rest of the your ownership of that device and since these models are open-source You can cache them modify them. You don't lose access to that. You also don't have open AI changing the model underneath your feet without you knowing right and It's probably too late to say it's at this point, but the views expressed here are not from any current previous or you know Future employer and they're also not the views of chat GPT, right? Yes, so I'll share that Basically next week because again like things change so quickly that I cannot share the raw code right now


chunk_text:
  month to to run models or an inference through open AI, right? Cost is a big one so if you're when I mentioned the difference between The previous generation of cloud-based AI systems and the current jet and the current generation You find that you're unable to get enough system memory to satisfy your needs with the GPU memory Almost ever without paying about at minimum like 700 bucks a month, right something crazy But you spend 1,500 or 2,000 once and you have a system that can do inference for you at this scale For the rest of the your ownership of that device and since these models are open-source You can cache them modify them. You don't lose access to that. You also don't have open AI changing the model underneath your feet without you knowing right and It's probably too late to say it's at this point, but the views expressed here are not from any current previous or you know Future employer and they're also not the views of chat GPT, right? Yes, so I'll share that Basically next week because again like things change so quickly that I cannot share the raw code right now There are like curse comments in that code, right? So some clean needs to happen, but this will all be shared detail. You plan to open source. Oh this absolutely Yeah, this is open. This is open sourceable. There's nothing here that isn't Something you can pick up and do today Like I mentioned three lines of code will get you to the point where you can do inference against these models The total retrieval system was about 100 lines of code And that's funny because I gave it Variance in terms of where I could pull context from like you probably saw on the slide before can read Google Wikipedia Gets pouring data from anywhere. I better understand something that your internal team might not well understand, right? Do you have any estimates on the time save versus the cost of actually running it? Rough estimates since we're pretty small team I would say that this is just being optimistic that I've saved about 80% of the time I would spend on design reviews or threat Models just through using this and getting that time back allows me to do research It's know my development teams better and also


chunk_text:
  There are like curse comments in that code, right? So some clean needs to happen, but this will all be shared detail. You plan to open source. Oh this absolutely Yeah, this is open. This is open sourceable. There's nothing here that isn't Something you can pick up and do today Like I mentioned three lines of code will get you to the point where you can do inference against these models The total retrieval system was about 100 lines of code And that's funny because I gave it Variance in terms of where I could pull context from like you probably saw on the slide before can read Google Wikipedia Gets pouring data from anywhere. I better understand something that your internal team might not well understand, right? Do you have any estimates on the time save versus the cost of actually running it? Rough estimates since we're pretty small team I would say that this is just being optimistic that I've saved about 80% of the time I would spend on design reviews or threat Models just through using this and getting that time back allows me to do research It's know my development teams better and also like improve the quality of these types of services Because I get the time to actually interact with human beings since the models are doing a lot of that heavy lifting You have to productize I definitely can't do that. But thank you What's that Hardware costs so as far as hardware costs, I recommend checking out lambda lambda labs to get some guidance on what Baseline hardware to build something like this looks like since this is always going to change due to GPU prices and you know Price is going up and down new boards being released, etc but I think like going through lambda labs looking at their desktop offering is a good way to kind of price out what the different system might look like and Keeping in mind the requirements grade that I mentioned before But you know, I think I kind of went overboard when I built mine So I don't recommend getting like 256 gigs of system memory and 12 terabytes of storage or anything like that but you can probably get away with spending about three grand on something like this using a consumer grade GPU like a RTX 3090 or something


chunk_text:
  like improve the quality of these types of services Because I get the time to actually interact with human beings since the models are doing a lot of that heavy lifting You have to productize I definitely can't do that. But thank you What's that Hardware costs so as far as hardware costs, I recommend checking out lambda lambda labs to get some guidance on what Baseline hardware to build something like this looks like since this is always going to change due to GPU prices and you know Price is going up and down new boards being released, etc but I think like going through lambda labs looking at their desktop offering is a good way to kind of price out what the different system might look like and Keeping in mind the requirements grade that I mentioned before But you know, I think I kind of went overboard when I built mine So I don't recommend getting like 256 gigs of system memory and 12 terabytes of storage or anything like that but you can probably get away with spending about three grand on something like this using a consumer grade GPU like a RTX 3090 or something like this Any more questions? So It's Realistically to build everything it took like a weekend, but to actually like build out the project It was many many months of iteration, right? so using all the different models that have come out seeing where they fail see if it makes more sense to spend the time to Truly fine-tune it or whether or not I can convince a model to behave a certain way by editing the conversation I had with it, which is prompt tuning in this case and nudge in the right direction Thank you Any other questions? Can you share all the models that you tried rather than just the top two? yeah, so there are about seven that I looked at and for the most part the gap between one and two and the rest was so Vast that I really only recommend using these two for open-source use cases like even with the base llama models or the base MPT model, I don't sorry like base Mistral model their lack of basic fine-tuning makes them very hard to use that thing in a lot of work and Especially when you have a limited context window


chunk_text:
  like this Any more questions? So It's Realistically to build everything it took like a weekend, but to actually like build out the project It was many many months of iteration, right? so using all the different models that have come out seeing where they fail see if it makes more sense to spend the time to Truly fine-tune it or whether or not I can convince a model to behave a certain way by editing the conversation I had with it, which is prompt tuning in this case and nudge in the right direction Thank you Any other questions? Can you share all the models that you tried rather than just the top two? yeah, so there are about seven that I looked at and for the most part the gap between one and two and the rest was so Vast that I really only recommend using these two for open-source use cases like even with the base llama models or the base MPT model, I don't sorry like base Mistral model their lack of basic fine-tuning makes them very hard to use that thing in a lot of work and Especially when you have a limited context window as many of these models do You're not able to actually even prompt to them in the right direction So that MPT 7 with 8k context is unique in that sense It's a small model that can take in a massive amount of input such that you can guide it in the right direction So these models were hosted on a physical machine, right? Because I kind of mentioned using a cloud offerings It tends to be pretty crazy, but you can obviously offload data storage to things like s3 and stream in The actual model files yourself and run them that way as opposed to storing them on a drive locally, and that's any other questions Yeah, so I mentioned workflow engines earlier, but I didn't really touch on their usage here But that's what the workflow engines come in every single step of that workflow Basically has an audit output as well as a lever that a dev can pull an engineer my team can pull to basically say Do one thing either the same or different and kind of make some adjustments Sometimes in real time like if you see that this is a file diff heavy


chunk_text:
  as many of these models do You're not able to actually even prompt to them in the right direction So that MPT 7 with 8k context is unique in that sense It's a small model that can take in a massive amount of input such that you can guide it in the right direction So these models were hosted on a physical machine, right? Because I kind of mentioned using a cloud offerings It tends to be pretty crazy, but you can obviously offload data storage to things like s3 and stream in The actual model files yourself and run them that way as opposed to storing them on a drive locally, and that's any other questions Yeah, so I mentioned workflow engines earlier, but I didn't really touch on their usage here But that's what the workflow engines come in every single step of that workflow Basically has an audit output as well as a lever that a dev can pull an engineer my team can pull to basically say Do one thing either the same or different and kind of make some adjustments Sometimes in real time like if you see that this is a file diff heavy PR versus a comment heavy PR You can actually switch the model downstream and say I actually want to use the Zephyr model from Hugging Face instead of the MPT model such that it handles a certain condition better and For future work I'll probably use some of these models in Parallel or in tandem because they're small enough to actually be run on some of these larger GPUs and have multiple models on the same GPU without any performance loss How small? Okay, so I think anything below 30 billion parameters is considered small But at some point basically next year anything we thought was large today will eventually be small anyway So long-term they're all gonna be small language models. I think assigning size to any kind of technology Is usually hubris as we tend to like go through Moore's law, right? But yeah, 7 billion parameters where I consider like small there is something like a million parameters But yeah, 7 billion parameters where I consider like small there are smaller ones, but they don't perform nearly as well There are things like mini LM and those are like 100 megs of disk space But they really


chunk_text:
  PR versus a comment heavy PR You can actually switch the model downstream and say I actually want to use the Zephyr model from Hugging Face instead of the MPT model such that it handles a certain condition better and For future work I'll probably use some of these models in Parallel or in tandem because they're small enough to actually be run on some of these larger GPUs and have multiple models on the same GPU without any performance loss How small? Okay, so I think anything below 30 billion parameters is considered small But at some point basically next year anything we thought was large today will eventually be small anyway So long-term they're all gonna be small language models. I think assigning size to any kind of technology Is usually hubris as we tend to like go through Moore's law, right? But yeah, 7 billion parameters where I consider like small there is something like a million parameters But yeah, 7 billion parameters where I consider like small there are smaller ones, but they don't perform nearly as well There are things like mini LM and those are like 100 megs of disk space But they really don't do the job well enough to actually be used in production You can see me later. Thank you You


chunk_text:
  { "webpage_url": "https://www.youtube.com/watch?v=612Pi_yk3s8", "title": "(KEYNOTE) Crazy Incentives And How They Drive Security Into No Man's Land by Dr. Christian Folini", "description": "", "channel_url": "https://www.youtube.com/channel/UCGdJZb8OWZPrRK8PuEeoZuA", "duration": 3549, "channel": "scrt.insomnihack", "uploader": "scrt.insomnihack", "upload_date": "20230411" } This text was transcribed using whisper model: large-v2 Thank you very much, Alain. It's nice to be back, actually. Welcome, Lausanne. Good. I'm going to talk about crazy incentives and how they drive security in no man's land. What do I mean by that? I mean, incentives are very often misaligned with security. So we start working away, we struggle, we invest time and resources, and instead of going forward, we're going backwards or in the wrong direction. So despite hard work, we're not getting security in return. And when I started to think about this concept, I realized very often it's incentives that drive us in the wrong direction. We're not aware of the incentives of people working with us. We're not aware of the incentives of vendors selling us their products. We're not even aware of our own incentives. So what I'm going to do is walk you through a couple of examples. Let's dive right in. This is from a video stream from Naomi Wu, a real sexy cyborg on Twitter. I think she walked around in Gansu or Shanghai in January or February and posted the video. And people went crazy immediately because what is happening here? What do you see? You see mostly young women flocking to an undercrossing to do streaming. At first, this looks like a sweatshop. However, they're all volunteers. They want to do this. They want to be there. They want to stream there. Why is that? It's not the beauty of the scenery. It's also not the good company. What is happening is they have incentives to gain the location feature of their streaming software. They want to stream from a certain location, from a good neighborhood, instead of being outside town where nobody would follow them. They want to stream from exactly this position. That's why in the night, in the evening, they flock together in an undercrossing. They have an incentive to gain the location feature, and the result is your PM being puzzled, going crazy over the video. But this exploded so quickly. What we see here is incentives at play that have very funny results. As I said, I'm Christian Folini. I'm a


chunk_text:
  Wu, a real sexy cyborg on Twitter. I think she walked around in Gansu or Shanghai in January or February and posted the video. And people went crazy immediately because what is happening here? What do you see? You see mostly young women flocking to an undercrossing to do streaming. At first, this looks like a sweatshop. However, they're all volunteers. They want to do this. They want to be there. They want to stream there. Why is that? It's not the beauty of the scenery. It's also not the good company. What is happening is they have incentives to gain the location feature of their streaming software. They want to stream from a certain location, from a good neighborhood, instead of being outside town where nobody would follow them. They want to stream from exactly this position. That's why in the night, in the evening, they flock together in an undercrossing. They have an incentive to gain the location feature, and the result is your PM being puzzled, going crazy over the video. But this exploded so quickly. What we see here is incentives at play that have very funny results. As I said, I'm Christian Folini. I'm a Swiss security engineer. I have a background in medieval history. This is what I did before I turned to security. This is where I get my helmets from. I'm a co-leader of the OWASP ModSecurity Coralset, a web application firewall project. I'm going to talk about that as well. I happen to be the program chair of the partnering conference Swiss Cyberstorm happening in October. You see you have a great company conference at the end of the winter, and you have another great conference in Bern in October. Please join us in Bern on October for Swiss Cyberstorm. Apparently, I'm a wearer of many helmets. I'm also working in e-voting security. I talked about that at a previous keynote. Now I'm going to talk about incentives. I'm taking this off because it's a bit heavy in the end. Incentives. First, what is an incentive? I turned to Wikipedia. What else would I do? Wikipedia tells me, in general, incentives are anything that persuades a person to alter their behavior. I guess we can agree on that. If we continue with the definition a bit, we get to higher incentives amount to greater levels of effort, and therefore higher levels


chunk_text:
  Swiss security engineer. I have a background in medieval history. This is what I did before I turned to security. This is where I get my helmets from. I'm a co-leader of the OWASP ModSecurity Coralset, a web application firewall project. I'm going to talk about that as well. I happen to be the program chair of the partnering conference Swiss Cyberstorm happening in October. You see you have a great company conference at the end of the winter, and you have another great conference in Bern in October. Please join us in Bern on October for Swiss Cyberstorm. Apparently, I'm a wearer of many helmets. I'm also working in e-voting security. I talked about that at a previous keynote. Now I'm going to talk about incentives. I'm taking this off because it's a bit heavy in the end. Incentives. First, what is an incentive? I turned to Wikipedia. What else would I do? Wikipedia tells me, in general, incentives are anything that persuades a person to alter their behavior. I guess we can agree on that. If we continue with the definition a bit, we get to higher incentives amount to greater levels of effort, and therefore higher levels of performance. Streaming in the Undercrossing, you get this. You make an effort to drive there, maybe to freeze in the winter for higher levels of performance. That's the view count. Performance already points to economics. Incentives is an economic term. It's definitely not a cybersecurity term. This is not a term that our industry coined. Incentives is an economic. It's a business term. This already shows you we have different incentives here. We have economic incentives, and we want to have security incentives. The two are absolutely not aligned with one another. Obviously, if they're misaligned, higher performance means better sales for the company selling you the snake oil. These are stupid tasks that we execute, and they result in higher level of friction. More work for us, but not security as a result. Ultimately, we just heat the climate by running more jobs on our computer without much results in terms of security. Think antivirus running on a Windows desktop. I'm not even saying that this is not necessary. It's a lot of investment of electricity and CPU cycles for relatively little gain just to keep things up. Optimal incentives mean that we get a maximum of security


chunk_text:
  of performance. Streaming in the Undercrossing, you get this. You make an effort to drive there, maybe to freeze in the winter for higher levels of performance. That's the view count. Performance already points to economics. Incentives is an economic term. It's definitely not a cybersecurity term. This is not a term that our industry coined. Incentives is an economic. It's a business term. This already shows you we have different incentives here. We have economic incentives, and we want to have security incentives. The two are absolutely not aligned with one another. Obviously, if they're misaligned, higher performance means better sales for the company selling you the snake oil. These are stupid tasks that we execute, and they result in higher level of friction. More work for us, but not security as a result. Ultimately, we just heat the climate by running more jobs on our computer without much results in terms of security. Think antivirus running on a Windows desktop. I'm not even saying that this is not necessary. It's a lot of investment of electricity and CPU cycles for relatively little gain just to keep things up. Optimal incentives mean that we get a maximum of security for a minimal investment of resources. This is what we actually would like to get. Ideally, we don't have friction. That means we have incentives pointing over there. We follow these incentives, and we get the same amount of security that we expected. If you follow bad incentives that lead you in a different direction, and you're a security engineer working in a blue team, so in defense setup, then you're not doing your job properly. Maybe without even being aware, you just follow incentives. You're not aware that the incentives are actually leading you away from security. If you implement the wrong incentives that fail to support the security of your users, then shame on you because you're placing incentives so people are getting bad security instead of what you're supposed to give them as a defense engineer. Then shame on you. Twitter announced a month ago that they would stop doing text message two-factor authentication for non-paying users. Whoever tried to convince their mother-in-law like I did or somebody else to enable two-factor authentication will realize that text message two-factor authentication is a weak form of two-factor authentication, but at least it is a little bit of security.


chunk_text:
  for a minimal investment of resources. This is what we actually would like to get. Ideally, we don't have friction. That means we have incentives pointing over there. We follow these incentives, and we get the same amount of security that we expected. If you follow bad incentives that lead you in a different direction, and you're a security engineer working in a blue team, so in defense setup, then you're not doing your job properly. Maybe without even being aware, you just follow incentives. You're not aware that the incentives are actually leading you away from security. If you implement the wrong incentives that fail to support the security of your users, then shame on you because you're placing incentives so people are getting bad security instead of what you're supposed to give them as a defense engineer. Then shame on you. Twitter announced a month ago that they would stop doing text message two-factor authentication for non-paying users. Whoever tried to convince their mother-in-law like I did or somebody else to enable two-factor authentication will realize that text message two-factor authentication is a weak form of two-factor authentication, but at least it is a little bit of security. So they have a relatively small, often people have relatively small security interest, small incentive for security, but text message you can kind of convince them if you make an effort and they get a bit of security out of this. And Twitter or Elon scrambling for money is now stopping that. What Elon is actually doing, he's doing A-B testing on the strength of the security desire of his users. He presents them with a choice. If you want security, you pay me. And if you want to save money, then you lose security. Or you upgrade to the Google Authenticator or other Authenticator apps. And that is simply too complicated for a lot of people. And I didn't manage to explain this to my mother-in-law. So Elon absolutely made a choice by placing incentives in front of his users, asking them to choose security or money. This was enacted this week, so we don't have the number. Actually, I'm also not sure we're getting to get the numbers. However, it's perfectly clear that if you want to save money, you're going to drop security. And if you want the security, that means more money for Twitter, but I


chunk_text:
  So they have a relatively small, often people have relatively small security interest, small incentive for security, but text message you can kind of convince them if you make an effort and they get a bit of security out of this. And Twitter or Elon scrambling for money is now stopping that. What Elon is actually doing, he's doing A-B testing on the strength of the security desire of his users. He presents them with a choice. If you want security, you pay me. And if you want to save money, then you lose security. Or you upgrade to the Google Authenticator or other Authenticator apps. And that is simply too complicated for a lot of people. And I didn't manage to explain this to my mother-in-law. So Elon absolutely made a choice by placing incentives in front of his users, asking them to choose security or money. This was enacted this week, so we don't have the number. Actually, I'm also not sure we're getting to get the numbers. However, it's perfectly clear that if you want to save money, you're going to drop security. And if you want the security, that means more money for Twitter, but I think it's more likely we're getting less security for users. Shame on you, Elon. Shame on you. Some of the examples I'm going to present here will not be quite obvious. And I think that is a problem. The Elon Musk and Twitter example is very obvious where the incentives are. Here, it's not so clear. And I have rounded up a couple of examples where we really need to think very hard. Where are the incentives, actually? How are we driven by them? How are users driven by these examples? This is a Nessus report. I have a bit of a knack for Nessus reports, and it's not a good one. It's small print. Don't try to read this. But it says critical security vulnerability is 44. Substantial. High security vulnerability is 309. So 309 high security vulnerabilities. I think that has a strong impact. I mean, if you read a report like that, that's a strong message. And the incentive that you're feeling, or I'm feeling, is let's jump to action. This is actionable. Every critical security vulnerability here is meant to be fixed, isn't it? I mean, you want to patch this. You want to get secure.


chunk_text:
  think it's more likely we're getting less security for users. Shame on you, Elon. Shame on you. Some of the examples I'm going to present here will not be quite obvious. And I think that is a problem. The Elon Musk and Twitter example is very obvious where the incentives are. Here, it's not so clear. And I have rounded up a couple of examples where we really need to think very hard. Where are the incentives, actually? How are we driven by them? How are users driven by these examples? This is a Nessus report. I have a bit of a knack for Nessus reports, and it's not a good one. It's small print. Don't try to read this. But it says critical security vulnerability is 44. Substantial. High security vulnerability is 309. So 309 high security vulnerabilities. I think that has a strong impact. I mean, if you read a report like that, that's a strong message. And the incentive that you're feeling, or I'm feeling, is let's jump to action. This is actionable. Every critical security vulnerability here is meant to be fixed, isn't it? I mean, you want to patch this. You want to get secure. You want to invest a lot of time and resources, namely, if audit is around the corner. This happened to a friend of mine. That's how I returned on Nessus reports last week. He was handed a 229 Nessus report by a customer. The customer didn't care about security for several years. Now, audit was coming back, and he, as a security consultant, received the 220 pages Nessus report. Please fix this for us. You have four weeks. Now, if you jump to action and start on page number one, or after the executive summary on page number two, you're doing a bad job here, unfortunately. Second screenshot is actually from the report my friend received. Nessus proposes to ensure that the proxy modules are disabled on a given server. The problem is that server was a proxy server. So Nessus effectively tried to convince my friend, please disable the central functionality of that server so you're secure. I mean, this is a classic false positive. This is stupid advice. Nessus has been around for more or less 20 years. Proxy technology on web servers has been around for 25 years at least. And Nessus is still not in a


chunk_text:
  You want to invest a lot of time and resources, namely, if audit is around the corner. This happened to a friend of mine. That's how I returned on Nessus reports last week. He was handed a 229 Nessus report by a customer. The customer didn't care about security for several years. Now, audit was coming back, and he, as a security consultant, received the 220 pages Nessus report. Please fix this for us. You have four weeks. Now, if you jump to action and start on page number one, or after the executive summary on page number two, you're doing a bad job here, unfortunately. Second screenshot is actually from the report my friend received. Nessus proposes to ensure that the proxy modules are disabled on a given server. The problem is that server was a proxy server. So Nessus effectively tried to convince my friend, please disable the central functionality of that server so you're secure. I mean, this is a classic false positive. This is stupid advice. Nessus has been around for more or less 20 years. Proxy technology on web servers has been around for 25 years at least. And Nessus is still not in a position to find out whether a server is a proxy server or not. I tell you, I'm working on a proxy server. It's a one-liner. It's so easy. If it has a back-end configured, chances are it's a proxy server. And Nessus is not able to do it. What is nice about Nessus is it tells you how it found out the proxy modules are enabled. This is the second slide here. There is a command. If you look at the command, it looks really sophisticated. Wow, AWK and stuff. Wow, grep. That's really nice. It's the wrong config file. Nessus takes the default config file. Chances are the server is running a different config file. Nessus is too stupid to identify the real config file. People pay money for this shit. I'm surprised. What are the incentives here? The incentives tell you, work through the spreadsheet. Check off the findings and you gain in security. This is what Nessus promises to you. But the result is actually time wasted fighting symptoms instead of fixing the core problem. What is the core problem here? We have hundreds of security vulnerabilities. We have bad policies and bad processes. That's the core


chunk_text:
  position to find out whether a server is a proxy server or not. I tell you, I'm working on a proxy server. It's a one-liner. It's so easy. If it has a back-end configured, chances are it's a proxy server. And Nessus is not able to do it. What is nice about Nessus is it tells you how it found out the proxy modules are enabled. This is the second slide here. There is a command. If you look at the command, it looks really sophisticated. Wow, AWK and stuff. Wow, grep. That's really nice. It's the wrong config file. Nessus takes the default config file. Chances are the server is running a different config file. Nessus is too stupid to identify the real config file. People pay money for this shit. I'm surprised. What are the incentives here? The incentives tell you, work through the spreadsheet. Check off the findings and you gain in security. This is what Nessus promises to you. But the result is actually time wasted fighting symptoms instead of fixing the core problem. What is the core problem here? We have hundreds of security vulnerabilities. We have bad policies and bad processes. That's the core problem here. And the individual high security finding, the server that is not patched, is only a symptom of that. So you can run after all these symptoms, try to fix them, but you're not adding anything. Because two weeks afterwards, you'll have new security vulnerabilities. The processes are completely wrong and Nessus is contributing to the problem. Nessus is not providing you with a solution. And I'm not saying there is no job for Nessus in this game. I think if you have a perfect setup and everything is fine, then Nessus can come in and reassure you, yes, good job, tap on the shoulder. Nicely done. Nicely done. The problem is, of course, if Nessus has to disable the proxy server, then it's no good. So too many false positives and too much friction working through false positives. Not good. Next idea or example. This is a random headline I found on the internet. And it says that over 14 million cyber attacks are blocked daily, says a report. If you read the small print, it actually says 14 million attacks on 1200 websites amongst 12 billion requests on a single day by super strong sensors by some


chunk_text:
  problem here. And the individual high security finding, the server that is not patched, is only a symptom of that. So you can run after all these symptoms, try to fix them, but you're not adding anything. Because two weeks afterwards, you'll have new security vulnerabilities. The processes are completely wrong and Nessus is contributing to the problem. Nessus is not providing you with a solution. And I'm not saying there is no job for Nessus in this game. I think if you have a perfect setup and everything is fine, then Nessus can come in and reassure you, yes, good job, tap on the shoulder. Nicely done. Nicely done. The problem is, of course, if Nessus has to disable the proxy server, then it's no good. So too many false positives and too much friction working through false positives. Not good. Next idea or example. This is a random headline I found on the internet. And it says that over 14 million cyber attacks are blocked daily, says a report. If you read the small print, it actually says 14 million attacks on 1200 websites amongst 12 billion requests on a single day by super strong sensors by some random security company. These are not attacks, gentlemen. These are bullets. What is an attack and what is a bullet? This is an industry that is not quite sure what an attack actually means. But that is an individual HTTP request here. So 14 million requests were intercepted. But that's not an attack. That's like going to war and counting bullets. That's no point. That is irrelevant information. They make a headline out of this because they profit from inflating the numbers. I mean, they place an incentive when you get action, get to work now, buy our stuff, and it goes, wow, 14 million attacks daily detected by us. We come to you and detect all these attacks hitting you. But it's actually quite irrelevant. I would be interested how many of these requests were actually dedicated attacks by persistent threat attacking these 1200 websites. But that information is not in the article. And I'm pretty sure the product is not able to give me that information. Who has an interest in inflate numbers? Who is typically inflating numbers? Spam detection software. Very high number. The more spam you detect, the better. Bigger licenses, you can say. There are


chunk_text:
  random security company. These are not attacks, gentlemen. These are bullets. What is an attack and what is a bullet? This is an industry that is not quite sure what an attack actually means. But that is an individual HTTP request here. So 14 million requests were intercepted. But that's not an attack. That's like going to war and counting bullets. That's no point. That is irrelevant information. They make a headline out of this because they profit from inflating the numbers. I mean, they place an incentive when you get action, get to work now, buy our stuff, and it goes, wow, 14 million attacks daily detected by us. We come to you and detect all these attacks hitting you. But it's actually quite irrelevant. I would be interested how many of these requests were actually dedicated attacks by persistent threat attacking these 1200 websites. But that information is not in the article. And I'm pretty sure the product is not able to give me that information. Who has an interest in inflate numbers? Who is typically inflating numbers? Spam detection software. Very high number. The more spam you detect, the better. Bigger licenses, you can say. There are license models around. For every spam message detected, you get more. So you have an interest to get more spam somehow to make money. Firewall administrators tend to block a lot of IP packages raising the number of attacks. It's only an IP package. And then web application firewall operators, people like me somehow, we have a tendency to inflate the numbers because every request that we're blocking is an attack. And it's perhaps only a bullet. Good. Even bigger numbers. Here is a report I found online. It came out in February, I believe. Yes, the end of February. A site called Textury. And they would talk about 300,000 new pieces of malware are created daily. Yeah, this is what happens when you create them in an automatic manner. I mean, you run and it generates malware for you. That doesn't mean that it's going to be deployed. This is the business these criminals are doing. So let's take a step back and remember that security resources are always limited. Every dollar and every hour spent on nonsense is a dollar without a return on investment. It's friction. It's heating up the climate, if you want. So if you


chunk_text:
  license models around. For every spam message detected, you get more. So you have an interest to get more spam somehow to make money. Firewall administrators tend to block a lot of IP packages raising the number of attacks. It's only an IP package. And then web application firewall operators, people like me somehow, we have a tendency to inflate the numbers because every request that we're blocking is an attack. And it's perhaps only a bullet. Good. Even bigger numbers. Here is a report I found online. It came out in February, I believe. Yes, the end of February. A site called Textury. And they would talk about 300,000 new pieces of malware are created daily. Yeah, this is what happens when you create them in an automatic manner. I mean, you run and it generates malware for you. That doesn't mean that it's going to be deployed. This is the business these criminals are doing. So let's take a step back and remember that security resources are always limited. Every dollar and every hour spent on nonsense is a dollar without a return on investment. It's friction. It's heating up the climate, if you want. So if you buy a license for some commercial software based on a shitty report with inflated numbers, you're actively burning dollars in a dumpster fire. Don't fall for such a trap. And shame on you if you create such reports with inflated numbers consciously or in a careless way because you think every IP package blocked is actually an attack. So we have an incentive to gain more budget through inflated numbers. And the result is company resources misassigned. They flow to the highest number. Or management stops trusting the reports. You tell me which one is worse of the two options. So we're getting close to dashboards now. These inflated numbers. Dashboards have a tendency to show you inflated numbers as well. In fact, I've been suffering from examining too many dashboards. Why is that? I'm working on a dashboard myself. I've been working on a dashboard all through 2022. And why is that? Because I've yet to see a decent dashboard. I've been in this industry like 20 years or so, doing a lot of web application, firewall, web service, security engineering. And I've never seen a good dashboard. Here are two examples. The top right is the infamous Norse


chunk_text:
  buy a license for some commercial software based on a shitty report with inflated numbers, you're actively burning dollars in a dumpster fire. Don't fall for such a trap. And shame on you if you create such reports with inflated numbers consciously or in a careless way because you think every IP package blocked is actually an attack. So we have an incentive to gain more budget through inflated numbers. And the result is company resources misassigned. They flow to the highest number. Or management stops trusting the reports. You tell me which one is worse of the two options. So we're getting close to dashboards now. These inflated numbers. Dashboards have a tendency to show you inflated numbers as well. In fact, I've been suffering from examining too many dashboards. Why is that? I'm working on a dashboard myself. I've been working on a dashboard all through 2022. And why is that? Because I've yet to see a decent dashboard. I've been in this industry like 20 years or so, doing a lot of web application, firewall, web service, security engineering. And I've never seen a good dashboard. Here are two examples. The top right is the infamous Norse dashboard. That came out like 2016 or so. It is animated. So what you see here is IP addresses sending IP packages across the globe. And then they hit data centers and they explode. And it's really cool. It's nice ice candy. And then lots of lists. And it's all animated. And it grabs your attention. And you get the feeling this is actionable for you. This is something where you have to do something about this. But that's not the case. It's only show. It's only eye candy. Consequently, Norse died a year or two afterwards. Nice video, but that's not security. What you're getting here. The lower left corner is from the Kibana documentation. This is an example dashboard that Kibana is proposing to you. That you should implement. This is how you could set up Kibana to look nicely. It doesn't have a lot of data. I mean, it's documentation. But you get the idea already. I mean, the numbers are too small. But there seem to be sparklines on top. Then we have bars on the left. The two very monolithic blocks. That's actually bar charts. And on top right and the lower right corner. We


chunk_text:
  dashboard. That came out like 2016 or so. It is animated. So what you see here is IP addresses sending IP packages across the globe. And then they hit data centers and they explode. And it's really cool. It's nice ice candy. And then lots of lists. And it's all animated. And it grabs your attention. And you get the feeling this is actionable for you. This is something where you have to do something about this. But that's not the case. It's only show. It's only eye candy. Consequently, Norse died a year or two afterwards. Nice video, but that's not security. What you're getting here. The lower left corner is from the Kibana documentation. This is an example dashboard that Kibana is proposing to you. That you should implement. This is how you could set up Kibana to look nicely. It doesn't have a lot of data. I mean, it's documentation. But you get the idea already. I mean, the numbers are too small. But there seem to be sparklines on top. Then we have bars on the left. The two very monolithic blocks. That's actually bar charts. And on top right and the lower right corner. We have two donuts or modern variants of pie charts. So the green areas on the left. I mean, what is the information? I mean, there's no information. It's just a block of color. Doesn't tell us anything. But do the pie charts. I'm explaining you the center lower. The lower one here. The very colorful donut. It maps colors to numbers that don't tell me anything. You cannot read the numbers. But khaki color means 200002147. And then the blue is 200002476. That is the information value in this. And this is in the documentation. This is what Kibana is proposing for you to do. Now, what is the idea behind a dashboard? Why do you want to have a dashboard? I mean, it's nice. You can show it to management. You can show it to your friends. And look at all the nice numbers. And what our servers are doing. And all the attacks that we're getting. Look at all these bullets flying in our direction. The promise of the dashboard is actionable. Actionability tells you, hey, this is the information where you get to work. And then you improve security. And A, I don't understand what this is


chunk_text:
  have two donuts or modern variants of pie charts. So the green areas on the left. I mean, what is the information? I mean, there's no information. It's just a block of color. Doesn't tell us anything. But do the pie charts. I'm explaining you the center lower. The lower one here. The very colorful donut. It maps colors to numbers that don't tell me anything. You cannot read the numbers. But khaki color means 200002147. And then the blue is 200002476. That is the information value in this. And this is in the documentation. This is what Kibana is proposing for you to do. Now, what is the idea behind a dashboard? Why do you want to have a dashboard? I mean, it's nice. You can show it to management. You can show it to your friends. And look at all the nice numbers. And what our servers are doing. And all the attacks that we're getting. Look at all these bullets flying in our direction. The promise of the dashboard is actionable. Actionability tells you, hey, this is the information where you get to work. And then you improve security. And A, I don't understand what this is telling me. And B, there is nothing I can do based on these color blocks or on the pie charts. I think pie charts are a really bad way usually of displaying information. I'm in a hate relationship with pie charts. I give you a next example. Yeah, let's dive into this. What is the promise? No, I have to repeat this. The point is that dashboards tend to give you not actionable information, but ready information. Actionable information would be something you need to do something. We have an imminent problem. Focus on this problem and you improve security. That is most of the time not what the dashboard is displaying you. Certainly not these dashboards here. What they're instead doing is displaying you ready information. Information they have at their fingertips that they can easily display to you to distract you actually. The result is distraction. The incentive is they do you a nice dashboard and the result is you're distracted. While you're looking at the dashboard, the attackers continue to attack you. And you're not even seeing it because of wrong information. Something that people in web application firewall dashboards typically show is the total number of


chunk_text:
  telling me. And B, there is nothing I can do based on these color blocks or on the pie charts. I think pie charts are a really bad way usually of displaying information. I'm in a hate relationship with pie charts. I give you a next example. Yeah, let's dive into this. What is the promise? No, I have to repeat this. The point is that dashboards tend to give you not actionable information, but ready information. Actionable information would be something you need to do something. We have an imminent problem. Focus on this problem and you improve security. That is most of the time not what the dashboard is displaying you. Certainly not these dashboards here. What they're instead doing is displaying you ready information. Information they have at their fingertips that they can easily display to you to distract you actually. The result is distraction. The incentive is they do you a nice dashboard and the result is you're distracted. While you're looking at the dashboard, the attackers continue to attack you. And you're not even seeing it because of wrong information. Something that people in web application firewall dashboards typically show is the total number of attacks, the total number of bullets if you want, hitting the application. That is completely irrelevant. Who cares? If you have a roof on your house, do you care how many raindrops it stopped? Are you interested in the number of raindrops stopped per roof tile? Do you want to see the metrics of the average raindrop count stopped per hour grouped by month? Who cares about this bullshit? It's completely irrelevant. Next one. This is an element of a typical web application firewall dashboard. It's the rule alert. Web application firewalls work with rules. They identify malicious requests and then they block them and then they do dashboards. This is important. This is what I did. This is where you put your attention. Obviously, the top line here must be the most important information. The top line says we have 1,259 alerts on the rule host header is a numeric IP address. That is where they focus your head on. This is where dashboard proposes, incentivizes you to put your brain share. Look at this. This is important. Now, as it happens, who triggers that rule? It's misconfigured local answers using the HTTP host header. They use it with


chunk_text:
  attacks, the total number of bullets if you want, hitting the application. That is completely irrelevant. Who cares? If you have a roof on your house, do you care how many raindrops it stopped? Are you interested in the number of raindrops stopped per roof tile? Do you want to see the metrics of the average raindrop count stopped per hour grouped by month? Who cares about this bullshit? It's completely irrelevant. Next one. This is an element of a typical web application firewall dashboard. It's the rule alert. Web application firewalls work with rules. They identify malicious requests and then they block them and then they do dashboards. This is important. This is what I did. This is where you put your attention. Obviously, the top line here must be the most important information. The top line says we have 1,259 alerts on the rule host header is a numeric IP address. That is where they focus your head on. This is where dashboard proposes, incentivizes you to put your brain share. Look at this. This is important. Now, as it happens, who triggers that rule? It's misconfigured local answers using the HTTP host header. They use it with an IP address when the RFC actually says it has to be a fully qualified domain name. It's sloppiness. It's misconfiguration. If you don't like this, configure the load plans correctly. And if you don't, nothing bad happens. Uptime agents, same behavior, and then people scanning the internet trigger this rule. Yes, it's an alert, but who cares? That's not danger. Somebody doing a cheap scan of the internet is no threat to you. If that person is a threat to you, then you have far bigger problems. The threat to you are those people who know how to set a host header correctly, so they don't trigger this rule. So what this dashboard element shows us has a lot to do with survivor bias. Wrong incentives. You probably know this graphic. It's quite famous in the IT security industry. It's a graphic of the Second World War. This is a bomber, an Allied bomber. And for every bomber returning from a bombing mission in Germany, they will count the bullets and put them on the graph. So every red dot is a bullet in a bomber who survived a bombing mission over Germany, perhaps. If you jump to conclusions


chunk_text:
  an IP address when the RFC actually says it has to be a fully qualified domain name. It's sloppiness. It's misconfiguration. If you don't like this, configure the load plans correctly. And if you don't, nothing bad happens. Uptime agents, same behavior, and then people scanning the internet trigger this rule. Yes, it's an alert, but who cares? That's not danger. Somebody doing a cheap scan of the internet is no threat to you. If that person is a threat to you, then you have far bigger problems. The threat to you are those people who know how to set a host header correctly, so they don't trigger this rule. So what this dashboard element shows us has a lot to do with survivor bias. Wrong incentives. You probably know this graphic. It's quite famous in the IT security industry. It's a graphic of the Second World War. This is a bomber, an Allied bomber. And for every bomber returning from a bombing mission in Germany, they will count the bullets and put them on the graph. So every red dot is a bullet in a bomber who survived a bombing mission over Germany, perhaps. If you jump to conclusions quickly, you would go, let's improve the armor on the bomber where we have the red dots. Isn't that natural? I mean, this is where they have been hit. We need to improve the bomber in these regions. And that's completely wrong. The point is, the bombers are already surviving hits in that area. The problem is the bomber hit by bullets that didn't return. And chances are, these are the bullets that hit the white areas. Those bombers didn't return. So what you need to do is improve the armor in the area, but there are no red dots here. So it's the other way around. By focusing on the wrong information, you're following the wrong incentives. And that's exactly the problem with dashboards. It's survivorship bias. It tells you what happened already, the stuff that you blocked. It's the rain that has been stopped by the roof. The one thing I'm interested in my roof is, where are the leaks? I'm not interested in the rain that it stops. I'm interested in the rain that it didn't stop, that is ruining my house. That is the problem. So there are two pieces of information that are really interesting


chunk_text:
  quickly, you would go, let's improve the armor on the bomber where we have the red dots. Isn't that natural? I mean, this is where they have been hit. We need to improve the bomber in these regions. And that's completely wrong. The point is, the bombers are already surviving hits in that area. The problem is the bomber hit by bullets that didn't return. And chances are, these are the bullets that hit the white areas. Those bombers didn't return. So what you need to do is improve the armor in the area, but there are no red dots here. So it's the other way around. By focusing on the wrong information, you're following the wrong incentives. And that's exactly the problem with dashboards. It's survivorship bias. It tells you what happened already, the stuff that you blocked. It's the rain that has been stopped by the roof. The one thing I'm interested in my roof is, where are the leaks? I'm not interested in the rain that it stops. I'm interested in the rain that it didn't stop, that is ruining my house. That is the problem. So there are two pieces of information that are really interesting here. A, only requests that target existing service endpoints are relevant. People scanning the internet are scanning everybody. They're not scanning your endpoints. What is going to happen with these requests? They're getting a 404 most of the time. They're not hitting your application anyway, so why do you care? It's completely irrelevant. They're triggering these rules on the dashboard, but they wouldn't mean no harm to you. They're no problem. And B, the real danger are requests by IP addresses that have triggered alerts, but they continue to improve their attack, and then they bypass the rule set. On subsequent requests, they're no longer being blocked, and then they start to hit relevant endpoints. These are the dangerous requests. This will be actionable. Hey, look, this is somebody who ran into the firewall. We blocked him on the first two, four, six attempts, but then he got his act together and suddenly he hit the backend application. Now we need to do something about it. That is a real attacker. That is a persistent attacker, and I failed to see a dashboard ever to give me this information, and that will be actionable and really important. So the incentives


chunk_text:
  here. A, only requests that target existing service endpoints are relevant. People scanning the internet are scanning everybody. They're not scanning your endpoints. What is going to happen with these requests? They're getting a 404 most of the time. They're not hitting your application anyway, so why do you care? It's completely irrelevant. They're triggering these rules on the dashboard, but they wouldn't mean no harm to you. They're no problem. And B, the real danger are requests by IP addresses that have triggered alerts, but they continue to improve their attack, and then they bypass the rule set. On subsequent requests, they're no longer being blocked, and then they start to hit relevant endpoints. These are the dangerous requests. This will be actionable. Hey, look, this is somebody who ran into the firewall. We blocked him on the first two, four, six attempts, but then he got his act together and suddenly he hit the backend application. Now we need to do something about it. That is a real attacker. That is a persistent attacker, and I failed to see a dashboard ever to give me this information, and that will be actionable and really important. So the incentives are look at the seemingly interesting numbers and maybe shiver in front of all the attacks on your screen, and the result is while you look at irrelevant track record, the attackers bypass the web application firewall, leaving no trace. Good. Let's talk about bug bounty hunter. I think it's time to talk about bug bounty hunters. Do we have bug bounty hunters in the room? Don't be shy. We don't have bug bounty. Oh, there is one at the very end. Hello, Ale. Okay, let's laugh a bit about Ale. Are pen testers in the room? Yeah, there are far more pen testers. That's one of the problems. Far more pen testers than bug bounty hunters. We get to you, pen testers, immediately after the bug bounty hunters. Is that a deal? Good. So bug bounty hunters have been around for a couple of years. Please appreciate the hoodie photo found. It's one of the photos that have not been used in any sales presentations before. I even paid money for that photo. I like it so much. It's so cool. So bug bounty has been around for a couple of years, at least in Switzerland. And we see


chunk_text:
  are look at the seemingly interesting numbers and maybe shiver in front of all the attacks on your screen, and the result is while you look at irrelevant track record, the attackers bypass the web application firewall, leaving no trace. Good. Let's talk about bug bounty hunter. I think it's time to talk about bug bounty hunters. Do we have bug bounty hunters in the room? Don't be shy. We don't have bug bounty. Oh, there is one at the very end. Hello, Ale. Okay, let's laugh a bit about Ale. Are pen testers in the room? Yeah, there are far more pen testers. That's one of the problems. Far more pen testers than bug bounty hunters. We get to you, pen testers, immediately after the bug bounty hunters. Is that a deal? Good. So bug bounty hunters have been around for a couple of years. Please appreciate the hoodie photo found. It's one of the photos that have not been used in any sales presentations before. I even paid money for that photo. I like it so much. It's so cool. So bug bounty has been around for a couple of years, at least in Switzerland. And we see tendencies now in the market. There was a promise around bug bounty hunting. I'm very much in favor of bug bounty hunting as well. I promoted that a lot. And now we're seeing maybe we don't really get what we expected from this new technique. Maybe something else is happening. Maybe it's still a net positive. But there are definitely incentives at play here that are not in favor of security. And you see where I'm taking this already. The question is, are bug bounty hunters interested in security? We said probably yes. Yeah, that's why they work in security. And that's certainly one of the incentives of how they start to bug bounty hunt in the first place. And the question more to the point, are bug bounty hunters interested in securities of the companies that they are probing? Of the programs that they're working for? Now, ideally, yes. However, in reality, I think the bounties are spoiling bug bounty hunters real quickly. It's like nice people entering an investment bank. Money spoils them really quickly. I think after a couple of months or years, they're more interested in personal fame. That's how red teamers work anyway. They want


chunk_text:
  tendencies now in the market. There was a promise around bug bounty hunting. I'm very much in favor of bug bounty hunting as well. I promoted that a lot. And now we're seeing maybe we don't really get what we expected from this new technique. Maybe something else is happening. Maybe it's still a net positive. But there are definitely incentives at play here that are not in favor of security. And you see where I'm taking this already. The question is, are bug bounty hunters interested in security? We said probably yes. Yeah, that's why they work in security. And that's certainly one of the incentives of how they start to bug bounty hunt in the first place. And the question more to the point, are bug bounty hunters interested in securities of the companies that they are probing? Of the programs that they're working for? Now, ideally, yes. However, in reality, I think the bounties are spoiling bug bounty hunters real quickly. It's like nice people entering an investment bank. Money spoils them really quickly. I think after a couple of months or years, they're more interested in personal fame. That's how red teamers work anyway. They want to be the greatest hackers in the world. I mean, I think that's natural for a red teamer. So they're interested in personal fame. Personal fame is directly tied to monetary incentives in bug bounty hunters. Higher bounties means you're cooler than the other guy. So it's five digits bounties that they're after. And how do you get five digits bounties? It's about automation of the bug discovery and optimizing. Ultimately, it's specialization. The bug bounty hunter has a strong incentive to specialize in certain area. As I came to see, I have a limited vision on this. I think it's cross-site scripting. A lot of bug bounty hunters are specializing in cross-site scripting because it's relatively easy to automate and it promises big payouts. It's a growing industry, namely in Switzerland. More and more programs pop up. It's like mushrooms in autumn. More and more programs. Relatively few bug bounty hunters around. It seems only Ale in the room is a bug bounty hunter. So there is a high demand for bug bounty hunters by so many programs and relatively little bug bounty hunters. As a bug bounty program, you struggle to get them onto your program. And then


chunk_text:
  to be the greatest hackers in the world. I mean, I think that's natural for a red teamer. So they're interested in personal fame. Personal fame is directly tied to monetary incentives in bug bounty hunters. Higher bounties means you're cooler than the other guy. So it's five digits bounties that they're after. And how do you get five digits bounties? It's about automation of the bug discovery and optimizing. Ultimately, it's specialization. The bug bounty hunter has a strong incentive to specialize in certain area. As I came to see, I have a limited vision on this. I think it's cross-site scripting. A lot of bug bounty hunters are specializing in cross-site scripting because it's relatively easy to automate and it promises big payouts. It's a growing industry, namely in Switzerland. More and more programs pop up. It's like mushrooms in autumn. More and more programs. Relatively few bug bounty hunters around. It seems only Ale in the room is a bug bounty hunter. So there is a high demand for bug bounty hunters by so many programs and relatively little bug bounty hunters. As a bug bounty program, you struggle to get them onto your program. And then these guys are specializing in a niche or in a certain area. And that means you get spotty coverage. And the promise of the bug bounty was you have thousands of hackers probing your servers and they'll have no stone unturned and ultimately they will discover every weakness. But what you're getting now is five professionals hitting you with all cross-site scripting around and the rest of your application is probably not covered properly. Let's turn to penetration testers. As I said, I've been in this industry for quite a long time and I've never seen, correct me if I'm wrong, open for feedback, but I've never seen a penetration tester kill a project. No. They always write nice reports. They always have critical vulnerability findings. But whenever I read these reports, I mean, I like these people. I like them. I'm really a fan of penetration test reports because it's so interesting. I learn so much. I can improve my setup and so on. But I always get the feeling it's very well balanced. The executive summary very often goes, you've done good work so far. However, we realize that there are still a couple of weaknesses in your


chunk_text:
  these guys are specializing in a niche or in a certain area. And that means you get spotty coverage. And the promise of the bug bounty was you have thousands of hackers probing your servers and they'll have no stone unturned and ultimately they will discover every weakness. But what you're getting now is five professionals hitting you with all cross-site scripting around and the rest of your application is probably not covered properly. Let's turn to penetration testers. As I said, I've been in this industry for quite a long time and I've never seen, correct me if I'm wrong, open for feedback, but I've never seen a penetration tester kill a project. No. They always write nice reports. They always have critical vulnerability findings. But whenever I read these reports, I mean, I like these people. I like them. I'm really a fan of penetration test reports because it's so interesting. I learn so much. I can improve my setup and so on. But I always get the feeling it's very well balanced. The executive summary very often goes, you've done good work so far. However, we realize that there are still a couple of weaknesses in your application. However, if you continue the good work, then you can make this work. I mean, you can do this. It's challenging, but you can do this and then we'll come back and do a retest of your application. So I think the backbone at the penetration tester has a very strong incentive in a follow-up contract. And the result is that the penetration tester will not turn every stone. A, the penetration test is scoped anyway. So there is a contract, but there's a post to look at. But it's not very holistic. It would be too expensive anyway. And the result, based on this incentive for the next contract to be nice, to write a nice executive management summary, is a nice contract and a nice report that guarantees a follow-up contract. Penetration testers tend to call this a partnership, a long-term partnership. But I think a long-term partnership is a nice word for follow-up contract. Let's continue to do business because I tap your shoulder and you give me money for it. It doesn't mean it's not security, but it means you'll get a limited view of the security of your online service. The penetration tester does


chunk_text:
  application. However, if you continue the good work, then you can make this work. I mean, you can do this. It's challenging, but you can do this and then we'll come back and do a retest of your application. So I think the backbone at the penetration tester has a very strong incentive in a follow-up contract. And the result is that the penetration tester will not turn every stone. A, the penetration test is scoped anyway. So there is a contract, but there's a post to look at. But it's not very holistic. It would be too expensive anyway. And the result, based on this incentive for the next contract to be nice, to write a nice executive management summary, is a nice contract and a nice report that guarantees a follow-up contract. Penetration testers tend to call this a partnership, a long-term partnership. But I think a long-term partnership is a nice word for follow-up contract. Let's continue to do business because I tap your shoulder and you give me money for it. It doesn't mean it's not security, but it means you'll get a limited view of the security of your online service. The penetration tester does not have an interest to rip you apart by telling you about things you've never heard of and they're completely undermining the security of your setup. You better stop working today and drop that project. It's not how this works. The result is again spotty coverage in the reports. So no matter what you seem to be doing, neither penetration tester nor bug bounty hunters give you a comprehensive coverage of your online service and their security. In fact, nobody can give you a substantial assessment of the services or the security of your services. The incentives are simply playing against you and your requirements and only time will tell whether you're secure or not. Good. Let's look at the Anthem breach. Probably don't remember that was in 2015. That was the big E-Health patient data breach. The incentives here in play are business. They build bigger and bigger and larger baskets and then they put all the eggs in the same. The incentive being centralized, digitalized, consolidate to get better revenue, higher margins. However, the result is, of course, a cluster risk. And when the database is exploited, you lose all the eggs or all the patient data, obviously.


chunk_text:
  not have an interest to rip you apart by telling you about things you've never heard of and they're completely undermining the security of your setup. You better stop working today and drop that project. It's not how this works. The result is again spotty coverage in the reports. So no matter what you seem to be doing, neither penetration tester nor bug bounty hunters give you a comprehensive coverage of your online service and their security. In fact, nobody can give you a substantial assessment of the services or the security of your services. The incentives are simply playing against you and your requirements and only time will tell whether you're secure or not. Good. Let's look at the Anthem breach. Probably don't remember that was in 2015. That was the big E-Health patient data breach. The incentives here in play are business. They build bigger and bigger and larger baskets and then they put all the eggs in the same. The incentive being centralized, digitalized, consolidate to get better revenue, higher margins. However, the result is, of course, a cluster risk. And when the database is exploited, you lose all the eggs or all the patient data, obviously. If you look through the newspaper article, this one is from Wall Street Journal, the affected products, let me quote them, are called Anthem Blue Cross, Anthem Blue Cross and Blue Shield, Blue Cross and Blue Shield of Georgia, Empire Blue Cross and Blue Shield, Amerigroup, Caremore, Unicare. These are all different brands on the markets and they all came back to the same database that was hacked. The incentives of business work contrary to security, very directly here. Now, a funny rumor is that Anthem actually had a cyber insurance with a coverage of 100 million and they used all that money for the notification process. So, really bad for Anthem. Good. Crisis communication. The incentives are twofold. This example came up on Monday. Swiss pen testing company Pentagrid revealed that Wizeka credit card company of Switzerland had a weakness on the server. You could look up credit card statements of companies without being the owner of this data. This is Heidi News talking about it. It was popularized by the Republic, a Swiss-German online magazine on Monday. And for the company Wizeka, they did not communicate this. Certainly not with the end users. However, they also informed the


chunk_text:
  If you look through the newspaper article, this one is from Wall Street Journal, the affected products, let me quote them, are called Anthem Blue Cross, Anthem Blue Cross and Blue Shield, Blue Cross and Blue Shield of Georgia, Empire Blue Cross and Blue Shield, Amerigroup, Caremore, Unicare. These are all different brands on the markets and they all came back to the same database that was hacked. The incentives of business work contrary to security, very directly here. Now, a funny rumor is that Anthem actually had a cyber insurance with a coverage of 100 million and they used all that money for the notification process. So, really bad for Anthem. Good. Crisis communication. The incentives are twofold. This example came up on Monday. Swiss pen testing company Pentagrid revealed that Wizeka credit card company of Switzerland had a weakness on the server. You could look up credit card statements of companies without being the owner of this data. This is Heidi News talking about it. It was popularized by the Republic, a Swiss-German online magazine on Monday. And for the company Wizeka, they did not communicate this. Certainly not with the end users. However, they also informed the banks that are connected to this user really, really late. So, in crisis communication, namely in IT security, I think there are two contradicting incentives at play. One is be proactive, be transparent about your problems, talk about it, put everything on the table so people can make up their mind, and don't downplay anything. However, the second incentive would be is a completely different strategy. Be secretive, be reactive, don't talk about it and don't admit anything that they cannot prove to you. The promise of the second scenario is nobody will find out and no problem. Unless, of course, newspapers start to talk about it and then everybody will tell you, hey, you should have been transparent from the beginning. However, I see in Switzerland the tendency to go with the second option so far. And they knew media will get this pentagram announced we're going to publish this and they had to expect me to jump on this. They still choose not a problem, no problem at all. And by the way, no customer data was ever revealed outside of a single funny user in the canton of Schwyz. And nobody knows why that record has been


chunk_text:
  banks that are connected to this user really, really late. So, in crisis communication, namely in IT security, I think there are two contradicting incentives at play. One is be proactive, be transparent about your problems, talk about it, put everything on the table so people can make up their mind, and don't downplay anything. However, the second incentive would be is a completely different strategy. Be secretive, be reactive, don't talk about it and don't admit anything that they cannot prove to you. The promise of the second scenario is nobody will find out and no problem. Unless, of course, newspapers start to talk about it and then everybody will tell you, hey, you should have been transparent from the beginning. However, I see in Switzerland the tendency to go with the second option so far. And they knew media will get this pentagram announced we're going to publish this and they had to expect me to jump on this. They still choose not a problem, no problem at all. And by the way, no customer data was ever revealed outside of a single funny user in the canton of Schwyz. And nobody knows why that record has been leaked. Good. Ransomware, closely related to that. Ransomware is all the rage. This here is the first cryptolocker from 1989. It's the HDOS Trojan. Very interesting. You had to send a check to Panama to get the decryption key. That business model didn't work so nicely. In the meantime, they've replaced Panama with Bitcoins. And that seems to work a lot nicer. I think ransomware is interesting from various perspectives. But one thing that I think is very interesting is the question, do you pay or are you not paying? Paying means the pain goes away immediately. Downtime is over. You can continue with your work. Unfortunately, the result is criminals get money in their hands and they get stronger over time. I'm seeing them getting more and more leverage. Ultimately, I think it will be no question whether you pay or not. I think the longer this goes, criminal will force you to pay and you cannot afford to not pay in the end over legal threats or going out of business immediately. This question whether to pay or not becomes even more interesting if you look at cyber insurance. There is relatively little news about this, but I


chunk_text:
  leaked. Good. Ransomware, closely related to that. Ransomware is all the rage. This here is the first cryptolocker from 1989. It's the HDOS Trojan. Very interesting. You had to send a check to Panama to get the decryption key. That business model didn't work so nicely. In the meantime, they've replaced Panama with Bitcoins. And that seems to work a lot nicer. I think ransomware is interesting from various perspectives. But one thing that I think is very interesting is the question, do you pay or are you not paying? Paying means the pain goes away immediately. Downtime is over. You can continue with your work. Unfortunately, the result is criminals get money in their hands and they get stronger over time. I'm seeing them getting more and more leverage. Ultimately, I think it will be no question whether you pay or not. I think the longer this goes, criminal will force you to pay and you cannot afford to not pay in the end over legal threats or going out of business immediately. This question whether to pay or not becomes even more interesting if you look at cyber insurance. There is relatively little news about this, but I found this Reuters article where it says insurers say some attackers may even check whether potential victims have policies that would make them more likely to pay out. If you manage to get a cyber insurance policy covering ransomware, that means the moment ransomware hits you, the cyber insurance takes over. From a criminal perspective, that is really attractive because they get to talk to the professionals. They get to talk to people to know how to transfer bitcoins, the cyber insurance company. They also don't have to do presentations where they present their business model and the service offerings to the victim. From an attacker perspective, it's very interesting to attack somebody who knows what the shot is, somebody with a cyber insurance. From the perspective of the victim, the thing gets more complicated. That means the cyber insurance promises coverage for your loss, for your damage, but somehow there seems to be incentives around that means you're probably more likely to be attacked by having the cyber insurance, certainly if you announce it, or there have been cyber insurances that have been hacked so the customer database was stolen and there were data where they had where subsequent


chunk_text:
  found this Reuters article where it says insurers say some attackers may even check whether potential victims have policies that would make them more likely to pay out. If you manage to get a cyber insurance policy covering ransomware, that means the moment ransomware hits you, the cyber insurance takes over. From a criminal perspective, that is really attractive because they get to talk to the professionals. They get to talk to people to know how to transfer bitcoins, the cyber insurance company. They also don't have to do presentations where they present their business model and the service offerings to the victim. From an attacker perspective, it's very interesting to attack somebody who knows what the shot is, somebody with a cyber insurance. From the perspective of the victim, the thing gets more complicated. That means the cyber insurance promises coverage for your loss, for your damage, but somehow there seems to be incentives around that means you're probably more likely to be attacked by having the cyber insurance, certainly if you announce it, or there have been cyber insurances that have been hacked so the customer database was stolen and there were data where they had where subsequent attacks resulted from the loss of the customer database. Good. We started 10 minutes late. Still have time. I'm getting to my last example now. It also happens to be the biggest one. Good. This is coming back to web application firewalls. This is like the only topic I really know anything about. The rest here was all somehow things that sprung to my mind. But web application firewalls, I've been working with for many, many years. And what is a web application firewall for repetition? It is something you place in front of your bank, of your online shop that does a first line of defense. It inspects the HTTP traffic and if it encounters something suspicious, malicious traffic, it blocks it and stops the attack from hitting the backend application. It's like the first line of defense, an additional layer of security in front of your application. If done right, that contributes to security. If done wrong, it's a lot of pain without a return on investment. My business case is making sure you do have a return on investment. Good. There is a market of maybe 100 commercial offerings around globally, including content delivery networks and online


chunk_text:
  attacks resulted from the loss of the customer database. Good. We started 10 minutes late. Still have time. I'm getting to my last example now. It also happens to be the biggest one. Good. This is coming back to web application firewalls. This is like the only topic I really know anything about. The rest here was all somehow things that sprung to my mind. But web application firewalls, I've been working with for many, many years. And what is a web application firewall for repetition? It is something you place in front of your bank, of your online shop that does a first line of defense. It inspects the HTTP traffic and if it encounters something suspicious, malicious traffic, it blocks it and stops the attack from hitting the backend application. It's like the first line of defense, an additional layer of security in front of your application. If done right, that contributes to security. If done wrong, it's a lot of pain without a return on investment. My business case is making sure you do have a return on investment. Good. There is a market of maybe 100 commercial offerings around globally, including content delivery networks and online cloud services. And there is a single open source offering, the OOS ModSecurity Coral set, the project I happen to run in my spare time, co-run in my spare time. So as a commercial vendor, you have two options. 100 commercial vendors around. They had to ask themselves the question, do we want to create a rule set on our own? Do we want to build something to detect all kinds of attacks and sell this to our customers? Or do we integrate the single open source offering into our solution? Two thirds of the market went with the second option. One third of the market continues to develop their own rule set. Now, this is a matrix based on a blog post by Tuomas Makkonen. It's two or three years old. And he would look at commercial cloud web application firewalls offerings. And then vertically, he would fire attacks against them. He would test their performance, their security performance. And then in the columns, you have the different commercial offering. First line, we're only looking at the first line. Remote command execution. Remote command execution is very bad. If you happen to execute this and it works, then you


chunk_text:
  cloud services. And there is a single open source offering, the OOS ModSecurity Coral set, the project I happen to run in my spare time, co-run in my spare time. So as a commercial vendor, you have two options. 100 commercial vendors around. They had to ask themselves the question, do we want to create a rule set on our own? Do we want to build something to detect all kinds of attacks and sell this to our customers? Or do we integrate the single open source offering into our solution? Two thirds of the market went with the second option. One third of the market continues to develop their own rule set. Now, this is a matrix based on a blog post by Tuomas Makkonen. It's two or three years old. And he would look at commercial cloud web application firewalls offerings. And then vertically, he would fire attacks against them. He would test their performance, their security performance. And then in the columns, you have the different commercial offering. First line, we're only looking at the first line. Remote command execution. Remote command execution is very bad. If you happen to execute this and it works, then you have a root shell on the server. That is not nice. You want to defend against remote command execution. Mind you, having the WAF ignore a remote command execution doesn't mean you have a root shell immediately. It just means that the first line of defense failed. You still need a weakness in your application. So even if it's very red here, that doesn't mean you're immediately exploited. It just means that the promise of security was not delivered. You purchased a commercial solution and it didn't work. First column is AWS WAF. That's what you get very cheaply from AWS. It has a performance of 4% of the attacks in the remote command execution category are blocked. 96% of the attacks will bypass the AWS WAF at least two years ago. Next column is AWS Fortinet rule set. So AWS, you can have commercial additional rule set. The Fortinet one is two and a half times better than the default one. Now we're up to 10%. So only 90% bypass the WAF. And then comes the third column. That's a stark contrast. Now we're over at the competition at Azure. And in the second line, you have in brackets


chunk_text:
  have a root shell on the server. That is not nice. You want to defend against remote command execution. Mind you, having the WAF ignore a remote command execution doesn't mean you have a root shell immediately. It just means that the first line of defense failed. You still need a weakness in your application. So even if it's very red here, that doesn't mean you're immediately exploited. It just means that the promise of security was not delivered. You purchased a commercial solution and it didn't work. First column is AWS WAF. That's what you get very cheaply from AWS. It has a performance of 4% of the attacks in the remote command execution category are blocked. 96% of the attacks will bypass the AWS WAF at least two years ago. Next column is AWS Fortinet rule set. So AWS, you can have commercial additional rule set. The Fortinet one is two and a half times better than the default one. Now we're up to 10%. So only 90% bypass the WAF. And then comes the third column. That's a stark contrast. Now we're over at the competition at Azure. And in the second line, you have in brackets the core rule set. That is my project. Azure is based on core rule set. They didn't develop their own rules. Primarily, they use the open source solution and they get a green column. Now they're up to 71. They're seven times stronger than Fortinet by using the open source solution. Next one is Barracuda. That's a company that's doing their own rules. Again, and then Alibaba. I don't know what they're doing exactly. But the fact that they have two green items means probably they're also based on CRS. Somebody told me they are, but I've never seen a hard date on this. Then you have Google Cloud Armor, 73%. And in the final column, CloudFlare WAF, also based on CRS. Everything green, with the exception of the Alibaba cloud, is known to be CRS and documented as being based on CRS. I don't know exactly what CloudFlare used to be doing two years ago in the remote command execution category. I think they disabled rules. And I also know for a fact they're improved in the meantime. So now that might be green in the meantime. But they disabled rules. So the open source offering gave them detection


chunk_text:
  the core rule set. That is my project. Azure is based on core rule set. They didn't develop their own rules. Primarily, they use the open source solution and they get a green column. Now they're up to 71. They're seven times stronger than Fortinet by using the open source solution. Next one is Barracuda. That's a company that's doing their own rules. Again, and then Alibaba. I don't know what they're doing exactly. But the fact that they have two green items means probably they're also based on CRS. Somebody told me they are, but I've never seen a hard date on this. Then you have Google Cloud Armor, 73%. And in the final column, CloudFlare WAF, also based on CRS. Everything green, with the exception of the Alibaba cloud, is known to be CRS and documented as being based on CRS. I don't know exactly what CloudFlare used to be doing two years ago in the remote command execution category. I think they disabled rules. And I also know for a fact they're improved in the meantime. So now that might be green in the meantime. But they disabled rules. So the open source offering gave them detection capability and they disabled it. And that leads us to the question, why would they disable it? And why would the commercial offerings or the commercial offerings being based on in-house rule set perform so much worse than the ones that use the open source offering? Why is the open source offering stronger than something commercial? Good. Let's look at a story that might explain this. Once upon a time, there was a boy. You probably know this story. The boy was a shepherd. Actually, he also had a ram in his shepherd in his herd for diversity reasons. And he would take the sheep to the forest. You know where I'm taking this. In the forest, there was a wolf. Very bad. So the boy screamed, raised the alarm, called the hunters with the idea the hunters will come, hunt down the wolf and make mincemeat out of the wolf. And it would be a big party in the village and everything was nice and the herd had no problem in the future. However, the problem is there was a false alarm. The boy called the hunters, but there was no wolf. Same happened the next day. Same happened


chunk_text:
  capability and they disabled it. And that leads us to the question, why would they disable it? And why would the commercial offerings or the commercial offerings being based on in-house rule set perform so much worse than the ones that use the open source offering? Why is the open source offering stronger than something commercial? Good. Let's look at a story that might explain this. Once upon a time, there was a boy. You probably know this story. The boy was a shepherd. Actually, he also had a ram in his shepherd in his herd for diversity reasons. And he would take the sheep to the forest. You know where I'm taking this. In the forest, there was a wolf. Very bad. So the boy screamed, raised the alarm, called the hunters with the idea the hunters will come, hunt down the wolf and make mincemeat out of the wolf. And it would be a big party in the village and everything was nice and the herd had no problem in the future. However, the problem is there was a false alarm. The boy called the hunters, but there was no wolf. Same happened the next day. Same happened the day afterwards. False alarms, no wolf. The hunters gave up on the boy. And when the real wolf struck, the real wolf struck, I want to show you this one. This happens. Good. Coming back. I think the story is very sad. It's also very bad. I think the way we're traditionally telling this story is we're telling it the wrong way. If you're a boy with sheep in a forest, it's really hard. There is a lot of missing information. The forest is dark. It's very dense. And the wolf is very cunning. So you don't know. You hear something. Maybe you see something. You don't know what is happening exactly. And you fear for your sheep. So you call the hunters, despite not knowing for a fact. You're not really sure. And you don't want to wait until the wolf has one sheep between his jaws. You want to be early. That's why you raise the alarm. I think the boy has very bad press in this story. But in fact, I think the boy is in a very bad position. And truth be told, this is my story. As an open source developers, we're the boy.


chunk_text:
  the day afterwards. False alarms, no wolf. The hunters gave up on the boy. And when the real wolf struck, the real wolf struck, I want to show you this one. This happens. Good. Coming back. I think the story is very sad. It's also very bad. I think the way we're traditionally telling this story is we're telling it the wrong way. If you're a boy with sheep in a forest, it's really hard. There is a lot of missing information. The forest is dark. It's very dense. And the wolf is very cunning. So you don't know. You hear something. Maybe you see something. You don't know what is happening exactly. And you fear for your sheep. So you call the hunters, despite not knowing for a fact. You're not really sure. And you don't want to wait until the wolf has one sheep between his jaws. You want to be early. That's why you raise the alarm. I think the boy has very bad press in this story. But in fact, I think the boy is in a very bad position. And truth be told, this is my story. As an open source developers, we're the boy. We're raising the alarm too early. We're provoking false positives. And this is the cost of doing business. Why are we doing this? Because the incentives are aligned for us to produce false positives. Honestly, if I produce a false positive, that's really bad. I'm sorry for the users. I'm sorry for the customers. Maybe somebody was checking out an online shop and then a false positive happened. Business didn't happen. Company didn't sell something. They lost money, consequently. And they complain to me and say, look, I'm really sorry for you. But you need to handle this. And you have to work around this. Here is the documentation. Help yourself. This is what I'm telling. Ultimately, I'm not affected. Or if I'm affected, it's very, very indirectly for my business. However, for a commercial company, situation looks very much different. For a commercial company, if somebody is not able to do business despite the product, they will get a call immediately. Hey, look, our CEO failed to log in to our platform today. And he was really pissed. And you guys need to get your act together. Because if you don't, we're stopping to pay you for the sheep.


chunk_text:
  We're raising the alarm too early. We're provoking false positives. And this is the cost of doing business. Why are we doing this? Because the incentives are aligned for us to produce false positives. Honestly, if I produce a false positive, that's really bad. I'm sorry for the users. I'm sorry for the customers. Maybe somebody was checking out an online shop and then a false positive happened. Business didn't happen. Company didn't sell something. They lost money, consequently. And they complain to me and say, look, I'm really sorry for you. But you need to handle this. And you have to work around this. Here is the documentation. Help yourself. This is what I'm telling. Ultimately, I'm not affected. Or if I'm affected, it's very, very indirectly for my business. However, for a commercial company, situation looks very much different. For a commercial company, if somebody is not able to do business despite the product, they will get a call immediately. Hey, look, our CEO failed to log in to our platform today. And he was really pissed. And you guys need to get your act together. Because if you don't, we're stopping to pay you for the sheep. No, we stop to pay your license. Every false positive is a direct threat on the business model of a commercial WAF vendor. It's very, very bad to have a false positive for a WAF vendor. How about false negatives? For me, I come to a security conference. We're going to do a coffee break afterwards. I want people to walk up to me and tap my shoulder and say, Hey, I tried to get past your CRS rules last week. Oh, boy, it was tough. I was working as a penetration tester. They had disabled the WAF. It was so strong. Good job. That's what I'm into this game. This is what I want to hear. The last thing I want to hear, people walking up to me, Ha, I run a penetration test and the WAF is bullshit. That's a joke what you're doing. All the hours you spend in your spare time was no match for my attack skills. That's the last thing I want to hear. I want to make a very, very strong WAF that is not allowing for any false negatives. These are my incentives. That's how the open source WAF tries to be


chunk_text:
  No, we stop to pay your license. Every false positive is a direct threat on the business model of a commercial WAF vendor. It's very, very bad to have a false positive for a WAF vendor. How about false negatives? For me, I come to a security conference. We're going to do a coffee break afterwards. I want people to walk up to me and tap my shoulder and say, Hey, I tried to get past your CRS rules last week. Oh, boy, it was tough. I was working as a penetration tester. They had disabled the WAF. It was so strong. Good job. That's what I'm into this game. This is what I want to hear. The last thing I want to hear, people walking up to me, Ha, I run a penetration test and the WAF is bullshit. That's a joke what you're doing. All the hours you spend in your spare time was no match for my attack skills. That's the last thing I want to hear. I want to make a very, very strong WAF that is not allowing for any false negatives. These are my incentives. That's how the open source WAF tries to be as strong as possible. And the commercial WAF tries to get out of the way of customers. They have a very strong incentive to avoid false positive and maybe a sack false negative. We can, in the coffee break, explore the false negative situation in a commercial setup. I think it's even more pervert than this, but I don't want to dive into that in the presentation too long. So time to wrap this up or time to come back to here. Now we have an explanation while the ones using commercial rules perform much weaker than the ones using open source rule set. And then in the case of Cloudflare, they looked at this and realized, Hey, this is causing too much business problems. So let's take out a few rules of the strong open source offering. Incentives are at play here. And we need to look at the incentives to realize where do we get the strong product that delivers security? And where do we get something that is a nice checkbox security? Interesting enough, that is a graphic you don't see in the Gartner report or the Forrester WAF quadrant. The last thing they're looking at is


chunk_text:
  as strong as possible. And the commercial WAF tries to get out of the way of customers. They have a very strong incentive to avoid false positive and maybe a sack false negative. We can, in the coffee break, explore the false negative situation in a commercial setup. I think it's even more pervert than this, but I don't want to dive into that in the presentation too long. So time to wrap this up or time to come back to here. Now we have an explanation while the ones using commercial rules perform much weaker than the ones using open source rule set. And then in the case of Cloudflare, they looked at this and realized, Hey, this is causing too much business problems. So let's take out a few rules of the strong open source offering. Incentives are at play here. And we need to look at the incentives to realize where do we get the strong product that delivers security? And where do we get something that is a nice checkbox security? Interesting enough, that is a graphic you don't see in the Gartner report or the Forrester WAF quadrant. The last thing they're looking at is WAF performance. Good. Time to wrap this up. I see two levels of bad incentives. An overly relaxed attitude, ignorance, negligence, carelessness leads to bad incentives for users. They trick them into weak decisions that ultimately undermine security. Think bad dashboards. Eating your brain shares. Nonsense. Because the guy developing the dashboard didn't think from your eyes, from an operating perspective, what is actually important? What is actionable? What information is useful? It was just carelessness. On the second level, however, people are deliberately following or setting crazy incentives for immediate gain. Financial benefit through bad incentives. They're consciously prioritizing financial benefit over security of users and their data. I see WAF vendors at play here and Elon Musk. Shame on him. So what do we do about this? I think it's our job, it's your job, to raise the alarm when you see incentives that are not aligned with security. In the projects that you're working on, you see people at the table who seem to have different incentives than the security goal that you actually have in mind. You need to step back and think about incentives when you make your business decisions or technical decisions. What are


chunk_text:
  WAF performance. Good. Time to wrap this up. I see two levels of bad incentives. An overly relaxed attitude, ignorance, negligence, carelessness leads to bad incentives for users. They trick them into weak decisions that ultimately undermine security. Think bad dashboards. Eating your brain shares. Nonsense. Because the guy developing the dashboard didn't think from your eyes, from an operating perspective, what is actually important? What is actionable? What information is useful? It was just carelessness. On the second level, however, people are deliberately following or setting crazy incentives for immediate gain. Financial benefit through bad incentives. They're consciously prioritizing financial benefit over security of users and their data. I see WAF vendors at play here and Elon Musk. Shame on him. So what do we do about this? I think it's our job, it's your job, to raise the alarm when you see incentives that are not aligned with security. In the projects that you're working on, you see people at the table who seem to have different incentives than the security goal that you actually have in mind. You need to step back and think about incentives when you make your business decisions or technical decisions. What are my incentives? What are the vendors' incentives? What is the consultant's incentive? What are the incentives of my boss, of the CISO, the security officer? Everybody at the table. And if the incentives and security don't align, then you need to cry wolf early enough. Thank you very much. So we started a bit later than expected, but it's almost coffee break. Can we take one or two questions? Ale? One question. Of course. Of course. There is one question at the end, at the back. Yeah, I've actually got a comment, not really a question, because you showed us the WAF comparison stuff. Yes. I personally don't really believe about WAFs, but yeah. So we are seeing this comparison between WAFs. The problem is I look at the blog post from the guy who posted it, and there is no reproducibility in it. Yes. So if we are doing such research, please, well, post your scripts that you are doing the research with and getting the results out of. Because otherwise, how can we really know whether this thing that we are seeing here is actually, you know, good or not? Like, if you look at the examples they


chunk_text:
  my incentives? What are the vendors' incentives? What is the consultant's incentive? What are the incentives of my boss, of the CISO, the security officer? Everybody at the table. And if the incentives and security don't align, then you need to cry wolf early enough. Thank you very much. So we started a bit later than expected, but it's almost coffee break. Can we take one or two questions? Ale? One question. Of course. Of course. There is one question at the end, at the back. Yeah, I've actually got a comment, not really a question, because you showed us the WAF comparison stuff. Yes. I personally don't really believe about WAFs, but yeah. So we are seeing this comparison between WAFs. The problem is I look at the blog post from the guy who posted it, and there is no reproducibility in it. Yes. So if we are doing such research, please, well, post your scripts that you are doing the research with and getting the results out of. Because otherwise, how can we really know whether this thing that we are seeing here is actually, you know, good or not? Like, if you look at the examples they used in this research, they, for example, sent to some, like, an application hosted on AWS, beyond one of the WAFs there, some payload that is, you know, a command injection payload for, like, PHP applications. But what if your app is not really PHP? Then they marked it as, you know, false positive or whatever. Like, you're not detecting this. And is that, you know, good? Probably not. And if you would, like, post or, like, publish your research, like, how did you do this? Then people can also, you know, improve. Yeah. Point taken. I mean, Tuomas Markon should have published the test scripts he used for this. He didn't. The reason I'm using this is because he's third party. I can reproduce this. I'm getting more or less the same results for these tests. But I'm obviously biased. Nobody would trust my numbers. So at least he's third party. Actually, if you reproduce this, you can also... No, I'm biased. I mean, I'm the wrong person to do that research. You should do the research and publish your results. I mean, nobody will trust my results anyway because I have skin in the game. Obviously, I'm the


chunk_text:
  used in this research, they, for example, sent to some, like, an application hosted on AWS, beyond one of the WAFs there, some payload that is, you know, a command injection payload for, like, PHP applications. But what if your app is not really PHP? Then they marked it as, you know, false positive or whatever. Like, you're not detecting this. And is that, you know, good? Probably not. And if you would, like, post or, like, publish your research, like, how did you do this? Then people can also, you know, improve. Yeah. Point taken. I mean, Tuomas Markon should have published the test scripts he used for this. He didn't. The reason I'm using this is because he's third party. I can reproduce this. I'm getting more or less the same results for these tests. But I'm obviously biased. Nobody would trust my numbers. So at least he's third party. Actually, if you reproduce this, you can also... No, I'm biased. I mean, I'm the wrong person to do that research. You should do the research and publish your results. I mean, nobody will trust my results anyway because I have skin in the game. Obviously, I'm the wrong person. But somebody should do this properly. He just happens to be the only guy who did a comparison. And I think, I mean, Gartner and Forrester should do that research. I mean, that's what they're selling anyway. The last thing is a comparison of the performance of the products that Gartner and Forrester are covering. They're looking at nice dashboards and stuff and machine learning, artificial intelligence. But ability to detect an eschol injection is no criteria for them. I think they should do that research. Thanks. Any other questions or remarks? Yeah, there is one. Thank you. I mean, if I'm not getting any bad feedback for this presentation, then I will be disappointed. I wasn't aggressive enough. I got this feeling after this presentation that we cannot trust commercial vendors somehow. Because I'm also working as a security engineer and we constantly check for new vendors. Starting from WAF to static code analysis, dynamic code analysis. And, you know, there are different vendors and they try to sell you like, oh, how great we are. But here we see that it's always the money is a huge incentive. And how or what would be the good


chunk_text:
  wrong person. But somebody should do this properly. He just happens to be the only guy who did a comparison. And I think, I mean, Gartner and Forrester should do that research. I mean, that's what they're selling anyway. The last thing is a comparison of the performance of the products that Gartner and Forrester are covering. They're looking at nice dashboards and stuff and machine learning, artificial intelligence. But ability to detect an eschol injection is no criteria for them. I think they should do that research. Thanks. Any other questions or remarks? Yeah, there is one. Thank you. I mean, if I'm not getting any bad feedback for this presentation, then I will be disappointed. I wasn't aggressive enough. I got this feeling after this presentation that we cannot trust commercial vendors somehow. Because I'm also working as a security engineer and we constantly check for new vendors. Starting from WAF to static code analysis, dynamic code analysis. And, you know, there are different vendors and they try to sell you like, oh, how great we are. But here we see that it's always the money is a huge incentive. And how or what would be the good way to differentiate, to really understand. Are they really interested in giving us a good security product? But rather than just keeping us, you know, on a three-year deal and then after that and so forth. Yeah, honestly, I think I have very strong negative opinions on that. As you've heard, I'm an open source enthusiast. And I'm always in favor of make up your mind yourself, get knowledge. And then you look at the product, evaluate them. But maybe not everybody can do that. And maybe it's not as negative as that. But I think what is definitely worthwhile is looking at incentives. Just take a step back and take a couple of minutes. Why are they doing this like that? Why is a spam detection software selling me a license on the number of spam detected? Why is a log platform selling a license based on the lines of logs that they're logging? So their incentive is obviously to log more. But is that security? I think that when you look at these deals and offerings, try to understand their incentives. And then you see, hey, look, no, they have the right thing in mind. And we know


chunk_text:
  way to differentiate, to really understand. Are they really interested in giving us a good security product? But rather than just keeping us, you know, on a three-year deal and then after that and so forth. Yeah, honestly, I think I have very strong negative opinions on that. As you've heard, I'm an open source enthusiast. And I'm always in favor of make up your mind yourself, get knowledge. And then you look at the product, evaluate them. But maybe not everybody can do that. And maybe it's not as negative as that. But I think what is definitely worthwhile is looking at incentives. Just take a step back and take a couple of minutes. Why are they doing this like that? Why is a spam detection software selling me a license on the number of spam detected? Why is a log platform selling a license based on the lines of logs that they're logging? So their incentive is obviously to log more. But is that security? I think that when you look at these deals and offerings, try to understand their incentives. And then you see, hey, look, no, they have the right thing in mind. And we know this company for a couple of years. It's a good partnership. Yes, we do follow up contracts, but we really benefit from this. Or do you see, hey, they just want to sell a license, shiny leaflet. And yeah, I'm probably not the best partner to talk about these questions because I have a very negative view on that. But ultimately, I'm only a security consultant myself. So I have my own incentives to pay my bills. Other questions? Is there a demand for coffee in the room? Oh, yeah, please. I have another comment. So regarding pen testing stuff and the incentive of cash and not giving you a negative feedback because we want to work with you or something like that. An interesting aspect of that is that when you have an adversarial pen test or an audit or something like that. So you as a company pay another company to audit not your software, but another software. And that's a really nice situation to be in, doing such an audit. But that's not an option. Yeah, and I haven't been in that situation. But true, then suddenly the incentives shift. I think that's an interesting proposal. Good.


chunk_text:
  this company for a couple of years. It's a good partnership. Yes, we do follow up contracts, but we really benefit from this. Or do you see, hey, they just want to sell a license, shiny leaflet. And yeah, I'm probably not the best partner to talk about these questions because I have a very negative view on that. But ultimately, I'm only a security consultant myself. So I have my own incentives to pay my bills. Other questions? Is there a demand for coffee in the room? Oh, yeah, please. I have another comment. So regarding pen testing stuff and the incentive of cash and not giving you a negative feedback because we want to work with you or something like that. An interesting aspect of that is that when you have an adversarial pen test or an audit or something like that. So you as a company pay another company to audit not your software, but another software. And that's a really nice situation to be in, doing such an audit. But that's not an option. Yeah, and I haven't been in that situation. But true, then suddenly the incentives shift. I think that's an interesting proposal. Good. So let's go for the coffee then. Thank you for your attention.


chunk_text:
  { "webpage_url": "https://www.youtube.com/watch?v=x609c-MUZ_g", "title": "Breaking And Fixing Azure AD Device Identity Security by Dirk-jan Mollema", "description": "", "channel_url": "https://www.youtube.com/channel/UCGdJZb8OWZPrRK8PuEeoZuA", "duration": 3255, "channel": "scrt.insomnihack", "uploader": "scrt.insomnihack", "upload_date": "20230411" } This text was transcribed using whisper model: large-v2 Good morning, everyone. Happy to see so many of you here. I'm glad to be here. Before we start, a little intro about me. So, my name is Dirk Jemollema. I am from the Netherlands. I've basically been in security for about almost seven years now. And last year, I started my own company where I do a mix of research, consultancy, and also giving training since this year. I've given talks at various conferences, like at DEF CON, et cetera. And most of my research I actually do by writing tools. So, I try to figure out how protocols work, how things all work together. Then I write a tool about it. And I've written several tools on Active Directory and also more on Azure Active Directory the last few years. I have a blog. I have a Twitter account. So, if you want to follow my work, then you can do that there. And so, what we're going to be talking about today is about Azure AD device identities. And it's structured a bit like first explaining what device identities are and why they are important. Then on the technicalities, like how does a device get an identity? How does that flow work? And then we get into all the tokens. So, we'll start talking about primary refresh tokens, how these are protected on modern Windows versions, and all the issues that I found in the implementation, and also how Microsoft resolved these issues. And also some scenarios that I found that could be a different option that you have to abuse this. So, just before we start, a little bit on terminology. So, this talk is really about Azure Active Directory. So, it's not about Azure as a cloud infrastructure provider. It's not about virtual machines, et cetera. But it's really about the Azure AD identity platform. And the identity platform is for Microsoft 365, also Azure Resource Manager, or basically any other app you want to integrate with it. So, this is kind of the follow-up from on-prem Active Directory, but it's really the identity part and not about cloud resources. So, all of this is quite important in the


chunk_text:
  today is about Azure AD device identities. And it's structured a bit like first explaining what device identities are and why they are important. Then on the technicalities, like how does a device get an identity? How does that flow work? And then we get into all the tokens. So, we'll start talking about primary refresh tokens, how these are protected on modern Windows versions, and all the issues that I found in the implementation, and also how Microsoft resolved these issues. And also some scenarios that I found that could be a different option that you have to abuse this. So, just before we start, a little bit on terminology. So, this talk is really about Azure Active Directory. So, it's not about Azure as a cloud infrastructure provider. It's not about virtual machines, et cetera. But it's really about the Azure AD identity platform. And the identity platform is for Microsoft 365, also Azure Resource Manager, or basically any other app you want to integrate with it. So, this is kind of the follow-up from on-prem Active Directory, but it's really the identity part and not about cloud resources. So, all of this is quite important in the whole zero trust architecture, even though it's a bit of a buzzword. If you look at the architectural overview of this, you see that there's some components involved. And of course, on the left top, you see user identities, because this is a very identity-focused security platform. So, user identity is very important with multi-factor authentication and other verification means. But you also see device identities. So, device identities is what we'll be talking about today, and also how they tie in with this whole security policy enforcement and the whole zero trust strategy, basically. So, a device gets an identity when it's either registered or joined to Azure Active Directory. And that can be a large range of devices. So, there's different scenarios, some different differences here. But basically, you could have a mobile operating system or a desktop operating system that is basically registered or joined to Azure Active Directory. And then the device identity exists as an object in Azure Active Directory. And optionally, you can manage this device by using Intune, which is Microsoft's mobile device management solution. Or you can also use some third-party MDM if you want. Now, in Azure AD Security, you have


chunk_text:
  whole zero trust architecture, even though it's a bit of a buzzword. If you look at the architectural overview of this, you see that there's some components involved. And of course, on the left top, you see user identities, because this is a very identity-focused security platform. So, user identity is very important with multi-factor authentication and other verification means. But you also see device identities. So, device identities is what we'll be talking about today, and also how they tie in with this whole security policy enforcement and the whole zero trust strategy, basically. So, a device gets an identity when it's either registered or joined to Azure Active Directory. And that can be a large range of devices. So, there's different scenarios, some different differences here. But basically, you could have a mobile operating system or a desktop operating system that is basically registered or joined to Azure Active Directory. And then the device identity exists as an object in Azure Active Directory. And optionally, you can manage this device by using Intune, which is Microsoft's mobile device management solution. Or you can also use some third-party MDM if you want. Now, in Azure AD Security, you have various options to really secure your infrastructure. And if you're looking at the strictest way that you really want to enforce people in your organization to only sign in from trusted devices, then you'll be joining your device to Azure Active Directory. You make sure that it's managed by an MDM such as Intune. And Intune applies all kinds of policies to devices. So, it manages the settings on the devices, all the configuration of the devices. And by applying the policies and checking if the settings are all correct, the device becomes compliant. And then people use conditional access policies to basically restrict access to resources such as SharePoint to only corporate compliant devices. Of course, in this part, then it's also important to ensure that any device that people use is also really a corporate device. So, we're also looking at that they lock down basically the environment by enforcing that you can only enroll devices that are trusted. So, they were purchased by the company itself and not just any device that you bought somewhere on eBay, for example. So, in Intune, you can set this as enrollment restrictions. And in the scenario that we're researching here,


chunk_text:
  various options to really secure your infrastructure. And if you're looking at the strictest way that you really want to enforce people in your organization to only sign in from trusted devices, then you'll be joining your device to Azure Active Directory. You make sure that it's managed by an MDM such as Intune. And Intune applies all kinds of policies to devices. So, it manages the settings on the devices, all the configuration of the devices. And by applying the policies and checking if the settings are all correct, the device becomes compliant. And then people use conditional access policies to basically restrict access to resources such as SharePoint to only corporate compliant devices. Of course, in this part, then it's also important to ensure that any device that people use is also really a corporate device. So, we're also looking at that they lock down basically the environment by enforcing that you can only enroll devices that are trusted. So, they were purchased by the company itself and not just any device that you bought somewhere on eBay, for example. So, in Intune, you can set this as enrollment restrictions. And in the scenario that we're researching here, Intune has been configured to block any devices that were not part of the organization. So, to give an overview of the research scenario using Windows 10 here. So, Windows 10 or 11. So, modern operating systems. Also, in the setup, we'll be using Autopilot, which is basically a way to ensure that devices are pre-registered already when you purchase them. So, you basically purchase the devices and they automatically become registered in your tenant. And then you know that the device that you purchased is the actual device that's accessing the resource, basically. We're locking down personal devices. So, Intune doesn't allow us to just register or join any device. And we require device compliancy with conditional access. And also, we have a protection of hardware protection of secrets by using a trusted platform module. So, it's basically the most lockdown scenario that you can have. So, when I started this, I had a couple of research questions. And first of all, so how are devices actually joined to Azure AD? And how are the secrets that are involved actually protected by hardware? And of course, if we know how it works, then we can also look at if


chunk_text:
  Intune has been configured to block any devices that were not part of the organization. So, to give an overview of the research scenario using Windows 10 here. So, Windows 10 or 11. So, modern operating systems. Also, in the setup, we'll be using Autopilot, which is basically a way to ensure that devices are pre-registered already when you purchase them. So, you basically purchase the devices and they automatically become registered in your tenant. And then you know that the device that you purchased is the actual device that's accessing the resource, basically. We're locking down personal devices. So, Intune doesn't allow us to just register or join any device. And we require device compliancy with conditional access. And also, we have a protection of hardware protection of secrets by using a trusted platform module. So, it's basically the most lockdown scenario that you can have. So, when I started this, I had a couple of research questions. And first of all, so how are devices actually joined to Azure AD? And how are the secrets that are involved actually protected by hardware? And of course, if we know how it works, then we can also look at if it's possible to extract these secrets anyway, even though they should be hardware protected. And if it's possible to bypass the device compliancy requirement, because of course, as an attacker, that's quite interesting to know. So, let's start by looking at the device join flow. And basically, the device join flow, if you have a device that was pre-registered using autopilot, then you'll typically set it up by when you install Windows, basically. So, it'll ask you for your corporate credentials, and then it will attempt to join it to Azure AD. You can also do this after the device was already set up. But basically, the flow is similar, just a different way to initiate it. So, on this screen, you would sign in with your account. You would do multi-factor authentication to validate that it's really your account. And then in the background, it initiates basically a technical flow. So, the device will generate two public-private key pairs. The first one is the device key, and the second one is the transport key. And these are asymmetric cryptographic keys. And a lot of the communication in the protocol, as we'll see later, is based on these cryptographic keys.


chunk_text:
  it's possible to extract these secrets anyway, even though they should be hardware protected. And if it's possible to bypass the device compliancy requirement, because of course, as an attacker, that's quite interesting to know. So, let's start by looking at the device join flow. And basically, the device join flow, if you have a device that was pre-registered using autopilot, then you'll typically set it up by when you install Windows, basically. So, it'll ask you for your corporate credentials, and then it will attempt to join it to Azure AD. You can also do this after the device was already set up. But basically, the flow is similar, just a different way to initiate it. So, on this screen, you would sign in with your account. You would do multi-factor authentication to validate that it's really your account. And then in the background, it initiates basically a technical flow. So, the device will generate two public-private key pairs. The first one is the device key, and the second one is the transport key. And these are asymmetric cryptographic keys. And a lot of the communication in the protocol, as we'll see later, is based on these cryptographic keys. And the public keys in this instance are sent to Azure AD. And of course, the private part of the key that remains on the device itself. So, Azure AD can basically validate that the device processes the private key by checking it with the public key that's registered. So, when the device registers itself or joins itself, it sends this request. It's just a HTTP post with some JSON in it. So, it's not super complex in this case. And just focusing a bit on the components, the first one we see is an authorization header. So, this is the header that actually contains the access token for the device registration service. So, first, the user requests a token by signing in. And then it sends this request to the device registration service to actually register the device. So, it authenticates the user doing the registration. It sends a certificate signing request. So, basically, the device is asking Azure AD, okay, I have these keys. Can you please issue me a certificate that I can use to identify my device later on? It also sends a transport key. And so, this is the public part of the transport key.


chunk_text:
  And the public keys in this instance are sent to Azure AD. And of course, the private part of the key that remains on the device itself. So, Azure AD can basically validate that the device processes the private key by checking it with the public key that's registered. So, when the device registers itself or joins itself, it sends this request. It's just a HTTP post with some JSON in it. So, it's not super complex in this case. And just focusing a bit on the components, the first one we see is an authorization header. So, this is the header that actually contains the access token for the device registration service. So, first, the user requests a token by signing in. And then it sends this request to the device registration service to actually register the device. So, it authenticates the user doing the registration. It sends a certificate signing request. So, basically, the device is asking Azure AD, okay, I have these keys. Can you please issue me a certificate that I can use to identify my device later on? It also sends a transport key. And so, this is the public part of the transport key. And that's basically a key that's used to provide additional encryption of any communication between Azure AD and the device. It also sends some properties. So, it will say this is the OS version. This is the device type. So, obviously, this will be different depending on the operating system. And it also shows the host name of the device. Those aren't actually validated. So, if you're doing this yourself, you can put here whatever you want. It's just the registration that counts, basically. The join type is in this case zero. So, zero means Azure AD join. There's different values if you're doing an Azure AD join, an Azure AD registration, or a hybrid join, for example. So, these are all different numbers. And lastly, there's this device ticket. And basically, we'll get into that later. But it's not strictly required. So, if you're registering a new device, you can also leave this out. Now, it sends this post request. And then it gets a response from Azure AD. Once again, it's relatively simple. It's just an object containing basically the certificate on the top. So, this is the signed certificate by Azure AD that the device can use


chunk_text:
  And that's basically a key that's used to provide additional encryption of any communication between Azure AD and the device. It also sends some properties. So, it will say this is the OS version. This is the device type. So, obviously, this will be different depending on the operating system. And it also shows the host name of the device. Those aren't actually validated. So, if you're doing this yourself, you can put here whatever you want. It's just the registration that counts, basically. The join type is in this case zero. So, zero means Azure AD join. There's different values if you're doing an Azure AD join, an Azure AD registration, or a hybrid join, for example. So, these are all different numbers. And lastly, there's this device ticket. And basically, we'll get into that later. But it's not strictly required. So, if you're registering a new device, you can also leave this out. Now, it sends this post request. And then it gets a response from Azure AD. Once again, it's relatively simple. It's just an object containing basically the certificate on the top. So, this is the signed certificate by Azure AD that the device can use to authenticate itself. And on the bottom, you also see some membership changes. So, for those who are familiar with Windows and how it structures local groups, this is the security identifier of the local administrators group. And basically, this is saying, okay, you're now joined to Azure AD. So, these users should be local administrators on the device. And it's doing that by passing some security identifiers in there that the device will put in the local administrator group so that Azure AD accounts that have an administrative role can basically manage the device. So, after the device is joined, we can see the status of the device so that you can do with the dsregcmd commandlet. And that basically will show that the device is Azure AD joined. It will also show the device ID. So, that's in the middle here. It will show the thumbprint of the certificate. And also, something that's interesting is that it says the key provider. So, that's where the private key basically is stored of the device certificate. It is the Microsoft Platform CryptoProvider. And it also says TPM protected, yes. So, this basically means if it shows the Platform CryptoProvider is


chunk_text:
  to authenticate itself. And on the bottom, you also see some membership changes. So, for those who are familiar with Windows and how it structures local groups, this is the security identifier of the local administrators group. And basically, this is saying, okay, you're now joined to Azure AD. So, these users should be local administrators on the device. And it's doing that by passing some security identifiers in there that the device will put in the local administrator group so that Azure AD accounts that have an administrative role can basically manage the device. So, after the device is joined, we can see the status of the device so that you can do with the dsregcmd commandlet. And that basically will show that the device is Azure AD joined. It will also show the device ID. So, that's in the middle here. It will show the thumbprint of the certificate. And also, something that's interesting is that it says the key provider. So, that's where the private key basically is stored of the device certificate. It is the Microsoft Platform CryptoProvider. And it also says TPM protected, yes. So, this basically means if it shows the Platform CryptoProvider is that Windows is using the additional security of a trusted platform module to protect this private key. And it's not stored in software somewhere that's attackers could extract it. So, the idea of a trusted platform module is that it's basically the trusted platform module itself. It has some private cryptographic keys. And if you generate keys inside a TPM, so if you generate private keys there, the key never leaves the TPM except in an encrypted format. So, the idea is that even if you like as an attacker has control over the whole operating system, because the private keys are actually stored in the TPM or they're at least tied to the TPM, then you cannot steal these keys. And basically, if the operating system wants to use the keys, it asks the TPM, hey, I want to perform this key operation. So, for example, it wants to perform some encryption or decryption or signing, it asks the TPM to do that for it. And the TPM basically gives back the result to the operating system. So, then even if the operating system itself is compromised, you wouldn't be able to extract these keys because they're TPM protected.


chunk_text:
  that Windows is using the additional security of a trusted platform module to protect this private key. And it's not stored in software somewhere that's attackers could extract it. So, the idea of a trusted platform module is that it's basically the trusted platform module itself. It has some private cryptographic keys. And if you generate keys inside a TPM, so if you generate private keys there, the key never leaves the TPM except in an encrypted format. So, the idea is that even if you like as an attacker has control over the whole operating system, because the private keys are actually stored in the TPM or they're at least tied to the TPM, then you cannot steal these keys. And basically, if the operating system wants to use the keys, it asks the TPM, hey, I want to perform this key operation. So, for example, it wants to perform some encryption or decryption or signing, it asks the TPM to do that for it. And the TPM basically gives back the result to the operating system. So, then even if the operating system itself is compromised, you wouldn't be able to extract these keys because they're TPM protected. And Azure AD uses this quite extensively to protect sensitive key material. So, after the device join, then the user can sign into the device. And basically, it signs in using their Azure AD username and password. And then the Cloud AP module in LSS, which is the module responsible for doing the authentication towards Azure AD, issues a primary refresh token. And a primary refresh token is a specific credential that the device can then use for a single sign-on. So, let's have a look at the flow there. So, the primary refresh token flow starts off by requesting a challenge. So, this is quite common in many of these operations. So, to prevent replay attacks, they always ask for a challenge or a nonce. And the nonce is basically this encrypted string that isn't truly a nonce because you can use it multiple times, but it does expire after five minutes. So, if you do a cryptographic operation with this nonce, then five minutes later, you wouldn't be able to do it anymore because the nonce basically expired. So, it first asks for a nonce, and then it sends some signed data to the server. So, here we


chunk_text:
  And Azure AD uses this quite extensively to protect sensitive key material. So, after the device join, then the user can sign into the device. And basically, it signs in using their Azure AD username and password. And then the Cloud AP module in LSS, which is the module responsible for doing the authentication towards Azure AD, issues a primary refresh token. And a primary refresh token is a specific credential that the device can then use for a single sign-on. So, let's have a look at the flow there. So, the primary refresh token flow starts off by requesting a challenge. So, this is quite common in many of these operations. So, to prevent replay attacks, they always ask for a challenge or a nonce. And the nonce is basically this encrypted string that isn't truly a nonce because you can use it multiple times, but it does expire after five minutes. So, if you do a cryptographic operation with this nonce, then five minutes later, you wouldn't be able to do it anymore because the nonce basically expired. So, it first asks for a nonce, and then it sends some signed data to the server. So, here we see something that looks a lot like a JSON Web Token, and it is a JSON Web Token. So, basically, it's sending a request to Azure AD saying, okay, I want to perform this action, and it's sending that as like a signed request. So, this token is actually signed with the device certificate so that Azure AD can basically authenticate that it's a real device doing the request. And if we look at the data content, because we can just decode it as it's a normal JSON Web Token, we can see that there's first the request nonce. So, it took the request nonce that it received from Azure AD, and it put that in that signed data content. And it also puts the ground type as password with the plain username and password in there. So, basically, when you authenticate on the Windows login screen on an Azure AD joint device, it just hands over that username and password to Azure AD via this basically signed JSON Web Token. All right. So, the reply from Azure AD, if the password is correct, of course, will be this primary refresh token assurance. So, it says refresh token, but this


chunk_text:
  see something that looks a lot like a JSON Web Token, and it is a JSON Web Token. So, basically, it's sending a request to Azure AD saying, okay, I want to perform this action, and it's sending that as like a signed request. So, this token is actually signed with the device certificate so that Azure AD can basically authenticate that it's a real device doing the request. And if we look at the data content, because we can just decode it as it's a normal JSON Web Token, we can see that there's first the request nonce. So, it took the request nonce that it received from Azure AD, and it put that in that signed data content. And it also puts the ground type as password with the plain username and password in there. So, basically, when you authenticate on the Windows login screen on an Azure AD joint device, it just hands over that username and password to Azure AD via this basically signed JSON Web Token. All right. So, the reply from Azure AD, if the password is correct, of course, will be this primary refresh token assurance. So, it says refresh token, but this is actually the primary refresh token that you get. Now, a primary refresh token, the token itself is not super secret because it's sent during various requests. But the thing that makes the primary refresh token very sensitive is especially this session key. So, you always need this cryptographic session key to prove that you actually possess this primary refresh token. So, the session key here is sent in an encrypted format, encrypted with the transport key, so that only the device itself can decrypt this key and then basically obtain the plain text transport key. We also see that it expires in two weeks. It's actually not true. So, primary refresh tokens are valid for 90 days, but it still says two weeks, and Windows also thinks two weeks, but they're, in fact, valid for 90 days. Now, this storage key or this session key at the bottom, that's the interesting part because you need this to actually use the primary refresh token, which makes it very similar to like a Kerberos ticket granting ticket because then you also need the session key to use it. And how this all works with the TPM is also quite interesting. So,


chunk_text:
  is actually the primary refresh token that you get. Now, a primary refresh token, the token itself is not super secret because it's sent during various requests. But the thing that makes the primary refresh token very sensitive is especially this session key. So, you always need this cryptographic session key to prove that you actually possess this primary refresh token. So, the session key here is sent in an encrypted format, encrypted with the transport key, so that only the device itself can decrypt this key and then basically obtain the plain text transport key. We also see that it expires in two weeks. It's actually not true. So, primary refresh tokens are valid for 90 days, but it still says two weeks, and Windows also thinks two weeks, but they're, in fact, valid for 90 days. Now, this storage key or this session key at the bottom, that's the interesting part because you need this to actually use the primary refresh token, which makes it very similar to like a Kerberos ticket granting ticket because then you also need the session key to use it. And how this all works with the TPM is also quite interesting. So, the TPM basically is quite a small chip, so you cannot store 100 keys in there, but the TPM has basically a root storage key that's stored in hardware, and then any additional keys such as storage keys that you can use to store encrypted data, they can be stored on disk, but they're always stored in an encrypted format. So, the transport key that the device generates is actually a storage key in TPM terms, and when Azure AD issues a session key, it will encrypt it with the public part of the transport key. So, basically, that means that you can only decrypt this key in the TPM, and you cannot use it in the OS, so the OS never gets access to the plaintext session key, but it can only be used by loading this transport key, which is a storage key in the TPM, and then asking the TPM to do certain operations with the session key. So, that's basically how it's tied to the TPM over these different requests. So, just to summarize this, there's the private key of the device certificate, there's the transport key, and the session key, and these are all tied


chunk_text:
  the TPM basically is quite a small chip, so you cannot store 100 keys in there, but the TPM has basically a root storage key that's stored in hardware, and then any additional keys such as storage keys that you can use to store encrypted data, they can be stored on disk, but they're always stored in an encrypted format. So, the transport key that the device generates is actually a storage key in TPM terms, and when Azure AD issues a session key, it will encrypt it with the public part of the transport key. So, basically, that means that you can only decrypt this key in the TPM, and you cannot use it in the OS, so the OS never gets access to the plaintext session key, but it can only be used by loading this transport key, which is a storage key in the TPM, and then asking the TPM to do certain operations with the session key. So, that's basically how it's tied to the TPM over these different requests. So, just to summarize this, there's the private key of the device certificate, there's the transport key, and the session key, and these are all tied to the TPM, and the operating system can use those keys, but it shouldn't have access to the plaintext key material. And the operating system then gets this primary refresh token, and that can be used to perform single sign-on to Azure AD resources. So, just looking at how this PRT is used, and what we can do with this from the endpoint, so there's basically quite some attacks that you can do if you are on a system as an attacker that has a primary refresh token. So, similar to how single sign-on works in on-prem AD cases, as a regular user, any process can basically request data for single sign-on purposes. So, it can just say, hey, I want to sign into this app, please give me some single sign-on data. So, a regular user can request to use this primary refresh token by asking LSS nicely, basically. Of course, if you're an administrator, or if you're a system on the device, then you can also dump the LSS memory, and you can also steal the primary refresh token in the session key, but not if it's protected by TPM. So, if there's no TPM, then all the


chunk_text:
  to the TPM, and the operating system can use those keys, but it shouldn't have access to the plaintext key material. And the operating system then gets this primary refresh token, and that can be used to perform single sign-on to Azure AD resources. So, just looking at how this PRT is used, and what we can do with this from the endpoint, so there's basically quite some attacks that you can do if you are on a system as an attacker that has a primary refresh token. So, similar to how single sign-on works in on-prem AD cases, as a regular user, any process can basically request data for single sign-on purposes. So, it can just say, hey, I want to sign into this app, please give me some single sign-on data. So, a regular user can request to use this primary refresh token by asking LSS nicely, basically. Of course, if you're an administrator, or if you're a system on the device, then you can also dump the LSS memory, and you can also steal the primary refresh token in the session key, but not if it's protected by TPM. So, if there's no TPM, then all the things will be stored in software, basically, and you can dump the plain key, but if it's stored in a TPM, then you cannot. Even if there is a TPM, then as administrator, of course, you can still interact with the PRT and the session key, and you can just ask the TPM to basically perform the cryptographic operations for you. So, we'll look at that in a bit as well. There's basically two ways that Windows uses PRTs, maybe more, but there's two ways that I'm aware of. So, for native apps, they'll actually ask the web account manager, which is a token broker component, so that handles all the authentication for the apps, and the apps don't need to worry about it. And this token broker basically passes the request onto LSS, and LSS then will ask for tokens using the primary refresh token. There's also a way to do single sign-on in the browser. So, then the browser will basically ask LSS for a specific header that it can use when authenticating, which is kind of like a PRT cookie. So, it will ask LSS for that cookie, and it will inject that in any authentication flow,


chunk_text:
  things will be stored in software, basically, and you can dump the plain key, but if it's stored in a TPM, then you cannot. Even if there is a TPM, then as administrator, of course, you can still interact with the PRT and the session key, and you can just ask the TPM to basically perform the cryptographic operations for you. So, we'll look at that in a bit as well. There's basically two ways that Windows uses PRTs, maybe more, but there's two ways that I'm aware of. So, for native apps, they'll actually ask the web account manager, which is a token broker component, so that handles all the authentication for the apps, and the apps don't need to worry about it. And this token broker basically passes the request onto LSS, and LSS then will ask for tokens using the primary refresh token. There's also a way to do single sign-on in the browser. So, then the browser will basically ask LSS for a specific header that it can use when authenticating, which is kind of like a PRT cookie. So, it will ask LSS for that cookie, and it will inject that in any authentication flow, so that Azure AD can see, okay, this is a device that has a PRT, so I can use this for single sign-on. So, as attackers, there's multiple ways that we can actually do this. So, there's some links here about two different approaches, one by me, one by another researcher, about how you can request this single sign-on data if you are running on the operating system as a normal user, and then you can steal this single sign-on data to perform the sign-on somewhere else. So, that's as a limited user. And as an admin, I work together with Benjamin Delphi basically to create an update for Mimikatz that can do this. So, here you see that Mimikatz can dump the CloudAP module in LSS and basically extract the primary refresh token from there. So, imagine that we are dumping such a primary refresh token, so now how does the authentication actually look with a primary refresh token? And so, I already mentioned shortly the PRT cookie, so that's what browsers use to do single sign-on to Azure AD with a PRT, and that's basically a JSON web token, so it will contain a header, a body with


chunk_text:
  so that Azure AD can see, okay, this is a device that has a PRT, so I can use this for single sign-on. So, as attackers, there's multiple ways that we can actually do this. So, there's some links here about two different approaches, one by me, one by another researcher, about how you can request this single sign-on data if you are running on the operating system as a normal user, and then you can steal this single sign-on data to perform the sign-on somewhere else. So, that's as a limited user. And as an admin, I work together with Benjamin Delphi basically to create an update for Mimikatz that can do this. So, here you see that Mimikatz can dump the CloudAP module in LSS and basically extract the primary refresh token from there. So, imagine that we are dumping such a primary refresh token, so now how does the authentication actually look with a primary refresh token? And so, I already mentioned shortly the PRT cookie, so that's what browsers use to do single sign-on to Azure AD with a PRT, and that's basically a JSON web token, so it will contain a header, a body with some claims, and a signature to prove that it's authentic basically. And in the header, we see first there's a CTX, so that's basically a cryptographic nonce that it uses. There's also the refresh token itself, so this is the primary refresh token that it's using to authenticate, and then there's also the request nonce. So, once again, this flow uses this request nonce to prevent basically this from being replayed after the nonce has expired. And then, of course, we also have the signature, so it basically uses the session key to calculate a signature, and that's used to sign this JSON web token, and Azure AD can then validate that it was all done correctly. So, the technical parts of this signing flow, so this is the simplified version if it's imagined that it's a software-only key. So, the OS basically generates some random bytes. This is called the context, and it uses this for the cryptographic operations, and it puts these random bytes in a key derivation function together with the session key to calculate a derived key, and this derived key, so this will be a different key for each random context that's generated, is actually


chunk_text:
  some claims, and a signature to prove that it's authentic basically. And in the header, we see first there's a CTX, so that's basically a cryptographic nonce that it uses. There's also the refresh token itself, so this is the primary refresh token that it's using to authenticate, and then there's also the request nonce. So, once again, this flow uses this request nonce to prevent basically this from being replayed after the nonce has expired. And then, of course, we also have the signature, so it basically uses the session key to calculate a signature, and that's used to sign this JSON web token, and Azure AD can then validate that it was all done correctly. So, the technical parts of this signing flow, so this is the simplified version if it's imagined that it's a software-only key. So, the OS basically generates some random bytes. This is called the context, and it uses this for the cryptographic operations, and it puts these random bytes in a key derivation function together with the session key to calculate a derived key, and this derived key, so this will be a different key for each random context that's generated, is actually used together with the primary refresh token with the nonce from Azure AD to create this PRT cookie signature. So, it basically calculates a new key for every request. So, this is the simplified version. Now, if there is a TPM involved, then the flow looks slightly more complicated. So, basically, because the session key is not in the operating system, but it's in the TPM, and only the TPM can basically perform crypto operations with it, then the operating system will generate these random bytes, then it will ask the TPM to do the key derivation, and then the operating system gets this derived key, and that it can combine with the primary refresh token with the nonce to create this PRT cookie. Now, there is a bit of an issue with this. So, if you look carefully, you see that the only thing that goes to the TPM is basically these random bytes, so the cryptographic context. So, if you are an attacker with access to the LSAT process, then you can basically perform like a lot of pre-calculation attacks. So, if I just give a lot of random bytes to the TPM, it will generate a lot


chunk_text:
  used together with the primary refresh token with the nonce from Azure AD to create this PRT cookie signature. So, it basically calculates a new key for every request. So, this is the simplified version. Now, if there is a TPM involved, then the flow looks slightly more complicated. So, basically, because the session key is not in the operating system, but it's in the TPM, and only the TPM can basically perform crypto operations with it, then the operating system will generate these random bytes, then it will ask the TPM to do the key derivation, and then the operating system gets this derived key, and that it can combine with the primary refresh token with the nonce to create this PRT cookie. Now, there is a bit of an issue with this. So, if you look carefully, you see that the only thing that goes to the TPM is basically these random bytes, so the cryptographic context. So, if you are an attacker with access to the LSAT process, then you can basically perform like a lot of pre-calculation attacks. So, if I just give a lot of random bytes to the TPM, it will generate a lot of derived keys for me, and because these derived keys do not expire, they don't have an expiry, they're just based on random bytes, basically, then if I have access to the operating system, I can just pre-calculate this and then use these keys whenever I want without even needing the session key. So, there's quite a flaw in this, and you can use Mimikatz to do this. So, if you load the session key, basically, so in this case, it's protected by the TPM, Mimikatz will generate a context for you, so these random bytes, and then it will give you a derived key, and that derived key is actually used to calculate the signature. So, if we use some tools that are part of my RoadTools framework, we can basically authenticate by using this, creating a PRT cookie ourselves, and then providing the PRT context and the derived key that belongs to it, and then basically generate our own cookies at runtime, and basically, whenever we want, we can authenticate using the single sign-on data without actually needing access to the TPM. So, the TLDR of this, of course, if there is no TPM at all, then you


chunk_text:
  of derived keys for me, and because these derived keys do not expire, they don't have an expiry, they're just based on random bytes, basically, then if I have access to the operating system, I can just pre-calculate this and then use these keys whenever I want without even needing the session key. So, there's quite a flaw in this, and you can use Mimikatz to do this. So, if you load the session key, basically, so in this case, it's protected by the TPM, Mimikatz will generate a context for you, so these random bytes, and then it will give you a derived key, and that derived key is actually used to calculate the signature. So, if we use some tools that are part of my RoadTools framework, we can basically authenticate by using this, creating a PRT cookie ourselves, and then providing the PRT context and the derived key that belongs to it, and then basically generate our own cookies at runtime, and basically, whenever we want, we can authenticate using the single sign-on data without actually needing access to the TPM. So, the TLDR of this, of course, if there is no TPM at all, then you can just steal the primary refresh token, including the session key as an attacker. If there is a TPM, then you could still acquire combinations of contexts and derived keys that could allow you to use the PRT without actually having access to that device. If you want a longer version of this, it's in my blog here. So, there was quite a flaw with this, of course, and Microsoft responded to that by basically changing the behavior of this. So, they made some changes in the whole PRT signing flow. They also changed the storage mechanism, which broke the CloudAP functionality in Mimikatz for a while. Nowadays, it's working again, so you can dump the PRT again in Mimikatz, but you cannot do the new version of the key derivation function. So, if you find blogs about this that are explaining how to do this, then some of them may be outdated and not work anymore. And basically, in the updated PRT structure, we see this added field, which says KDF version 2. So, that uses a new key derivation function to basically calculate these derived keys. And if we look in the documentation, then they basically changed it


chunk_text:
  can just steal the primary refresh token, including the session key as an attacker. If there is a TPM, then you could still acquire combinations of contexts and derived keys that could allow you to use the PRT without actually having access to that device. If you want a longer version of this, it's in my blog here. So, there was quite a flaw with this, of course, and Microsoft responded to that by basically changing the behavior of this. So, they made some changes in the whole PRT signing flow. They also changed the storage mechanism, which broke the CloudAP functionality in Mimikatz for a while. Nowadays, it's working again, so you can dump the PRT again in Mimikatz, but you cannot do the new version of the key derivation function. So, if you find blogs about this that are explaining how to do this, then some of them may be outdated and not work anymore. And basically, in the updated PRT structure, we see this added field, which says KDF version 2. So, that uses a new key derivation function to basically calculate these derived keys. And if we look in the documentation, then they basically changed it so that they don't just put like these random bytes called the context to derive the signing key, but they actually use a sha256 hash of that context plus the body of the JSON web token to calculate this derived key. This is documented. I actually only found out it was documented after I reverse engineered it all, so I could have saved myself some issues here by reading the documentation. But basically, if we update this graph, then now it first takes the nonce from Azure AD, so that's something that expires after a few minutes. It combines that with the primary refresh token, and it uses that to create the JSON web token body. Then it combines that with these random bytes, and then it takes the sha256 hash of this whole thing, and that's called the KDF context. And the KDF context is actually used in the key derivation with the TPM to get this derived key. So, you see that because it now includes this nonce, which is time bound, so it only works for a few minutes, then you can no longer do this pre-calculation and pre-calculate these derived keys to use at later points,


chunk_text:
  so that they don't just put like these random bytes called the context to derive the signing key, but they actually use a sha256 hash of that context plus the body of the JSON web token to calculate this derived key. This is documented. I actually only found out it was documented after I reverse engineered it all, so I could have saved myself some issues here by reading the documentation. But basically, if we update this graph, then now it first takes the nonce from Azure AD, so that's something that expires after a few minutes. It combines that with the primary refresh token, and it uses that to create the JSON web token body. Then it combines that with these random bytes, and then it takes the sha256 hash of this whole thing, and that's called the KDF context. And the KDF context is actually used in the key derivation with the TPM to get this derived key. So, you see that because it now includes this nonce, which is time bound, so it only works for a few minutes, then you can no longer do this pre-calculation and pre-calculate these derived keys to use at later points, because you will always need to have a recent nonce, and the nonces are controlled by Microsoft, so you cannot create those yourself, basically. So, this was patched with this CVE number, and basically the new method prevents you from pre-calculating these derived keys by putting the nonce as part of the key derivation function. And they also prevented you from downgrading the key derivation function, so I assume it's stored somewhere in the PRT, which key derivation function was used when issuing it, so that you cannot simply remove the header and then use the old calculation. So, this is fixed quite well. So, some other scenarios, like there's more stuff we can do. So, of course, there's some downsides to what we've discussed previously. First, you need to be admin on the device, you need to dump LSAS, and dumping LSAS is something that a lot of EDRs are very interested in, so it's quite noisy if you're doing it as an attacker. If it's stored in the TPM, if it's modern security protected, then you can no longer steal it, you can also no longer pre-calculate these things. And of course, if the device is disabled, then


chunk_text:
  because you will always need to have a recent nonce, and the nonces are controlled by Microsoft, so you cannot create those yourself, basically. So, this was patched with this CVE number, and basically the new method prevents you from pre-calculating these derived keys by putting the nonce as part of the key derivation function. And they also prevented you from downgrading the key derivation function, so I assume it's stored somewhere in the PRT, which key derivation function was used when issuing it, so that you cannot simply remove the header and then use the old calculation. So, this is fixed quite well. So, some other scenarios, like there's more stuff we can do. So, of course, there's some downsides to what we've discussed previously. First, you need to be admin on the device, you need to dump LSAS, and dumping LSAS is something that a lot of EDRs are very interested in, so it's quite noisy if you're doing it as an attacker. If it's stored in the TPM, if it's modern security protected, then you can no longer steal it, you can also no longer pre-calculate these things. And of course, if the device is disabled, then any primary refresh tokens from that device are also disabled. So, I wanted to come up with a new way to do this. And basically, it combines some knowledge of the two previous parts. So, I looked at, because we know how the device registration flow works, we can basically emulate that ourselves, and we know how to get our own primary refresh token, and we know how to get access tokens for the user by using single sign-on. So, I thought, okay, so if I can get a single sign-on token from the user session, can I also get a single sign-on token for the device registration service? And it turned out that's possible. So, you can first initialize this single sign-on flow, and that's basically done with a small tool that's called rowtoken. So, you give it the nonce from Azure AD, and then it will calculate this XMS refresh token credential, which is the signed primary refresh token that browsers use. And then you can get a token for the device registration service with rowtx, which is a tool I wrote to exchange tokens, basically. So, you see it asks a token for the DRS. So, DRS


chunk_text:
  any primary refresh tokens from that device are also disabled. So, I wanted to come up with a new way to do this. And basically, it combines some knowledge of the two previous parts. So, I looked at, because we know how the device registration flow works, we can basically emulate that ourselves, and we know how to get our own primary refresh token, and we know how to get access tokens for the user by using single sign-on. So, I thought, okay, so if I can get a single sign-on token from the user session, can I also get a single sign-on token for the device registration service? And it turned out that's possible. So, you can first initialize this single sign-on flow, and that's basically done with a small tool that's called rowtoken. So, you give it the nonce from Azure AD, and then it will calculate this XMS refresh token credential, which is the signed primary refresh token that browsers use. And then you can get a token for the device registration service with rowtx, which is a tool I wrote to exchange tokens, basically. So, you see it asks a token for the DRS. So, DRS stands for device registration service. So, it asks a token for the device registration service, and it uses this PRT cookie to authenticate the request. And this will give you an access token that can be used to register devices. And then you can register the device. So, this is an older proof of concept that I wrote. Nowadays, this is all in rowtx. But basically, this performs the device registration flow, and you see the result here that contains the certificate of the device and also the SIDs. And basically, this gives us a device certificate with private keys that we control because we generated them on the fly. And this certificate can be used to, like, pretend that we are a real device towards Azure AD. Now, once we have this, we can get a primary refresh token. And for that, because it emulates the flow on the sign-in screen, we only basically need the password. So, we send the user's password to Azure AD, and we get back a primary refresh token. We authenticate the request by using the device, and then we get back a primary refresh token and the session key that also belongs to


chunk_text:
  stands for device registration service. So, it asks a token for the device registration service, and it uses this PRT cookie to authenticate the request. And this will give you an access token that can be used to register devices. And then you can register the device. So, this is an older proof of concept that I wrote. Nowadays, this is all in rowtx. But basically, this performs the device registration flow, and you see the result here that contains the certificate of the device and also the SIDs. And basically, this gives us a device certificate with private keys that we control because we generated them on the fly. And this certificate can be used to, like, pretend that we are a real device towards Azure AD. Now, once we have this, we can get a primary refresh token. And for that, because it emulates the flow on the sign-in screen, we only basically need the password. So, we send the user's password to Azure AD, and we get back a primary refresh token. We authenticate the request by using the device, and then we get back a primary refresh token and the session key that also belongs to the primary refresh token. So, this is no longer protected by any DPMs, but we now have this on disk, basically. So, we can use this PRT and this session key to sign in. So, this is just to get an access token. There's also features in RoadTX that allow you to use this in an interactive flow in a browser. But basically, we authenticate with this PRT. And if we look at the token, then we see that we got an access token, which not only contains the password claim, but also the MFA claim. So, that's quite an interesting scenario because we never did any MFA for this user, but just because we registered, basically, the device with a token that already included an MFA claim, because the token originated from a device on which MFA was done, then the tokens we get after with our registered PRT also contain an MFA claim. So, this is basically a way to extract these tokens and to steal tokens with an MFA claim without ever doing MFA for the user. So, to summarize this, we can request single sign-on tokens from a user session without any admin privileges. These access


chunk_text:
  the primary refresh token. So, this is no longer protected by any DPMs, but we now have this on disk, basically. So, we can use this PRT and this session key to sign in. So, this is just to get an access token. There's also features in RoadTX that allow you to use this in an interactive flow in a browser. But basically, we authenticate with this PRT. And if we look at the token, then we see that we got an access token, which not only contains the password claim, but also the MFA claim. So, that's quite an interesting scenario because we never did any MFA for this user, but just because we registered, basically, the device with a token that already included an MFA claim, because the token originated from a device on which MFA was done, then the tokens we get after with our registered PRT also contain an MFA claim. So, this is basically a way to extract these tokens and to steal tokens with an MFA claim without ever doing MFA for the user. So, to summarize this, we can request single sign-on tokens from a user session without any admin privileges. These access tokens contain an MFA claim. If you use this to register a device, then any PRTs you request with that device that belong to the same user also inherit this MFA claim. And you only need the passwords to get a PRT. If you have ADFS, you could probably also do this only using single sign-on, but that's not something I've researched. So, basically, this is a free MFA upgrade from your token because we only use the password to actually get the PRT. So, this also has some upsides and downsides. So, because we registered a new device, it's basically a separate device. It's not a real device. But if the old device is disabled, then the PRT from our new device will still work. Of course, the downside is that this requires permissions to register devices or join devices, which is not always allowed. You can lock that down in Azure AD. And this also doesn't mean that the device will be allowed to enroll in Intune because we registered a new device, which is not known as a corporate device. And if we need to access anything that's protected by conditional access to only allow compliant devices,


chunk_text:
  tokens contain an MFA claim. If you use this to register a device, then any PRTs you request with that device that belong to the same user also inherit this MFA claim. And you only need the passwords to get a PRT. If you have ADFS, you could probably also do this only using single sign-on, but that's not something I've researched. So, basically, this is a free MFA upgrade from your token because we only use the password to actually get the PRT. So, this also has some upsides and downsides. So, because we registered a new device, it's basically a separate device. It's not a real device. But if the old device is disabled, then the PRT from our new device will still work. Of course, the downside is that this requires permissions to register devices or join devices, which is not always allowed. You can lock that down in Azure AD. And this also doesn't mean that the device will be allowed to enroll in Intune because we registered a new device, which is not known as a corporate device. And if we need to access anything that's protected by conditional access to only allow compliant devices, then basically it won't work. So, this doesn't cover our full scenario goals. So, for the last step, looking at bypassing Intune restrictions. And basically, this is another dive in how these Intune things work. So, something to consider is when you enroll a device in MDM, that's actually a different process from registering the device itself. So, if you look at this from a user perspective, they only enter their credentials once and then the device will first register itself and then enroll itself in MDM. But that's not how it works under the hood. So, it's two different registration requests. And even if you block non-corporate devices in Intune, you could still register or join the device in Azure AD because this is something that's separately controlled. If you do this in Windows, then if Windows sees an error, so if it sees it's not allowed to enroll the device in Intune, it'll automatically roll back the Azure AD registration as well. But if you do it yourself, you can just do those separately and you can still register or join a device even if you're not allowed to enroll it in Intune. Of course, this doesn't really


chunk_text:
  then basically it won't work. So, this doesn't cover our full scenario goals. So, for the last step, looking at bypassing Intune restrictions. And basically, this is another dive in how these Intune things work. So, something to consider is when you enroll a device in MDM, that's actually a different process from registering the device itself. So, if you look at this from a user perspective, they only enter their credentials once and then the device will first register itself and then enroll itself in MDM. But that's not how it works under the hood. So, it's two different registration requests. And even if you block non-corporate devices in Intune, you could still register or join the device in Azure AD because this is something that's separately controlled. If you do this in Windows, then if Windows sees an error, so if it sees it's not allowed to enroll the device in Intune, it'll automatically roll back the Azure AD registration as well. But if you do it yourself, you can just do those separately and you can still register or join a device even if you're not allowed to enroll it in Intune. Of course, this doesn't really gain you anything because the goal of conditional access... The goal of this was to bypass conditional access policy to require a compliant device state. You only get that by enrolling it in Intune. So, by just joining the device, we cannot meet this compliancy. So, some more digging into this. So, there's a few interesting parts here. So, if you're using Autopilot, basically the devices are already registered in your tenant based on their hardware properties. And if you wipe those devices and register them again, then it will reuse like the same device ID in Azure AD. So, my question was so how does Azure AD actually know that this is the same device? Like if I wipe this laptop and I register it again, it will use the same device. It will override the same device and it will know that this was like a pre-registered device. So, that's the part that comes into play is this registration request again and especially this part on the bottom. So, this is actually a device ticket that proves to Azure AD which device it is. So, of course, I was wondering, okay, how is this structured and where does


chunk_text:
  gain you anything because the goal of conditional access... The goal of this was to bypass conditional access policy to require a compliant device state. You only get that by enrolling it in Intune. So, by just joining the device, we cannot meet this compliancy. So, some more digging into this. So, there's a few interesting parts here. So, if you're using Autopilot, basically the devices are already registered in your tenant based on their hardware properties. And if you wipe those devices and register them again, then it will reuse like the same device ID in Azure AD. So, my question was so how does Azure AD actually know that this is the same device? Like if I wipe this laptop and I register it again, it will use the same device. It will override the same device and it will know that this was like a pre-registered device. So, that's the part that comes into play is this registration request again and especially this part on the bottom. So, this is actually a device ticket that proves to Azure AD which device it is. So, of course, I was wondering, okay, how is this structured and where does it get that? So, how does it do that? And also, like if you reuse the same parameter, it will always override the device. So, this is apparently something that is really tied to the device and it knows the device hardware based on this ticket. It does seem to expire after a certain period of time. So, it's not something that's like fixed or something. So, it is some device tickets for your device, but it's not related to Azure AD basically. So, we did some reversing of this which was quite annoying because the registration flow is a web-based app. It uses the WinRT framework and it also calls WinRT APIs which means that it's doing some inter-process communication. So, it actually spawns a different process to do the registration than the one that does the app. But eventually, there will be a DLL host that actually spawns with the DLL in there that contains the registration logic. So, once I found that out, I just looked into reversing this and I basically found this string. So, it's probably a bit small to read, but you can see that it instantiates a class of the Windows security authentication


chunk_text:
  it get that? So, how does it do that? And also, like if you reuse the same parameter, it will always override the device. So, this is apparently something that is really tied to the device and it knows the device hardware based on this ticket. It does seem to expire after a certain period of time. So, it's not something that's like fixed or something. So, it is some device tickets for your device, but it's not related to Azure AD basically. So, we did some reversing of this which was quite annoying because the registration flow is a web-based app. It uses the WinRT framework and it also calls WinRT APIs which means that it's doing some inter-process communication. So, it actually spawns a different process to do the registration than the one that does the app. But eventually, there will be a DLL host that actually spawns with the DLL in there that contains the registration logic. So, once I found that out, I just looked into reversing this and I basically found this string. So, it's probably a bit small to read, but you can see that it instantiates a class of the Windows security authentication online ID, online ID service tickets request. I was like, okay, what is this? I've never heard of this, but it's documented. So, the online ID service ticket request basically to sign in a user or device to the live endpoint. So, that's also what you use for Microsoft accounts. So, apparently, your device has its own Microsoft account that it uses separately from any Azure AD or Intune registrations and it uses these APIs to ask tickets for that account. So, your device has its own Microsoft account. This is also what it uses for licensing checks and such. And when device-specific authentication is needed, so when it does the device identity authentication, then it uses these tickets to prove its identity. These tickets are cached in the registry, actually in the current user registry hive. So, any tickets that were used in your user session, you can query from here. They are DPAPI protected, but the protection is tied to the machine. So, any user on the machine can decrypt them. And basically, you can just enumerate these using PowerShell. So, by enumerating these keys and then basically decrypting them, we find this token. So, here we see


chunk_text:
  online ID, online ID service tickets request. I was like, okay, what is this? I've never heard of this, but it's documented. So, the online ID service ticket request basically to sign in a user or device to the live endpoint. So, that's also what you use for Microsoft accounts. So, apparently, your device has its own Microsoft account that it uses separately from any Azure AD or Intune registrations and it uses these APIs to ask tickets for that account. So, your device has its own Microsoft account. This is also what it uses for licensing checks and such. And when device-specific authentication is needed, so when it does the device identity authentication, then it uses these tickets to prove its identity. These tickets are cached in the registry, actually in the current user registry hive. So, any tickets that were used in your user session, you can query from here. They are DPAPI protected, but the protection is tied to the machine. So, any user on the machine can decrypt them. And basically, you can just enumerate these using PowerShell. So, by enumerating these keys and then basically decrypting them, we find this token. So, here we see also the token scope is the enterprise registration with Windows.net. So, it's a device ticket for the device registration. Now, of course, these cache tickets are nice, but like I said, they expire after a certain time. So, if the device registration was done a long time ago, then these cache tickets are useless. So, I did some more reversing and found the exact WinRT API calls that were needed. So, there's this endpoint involved that it asks the ticket for, and there's the app GUID that's used for registration. And basically, when you know all these parameters, you can write a very simple .NET program, which basically creates such an online ID system authenticator request. And it asks a ticket for the enterprise registration of Windows.net. It sets the app ID to the same ID that Windows uses, basically. And then it requests this ticket from the system, and it prints it to the console. So, if we compile this and we do this, then basically we get this device ticket that contains the device identity. And with this ticket, if we actually do the registration again, but now provide this device ticket, then we will actually override the


chunk_text:
  also the token scope is the enterprise registration with Windows.net. So, it's a device ticket for the device registration. Now, of course, these cache tickets are nice, but like I said, they expire after a certain time. So, if the device registration was done a long time ago, then these cache tickets are useless. So, I did some more reversing and found the exact WinRT API calls that were needed. So, there's this endpoint involved that it asks the ticket for, and there's the app GUID that's used for registration. And basically, when you know all these parameters, you can write a very simple .NET program, which basically creates such an online ID system authenticator request. And it asks a ticket for the enterprise registration of Windows.net. It sets the app ID to the same ID that Windows uses, basically. And then it requests this ticket from the system, and it prints it to the console. So, if we compile this and we do this, then basically we get this device ticket that contains the device identity. And with this ticket, if we actually do the registration again, but now provide this device ticket, then we will actually override the old device. So, if you use the Rotex device module and you specify the device ticket that we obtained from the Windows host, basically, then it will override this old device. And now we have a device that we control, which has the same ID, but we have the keys actually stored on disk instead of them being TPM protected. So, you also see that, like here, a registered device, and you see the device ID is the same device ID than what Windows reports on the old device. So, you can just request a device ticket from a low-privileged session, and then override the old device. And the nice part is that the device basically retains the original properties. So, before they basically changed this, then it used to be that even if you overrode the device, it was still considered compliant. And any sign-ins you did using this stolen device identity would also be considered compliant. So, you could steal that, and basically, the original device was no longer working properly, but you have the device as an attacker, and the Azure AD still considers any tokens issued to that device compliant. So, just to summarize this, any


chunk_text:
  old device. So, if you use the Rotex device module and you specify the device ticket that we obtained from the Windows host, basically, then it will override this old device. And now we have a device that we control, which has the same ID, but we have the keys actually stored on disk instead of them being TPM protected. So, you also see that, like here, a registered device, and you see the device ID is the same device ID than what Windows reports on the old device. So, you can just request a device ticket from a low-privileged session, and then override the old device. And the nice part is that the device basically retains the original properties. So, before they basically changed this, then it used to be that even if you overrode the device, it was still considered compliant. And any sign-ins you did using this stolen device identity would also be considered compliant. So, you could steal that, and basically, the original device was no longer working properly, but you have the device as an attacker, and the Azure AD still considers any tokens issued to that device compliant. So, just to summarize this, any user that had a session on this device could request this device ticket, which could be used to override the device. If it was pre-registered using autopilots, then it will do this hardware matching and actually override the device based on the ticket. It overrides the device in Azure AD using, of course, our own cryptographic keys. So, if we request any tickets after that, then we can just decrypt the session key using our own transport key instead of it being tied to a TPM. So, in this case, you don't need to steal any PRTs from LSS. You don't need to worry about TPMs, and actually, you don't need any administrative privileges at all because you can just request this from a normal user session. Some bonus features. So, you could use any identity to override the device. It didn't necessarily have to be the original user that was using the device. The device ticket also stays valid after wiping the device. So, if I steal the device ticket, I wipe the device because it was compromised, for example, then I could still use that device ticket somewhere else. And if you use a new account to override


chunk_text:
  user that had a session on this device could request this device ticket, which could be used to override the device. If it was pre-registered using autopilots, then it will do this hardware matching and actually override the device based on the ticket. It overrides the device in Azure AD using, of course, our own cryptographic keys. So, if we request any tickets after that, then we can just decrypt the session key using our own transport key instead of it being tied to a TPM. So, in this case, you don't need to steal any PRTs from LSS. You don't need to worry about TPMs, and actually, you don't need any administrative privileges at all because you can just request this from a normal user session. Some bonus features. So, you could use any identity to override the device. It didn't necessarily have to be the original user that was using the device. The device ticket also stays valid after wiping the device. So, if I steal the device ticket, I wipe the device because it was compromised, for example, then I could still use that device ticket somewhere else. And if you use a new account to override the device, it became the new device owner. So, then it can actually recover the BitLocker keys that are stored in the device because, by default, users can read BitLocker keys for devices that they own. So, if the BitLocker keys were stored in Azure AD, then you could read this after you overrode the device. You could read this after you overrode the device. And also, the nice part is that the original device still kept its link to Intune. So, like I said, the registration in Azure AD and Intune is separate. So, if you overrode the Azure AD part, the device would still be talking to Intune and would still report its compliancy. And basically, Intune would keep reporting the device as compliant to Azure AD even though the device was no longer the original one, but was now one controlled by the attacker. And the device retained its compliancy status. So, just to the complete chain, you could do a few commands in a low-privilege session of the victim. You could request a single sign-on token to register a new device. You could request a device ticket to override the legitimate compliant device. And then you


