# Media

url: https://www.youtube.com/watch?v=p8NMZoPM8Yc
type: video
author: DEFCONConference
ingestion_date: 2024-11-09
transcription_model: large-v2
chunking_status: completed
title:
DEF CON 32 - Speed Bumps and Speed HacksP: Adventures in Car Mfg Security - Paulo Silva, David Sopas

content:
{
  "webpage_url": "https://www.youtube.com/watch?v=p8NMZoPM8Yc",
  "title": "DEF CON 32 - Speed Bumps and Speed HacksP: Adventures in Car Mfg Security - Paulo Silva, David Sopas",
  "description": "Since the first car hit the road, manufacturers have been obsessed with safety. But hey, as AI wisely points out, determining the absolute 'safest' car can be as subjective as trying to decide on the best pizza topping! So, we decided to shift gears\u2026 into the world of car manufacturers' (application) security.\n\nIn this talk, get ready for a wild ride as we unveil the security findings from our research, affecting at least eleven major car manufacturers. Buckle up, folks, because your favorite brand might be on our list\u2014along with your personal info!\n\nBring popcorn to watch some proof-of-concept videos.\n\nWith this talk we aim to demonstrate:\n\nthat modern Web Applications are still affected by old/traditional vulnerabilities,\nhow security issues can be chained together to build real attacks/demonstrate impact,\nhighlight a common pattern on running unpatched third-party software,\nhow organizations benefit from a responsible disclosure policy.",
  "channel_url": "https://www.youtube.com/channel/UC6Om9kAkl32dWlDSNlDS9Iw",
  "duration": 1890,
  "channel": "DEFCONConference",
  "uploader": "DEFCONConference",
  "upload_date": "20241016"
}

4.18s - 8.94s | This text was transcribed using whisper model: large-v2

 Okay, first of all, thank you for joining. It's a great pleasure to be here today
9.86s - 12.94s |  I'm not sure how many of you have a driving license, but I
13.46s - 17.74s |  Kindly ask you to fasten your seatbelts because this is gonna be a quick one
18.74s - 22.70s |  My name is Paul Silva. I'm a security researcher and pen tester at chart 49
23.42s - 27.50s |  But I've started as a software developer. I did it for more than 15 years
27.50s - 30.50s |  What was plenty of time to do all sort of mistakes?
30.98s - 36.46s |  I'm also co-author of something that you may have heard about like the last BPI security top 10
37.42s - 41.78s |  But the topic here today is another one with us. We we have David
43.10s - 49.06s |  Yeah, so my name is David Sopas. I'm a security researcher for a long time
50.14s - 55.22s |  more than 15 years, I especially like research and
55.78s - 56.82s |  And
56.82s - 61.06s |  Focusing also on I'll have open testing. I usually I
62.14s - 65.58s |  Already spoke on events like RSA
66.14s - 72.86s |  Defcon this is my first time here in Vegas because the first time I spoke on on Defcon
72.86s - 79.54s |  It was unsafe mode. So I was at home not that fun, but still I did the shots with my friends
79.54s - 81.38s |  So shut the knob
81.42s - 86.70s |  And besides and props to besides Lisbon which some guys over here I
87.22s - 91.86s |  Did some API security methodology. I don't know if you guys know it mind API
92.46s - 94.46s |  It's part of my open source work
94.98s - 96.98s |  And that's it. Let's start
98.54s - 104.34s |  Before some of the work that we the research that we were presenting I
105.02s - 107.62s |  Would like to say that this is not
108.14s - 113.78s |  We will not teach you how to act web applications or APIs all the targets
114.50s - 120.66s |  manufacturers you presented have a bug bounty program responsible disclosure program or whatever, so
122.10s - 123.70s |  This is
123.70s - 130.70s |  Everything was reported to the organizations. Some of them were like late to respond but in the end we
130.70s - 132.66s |  We did it
132.66s - 140.82s |  We'll focus this presentation with nine brands. I hope that at least
141.94s - 146.02s |  One of them. It's your your car manufacturer
147.50s - 149.58s |  But let's start with this one
150.10s - 155.58s |  I don't know if you know this this brand, but we will not mention it. It's a vulnerable
156.38s - 161.98s |  outdated component, but before that I would like to tell you a little bit about the
162.70s - 164.82s |  API
164.82s - 170.10s |  In the API world, which means you have a lot of API architectures
170.10s - 173.74s |  I'm talking about the most popular ones like rest graph QL
174.94s - 177.98s |  RPC you name it all of them
178.74s - 185.02s |  Make sense to to mention that most of the abilities that were present was focused on these architectures
186.02s - 188.02s |  Open
189.46s - 191.58s |  API is one of the most common ones
192.58s - 194.58s |  Usually it's like
195.02s - 196.86s |  Connected in a way to
196.86s - 200.94s |  Swagger, I hope anyone knows swagger. I believe everyone knows
201.78s - 203.50s |  so
203.50s - 205.50s |  the problem is that
206.70s - 213.26s |  It helps out developers and even pen testers in researchers because it documents everything and
214.10s - 220.62s |  You it will help you not to make any mistakes and leave the documentation open, but
221.54s - 223.50s |  one of the things that
223.50s - 227.18s |  Because usually individuals like shiny things and web interfaces
227.74s - 231.10s |  Swagger UI is one of them. It's the most popular one and
231.82s - 236.50s |  Usually is left public of available. This is the example of
237.14s - 242.78s |  Swagger pet store, but like every software it also has vulnerabilities
242.90s - 244.90s |  so
244.94s - 247.14s |  Especially the vulnerability from
248.50s - 253.02s |  3.71 which is the cross-site scripting which allows you to put
255.22s - 257.02s |  Specially crafted
257.02s - 258.26s |  JSON
258.26s - 265.58s |  File that you can control and put cross-site scripting payload and you can do a lot of a lot of stuff with that
266.58s - 268.98s |  This is the classic example of a
269.62s - 274.06s |  Swagger UI which is presented on the brand that I showed you
274.70s - 276.70s |  this is come to a reminder that
277.46s - 279.46s |  if you use
280.22s - 283.62s |  or use third-party software, please
284.42s - 291.58s |  Keep it updated because it can be a way that attackers can achieve to get into your organization
293.22s - 295.22s |  This brand which is
296.14s - 297.14s |  I
297.14s - 303.46s |  I'm guessing nobody here has this brand in their garage if so kudos
304.42s - 308.38s |  It has a vulnerable outdated component this time. It's not
309.98s - 315.02s |  Swagger but we we found that a subdomain on this company
316.06s - 318.06s |  have
318.14s - 325.42s |  Immediately cuts all attention because we saw form and we like okay form let's try to approach this the security
325.42s - 327.42s |  research way
327.70s - 333.38s |  We immediately went to the inspector because it's faster than trying to do some proxy stuff
333.98s - 340.70s |  We noticed that it was a wordpress because it has a lot of tips like WP content WP plugins
340.82s - 345.22s |  You can use for example for enumeration tools like WP scan
345.22s - 349.02s |  But in this case, we immediately saw that the plugin that we wanted
349.62s - 353.66s |  And we could see if any vulnerability could be in place
354.26s - 355.98s |  And it was
355.98s - 358.66s |  this subdomain from from this manufacturer
359.42s - 360.90s |  uses
360.90s - 362.90s |  Use double three total cash
363.62s - 370.22s |  it was a vulnerable version and this was very cool for us to test especially for
371.98s - 378.78s |  These manufacturer that I don't know if they could like offers a good reward like a car
379.78s - 382.34s |  This is the CVE from the plugin
383.06s - 385.06s |  It's very cool. So it's vulnerable to
385.62s - 389.14s |  You could read arbitrary files from the subdomain
390.02s - 396.70s |  And the main goal here. Okay, let's to create the POC and to respect the responsible disclosure program
396.70s - 403.90s |  Let's like grab WP config and drop the database details and tell them way. This is the your database
403.98s - 408.18s |  Okay, so at least the credentials and that's what we did
408.26s - 410.26s |  so this is the
410.66s - 415.62s |  Exploit the first thing that we wanted to know and to play around with the file path
416.02s - 422.70s |  Was to see what was the operating system running. So we do the curl, but the full
423.90s - 430.30s |  URL the the plug-in place the vulnerable parameter was which was the subscriber URL
430.46s - 435.30s |  So we put etc issue just to see if it's it's running Linux and it was
435.74s - 442.46s |  The the response it's below so it's Ubuntu not latest version, but it doesn't matter but they give us
442.82s - 449.98s |  With this operating system gave us clues to understand the path and the configuration files for other stuff
450.02s - 456.06s |  So moving forward. So now we noticed that is running virtual hosts
456.06s - 458.14s |  So we did the past golf
458.46s - 463.30s |  It's like going through to see how we can get the full path
464.06s - 466.06s |  Internal path for
466.22s - 467.98s |  the WordPress
467.98s - 473.66s |  Installation and get the WP config sites enabled. We didn't we we needed to know the
474.54s - 477.90s |  Domain itself. So we tried it and in the end after
478.66s - 481.98s |  Like experience because sites enable usually it's sub domain
481.98s - 486.54s |  We tried that on the subscriber URL and we got to document roots
486.54s - 490.50s |  Which was this one and with that we just try it
491.30s - 494.50s |  Just WP config and we got to database from this
495.18s - 496.46s |  high-level
496.46s - 500.98s |  Manufacturer that in the end decided to tell us good work
501.74s - 503.74s |  And it fix it
503.94s - 506.30s |  another brand, this is my favorite because
507.30s - 510.42s |  It's a brand that I'm very close to
511.34s - 515.58s |  And we found out some cool stuff like bola reflected
516.30s - 518.30s |  Cross-site scripting and again
518.70s - 520.70s |  components that are vulnerable but
521.46s - 526.10s |  It sees it's very hard sometimes to get VINs from cars as you can see
526.98s - 528.82s |  Usually it's placed on the windshield
528.82s - 534.74s |  But you can also try other state out other things like going to Google and search for VINs
535.34s - 537.34s |  And we did that
538.38s - 542.30s |  Not only that we decided okay, let's see if we can create
543.30s - 544.50s |  on
544.50s - 546.74s |  this manufacturer account dashboard
549.14s - 554.86s |  Add these cars without any confirmation. So we create a persona, which is g1
555.62s - 561.94s |  And we added a bunch of cars like you can see on the picture. Sorry. It's in Portuguese
561.94s - 569.22s |  We are from Portugal. So it's automatic translated, but it's it's it works in every language. So we added these cars
569.98s - 573.46s |  on other brands and I can I can tell you that
574.50s - 577.58s |  Like on the nine manufacturers that we tried
578.26s - 580.26s |  at least three
580.58s - 584.54s |  You need to confirm on the infotainment talk inside the car
585.02s - 591.54s |  To have the car added to your dashboard account on the car on the car website in this case
591.74s - 596.14s |  like you can add it and get information from this is the
598.18s - 602.30s |  Information you added without any interaction from the user inside the car
602.38s - 608.74s |  So that's a big issue some brands tells you that no this it's normal, but you can see in
609.54s - 611.54s |  After a while, that's it's not
611.66s - 616.46s |  so on these information that we get on do you why dashboard
617.10s - 621.60s |  You know that your you have an outdated car, for example
622.62s - 626.54s |  You can see the versions from the infotainment system
628.06s - 629.98s |  You can see
629.98s - 633.78s |  Issues related to guarantees if you paid for
636.14s - 640.26s |  Years for to have more guarantee on your vehicle
641.30s - 647.02s |  Also more information that what you need to do when you go to an inspection car
648.74s - 655.14s |  But one thing that we noticed is that is that we could get more information on the API
655.34s - 659.04s |  That was not present on the UI and this is cool
659.24s - 661.24s |  okay, so
661.36s - 666.80s |  We try that a in by the way with this endpoint, which is already fixed. So disclaimer
668.56s - 674.12s |  We can you don't need even to be authenticated this is like public information
675.88s - 679.22s |  On the API we can get over meter fuel
679.36s - 685.20s |  But is battery status if it's electric or life tire pressure for each wheel
685.52s - 689.36s |  For each tire and that's after you can see the screenshot on the API
689.36s - 693.80s |  This information is not present on the UI by the way, so we get that
694.64s - 696.64s |  It will be cool like to get
697.96s - 705.12s |  For example on this manufacturer for a friend and you can go check out the is been and bet with him
705.12s - 706.52s |  okay, if I
706.52s - 714.04s |  Know the exactly number of kilometers. That's your car as you buy me a beer. So this is win-win. So
716.20s - 721.52s |  Other information, I think that's that's another end point that we found using
722.52s - 729.88s |  Our intersection only using a proxy was like type of vehicle which is in the car. Not that relevant
731.00s - 738.52s |  So model one thing and I put two smileys on Wi-Fi hotspot. Very cool. I love cars with Wi-Fi hotspot
739.12s - 741.88s |  Also, if it's connected to the Internet even better
742.64s - 747.96s |  And as you can see we can do a lot of things especially with Wi-Fi hotspot
749.48s - 752.00s |  We can wiggle wiggle so I
753.04s - 755.04s |  got the hotspot from the
755.56s - 759.48s |  Sorry, the broadcast MAC address. I grabbed it
760.04s - 764.92s |  And go to wiggle and then we get location from the vehicles
765.44s - 771.08s |  And that's very cool because I'm talking about millions of these manufacturer
771.08s - 776.68s |  But it works also for other brands. So this is like crazy
780.08s - 784.60s |  Firmware update is something that we still like a work in progress
786.16s - 793.00s |  Because on this specific request you get information from firmware, but not only that we can download it
793.92s - 799.80s |  Unpack it and what we did is just I mounted on my operating system
799.84s - 803.84s |  I noticed what was the operating system running on the infotainment
803.84s - 811.52s |  It was armaments, but I didn't check for more information so far. I was focused on the API stuff. So I move forward
812.68s - 815.80s |  Recalls, this is something that the manufacturer was like
816.64s - 819.00s |  Tell me that this was an acceptable risk
819.68s - 823.28s |  Again, I don't understand why but I will explain that later
824.28s - 831.28s |  The recalls it's something that the information that it comes is very juicy because especially, you know
831.28s - 833.28s |  if the car was like
834.12s - 839.52s |  Have problems with something and get information about that. But not only that
840.32s - 847.28s |  we noticed that one of the fields returning on the response or this API request was the description part and
848.12s - 849.80s |  it was
849.80s - 856.84s |  Maybe used by an operator a mechanical part. I don't know. I don't know who uses this this this field because
857.80s - 861.12s |  most of the PYY information that we got was
861.92s - 863.92s |  added to this field, so I
864.56s - 866.56s |  find out something like
867.36s - 874.36s |  Please return the car to the home or to the owner at address with his phone and everything. So
875.24s - 877.24s |  this is like
877.48s - 883.36s |  Public information according to the this manufacturer, but it's you can see name
884.36s - 888.12s |  location of the the owner of the car and everything I
889.68s - 893.48s |  Communicated that to the manufacturer. He told me it's normal. So
894.20s - 895.32s |  maybe
895.32s - 901.88s |  It's it's kind of weird to receive sometimes this this kind of responses, but okay, so what?
904.64s - 906.64s |  With all this information together
907.24s - 910.56s |  We can do a lot of stuff. Okay, we will data we can track devices
911.16s - 914.76s |  to pinpoint locations we can place for example some
915.88s - 920.12s |  antennas on specific spots and try to track vehicles
921.44s - 924.40s |  Analyzing over meter. For example, you you can create
925.04s - 930.80s |  You can know routines from from the the the habits from the victim for example
931.44s - 935.64s |  Insights from from everything how much fuel and you can calculate
936.44s - 944.40s |  When the the person needs to go refuel or something like that a lot of stuff recall information gives you PYY information
944.40s - 946.40s |  so it's it's
946.52s - 953.92s |  It's crazy. But yeah, and by the way, yeah outdated components again, but not only one
954.64s - 958.28s |  if he's like loathing it takes a while because it's
959.32s - 961.32s |  Doesn't work. Okay, it's
961.92s - 963.44s |  This is all for
963.44s - 967.71s |  the same manufacturer halfway and
970.67s - 972.67s |  Still but
973.35s - 975.35s |  That's it
975.87s - 983.59s |  They control the documentation and the third-party software very well, so I think that's it I think to Paul
985.07s - 990.83s |  Okay from now on it's with me I believe the point here we this is not a full assessment on car manufacturers
990.83s - 993.35s |  We are not trying to assess car safety
993.87s - 1000.51s |  This was like a free time to assess how car manufacturers handle their digital asset security, so
1001.11s - 1007.59s |  We were looking for a few bugs that we could chain together and prove them the impact. So this is why you're
1008.55s - 1012.23s |  You're gonna see a lot cross-site scripting that we can chain with something else
1012.83s - 1016.35s |  And prove them the impact and get the system patch patched
1016.99s - 1020.71s |  So another manufacturer, I'm not sure how many of you are familiar
1021.55s - 1026.55s |  With the return URI query string perimeter, but when I see one I get excited
1027.19s - 1032.79s |  Probably I'm gonna get an open redirect, but then I have an additional feeling
1032.79s - 1038.83s |  maybe I can get the cross-site scripting which is even better and all we have to do is change the URI scheme and
1039.91s - 1045.67s |  change from HTTP or HTTPS to JavaScript and we get a cross-site scripting easily and
1046.67s - 1051.03s |  Chances are that we can access cookies or local storage using our
1052.11s - 1054.11s |  cross-site scripting and
1054.23s - 1058.99s |  Probably the ALT token is in there and we can use it to exfiltrate the token
1058.99s - 1062.87s |  So this is how we prove the the impact here. It's a quick demo
1063.55s - 1065.79s |  So you can use it on a phishing
1066.87s - 1068.15s |  email
1068.15s - 1074.63s |  With the malicious payload and as soon as as soon as the victim logins we get the ALT token
1074.63s - 1081.51s |  So this way they it's not only a cross-site scripting that probably no one wants are willing to fill to fix
1082.19s - 1089.63s |  There's a there's the impact here to exfiltrate PII and this is a bigger deal for the manufacturer
1091.91s - 1099.19s |  But usually you also find let's call it less interesting stuff like click jacking or UI redressing
1100.19s - 1106.43s |  If you're familiar with it or not what we have on the screen
1107.99s - 1115.63s |  Are the same but there's some red fields and those fields we are loading the login form on our one domain attacker.com and
1116.83s - 1121.83s |  The red ones are fields that we put on top of the the legit login form
1121.83s - 1124.71s |  So this is why redressing is how the attack works
1124.71s - 1127.79s |  It's not that fancy and probably no one is gonna fix
1128.31s - 1133.51s |  UI redressing as long as you have you can prove the impact it it may have and
1134.23s - 1142.11s |  Chances are this is just an hypothesis that the same the same company or organization has a subdomain
1143.59s - 1147.39s |  Pointing to an AWS cloud front and when you request
1148.23s - 1153.67s |  Some resources from it you realize it comes from S3. Not sure whether you're familiar with S3
1153.91s - 1155.83s |  It's a cloud storage
1155.83s - 1163.05s |  You can put there. They are called buckets. You can put there. Whatever you want. One of the features is hosting static websites
1164.79s - 1166.71s |  But as
1166.71s - 1169.43s |  Amazon and S3 permissions are not that easy and
1170.55s - 1175.63s |  Quite often we choose to go with the star. We allow everything to everyone and
1177.07s - 1180.63s |  S3 has a public API rest API. You can read the docs online
1181.39s - 1184.19s |  And chances are that we can write to the bucket
1184.47s - 1190.43s |  I'm not saying that that was the case, but chances are you can write to the bucket and upload whatever you want and
1191.27s - 1196.35s |  If you have a UI redressing issue, maybe you can even deploy the login form
1197.43s - 1199.15s |  deploy the
1199.15s - 1201.15s |  malicious login form on their own
1201.67s - 1208.63s |  Subdomain and this is how it would potentially look like to exfiltrate credentials
1209.07s - 1215.99s |  So you can upload whatever you want to a public S3 or at least internet facing
1216.55s - 1218.19s |  S3
1218.19s - 1219.27s |  buckets
1219.27s - 1224.31s |  Then you get your UI redressing page deployed on the subdomain
1225.15s - 1230.19s |  The victims fill the form and you get the credentials on your on your own server
1231.47s - 1234.23s |  This is exactly what we have in this demo. So
1235.23s - 1239.43s |  With the impact proved they they fixed it
1240.35s - 1244.19s |  Otherwise probably they would look like to it like a UI redressing. No one cares
1244.95s - 1246.95s |  So let's move on to another manufacturer
1247.79s - 1253.71s |  And you may may be asking yourselves. Why didn't you fuzz to find new endpoints? We did it
1254.79s - 1255.79s |  and
1255.79s - 1257.79s |  We found something
1257.79s - 1259.11s |  called site
1259.11s - 1266.55s |  Site minder. We were not aware what site minder was and we asked shut GPT or AI you may
1267.03s - 1271.83s |  You may be missing the AI buzzword and charge a pity told us
1271.83s - 1277.39s |  It's a widely used access management and authentication solution developed by CA technologies
1277.63s - 1283.27s |  Okay, so it's a third-party software, but you have a domain pointing to it. So we did
1283.79s - 1287.87s |  What this company was supposed to do we checked for CVS?
1288.27s - 1294.19s |  One of them was a cross-site scripting on site minder so you may know where this is gonna end up
1294.71s - 1298.91s |  Just crafting a simple payload. It's an image on the username parameter
1299.75s - 1303.55s |  Running some some code some JavaScript code and you get the cross-site scripting
1304.47s - 1305.55s |  but
1305.55s - 1309.39s |  Instead of a single site minder they had tons of them
1309.39s - 1314.79s |  So tons of chances to exploit cross-site scriptings on this car manufacturer
1315.79s - 1318.47s |  So moving on to another manufacturer
1319.31s - 1321.35s |  This is the another another
1322.19s - 1324.19s |  Time when I get the 404 not found
1324.75s - 1328.99s |  My heart starts beating faster because this is again
1329.55s - 1333.31s |  AWS S3 buckets and when you get the 404 means that someone deleted
1333.79s - 1338.27s |  The buckets and buckets they the name should be unique across all AWS
1339.51s - 1344.27s |  Regions, so if the bucket was deleted, but there's still a subdomain pointing to it
1344.79s - 1351.67s |  You can create the buckets and you're gonna control you're gonna get control over the content that served on this subdomain
1351.67s - 1354.15s |  So this is a subdomain takeover technically
1354.91s - 1359.67s |  But we need to figure out in which region the bucket should be created where it was created
1360.83s - 1365.19s |  Before but you know if this is not magic you can use several tools to do it
1365.71s - 1370.31s |  Checking the DNS you get the region. It's a European Europe central
1370.43s - 1374.51s |  You just create the bucket upload the content and from that moment on you're serving
1375.15s - 1382.95s |  Content whatever you want on the on the organization subdomain. This is to be honest. This is quite common on S3
1383.47s - 1385.47s |  so
1385.55s - 1391.51s |  Keep keep watching your subdomains monitor what whatever subdomains you have buckets your assets
1392.91s - 1400.07s |  But before moving to the next manufacturer still cross-site scripting on swagger UI so you're gonna find them
1400.51s - 1403.03s |  deployed everywhere and no one cares about
1404.39s - 1407.35s |  Vulnerable non vulnerabilities on this on this software
1408.83s - 1410.83s |  So
1410.87s - 1417.99s |  Let's move move on to another manufacturer we were looking to Portia. It's not a secret here
1418.99s - 1425.07s |  We found out this website called Portia experience. We looked under the woods to understand how it worked
1425.07s - 1428.11s |  It's a single page application backed by a graph QL API
1428.55s - 1430.15s |  So this time is not rest
1430.15s - 1437.19s |  But we noticed noticed something quite strange the ALT token was included twice on the graph QL requests
1437.19s - 1440.15s |  Both on the cookies and in the authorization header
1440.87s - 1443.43s |  It depends how familiar you with you are with
1444.67s - 1452.71s |  HTTP but adding the ALT token on the the request header is a game-changer in terms of cross-site scripting if
1453.03s - 1458.91s |  The API picks the ALT token from the cookie cookies are managed by the web browser and they are
1459.43s - 1461.87s |  automatically added to the request so
1462.87s - 1466.03s |  They come up with the idea. Maybe we can exploit the cross-site
1466.95s - 1470.19s |  Request forgery. Let's see. Let's see if that happens or not
1470.63s - 1475.47s |  so do that the API should get the ALT token from the cookie instead of the
1475.83s - 1479.71s |  Authorization header and that was what was happening with the graph QL API
1480.71s - 1484.95s |  Then we need the API to be willing to fulfill requests coming from
1485.63s - 1492.27s |  Or started from another origin which the API is also willing to do so we still have some
1493.07s - 1497.11s |  cross-origin request sharing issue here misconfiguration on the API
1497.71s - 1501.55s |  But then browsers include several security features
1502.63s - 1504.71s |  namely on to protect your cookies and
1505.11s - 1511.35s |  The browser they were not willing to include the cookie on requests started from external origins
1511.63s - 1516.67s |  So it was like, okay, there's no way here. It's a dead end. But in fact
1517.55s - 1520.35s |  Cross-site scripting would make this a great a great issue
1520.71s - 1525.43s |  so instead of one we found three cross-site scriptings in different websites and
1525.79s - 1529.79s |  We decided to go with the campaigns dot Porsche comm for a phishing email
1529.79s - 1534.99s |  It sounds kind of catchy if you can offer something on a campaigns dot Porsche comm
1535.51s - 1542.79s |  So this is the vulnerable URL the parameter you should keep the integer for some reason
1543.11s - 1545.11s |  But you can then append
1545.47s - 1546.75s |  HTML
1546.75s - 1548.55s |  So this is the loader
1548.55s - 1555.19s |  We're gonna we're gonna add to the URL that we are gonna send to victims to load a JavaScript file from a remote server
1556.03s - 1562.35s |  Then we have to do some encoding because you cannot it's not that easy to manage spaces and special characters on the URL
1562.91s - 1567.87s |  And this is what's fetched from our remote server. So we're gonna do a GraphQL query
1568.63s - 1572.71s |  Fetch victims PII and exfiltrate it to the remote server
1572.91s - 1578.27s |  Then we just need to trick victims to click the URL so offer them something like a Porsche experience
1578.27s - 1581.43s |  Why not and this is how it goes
1581.43s - 1594.29s |  So we are oops we are waiting victims data so let's email mail them we get the click
1595.17s - 1597.89s |  So as you as you can see, I read the emails
1599.17s - 1605.93s |  Then I click them the links and from that moment on we get the PII on our remote server
1605.93s - 1607.93s |  Again, the point here is not
1608.45s - 1613.45s |  None of the issues here are that fancy. The point is chaining them together
1613.45s - 1618.57s |  The point is chaining them together and prove the impact and this way we get all of them fixed
1621.45s - 1624.61s |  So yes, I know you
1625.57s - 1627.57s |  so we talked about
1627.81s - 1632.57s |  Subdomain takeover. So this is another another subdomain takeover on a different manufacturer
1632.57s - 1636.65s |  But here the the issue is a bit different in this case
1637.17s - 1644.33s |  And sometimes if you want to integrate with some third party, they're gonna say point subdomain to our one
1645.13s - 1652.65s |  Domain or subdomain they create a custom subdomain so that you can point something to them. But then at some point the company
1653.65s - 1655.45s |  Gave up on the the product
1655.45s - 1662.93s |  But your your DNS record is still pointing to their their wound and if it was a bucket and they just deleted their buckets
1663.25s - 1666.25s |  We can create the bucket. That was that was the case here
1666.41s - 1672.81s |  There was a DNS record pointing to another company and the company gave up on the bucket and we created the bucket and we got
1672.93s - 1676.13s |  Our content delivered through their buckets
1677.89s - 1679.17s |  So
1679.17s - 1680.37s |  again
1680.37s - 1688.13s |  Swagger UI so if there's a big takeaway today go home and check whether your company has swagger UI deployed or any other
1688.45s - 1694.57s |  Third-party piece of software and check whether there are known vulnerabilities and patch the systems
1695.45s - 1700.05s |  So time for another one. Nothing fancy again, just a cross-site scripting
1700.65s - 1702.65s |  on swagger UI
1703.81s - 1705.81s |  So we've talked about
1706.85s - 1708.85s |  misconfigured AWS
1709.09s - 1713.57s |  S3 buckets. So here's another another use use case the same issue
1714.25s - 1716.25s |  public writable
1716.77s - 1718.01s |  S3 buckets
1718.01s - 1724.13s |  so the same thing you get your you get your own codes running on their one of their subdomains and
1724.45s - 1726.25s |  Then you can access
1726.25s - 1728.25s |  APIs and other stuff
1729.25s - 1737.41s |  Because you're running on their one subdomain, so most of the policies can be bypassed or will be bypassed this way
1738.97s - 1740.13s |  So
1740.13s - 1745.36s |  We just got our content again. We use a phishing email
1745.36s - 1751.60s |  It's I believe we ask chat GPT to write it not exactly. Give me a phishing email because
1752.12s - 1754.12s |  Otherwise we won't get it
1755.12s - 1760.16s |  But as it happened before with this simple trick we get
1761.44s - 1763.92s |  The ALF token that we can then use
1765.08s - 1767.08s |  this case I
1767.36s - 1772.12s |  Believe we got the it was PII from from the API
1772.72s - 1779.20s |  But that was not the only one issue with this manufacturer as David said you have a vehicle identification number
1779.20s - 1781.20s |  You can get them from your windshield
1781.36s - 1785.52s |  So let's call it public because anyone can get it from your car part from the car park
1785.52s - 1792.68s |  Or if you're lazy enough as you we are you go to Google you search and you get one million eight hundred and ninety
1793.88s - 1795.88s |  vehicle identification numbers and
1796.44s - 1803.20s |  Then you can add them to your to you can use the API to fetch PII drivers PII owners PII
1803.80s - 1805.80s |  There was something interesting here
1806.12s - 1810.04s |  We were not able to use the this API in point
1811.00s - 1814.08s |  Before adding them. Okay, just a few seconds
1814.88s - 1820.44s |  Before adding one car. We need to be a warner, but we can be a warner of anyone's car
1820.80s - 1822.80s |  Just pick the V in add to your account
1822.80s - 1829.68s |  You become a warner and from that moment on you can do thousands of requests to just scrape the data from the API
1832.20s - 1836.00s |  Maybe you're lucky and you're gonna get again a cross-site scripting from some
1836.72s - 1841.08s |  Redirect URI and you can do the same magic we did before
1842.80s - 1846.94s |  Okay, I'm not sure we're gonna be we're gonna have time for all of them, but those are quick
1848.24s - 1851.68s |  Let's see. If you have siteminder go home and patch it
1852.12s - 1857.88s |  We have 91 instance of siteminder in the same company. So go home and fix it
1859.00s - 1860.52s |  Very fast
1860.52s - 1863.92s |  Just to tell you you have the table with all the information
1864.60s - 1871.56s |  Organizations subscribe to vulnerability databases, of course create a safe harbor for us hackers. So that's very cool
1871.68s - 1875.28s |  Reply to our emails. It's for everyone security
1876.00s - 1880.04s |  AI won't make your assets safe, but only safer
1880.16s - 1885.40s |  That's a QR code if you want to promo on API security training, so that's it. Thank you


url: https://www.youtube.com/watch?v=vEzLsRXjatw
title: DEF CON 32 - Securing Frontends at Scale;Paving our Way to Post XSS World - jen ozmen, Aaron Shim
type: video
author: DEFCONConference
ingestion_date: 2024-11-09
transcription_model: large-v2
chunking_status: completed
content:
{
  "webpage_url": "https://www.youtube.com/watch?v=vEzLsRXjatw",
  "title": "DEF CON 32 - Securing Frontends at Scale;Paving our Way to Post XSS World - jen ozmen, Aaron Shim",
  "description": "Cross-site scripting (XSS) remains a top web vulnerability. Google has invested heavily in defenses, and in this talk, we'll share our blueprint for protecting your code. We'll discuss how we implemented runtime and compile-time protections across hundreds of products used by billions, highlighting technical lessons and best practices. We'll also glimpse into the future of anti-XSS defenses and explore how we can make the web safer for everyone.",
  "channel_url": "https://www.youtube.com/channel/UC6Om9kAkl32dWlDSNlDS9Iw",
  "duration": 1481,
  "channel": "DEFCONConference",
  "uploader": "DEFCONConference",
  "upload_date": "20241016"
}

2.10s - 6.10s | This text was transcribed using whisper model: large-v2

 Hello, my name is Aaron, and I guess I have to apologize.
6.10s - 9.34s |  The resemblance is a little hard to see in that picture
9.34s - 11.26s |  because my hair is not nearly as luscious
11.26s - 12.62s |  as it is in that photo.
12.62s - 15.46s |  But I'm sorry, your mineral content in the water,
15.46s - 17.74s |  as you've all found out, really awful here.
19.06s - 21.98s |  But yeah, like Jen, I work at Google.
21.98s - 24.22s |  We do web security on hundreds
24.22s - 25.58s |  and thousands of different web apps.
25.58s - 28.22s |  I've been at Google for a little under seven years,
28.22s - 29.86s |  and before my stint in security,
29.86s - 31.34s |  I came from the developer side.
31.34s - 34.42s |  I've worked on products like Google Docs and GCP,
34.42s - 35.62s |  on customer-facing stuff.
35.62s - 37.86s |  So when we approach this work
37.86s - 40.82s |  of trying to make our web apps as secure as possible,
40.82s - 43.18s |  we approach it with a special empathy for the developer,
43.18s - 46.02s |  where we want to ship the most secure products,
46.02s - 47.50s |  but also we don't want to get in the way
47.50s - 51.10s |  of delivering really dynamic and new user features.
52.14s - 53.86s |  Cool, thanks, Aaron.
53.86s - 55.90s |  Here's a quick look at our agenda for today.
55.90s - 58.98s |  So we'll start with a quick intro to XSS.
58.98s - 60.66s |  I assume most of you are familiar with it,
60.98s - 62.50s |  so we'll still go over some examples.
62.50s - 64.42s |  And then we'll talk about runtime mitigation,
64.42s - 66.98s |  specifically those that come with content security policy,
66.98s - 68.42s |  and it's different flavors.
68.42s - 70.02s |  Then we'll talk about how we use those
70.02s - 72.74s |  to secure our applications at scale.
72.74s - 75.46s |  And we'll move on to how we imagine the post-XSS world,
75.46s - 76.96s |  and end with a call to action.
77.94s - 79.94s |  Cool, so cross-site scripting.
81.30s - 84.18s |  It's just some data to motivate that this is a big problem.
84.18s - 86.16s |  Google runs a vulnerability reward program
86.16s - 88.18s |  where we pay out security researchers
88.18s - 90.10s |  when they report their bugs to us.
90.82s - 93.82s |  As you can see here, over 50% of the bugs reported
93.82s - 95.58s |  are web-related, that makes sense.
95.58s - 97.18s |  Google is a web company.
97.18s - 102.18s |  And then the rest, out of those 50%, almost 35 are XSS.
103.18s - 104.70s |  So this was a really big problem,
104.70s - 106.50s |  and that's why we've developed techniques
106.50s - 107.94s |  to remediate this at scale.
109.02s - 111.26s |  So let's talk about what XSS really is.
111.26s - 114.26s |  The root cause is user input gets mixed up with code,
114.26s - 116.42s |  and then gets interpreted as code.
116.42s - 119.18s |  Because HTML is dynamic and mixes static data
119.18s - 121.86s |  with logic that should execute, as you see here.
121.86s - 122.86s |  This is very prevalent.
122.86s - 124.70s |  And this leads to malicious scripts
124.70s - 126.94s |  getting injected into a trusted site.
126.94s - 130.22s |  In this example, you see, very simple example,
130.22s - 132.18s |  there's a description that's supposed to be dynamic,
132.18s - 133.78s |  but if this is user-controlled,
133.78s - 136.66s |  we can inject the following script and get it to execute.
136.66s - 138.06s |  And there, you have an XSS.
139.34s - 141.54s |  An XSS, when exploited, lets the attacker
141.54s - 144.26s |  take control of the account of a logged-in user.
144.26s - 146.48s |  This means the attacker can do anything the victim can,
146.48s - 147.46s |  so it's pretty powerful,
147.50s - 149.90s |  and that's why we try really hard to avoid it.
149.90s - 152.06s |  We also call this the client-side version
152.06s - 154.10s |  of remote code execution,
154.10s - 157.30s |  as they share a similar concept of arbitrary code execution.
157.30s - 161.26s |  And here's another format you might be used to seeing XSS.
161.26s - 163.62s |  We usually pop an alert with the document domain
163.62s - 166.06s |  as a proof of concept when we find an XSS vector.
167.62s - 169.98s |  So a quick example of DOM XSS,
169.98s - 173.58s |  the client-signed variant of cross-site scripting.
173.58s - 177.30s |  The root cause here is DOM APIs are not secure by default.
178.14s - 181.54s |  JavaScript and JavaScript loves turning strings into code
181.54s - 185.50s |  through APIs like innerHTML or script.innerText.
185.50s - 187.94s |  Here's a very simple example of how this might happen.
187.94s - 189.66s |  In this code snippet, location hash
189.66s - 191.22s |  is a user-controlled string.
191.22s - 194.30s |  We take that and happily insert it into innerHTML.
194.30s - 196.70s |  In this case, with the example here,
196.70s - 199.46s |  the onError alert statement will execute,
199.46s - 200.82s |  and then we have XSS.
200.82s - 202.46s |  But no one codes like this anymore, right?
202.46s - 204.20s |  Thanks to modern frameworks like React
204.20s - 205.92s |  that abstract away DOM manipulations,
205.92s - 207.02s |  nobody has to do this.
208.30s - 211.30s |  Except not.
211.30s - 214.38s |  As you can see in this fresh bit of internet outrage,
214.38s - 216.58s |  an enterprising developer has managed
216.58s - 218.94s |  to work in a innerHTML assignment
218.94s - 221.54s |  inside a very functional React component.
221.54s - 223.54s |  It's supposed to be a really advanced system
223.54s - 225.74s |  that takes care of the virtual DOM dipping,
225.74s - 227.30s |  all of the different re-rendering things
227.30s - 228.86s |  so that you don't have to do this yourself,
228.86s - 230.74s |  but this developer, they really want to take it
230.74s - 232.18s |  into their own hands.
232.18s - 234.18s |  And this is the crux of the problem.
234.18s - 236.62s |  Just because a framework can abstract away
236.66s - 238.46s |  the complex DOM operations,
238.46s - 239.90s |  doesn't mean that it can prevent
239.90s - 241.54s |  a developer from using them.
241.54s - 243.30s |  And these issues are really easy
243.30s - 244.62s |  to introduce in your code base.
244.62s - 246.94s |  It really takes a developer, a code reviewer,
246.94s - 250.34s |  having a bad day, and once it sneaks into your product,
250.34s - 252.58s |  then it's really hard to eradicate.
252.58s - 256.18s |  So do we stand a chance against this class of problems?
256.18s - 257.22s |  And we think we do.
257.22s - 259.64s |  We've all heard of content security policy
259.64s - 262.36s |  and how it can solve all your process scripting, right?
262.36s - 265.86s |  Well, we do admit that content security policy
265.86s - 268.82s |  is a little bit difficult to understand and configure.
268.82s - 272.10s |  It really is more like a meta feature than a feature.
272.10s - 274.30s |  It has a lot of bells and whistles and knobs
274.30s - 276.82s |  that you can change, a lot of directives,
276.82s - 278.46s |  what do you have to set on these directives,
278.46s - 280.50s |  and it is a little confusing
280.50s - 282.18s |  because it tries to cover everything,
282.18s - 284.80s |  any sort of resource that loads on your page.
284.80s - 286.78s |  But hopefully, when we go on this journey together,
286.78s - 289.38s |  we can talk about some of the configurations
289.38s - 291.66s |  that we've found to work at scale
291.66s - 294.02s |  and cover many classes of cross-site scripting,
294.02s - 298.00s |  and we'll see why they work together.
298.90s - 299.78s |  And before we begin,
299.78s - 303.54s |  since we are talking about cross-site scripting
303.54s - 307.30s |  and CSP and protections that Google uses,
307.30s - 308.22s |  maybe we'll take a look
308.22s - 310.68s |  at what is actually working in the wild.
310.68s - 313.54s |  So here we have an example of particular Google web app
313.54s - 315.26s |  that some of you may have used.
315.26s - 317.10s |  You know it has a really complicated UI
317.10s - 319.92s |  with a lot of interactive user features
319.92s - 322.12s |  and a small call-out to this team.
322.16s - 323.40s |  They've been great partners
323.40s - 326.04s |  in really shipping the best and greatest
326.04s - 329.08s |  and the latest web protection mitigations
329.08s - 331.60s |  and working really hard to keep their end users safe.
331.60s - 333.68s |  So let's open up Chrome DevTools
333.68s - 335.20s |  and see what we are actually shipping
335.20s - 336.88s |  on a real-life product.
336.88s - 338.36s |  And as you can see here,
338.36s - 341.90s |  we see a couple different content security policies,
341.90s - 343.26s |  a lot of them, as a matter of fact.
343.26s - 344.84s |  But like we said before,
344.84s - 347.12s |  not all of them are specifically
347.12s - 348.76s |  for cross-site scripting defenses
348.76s - 350.28s |  because it's a meta feature
350.28s - 352.44s |  that can also turn off other
352.44s - 354.28s |  potential dangerous vectors as well.
354.28s - 356.00s |  We don't have time to get into the other protections,
356.00s - 357.24s |  but if you have questions,
357.24s - 359.28s |  we will be sticking around for a little bit after
359.28s - 361.86s |  so we can go into more detail after the talk.
361.86s - 364.24s |  But for now, we'll focus on these two,
364.24s - 367.32s |  the ones that we find really powerful protections
367.32s - 368.76s |  against cross-site scripting,
368.76s - 371.44s |  namely the nonce-only CSP
371.44s - 375.19s |  and the trusted types header.
375.19s - 377.11s |  Cool, and we'll go into some details.
377.11s - 378.83s |  So nonce-only CSP,
378.83s - 381.21s |  in the example we saw before,
381.61s - 382.65s |  it's pretty self-explanatory.
382.65s - 384.55s |  It's the one with the nonce directive.
385.81s - 387.61s |  More specifically, a nonce-only policy
387.61s - 390.09s |  takes a form that looks a bit like this.
390.09s - 391.97s |  Internally, we recommend a nonce-only
391.97s - 395.09s |  or a strict CSP for most of our modern web apps.
395.09s - 397.39s |  What this does is only what's allow-listed
397.39s - 398.97s |  under the script source directive
398.97s - 400.11s |  can run on the page.
400.11s - 401.57s |  Other scripts will be blocked.
401.57s - 404.01s |  So in this case, any script with the right nonce,
404.01s - 406.89s |  which is a random value that's generated per request,
407.77s - 408.73s |  can run on the page.
408.73s - 411.01s |  So the templating systems usually propagate
411.77s - 413.09s |  the nonce for each script tag on the HTML,
413.09s - 416.93s |  and the header has to send the right value for the nonce.
416.93s - 418.49s |  So the header and the HTML work together
418.49s - 420.05s |  to indicate which scripts are trusted,
420.05s - 422.65s |  and the browser can act accordingly.
422.65s - 424.41s |  Another CSP directive that's useful
424.41s - 427.77s |  for a nonce-only policy is the strict dynamic directive.
427.77s - 431.13s |  This directive enables a transitive trust for scripts.
431.13s - 433.43s |  So if you have one script with the right nonce,
433.43s - 435.25s |  it's allowed to load other scripts,
435.25s - 437.21s |  and the trust is propagated.
437.21s - 438.53s |  When we don't have this directive,
438.53s - 440.63s |  only scripts with the right nonce can execute
441.25s - 443.45s |  and if they load other scripts, they'll be blocked.
443.45s - 445.79s |  So although this slightly relaxes the policy,
445.79s - 449.03s |  we found that it's also made adoptions a lot easier
449.03s - 450.07s |  for complex web apps,
450.07s - 452.47s |  so that's the trade-off that we're working with.
454.19s - 455.07s |  So thanks, Jen.
455.07s - 457.83s |  So as Jen mentioned, a nonce-only CSP is great
457.83s - 460.27s |  if you have the backend or the server-side render content
460.27s - 461.75s |  working in close conjunction
461.75s - 463.47s |  with the frontend or client-side code
463.47s - 464.99s |  because you have to generate these nonces
464.99s - 467.99s |  that are new for every single response.
467.99s - 470.39s |  But what if you don't have control over the backend?
471.15s - 472.27s |  Then we have to use a close relative
472.27s - 475.11s |  of the nonce-only CSP called the hash-based CSP.
475.11s - 477.23s |  And the way it works is it's very similar
477.23s - 479.71s |  to a nonce-based policy, but as you've noticed,
479.71s - 482.75s |  instead of the nonce value, we provide a hash value.
482.75s - 484.45s |  And this is really powerful
484.45s - 487.49s |  because we don't have to regenerate this CSP
487.49s - 489.11s |  for every response load.
489.11s - 493.03s |  The hash is fixed, presumably because on every response,
493.03s - 495.91s |  your script is mostly the same.
495.91s - 498.51s |  And this is really powerful
498.51s - 500.55s |  when you don't have control over that backend
500.55s - 505.55s |  so you can put in the logic to do this nonce generation.
505.63s - 507.53s |  And this is also super useful
507.53s - 510.39s |  for the SBA, single-page applications,
510.39s - 513.27s |  because when you are doing the final bundling process
513.27s - 516.07s |  for your final HTML, you know what script has
516.07s - 518.27s |  are going to be there in your SBA bundle
518.27s - 520.71s |  and use that to generate your hash.
520.71s - 522.85s |  And there you go, a fixed CSP policy
522.85s - 524.63s |  that offers this protection
524.63s - 527.45s |  that you can even drop into a meta tag
527.45s - 529.13s |  at the top of the HTML file.
529.13s - 532.69s |  And in more detail, so what you have on the left
532.69s - 534.65s |  is sort of like what your bundle might look like
534.65s - 536.33s |  for a particular single-page application.
536.33s - 537.61s |  You got a couple scripts loading
537.61s - 540.73s |  to actually get the user interaction going.
540.73s - 543.25s |  And we would actually rewrite this, in this case,
543.25s - 547.43s |  into a dynamic loader inside an inline script block
547.43s - 549.37s |  because that is one of the characteristics
549.37s - 550.77s |  of a hash-based CSP for now.
550.77s - 553.97s |  You need to know exactly what's in the script content
553.97s - 555.09s |  in order to make the hash.
555.09s - 558.73s |  So usually you do this by actually inlining the block.
558.73s - 562.85s |  And if you have resources that would load over the network,
562.85s - 565.49s |  then you turn it into a JavaScript
565.49s - 567.85s |  that would dynamically add it to the page.
567.85s - 571.69s |  And once we have that, we also added some preload tags
571.69s - 574.45s |  so that we actually get the parallelization going
574.45s - 575.87s |  for fetching these resources.
575.87s - 578.01s |  You can take the script block, create the hash,
578.01s - 581.57s |  and stick it in as a meta tag that carries a CSP,
581.57s - 582.87s |  and there you go.
582.87s - 585.91s |  The HTML is self-contained and offers its protection
585.91s - 592.20s |  with a CSP baked in.
592.20s - 594.28s |  The next CSP-delivered mitigation
594.28s - 596.20s |  that we want to talk about is Trusted Types.
596.20s - 597.88s |  Specifically, Trusted Types is geared
597.88s - 600.36s |  towards protecting DOM-based XSS.
600.36s - 602.48s |  So in the example that we saw before,
602.48s - 605.16s |  it's the one that says Require Trusted Types For,
605.16s - 609.24s |  and that's the directive that you need to send.
609.24s - 611.36s |  So we want to point out that the attack surface is
611.36s - 613.16s |  large here for DOM XSS.
613.16s - 614.96s |  There's over 60 JavaScript APIs that
614.96s - 617.40s |  can turn strings into code and lead to arbitrary code
617.40s - 620.08s |  execution, and DOM XSS is usually
620.08s - 622.08s |  very easy to introduce and hard to detect.
622.08s - 623.64s |  Calls to these APIs could be going
623.64s - 625.68s |  through multiple levels of indirections.
625.68s - 627.52s |  It could be in a transitive dependency,
627.52s - 629.08s |  or it could exist in libraries that we
629.08s - 630.32s |  don't have control over.
630.32s - 632.32s |  So how can we make this better?
632.32s - 634.60s |  And we need a runtime mitigation for this,
634.60s - 636.36s |  and this is where Trusted Types comes in.
636.36s - 637.84s |  The main idea behind Trusted Types
637.84s - 640.76s |  is to make security decisions explicit and auditable.
640.76s - 642.96s |  To do this, Trusted Types will restrict the access
642.96s - 644.84s |  to the dangerous DOM APIs.
644.84s - 646.56s |  We mentioned previously, and it'll
646.56s - 649.24s |  require typed objects instead of plain strings
649.24s - 652.72s |  to be passed into the over 60 APIs.
652.72s - 654.40s |  So in this case, instead of strings
654.40s - 657.12s |  that represent script URLs, HTML, or scripts,
657.12s - 659.84s |  we'll require typed objects such as TrustedScriptURL.
659.84s - 662.12s |  And we'll discuss how to generate these typed objects,
662.12s - 665.20s |  but this is the main idea.
665.20s - 667.28s |  This is what the header looks like, again.
667.28s - 669.68s |  This tells the browser to reject string assignments
669.68s - 671.40s |  to injection syncs, and effectively, it
671.40s - 673.20s |  provides a runtime type checking mechanism
673.20s - 675.28s |  on top of JavaScript.
675.28s - 678.44s |  If you remember the example from our DOMXSS slide before,
678.44s - 680.80s |  we're using a plain string here within our HTML.
680.80s - 682.44s |  And if Trusted Types were enforced,
682.44s - 685.20s |  we would throw an error that looks like this.
685.20s - 687.80s |  So the browser will tell you you need a trusted HTML,
687.80s - 690.52s |  and it won't use a string.
690.52s - 692.88s |  We want to highlight that most security guarantees
692.88s - 694.84s |  of Trusted Types come from a policy,
694.84s - 696.68s |  and the policy is what I mentioned
696.68s - 700.00s |  of how we could generate a trusted object.
700.00s - 702.72s |  So you can think of it as a wrapper that takes in a string
702.72s - 704.76s |  and returns a TrustedType.
704.76s - 706.30s |  And we apply this policy to strings
706.30s - 708.20s |  before inserting them into the DOM sync.
708.20s - 710.08s |  And the processing you do within the policy
710.08s - 711.96s |  should make sure the string won't
711.96s - 714.60s |  lead to any unintended code execution,
714.60s - 716.80s |  therefore preventing DOMXSS.
716.80s - 719.84s |  So in this example, we use a sanitizer from DOMPurify.
719.84s - 721.68s |  We check for the presence of TrustedTypes
721.68s - 724.76s |  in the JavaScript namespace for browser compatibility,
724.76s - 726.90s |  and then use the correct factory if it exists.
726.90s - 729.40s |  But if you don't want to keep writing this verbose if block
729.40s - 732.32s |  because it is a bit clunky, DOMPurify also
732.32s - 734.20s |  works with TrustedTypes off the shelf.
734.20s - 737.32s |  So it will do the policy creation and the browser
737.32s - 739.64s |  compatibility checking under the hood for you
739.64s - 745.19s |  if you pass in the right config for TrustedTypes.
745.19s - 746.03s |  Thanks, Jen.
746.03s - 748.31s |  So now that we've talked about our security features
748.31s - 750.63s |  that we believe are powerful against cross-site scripting
750.63s - 752.35s |  today, we'll touch a little bit on how
752.35s - 755.03s |  to use them in your application and offer a bit of the lessons
755.03s - 758.39s |  that we've learned from securing our apps at scale at Google.
758.39s - 760.51s |  So before we begin, we'd like to make a quick plug
760.51s - 762.39s |  for an article that presents many of the ideas
762.39s - 765.25s |  that we'll discuss in the next section, but in written form.
765.25s - 767.65s |  This approach is used widely in our team and in our org,
767.65s - 769.35s |  and this article goes into a little bit more detail
769.35s - 771.55s |  than what we have time for in some of the ideas
771.55s - 773.37s |  that we'll elaborate on next.
773.37s - 775.85s |  It's on the Google Security Engineering blog
775.85s - 779.97s |  at bughunters.google.com, along with other great content
779.97s - 781.81s |  like this, and also similar to, I think,
781.81s - 782.89s |  the theme of this village.
782.89s - 784.81s |  So please check us out.
784.81s - 787.37s |  So back to the main show, what are some of the tactics
787.37s - 790.97s |  that we use to adopt these security features at scale
790.97s - 792.61s |  on hundreds of different web apps?
792.61s - 793.69s |  So some of the lessons that we've
793.69s - 796.21s |  learned that we find most useful are, number one,
796.21s - 798.97s |  use the frameworks that will help you in this journey.
798.97s - 803.13s |  Number two, try to shift left to get developers an indication
803.13s - 805.39s |  whether this feature is incompatible with code
805.39s - 808.33s |  that they're writing as early as possible in the process.
808.33s - 811.77s |  Number three, some tools that we've built and we also
811.77s - 814.41s |  rely on throughout this process that we found useful.
814.41s - 817.81s |  And number four, a quick note about third-party dependencies
817.81s - 820.53s |  and how they might complicate your adoption process.
820.53s - 822.93s |  So the first big idea is, if you're
822.93s - 825.67s |  starting a new code base from scratch,
825.67s - 827.05s |  let's think about which frameworks
827.05s - 829.17s |  we can use that will give us security superpowers.
829.17s - 832.45s |  And the big idea behind that is that, at our scale,
832.45s - 835.57s |  at at-scale defense against cross-site scripting at Google,
835.57s - 837.65s |  we rely on frameworks with, number one,
837.65s - 840.41s |  context-aware templating, and number two,
840.41s - 842.81s |  built-in compatibility with these security features
842.81s - 846.13s |  that are secure by default. And as a quick aside,
846.13s - 848.65s |  we haven't talked too much about context-aware templating
848.65s - 851.21s |  right now, but we don't have too much time to get into that.
851.21s - 854.05s |  But the gist of it is, you want to have a templating system
854.05s - 856.01s |  that does reasonable enough things
856.01s - 859.57s |  to prevent a lot of server-side rendered cross-site scripting
859.57s - 860.37s |  from happening.
860.37s - 861.65s |  Catch us after the talk, and we are
861.65s - 863.15s |  happy to chat about templating, too,
863.15s - 865.73s |  which is beyond the scope of this talk.
865.73s - 868.93s |  So what we want to cover on this slide
868.93s - 870.73s |  and for the rest of the talk is what
870.73s - 873.17s |  we call the safe coding philosophy at Google.
873.17s - 875.25s |  And this is the idea that we want
875.25s - 877.77s |  to give tools to developers that take away
877.77s - 880.29s |  a lot of the uncertainty from the developer
880.29s - 882.49s |  on how to use and configure these security
882.49s - 884.57s |  features correctly, because they are really
884.57s - 887.29s |  easy to misconfigure in really subtle ways.
887.29s - 889.05s |  And we want the security to not really
889.05s - 891.81s |  get in the way of feature development
891.81s - 893.69s |  and shipping great things to our users,
893.69s - 895.81s |  so we want to abstract away some of these decisions
895.81s - 898.29s |  in a clear and understandable manner.
898.29s - 901.45s |  And the key to do this is through our frameworks
901.45s - 902.61s |  and our APIs.
902.61s - 905.73s |  And we'll say that this is possible at Google
905.73s - 907.41s |  because you do have tight control
907.41s - 909.77s |  over the internal frameworks that you use in-house
909.77s - 911.65s |  and also tight control over the guidance
911.65s - 914.57s |  that we give to our developers on how to use these frameworks.
914.57s - 916.69s |  But I imagine many of the web developers
916.69s - 919.31s |  here aren't building internal web apps for Google,
919.31s - 921.85s |  so how can we get some of these benefits?
921.85s - 924.15s |  And thankfully, our colleagues have
924.15s - 926.47s |  worked hard to ensure compatibility
926.47s - 929.41s |  with some of these security features in frameworks
929.41s - 932.03s |  that are available outside of our internal ecosystem
932.03s - 932.65s |  as well.
932.65s - 934.19s |  For instance, some of the frameworks
934.19s - 937.95s |  are also available externally, like Angular and Lit.
937.95s - 940.11s |  All the hardening work that we do there
940.11s - 942.23s |  translates to benefits for the ecosystem
942.23s - 944.15s |  and for everyone in the world to use.
944.15s - 945.79s |  But if you're also using frameworks
945.79s - 949.67s |  that are not entirely within the Google ecosystem
949.67s - 953.07s |  or very Google-inspired, we also have our colleagues
953.07s - 955.39s |  at Meta, who did a lot of work to make
955.39s - 957.59s |  React compatible with CSV and Trusted Types.
957.59s - 959.75s |  And some of the frameworks that are based on React,
959.75s - 962.79s |  like Next.js, thanks to volunteers and other colleagues,
962.79s - 965.11s |  have compatibility as well.
965.11s - 965.75s |  Great.
965.75s - 969.17s |  So on to the second idea, how to shift left and enforce early.
969.17s - 970.63s |  And this is just the idea that we
970.63s - 972.43s |  want to add these headers and protections
972.43s - 974.75s |  as early as possible, maybe in your dev server,
974.75s - 976.87s |  so that when things go wrong, your developers
976.87s - 979.19s |  can see the error and act on it as soon
979.19s - 981.87s |  as they've written the code so that they can fix it
981.87s - 984.03s |  while the context of what they're trying to do
984.03s - 985.59s |  is fresh in their minds.
985.59s - 989.23s |  And more technically, a way to do this is, well,
989.23s - 992.27s |  here in this example, we have a configuration
992.27s - 993.67s |  for a Vite dev server.
993.67s - 996.59s |  I hear that's what all the web developers are using nowadays.
996.59s - 998.87s |  We are all onto Vite now.
998.87s - 1002.75s |  And here we see a Node.js express-style middleware
1002.75s - 1004.79s |  inside this configuration for the dev server.
1004.79s - 1008.67s |  That just dumps in these headers.
1008.67s - 1010.63s |  But if you don't have control over your dev
1010.63s - 1013.75s |  server or your serving stack, as you mentioned before,
1013.75s - 1016.43s |  meta tags at the top of an HTML file
1016.43s - 1018.27s |  also can deliver these headers.
1018.27s - 1020.19s |  And for the observant among you, you
1020.19s - 1023.55s |  will see that here we have our hash-based CSP again.
1023.55s - 1027.99s |  Because if you are baking a CSP into a static HTML page,
1027.99s - 1029.67s |  you really want it to be a hash-based CSP
1029.67s - 1032.51s |  so that the nonce value isn't blown
1032.51s - 1034.03s |  the first time you use it.
1034.03s - 1037.47s |  And we saw an example of how to do the refactoring
1037.47s - 1038.99s |  to make a hash-based CSP before.
1038.99s - 1040.27s |  And it was a little cumbersome.
1040.27s - 1042.51s |  So how can you do that more easily?
1042.51s - 1046.59s |  And the answer is we have an experimental tool
1046.59s - 1048.43s |  available in GitHub and NPM that does
1048.43s - 1050.95s |  a lot of this transformation for you.
1050.95s - 1053.63s |  This is available as a Webpack plugin,
1053.63s - 1056.39s |  but also as a utility library that
1056.39s - 1058.39s |  takes in a string of the HTML and does
1058.39s - 1060.39s |  a refactoring like this.
1060.39s - 1062.47s |  I mean, I know there's a lot of code on the screen.
1062.47s - 1064.91s |  But it just takes in a string and outputs a string.
1064.91s - 1067.15s |  And it will shuffle around the nodes,
1067.15s - 1069.35s |  do the hashing, add the meta tags for you.
1069.35s - 1071.23s |  And this is really powerful because this
1071.23s - 1073.95s |  is a very framework-agnostic way that now has
1073.95s - 1077.67s |  proven to have outlasted the popularity of Webpack plugins
1077.67s - 1080.07s |  because they just keep changing the configurations.
1080.07s - 1083.55s |  But yeah, but this is really powerful
1083.55s - 1085.95s |  because if we remember the example from before
1085.95s - 1089.39s |  with the Vite middleware, it composes almost perfectly.
1089.39s - 1092.07s |  This is just a copy-paste job of the last two slides.
1092.07s - 1094.43s |  And yeah, and I think one of the lessons
1094.43s - 1096.43s |  we've learned in building some of this tooling
1096.43s - 1099.39s |  is if you keep it as agnostic as possible in terms
1099.39s - 1102.79s |  of frameworks and just separate the logic from the framework,
1102.79s - 1105.19s |  then you can just snap the pieces together
1105.19s - 1108.39s |  when a new framework comes along.
1108.39s - 1111.91s |  So on to the next section.
1111.91s - 1113.19s |  Thanks, Eric.
1113.19s - 1116.27s |  In addition to using the header to spot violations early,
1116.27s - 1118.31s |  we really advocate for using some other tools
1118.31s - 1120.59s |  to help you with adopting these defenses.
1120.59s - 1122.51s |  We'll introduce a bunch of helpful tools,
1122.51s - 1126.11s |  but they can categorically be broken down into, first
1126.11s - 1129.35s |  of all, tools for understanding features and violations.
1129.35s - 1131.23s |  We mentioned that CSP is notoriously
1131.23s - 1132.99s |  hard to configure and understand.
1132.99s - 1135.27s |  And the CSP evaluator solves that problem.
1135.27s - 1138.27s |  It's available as either a website or a Chrome extension,
1138.27s - 1140.11s |  which will give you some high-level guidance
1140.11s - 1141.59s |  on the configuration of your CSP.
1141.59s - 1146.67s |  It can detect directives and then kind of explain them.
1146.67s - 1148.47s |  We're also actively working on a tool
1148.47s - 1149.83s |  called Trusted Types Helper.
1149.83s - 1153.03s |  This isn't released yet, but it will be soon,
1153.03s - 1155.51s |  for streamlining the Trusted Types adoption process.
1155.51s - 1157.87s |  One of the most time-consuming and painful stages
1157.87s - 1160.07s |  of Trusted Types enforcement is finding the source
1160.07s - 1161.83s |  of violations and trying to fix them.
1161.83s - 1164.63s |  So this Chrome extension will create a DevTools tab
1164.63s - 1166.87s |  that helps you demystify Trusted Types violations
1166.87s - 1168.07s |  and suggest some refactoring.
1168.07s - 1170.31s |  So stay tuned.
1170.31s - 1172.09s |  Another category of tools are those
1172.09s - 1175.75s |  that help with refactoring using static analysis.
1175.75s - 1178.47s |  We have a huge static analysis pipeline at Google.
1178.47s - 1180.01s |  And we took the most effective checks
1180.01s - 1182.39s |  that we run on our internal static analysis
1182.39s - 1185.49s |  tooling and linter and built it on top of ESLint.
1185.49s - 1187.15s |  And that is Safety Web.
1187.15s - 1189.79s |  This project is also in active development,
1189.79s - 1191.87s |  but it is available on GitHub.
1191.87s - 1193.43s |  As you can see in the example here,
1193.43s - 1196.03s |  it'll detect unsafe usages of risky APIs.
1196.03s - 1197.83s |  And you can refactor these unsafe calls
1197.83s - 1200.63s |  with another tool we provide called Safe Values.
1200.63s - 1204.39s |  So Safe Values is a library designed to steer developers
1204.39s - 1206.39s |  to make safer choices.
1206.39s - 1208.27s |  It will prevent unsafe assignments
1208.27s - 1211.27s |  using TypeScript type checking unless the value is trusted,
1211.27s - 1213.91s |  either from a trusted source or by transformation
1213.91s - 1214.83s |  that makes it trusted.
1214.83s - 1219.47s |  For example, sanitizing in HTML makes it trusted.
1219.47s - 1221.55s |  As we mentioned, we were going to add a quick note
1221.55s - 1223.39s |  about third-party dependencies.
1223.39s - 1225.43s |  This is the complicated part.
1225.43s - 1227.97s |  And it's a big aspect of making your application
1227.97s - 1230.19s |  compatible with these features.
1230.19s - 1232.43s |  For both trusted types and CSV, all code
1232.43s - 1234.75s |  running on your site needs to be free of violations
1234.75s - 1235.67s |  of these policies.
1235.67s - 1238.27s |  Otherwise, they will be blocked by the browser.
1238.27s - 1240.07s |  This is a hard part because we won't always
1240.07s - 1242.99s |  know which dependencies are incompatible until we've
1242.99s - 1243.99s |  introduced them.
1243.99s - 1246.03s |  And it's also hard to fix because the potential
1246.03s - 1247.99s |  refactoring sites don't exist in our code,
1247.99s - 1250.91s |  but code that others have written and others control.
1250.91s - 1252.35s |  We're currently working on tooling
1252.35s - 1254.39s |  that will make informed decisions about which
1254.39s - 1256.27s |  dependencies might be compatible with which
1256.27s - 1257.79s |  features in the ecosystem.
1257.79s - 1260.47s |  But this is a part where we could use some help as well.
1260.47s - 1263.23s |  As CSV and trusted types gain more widespread browser
1263.23s - 1265.75s |  adoption, you can help us by making pull requests
1265.75s - 1267.83s |  if you do end up patching third-party dependencies
1267.83s - 1273.16s |  that you find along your journey of adopting these features.
1273.16s - 1273.72s |  Thanks, Jen.
1273.72s - 1276.68s |  So I know we talked a lot about the mitigations
1276.68s - 1279.00s |  and the tools that are used today to protect our apps.
1279.00s - 1280.50s |  But let's talk a bit about where we
1280.50s - 1282.80s |  want to go from here and the more secure world
1282.80s - 1285.40s |  that we'd like to see in the future.
1285.40s - 1288.12s |  And here are the key ideas we talked about today
1288.12s - 1291.44s |  that form the basis of this protection, this pipeline
1291.44s - 1294.60s |  of protection, if you will.
1294.60s - 1297.68s |  So the running theme is abstracting security decisions
1297.68s - 1301.16s |  away from developers in a clear and secure by default way.
1301.16s - 1302.88s |  When all of these elements work together,
1302.88s - 1305.08s |  starting from the first stage when the code is written
1305.08s - 1306.54s |  all the way to the final point when
1306.54s - 1309.02s |  the code is executed in the client's browser,
1309.02s - 1311.74s |  we have protection at every stage of the development
1311.74s - 1312.86s |  lifecycle.
1312.86s - 1315.50s |  And this should make most cross-site scripting solvable.
1315.50s - 1317.94s |  And we see this working at Google scale.
1317.94s - 1320.22s |  On our most recent in-house frameworks
1320.22s - 1323.22s |  that follow a lot of these guidelines from these pillars,
1323.22s - 1325.10s |  we see so few cross-site scripting.
1325.10s - 1327.50s |  And none of them are trivial attack chains.
1327.50s - 1331.18s |  And all of them really have some weird logic bug
1331.18s - 1332.98s |  or some legacy system involved.
1332.98s - 1335.58s |  So we are seeing the effects of this working
1335.58s - 1337.50s |  at scale across billions of users
1337.50s - 1339.18s |  and hundreds of different web apps.
1339.18s - 1340.54s |  But how can we make this safer?
1340.54s - 1341.62s |  Where do we go from here?
1341.62s - 1343.42s |  What's the next logical step?
1343.42s - 1345.42s |  And we believe that, remember how
1345.42s - 1346.62s |  we talked about trusted types?
1346.62s - 1351.42s |  And it blocks a lot of these inherently unsafe DOM APIs
1351.42s - 1354.26s |  in the web platform, or at least makes it harder to use.
1354.26s - 1357.14s |  What if we take that to the conclusion of the idea
1357.14s - 1359.46s |  and actually ban all of those usages
1359.46s - 1362.06s |  at runtime of these legacy DOM APIs
1362.06s - 1366.62s |  that are a little bit inherently unsafe as they've
1366.62s - 1367.54s |  been designed?
1367.54s - 1371.42s |  Can you build apps where we don't use any of those APIs?
1371.42s - 1373.82s |  And that's our future vision of where we go from here.
1373.82s - 1376.74s |  If we have new platform-level APIs for operations
1376.74s - 1379.26s |  that we currently use DOM APIs for,
1379.26s - 1381.86s |  such as maybe a browser-included sanitizer
1381.86s - 1385.10s |  implementation for HTML, maybe a templating solution,
1385.10s - 1386.86s |  then we've really come full circle
1386.86s - 1388.74s |  and have our runtime enforcements
1388.74s - 1391.66s |  influence a new suite of secure-by-default web platform
1391.74s - 1394.38s |  APIs to bring decisions that nudge developers
1394.38s - 1397.42s |  towards safer choices, maybe at the framework level,
1397.42s - 1399.62s |  from the very beginning when the first line of code
1399.62s - 1401.94s |  is written in a new code base, all the way
1401.94s - 1407.38s |  to the moment when it is executed on the client's
1407.38s - 1408.70s |  browser.
1408.70s - 1410.54s |  And this is a future we hope to get to,
1410.54s - 1412.94s |  one where we can safely lock down all the accesses
1412.94s - 1416.14s |  to many DOM APIs, along with a new suite of platform APIs
1416.14s - 1419.14s |  to replace those operations in a safe-by-default manner.
1419.14s - 1420.50s |  And we call this approach, we've
1420.50s - 1421.90s |  nicknamed it Perfect Types.
1421.90s - 1424.74s |  It is the logical evolution of Trusted Types.
1424.74s - 1426.50s |  And we see this as the next evolution
1426.50s - 1428.62s |  of Trusted Types and a post-cross-site scripting
1428.62s - 1429.26s |  world.
1429.26s - 1431.74s |  This, combined with the strict CSP configurations
1431.74s - 1434.50s |  we talked about and our framework-based approaches,
1434.50s - 1436.18s |  secure-by-design, safe-by-design,
1436.18s - 1438.18s |  should guide us to a new post-cross-site scripting
1438.18s - 1438.70s |  world.
1438.70s - 1441.70s |  And taking a step back for a second for an even bigger
1441.70s - 1445.82s |  picture, and as a final plug, this safe coding philosophy
1445.82s - 1447.98s |  and safe-by-design, trying to steer developers
1447.98s - 1449.48s |  in the right direction, is something
1449.48s - 1451.66s |  that extends beyond just JavaScript, cross-site
1451.66s - 1455.10s |  scripting, and is a core priority for all Google-scale
1455.10s - 1456.18s |  security efforts.
1456.18s - 1458.46s |  You can read more about our organization's work
1458.46s - 1459.14s |  at these links.
1459.14s - 1462.20s |  And Google is a signer of CISA's Secure-by-Design Pledge,
1462.20s - 1465.02s |  along with other industry and government leaders in the space.
1465.02s - 1466.82s |  And we've committed to sharing our knowledge
1466.82s - 1470.26s |  on how to build a safer future together.
1470.26s - 1472.74s |  All right, well, that is the end of our talk.
1472.74s - 1474.38s |  So thank you, everyone, for listening.
1474.38s - 1476.02s |  Hopefully, this was a good guide on how
1476.02s - 1477.46s |  to think about solving XSS.
1477.46s - 1479.62s |  I'll leave you to it.


url: https://www.youtube.com/watch?v=WsRSzJsrWAw
title: DEF CON 32 - Defeating Secure Code Review GPT Hallucinations - Wang Zhilong, Xinzhi Luo
type: video
author: DEFCONConference
ingestion_date: 2024-11-09
transcription_model: large-v2
chunking_status: completed
content:
{
  "webpage_url": "https://www.youtube.com/watch?v=WsRSzJsrWAw",
  "title": "DEF CON 32 - Defeating Secure Code Review GPT Hallucinations - Wang Zhilong, Xinzhi Luo",
  "description": "In this talk, we will discuss the strengths and limitations of LLMs for code analysis tasks like code search and code clone detection. We will show when the LLMs make mistakes and what kinds of mistakes they make. For example, we observe that the performance of popular LLMs heavily relies on the well-defined variable and function names, therefore, they will make mistakes when some misleading variable name is given. Anyone interested in exploring the intersection of AI and code security analysis can attend this talk.",
  "channel_url": "https://www.youtube.com/channel/UC6Om9kAkl32dWlDSNlDS9Iw",
  "duration": 1077,
  "channel": "DEFCONConference",
  "uploader": "DEFCONConference",
  "upload_date": "20241016"
}

0.00s - 8.00s | This text was transcribed using whisper model: large-v2

 Alright everyone, thank you very much for joining us. Our next talk is SDLC Nightmares Defeating
8.00s - 26.70s |  Secure Code Review H- er GPT Hallucination with Julo and April. Hello my name is Julo and uh I
26.70s - 33.70s |  just graduated from the Pennsylvania State University last year and now I'm working at the
33.70s - 40.70s |  security engineering industry and uh my research interest is focused on applied deep learning in
40.70s - 49.14s |  cyber security. And I'm April, Carnegie Mellon University graduate and currently a security
49.14s - 56.14s |  engineer as well. A fun fact about me is I learn and apply my infosec knowledge during writing
56.14s - 65.14s |  science fiction. Okay let's get started. As most of us might have noticed, nowadays large language
65.14s - 72.14s |  models have been playing an increasingly important role in our daily work and lives. Today we'll
72.14s - 79.14s |  discuss how LLMs can be applied to security code analysis and evaluate their reliability in
80.14s - 89.65s |  understanding program logic. Uh overview of our talk today, first we'll introduce LLMs and their
89.65s - 96.65s |  strengths in SDLC, then we'll uncover their limitations through an experiment. Based on our findings
96.65s - 103.65s |  we'll further discuss the root causes and their other related issues. Finally we'll summarize key
103.65s - 114.11s |  takeaways on how to better utilize LLMs in our work. So first of all what is large language model?
114.11s - 123.11s |  Basically LLM is advanced machine learning models that understand and generate human like text uh by
123.11s - 132.11s |  learning from fast data sets. They use deep learning especially uh transformer architectures to predict
132.11s - 140.11s |  the most probable word in a sentence. For example in this figure it estimates uh 91% probability that
140.11s - 149.11s |  the word blue follows the phrase the color of the sky is. LLMs generate coherent and contextually
149.11s - 156.11s |  appropriate text enabling tasks like answering questions, uh translation and our topic today
156.11s - 166.11s |  code analysis. Now let's talk about SDLC. A typical software development life cycle involves various
166.11s - 176.11s |  stages where LLM can significantly assist. By understanding semantics and recognizing patterns, LLMs
176.11s - 185.11s |  can comprehend natural language and analyze code. Performing tasks like code generation, test case
185.11s - 194.11s |  creation and also documentation. From a security perspective they also help with threat modeling and
194.11s - 203.11s |  identifying vulnerabilities. Ultimately boosting the efficiency and uh quality throughout the SDLC.
203.11s - 211.11s |  However LLMs can make mistakes. Here are two famous examples. When Google introduced their chatbot
211.11s - 220.11s |  bot it made a factual error during its very first demo. Another instance involved chat GPT which
220.11s - 229.11s |  confidently asserted that 9.11 is greater than 9.9. A mistake even an elementary student will recognize.
230.11s - 239.11s |  These errors stem from their tendency to state incorrect information as facts. LLMs often hallucinate like
239.11s - 245.11s |  what I do every time before sleeping or conceiving a science fiction story. They tend to make up
245.11s - 254.11s |  information because they're essentially autocomplete system based on probability rather than facts. Now
254.11s - 263.11s |  let's see their performance in code analysis. Before diving into the experiment let's first take a closer
263.11s - 271.11s |  look at the code itself. Technically source code is a sequence of tokens consisting of three types.
271.11s - 280.11s |  Keywords, operators and user defined names. Keywords and operators are defined by the specification of
280.11s - 288.11s |  each programming language and they control the actual logic flow of the program execution. So we categorize
288.11s - 297.11s |  them as logical features. Meanwhile user defined names, annotations and comments which don't influence the
297.11s - 307.11s |  program flow once it's compiled are categorized as literal features. So our goal for the experiment is to
307.11s - 315.11s |  determine what types of features in the code LLMs can actually learn and process. We'll mask the literal features
315.11s - 323.11s |  by replacing the changeable names that don't affect the program flow and then use the LLM to perform code
323.11s - 332.11s |  analysis tasks on this modified code. By evaluating the performance we'll assess whether LLM can effectively
332.11s - 341.42s |  process the logical features when literal features are absent or even misleading. To conceal the literal features
341.42s - 350.42s |  we'll adopt three kinds of anonymization. So here is an example of the bubble sort function in Java and the first
350.42s - 358.42s |  anonymization is for function name. So here we can replace the bubble sort string with our customized string.
359.42s - 367.42s |  And the second one is for variable names. Two examples here is the begin and end which we can replace with our
367.42s - 376.42s |  strings. And the last one is the method invocation name. Here we have a function pret as one of the arguments for
376.42s - 385.77s |  the bubble sort function so we can replace the invocation name with our customized string. Meanwhile to understand
386.77s - 394.77s |  how literal features affect LLM comprehension we also use two name replacement strategies. The first one is just
394.77s - 403.77s |  replacing it with some randomly generated string while the second one is shuffling the names with other meaningful
403.77s - 414.77s |  names from the code making it more misleading for users. For readers, sorry. So after literal features are concealed
414.77s - 424.77s |  the LLM performs two main tasks related to code analysis. The first one is code clone detection which identifies and
424.77s - 433.77s |  measures the similarity between different code fragments to see if they generate the same result when given the same
433.77s - 443.77s |  input. The second one is code search where it tries to retrieve the code fragment from a larger code corpus that
443.77s - 455.11s |  closely match a developer's intent which is expressed in natural language as the input. So here are the results we got.
455.11s - 463.11s |  It shows that anonymizing names significantly decreases model performance especially when we replace all three kinds of
463.11s - 472.11s |  names. In the code search task we also notice that anonymizing master definition name causes more degradation than
473.11s - 481.11s |  anonymizing master invocation name indicating a stronger correlation between the master definition name and their
481.11s - 491.11s |  purpose. As for different languages, anonymizing a single attribute affects Python more but changing all naming
491.11s - 501.11s |  attributes impact Java the most probably because the verbosity and the structure in Java code that can mitigate partial
501.11s - 512.10s |  anonymization are fully disrupted. Compared to the code clone detection task, the performance degradation is more
512.10s - 520.10s |  pronounced in code search task since will define variable and method names are crucial for relating the code snippet to its
520.10s - 531.10s |  descriptive purpose. So overall the results clearly show that while logical features are what matters for program to
531.10s - 541.10s |  execute, what LLMs processed are mostly the literal features from the code. Now I'll hand it over to Dr. Zhilong to discuss the
541.10s - 544.10s |  reasons and possible solutions.
544.10s - 557.52s |  Okay. So let's discuss some root cause for the misleading names and comments in the code. Basically we just observed three
558.52s - 566.52s |  types of the root cause for these misleading names and comments and they are malicious code committers and outdated comments
566.52s - 580.36s |  and bad coding practice. Malicious code contributor can contribute some malicious code to the open source projects and to avoid
580.36s - 590.36s |  being detected by the code reviewers and they usually can change the naming to some misleading names to avoid being detected by
590.36s - 607.10s |  the code reviewer. Secondly, in software development programs often change frequently update their code to fix some bugs or add
607.10s - 619.10s |  some new features. Well the comments corresponding to the code may not be updated in time. In this example, partial code was
619.10s - 632.06s |  removed during a new commit and however the corresponding comment was not removed accordingly. Comments that are not updated can
632.06s - 642.06s |  cause inconsistency with the code and we call them outdated comments. And outdated comments can mislead the light learning model
642.06s - 656.87s |  during code analysis. Also in bad coding practice, confusing variable names and misleading variable names could be given by even a benign
656.87s - 673.26s |  programmer and developer which can mislead the code reviewer too. So next we are going to cover some other issues when using the light
673.26s - 690.51s |  learning model to do the code analysis. Besides aforementioned limitations we also observe two limitations. The first one is lack of context
690.51s - 708.20s |  and the second one is diverse performance on different program language. We will show the detail. Most light learning models can only
708.20s - 725.20s |  handle a sequence of limited length. For example, GPT-3 can handle a sequence with 2,048 tokens at most. And the GPT-4 can handle 32,000 tokens
725.20s - 739.20s |  at most. However, the code for even small projects can have thousands lines of code and each line of code contains usually more than 10 tokens.
739.20s - 752.45s |  Therefore, the light learning model cannot consume all the code at once which will lead to lack of context issue. Let's show through this figure.
752.45s - 767.45s |  We have a big project here showing this figure and let's consider context of the light learning model through this blue box.
767.45s - 792.35s |  When we use the light learning model to analyze this whole project, we need to analyze part of them individually. In most cases, the light learning model is used to
792.35s - 807.35s |  analyze each function individually and cannot have a big picture of the whole project. So, we actually have written an ARXIV preprint to dive into this problem.
807.35s - 830.20s |  We also propose some solution to solve this problem and for people who are interested in the details can read this ARXIV preprint. Also, we observe that light learning model has a diverse performance
830.20s - 850.20s |  across different program languages and different light learning models also have different strengths across different program languages. We did a quick experiment to evaluate the light learning model's ability to infer the program logic on two program languages.
850.20s - 878.26s |  The result is shown in the table here. We can observe that the GPT 3.5 has shown the best results on PHP and Go. However, Gemini can show the best results on PHP and Python.
878.26s - 904.89s |  Next, we will discuss how to avoid this issue in using light learning model for code analysis. We start from some ad hoc solutions. If the code is committed by a trusted developer, we should expect them to improve the quality of the name convention.
904.89s - 921.89s |  If the code is committed by some untrusted developer, we should pay special attention in case the light learning model is fooled by the malicious code committer, which can easily be achieved based on our experiments.
921.89s - 952.76s |  For example, we could deploy some naming logical inconsistency detection and use it to detect the inconsistency before we perform the code review. However, to really solve this problem, we believe that we need to change the model a little bit specifically for the code analysis.
952.76s - 966.76s |  A current popular pre-training task for the light learning model, as far as we know, is mask token prediction. In the mask token prediction task, it masks off some tokens in the sequence.
967.76s - 985.76s |  Take this as an example. It masks off some tokens from its sequence and uses neural network to predict this token. This is how current light learning model is trained, as far as we know.
985.76s - 1006.76s |  The mask token prediction task mainly focuses on the literal feature when applied to the source code analysis. But if we apply this model to the neural language NLP, it's okay because it does not have too much of the logic features.
1006.76s - 1025.76s |  But for the source code, we have a lot of the logic features. So, to really solve this problem, we think that we need to add some more pre-training tasks to train a new large program language model.
1025.76s - 1048.76s |  We should add some logic feature prediction tasks so that it can better capture the logic feature, not the literal feature. Because, as we show, relying too much on the literal feature will result in some issues, as we have demonstrated in the experiments.
1048.76s - 1072.24s |  So, for the audience who are interested in the details of this talk, you can refer to this ARX that we pre-printed for more details. Here is our contact information. And that's all of our talk.


url: https://www.youtube.com/watch?v=1upEyCKVpkI
title: DEF CON 32 - The Immortal Retrofuturism of Mainframes and How to Keep Them Safe- Michelle Eggers
type: video
author: DEFCONConference
ingestion_date: 2024-11-09
transcription_model: large-v2
chunking_status: completed
content:
{
  "webpage_url": "https://www.youtube.com/watch?v=1upEyCKVpkI",
  "title": "DEF CON 32 - The Immortal Retrofuturism of Mainframes and How to Keep Them Safe- Michelle Eggers",
  "description": "When you used your debit card today, do you know where that transaction was sent? Though it may conjure archival images of a 1950\u2019s IT room stocked with enormous, low-tech machines, Mainframe technology is both modernized and heavily relied upon today.\n\nMainframe architecture is some of the most reliable tech available, able to manage incredibly large input/output volumes with low risk of downtime and there are few signs of it being sunset in the decades to come. As protectors of the cyber landscape, understanding how mainframes are incorporated into a businesses topology and ways to secure mainframe architecture will remain important for any entity that utilizes this technology.\n\nIn this talk we'll explore the pervasiveness of mainframe technology, why it will remain relevant to the future landscape of mission critical-applications, and several trusted solutions for helping to secure these incredible computers.",
  "channel_url": "https://www.youtube.com/channel/UC6Om9kAkl32dWlDSNlDS9Iw",
  "duration": 1547,
  "channel": "DEFCONConference",
  "uploader": "DEFCONConference",
  "upload_date": "20241016"
}

2.54s - 4.04s | This text was transcribed using whisper model: large-v2

 All right
4.04s - 11.44s |  Thanks everyone for being here. We know it's Sunday and 10 a.m. So my heart goes out. I appreciate you so much today
11.44s - 16.48s |  We're gonna talk about the immortal retrofuturism of mainframe computers and how to keep them safe
16.48s - 22.24s |  I will give you a small disclaimer. I am coming from of course the red team side offensive security side
22.24s - 28.16s |  So this how to keep safe tips will not be as expansive as perhaps someone who is on the defensive side
28.16s - 30.28s |  But we will touch on it a little bit. So to begin
30.64s - 32.64s |  Let me see
32.68s - 33.88s |  Yes
33.88s - 35.88s |  It worked. Okay. This is me
35.88s - 40.72s |  It looks a little weird. It's all blown out. But I am Michelle Eggers. I'm a security consultant with Netspy
40.72s - 46.80s |  I've been with them for a couple of years. I specialize in mainframe web application and network penetration testing
46.80s - 48.56s |  I have a background in accounting and finance
48.56s - 53.88s |  Which is a little bit linked to mainframe because you know, the financial industry is essentially held up by mainframe
53.88s - 59.32s |  We'll touch on that in a minute more here, and I'm passionate about legacy tech. I like old things
59.32s - 65.60s |  I think they're interesting and cool and we are still using them. So it's important that we make sure we're still securing them
65.60s - 69.72s |  We can't forget to pay attention to the entire topology. All right, so
70.92s - 73.48s |  This is our outline. We're gonna do a history debrief
73.64s - 77.48s |  We're gonna look at how it kind of started and some things that have changed over the years
77.64s - 83.44s |  We're gonna look at mainframes today why people still elect to use them. What's special about them? What even are they?
83.84s - 86.44s |  We're gonna look at a few modern threats some notable
87.24s - 92.80s |  Breaches that have happened in the last decade or so and then some things that we see in our pen tests currently
93.68s - 97.08s |  Yeah, and I have a demo which is really cool. That shows a real thing that happened
97.08s - 100.48s |  It's a little scary, but it'll be interesting and then of course the tips to secure
101.28s - 107.00s |  Okay, so before we get into the history and all of that. I want to know whoever doesn't currently work with mainframe
108.04s - 113.72s |  Anybody want to volunteer what you think mainframe is give a description on it?
113.72s - 117.88s |  Just a sentence or or two any takers
118.96s - 139.88s |  Yes, right. So we have she said a cluster of computers with sort of different
140.76s - 142.76s |  applications use cases
143.00s - 145.00s |  Yes
145.52s - 148.12s |  Close you're on your way
149.72s - 155.57s |  What's that everything's bigger yeah, well depending yes
155.57s - 157.57s |  Hmm
157.65s - 162.25s |  Yes, wait, yeah, they're they're big systems obviously she's yes, of course
166.45s - 174.55s |  Yes, I love it. Okay, great. Everybody's super smart. You'll get big gold stars
176.15s - 182.07s |  So mainframes are actually just computers, right they're specialized computers that have insane
182.43s - 188.59s |  Processing power storage power their input output stuff it billions of turns that transactions a day literally billions
188.83s - 193.09s |  So it's not something like a quantum computer where you're having a highly
193.91s - 195.55s |  complex
195.55s - 198.63s |  Transaction that's occurring. They're more simplistic, but they have to be
199.15s - 204.35s |  Incredibly precise and very very fast and happening in tandem at the same time happened
204.35s - 211.43s |  They did parallel sysplex makes this possible and they have unique operating systems like, you know, IBM Z right and Linux
211.79s - 217.11s |  So very cool. You're not gonna find Windows on a mainframe. Just I hope not. Please don't
219.59s - 221.31s |  I don't do it
221.31s - 223.31s |  So let's look at a few myth versus facts
223.79s - 228.43s |  Something that has been fun for me this week is this first myth is something I've had to fight with people
228.91s - 236.27s |  Basically every day on like Oh mainframe. Oh, that's like 1960s as 4000 and I'm like no no
238.71s - 243.87s |  They've continuously evolved right there are some of the first systems to introduce virtualization
244.55s - 250.31s |  Built-in encryption their continue. I mean, there's more things coming out all the time. There's real-time a inferencing
250.31s - 254.73s |  There are new things coming out next year from IBM. I'll tell you a little bit more about that, too
255.39s - 257.51s |  But they're not locked in the past
257.51s - 264.71s |  It's not like this green screen terminal from a 1980s movie or this image of this woman in this room with base
264.71s - 268.95s |  That's basically one computer in the entire room. We're way beyond that
269.23s - 272.07s |  Okay, another myth is that cloud replaces mainframe?
272.63s - 276.91s |  Due to regulatory compliance reasons, this is impossible in many scenarios
277.59s - 280.59s |  Also, it's kind of expensive to move off of the mainframe
280.63s - 285.99s |  So that's another reason why if you know if you have a use case for it and you've implemented it
285.99s - 291.03s |  It's kind of a who's you to keep it running because once you have it, it's actually pretty inexpensive
291.15s - 297.39s |  It's less expensive than cloud in that regard. You can also use them to host your own virtual private cloud, which is nice
297.91s - 303.35s |  And then hybridization, okay, so if you're gonna look at cloud and mainframe, it's not gonna knock mainframe away
303.35s - 304.47s |  You're not gonna replace it
304.47s - 310.51s |  But we are seeing more and more cases of hybridization where certain applications and functions are in the cloud
310.59s - 314.43s |  Right, and there's you know, just API calls or whatever making these things possible
315.15s - 319.19s |  Oh, yeah, and then I like this the CLT that is extract load and transform
319.19s - 326.87s |  So that's a way of getting your data off of the mainframe into a different environment so that you can do more analysis, right?
327.87s - 331.83s |  Love it. Okay, and then another myth is it's just too specialized. No one knows what it is
331.83s - 337.43s |  Nobody knows how to do it. There really are literally maybe like five to ten mainframe hackers in the world
337.87s - 340.87s |  So that's real. That's true, but there are
341.51s - 343.35s |  moves underway
343.35s - 349.77s |  To rebuild that workforce. It sort of took a hit in the mid 90s to the early 2010s in staffing
349.91s - 354.63s |  The funding went down as far as you know building people bringing them into the workforce
354.75s - 355.95s |  But
355.95s - 360.43s |  I was actually I was at share which is like one of the biggest mainframe conferences that happens in a year
360.43s - 366.39s |  They do it twice and I met a lot of people that were early career professionals that are you know in college and they're learning
366.39s - 372.79s |  Cobol and we're like really? Oh, that's great. They're becoming these sort of system programmers in the mainframe environment
372.79s - 378.43s |  There's there's money going into it now because they're still being used and there's still innovation happening and people kind of woke up and said
378.43s - 383.75s |  Oh, they're not dead. We have to keep going. So oh and then this Watson X code assist
383.91s - 390.07s |  That's that's a cool feature that's coming out and it's basically just means you can take your cobalt and use these tools
390.11s - 393.27s |  To help translate it into a language like Java or Python
393.27s - 396.59s |  It's a lot easier to find developers who are specialized in those languages
396.59s - 401.07s |  It's much more common than someone who specializes in cobalt, even though we are rebuilding cobalt people. So
401.79s - 404.91s |  All right. I love old pictures. I hope you do too brief history
405.91s - 408.19s |  We need these lights down. It's all washed out
409.19s - 416.23s |  1937 okay. We've got u.s. U.s. Navy Bureau ships pre mainframe technology. These were not digital systems, right?
416.39s - 421.07s |  We're just talking about analog computers mechanical calculators. These are fairly gunnery
421.67s - 423.15s |  calculations
423.15s - 429.83s |  Navigation this is very military. This is pre-world war two stuff, but it's sort of like the the bones where it began
429.83s - 432.75s |  This is where we started seeing innovation in this area
433.51s - 435.51s |  1951 the Eckert Mockley Computer Corporation
436.03s - 442.07s |  They made it possible for businesses to now get this technology for their own use
442.07s - 448.23s |  It was no longer simply relegated to government. However, it was the UNIVAC system. That was for the census
448.23s - 455.35s |  I there's 1950 census and then they rolled out the UNIVAC in 1951 to help process that data that they pulled in that year
455.35s - 457.11s |  So really cool
457.11s - 458.87s |  Time marches on
458.87s - 465.55s |  When is this next one? Yes, I oh, I love this one. Okay, so unified architecture was a major achievement
466.71s - 472.51s |  Previous to the unified architecture you had let's say you had several different types of mainframe boxes
472.87s - 474.87s |  They could not speak to each other
475.19s - 480.15s |  They didn't have you know similar software or even the language wasn't quite the same
480.15s - 486.61s |  And so you'd have to have a person trained up on each one or different people trained on each one and they could not work together
487.29s - 490.93s |  But with unified architecture it changed all of that and now you could have you know
490.93s - 494.61s |  A few different types of machines and they could communicate together really nice
494.61s - 498.49s |  And then of course the vacuum tubes were replaced by this
499.29s - 503.81s |  Sorry, the stall solid-state technology if you have vacuum tubes in an earthquake
504.77s - 508.73s |  That's just bad news. Like it's not gonna live. So it got a lot better
509.61s - 514.35s |  1970 the innovation continued we had magnetic ferrite cores replaced by silicon drem memory chips
515.03s - 519.15s |  Virtual memory was introduced and dynamic address translation was also introduced
519.15s - 526.03s |  So as we're seeing mainframe is really pushing right? It's not lagging behind. It's constantly sort of at this
526.99s - 528.19s |  parallel
528.19s - 533.79s |  Level with innovation as along with other technologies. It's just maybe not as talked about
535.51s - 540.11s |  1991 the saga continues. This is great. This guy Stuart Alsop. He was
540.87s - 544.59s |  He was like a talking head right for for the technology
546.63s - 549.91s |  You really wrote new newspaper articles or something but
550.99s - 553.99s |  He said that mainframe would be dead that it would be sunset
553.99s - 559.35s |  The last one's gonna be unplugged and this very specific date and that was published that date
559.35s - 562.59s |  He like stood on that hill and he was very much wrong
563.67s - 567.43s |  Because as we can see mainframe is still being used, right?
568.23s - 573.75s |  By finance by military health care if you use your debit card today
574.43s - 581.11s |  That transaction touched a mainframe if you have ever been to a doctor or you go to a hospital for something
581.11s - 586.75s |  Your information is in a mainframe somewhere. It is so now, you know
587.95s - 591.03s |  And then we've got the new developments coming along something cool
591.79s - 593.15s |  so
593.15s - 596.51s |  There's already AI on the I have the z15
597.07s - 601.39s |  But this next coming year IBM is releasing a new box. That's gonna have
602.03s - 606.53s |  more increased capabilities and what we're gonna see is the the real-time
606.87s - 612.23s |  Inferencing is going to step in in such a way that if there's fraud, right?
612.43s - 618.07s |  So this actually happened to my family pretty recently like someone's at home and then they get a call 10 or 15 minutes later
618.07s - 621.47s |  Hey, are you at Legoland? Did you just spend $800? No, okay
621.47s - 623.75s |  We're gonna cancel your card and reissue a new one
623.75s - 628.63s |  It's this whole process and then that company has to eat that cost because you as you know
628.63s - 631.03s |  The account holder and there's fraud you're not gonna pay for that
631.03s - 635.47s |  Hopefully you shouldn't pay for that and then are they gonna catch the person who did the fraud likely?
635.47s - 638.79s |  No, so these companies are just eating all these fraud costs constantly
638.91s - 645.07s |  Well, if they can catch the fraudulent charge as it's occurring instead of you know, even five minutes later
645.23s - 647.23s |  It's gonna really reduce
647.35s - 651.15s |  That cost that inconvenience having to get a new card. I
652.11s - 656.25s |  Love it. All right. So why do people use mainframe now?
657.91s - 659.31s |  Why do we care
659.31s - 662.95s |  There are three reasons you'll hear this acronym in the mainframe community
662.95s - 668.19s |  It's kind of like sell sales talking points, but reliability availability and serviceability
668.19s - 672.19s |  So if you're looking at the New York Stock Exchange or Nasdaq something like that
672.19s - 678.83s |  They have to have what is called the five nines of reliability. So it's ninety nine point nine nine nine percent uptime
678.83s - 683.83s |  They cannot have more than five and a half minutes of downtime in a given year
684.95s - 691.19s |  They cannot right and so mainframe is able to do this. It's got well, it's not gonna go through
691.35s - 697.03s |  Let's break out the slides a little more built-in redundancy across your hardware levels your input output paths your power supply
697.47s - 701.99s |  You've got error detection correction. You've got these machines that are so robust
702.39s - 707.55s |  They can withstand up to seven magnitude earthquakes, right? The vacuum tubes are gone. They're strong now
709.31s - 712.11s |  Availability, right? We want to make sure that the machine itself
713.67s - 718.87s |  Beyond it just being strong you can still actually get your data out of it and it's accurate data at any given time, right?
718.87s - 724.11s |  It's available to you. So you've got transaction rollback checkpoint restart complex job scheduling
724.15s - 728.11s |  they have a hundred ninety configurable cores in this one specific example of the
728.83s - 734.19s |  Z15 right so you can basically finally tune your machine you make like a bespoke system for yourself
734.87s - 738.67s |  Basically out of the box you can start tuning it and fitting it to your specific use case
738.91s - 743.51s |  And again with the unified architecture, they can all talk to each other. So we love it and then the service ability
743.51s - 749.27s |  This is actually one of my favorite features. They're modular by design. Okay, so and then I mentioned parallel sysplex
749.27s - 755.71s |  It means if you have a workflow or process that is relegated to one L power one logical partition, right? It's like the virtual
756.95s - 758.87s |  segmentation within your mainframe
758.87s - 765.87s |  You can just swap those workflows over to another section of your box without any interruption without any issues
765.87s - 770.79s |  So if you need to change a card out or you need to do some kind of repair the machine itself never has to go
770.79s - 778.27s |  Down ever you just move the pieces you need to move when you need to move them because they're so modular. It's intentional
778.87s - 784.59s |  Got the in-depth logging for problem-solving on-site accessible for repair maintenance. So I
785.67s - 792.75s |  Live by a Google plant, right and you know, they're just what is the clouds just other people's computers. We understand this
792.75s - 799.03s |  So I'm driving home and it's dark and I'm on the freeway and I can see the plant kind of over near the river
799.03s - 803.43s |  and there are all these police cars and a couple fire engines an
803.71s - 808.79s |  Ambulance and all the lights are going like this and then keep driving like what's going on and then the road to get to the plant
808.79s - 814.43s |  Same thing it's blocked off. I don't know what's going on if your data is there. Are they gonna tell you?
815.27s - 817.03s |  We're like, oh, well, that's just one region
817.03s - 817.23s |  well
817.23s - 818.19s |  what if there was I mean
818.19s - 819.51s |  I don't know if it was a
819.51s - 824.83s |  Physical breach or if there was a fire if there's some other kind of problem and they'll never tell you like well
824.83s - 826.63s |  Oh, well, it's a redundancy. We have different regions
826.63s - 826.87s |  Okay
826.87s - 832.07s |  what if there was a coordinated attack against multiple regions at the same time and they just so happen to be the regions that
832.07s - 834.07s |  your data
834.27s - 835.71s |  Are stored in
835.71s - 842.75s |  something to think about if it's on-site and you have incredibly sensitive business critical data and
843.39s - 845.39s |  You can protect it with your own security
845.39s - 850.07s |  You know if there's a fire, you know, your organization knows if there's some kind of physical breach
850.71s - 852.71s |  You're gonna you have a better
852.95s - 856.59s |  Just you have more insight into the security of your own systems
856.87s - 857.79s |  You're not you know
857.79s - 863.87s |  Hoping Azure is gonna call you and say hey somebody broken and start a bunch of fires and hopefully didn't affect your stuff. So
864.95s - 866.47s |  and
866.47s - 872.39s |  This is this I love this. Okay. So this is a little demo. Oh, I wonder if it'll play. Oh, I wish it will
873.39s - 881.07s |  It may not darn well basically it's not gonna play they do this testing I'll just simulate
887.75s - 891.15s |  So in New Jersey about eight months ago there was a
891.99s - 898.11s |  4.5 magnitude earthquake and there's a campus near there. I mean that earthquake went up the coast as well. It affected a lot of places
898.67s - 905.83s |  There is a campus with 200 mainframes and not a single one was affected there was no damage whatsoever
906.51s - 908.51s |  You want to start an earthquake?
909.35s - 911.83s |  In your server room and see how it goes
912.71s - 914.63s |  Probably not, right?
914.63s - 919.87s |  Yeah, yeah a little bit scary mainframes can do that. They're literally built to do that
919.99s - 923.07s |  So apologies for the lack of the shaking I can show you later
924.07s - 926.07s |  All right, so
926.23s - 932.15s |  We've gotten a little bit of an understanding about why people care about mainframe why they use it. What are the benefits of it?
932.87s - 936.87s |  What are the things that can affect it? What are we scared of? Why?
937.55s - 939.67s |  Do we need to think about the security? So
940.59s - 946.23s |  One of the big things that affects mainframe are the integrations, right? We see a lot of
946.39s - 948.39s |  Entry
948.47s - 953.67s |  Into the mainframe environment not by not starting at the mainframe level, of course
953.67s - 959.15s |  But with things like web applications that connect to it or just the general network in this case
959.15s - 963.65s |  Even it was the Wi-Fi network. So the TJX company's data breach payment card data breach
964.59s - 970.75s |  45.7 million user cards were affected exfiltrated. This is a weakness in the wireless network. That's how that one kicked off
970.75s - 973.87s |  We're gonna see a trend here Heartland payment systems breach
974.59s - 980.23s |  Doubled the last hundred million cards compromised. This is at the hands of a global cyber fraud operation
980.31s - 983.91s |  Okay, and this was a combined attack of network and application vulnerabilities
984.31s - 986.31s |  Then we see the Equifax data breach
986.63s - 992.03s |  Okay, this was now a hundred and forty seven million users exposed which is just goes up every single time
992.15s - 997.51s |  It's worse every time in this instance. It was the web application that provided the initial foothold
998.27s - 1004.11s |  And this one is a little bit harder to find information on this is the logica
1004.51s - 1007.67s |  Data breach this was in Sweden and it affected the government
1008.19s - 1014.35s |  And it was so bad and word got out that the government just published the documents. They just put everything public
1014.35s - 1019.07s |  They said just here go ahead and find it. You can find it if you want to it's in Swedish
1019.11s - 1021.55s |  But you can find all the documents about this
1021.63s - 1023.63s |  This
1023.63s - 1028.59s |  This was initial access to be in the FTP network so unencrypted protocol, okay
1029.27s - 1034.23s |  The hackers use something like Hercules, which is an emulator. I use an emulator in my testing
1034.23s - 1035.83s |  I use x 3270
1035.83s - 1041.19s |  But I tell you this to say it's not that hard to get some of these tools to start getting into the places that maybe
1041.19s - 1042.67s |  You shouldn't be getting into
1042.67s - 1049.67s |  this attack specifically leveraged some zero days, which may be a little more difficult to you know make happen, but
1050.51s - 1052.95s |  They were based on default configurations
1053.51s - 1058.95s |  Right, so this is something that is easy to fix and not have in your environment
1058.95s - 1063.35s |  Please if just as soon as you have something new turn off your default configurations
1064.15s - 1066.15s |  Change it as soon as possible
1067.63s - 1069.27s |  120,000 username stolen from rack F
1069.27s - 1075.99s |  Rack F is like a security management tool in the mainframe environment and the guy you know what guy head guy who's in charge of it?
1075.99s - 1079.43s |  He was in Cambodia at the time stole a bunch of money. It was it was very bad
1079.43s - 1082.51s |  It was very bad and the entire government made all these new policies
1083.27s - 1085.55s |  specifically because of this this breach
1086.71s - 1090.19s |  Okay, so next up. I'm going to show you a demo. I'll give a little information on it
1090.95s - 1093.63s |  What you'll see me doing. Hopefully you'll be able to see it is
1093.63s - 1101.35s |  I'm authenticating to an L power to a logical partition right assume that the endpoints that I navigate to have already been enumerated
1101.95s - 1107.03s |  Previously during the engagement, so I'm not gonna you don't see the enumeration part in this demo
1107.03s - 1110.15s |  But you'll see me sort of navigate to some endpoints. That's why
1111.03s - 1117.27s |  The user I am that you'll see is a lower level user. This is not an admin user, right?
1117.27s - 1120.99s |  So of course it means there are specific resources that should they should not be able to access
1121.55s - 1123.55s |  Okay, so the goal of this
1124.19s - 1126.19s |  Hack was to try to access
1127.15s - 1129.71s |  of course more privileged information, so
1131.19s - 1133.87s |  Okay, and we're up and running okay, so
1134.87s - 1137.19s |  This is me logging in
1138.51s - 1140.51s |  Got TSO here
1140.95s - 1142.19s |  All right is PF
1142.19s - 1148.51s |  And then now you're gonna see me begin to access a command-line utility which is a Unix like feature within the mainframe environment
1149.39s - 1150.95s |  through OMVS
1150.95s - 1154.59s |  All right
1154.59s - 1159.07s |  So it's if it looks familiar to you a Unix base. You can just use your similar commands, right?
1159.11s - 1164.19s |  So I'm gonna navigate to this opt secrets location. I'm gonna see what's in there
1167.10s - 1170.58s |  List it out. We've got a readme.txt and an SSH key
1170.90s - 1174.94s |  We're gonna cat the readme.txt file says this is the SSH private key
1174.94s - 1177.18s |  You need to log on as an admin do not share well
1177.18s - 1180.30s |  Can I just cat SSH key and just maybe access it anyway?
1180.78s - 1183.66s |  No permission denied game over right wrong
1183.66s - 1189.90s |  I used the same IP that the LPAR is on and just navigated it to it in the web browser over 8080, right?
1190.90s - 1196.18s |  Again we already had the endpoints enumerated. Okay, so this specific location
1197.10s - 1200.22s |  The requests that are generated pull information
1200.90s - 1202.90s |  From the mainframe, right?
1203.02s - 1209.84s |  So just open up the network tab and you can look at the requests that have made and maybe modify it and see what happens
1209.84s - 1210.78s |  Okay
1210.78s - 1216.66s |  So I'm generating the request it makes a post request that pulls information from the mainframe. All right, we're gonna crack that open
1217.54s - 1223.64s |  You don't even need to use an intercepting tool for this you really can just use the network tab in your browser
1223.80s - 1225.52s |  It's a little bit insane
1225.52s - 1228.92s |  So in the message body, there's a request for a specific file
1229.64s - 1236.16s |  Modify that to point to what was disallowed earlier. Keep in mind. I have not authenticated. This is simply in the web browser
1237.04s - 1243.52s |  Okay, got a 200. Okay, that sounds promising. There's the endpoint I went to with that SSH key
1244.28s - 1246.28s |  Let's see what's in the response
1247.57s - 1249.57s |  There it is
1250.53s - 1252.53s |  I
1261.41s - 1265.97s |  Will just say within the past year to be somewhat vague and it was not an SSH key
1265.97s - 1274.09s |  But it was definitely some very very private information completely unauthenticated access not even on within the mainframe emulated environments
1274.13s - 1275.97s |  Just over the browser
1275.97s - 1280.13s |  So this is why I'm saying we need to look at the entire topology
1280.65s - 1286.97s |  And make sure we're investigating the integrations with the mainframe because people say well mainframe is it's older. It's safe
1287.49s - 1293.05s |  Yes, and no. All right. So we've talked about integration once what about the mainframe itself?
1293.05s - 1298.37s |  If you're just looking at the mainframe, we're gonna stay in this little box of the mainframe. Is it safe all by itself?
1299.25s - 1304.89s |  These are things we find on a regular basis within our pen tests. Okay local file include
1305.97s - 1307.97s |  We find insecure
1308.33s - 1309.41s |  Insecure
1309.41s - 1312.93s |  Protocols all the time. There's so much 23. It's yeah
1313.65s - 1316.73s |  Unauthenticated access happens. So there been instances where I will
1317.53s - 1323.97s |  Begin the authentication process and then back out of it not complete authenticating and I can still run kicks CICS commands
1324.37s - 1329.29s |  Or you know run Kim's things like this, which you should you should be authenticated to run these
1329.85s - 1334.25s |  So we do find this a lot injection sequel injection into DB to databases
1334.89s - 1339.17s |  JCL job job control language injections or rec scripts injections if you're able to
1339.89s - 1343.01s |  Use these when you're not supposed to you could delete modify
1343.89s - 1345.89s |  erase entire data sets
1346.37s - 1350.65s |  Bad news. Okay, and then we do see security misconfigurations
1350.65s - 1358.37s |  I I don't see a lot of default credentials thankfully, but it does occur weak password policies often. I will see
1359.21s - 1361.65s |  They'll allow mixed case passwords
1361.65s - 1363.81s |  So let's say you're supposed to have a cap
1363.81s - 1370.13s |  but like your actual password is capital this and lowercase this and some numbers and whatever else you can switch the case around and do
1370.13s - 1372.13s |  Whatever you want and we'll still accept it
1372.13s - 1373.89s |  It's like don't do that
1373.89s - 1374.93s |  and
1374.93s - 1382.97s |  Not using external security management tools like raka for top secret. Thankfully most shops that have mainframe are implementing these things
1383.97s - 1389.09s |  But it's just as a safeguard as a you know, just make sure that you're using these these things
1389.89s - 1393.05s |  Okay, and then now the promised ways to secure
1393.85s - 1399.01s |  From the blue team side properly configure your network topology as a whole. I can't stress that enough
1399.13s - 1403.21s |  Please mainframe is not its own weird thing off to the side
1403.21s - 1408.93s |  It is part of your stack treated as such make appropriate use of your logical partitions
1408.93s - 1415.25s |  Make sure if a user has access to L par one, they can't also somehow navigate to L part two and three and four
1416.77s - 1422.33s |  Your regions your ESM tools and see this again disallow unencrypted protocols secure with compliance
1422.89s - 1425.45s |  the CIS benchmarking is really great for
1426.17s - 1433.37s |  Regular auditing purposes and then the DSA stigs. These are your Department of Defense guides from NIST. I've used these on a test
1433.45s - 1437.17s |  I've actually it's really nice. So I had a DB to test where
1437.65s - 1439.65s |  It's basically sequel based and
1440.09s - 1444.41s |  They have whole lists of commands you can run to check. Okay. Well if I run this command
1444.41s - 1446.89s |  Can I access this when I shouldn't if I run this command?
1446.89s - 1452.69s |  Can I elevate my privileges and so on and like this is this is great. I love this very helpful helpful tools
1452.93s - 1459.77s |  Secure the ongoing reviews. This is real basic stuff that everybody should be doing regardless of its mainframe or something else
1460.17s - 1463.33s |  Audit your identity and access management on a regular basis, right?
1463.77s - 1468.25s |  Establish recurring audits for all features and do recurring pen tests
1468.25s - 1472.81s |  I love a good internal team literally like you're closer to your stack
1472.81s - 1478.29s |  You're rolling out new features you sort of understand what's going on and you need to be there, right?
1478.89s - 1482.77s |  But you know, I am biased again because I'm third-party pen tester
1482.77s - 1489.81s |  Which is I think is incredibly important to get fresh eyes on your about five minutes. Okay. Thank you. I'm hurry up here
1491.05s - 1493.05s |  fresh set of eyes
1494.01s - 1500.01s |  Not having the source code being you know, somewhat unfamiliar with the environment and then getting in and just
1500.81s - 1505.83s |  Gently breaking it so you can help the company fix it. I think is really really nice
1505.83s - 1511.01s |  It's a special place to be I love it. So again, what have we learned today?
1511.01s - 1513.29s |  We're gonna cover it real quick and a few bullet points
1514.17s - 1518.01s |  Mainframe is not going away anytime soon. Okay, please keep paying attention to it
1518.01s - 1525.11s |  We like we rely on it globally to support finance health care government all these critical industries
1525.93s - 1531.85s |  Okay, so we need to keep securing it. There are more possible vulnerabilities now due to the prevalence of integrations
1531.85s - 1533.53s |  It's just gonna keep increasing
1533.53s - 1540.01s |  But if we work together we can combat it. So thank you so much for coming to learn about mainframe


url: https://www.youtube.com/watch?v=6SNy0u6pYOc
title: DEF CON 32 - The Darkest Side of Bug Bounty - Jason Haddix
type: video
author: DEFCONConference
ingestion_date: 2024-11-09
transcription_model: large-v2
chunking_status: completed
content:
{
  "webpage_url": "https://www.youtube.com/watch?v=6SNy0u6pYOc",
  "title": "DEF CON 32 - The Darkest Side of Bug Bounty - Jason Haddix",
  "description": "Bug bounty is an intricate game between the bug hunter, the clients, and the intermediary.\n\nLike any game, it can be hacked. Like some games, it can be unfair.\n\nJoin Jason as he walks you through the darker secrets of bug bounty , tips and tricks to address them, and in some cases, commiserate that there are just bad realities to the game.\n\nJason will address the problems hardly talked about in the system, from a hacker, program owner, and platform runner.",
  "channel_url": "https://www.youtube.com/channel/UC6Om9kAkl32dWlDSNlDS9Iw",
  "duration": 1949,
  "channel": "DEFCONConference",
  "uploader": "DEFCONConference",
  "upload_date": "20241016"
}

6.51s - 8.51s | This text was transcribed using whisper model: large-v2

 Thank you, thank you so much.
8.51s - 10.51s |  My name is Jason.
10.51s - 12.51s |  I'm going to try to keep my energy up for this talk.
12.51s - 14.51s |  I usually do when I speak,
14.51s - 16.51s |  but I have a little bit of a
16.51s - 18.51s |  situation going on right now at home.
18.51s - 20.51s |  My wife's in the ER.
20.51s - 22.51s |  My kids are watching, my other kids,
22.51s - 24.51s |  and I've done four talks this week already.
24.51s - 26.51s |  So I'm going to try to keep it up, but if I'm a little bit
26.51s - 28.51s |  less than super, super
28.51s - 30.51s |  Jason Haddix, I'm sorry.
30.51s - 32.51s |  But I'm going to do my best.
32.51s - 34.51s |  This talk I've wanted to do for a long time.
34.51s - 36.51s |  It's called The Dark Side of Bug Bounty.
36.51s - 38.51s |  And it is a collection
38.51s - 40.51s |  of notes around
40.51s - 42.51s |  the kind of worst possible
42.51s - 44.51s |  scenarios that can happen in bug bounty programs.
44.51s - 46.51s |  So we did a little survey
46.51s - 48.51s |  originally about what you guys do, but
48.51s - 50.51s |  there are
50.51s - 52.51s |  four primary types of people, or three primary
52.51s - 54.51s |  types of people in the
54.51s - 56.51s |  equation of bug bounty. So there are
56.51s - 58.51s |  bug bounty hunters, people who find
58.51s - 60.51s |  vulnerabilities. How many bug bounty hunters do we have
60.51s - 62.51s |  in here? Okay,
62.51s - 64.51s |  good amount. There are
64.51s - 66.51s |  program owners, so they work
66.51s - 68.51s |  at the company that runs the bug bounty program.
68.51s - 70.51s |  Who are those people in the audience?
70.51s - 72.51s |  You could be triage, you could be program management,
72.51s - 74.51s |  you could be vulnerability management, stuff like
74.51s - 76.51s |  that. And then you have people that work on
76.51s - 78.51s |  the platform that do the triage, or any platform
78.51s - 80.51s |  people in here, like Bug Crowd, SYNAC,
80.51s - 82.51s |  HackerOne, Integrity, things like that.
82.51s - 84.51s |  Do we have any of those people in here?
84.51s - 86.51s |  You wouldn't want to raise your hand for this one.
86.51s - 88.51s |  I'm just kidding.
88.51s - 90.51s |  Okay, so why am I uniquely
90.51s - 92.51s |  situated to give this talk?
92.51s - 94.51s |  Because I've been
94.51s - 96.51s |  on all sides of the equation. So
96.51s - 98.51s |  I am ranked number
98.51s - 100.51s |  57th overall on Bug
100.51s - 102.51s |  Crowd as a hacker. I have been
102.51s - 104.51s |  hacking on Bug Crowd for a long, long time.
104.51s - 106.51s |  Recently I've started to build up
106.51s - 108.51s |  my rep on Integrity as well. I've
108.51s - 110.51s |  done HackerOne live events, I've done
110.51s - 112.51s |  Bug Crowd live events. I'm pretty much
112.51s - 114.51s |  well versed in the hacker area
114.51s - 116.51s |  of bug bounties.
116.51s - 118.51s |  I have also been on
118.51s - 120.51s |  the platform side. I worked at
120.51s - 122.51s |  Bug Crowd for six years.
122.51s - 124.51s |  Four of them as
124.51s - 126.51s |  the head of operations,
126.51s - 128.51s |  meaning I managed the triage team globally.
128.51s - 130.51s |  So that means I got to see
130.51s - 132.51s |  every bug bounty program that they ran.
132.51s - 134.51s |  Through my tenure I saw
134.51s - 136.51s |  tens of thousands of bugs filter
136.51s - 138.51s |  through the queue at Bug Crowd and managed
138.51s - 140.51s |  my teams to triage them.
140.51s - 142.51s |  I've
142.51s - 144.51s |  also known triagers that have gone
144.51s - 146.51s |  to other platforms and stay in touch with them.
146.51s - 148.51s |  Some of this stuff is not my direct experience,
148.51s - 150.51s |  it's notes from other people on other platforms.
150.51s - 152.51s |  I don't mention any platforms in this.
152.51s - 154.51s |  I think that all the platforms have
154.51s - 156.51s |  a lot of the same problems that I'm going
156.51s - 158.51s |  to talk about in these slides.
158.51s - 160.51s |  And then I've also been a program owner.
160.51s - 162.51s |  I've run a multinational company
162.51s - 164.51s |  as a CISO and so I had to purchase
164.51s - 166.51s |  Bug Bounty as a part of
166.51s - 168.51s |  a security program, as part of a vulnerability management
168.51s - 170.51s |  program and assessment program.
170.51s - 172.51s |  So we had to build a scope
172.51s - 174.51s |  and deal with running
174.51s - 176.51s |  a program just like everybody else does.
176.51s - 178.51s |  So I've got to see this
178.51s - 180.51s |  ecosystem grow from its
180.51s - 182.51s |  infancy to its adulthood
182.51s - 184.51s |  and got to see every skeleton in the
184.51s - 186.51s |  closet basically. So we're going to talk about some of the
186.51s - 188.51s |  notes I have. They're a little bit scattered but
188.51s - 190.51s |  hopefully they're fun to talk about.
190.51s - 192.51s |  The first thing we're going to talk about
192.51s - 194.51s |  is some of the really crappy things
194.51s - 196.51s |  that are going on right now in the
196.51s - 198.51s |  bug bounty industry in general.
198.51s - 200.51s |  And this is kind of
200.51s - 202.51s |  goes back to a thing that me and my friend
202.51s - 204.51s |  talk about is that
204.51s - 206.51s |  a lot of people have pivoted from
206.51s - 208.51s |  celebrating the creativity
208.51s - 210.51s |  of the bug bounty hunters who are
210.51s - 212.51s |  submitting really cool vulnerabilities
212.51s - 214.51s |  and scaling your security program in a way
214.51s - 216.51s |  that you really couldn't do before.
216.51s - 218.51s |  You had to really consult with pen testers and stuff like that
218.51s - 220.51s |  before. Now they're starting to
220.51s - 222.51s |  exploit them a little bit. And so we're going to talk about
222.51s - 224.51s |  two things that I think are kind of shitty
224.51s - 226.51s |  in that way. You guys don't mind if I drop
226.51s - 228.51s |  a curse word every once in a while, right?
228.51s - 230.51s |  Alright.
230.51s - 232.51s |  The first thing I'm going to talk about is AI.
232.51s - 234.51s |  So right now at all of the platforms
234.51s - 236.51s |  what they're doing is
236.51s - 238.51s |  they're taking bug bounty hunters
238.51s - 240.51s |  attack traffic and they're training
240.51s - 242.51s |  AI models on this attack traffic.
242.51s - 244.51s |  And what they're doing is they're building services out of it.
244.51s - 246.51s |  They're building threat intel feeds for
246.51s - 248.51s |  program owners to understand
248.51s - 250.51s |  what type of attacks exist
250.51s - 252.51s |  in hardware hacking, web hacking, all kinds
252.51s - 254.51s |  of different hackings. And
254.51s - 256.51s |  then they're also building vulnerability scanners
256.51s - 258.51s |  out of training off of these models
258.51s - 260.51s |  on this attack traffic.
260.51s - 262.51s |  And what's happening is that they're basically stealing
262.51s - 264.51s |  attack data and research data from
264.51s - 266.51s |  researchers and they give no payments to
266.51s - 268.51s |  the bug hunters for this.
268.51s - 270.51s |  We get nothing for it basically.
270.51s - 272.51s |  So for instance
272.51s - 274.51s |  I find a zero day tomorrow
274.51s - 276.51s |  on SAP
276.51s - 278.51s |  or something like that. And I submit it to a bug bounty
278.51s - 280.51s |  platform. They will automatically train on it
280.51s - 282.51s |  with the attack string I used. And then
282.51s - 284.51s |  they'll either turn it into a threat feed or a scanner
284.51s - 286.51s |  give the benefit to all of their
286.51s - 288.51s |  customers but never give me any
288.51s - 290.51s |  kickback on that vulnerability. So I only get
290.51s - 292.51s |  paid once and maybe they found hundreds of
292.51s - 294.51s |  thousands of instances of that vulnerability.
294.51s - 296.51s |  So this is a really shitty part of the
296.51s - 298.51s |  industry and I'm hoping that this changes.
298.51s - 300.51s |  I'm hoping that we can move towards a model
300.51s - 302.51s |  where
302.51s - 304.51s |  if that happened and I didn't have
304.51s - 306.51s |  access to a program where
306.51s - 308.51s |  I couldn't submit that SAP vulnerability
308.51s - 310.51s |  that they would automatically be able to tell
310.51s - 312.51s |  me and say hey we did locate
312.51s - 314.51s |  this on 35 other customers
314.51s - 316.51s |  here's a 20% kickback
316.51s - 318.51s |  because it was your vulnerability research.
318.51s - 320.51s |  So this is how I'm hoping this changes
320.51s - 322.51s |  but right now there's no indication that that's going to happen
322.51s - 324.51s |  and they will continue stealing
324.51s - 329.07s |  from bug hunters.
329.07s - 331.07s |  Another one is cloud
331.07s - 333.07s |  WAFs. So
333.07s - 335.07s |  in the industry obviously you have
335.07s - 337.07s |  companies that are cloud based
337.07s - 339.07s |  web application firewalls or cloud based
339.07s - 341.07s |  protection suites. These
341.07s - 343.07s |  products are ubiquitous to
343.07s - 345.07s |  every application these days and
345.07s - 347.07s |  all of your internet connected infrastructure.
347.07s - 349.07s |  These are things like Akamai and CloudFlare
349.07s - 351.07s |  and
351.07s - 353.07s |  some of the other firewall vendors have these type of things.
353.07s - 355.07s |  So
355.07s - 357.07s |  a couple weeks ago
357.07s - 359.07s |  a friend of mine was streaming
359.07s - 361.07s |  and he's streaming on Twitch
361.07s - 363.07s |  and he's streaming hacking stuff
363.07s - 365.07s |  and we're talking
365.07s - 367.07s |  in Twitch chat because we all like to support
367.07s - 369.07s |  each other and he's like a content creator slash
369.07s - 371.07s |  streamer and we get to talking
371.07s - 373.07s |  and we're talking about a
373.07s - 375.07s |  new AI tool that we're going to build
375.07s - 377.07s |  and live in chat a representative
377.07s - 379.07s |  from one of these
379.07s - 381.07s |  companies basically says
381.07s - 383.07s |  oh that's really cool I'm going to build a defense against
383.07s - 385.07s |  that and take it. So then
385.07s - 387.07s |  we end up talking to a whole bunch of
387.07s - 389.07s |  researchers around
389.07s - 391.07s |  kind of around the scene
391.07s - 393.07s |  and we compare some notes
393.07s - 395.07s |  and some conversations we've had
395.07s - 397.07s |  and it turns out that there's
397.07s - 399.07s |  a lot of kind of nefarious
399.07s - 401.07s |  stuff going on. So
401.07s - 403.07s |  if you're a bug bounty hunter and you submit
403.07s - 405.07s |  attack traffic sometimes
405.07s - 407.07s |  the platforms ask you to put a web header in there
407.07s - 409.07s |  so they can recognize it's you.
409.07s - 411.07s |  Now anybody between your home
411.07s - 413.07s |  machine and the client you're attacking
413.07s - 415.07s |  and the bug bounty hunting
415.07s - 417.07s |  company when you submit the report
417.07s - 419.07s |  can identify you by that header. It's a unique
419.07s - 421.07s |  header. It's usually
421.07s - 423.07s |  like hacker one something or whatever
423.07s - 425.07s |  you know. Now we didn't
425.07s - 427.07s |  know until very recently is that
427.07s - 429.07s |  Akamai, CloudFlare
429.07s - 431.07s |  is like this. Do any of you work for Akamai or CloudFlare?
433.07s - 435.07s |  They're monitoring
435.07s - 437.07s |  about a list of
437.07s - 439.07s |  250 of the best bug bounty hunters
439.07s - 441.07s |  in the world. They're monitoring
441.07s - 443.07s |  our traffic, our attack patterns, what we're
443.07s - 445.07s |  doing and they're trying to build just in time
445.07s - 447.07s |  protections for the type of attacks we're doing
447.07s - 449.07s |  and sell it as part of their rule set
449.07s - 451.07s |  without giving us any credit.
451.07s - 453.07s |  They're also
453.07s - 455.07s |  sending representatives to watch our streams
455.07s - 457.07s |  monitor our
457.07s - 459.07s |  GitHub commits, go to our talks
459.07s - 461.07s |  that's why I asked if anybody was in here
461.07s - 463.07s |  and monitor our chats
463.07s - 465.07s |  on the
465.07s - 467.07s |  open bug bounty discords and slacks.
467.07s - 469.07s |  Then they
469.07s - 471.07s |  take this data and they share it with each other
471.07s - 473.07s |  so they have agreements
473.07s - 475.07s |  with each other to share attack data so that they can
475.07s - 477.07s |  do analytics on it. And in some of the
477.07s - 479.07s |  worst cases we've had, bug bounty
479.07s - 481.07s |  hunters have released tools
481.07s - 483.07s |  that were really, really good tools in the offensive
483.07s - 485.07s |  security space and these
485.07s - 487.07s |  vendors have gone out and strong armed
487.07s - 489.07s |  young impressionable researchers and
489.07s - 491.07s |  forced them to take these tools offline
491.07s - 493.07s |  or at least pressured them to do so.
493.07s - 495.07s |  So this is the kind of stuff we deal
495.07s - 497.07s |  with. Now the solutions here
497.07s - 499.07s |  are just better
499.07s - 501.07s |  communication and a technical
501.07s - 503.07s |  solution to bug bounty hunting.
503.07s - 505.07s |  I'm a big believer that
505.07s - 507.07s |  that's top 250
507.07s - 509.07s |  people that they feel like they need to monitor
509.07s - 511.07s |  for that kind of stuff.
511.07s - 513.07s |  First of all, they just get in our way. They're
513.07s - 515.07s |  free by obscurity. It doesn't take
515.07s - 517.07s |  me long to bypass one of those cloud
517.07s - 519.07s |  based wafts. It's just kind of an annoyance.
519.07s - 521.07s |  But I don't feel good about
521.07s - 523.07s |  the data sharing between companies.
523.07s - 525.07s |  And so if they want to just
525.07s - 527.07s |  put some research time rather
527.07s - 529.07s |  into doing all this against us and just
529.07s - 531.07s |  make a global header that allows
531.07s - 533.07s |  a background checked bug bounty
533.07s - 535.07s |  researcher who is well known in the community
535.07s - 537.07s |  to bypass all this stuff
537.07s - 539.07s |  and they don't have to monitor and I'll just work with them.
539.07s - 541.07s |  I'd much rather have a community that worked like that
541.07s - 543.07s |  than one that works like this.
543.07s - 547.86s |  Okay. Let's talk about
547.86s - 549.86s |  triage in bug bounty.
549.86s - 554.86s |  Okay. So if you've ever run a bug bounty
554.86s - 556.86s |  how budget works
556.86s - 558.86s |  in a bug bounty is and how you purchase
558.86s - 560.86s |  a bug bounty is usually you go into
560.86s - 562.86s |  the bug bounty platform and you're like, hey, I want
562.86s - 564.86s |  to negotiate. I want to buy bug bounty.
564.86s - 566.86s |  And what you're buying when you buy bug bounty
566.86s - 568.86s |  is you're buying the platform
568.86s - 570.86s |  to ingest the bugs usually.
570.86s - 572.86s |  And then you're buying maybe triage
572.86s - 574.86s |  associated to that. You're getting triage
574.86s - 576.86s |  from the platform.
576.86s - 578.86s |  And then you're also paying
578.86s - 580.86s |  your reward pool
580.86s - 582.86s |  in the initial payment.
582.86s - 584.86s |  And so it's usually structured as a one year deal
584.86s - 586.86s |  or a three year deal.
586.86s - 588.86s |  You get a price break if you go
588.86s - 590.86s |  for a three year deal with the platforms.
590.86s - 592.86s |  Now the reason that
592.86s - 594.86s |  you pay for all of these things
594.86s - 596.86s |  in one lump sum is
596.86s - 598.86s |  usually because you're trying to skirt your
598.86s - 600.86s |  finance group at your company.
600.86s - 602.86s |  So it's a
602.86s - 604.86s |  financial
604.86s - 606.86s |  trap to
606.86s - 608.86s |  pay bug bounty hunters out
608.86s - 610.86s |  from your company. It requires a whole
610.86s - 612.86s |  bunch of tax paperwork for each bug
612.86s - 614.86s |  that you pay out individually. This is why
614.86s - 616.86s |  people use the platforms a lot.
616.86s - 618.86s |  So what you
618.86s - 620.86s |  do is you pay for the platform, you pay for the triage
620.86s - 622.86s |  and then you pay for your first
622.86s - 624.86s |  year of bug bounty rewards all as
624.86s - 626.86s |  one lump sum to the platform.
626.86s - 628.86s |  And then they hold an escrow
628.86s - 630.86s |  the money for the payouts to the hackers.
630.86s - 632.86s |  And so that's called the reward pool.
632.86s - 634.86s |  And
634.86s - 636.86s |  that way it's just one lump
636.86s - 638.86s |  sum that your finance team sees and
638.86s - 640.86s |  you don't have to do the taxes. BugCrowd has
640.86s - 642.86s |  to do the taxes or HackerOne has to do
642.86s - 644.86s |  the taxes for each bug
644.86s - 646.86s |  payout which is a tremendous
646.86s - 648.86s |  benefit to your financial teams.
648.86s - 650.86s |  And so that's kind of how budgeting
650.86s - 652.86s |  works and how payment works for a program.
652.86s - 656.27s |  Now
656.27s - 658.27s |  a very common situation
658.27s - 660.27s |  in the worst part of bug bounty
660.27s - 662.27s |  is that you start a bug
662.27s - 664.27s |  bounty and you haven't had enough security testing
664.27s - 666.27s |  yet or you just didn't
666.27s - 668.27s |  know that you had a whole bunch of assets out there
668.27s - 670.27s |  or you just mis
670.27s - 672.27s |  kind of judged
672.27s - 674.27s |  how bad you were going to get crushed
674.27s - 676.27s |  by the bug bounty hunters.
676.27s - 678.27s |  And this happens so often.
678.27s - 680.27s |  So you open a public program
680.27s - 682.27s |  and you get hundreds
682.27s - 684.27s |  of submissions
684.27s - 686.27s |  like hundreds, like 800 to
686.27s - 688.27s |  maybe even 1,200. I've seen programs
688.27s - 690.27s |  I've saw a program once get
690.27s - 692.27s |  8,000 submissions in a week
692.27s - 694.27s |  from the bug bounty hunter community.
694.27s - 696.27s |  Now obviously your triagers at the
696.27s - 698.27s |  platform are there to lessen that
698.27s - 700.27s |  load but it's still a lot of load
700.27s - 702.27s |  on your team and
702.27s - 704.27s |  you've made a contract and
704.27s - 706.27s |  you've opened a program and all these bugs come
706.27s - 708.27s |  rushing in at once immediately
708.27s - 710.27s |  your scope or
710.27s - 712.27s |  immediately your budget, it gets exhausted
712.27s - 714.27s |  like it's gone within a day
714.27s - 716.27s |  like a lot of people grossly underestimate
716.27s - 718.27s |  how much it's going to take to run
718.27s - 720.27s |  a public bug bounty. A lot of people
720.27s - 722.27s |  go in thinking I'm going to spend
722.27s - 724.27s |  the first year maybe 80k
724.27s - 726.27s |  on bug bounty payouts.
726.27s - 728.27s |  The reality is if you have a large scale program
728.27s - 730.27s |  you're going to spend somewhere between
730.27s - 732.27s |  300 to 500k.
732.27s - 734.27s |  If you keep it open and you don't
734.27s - 736.27s |  pause it and you don't do some of the shitty things I'm about
736.27s - 738.27s |  to tell you that some people do.
738.27s - 740.27s |  So what happens
740.27s - 742.27s |  in this situation with the worst customers
742.27s - 744.27s |  is they get
744.27s - 746.27s |  really scared about the bug bounty and
746.27s - 748.27s |  their budget getting exhausted
748.27s - 750.27s |  and they get really conservative with payouts.
750.27s - 752.27s |  And so what they do
752.27s - 754.27s |  is they take the contextual
754.27s - 756.27s |  ratings of CVSS
756.27s - 758.27s |  which most of the platforms use
758.27s - 760.27s |  to rate the impact of a bug
760.27s - 762.27s |  and they start downgrading little edge cases
762.27s - 764.27s |  out of them. I'll highlight the specific ones
764.27s - 766.27s |  that they downgrade.
766.27s - 768.27s |  But the customers will downgrade these to make the bug
768.27s - 770.27s |  fall to a lower payout level so that
770.27s - 772.27s |  they can have more remaining budget
772.27s - 774.27s |  on the other side.
774.27s - 776.27s |  The other way that they try to protect against
776.27s - 778.27s |  budget exhaustion when they get crushed by a
778.27s - 780.27s |  bug bounty program
780.27s - 782.27s |  is they try to group bugs together.
782.27s - 784.27s |  So instead of individually paying out every
784.27s - 786.27s |  XSS that they find, they'll be
786.27s - 788.27s |  like, oh, we're just going to
788.27s - 790.27s |  pay out the first few and we're going to say
790.27s - 792.27s |  that holistically XSS is a problem
792.27s - 794.27s |  and we're not going to pay out XSS anymore.
794.27s - 796.27s |  And they're going to say, oh yeah,
796.27s - 798.27s |  it's because of a framework
798.27s - 800.27s |  level problem. And once we fix the framework
800.27s - 802.27s |  level problem, all XSS will be fixed,
802.27s - 804.27s |  which is a bunch of bullshit.
804.27s - 806.27s |  But they do this very often.
806.27s - 808.27s |  Sometimes even at the
808.27s - 810.27s |  suggestion of the platforms, which is kind
810.27s - 816.66s |  of nefarious.
816.66s - 818.66s |  So in triage, clients get
818.66s - 820.66s |  in the loop with the triage after
820.66s - 822.66s |  the platform does.
822.66s - 824.66s |  So the platform triage usually helps
824.66s - 826.66s |  rewrite the bug,
826.66s - 828.66s |  decide if the bug
828.66s - 830.66s |  is valid, and then
830.66s - 835.54s |  passes it on to the customer.
835.54s - 837.54s |  When customers get in the
837.54s - 839.54s |  loop,
839.54s - 841.54s |  stuff can get real crazy.
841.54s - 843.54s |  So if a customer is not
843.54s - 845.54s |  technical enough,
845.54s - 847.54s |  just on accident, they can
847.54s - 849.54s |  mess up the scoring of the impact of a vulnerability.
849.54s - 851.54s |  And most often,
851.54s - 853.54s |  either the nefarious downgrade of a vulnerability
853.54s - 855.54s |  or the accidental downgrade
855.54s - 857.54s |  of a vulnerability, sorry about the typo,
857.54s - 859.54s |  is in four sections in CBSS4.
859.54s - 861.54s |  It's the access requirements,
861.54s - 863.54s |  attack requirements, user interaction
863.54s - 865.54s |  requirements, and privilege requirements
865.54s - 867.54s |  in CBSS4.
867.54s - 869.54s |  These are all dials that you turn on CBSS4.
869.54s - 871.54s |  And whether you
871.54s - 873.54s |  say that you need
873.54s - 875.54s |  to have network access or you need to have
875.54s - 877.54s |  authenticated access or something like that,
877.54s - 879.54s |  the bug severity goes up and down.
879.54s - 881.54s |  Now a lot of customers
881.54s - 883.54s |  make the first mistake just because
883.54s - 885.54s |  they don't understand CBSS. They say,
885.54s - 887.54s |  oh, the privilege requirements
887.54s - 889.54s |  are that you have
889.54s - 891.54s |  to be authenticated.
891.54s - 893.54s |  And they
893.54s - 895.54s |  consider that not
895.54s - 897.54s |  self-signup. They consider that employee
897.54s - 899.54s |  level access. And so they think,
899.54s - 901.54s |  oh, well,
901.54s - 903.54s |  they had to self-register
903.54s - 905.54s |  for the site to get access
905.54s - 907.54s |  to this function and then hack this site.
907.54s - 909.54s |  But when that's free registration, really
909.54s - 911.54s |  that's open access. Anybody can register
911.54s - 913.54s |  for a website, get an account,
913.54s - 915.54s |  find a bug. And so they'll
915.54s - 917.54s |  do this on accident. They'll underscore
917.54s - 919.54s |  this. So there's several ways that they can do this by accident.
919.54s - 921.54s |  There's also
921.54s - 923.54s |  this world, and I've seen it
923.54s - 925.54s |  many times, where
925.54s - 927.54s |  the second level of triage at the
927.54s - 929.54s |  client is done by a security
929.54s - 931.54s |  engineer at the client site.
931.54s - 933.54s |  And, you know,
933.54s - 935.54s |  they're probably a really good security engineer,
935.54s - 937.54s |  but they're jaded because they have 800
937.54s - 939.54s |  submissions in the queue.
939.54s - 941.54s |  And so they start rating
941.54s - 943.54s |  bugs with a custom rubric that
943.54s - 945.54s |  they make themselves.
945.54s - 947.54s |  And it is
947.54s - 949.54s |  not very
949.54s - 951.54s |  data-driven. And so
951.54s - 953.54s |  what they'll do is they'll start
953.54s - 955.54s |  triaging based on
955.54s - 957.54s |  how long it takes to
957.54s - 959.54s |  exploit the bug, which shouldn't
959.54s - 961.54s |  matter, how
961.54s - 963.54s |  technical the bug is.
963.54s - 965.54s |  Sometimes you find really stupid stuff
965.54s - 967.54s |  like a website that
967.54s - 969.54s |  has a default password
969.54s - 971.54s |  and that lets a hacker into your systems.
971.54s - 973.54s |  But the impact
973.54s - 975.54s |  of that is what you should be, which should be
975.54s - 977.54s |  generating value for that, right?
977.54s - 979.54s |  And so if that DevSecOps panel
979.54s - 981.54s |  that had a default password on it
981.54s - 983.54s |  got them access to all your customer
983.54s - 985.54s |  data or all of your DevSecOps data
985.54s - 987.54s |  or all your logs or something, that
987.54s - 989.54s |  should be the end-all be-all way
989.54s - 991.54s |  that you triage a bug like that.
991.54s - 993.54s |  And then
993.54s - 995.54s |  sometimes they lower triage
995.54s - 997.54s |  or they lower impact
997.54s - 999.54s |  by saying that there was mitigating
999.54s - 1001.54s |  controls behind
1001.54s - 1003.54s |  the controls that
1003.54s - 1005.54s |  were passed, which there's no way for a
1005.54s - 1007.54s |  bug hunter to verify that.
1007.54s - 1009.54s |  And so a lot of times you get in a
1009.54s - 1011.54s |  lot of arbitration and
1011.54s - 1013.54s |  kind of bad
1013.54s - 1015.54s |  situations between people are like,
1015.54s - 1017.54s |  oh, we would have noticed that in our logs, or
1017.54s - 1019.54s |  we had a mitigating
1019.54s - 1021.54s |  manual check of this process.
1021.54s - 1023.54s |  And so the bug hunter doesn't know that.
1023.54s - 1025.54s |  They accessed your system,
1025.54s - 1027.54s |  accessed your logs, they had it, or
1027.54s - 1029.54s |  they exploited this vulnerability.
1029.54s - 1031.54s |  Pretty sure that it counts
1031.54s - 1033.54s |  for a bug bounty.
1033.54s - 1035.54s |  The other one that they can do, that really
1035.54s - 1037.54s |  shitty clients can do, is they can say there was an
1037.54s - 1039.54s |  existing issue in their backlog,
1039.54s - 1041.54s |  which I have seen outright
1041.54s - 1043.54s |  customers lie about.
1043.54s - 1045.54s |  They say, oh yeah, we knew about this issue.
1045.54s - 1047.54s |  It came from our pen testers two years
1047.54s - 1049.54s |  ago. We just haven't had time to fix it.
1049.54s - 1051.54s |  Well, if it was two fucking years ago,
1051.54s - 1053.54s |  that's bad.
1053.54s - 1055.54s |  And anyway, show me proof.
1055.54s - 1057.54s |  This is a very common one.
1057.54s - 1059.54s |  And again, I'm not saying
1059.54s - 1061.54s |  this is you guys. I'm saying this is the worst of all
1061.54s - 1063.54s |  customers I've ever seen. But there
1063.54s - 1068.38s |  are some shitty customers out there.
1068.38s - 1070.38s |  Okay, so another thing shitty
1070.38s - 1072.38s |  customers do is they make
1072.38s - 1074.38s |  payout ranges. And so
1074.38s - 1076.38s |  they'll say, hey, a critical bug is
1076.38s - 1078.38s |  worth like 20k
1078.38s - 1080.38s |  to us, or 50k, or
1080.38s - 1082.38s |  100k if they're going ham.
1082.38s - 1084.38s |  And
1084.38s - 1089.97s |  sorry, give me one second.
1089.97s - 1091.97s |  And so they'll make
1091.97s - 1093.97s |  these ranges for payment. They'll say
1093.97s - 1095.97s |  anything between 10 and
1095.97s - 1097.97s |  40k, depending on what we decide,
1097.97s - 1099.97s |  right? Less than 15%
1099.97s - 1101.97s |  of clients who use
1101.97s - 1103.97s |  ranges, sorry about the typos. I made the slides
1103.97s - 1105.97s |  last night. Less than 15%
1105.97s - 1107.97s |  of clients who use ranges pay anything above
1107.97s - 1109.97s |  the minimum of that range ever.
1109.97s - 1111.97s |  You have some gold star customers who do
1111.97s - 1113.97s |  that, like Google and Meta, and they
1113.97s - 1115.97s |  really have a vulnerability rating board
1115.97s - 1117.97s |  internally in the company.
1117.97s - 1119.97s |  But most of the time, they just pay
1119.97s - 1121.97s |  the bare minimum. And so setting
1121.97s - 1123.97s |  as a career hacker who's going to do bug bounty,
1123.97s - 1125.97s |  you can
1125.97s - 1127.97s |  really get kind of skunked on
1127.97s - 1129.97s |  what you think is going
1129.97s - 1131.97s |  to be a great program, and then you get wrecked.
1131.97s - 1133.97s |  This is very common right now in the web
1133.97s - 1135.97s |  3 space. So there are
1135.97s - 1137.97s |  a lot of web 3 bounties that want
1137.97s - 1139.97s |  to make a splash in the press and say,
1139.97s - 1141.97s |  if you can hack our crypto
1141.97s - 1143.97s |  system, we will give you
1143.97s - 1145.97s |  a million dollars. There's several of these
1145.97s - 1147.97s |  going on right now. And then
1147.97s - 1149.97s |  I've had several bug hunters break
1149.97s - 1151.97s |  into their sites, get all their customer
1151.97s - 1153.97s |  data, have the ability to backdoor their
1153.97s - 1155.97s |  algorithms, and basically mess
1155.97s - 1157.97s |  up their whole exchange. But
1157.97s - 1159.97s |  because the bug wasn't a
1159.97s - 1161.97s |  crypto-related bug, like a smart
1161.97s - 1163.97s |  contract bug, because it was a web
1163.97s - 1165.97s |  bug, they got paid
1165.97s - 1167.97s |  20K instead of a million.
1167.97s - 1169.97s |  And so there is
1169.97s - 1171.97s |  some really shady stuff going on like
1171.97s - 1173.97s |  that right now in the industry.
1173.97s - 1175.97s |  And those million dollar
1175.97s - 1177.97s |  payouts, they're mostly for marketing.
1177.97s - 1179.97s |  They're mostly a marketing
1179.97s - 1181.97s |  ploy to make the company look like they're more
1181.97s - 1183.97s |  secure. The only people I've seen
1183.97s - 1185.97s |  that pay million dollar bugs
1185.97s - 1187.97s |  at least recently, or close
1187.97s - 1189.97s |  to a million dollars, has been
1189.97s - 1191.97s |  T-Mobile has a really forward-facing
1191.97s - 1193.97s |  bug bounty program, and they've paid out really
1193.97s - 1195.97s |  high rewards for bug hunters
1195.97s - 1197.97s |  who did very specific things.
1197.97s - 1201.89s |  All right. So some of these
1201.89s - 1203.89s |  same things happen
1203.89s - 1205.89s |  at live events. So if you've never
1205.89s - 1207.89s |  been to a live event, how many of you have been to a live event
1207.89s - 1209.89s |  before, a hacker live event?
1209.89s - 1211.89s |  Cool. So some of you, yeah.
1211.89s - 1213.89s |  So if you haven't been,
1213.89s - 1215.89s |  live events at BugCrowd, HackerOne, Integrity,
1215.89s - 1217.89s |  Let'sHack, or
1217.89s - 1219.89s |  YesWeHack,
1219.89s - 1221.89s |  they're basically giant hacker
1221.89s - 1223.89s |  parties where
1223.89s - 1225.89s |  you fly out a whole bunch of really
1225.89s - 1227.89s |  good, really high-scoring, really well-known
1227.89s - 1229.89s |  bug hunters to a location
1229.89s - 1231.89s |  and they
1231.89s - 1233.89s |  basically focus on a single
1233.89s - 1235.89s |  company, and they
1235.89s - 1237.89s |  increase the rewards substantially
1237.89s - 1239.89s |  for this period, and
1239.89s - 1241.89s |  they open up NewScope.
1241.89s - 1243.89s |  Now several live events,
1243.89s - 1245.89s |  and you can talk to a bunch of the bug bounty
1245.89s - 1247.89s |  hunters, but
1247.89s - 1249.89s |  several live events have been the most
1249.89s - 1251.89s |  disastrous thing ever, where
1251.89s - 1253.89s |  the client puts in NewScope into the
1253.89s - 1255.89s |  program for the live event
1255.89s - 1257.89s |  and says, hey,
1257.89s - 1259.89s |  we're going to pay $20k
1259.89s - 1261.89s |  per critical vulnerability,
1261.89s - 1263.89s |  and then in 40 minutes
1263.89s - 1265.89s |  you get
1265.89s - 1267.89s |  40 critical vulnerabilities and completely
1267.89s - 1269.89s |  exhaust the whole live event budget.
1269.89s - 1271.89s |  And this is before, this hacking,
1271.89s - 1273.89s |  you have two weeks before you even go on-site
1273.89s - 1275.89s |  to do some of this hacking, so the budget
1275.89s - 1277.89s |  gets exhausted, like we saw
1277.89s - 1279.89s |  in previous slides, before the live event really
1279.89s - 1281.89s |  even happens. And then you
1281.89s - 1283.89s |  fly all these hackers out there, and then the customers
1283.89s - 1285.89s |  are nickel-and-diming the hackers, downgrading
1285.89s - 1287.89s |  their reports, out-of-scoping
1287.89s - 1289.89s |  their reports, combining them with other reports,
1289.89s - 1291.89s |  really unfairly, because they
1291.89s - 1293.89s |  basically have no budget left for the
1293.89s - 1295.89s |  project.
1295.89s - 1297.89s |  And it's kind of the fault,
1297.89s - 1299.89s |  in this case, of both
1299.89s - 1301.89s |  the expectations of the clients,
1301.89s - 1303.89s |  the non-understanding of the
1303.89s - 1305.89s |  scope, and non-understanding
1305.89s - 1307.89s |  about how effective live events are at
1307.89s - 1315.97s |  finding vulnerabilities.
1315.97s - 1317.97s |  So, there was a panel
1317.97s - 1319.97s |  earlier this week at the Bug Bounty Village,
1319.97s - 1321.97s |  there's a Bug Bounty Village here if you didn't know,
1321.97s - 1323.97s |  and the panel was
1323.97s - 1325.97s |  some of my closest friends
1325.97s - 1327.97s |  on this panel, and so I don't want
1327.97s - 1329.97s |  to, like, crap on them, really.
1329.97s - 1331.97s |  But they said something that really pissed me off.
1331.97s - 1333.97s |  So, basically
1333.97s - 1335.97s |  I'm paraphrasing here, but they said, like,
1335.97s - 1337.97s |  going public or tweeting or making
1337.97s - 1339.97s |  the platform's job harder
1339.97s - 1341.97s |  in any way will make them not
1341.97s - 1343.97s |  want to work with you and not get invited to
1343.97s - 1345.97s |  programs, which I think is a bunch
1345.97s - 1347.97s |  of bullshit, because we are the product.
1347.97s - 1349.97s |  All you have, if you are a
1349.97s - 1351.97s |  platform, is a goddamn webpage.
1351.97s - 1353.97s |  You don't have any way of finding vulnerabilities.
1353.97s - 1355.97s |  The hackers are the product.
1355.97s - 1357.97s |  And so if you think that
1357.97s - 1359.97s |  sharing data between platforms
1359.97s - 1361.97s |  and stuff is right, and, like, kind of blackballing
1361.97s - 1363.97s |  researchers,
1363.97s - 1365.97s |  you have grossly
1365.97s - 1367.97s |  not oriented
1367.97s - 1369.97s |  kind of your whole company the right way.
1369.97s - 1371.97s |  Your hackers are the only thing you really have.
1371.97s - 1373.97s |  Otherwise, you're a glorified
1373.97s - 1375.97s |  ticketing system.
1375.97s - 1377.97s |  Now, this happens
1377.97s - 1379.97s |  because platform triage
1379.97s - 1381.97s |  gets burnt out. They handle
1381.97s - 1383.97s |  hundreds of thousands of bugs a week.
1383.97s - 1385.97s |  I mean, I've managed these
1385.97s - 1387.97s |  teams, but it is a
1387.97s - 1389.97s |  core
1389.97s - 1391.97s |  need for that
1391.97s - 1393.97s |  team to be the stalwart defenders
1393.97s - 1395.97s |  of the hackers. The triage team at
1395.97s - 1397.97s |  the platform needs to make sure at every
1397.97s - 1399.97s |  opportunity the hackers are treated like rock stars
1399.97s - 1401.97s |  and they are stood up for.
1401.97s - 1403.97s |  Because if any time that any of these
1403.97s - 1405.97s |  hackers decide to not work on the program, they have
1405.97s - 1407.97s |  nothing to sell. And so
1407.97s - 1409.97s |  this paraphrase here of
1409.97s - 1411.97s |  those leaders at those companies
1411.97s - 1413.97s |  really made me mad, even though I knew them.
1413.97s - 1415.97s |  And so this goes back to the idea
1415.97s - 1417.97s |  that the hackers are the product and they're
1417.97s - 1419.97s |  being kind of mistreated in a lot of ways.
1419.97s - 1421.97s |  On some programs, not all.
1421.97s - 1427.30s |  The worst scenario of all
1427.30s - 1429.30s |  time that's happened at least once
1429.30s - 1431.30s |  documented was that internal
1431.30s - 1433.30s |  triagers at a platform
1433.30s - 1435.30s |  stole research from
1435.30s - 1437.30s |  a bug hunter and then resubmitted
1437.30s - 1439.30s |  it to other bug bounty programs in their own
1439.30s - 1441.30s |  name. So this has happened
1441.30s - 1443.30s |  at least once that we know of
1443.30s - 1445.30s |  and it is absolutely
1445.30s - 1447.30s |  atrocious. And so it caused
1447.30s - 1449.30s |  a kickoff of
1449.30s - 1451.30s |  basically a whole bunch of process
1451.30s - 1453.30s |  at this place where this happened.
1453.30s - 1455.30s |  And so
1455.30s - 1457.30s |  I would like to say that I
1457.30s - 1459.30s |  thought that that was the only instance that ever
1459.30s - 1461.30s |  happened was the one that got caught and publicized.
1461.30s - 1463.30s |  But I don't think it is.
1463.30s - 1465.30s |  I think that triage
1465.30s - 1467.30s |  jobs
1467.30s - 1469.30s |  they
1469.30s - 1471.30s |  basically those people get
1471.30s - 1473.30s |  to see all the research and it should be
1473.30s - 1475.30s |  held to a very high ethical standard for
1475.30s - 1477.30s |  keeping that vulnerability data
1477.30s - 1482.82s |  private.
1482.82s - 1484.82s |  Another thing that happens
1484.82s - 1486.82s |  in the bug bounty industry that's kind of crappy
1486.82s - 1488.82s |  is celebrity. And I'm guilty of this.
1488.82s - 1490.82s |  I am an infosex celebrity. I have
1490.82s - 1492.82s |  160,000 Twitter followers.
1492.82s - 1494.82s |  I connect with everybody on LinkedIn.
1494.82s - 1496.82s |  But I earned it. I have been
1496.82s - 1498.82s |  doing work and research for 20
1498.82s - 1500.82s |  years. And a lot of the big
1500.82s - 1502.82s |  bug hunters earn it. But what it
1502.82s - 1504.82s |  does is it creates a power disparity
1504.82s - 1506.82s |  with newer bug hunters who don't have
1506.82s - 1508.82s |  that celebrity. So let me give you some
1508.82s - 1510.82s |  examples of the advantages celebrity gives you
1510.82s - 1512.82s |  in bug bounty. Big names
1512.82s - 1514.82s |  get big service. So for instance
1514.82s - 1516.82s |  if I submit a bug, I
1516.82s - 1518.82s |  know Casey Ellis. He is the CTO
1518.82s - 1520.82s |  of Bug Crowd. I worked with him for five years
1520.82s - 1522.82s |  at Bug Crowd. If I think I'm being treated unfairly
1522.82s - 1524.82s |  if I really want to go out of my way
1524.82s - 1526.82s |  I can DM him and be like,
1526.82s - 1528.82s |  yo, why are you guys being stupid?
1528.82s - 1530.82s |  This is a good bug. I'm submitting it.
1530.82s - 1532.82s |  Talk to the triage team.
1532.82s - 1534.82s |  And so a lot of big name bug
1534.82s - 1536.82s |  hunters have this power. They know
1536.82s - 1538.82s |  internal employees. Some of them
1538.82s - 1540.82s |  at the platforms get their own CS reps.
1540.82s - 1542.82s |  So they get an actual CS rep
1542.82s - 1544.82s |  assigned to them to help them
1544.82s - 1546.82s |  get their bugs pushed through faster.
1546.82s - 1548.82s |  So they'll get out of band or instant
1548.82s - 1550.82s |  support. And
1550.82s - 1552.82s |  basically what it means is that these people
1552.82s - 1554.82s |  have like an unfair power dynamic
1554.82s - 1556.82s |  to somebody who's just coming up. In which
1556.82s - 1558.82s |  I've learned some but I still think there's a system
1558.82s - 1560.82s |  in where no name people
1560.82s - 1562.82s |  who don't have any rep or any research
1562.82s - 1564.82s |  to their name but find really really good bugs
1564.82s - 1566.82s |  which is the spirit of Bug Bounty. Anybody
1566.82s - 1568.82s |  can find a bug. I mean there was
1568.82s - 1570.82s |  there was this guy one time who had never done
1570.82s - 1572.82s |  Bug Bounty. He was a developer. I was working
1572.82s - 1574.82s |  on the platform.
1574.82s - 1576.82s |  And he's now a very famous
1576.82s - 1578.82s |  Bug Bounty hunter but he started off as a developer. And he found
1578.82s - 1580.82s |  the most amazing SSO
1580.82s - 1582.82s |  misconfiguration I've ever seen.
1582.82s - 1584.82s |  Completely wrecked this program
1584.82s - 1586.82s |  by basically being able to log in
1586.82s - 1588.82s |  as anyone in their whole SSO
1588.82s - 1590.82s |  kind of staple. Anyone in the company.
1590.82s - 1592.82s |  And that's the kind of
1592.82s - 1594.82s |  bugs you want. You want people coming out of the blue
1594.82s - 1596.82s |  and just finding amazing stuff.
1596.82s - 1598.82s |  But sometimes it doesn't happen like that
1598.82s - 1600.82s |  because nobody knows who they are.
1600.82s - 1602.82s |  So there should be a better system
1602.82s - 1607.18s |  for this. Other things that
1607.18s - 1609.18s |  really suck about the process is that after
1609.18s - 1611.18s |  after you have a couple
1611.18s - 1613.18s |  rounds of conversation with the platform
1613.18s - 1615.18s |  really it's done. The door is closed.
1615.18s - 1617.18s |  Like if you have two comments on
1617.18s - 1619.18s |  a submission and you disagree with
1619.18s - 1621.18s |  the rating or anything happens
1621.18s - 1623.18s |  there's no more recourse for you. They just shut you down.
1623.18s - 1625.18s |  Like that's it. You can comment
1625.18s - 1627.18s |  50 more times
1627.18s - 1629.18s |  and they can see it in the back end. They get alerts
1629.18s - 1631.18s |  but they just won't reply to you.
1631.18s - 1633.18s |  So at some point there's really no
1633.18s - 1635.18s |  recourse for bug hunters when they think that they've
1635.18s - 1639.20s |  been treated unfairly.
1639.20s - 1641.20s |  Five minutes right?
1641.20s - 1643.20s |  So I'm going to roll through this.
1643.20s - 1645.20s |  The worst form of Bug Bounty runner
1645.20s - 1647.20s |  is somebody who does a public VDP
1647.20s - 1649.20s |  to get free bugs and then runs a private
1649.20s - 1651.20s |  paid program
1651.20s - 1653.20s |  to basically funnel people into.
1653.20s - 1655.20s |  It's basically farming free work from bug hunters.
1655.20s - 1657.20s |  So that kind of sucks for
1657.20s - 1659.20s |  the community in general.
1659.20s - 1661.20s |  Let's see here.
1661.20s - 1663.20s |  Some other shitty things
1663.20s - 1665.20s |  are the platforms treat
1665.20s - 1667.20s |  hackers as the product in more than just
1667.20s - 1669.20s |  the hacking ways. They also
1669.20s - 1671.20s |  have in several
1671.20s - 1673.20s |  instances used them
1673.20s - 1675.20s |  as marketing tools as
1675.20s - 1677.20s |  many celebrities and they very
1677.20s - 1679.20s |  rarely pay them for that work or
1679.20s - 1681.20s |  anything like that.
1681.20s - 1683.20s |  Okay some tips and tricks
1683.20s - 1685.20s |  if you are a bounty hunter. Some of you did raise your
1685.20s - 1687.20s |  hand. I think like maybe 20 of you.
1687.20s - 1689.20s |  So if you want to engage in debating
1689.20s - 1691.20s |  criticality, one of my tips is
1691.20s - 1693.20s |  pre-provide your justification
1693.20s - 1695.20s |  in CVS 4.0
1695.20s - 1697.20s |  in the original submission. So don't let
1697.20s - 1699.20s |  them make the mistake.
1699.20s - 1701.20s |  Pre-write it out. Here is why this is
1701.20s - 1703.20s |  access level this. Here is why privileges
1703.20s - 1705.20s |  required are this.
1705.20s - 1707.20s |  If you want your bug to get triaged
1707.20s - 1709.20s |  with high impact
1709.20s - 1711.20s |  also do some research on what
1711.20s - 1713.20s |  compliance they may be subject to and
1713.20s - 1715.20s |  reference that the bug might cause them to be
1715.20s - 1717.20s |  out of compliance with X, Y, and
1717.20s - 1719.20s |  Z. And then write out your impact
1719.20s - 1721.20s |  scenario in your bug really
1721.20s - 1723.20s |  really well. Scare the shit out of them if
1723.20s - 1725.20s |  you can. Use chat GPT if you have to
1725.20s - 1727.20s |  do it. But do this
1727.20s - 1729.20s |  and you will get paid more consistently.
1729.20s - 1732.86s |  If you have a bug
1732.86s - 1734.86s |  that is in a multi-use platform
1734.86s - 1736.86s |  don't submit
1736.86s - 1738.86s |  it to everything all at once.
1738.86s - 1740.86s |  Submit it to one program
1740.86s - 1742.86s |  get it paid, wait
1742.86s - 1744.86s |  for a little bit and then submit it to another program
1744.86s - 1746.86s |  and then wait for a little bit and then submit it
1746.86s - 1748.86s |  to another program to make sure that
1748.86s - 1750.86s |  they're not doing anything nefarious like sharing
1750.86s - 1752.86s |  your attack strings among their customers
1752.86s - 1754.86s |  and just kind of visit
1754.86s - 1756.86s |  and just kind of make sure that
1756.86s - 1758.86s |  you can stagger your bug.
1758.86s - 1760.86s |  A lot of times what customers will do is they'll say
1760.86s - 1762.86s |  oh we're going to
1762.86s - 1764.86s |  make one fix and it'll fix all
1764.86s - 1766.86s |  these things if you shoot
1766.86s - 1768.86s |  a vulnerability class at them.
1768.86s - 1770.86s |  So if you stagger your reports
1770.86s - 1772.86s |  and then they give you that answer
1772.86s - 1774.86s |  or if you stagger your reports
1774.86s - 1776.86s |  they'll fix the one bug and then you can wait a little while
1776.86s - 1778.86s |  and send another one and then eventually
1778.86s - 1780.86s |  if they say hey we're going to do one fix for all of this
1780.86s - 1782.86s |  then you can test after
1782.86s - 1784.86s |  that too. And then revisit all your old
1784.86s - 1786.86s |  bugs for bypasses as well.
1786.86s - 1793.66s |  Let's see here.
1793.66s - 1795.66s |  Some tips if you
1795.66s - 1797.66s |  want to kind of go out of scope every once in a while
1797.66s - 1799.66s |  or you find something that you think is out of scope
1799.66s - 1801.66s |  I have used a submission
1801.66s - 1803.66s |  just to ask the client a question
1803.66s - 1805.66s |  which puts the bug bounty platform in a little
1805.66s - 1807.66s |  bit of a weird way. I don't care it doesn't hurt
1807.66s - 1809.66s |  my submission score a lot but like here's
1809.66s - 1811.66s |  an example. So for State Farm
1811.66s - 1813.66s |  they didn't have creds on
1813.66s - 1815.66s |  GitHub in their scope
1815.66s - 1817.66s |  and so I just burned a submission
1817.66s - 1819.66s |  and I said hey I think I found like 20 creds
1819.66s - 1821.66s |  for people in
1821.66s - 1823.66s |  GitHub.
1823.66s - 1825.66s |  Do you want me to submit this?
1825.66s - 1827.66s |  And so then the triage team
1827.66s - 1829.66s |  has to forward it to the customer and they have to answer
1829.66s - 1831.66s |  it honestly. Now it has to be worded
1831.66s - 1833.66s |  carefully because otherwise that could seem
1833.66s - 1835.66s |  like extortion and I'm going to give
1835.66s - 1837.66s |  them the bugs anyway. That's what I'm going to do.
1837.66s - 1839.66s |  If they say no I'm going to be like cool
1839.66s - 1841.66s |  sucks you don't take it here's 20
1841.66s - 1843.66s |  things. So I'm never
1843.66s - 1845.66s |  on the edge of extortion but at
1845.66s - 1847.66s |  least they see it and have the opportunity to
1847.66s - 1849.66s |  say okay yeah this is cool
1849.66s - 1851.66s |  we'll pay this guy. That's a tip there.
1853.66s - 1855.66s |  When push comes
1855.66s - 1857.66s |  to shove at the end when
1857.66s - 1859.66s |  you are locked out you can't make any more
1859.66s - 1861.66s |  comments you've done all kinds of escalations on the
1861.66s - 1863.66s |  platforms the customer is not agreeing
1863.66s - 1865.66s |  with you but you still have a good bug
1865.66s - 1867.66s |  well then you can ask for disclosure
1867.66s - 1869.66s |  or you can just YOLO disclose the bug
1869.66s - 1871.66s |  via a blog post or Twitter.
1871.66s - 1873.66s |  And so
1873.66s - 1875.66s |  this happens often when companies
1875.66s - 1877.66s |  think it's out of scope or NA
1877.66s - 1879.66s |  a bug not applicable
1879.66s - 1881.66s |  a bug
1881.66s - 1883.66s |  and so if they think it's out of scope
1883.66s - 1885.66s |  or not applicable or no security impact
1885.66s - 1887.66s |  these are all the three ways they can rate it
1887.66s - 1889.66s |  then by contract
1889.66s - 1891.66s |  I actually don't have the right to blog about
1891.66s - 1893.66s |  it on the platforms. As soon as I press
1893.66s - 1895.66s |  the submit button for a vulnerability on a platform
1895.66s - 1897.66s |  it belongs to the platform that data
1897.66s - 1899.66s |  and that is the contract I have to sign
1899.66s - 1901.66s |  but I think that we should have the right
1901.66s - 1903.66s |  as bug bounty hunters to disclose if someone
1903.66s - 1905.66s |  thinks it's out of scope or they don't care about it
1905.66s - 1907.66s |  we should be able to write about it we spent time
1907.66s - 1909.66s |  researching it and trying to help them
1909.66s - 1911.66s |  and so I hope that changes over the years.
1911.66s - 1913.66s |  Closing thoughts
1913.66s - 1915.66s |  platforms are nothing
1915.66s - 1917.66s |  without their product the hackers
1917.66s - 1919.66s |  heavy education needs to happen
1919.66s - 1921.66s |  to clients on what kind of the core
1921.66s - 1923.66s |  of the bug bounty contract is
1923.66s - 1925.66s |  better communication needs to improve
1925.66s - 1927.66s |  on all platform features
1927.66s - 1929.66s |  and every function of a bug bounty in the
1929.66s - 1931.66s |  triage cycle and in the
1931.66s - 1933.66s |  creation cycle needs to be documented
1933.66s - 1935.66s |  just like source code is just like you can
1935.66s - 1937.66s |  review a revision history to keep everybody
1937.66s - 1939.66s |  honest
1939.66s - 1941.66s |  that's it


url: https://www.youtube.com/watch?v=3y_6oSZwLr0
title: DEF CON 32 - Relative Path File Injection The Next Evolution in RPO - Ian Hickey
type: video
author: DEFCONConference
ingestion_date: 2024-11-09
transcription_model: large-v2
chunking_status: completed
content:
{
  "webpage_url": "https://www.youtube.com/watch?v=3y_6oSZwLr0",
  "title": "DEF CON 32 - Relative Path File Injection  The Next Evolution in RPO - Ian Hickey",
  "description": "Most web security professionals are familiar with Relative Path Overwrite (RPO) attacks that allow injecting malicious CSS via a quirk in how browsers handle paths. But what if you could use a similar technique to get victims to download malicious files by clicking an innocuous looking download link on a trusted site? In this presentation, we'll unveil a new attack vector dubbed Relative Path File Injection (RPFI) that abuses path handling to turn benign websites into malware delivery platforms. Attendees will learn the anatomy of an RPFI attack, see demos of it in action, and learn how to detect this overlooked vulnerability class in the wild. We'll also release an open source GitHub repo with proof of concepts for users to try for themselves. RPFI represents a new breed of polyglot-based attack that exploits gaps between web specifications and real-world implementations.",
  "channel_url": "https://www.youtube.com/channel/UC6Om9kAkl32dWlDSNlDS9Iw",
  "duration": 1750,
  "channel": "DEFCONConference",
  "uploader": "DEFCONConference",
  "upload_date": "20241016"
}

0.00s - 5.00s | This text was transcribed using whisper model: large-v2

 So it is my pleasure to welcome Ian Hickey to the stage talking about relative path file
8.54s - 15.46s |  injection, the next evolution in relative path. Right? Okay. Let's give it up.
25.24s - 30.44s |  Thank you for having me today at the AppSec Village, DEF CON 32. Thank you for having
30.44s - 35.44s |  AppSec Village here as well. I'm a software developer. I don't work in security. I work in
39.00s - 44.74s |  ed tech. But every once in a while I like to spend some brain power looking at some
44.74s - 52.03s |  security things. And the last time I was doing this in April or May, I had a couple of
52.03s - 58.87s |  discoveries that I thought were interesting and I'm going to share them with you today. So
58.87s - 63.67s |  many of you know that there's something called relative path overwrite. That was
63.67s - 68.67s |  discovered in 2014 by Gareth Hayes of Port Swigger Research. And relative path overwrite
71.55s - 76.55s |  is a way to if you can inject CSS onto an HTML page and if a website is vulnerable to path
83.13s - 88.13s |  confusion, then you can actually get the default link tag in HTML to interpret the current
88.89s - 93.89s |  HTML page as a CSS style sheet and under some circumstances it will execute the CSS in older
105.67s - 111.01s |  Internet Explorer, used to be able to use things like an expression tag and execute
111.01s - 116.01s |  JavaScript from CSS. So it was a pretty ‑‑ it was a pretty interesting way to do that.
118.25s - 123.25s |  To get JavaScript execution. So what I've been looking at is Port Swigger Research has these
128.21s - 132.51s |  impossible labs and one of these impossible labs I've been looking at for a couple of
132.51s - 136.71s |  years and I'll go back and I'll play with different things and just see if I can make any
136.71s - 141.71s |  progress on it. In this case it was ‑‑ there's context in HTML where you can't execute
142.71s - 147.71s |  JavaScript. The browser won't let you. And some of those places are no scripts or within a
151.55s - 156.55s |  frame set tag. It interprets ‑‑ it interprets any HTML that you can inject as just plain
161.51s - 169.70s |  text. And because it's plain text, you can't do anything with it. So the first insight
169.70s - 174.70s |  that I had into this was ‑‑ see here, here's an example of a B tag but it's just text. You
177.20s - 183.54s |  can't display anything. The first interesting sort of insights I had into this was actually
187.54s - 192.54s |  you can execute JavaScript but you would have to have AngularJS installed. And you'd have
195.84s - 200.84s |  to be using AngularJS and it's been end of life for quite some time. But you can't
201.62s - 206.62s |  ‑‑ there is a way to do it. So if you have a no frames tag inside of a frame set tag and
209.34s - 214.34s |  there happens to be AngularJS on the page, then you can just instantiate AngularJS and
217.50s - 222.50s |  execute some JavaScript. That's cool. In the original relative path overwrite, if you needed
223.50s - 228.50s |  to inject CSS into the page, and again, it didn't matter where you did it. Because in both
236.22s - 241.22s |  cases, those parsers are not taking into consideration the rules that the browser follows.
246.08s - 251.08s |  Angular will happily execute JavaScript anywhere on the page it finds it. And in this case,
252.80s - 257.80s |  when it's interpreting a HTML page as a style sheet, it doesn't care where it finds CSS, it
258.64s - 263.42s |  will just execute it. So that was a starting point. And then the epiphany was sort of, okay,
263.42s - 269.30s |  well, you can use other parsers to execute JavaScript within these contexts where you're not
269.30s - 275.22s |  supposed to be able to do it. So what other parsers are there that will do the same thing?
275.22s - 281.46s |  One thing to note about both of these things is that they're lenient parsers. Meaning it does
281.46s - 288.22s |  not matter that there's HTML surrounding the thing that they're looking for. All that it
288.22s - 293.42s |  matters is that it eventually finds it. So that was my starting point. What else is there? So
293.42s - 298.42s |  I started making a list of all the different types of parsers there are. There's SVG parsers
301.22s - 306.22s |  and XML and PDF and rich text and markdown and after spending more time than I would care to
306.78s - 311.78s |  admit with markdown parsers, I started thinking about PDF a lot. PDF can actually be
316.90s - 321.40s |  handwritten. A lot of people think of it as a binary format and there might be binary mixed in
321.40s - 326.40s |  with it but you can actually just write a PDF file and as plain text. So if you have plain text
327.40s - 333.75s |  injection like we're talking about here in one of these contexts such as no script, no frames,
341.55s - 346.55s |  then you can inject PDF. Well, PDF can execute JavaScript. PDF can submit forms. PDF can
351.75s - 356.75s |  there's tons of research into malicious PDFs. So I started playing with it. I started
357.75s - 362.75s |  using it. And the second big epiphany that I had that just really surprised the shit out of me is
364.89s - 369.89s |  that PDFs are also super lenient because PDFs start with these magic characters and end with
373.39s - 378.13s |  magic characters. It doesn't really care what it finds before and after that. So you can
378.13s - 385.13s |  actually take this HTML document with PDF persistently injected into it and you can save
385.13s - 390.13s |  this as a PDF and it will actually execute. And this is what this is here. This is just an
394.13s - 399.13s |  embedded HTML file renamed as PDF and it displays as a PDF and it executes JavaScript. How
404.30s - 409.30s |  many people how are you going to convince people to download an HTML file as a PDF? Like are
410.18s - 415.18s |  you going to put something on the page that says right click here, save it as whatever? They
420.34s - 425.34s |  just won't do it. So I started thinking about ways to get people to download these PDFs
430.72s - 435.72s |  without me having to explicitly convince them of anything. And when I started thinking about
436.72s - 441.72s |  the original relative path overwrite which interprets the current page as a CSS style sheet,
443.84s - 449.48s |  well, the reason why it does that is because the browsers are stupid. They just whenever you
449.48s - 454.98s |  return back from the server it's going to use and if you happen to be using a server that when
454.98s - 460.90s |  you add a forward slash at the ends returns the current page back, well, if the current page
460.90s - 465.90s |  happens to be have PDF injected into it, then that's what you're going to get. So that's exactly
471.86s - 476.86s |  what I did and I'm going to show you how it works now and before we even get into it, just so
479.96s - 484.96s |  everyone sees because it's easier to simply see what's happening than to explain it. Someone
485.88s - 490.88s |  goes to a website. There is a download link on the website and the download link happens to use a
499.88s - 506.58s |  relative path. Well, if you click this download link, all that's going to happen is it's going to
506.58s - 514.12s |  download whatever the file was that you intended to download and if we open this up, all this
514.82s - 524.71s |  is is, you know, oh, sorry, sorry. Let's see if I can make that bigger for you. Sorry, let me go to,
543.92s - 548.92s |  whoop, whoop, one second. Okay. Good PDF download. So I've downloaded this PDF and I'm
552.04s - 557.04s |  going to open it with Google Chrome. It's just a normal harmless PDF. No big deal. However, all you
562.98s - 567.98s |  have to do is add a forward slash at the ends and you notice it lost its styling. The reason it lost
570.66s - 575.66s |  its styling is because now the browser is confused. If you look in the, if you look at what it's
576.52s - 581.52s |  actually trying to include as the style sheet, what you find now is it's actually getting the
586.76s - 591.76s |  current page that's relative path over it. It's confused so it's returning this back. The effect
594.10s - 599.10s |  of that, once it loses its styling, is that if you now click this, it's actually going to return the
599.82s - 604.82s |  current page as the PDF. Which means when someone opens this, it's not going to be the same PDF
609.10s - 618.71s |  that, that they expected to open. Instead, it's going to execute JavaScript. So what just
623.11s - 628.11s |  happened? Where did, where did the new PDF come from? Well, the PDF lives on the server because
629.05s - 634.05s |  the HTML lives on the server and you've persistently saved PDF content into that HTML page,
639.73s - 644.73s |  right? And it's sufficiently confused so when they go to download, it downloads what you want
645.57s - 650.57s |  them to download instead of whatever they were trying to download. I mean, this is dangerous. If
652.07s - 657.07s |  you think about it, what if this was a, what if this was a text PDF site? What if this was a text
658.11s - 663.11s |  PDF site or something? It's not going to be just one link that this happens with. It'll be every
663.11s - 668.11s |  link on the page that uses a relative download. As soon as they click it, it's going to download
668.11s - 674.13s |  whatever that PDF is. So the things that you can do with PDF are pretty interesting, right?
674.13s - 679.13s |  Depending on the PDF reader that the user happens to be using. So you have, you have something
680.13s - 685.13s |  like Fox Pro, I think it's called, that allows you to do commands when the PDF opens, right? You
693.37s - 699.21s |  can execute shell commands. So there's nothing stopping you to simply populate your PDF with a
699.21s - 706.21s |  launch command that executes shell commands as long as the user has permission to be doing it to do
706.21s - 711.21s |  whatever you want, right? You can run scripts. You can download new scripts. You can, you know,
713.47s - 717.97s |  whatever. In fact, if you want to stay in the web context, I imagine you could probably open a
717.97s - 724.15s |  Chrome browser headless with the user's data set as their data and then just go explore the
724.15s - 729.15s |  Internet as them, right? Because why couldn't you? So it's pretty dangerous. But thinking this
729.87s - 734.87s |  through a little bit more, it's interesting that you can do it with PDF and all I did here was this
740.15s - 745.15s |  would be, you know, say it's a profile page and you have some public, you know, blurb about
747.75s - 752.75s |  yourself that you've put this PDF in. However you get the persistent content injection onto the
753.55s - 758.55s |  page. But as long as it's there in some capacity and as long as you can save it as a PDF, which we can,
762.79s - 769.17s |  because there's a simple download link, then it will execute. So in the original relative path
769.17s - 774.17s |  overwrite, what you needed was a link tag with a relative with a relative path to the style sheet. In
775.07s - 780.07s |  this case, you need an anchor tag with a relative path to the PDF. And if it's not HTML, the
787.35s - 792.35s |  download has to be fully qualified. Meaning if you don't have sample PDF here as download, then it
794.65s - 799.65s |  won't accept downloading the HTML as a PDF. The content disposition will be wrong. So that's the
799.65s - 804.65s |  basic requirement for executing this. It has to be a page. So think like a, you know, PDF
810.39s - 816.93s |  library or, you know, places where you can inject content and then it says, hey, save your whole
816.93s - 821.93s |  history or something like that. But this technique is going to be more useful the more file types
821.93s - 828.34s |  that we find that you can do this with, right? So so PDF is great. If they happen to be using a PDF
837.42s - 843.56s |  viewer that supports executing shell commands, et cetera. But there has to be other lenient
843.56s - 849.86s |  parsers and lenient file types that you can use to do similar things. So then I started exploring
849.86s - 864.74s |  that. So Gareth Hayes, I was talking to him about this just to make sure that he hadn't heard of it
864.74s - 870.62s |  being discovered before either. And he suggested I talk to a gentleman by the name of Orange
870.62s - 875.62s |  Albertini who specializes in polyglot file types. He's he's a maven. He's the type of guy that his
879.16s - 884.16s |  business card, you can put it into a super Nintendo emulator. And even though it's a PDF, it'll still
884.48s - 889.48s |  play in the emulator. It's crazy. And we were brainstorming other types of lenient file types where
894.70s - 901.28s |  this sort of thing might work with. And we went, you know, zip file types to tar. Tar is another
901.28s - 906.24s |  like plain text one that you can sort of hand write. And it took me down this road looking at that.
906.24s - 910.46s |  And while I was exploring it, I was thinking about, well, what would you put in it? Shell scripts.
910.46s - 915.46s |  And then anyone who's dealt with enough shell scripts knows that shell scripts can actually be really
917.02s - 922.02s |  lenient when they're run too. So what this means is if you happen to be what the user is trying to download
927.98s - 932.98s |  is a shell script, and you have some sort of content injection, you can actually get them to get them to
933.50s - 938.50s |  download, you know, whatever, PowerShell file or a batch file or, you know, batch scripts. And in some
951.40s - 956.40s |  Linux distributions, and I'm looking at especially Windows sub WSL2 with U12, et cetera, you can
963.04s - 968.04s |  actually get them to download. And with some of these things, it will actually happily execute an HTML
969.34s - 974.42s |  file as a shell script. And it will error on every single line until it gets to something that it
974.42s - 979.42s |  understands and then it will execute it. But if someone's going to a site looking to download a shell
980.80s - 985.80s |  script, it's going to say, hey, download this code and then run it as a super user, right, because that's
986.46s - 991.46s |  what they always say. So if they do that, and the link that they clicked happens to have a forward
995.30s - 1000.30s |  slash at the end, it's susceptible to, you know, path confusion, et cetera, then the script that they
1003.54s - 1008.54s |  get is an HTML page, and when they run it, it's going to run. And what that looks like is what that
1009.40s - 1015.80s |  looks like is something like this where every single line, HTML, the head, all of it, errors, line by
1028.38s - 1034.66s |  line by line, and then it gets to echo hello world and it executes it and then it exits and that's it.
1034.66s - 1040.22s |  By the time they realize something happened, it's going to be too late, right? So I managed to get
1040.24s - 1045.24s |  this working with WSL2. I managed to get it working with PowerShell, which does the exact same thing,
1048.58s - 1054.62s |  right? It will execute every single line of the HTML file because it's a PS1 file or whatever, and then
1054.62s - 1060.62s |  it gets to hello world, it executes it, and then it continues, you know, just executing nonsense. The
1060.62s - 1065.64s |  closest I could get it with regular Linux shells is you have to be able to inject something right after
1066.04s - 1071.04s |  HTML, so it's far-fetched, but I threw it in there in case anyone has any ideas on how to get that
1075.18s - 1080.18s |  working. I did actually get this similar thing working on OSX and Linux if a user is using, you know,
1086.72s - 1091.72s |  Box Pro or whatever it is, because you can have a launch command that simply reads whatever it is in
1092.72s - 1097.72s |  its own file and then, you know, executes it line by line. So it can be done. It can be done. And so
1105.10s - 1110.10s |  that's, so now we have two, three, four types of files. Ideally we would have dozens, hundreds.
1113.56s - 1118.56s |  There's so many file types out there to get this type of stuff working well. So I kept on, you know,
1121.74s - 1126.74s |  I kept on investigating other things. What does not mind there being HTML, right? What will
1128.04s - 1133.04s |  someone simply download from the internet and then throw into something and have it execute, right?
1134.72s - 1139.72s |  So pretty much all server-side, server-side templating languages are like this, right? So if
1145.26s - 1150.26s |  someone's looking for, you know, a Ruby or a whatever, a cold fusion script or a, you know, a
1152.20s - 1157.20s |  anything else, they're probably going to grab it and then just throw it into whatever it is to test
1160.54s - 1165.88s |  it, at which point it's already executed anything it needs to execute, right? And the cool thing
1165.88s - 1171.38s |  there is that any of the server-side languages, they don't, they're expecting there to be HTML,
1171.38s - 1175.92s |  right? They're expecting there to be all this, all this other stuff in there so they won't
1175.92s - 1180.92s |  complain. And so these are the ones that I found that it would work with out of box, right? So
1182.56s - 1187.56s |  you have this PHP. Similar thing right here, if we look at this vulnerable PHP. So all this is
1194.44s - 1199.44s |  going to do, if we go here, is the safe download, let's unblock this, safe download goes ahead
1208.21s - 1213.21s |  and it downloads this PHP file, open it up, show you what this, it's generated. And what this did is
1219.52s - 1224.52s |  because it's server-side it already executed this and it says welcome user, no big deal. We do it
1226.26s - 1242.11s |  the other way, forward slash, all of a sudden they download it, this time welcome bad guy, right?
1244.95s - 1249.95s |  So there's nothing stopping you, it doesn't have to be bad guy, it could be read the contents of a
1251.29s - 1256.29s |  password file, whatever, send it somewhere, anything you want it to. If you can get people, the
1258.99s - 1263.33s |  thing that's sort of insidious about this, even though there's like user interaction that could take
1263.33s - 1268.33s |  place, is that the user's going to a server that's trusted, right? There's nothing untrusted about this
1271.71s - 1277.35s |  server, they've been to it a thousand times and they're clicking a download link. There's no
1277.37s - 1282.37s |  difference if that, if you were able to just break into the server and replace the file, right?
1282.37s - 1287.37s |  Because the effect's the same. And, but what would be really interesting is if you didn't have to do
1293.67s - 1298.67s |  this whole download portion of it at all, right? And that's the hardest part, is how do you get someone
1299.67s - 1304.67s |  to, you know, if someone's going to go download a PDF, click on a link and download a PDF, there's a
1309.55s - 1314.75s |  high chance that they're going to go look at the PDF, right? I mean, that's what it's intended for,
1314.75s - 1319.39s |  they're going to trust that whatever's in there, it should be, you make a PDF that looks like a tax
1319.39s - 1325.81s |  document and submits back to your server, whatever. But it would be great if they didn't have to
1325.83s - 1331.29s |  download the thing in the first place. So I started thinking about how that could possibly work.
1331.29s - 1336.29s |  And that's pretty tricky, right? But there's been a rise in all of these HTML over the wire sort of
1344.13s - 1349.81s |  frameworks. There's live wire, there's hot wire, there's HTMX. And all of these things, what
1349.81s - 1354.81s |  they're doing is they're getting HTML from the server instead of putting all the HTML client side,
1359.81s - 1364.91s |  right? And the HTML they get from the server could be, you know, pre-rendered with, like, the
1364.91s - 1370.41s |  user's profile picture and, you know, user name or something like that. But they're going to suffer
1373.91s - 1379.79s |  the same, the same type of issue if they happen to be on a server that returns the current
1379.79s - 1385.79s |  page when there's a forward slash at the end. So something like that, what it would look like is,
1396.58s - 1402.58s |  so I'm not using any of the frameworks. This is just straight jQuery. But what this does is, when
1405.00s - 1411.00s |  the, when the page loads, it attempts to get some, get some DOM element and then inject it into
1417.44s - 1423.44s |  this main div, right? And again, it's the same thing. All we're doing is, you know, doing a, you
1429.28s - 1435.28s |  don't need to do this. All it's doing is just, you know, using jQuery HTML, just setting the HTML.
1437.28s - 1449.36s |  No big deal. And if we go look at what this script does, what's that? There's nothing on this page.
1450.36s - 1456.36s |  There's literally nothing on this page. So all this does is it doesn't do anything initially. You can
1460.78s - 1469.59s |  see when it goes to get the, let me see if I can make this bigger. Excuse me? Oh, gotcha, gotcha.
1473.63s - 1480.17s |  So, so nothing too interesting is happening on this page yet. It gets jQuery. It reaches out and
1480.17s - 1485.37s |  it gets this thing that says welcome user. No big deal. And then it tries to inject the welcome
1485.37s - 1491.37s |  user into the page. But you add a forward slash on to the end and now it actually gets the current
1494.71s - 1505.77s |  page. And one second. Let's see what just happened. What did I do? Oh, there it goes. So, so in
1506.35s - 1516.12s |  this case what just happened is when it reached out to get it, the, the ID that it found was for a
1527.96s - 1534.36s |  frame set and HTML is probably the most lenient parser of all because it has this really weird
1534.36s - 1540.36s |  behavior where if you try to inject HTML tags into other HTML tags that it doesn't like, it will
1541.70s - 1547.70s |  simply strip those tags away. And that's exactly what happened here is because when it
1550.98s - 1563.16s |  founds, whoop, it's not letting me leave. Okay. Whoop. Yeah, so when it founds the, the string
1570.54s - 1576.54s |  frame set, it, when it injected it into this main tag, it actually stripped the frame set away. So
1577.74s - 1582.70s |  even though the browser is saying hey, this frame set is completely innocuous, we're just going to
1582.70s - 1588.70s |  treat it like text, the framework itself can take that and inject it automatically. So, there's
1590.12s - 1596.70s |  places where maybe you weren't concerned about people getting content injection before but now you
1596.70s - 1600.48s |  have to be worried about it. And how do you mitigate something like this? Well, there's all the
1600.48s - 1606.48s |  standard ways to mitigate it. Like one, you have to stop using, either use a base tag so that things
1608.90s - 1614.82s |  aren't relative. They have a, things aren't relative. You can make them root relative by adding a
1614.82s - 1620.82s |  forward slash at the beginning, right? Content security policy, all these things, you have to do
1621.06s - 1627.06s |  something. But also educating people, you can't actually trust a file download if it's using a
1628.90s - 1634.90s |  relative, if it's using a relative path. So, basic mitigations in place. Yeah. All the, all the
1635.72s - 1644.38s |  basic mitigations, so path handling, it's important to check if the server that you're using is
1657.06s - 1663.30s |  vulnerable to this type of path handling. If you add a forward slash at the end of the URL and it
1663.30s - 1669.30s |  loses all of its styling, it's a pretty good indication that it has this issue. Default PHP
1669.30s - 1675.30s |  server has this issue. I did make a GitHub for this. If anyone wants to play with it themselves,
1675.30s - 1683.63s |  you can find it on GitHub. Yeah. So, this is a novel file injection vector. I figured out a
1683.91s - 1689.91s |  number of server side scripts, shell scripts, PDFs, but there has to be, there has to be dozens
1700.37s - 1707.29s |  more if, you know, people want to explore it. I rarely do bug bounties because I don't work in
1707.33s - 1713.33s |  security. So I do do it sometimes. But it feels like a job to me. So, so I try not to. I would
1715.25s - 1721.83s |  imagine there's a number of these vulnerabilities that you can go and find. If you're so
1721.83s - 1727.83s |  inclined, no one's ever looked for them before. So, yeah. I think, I think that there's a lot
1730.61s - 1736.21s |  more that can be done there with the shell scripts on Linux to figure out how to get them to
1736.21s - 1742.21s |  execute. And, yeah. I think if any of you do bug bounties, you should go out and try to find
1743.95s - 1745.71s |  it. And that is my talk.


url: https://www.youtube.com/watch?v=bVKtcu82a8w
title: DEF CON 32 -Ticking SQLi - Iggy
type: video
author: DEFCONConference
ingestion_date: 2024-11-09
transcription_model: large-v2
chunking_status: completed
content:
{
  "webpage_url": "https://www.youtube.com/watch?v=bVKtcu82a8w",
  "title": "DEF CON 32 -Ticking SQLi - Iggy",
  "description": "Explore the intricacies of time-based SQL injection through the lens of Operation GhostShell. This session delves into the methods used by attackers to exploit time delays for data extraction, highlighting real-world examples and the impact of these breaches on major universities. Attendees will gain a deep understanding of the technical aspects, see live demonstrations, and learn practical defense strategies to safeguard against such vulnerabilities. Perfect for security professionals seeking to enhance their knowledge of advanced SQLi techniques and mitigation.",
  "channel_url": "https://www.youtube.com/channel/UC6Om9kAkl32dWlDSNlDS9Iw",
  "duration": 1508,
  "channel": "DEFCONConference",
  "uploader": "DEFCONConference",
  "upload_date": "20241016"
}

0.66s - 7.68s | This text was transcribed using whisper model: large-v2

 Alright folks, it is that time. If you have a seat that is in between you, please collapse
7.68s - 14.18s |  down. Specifically you sir, thank you very much. If you would bring it in close. Alright
14.18s - 18.96s |  with that we are gonna go ahead and kick everything off. I would like to introduce Iggy to the
18.96s - 20.96s |  stage. Thank you.
20.96s - 35.44s |  So hello DEF CON and welcome to my talk called Ticking SQLI. And we will begin. So what's
35.44s - 41.36s |  in it for you today? Today we will explore types of SQL injection. We will focus a bit
41.36s - 48.40s |  on time based SQL injection. We will see the impact team Ghost Shell did in the past using
48.40s - 53.08s |  these kinds of techniques. We will understand why it's still relevant today and you will
53.08s - 60.20s |  have a good time with a pretty much good presenter today. So who am I? My name is Igor but please
60.20s - 66.60s |  call me Iggy. It's my nickname since childhood. I'm a security engineer at Axonius. One sentence
66.60s - 74.56s |  about Axonius is it's a cyber security asset management platform that maps all your digital
74.56s - 79.72s |  assets across your organization, normalize them, deduplicate them and give you comprehensive
79.72s - 85.24s |  insights about them and much more. And why I'm here today? I'm here today because I love
85.24s - 92.32s |  learning stuff and especially in cyber security. Since from low level to high level, from lock
92.32s - 98.88s |  picking to RFIDs and I'm studying a lot and for my studies I came to this idea to talk
98.88s - 104.28s |  about this topic today. And the last thing about me, it's the first time I'm presenting
104.28s - 110.00s |  so thank you AppSecVillage for giving me this opportunity today to be in such an honorable
110.00s - 117.28s |  stage as AppSecVillage and DEF CON. I truly appreciate it. And we'll begin. We'll begin
117.28s - 123.88s |  by introducing what is SQL injection. SQL injection is a technique where attackers insert
123.88s - 130.88s |  malicious SQL code into some input fields. It can be login forms, search boxes or URL
130.88s - 139.00s |  parameters and they do it mainly to retrieve sensitive information from the database for
139.00s - 147.32s |  the application. They can retrieve sensitive information, delete tables and a lot of things
147.32s - 154.16s |  that we'll see in the next slides. But the bottom line here is the attackers using some
154.16s - 159.92s |  malicious SQL payload to manipulate the web application and the database behind it to
159.92s - 175.84s |  retrieve some and to do some sensitive operations. The example that I like the most to show what
175.88s - 181.20s |  it is SQL injection is to see how it looks through the code. We have a query where the username
181.20s - 186.76s |  and the password flows to its parameters. In the UI it looks something like this. You get a
186.76s - 193.68s |  login form. It's like to demonstrate the most basic type of SQL injection. You have the
193.68s - 199.72s |  username, you can enter admin and some malicious payload to the password that basically means
199.72s - 207.68s |  a single quote and some old condition that always evaluates to true. This flows to the query.
207.68s - 213.52s |  The username gets the admin and the password gets some payload that always evaluates to true
213.52s - 219.80s |  and by that the attacker can bypass the authentication because it's sending it to the web
219.80s - 229.91s |  application that's sending it to its database and it retrieves authentication bypass. We have
229.91s - 236.95s |  some different types of SQL injection. First type called in-band or classic what essentially
236.95s - 244.55s |  it means that you see the results of your payload directly in HTTP response. Actually it also
244.55s - 252.03s |  can be served as self-category but I want to stick as many resources I'll state in it. So we
252.03s - 257.47s |  have the in-band that you see the response directly into the web application. It also divided
257.51s - 267.51s |  into two categories. One of them is error-based. Error-based what it means is that you rely on
267.51s - 276.31s |  errors that you retrieve from the database to see some sensitive information and you have a
276.31s - 282.31s |  union-based type. What it means in general we will in a second see demonstration for this but in
282.31s - 290.63s |  one sentence union-based is concatenating two data, two data or tables into one that contains
290.63s - 297.95s |  both of them or we can say that it's two select statements that combined into one result. We
297.95s - 306.15s |  combine to some expected data additional concatenation of a table that retrieves sensitive
306.19s - 313.99s |  information. We will see it in the next slides. So to demonstrate error-based SQL injection we
313.99s - 321.99s |  can see that we we should try to probe the application. Most of the times we will try through
321.99s - 326.83s |  one single quote and two single quotes is the most type of entry points but you have additional
326.83s - 333.19s |  entry points and comments and you starting to get a sense of some error-based attack when you
333.19s - 341.95s |  get from the one single quote uh five hundred internal server error and and for the double
341.95s - 347.27s |  single quotes uh two hundred okay. What it means that the SQL query behind it in the backend
347.27s - 355.35s |  cannot uh uh process the uh the query correctly because it's not expected to receive five
355.35s - 361.15s |  hundred internal server error and if you close it in single quotes it suddenly is okay. So there
361.15s - 367.03s |  are some probably manipulation that you can do on on the SQL. So we'll try to do some
367.03s - 373.39s |  manipulation. We can send a curl command that uh looks a bit messy but it's very simple. It's
373.39s - 378.95s |  uh trying to convert the version uh function that you send to the database to retrieve its
378.95s - 385.35s |  version. You try to convert it to an integer and compare it to one just the so the well
385.35s - 390.43s |  closed will operate correctly. Uh it will look something like this when it flows to the
390.43s - 400.43s |  query and you get the error uh conversion conversion failed. You get the MySQL that the
400.43s - 406.07s |  server uses MySQL with a specific version and by this by this will retrieve a sensitive
406.07s - 411.35s |  information. Uh I want to state here that uh something important. I stick to this example
411.35s - 416.09s |  of uh of retrieving the version of the database in many additional slides that you will see
416.09s - 420.73s |  later. I like this example because it's not too basic and not too advanced. It's something in
420.73s - 425.73s |  the middle so it's easy to show it but in this techniques you can do something even more
425.73s - 431.21s |  advanced to retrieve hash passwords and so on. So but we'll stick to the version because it's
431.21s - 438.41s |  truly a real world example on one hand but on the other hand it's not too complex. So now we'll
438.41s - 443.41s |  move to the union based as we showed before and this uh presenting uh expected regular uh
446.15s - 452.15s |  curl command to a server that I build. We send query equals to Alice and we'll retrieve some
452.15s - 458.33s |  expected information. But when we try to manipulate with a union uh attack we will edit a
458.33s - 463.33s |  union select the first oh the user that is currently connected to the database uh through the
464.33s - 469.33s |  web application uh we of course need to encode it and send it to the server and that's what we
472.97s - 479.97s |  get. We get the Alice the query equals to Alice as we see so before and we get the current user
479.97s - 485.17s |  that's connected to the database for the web application and this is information that not
485.17s - 490.19s |  supposed to be retrieved especially so easy so this is how union based attack looks like.
491.19s - 498.19s |  After that we have many resources uh dividing into these categories uh one of them is out of
498.19s - 503.19s |  band I won't uh elaborate about it too much because it's something not so popular but to say it uh to
506.57s - 511.81s |  explain it in one or two sentences out of band attacks are the kind of attacks that you cannot
511.81s - 516.91s |  communicate with the web application the server directly and you need to utilize some external
516.93s - 523.33s |  server to exfiltrate data through the database and you use uh some HTTP request and DNS
523.33s - 530.11s |  queries to exfiltrate this data but it's something much uh not so common so we won't elaborate
530.11s - 535.11s |  about it too much. Uh the third kind uh inferential or blind uh type of SQL injection is in
537.25s - 543.69s |  general a bit of the opposite of the in band of course it's type of attacks that you cannot um uh
543.71s - 548.91s |  see your payload or the results of your payload directly in the webpage and you need to infill the
548.91s - 555.15s |  results of your payload uh from the behavior of the web application. It is divided to two
555.15s - 560.15s |  categories one of them is boolean based uh I will explain that by example uh but generally
563.49s - 569.53s |  speaking it's evaluates your payload to true or false uh in some way of behavior of the web
569.57s - 576.31s |  application and like that you can infill the results uh to see example of it uh I will show you by
576.31s - 581.31s |  demonstrating on B-Web it's a buggy web application it's a cool application to practice on you can
583.55s - 588.55s |  see here a payload of substring uh to the version of the database uh in short what it means it's uh
590.89s - 595.89s |  takes the version, checks the fields character and compare it's uh to six in this case and comments
596.29s - 601.29s |  all the rest. We flow it to the search field and we see that the movie does not exist. But when we
607.47s - 613.53s |  change the six to a five and we'll flow it again to the search box we get that the movie does
613.53s - 618.71s |  exist. So we can infill by that that when you're comparing to fives the first character of the
618.71s - 624.59s |  version of the database is starting with a five. And this kind of uh in this kind of blind SQL
624.59s - 629.59s |  injections uh you need uh enumerate uh enumerate uh each character and uh until you get the whole
632.67s - 640.64s |  picture uh but that's how it looks like. And the last type of uh influential blind that we'll focus
642.98s - 648.98s |  a bit more today called time based. Time based is in a way it's similar to Boolean based but it relies
648.98s - 653.98s |  on a time delays that you get from the server and that's how you can infill your uh SQL payload uh
654.48s - 659.48s |  by sending a time delay of for example uh ten seconds the database is delayed with this it's
662.72s - 668.52s |  response for ten seconds and that's how you can infill if your payload was evaluated to true or
668.52s - 676.12s |  false. Uh and that's how a time blind SQL injection will look like. Uh to my web server I added uh a
679.70s - 685.00s |  parameter of elapsed time to see how much it takes to the server to respond. So in particular
685.02s - 690.02s |  query uh where it equals to Alice the elapsed time was immediate but when we try to run a time based
692.70s - 697.70s |  payload uh uh for example that will look like this with uh closing single quote uh so the SQL uh
700.30s - 705.84s |  payload will operate uh normally and a sleep command that causes the the server to delay for five
705.84s - 710.84s |  seconds we encode it and we will uh construct in the URL sending it back and we can suddenly see that
715.02s - 720.32s |  with the sleep command of five seconds the elapsed time was five seconds. So in general what it
720.32s - 725.32s |  means that the time delay was successfully executed. Uh before we continue about time delays I want to
727.00s - 732.00s |  elaborate a bit about tools to be connected. Uh there are many tools to uh uncover SQL injection
734.50s - 739.50s |  one of them is a bit less popular uh called Havij. I decided to include it because when I research
740.14s - 745.14s |  about Tim Goschel uh some resources stated they use this tool and I never met it before and it's a
749.32s - 754.32s |  tool that was used back then uh few years ago but it's still uh people can use it today but it's much
755.66s - 760.66s |  less popular or good tool. Uh we have uh many dust tools like uh uh personally here is uh a
761.66s - 766.66s |  belt suit by Paul Swigel that can scan your site uh it's a dust tool and find SQL injection as we
773.04s - 779.18s |  see here in the picture that I scanned my uh web server. But the most uh common tool for SQL
779.18s - 784.18s |  injection and uncovering them is called SQL map. Uh I'm so happy that I uh zoomed in on what's
784.88s - 789.88s |  relevant so SQL map will look something like this. Uh you'll run SQL command on the URL that you
793.88s - 798.86s |  want to scan. In this example we see batch command that what it means is uh non-interactive
798.86s - 803.86s |  command that uh won't interfere in the process of executing and please retrieve us the database
805.62s - 810.64s |  version. When we run it we can see that uh SQL map uncovers that we have a Boolean based uh time
811.10s - 816.10s |  based and union union attack seal and we see the stuff that we wanted to retrieve that the
819.34s - 824.34s |  database uses uh SQLite behind the curtain. Uh to show you of course SQL map can do a bit more
826.78s - 831.78s |  advanced stuff like in this example uh we can execute SQL map with uh time delay of five
832.44s - 837.44s |  seconds. I'll run it with a batch command as before and leave please uh return us the banner of
843.48s - 848.48s |  the database and we can see that it successfully works. We can retrieve the banner of uh the
850.92s - 857.83s |  two equals to eight point three point zero. And before I continue to the next slides I want to
857.91s - 863.95s |  emphasize something important that uh for my personal opinion we cannot realize on uh many
863.95s - 870.09s |  automation tools. Uh we always need to have the ability to write our own scripts to some
870.09s - 875.79s |  particular cases we don't need to be afraid and like in a perfect world we need to master a
875.79s - 882.07s |  combination of the two. We can master the automating tools and writing our own scripts. And a
882.07s - 888.35s |  script of uh executing time delay can look something uh simple as as this. We can have a
888.35s - 894.33s |  script with of course some constants and uh some expected parameters but in general it will be
894.33s - 900.13s |  divided to three functions. One function that will check the delay of the HTTP response and
900.13s - 905.97s |  will record it. The second function will extract the version character it uh takes the position
905.97s - 911.91s |  that we are checking in our payload uh it compares it to the character, character uh
911.91s - 917.21s |  enumerates for all the ASCII's. It compares to the position in the version that we want to
917.21s - 922.23s |  retrieve. If it got the sleep time then it means that we successfully executed it or if not uh it
924.55s - 929.57s |  didn't work. And the first function it's uh uh continues to enumerate all the next indexes in
930.43s - 937.61s |  the version number and recalls the version that we are retrieving of the database. And
937.61s - 944.62s |  eventually it will look something like this when we're running it. Extracting the database
944.62s - 952.62s |  version. So it's very simple, simple as that uh as we can see. Uh to move to a bit uh more real
957.06s - 964.06s |  world examples I want to show you uh few stuff. One of them a real world example uh of a
964.06s - 970.62s |  guy called Mola here is uh YouTube and uh his uh site. I really like to learn about uh real
970.62s - 976.02s |  world examples because it uh really demonstrates what's what's we need, what's truly
976.02s - 983.16s |  important. And here he claims that he earned a bounty of five thousand dollars uh finding the
983.16s - 990.04s |  next payload that you will see to in a first billing uh site. He uh what he claims that he did
990.06s - 996.80s |  to execute this bounty he ran a sleep command that we saw before uh concatenated to some uh
996.80s - 1003.14s |  parameter that the site had uh per page equals to one and executed time delay. He got truly uh
1003.14s - 1009.44s |  five seconds delay. Then he executed this payload. This this payload is exactly as the substring
1009.44s - 1015.42s |  we saw before. The mid is equals to the substring but uh particular to my SQL version. And it
1015.44s - 1020.54s |  checks the version fills character uh if it equals to five go to sleep for fifteen seconds. If
1020.54s - 1027.58s |  not return zero. And in this case he doesn't he didn't get uh any delay. But when he uh changed
1027.58s - 1033.42s |  the five to eight he could successfully gain uh fifteen seconds delay and by that he gained the
1033.42s - 1042.04s |  bounty. Just to show you a bit more real world examples uh one of them uh looks something like
1042.08s - 1048.52s |  this. It's some bounty that uses XOR interesting uh payload that always evaluates to true. And
1048.52s - 1055.16s |  some bounty that was discovered in department of uh defense uh in HackerOne system. The second
1055.16s - 1062.14s |  one was uh pretty simple uh concatenation to uh expected parameters of uh sleep seven seconds
1062.14s - 1067.14s |  also disclosed to HackerOne um and found in the automatic uh website. And the third one my
1067.82s - 1072.82s |  favorite uh combines uh script, union, and sleep commands. Uh some build uh combination of
1078.12s - 1086.41s |  both of all of them. Uh ah and the third bounty gained uh what I found from uh resources when I
1089.35s - 1094.35s |  checked it uh got bounty of two thousand and one hundred dollars. So uh all of the stuff that we
1094.71s - 1099.71s |  seen was utilized by uh Tim Goshell in the past uh they used a lot of SQL injection
1104.55s - 1110.29s |  especially time delays. Something even uh what they did uh they was they were operating uh in
1110.29s - 1116.67s |  two thousand and twelve. And they utilized even simpler uh SQL commands that we SQL injection
1116.67s - 1123.27s |  commands than than what we saw before. And as I said they were operating in two thousand and
1123.29s - 1128.33s |  twelve. They are not related to APT39 or some Iranian government. They actually were a
1128.33s - 1133.33s |  hacktivist group. Um I'm just curious how many people of you uh know what hacktivism is? Please
1136.27s - 1141.27s |  raise your hands. Okay okay not bad. Approximately half. So hacktivism is uh when you attacking
1144.31s - 1150.59s |  group that uh trying to promote some uh political change or some social agenda that you believe
1150.67s - 1156.95s |  in and you promote it by hacking. And that's what Tim Goshell were. They were a hacktivist
1156.95s - 1163.65s |  collective. And just to see what they did and who they were uh they started to operate in two
1163.65s - 1168.63s |  thousand and twelve. They hacked the University of Arkansas uh because some one of the guys
1168.63s - 1175.33s |  there uh called Zoom was arrested. So they hacked it in terms of protest. Uh after the dead
1175.35s - 1182.85s |  Melox uh uh was will appealed was appealed. Uh dead Melox was the admin of the group. He
1182.85s - 1188.15s |  appealed in their uh main communication channel uh in Facebin stating that he doesn't in
1188.15s - 1192.63s |  ironic way that he doesn't want to fight with FBI or CIA or something like this. But on the
1192.63s - 1197.63s |  other hand we we leak bunch of stuff uh in the uh coming year. So as he promised he started uh
1198.61s - 1203.61s |  they started to uh make additional hacking projects. Uh they uh had the project Dragonfly
1208.41s - 1215.15s |  where they hacked uh China government and leaked uh thousands of uh sensitive information.
1215.15s - 1221.13s |  Uh it included like hash passwords, some personal data and like everything they got. Later
1221.13s - 1226.27s |  Hellfire in project Hellfire they hacked uh governments, financial sectors and uh other
1226.27s - 1231.35s |  companies. In project West Wing they focused on universities around the globe from
1231.35s - 1236.35s |  Harvard, Stanford to University of Rome in Tokyo. And they even declared a war uh in project
1238.55s - 1243.55s |  Black Star against Russia. Uh in in all of these projects they hacked like uh they uh leaked
1248.43s - 1253.43s |  uh thousands to millions of uh records from uh personal information to hash passwords as I
1253.73s - 1258.73s |  said before but many of this information was publicly already disclosed. Still on this other
1261.83s - 1267.33s |  hand it was uh sensitive information that truly was uh released but what they achieved from
1267.33s - 1272.35s |  it it uh they achieved their main cause to raise uh awareness to their hacktivism and to their
1274.05s - 1279.89s |  operations. So later they in project White Fox they basically hacked everyone to raise
1279.91s - 1284.91s |  awareness to hacktivism and they uh introduced their uh new cool logo. Later came Sunrise
1289.25s - 1294.25s |  hacked South Africa. Uh 2015 introduced a new term of dark hacktivism that raising the normal
1295.75s - 1300.75s |  hacktivism into cyber warfare levels in a much more scale. And in 2016 this story came to an end
1302.39s - 1308.23s |  when the dead Melox the leader came clean. He came clean to some uh newspaper uh send them an
1308.25s - 1313.81s |  email that hi it's me I'm uh Eugene the man who behind the dead Melox the leader of Tim
1313.81s - 1320.39s |  Goshell he sent his passport and even a picture. They still didn't believe them him so he posted
1320.39s - 1325.19s |  on his Twitter account to the uh next web the journal that he came clean to that hello it's
1325.19s - 1331.92s |  truly me. So why why they did what they did uh as typical for hacktivism for example Westwind
1332.76s - 1337.76s |  the hack to uh raise awareness oops raise awareness to the foreign standards of the education
1342.70s - 1347.70s |  today. In Black Star they uh uh claimed that the Russian is a government of tyranny and uh and
1350.34s - 1355.34s |  regret they uh criticized their government. In White Fox uh essentially they wanted to promote
1355.68s - 1360.68s |  hacktivism worldwide and draw attention to the freedom of uh information and they had many
1364.88s - 1369.88s |  other causes like even on Japan they wanted to piss them off just just as it is. So why the hell
1372.38s - 1377.86s |  the I want you all of you to know this today. I know that you are thinking SQL injection is old
1377.86s - 1384.86s |  and Tim Goshell was operated in 2012 but I will show you that what is old is still worth gold. Uh
1384.88s - 1390.22s |  we can see some statistics you know as top 10 uh SQL injection is part of the injection section
1390.22s - 1399.10s |  it's still on the top uh in many years it was the first one and now it's like the third one. Ah
1399.10s - 1404.10s |  whoops I didn't like confuse me for a second. Yes? There was? Okay. NVD is ranking uh SQL
1406.54s - 1411.54s |  injection uh ranking it uh in the second place for many many years. Uh it considered one of the
1412.16s - 1417.16s |  most stubborn weaknesses ranked in the third place uh by Mitre and um GitHub security
1422.30s - 1427.34s |  advisor also stating that and it's truly stated that SQL injection is not relevant to model
1427.34s - 1432.92s |  languages like Vue, React, Python and stuff like this but for my opinion what important to know
1432.92s - 1438.42s |  here that the concepts or uh the mindset that you set for yourself when you are learning about
1438.44s - 1444.84s |  these attacks can be implemented to many other fields because they are all related uh you can use
1444.84s - 1451.12s |  similar techniques to other attacks. And if I still didn't convince you, you can earn bug
1451.12s - 1457.22s |  bounties easily using those kinds of techniques like this one uh was disclosed critical
1457.22s - 1464.05s |  vulnerability to Department of Defense and so we are coming to an end and I want to leave you
1464.05s - 1469.11s |  with some homework. I don't want to state here uh 10 resources but I want to state one and good
1469.15s - 1475.09s |  one that uh I use, I don't get money by Boltzweigle by the way, I truly recommend them because
1475.09s - 1480.09s |  it's free resource uh theoretical and practical combined, very recommended and it dives deep. So
1482.67s - 1489.67s |  to recap, it's hard to make a 25 minute lecture because I want to talk about many stuff and I only
1489.67s - 1495.71s |  have 25 minutes so I tried my best and if you gain even one small little thing today I will be
1495.71s - 1500.75s |  genuinely happy and that's it if you want to connect with me like LinkedIn and Twitter and that's it.


url: https://www.youtube.com/watch?v=WG_oXtUTEoI
title: DEF CON 32 - Transforming AppSec Protecting 'Everything as Code' - Kunal Bhattacharya
type: video
author: DEFCONConference
ingestion_date: 2024-11-09
transcription_model: large-v2
chunking_status: completed
content:
{
  "webpage_url": "https://www.youtube.com/watch?v=WG_oXtUTEoI",
  "title": "DEF CON 32 - Transforming AppSec  Protecting 'Everything as Code' - Kunal Bhattacharya",
  "description": "The continuous adoption of emerging technological trends like Mobile, IoT, Cloud, Blockchains, and now GenAI has transformed application security from simple threat modeling and SAST/DAST scans to comprehensive proactive prevention and real-time detection of security anomalies. This panel will discuss the dynamic nature of AppSec as the lines between traditional infrastructure and cloud environments blur. We will explore the importance of maintaining a comprehensive security posture within 'everything as code' ecosystems, emphasizing proper configuration and secret management to secure infrastructure effectively.",
  "channel_url": "https://www.youtube.com/channel/UC6Om9kAkl32dWlDSNlDS9Iw",
  "duration": 2698,
  "channel": "DEFCONConference",
  "uploader": "DEFCONConference",
  "upload_date": "20241016"
}

0.11s - 3.11s | This text was transcribed using whisper model: large-v2

 Hi, my name is Kunal Bhattacharya.
3.11s - 6.11s |  I'm a senior security leader and advisor.
6.11s - 9.11s |  My security journey has been pretty long.
9.11s - 13.11s |  Before I came into security, I've done a bunch of different things.
13.11s - 18.11s |  I was a developer, was a sysadmin, was a QA architect.
18.11s - 22.11s |  I finally moved to security and never looked back.
22.11s - 27.11s |  I also have had the privilege of building a homegrown bug bounty program
27.11s - 31.11s |  before the likes of HackerOne. I don't know if they are the sponsor.
31.11s - 36.11s |  And then eventually moved into a lot of the security stuff.
36.11s - 42.11s |  And I just consider myself to be a lifelong learner.
42.11s - 50.11s |  Apart from security, I guess I'm an avid runner, swimmer, dad, whatever.
50.11s - 55.11s |  Yeah, so that's me. Sarah, you want to introduce yourself?
55.11s - 57.11s |  Hey, everyone. My name is Sarah Dharzadeh.
57.11s - 59.11s |  I lead vulnerability management at DocuSign.
59.11s - 64.11s |  I've been in the vulnerability management space specifically for about five years
64.11s - 67.11s |  and really passionate about infrastructure security,
67.11s - 71.11s |  about patch automation, and also emergent vulnerability response.
71.11s - 73.11s |  As it relates to everything as code,
73.11s - 78.11s |  my first introduction to everything as code was infrastructure as code.
78.11s - 81.11s |  When I was building patch workflows in AWS,
81.11s - 85.11s |  utilizing everything, all of my resources were deployed using CloudFormation.
85.11s - 90.54s |  So I'm really passionate about this talk as well.
90.54s - 93.54s |  Hi, everyone. My name is Shaka Aman,
93.54s - 97.54s |  and I'm the CEO and co-founder of Backslash Security.
97.54s - 103.08s |  Backslash is doing reachability-based AppSec.
103.08s - 109.08s |  What do I want for AppSec?
109.08s - 111.08s |  How about now? Better?
111.08s - 120.91s |  Hi. Can you hear me at the back? Perfect.
120.91s - 125.91s |  So, again, Shaka Aman, co-founder and CEO of Backslash Security.
125.91s - 131.91s |  Previously, I was leading the engineering team of a company called Aqua Security,
131.91s - 135.91s |  including the product management for putting out there
135.91s - 138.91s |  very nice open-source tools for the community,
138.91s - 141.91s |  like Trivi, CubeHunter, if you're using some of those.
141.91s - 145.91s |  And most of my career I spent with the enterprise leading engineering team,
145.91s - 150.91s |  mainly at SAP, both in Germany and Tel Aviv.
150.91s - 152.91s |  Happy to be here today.
152.91s - 157.15s |  Thank you, Shahal.
157.15s - 159.15s |  Before I introduce myself,
159.15s - 162.15s |  it's an absolute honor and privilege to be here
162.15s - 166.15s |  and sharing stage with these seasoned leaders.
166.15s - 170.15s |  But I do want to take 10 seconds to express silent gratitude
170.15s - 172.15s |  to our awesome volunteers.
172.15s - 177.15s |  So let's have a big round of applause for our volunteers.
177.15s - 185.13s |  You know, they spend a lot of hours for these operations to run smoothly.
185.13s - 188.13s |  And I really want to make sure their efforts
188.13s - 193.13s |  and all the hard work they put shouldn't get unrecognized.
193.13s - 200.46s |  So thank you again, all the awesome volunteers of DEF CON.
200.46s - 202.46s |  Everyone, my name is Trukti.
202.46s - 205.46s |  I'm director of product security at DocuSign.
205.46s - 211.46s |  And before entering the awesome field of software security,
211.46s - 214.46s |  18 years back, I was a mobile game developer.
214.46s - 218.46s |  So I'm a developer turned cybersecurity professional.
218.46s - 220.46s |  And I absolutely love doing security.
220.46s - 222.46s |  I don't know what else I'll do.
222.46s - 224.46s |  It's a passion. It's a purpose.
224.46s - 227.46s |  I even call this is my Ikegai.
227.46s - 229.46s |  When I'm not doing security,
229.46s - 234.46s |  I like to paint, do nature work, and meditate.
234.46s - 236.46s |  So that's about me.
236.46s - 239.46s |  Thanks, everyone, for the introduction.
239.46s - 241.46s |  We have some veterans here.
241.46s - 244.46s |  So quickly jumping to the topic.
244.46s - 247.46s |  AppSec, it's a very loaded term these days.
247.46s - 251.46s |  And people ask me, what is AppSec?
251.46s - 255.46s |  And why do you want to transform it?
255.46s - 257.46s |  So what is AppSec?
257.46s - 260.46s |  It is application security, as everyone knows.
260.46s - 265.46s |  I want to talk about how AppSec has been in the past
265.46s - 267.46s |  and what we are seeing now as trends.
267.46s - 272.46s |  And what do we anticipate how things will be going forward?
272.46s - 276.46s |  Probably like a prediction from the veterans here.
276.46s - 281.46s |  With that said, we'll get to the questions that I had.
281.46s - 284.46s |  What are the key challenges in securing everything as code?
284.46s - 286.46s |  Nowadays, everything is code.
286.46s - 287.46s |  Policy is code.
287.46s - 289.46s |  Code is code, for sure.
290.46s - 295.46s |  Including things like ISE, GitOps, serverless.
295.46s - 297.46s |  What are the bigger challenges that exist today?
297.46s - 301.06s |  Sarah, do you want to take that?
301.06s - 305.06s |  So I'd say before the popularization of everything as code,
305.06s - 308.06s |  I think things were a lot more straightforward and clean-cut.
308.06s - 313.06s |  I think in terms of separation of roles and duties,
313.06s - 317.06s |  the network engineer versus the application developer
317.06s - 319.06s |  versus security versus ops.
319.06s - 323.06s |  Those had very specific clean-cut roles and responsibilities.
323.06s - 327.06s |  And the understanding of what their assets were
327.06s - 331.06s |  and how they applied to them was also really straightforward.
331.06s - 334.06s |  And then on the security side, similarly,
334.06s - 336.06s |  because of that same asset delineation,
336.06s - 340.06s |  I think knowing what we were going to run assessments on
340.06s - 342.06s |  tended to be a lot more straightforward.
342.06s - 344.06s |  We knew what we were going to run, where we were going to run it,
344.06s - 348.06s |  how we were going to run it, what those scans were going to produce.
348.06s - 354.06s |  And that made things a lot more easy to process.
354.06s - 358.06s |  In terms of today, I see three main challenges with everything as code.
358.06s - 364.06s |  I think the first thing is that there's really no longer that separation of duties.
364.06s - 368.06s |  So everybody is sort of speaking in similar languages.
368.06s - 371.06s |  Some may be more familiar with those languages than others.
371.06s - 376.06s |  A lot of people are going to be learning completely new languages,
376.06s - 378.06s |  all of us at the same time.
378.06s - 382.06s |  And that can present challenges in terms of understanding who owns what,
382.06s - 385.06s |  how do we get the right context to the right people,
385.06s - 390.06s |  and who's going to ultimately do remediation work.
390.06s - 392.06s |  In terms of other challenges that I see,
392.06s - 397.06s |  I think data is another big one on the security side.
397.06s - 403.06s |  With everything as code, we are just producing a crap ton of data,
403.06s - 405.06s |  especially on the vulnerability management side.
406.06s - 410.06s |  Unfortunately, I think a lot of security budgets don't include a ton of money
410.06s - 414.06s |  for really awesome data architects to help us make sense of it all.
414.06s - 418.06s |  So how do we normalize and correlate all of this data?
418.06s - 421.06s |  How do we build complex data models?
421.06s - 427.06s |  How do we ensure that we have good performance against all of our data sets?
427.06s - 429.06s |  That's another really big challenge.
429.06s - 430.06s |  Yeah.
430.06s - 431.06s |  That's it.
431.06s - 433.06s |  Some big things in mind.
433.06s - 434.06s |  Thanks, Sarah.
434.06s - 435.06s |  That was very good.
435.06s - 438.06s |  You talked about the data side, which, of course, goes into AI.
438.06s - 441.06s |  So I'm just going to hold on until I get to that.
441.06s - 446.06s |  But, Trupti, what are your views on this?
446.06s - 452.57s |  I think our high-tech industry has seen numerous emerging trends.
452.57s - 457.57s |  How many of you remember evolution of iPhone and iPhone-based application,
457.57s - 459.57s |  the mobile era?
459.57s - 462.57s |  How many of you remember the IoT era?
462.57s - 468.88s |  And then how many of you remember the cryptocurrency era?
468.88s - 472.88s |  Who actually thought they will make enough money that they can retire?
472.88s - 473.88s |  Good.
473.88s - 475.88s |  All of you are smart.
475.88s - 478.88s |  And then recently we saw the Gen AI era, right?
478.88s - 484.88s |  So all these emerging trends, and as we move from monolith to microservice
484.88s - 489.88s |  and low-code, no-code type of architectural style enterprise application
489.88s - 493.88s |  deployed in cloud, what is one thing common?
493.88s - 496.88s |  Everything is accessible via API.
496.88s - 497.88s |  Everything is API-based.
497.88s - 498.88s |  What does that mean?
498.88s - 501.88s |  Everything has become code, right?
501.88s - 506.88s |  So no matter what newer technology comes, everything is code nowadays.
506.88s - 511.88s |  And we are supposed to launch our products where APIs are available.
511.88s - 518.88s |  So I see for product security, Kunal, that we as a security engineer,
518.88s - 521.88s |  product security engineer, we are supposed to quickly learn
521.88s - 526.88s |  about the emerging technologies, not only how to secure them,
526.88s - 531.88s |  but the business logic context and whatnot, and stay ahead of the game.
531.88s - 533.88s |  That's the biggest challenge I see.
533.88s - 536.88s |  What about you, Shahar?
536.88s - 538.88s |  Yeah, maybe I'll add to that.
538.88s - 541.88s |  You know, in the past, things used to be very simple.
541.88s - 544.88s |  I remember my days back at SAP.
544.88s - 547.88s |  It was a clear separation between the application guys
547.88s - 548.88s |  and the infrastructure guys.
548.88s - 550.88s |  It was like a huge wall.
550.88s - 553.88s |  When you're done with developing, you just throw it over the wall
553.88s - 556.88s |  to the infrastructure guys.
556.88s - 559.88s |  They'll need to deploy it, make sure it's secure, et cetera.
559.88s - 565.88s |  And now, with the beasts coming from the infrastructure world
565.88s - 571.88s |  into the application space, living in the Git as a code artifact,
571.88s - 574.88s |  it brings a very different mindset, not only technology,
574.88s - 576.88s |  to the different parts.
576.88s - 580.88s |  So maybe just to give one example.
580.88s - 585.88s |  In a raise of hands, how many of you have worked with containers?
585.88s - 587.88s |  Right, so it's becoming very popular.
587.88s - 592.88s |  But it's not that obvious that developers need to take part
592.88s - 597.88s |  in the infrastructure layer of the base image, for example.
597.88s - 600.88s |  It contains stuff that was not in our responsibility.
600.88s - 602.88s |  You know, as developers, we just want things to work.
602.88s - 605.88s |  Once they work, we're fine, we're happy.
605.88s - 609.88s |  We don't just start looking into vulnerabilities in the operating system,
609.88s - 610.88s |  in the core.
610.88s - 613.88s |  When it works, we're happy.
613.88s - 617.88s |  So the challenge is now responsibility have shifted left,
617.88s - 619.88s |  have shifted to where the code lives,
619.88s - 622.88s |  also for artifacts that traditionally
622.88s - 625.88s |  was not part of the engineering responsibilities.
625.88s - 628.88s |  And I think this is one of the biggest challenges
628.88s - 631.88s |  to understand that infrastructure has moved
631.88s - 634.88s |  and is being maintained in another part of the house.
634.88s - 639.88s |  Now, while this is challenging from security point of view,
639.88s - 642.88s |  it brings a lot of opportunities and speed
642.88s - 644.88s |  into the development hands, right?
644.88s - 647.88s |  If you build your new terraform, cloud formation,
647.88s - 649.88s |  base images yourself in the team
649.88s - 652.88s |  and you have good practices to handle them,
652.88s - 654.88s |  you don't need to wait for infrastructure guys
654.88s - 657.88s |  to upgrade your VMs, upgrade your base images, etc.
657.88s - 659.88s |  You control that.
659.88s - 663.88s |  So, you know, as they say, you need brakes in your car
663.88s - 666.88s |  to run faster, right?
666.88s - 669.88s |  Because without having brakes, you wouldn't go that fast.
669.88s - 672.88s |  The same with security and infrastructure, in my opinion.
672.88s - 674.88s |  As long as you have the right brakes,
674.88s - 677.88s |  you can go as fast as you want with engineering,
677.88s - 680.88s |  building and upgrading the artifacts
680.88s - 682.88s |  also for the infrastructure part.
682.88s - 684.88s |  So these are the two things that are,
684.88s - 687.88s |  two insights coming to mind when talking about it.
687.88s - 689.88s |  Yeah, thanks, Shahar and Dhrupti.
689.88s - 692.88s |  Since you talked about the developer part,
692.88s - 695.88s |  one question that comes to mind is basically,
695.88s - 698.88s |  how do you guys see the developer education
698.88s - 700.88s |  in terms of application security?
701.88s - 704.88s |  Because coming from a development background,
704.88s - 707.88s |  I feel it gives a different perspective
707.88s - 710.88s |  when you can actually see things from a developer perspective
710.88s - 712.88s |  and then protect it.
712.88s - 715.88s |  But at the same time, there's also emphasis
715.88s - 718.88s |  that needs to be laid on developer education.
718.88s - 721.88s |  And when we talk about everything as code,
721.88s - 723.88s |  things like ISE,
723.88s - 725.88s |  I think, Shahar, you touched upon it a little bit,
725.88s - 727.88s |  saying that you just throw it over the fence
727.88s - 729.88s |  to operations previously.
729.88s - 733.88s |  I've still seen that happen with ISE code.
733.88s - 735.88s |  Oh, this is just a template.
735.88s - 737.88s |  It's not really code.
737.88s - 741.88s |  And I've seen security engineers or leaders fight saying,
741.88s - 742.88s |  it is code.
742.88s - 745.88s |  And how do you guys see that in that context?
745.88s - 748.88s |  Yeah, maybe I hope we're not going to start a fight
748.88s - 750.88s |  between Dhrupti and myself,
750.88s - 752.88s |  but we see things very differently.
752.88s - 753.88s |  So from my perspective,
753.88s - 756.88s |  leading engineering team for the last 20 years,
756.88s - 761.88s |  I think the exercise of educating developers,
761.88s - 765.88s |  being more security aware, kind of failed.
765.88s - 767.88s |  I mean, let's face it.
767.88s - 771.88s |  Developers, they don't care about security for good reasons.
771.88s - 774.88s |  They're passionate about developing features,
774.88s - 775.88s |  about going fast,
775.88s - 780.88s |  about the functionality that they bring to the world,
780.88s - 781.88s |  and they get paid for that.
781.88s - 784.88s |  So it's always a burden or a hustle for them
784.88s - 787.88s |  to treat security in the right way.
787.88s - 789.88s |  We expect them to do that, right?
789.88s - 791.88s |  So, and not only security.
791.88s - 794.88s |  When we talk to developers these days,
794.88s - 797.88s |  they feel like everything is being shifted left.
797.88s - 799.88s |  There's no QA paradigm anymore.
799.88s - 801.88s |  They need to do it themselves.
801.88s - 802.88s |  They need to do security.
802.88s - 804.88s |  They need to do documentation of the code.
804.88s - 806.88s |  So out of their, you know,
806.88s - 810.88s |  100% of writing code and creating functionality,
810.88s - 812.88s |  they need to spend a lot of time on that.
812.88s - 813.88s |  This is a challenge.
813.88s - 818.88s |  So I think shifting left might have gone too much,
818.88s - 824.88s |  expecting developers to create security on their part.
824.88s - 825.88s |  I don't think that's fair.
825.88s - 827.88s |  I think the gates may have shifted left,
827.88s - 831.88s |  and that's very right to put it as part of the CICD, et cetera.
831.88s - 835.88s |  But the wisdom and the expertise of doing the right policies,
835.88s - 836.88s |  the right security,
836.88s - 838.88s |  it's still with the AppSec team,
838.88s - 840.88s |  with the DevSecOps guys.
840.88s - 844.88s |  It's a profession to be an application security person.
844.88s - 847.88s |  This is why we've chosen to be in that profession.
847.88s - 851.88s |  And if you're able in a program
851.88s - 855.88s |  to get the wisdom of security into the policies
855.88s - 860.88s |  and let alone the developers to work under these policies,
860.88s - 861.88s |  that's the balanced way,
861.88s - 863.88s |  not just giving them more and more education
863.88s - 867.88s |  about SQL injection, past reversal, you name it.
867.88s - 868.88s |  I mean, they got it.
869.88s - 873.88s |  Second point is that you can educate 99% of your developers,
873.88s - 877.88s |  but your security program is only as strong
877.88s - 879.88s |  as your weakest developer, right?
879.88s - 881.88s |  So you can have one new guy,
881.88s - 883.88s |  a junior guy who joined the development team,
883.88s - 884.88s |  just added a code.
884.88s - 886.88s |  He didn't know exactly how to do that.
886.88s - 888.88s |  He was not educated yet,
888.88s - 891.88s |  but you've compromised your entire production
891.88s - 893.88s |  because of this particular code.
893.88s - 895.88s |  So this is where I am right now
895.88s - 897.88s |  with my thinking around development
897.88s - 899.88s |  and AppSec responsibility.
899.88s - 902.88s |  And I'll let Drupty very nicely contradict me.
906.58s - 909.58s |  So Shahar, I do understand your concerns
909.58s - 912.58s |  about shifting security left completely,
912.58s - 914.58s |  but let me ask the audience,
914.58s - 918.58s |  how many of you think we can scale product security,
918.58s - 920.58s |  application security without developer?
920.58s - 922.58s |  Please raise your hand.
923.58s - 925.58s |  Not a single hand.
926.58s - 929.58s |  So thank you, audience, for being with me.
931.58s - 933.58s |  Okay, all right.
935.58s - 937.58s |  So, you know, when we provide
937.58s - 940.58s |  our developers generic security trainings,
940.58s - 942.58s |  obviously they are not going to get, you know,
942.58s - 945.58s |  full knowledge that is needed
945.58s - 948.58s |  to take those security decisions.
948.58s - 951.58s |  So the security champions program that I have designed,
951.58s - 953.58s |  I always make sure,
953.58s - 955.58s |  depending on the developer's role,
955.58s - 958.58s |  the kind of technology they are using,
958.58s - 962.58s |  provide that customized contextual training to them
962.58s - 964.58s |  so that they understand the attacks
964.58s - 966.58s |  which are related to their technology,
966.58s - 969.58s |  their application, their SDLC,
969.58s - 973.58s |  and they can understand the actionable remediation guidance.
973.58s - 975.58s |  What I have seen in my last 10 years
975.58s - 978.58s |  of running security champions program,
978.58s - 980.58s |  when we provide that level of customized
980.58s - 983.58s |  targeted knowledge to our developers,
983.58s - 986.58s |  they exactly know how to solve the problem,
986.58s - 989.58s |  how to identify security problem and fix it right away.
989.58s - 993.58s |  And sometimes developers are so smart,
993.58s - 996.58s |  they have identified deeper level of issues
996.58s - 998.58s |  that product security engineers have not.
998.58s - 1000.58s |  Because product security engineers
1000.58s - 1002.58s |  go from application to application,
1002.58s - 1004.58s |  one system to another system,
1004.58s - 1008.58s |  they rarely develop that deeper level of context.
1008.58s - 1011.58s |  So I have seen very successful cases.
1011.58s - 1015.58s |  One of the thing that my good friend Sarah brought up,
1015.58s - 1017.58s |  if you want to have security checks
1017.58s - 1020.58s |  before every PR merge,
1020.58s - 1022.58s |  can we have security engineers do that?
1022.58s - 1023.58s |  Absolutely not.
1023.58s - 1026.58s |  In this way, security becomes the bottlenecks.
1026.58s - 1028.58s |  We can't match the speed of innovation.
1028.58s - 1033.58s |  So I really think if product security invest enough time
1033.58s - 1035.58s |  building and delivering
1035.58s - 1037.58s |  those contextual trainings to our developer,
1037.58s - 1041.58s |  they absolutely love and welcome this level of knowledge.
1041.58s - 1043.58s |  And last thing,
1043.58s - 1045.58s |  it's not the developers that we need to target,
1045.58s - 1048.58s |  we also need to educate our product managers
1048.58s - 1050.58s |  on how to match business requirement
1050.58s - 1052.58s |  with security requirement.
1052.58s - 1055.58s |  We also need to educate our TPMs,
1055.58s - 1056.58s |  engineering managers,
1056.58s - 1059.58s |  so that they know how to prioritize security backlog
1059.58s - 1062.58s |  and systematically reduce risk.
1062.58s - 1069.34s |  I would say that I agree with both of you
1069.34s - 1070.34s |  in different ways.
1071.34s - 1073.34s |  Realistically, yes.
1073.34s - 1075.34s |  I don't think that you're going to have...
1075.34s - 1077.34s |  I don't think your dev team is going to be very happy
1077.34s - 1080.34s |  if your team of five application security engineers
1080.34s - 1082.34s |  are the bottleneck for six to 12 hours
1082.34s - 1084.34s |  waiting for a PR to be merged
1084.34s - 1086.34s |  because it's in the pipeline,
1086.34s - 1087.34s |  there's a gate check,
1087.34s - 1090.34s |  and they either don't have the context
1090.34s - 1091.34s |  to figure it out themselves
1091.34s - 1095.34s |  or they need a validation or an exception,
1095.34s - 1096.34s |  and they're just waiting.
1096.34s - 1097.34s |  That's not realistic.
1098.34s - 1099.34s |  And so to that regard,
1099.34s - 1100.34s |  I agree with you.
1100.34s - 1102.34s |  I think it cannot just be
1102.34s - 1103.34s |  application security engineers
1103.34s - 1105.34s |  and vulnerability management engineers
1105.34s - 1108.34s |  that are performing these tasks
1108.34s - 1111.34s |  and doing the validation.
1111.34s - 1113.34s |  However, I do also think that
1113.34s - 1114.34s |  on the training side,
1114.34s - 1116.34s |  on effectiveness of training,
1116.34s - 1119.34s |  I think realistically in the future,
1119.34s - 1120.34s |  I wouldn't be surprised
1120.34s - 1123.34s |  if most formal structured training
1123.34s - 1125.34s |  was really minimized
1125.34s - 1127.34s |  while constant learning
1127.34s - 1129.34s |  through things like co-pilot, et cetera,
1129.34s - 1131.34s |  were much more relevant
1131.34s - 1133.34s |  and those reminders in the IDE
1133.34s - 1134.34s |  and your linters
1134.34s - 1137.34s |  were used as the primary source of training
1137.34s - 1141.34s |  on the fly with your actual application code.
1141.34s - 1144.34s |  Yeah, training is definitely a very key component.
1144.34s - 1145.34s |  I remember one of the organizations
1145.34s - 1146.34s |  that I was at,
1146.34s - 1150.34s |  we tried to gamify the whole training.
1150.34s - 1152.34s |  We put together something
1152.34s - 1154.34s |  on the theme of Game of Thrones
1155.34s - 1156.34s |  so we had night commanders
1156.34s - 1158.34s |  and all of that,
1158.34s - 1160.34s |  which went pretty cool.
1160.34s - 1162.34s |  So it really depends upon
1162.34s - 1164.34s |  the organization culture.
1164.34s - 1165.34s |  But one other thing
1165.34s - 1167.34s |  that I would like to say is
1167.34s - 1168.34s |  when you talk about shift left,
1168.34s - 1169.34s |  that's overtly used
1169.34s - 1171.34s |  and sometimes abused.
1171.34s - 1173.34s |  But Shahar, what you were talking about
1173.34s - 1175.34s |  is basically putting it towards the right.
1175.34s - 1176.34s |  I think it's all a cycle.
1176.34s - 1179.34s |  So sometimes you need to shift right
1179.34s - 1180.34s |  to actually shift left.
1180.34s - 1182.34s |  I know it sounds counterproductive,
1182.34s - 1183.34s |  but this is the trend
1183.34s - 1185.34s |  that I've been seeing for some time
1185.34s - 1188.34s |  because let's say you're starting off a program
1188.34s - 1190.34s |  at the very early phases.
1190.34s - 1191.34s |  You want to stop the bleed.
1191.34s - 1192.34s |  So in that case,
1192.34s - 1194.34s |  you want to put things to the right
1194.34s - 1196.34s |  so that you're able to protect your crown jewels
1196.34s - 1199.34s |  and then gradually try to find it earlier
1199.34s - 1200.34s |  in the cycle.
1200.34s - 1203.34s |  So I think both are right to a certain extent.
1203.34s - 1205.34s |  Yeah, I have a thought.
1205.34s - 1208.34s |  After hearing to all the panelists
1208.34s - 1209.34s |  and moderator,
1209.34s - 1211.34s |  I genuinely think that we need
1211.34s - 1213.34s |  to strengthen our right
1213.34s - 1215.34s |  while shifting left like a boss
1215.34s - 1218.34s |  and have a really strong governance program
1218.34s - 1221.34s |  so that even if our awesome developers
1221.34s - 1224.34s |  who are our unsung security warriors
1224.34s - 1227.34s |  miss on certain vulnerabilities or fixes,
1227.34s - 1229.34s |  we should be able to catch it.
1229.34s - 1230.34s |  So yeah, I'm going to reiterate,
1230.34s - 1232.34s |  let's strengthen our right
1232.34s - 1235.34s |  while shifting left like a boss.
1237.34s - 1238.34s |  Thanks, Shruti.
1238.34s - 1240.34s |  I know you're very passionate about this.
1240.34s - 1242.34s |  Moving on,
1242.34s - 1244.34s |  how can organizations leverage,
1244.34s - 1245.34s |  we talked about the pipeline, right?
1245.34s - 1247.34s |  How can organizations leverage automation
1247.34s - 1250.34s |  and a CI-CD pipeline,
1250.34s - 1253.34s |  especially in the context of everything as tech,
1253.34s - 1256.34s |  things like ISE and GitOps and things?
1258.34s - 1259.34s |  Yeah, maybe I can start.
1259.34s - 1260.34s |  So as mentioned,
1260.34s - 1265.34s |  I think that the trend of having more visibility
1265.34s - 1266.34s |  in the pipeline
1266.34s - 1269.34s |  to what's going on in terms of security
1269.34s - 1270.34s |  is welcome.
1270.34s - 1273.34s |  And I didn't see many brave organization
1273.34s - 1274.34s |  that actually forces
1274.34s - 1278.34s |  and stops the build for the developers
1278.34s - 1279.34s |  just for having them,
1279.34s - 1281.34s |  you know, fixing their security stuff
1281.34s - 1283.34s |  because there are just too many of those.
1283.34s - 1284.34s |  You would allow that
1284.34s - 1286.34s |  if you have five or four in the CI-CD,
1286.34s - 1288.34s |  okay, we'll stop, we understand.
1288.34s - 1291.34s |  But then if their request is to stop your build
1291.34s - 1295.34s |  and go ahead, fix your 1500 vulnerability issues,
1295.34s - 1297.34s |  I just wanted to add another button
1297.34s - 1298.34s |  to the application.
1298.34s - 1299.34s |  That's not going to work.
1299.34s - 1303.34s |  And, you know, sometimes we hear the dynamic
1303.34s - 1305.34s |  that application security teams ask
1305.34s - 1307.34s |  to prioritize the issues.
1307.34s - 1308.34s |  And you know what?
1308.34s - 1310.34s |  Let's just prioritize them.
1310.34s - 1313.34s |  Now, here's the thing with priorities.
1313.34s - 1317.34s |  That's sometimes humanly impossible.
1317.34s - 1320.34s |  Just think about a list of 10 items
1320.34s - 1323.34s |  that you would like to prioritize.
1323.34s - 1324.34s |  That's a small list.
1324.34s - 1325.34s |  It seems manageable.
1325.34s - 1326.34s |  We do it every day.
1326.34s - 1328.34s |  But if you want to do it right,
1328.34s - 1330.34s |  there are more than 3 million options
1330.34s - 1334.34s |  to prioritize a 10 items list, right?
1334.34s - 1336.34s |  That's the numbers.
1336.34s - 1337.34s |  That's the math behind it.
1337.34s - 1338.34s |  So just imagine what happens
1338.34s - 1341.34s |  when you look at a thousand items list
1341.34s - 1343.34s |  and you ask them to prioritize.
1343.34s - 1345.34s |  So we've all settled
1345.34s - 1350.34s |  on a stateless generic prioritization system
1350.34s - 1353.34s |  like CVSS, EPSS from the outside.
1353.34s - 1354.34s |  Okay, you know what?
1354.34s - 1355.34s |  Let's just do the critical.
1355.34s - 1357.34s |  But then the frustration is that
1357.34s - 1358.34s |  there's no real correlation
1358.34s - 1361.34s |  between my intimate understanding
1361.34s - 1363.34s |  of the risk of the application
1363.34s - 1365.34s |  and the CVSS scoring outside.
1365.34s - 1366.34s |  So here's another triage
1366.34s - 1369.34s |  that needs to go for each one of them.
1369.34s - 1371.34s |  So it all comes down to the numbers
1371.34s - 1373.34s |  that you need to handle.
1373.34s - 1376.34s |  You cannot just move everything left
1376.34s - 1377.34s |  if you have not cleaned up
1377.34s - 1379.34s |  with the right policies.
1379.34s - 1380.34s |  What do you want to fix
1380.34s - 1381.34s |  that it's actionable
1381.34s - 1382.34s |  for the developer teams?
1382.34s - 1384.34s |  I think that's the right balance.
1384.34s - 1385.34s |  If you figure that out,
1385.34s - 1387.34s |  this will be a working,
1387.34s - 1390.34s |  a successful shift left program.
1390.34s - 1392.82s |  Thank you, Shahar,
1392.82s - 1395.82s |  for bringing up the prioritization part.
1395.82s - 1398.82s |  I want to focus on two other things.
1398.82s - 1400.82s |  Before automating your CICD,
1400.82s - 1402.82s |  make your SREs and DevOps folks
1402.82s - 1404.82s |  your best friend.
1404.82s - 1408.82s |  Try to get inventory of your CICD.
1408.82s - 1411.82s |  Not all organization have unified CICD,
1411.82s - 1413.82s |  especially organizations
1413.82s - 1415.82s |  which have legacy systems
1415.82s - 1418.82s |  or organizations which are very heavy
1418.82s - 1420.82s |  on merger acquisition.
1420.82s - 1423.82s |  So we may have multiple flavors of CICD.
1423.82s - 1426.82s |  So get the asset inventory first.
1426.82s - 1427.82s |  Try to figure out
1427.82s - 1429.82s |  how does your CICD look like.
1429.82s - 1431.82s |  And then second thing is
1431.82s - 1433.82s |  many CICD pipelines
1433.82s - 1436.82s |  have some inherent native security controls.
1436.82s - 1437.82s |  For example,
1437.82s - 1439.82s |  GitHub has branch protection.
1439.82s - 1442.82s |  So definitely leverage those
1442.82s - 1445.82s |  before buying another third-party commercial tool.
1445.82s - 1447.82s |  Sarah?
1447.82s - 1449.82s |  Yeah.
1449.82s - 1453.82s |  I would say that I think generally
1453.82s - 1455.82s |  there is so much that CICD
1455.82s - 1457.82s |  and policy as code
1457.82s - 1458.82s |  and infrastructure as code
1458.82s - 1460.82s |  has really opened up the door
1460.82s - 1464.82s |  for infrastructure security specifically.
1464.82s - 1466.82s |  Immutability wasn't even like a topic
1466.82s - 1468.82s |  that was possible before that.
1468.82s - 1470.82s |  And so it's really awesome
1470.82s - 1472.82s |  to see the sustainability
1472.82s - 1475.82s |  that is kind of embedded in that
1475.82s - 1478.82s |  and how you use infrastructure as code
1478.82s - 1481.82s |  and everything as code in your pipelines.
1481.82s - 1483.82s |  I think I would also agree
1483.82s - 1485.82s |  that it's hard.
1485.82s - 1486.82s |  It's complex.
1486.82s - 1487.82s |  The cloud is complex.
1487.82s - 1489.82s |  It's going to take a lot of work
1489.82s - 1491.82s |  to figure out the right set of rules.
1491.82s - 1494.82s |  You have to be really intentional about that.
1494.82s - 1496.82s |  You have to probably roll it out pretty slowly.
1496.82s - 1498.82s |  You have to think about exceptions
1498.82s - 1500.82s |  and how you're going to scale exceptions.
1500.82s - 1502.82s |  It's probably not going to be
1502.82s - 1505.82s |  by individuals, people responding to exceptions.
1505.82s - 1507.82s |  So how are you going to do that at scale?
1507.82s - 1511.82s |  And that's a huge topic in enterprise security
1511.82s - 1513.82s |  and application security.
1513.82s - 1515.82s |  Yeah, totally makes sense.
1515.82s - 1518.82s |  Kind of segwaying into something
1518.82s - 1520.82s |  which is what you talked about, Sarah,
1520.82s - 1522.82s |  and I think, Shahar, you talked about earlier as well,
1522.82s - 1525.82s |  which is around ESE.
1525.82s - 1529.82s |  And in the context of the lines getting blurred
1529.82s - 1531.82s |  with application code, infrastructure code,
1531.82s - 1534.82s |  everything as code,
1534.82s - 1537.82s |  where does vulnerability management really fit in?
1537.82s - 1540.82s |  Is the concept of unified vulnerability management
1540.82s - 1542.82s |  something that would really work?
1542.82s - 1544.82s |  Because I worked in organizations
1544.82s - 1546.82s |  where you have different SLAs
1546.82s - 1548.82s |  for application-level vulnerabilities
1548.82s - 1551.82s |  and infrastructure-level vulnerabilities.
1551.82s - 1554.82s |  Does this concept of unified vulnerability management
1554.82s - 1556.82s |  really resonate in today's world
1556.82s - 1558.82s |  with everything becoming code?
1558.82s - 1560.82s |  Sarah?
1560.82s - 1563.82s |  No, yes, maybe, probably.
1563.82s - 1567.82s |  I think that's a really hard question to answer.
1567.82s - 1570.82s |  I think, in general,
1570.82s - 1571.82s |  you have to know what you mean
1571.82s - 1574.82s |  by unified vulnerability management.
1574.82s - 1577.82s |  Are we talking about unifying the tooling sets
1577.82s - 1579.82s |  of what's performing assessments?
1579.82s - 1583.82s |  Or are we talking about unifying data and reporting?
1583.82s - 1585.82s |  Those are two separate concepts entirely.
1585.82s - 1588.82s |  And I think, with tooling,
1588.82s - 1592.82s |  it really depends on your business's structure
1592.82s - 1596.82s |  and requirements, to your point, Tripti.
1596.82s - 1597.82s |  And I think, a lot of times,
1597.82s - 1599.82s |  that's going to be a lot harder,
1599.82s - 1604.82s |  especially as you grow in size of your company.
1604.82s - 1606.82s |  I think it's going to be a lot harder to optimize that,
1606.82s - 1610.82s |  but I think it's a worthwhile endeavor for a lot of cases
1610.82s - 1612.82s |  because the complexity of all these scans
1612.82s - 1615.82s |  becomes very, very nebulous.
1615.82s - 1617.82s |  But what's probably more realistic
1617.82s - 1620.82s |  in terms of unification is unification of data.
1620.82s - 1623.82s |  I mean, ultimately, a lot of this stuff
1623.82s - 1625.82s |  is the same conceptually.
1625.82s - 1627.82s |  There is a thing that we have found,
1627.82s - 1629.82s |  and it needs to be fixed,
1629.82s - 1631.82s |  and it needs to be fixed at this time frame,
1631.82s - 1633.82s |  and it needs to be fixed by this person.
1633.82s - 1637.82s |  And that is something that can be unified.
1637.82s - 1639.82s |  But I did talk about some of the data challenges
1639.82s - 1641.82s |  embedded into that,
1641.82s - 1644.82s |  but that's where I would kind of lean into.
1644.82s - 1647.82s |  Just to provide a little bit more color to the question,
1647.82s - 1649.82s |  when I talk about unified volume management,
1649.82s - 1651.82s |  definitely data is one aspect.
1651.82s - 1653.82s |  But when I say volume management,
1653.82s - 1656.82s |  most organizations are policy-driven.
1656.82s - 1661.82s |  So you have a policy of saying that any issue,
1661.82s - 1664.82s |  any vulnerability needs to be fixed in,
1664.82s - 1667.82s |  let's say, one day or two days for critical,
1667.82s - 1672.82s |  five days for high and medium, so on and so forth.
1672.82s - 1676.82s |  So my question was mainly in the context of,
1676.82s - 1678.82s |  do you have the same SLAs?
1678.82s - 1680.82s |  And if so, can you put the same logic?
1680.82s - 1683.82s |  So, for example, the difference that I would see
1683.82s - 1687.82s |  in infrastructure-level vulnerabilities would be,
1687.82s - 1689.82s |  you just need to upgrade.
1689.82s - 1691.82s |  But there could be thousands of upgrades
1691.82s - 1692.82s |  that you would need to do,
1692.82s - 1694.82s |  which is where the automation comes into play,
1694.82s - 1696.82s |  versus, let's say, on the application level,
1696.82s - 1698.82s |  you actually need a design fix.
1698.82s - 1701.82s |  So does it really make sense to unify it?
1701.82s - 1702.82s |  So that's the...
1705.82s - 1707.82s |  No.
1707.82s - 1711.82s |  I don't think total unification of SLAs and activities
1711.82s - 1712.82s |  makes sense.
1712.82s - 1715.82s |  I think there are certain things
1715.82s - 1718.82s |  that you can potentially unify around that,
1718.82s - 1722.82s |  but perhaps things that you're finding in code
1722.82s - 1724.82s |  before it goes into an image
1726.82s - 1728.82s |  might have a different risk
1728.82s - 1731.82s |  compared to things that are already actively running
1731.82s - 1732.82s |  on a container image,
1732.82s - 1735.82s |  and you probably need to prioritize those differently,
1735.82s - 1736.82s |  for sure.
1736.82s - 1738.82s |  So I think there's a lot of, again, complex rule sets
1738.82s - 1740.82s |  that have to be embedded,
1740.82s - 1743.82s |  and your teams are going to have to sit down
1743.82s - 1746.82s |  and identify what those rule sets really look like
1746.82s - 1749.82s |  in the context of your business's risk tolerance.
1750.82s - 1752.82s |  Yeah, I agree with that.
1752.82s - 1754.82s |  Many times, organization,
1754.82s - 1757.82s |  for the sake of compliance teams and GRC teams,
1757.82s - 1759.82s |  if there are any GRC teams here,
1759.82s - 1761.82s |  please don't hate me.
1761.82s - 1765.82s |  So, yeah, for the sake of convenience,
1765.82s - 1767.82s |  people like to have same SLAs
1767.82s - 1769.82s |  for application security vulnerabilities
1769.82s - 1772.82s |  as well as infrastructure security vulnerability,
1772.82s - 1776.82s |  but if you look at the remediation phases,
1776.82s - 1778.82s |  application security vulnerability
1778.82s - 1780.82s |  sometimes are very hard to fix,
1780.82s - 1782.82s |  especially if it's going to require
1782.82s - 1784.82s |  design and architecture change,
1784.82s - 1785.82s |  and at that time,
1785.82s - 1788.82s |  we really have to be with our developers on their side
1788.82s - 1790.82s |  defending, you know,
1790.82s - 1793.82s |  why it's going to take four months or five months
1793.82s - 1794.82s |  and what it takes,
1794.82s - 1795.82s |  because oftentimes,
1795.82s - 1797.82s |  those who have never written code
1797.82s - 1799.82s |  or never done any development,
1799.82s - 1802.82s |  they don't understand what it takes to fix things, right?
1802.82s - 1804.82s |  So I genuinely think that we should have
1804.82s - 1806.82s |  different set of SLA
1806.82s - 1809.82s |  depending on the remediation efforts required.
1809.82s - 1813.26s |  Yeah, maybe I'll just add to that shortly.
1813.26s - 1816.26s |  I think from management perspective,
1816.26s - 1818.26s |  they really like to have one, you know,
1818.26s - 1819.26s |  single pane of glass
1819.26s - 1821.26s |  on a certain business unit or product,
1821.26s - 1822.26s |  but they can see,
1822.26s - 1825.26s |  they don't care if you have 500 or 300 issues,
1825.26s - 1826.26s |  they want to see the trend.
1826.26s - 1828.26s |  You used to have 500,
1828.26s - 1829.26s |  one quarter later,
1829.26s - 1831.26s |  you have 200 or 300,
1831.26s - 1834.26s |  so it's nice to capture them together in one tool,
1834.26s - 1836.26s |  but from pragmatic approach,
1836.26s - 1838.26s |  I think these are all different beasts.
1838.26s - 1839.26s |  If you find, you know,
1839.26s - 1842.26s |  a virtual machine you need to fix
1842.26s - 1843.26s |  or to patch
1843.26s - 1845.26s |  or a container-based image
1845.26s - 1847.26s |  or a vulnerability in the code
1847.26s - 1849.26s |  where you just need to clean up a parameter
1849.26s - 1851.26s |  and that could be very quickly,
1851.26s - 1852.26s |  it really needs to be aggregated
1852.26s - 1854.26s |  from tool set perspective
1854.26s - 1857.26s |  where the actionable item happens, right?
1857.26s - 1860.26s |  So developers should have their own view
1860.26s - 1861.26s |  on what's fixable for them.
1861.26s - 1862.26s |  If they cannot do anything
1862.26s - 1865.26s |  about patching a certain package
1865.26s - 1866.26s |  from the infrastructure,
1866.26s - 1868.26s |  why should they care about it?
1868.26s - 1870.26s |  Why should they even handle that?
1870.26s - 1872.26s |  So reporting is important,
1872.26s - 1873.26s |  but where the data sits,
1873.26s - 1878.26s |  it really depends how you structure your program.
1878.26s - 1881.26s |  And you talked about context being king, right?
1881.26s - 1883.26s |  So I think context also matters a lot,
1883.26s - 1886.26s |  especially when you talk about,
1886.26s - 1887.26s |  you know, for example,
1887.26s - 1890.26s |  a library which needs to be upgraded
1890.26s - 1893.26s |  that would technically be under an SCA.
1893.26s - 1896.26s |  And if that library is actually being utilized
1896.26s - 1898.26s |  or a function in the library is being utilized,
1898.26s - 1900.26s |  would you really want to patch it if it's not?
1900.26s - 1903.26s |  I mean, that's the challenge that we run into, right?
1903.26s - 1905.26s |  A lot of times.
1905.26s - 1907.26s |  Thanks, Shah.
1907.26s - 1909.26s |  So going to the next steps,
1909.26s - 1912.26s |  which is around AI, as I promised,
1912.26s - 1914.26s |  do you see AI,
1914.26s - 1916.26s |  and this is a very generic kind of question
1916.26s - 1919.26s |  when we talk about for the future, right?
1919.26s - 1922.26s |  We have been seeing AppSec being in the same state
1922.26s - 1926.26s |  of morasses for maybe 20 years now,
1926.26s - 1928.26s |  where we have SCA tools,
1928.26s - 1930.26s |  we have SAS tools,
1930.26s - 1933.26s |  which are kind of struggling
1933.26s - 1935.26s |  because there's a lot of context,
1935.26s - 1938.26s |  which is really hard to get.
1938.26s - 1941.26s |  Do we see AI as a panacea,
1941.26s - 1942.26s |  which will solve these problems
1942.26s - 1946.26s |  and bring us towards this glorious green future?
1946.26s - 1948.26s |  Yeah, maybe I'll start.
1948.26s - 1950.26s |  So in a nutshell,
1950.26s - 1953.26s |  I think we're going into as an industry
1953.26s - 1957.26s |  to the AI era with our eyes wide open.
1957.26s - 1959.26s |  I mean, we've been there before
1959.26s - 1960.26s |  with the machine learning trends,
1960.26s - 1962.26s |  this will save everything, right?
1962.26s - 1965.26s |  And now every company would put an AI sticker
1965.26s - 1968.26s |  on what they can do.
1968.26s - 1971.26s |  However, I think there are very specific use cases
1971.26s - 1972.26s |  that AI can help us.
1972.26s - 1975.26s |  It's not just a generic behind the scenes
1975.26s - 1977.26s |  under the hood functionality,
1977.26s - 1978.26s |  it's very specific.
1978.26s - 1979.26s |  I mean, let's face it,
1979.26s - 1982.26s |  in the last two decades,
1982.26s - 1984.26s |  the industry with all the money
1984.26s - 1987.26s |  put into the industry of application security,
1987.26s - 1988.26s |  vendors, et cetera,
1988.26s - 1992.26s |  kind of failed fixing the same old things.
1992.26s - 1996.26s |  If you look at the top trending vulnerabilities right now,
1996.26s - 1997.26s |  same thing.
1997.26s - 2000.26s |  I mean, SQL injection, number three,
2000.26s - 2002.26s |  out of bound memory,
2002.26s - 2004.26s |  RAID is one,
2004.26s - 2005.26s |  sorry, RAID is one,
2005.26s - 2007.26s |  RAID is seven.
2007.26s - 2008.26s |  This is the ranking, right?
2008.26s - 2010.26s |  We've seen CrowdStrikes,
2010.26s - 2012.26s |  out of bound recently,
2012.26s - 2014.26s |  past reversal, still trending.
2014.26s - 2015.26s |  With all the tools that we have,
2015.26s - 2016.26s |  that's the same thing,
2016.26s - 2018.26s |  we cannot get rid of them as an industry.
2018.26s - 2021.26s |  So I think something fundamental should change
2021.26s - 2023.26s |  with our expectation from code writing
2023.26s - 2025.26s |  in that perspective.
2025.26s - 2027.26s |  All the recent attacks,
2027.26s - 2029.26s |  same holes in the application,
2029.26s - 2031.26s |  everything stays the same.
2031.26s - 2032.26s |  I think there,
2032.26s - 2034.26s |  before we even talk about AI,
2034.26s - 2036.26s |  and I'll touch it in a second,
2036.26s - 2038.26s |  there are so many things that we can do right now
2038.26s - 2041.26s |  in order to improve the current situation
2041.26s - 2042.26s |  because many people,
2042.26s - 2043.26s |  many CISOs,
2043.26s - 2046.26s |  head of app security we talked to,
2046.26s - 2049.26s |  kind of settled with the current situation.
2049.26s - 2051.26s |  So 90% of my code is coming
2051.26s - 2054.26s |  probably from open source, right?
2054.26s - 2057.26s |  But just recently the industry realized
2057.26s - 2060.26s |  that out of the 90%,
2060.26s - 2062.26s |  90% of the packages you bring in
2062.26s - 2065.26s |  are not being used by your team, right?
2065.26s - 2067.26s |  And all the scanners you'll go to
2067.26s - 2068.26s |  will tell you
2068.26s - 2070.26s |  you need to fix those vulnerabilities
2070.26s - 2071.26s |  although they're not reachable.
2071.26s - 2073.26s |  So the poor developer,
2073.26s - 2075.26s |  he just brought in a framework
2075.26s - 2077.26s |  in order to use a certain functionality.
2077.26s - 2078.26s |  This framework brought
2078.26s - 2080.26s |  all the transitive dependencies with it,
2080.26s - 2082.26s |  around 100, sometimes more.
2082.26s - 2085.26s |  And now he has tons of vulnerabilities in the code,
2085.26s - 2086.26s |  but no one cares about it
2086.26s - 2088.26s |  because these areas of the code
2088.26s - 2090.26s |  are not reachable, kind of dead.
2090.26s - 2091.26s |  So how about just focusing
2091.26s - 2093.26s |  on what really is being accessed,
2093.26s - 2095.26s |  being approached, et cetera?
2095.26s - 2097.26s |  So these are important problems
2097.26s - 2099.26s |  we should solve as an industry.
2099.26s - 2101.26s |  When it comes to AI,
2101.26s - 2103.26s |  I think we can do
2103.26s - 2104.26s |  a lot of interesting things
2104.26s - 2106.26s |  when it comes to patching
2106.26s - 2108.26s |  and fixing the problems.
2108.26s - 2110.26s |  So no company will allow
2110.26s - 2111.26s |  that any of its code line
2111.26s - 2115.26s |  will go into an AI engine,
2115.26s - 2118.26s |  let's say JetGPD, whatever, LLM,
2118.26s - 2120.26s |  and the code, the IP of the company
2120.26s - 2122.26s |  will be extracted there.
2122.26s - 2123.26s |  But there is a wisdom there
2123.26s - 2125.26s |  that will help you fix and remediate
2125.26s - 2128.26s |  and create automatically PR requests
2128.26s - 2130.26s |  for your developers.
2130.26s - 2131.26s |  And I think we're going to see that
2131.26s - 2133.26s |  coming in the next couple of years.
2133.26s - 2135.26s |  This is kind of obvious use case
2135.26s - 2137.26s |  we can leverage.
2137.26s - 2140.26s |  Subti, you have any thoughts?
2140.26s - 2142.26s |  AI?
2142.26s - 2144.26s |  No.
2144.26s - 2146.26s |  I think AI has been around
2146.26s - 2148.26s |  since 1960s,
2148.26s - 2149.26s |  and we have seen different flavors
2149.26s - 2151.26s |  like deep learning,
2151.26s - 2154.26s |  then the most recent Gen AI.
2154.26s - 2156.26s |  And if you look at
2156.26s - 2158.26s |  some of the security vendors,
2158.26s - 2159.26s |  AI has always been part
2159.26s - 2162.26s |  of their product strategy.
2162.26s - 2164.26s |  They have been using AI
2164.26s - 2165.26s |  to do a better accurate
2165.26s - 2167.26s |  vulnerability correlation.
2167.26s - 2170.26s |  Then we see a lot of Gen AI usage
2170.26s - 2173.26s |  to produce contextual security reports.
2173.26s - 2176.26s |  So I think we should totally use
2176.26s - 2177.26s |  different flavors of AI
2177.26s - 2179.26s |  depending on the use case.
2179.26s - 2182.26s |  What really scares me is
2182.26s - 2184.26s |  allowing our AI agent
2184.26s - 2186.26s |  to do auto remediation.
2186.26s - 2189.26s |  I think that's a nice goal to have.
2189.26s - 2191.26s |  That's a nice state to aspire.
2191.26s - 2192.26s |  But we have to move
2192.26s - 2194.26s |  in that direction very slowly.
2194.26s - 2197.26s |  So let it take three years, five years.
2197.26s - 2200.26s |  Today, I am not ready.
2200.26s - 2202.26s |  That's fair.
2202.26s - 2205.26s |  I try to not be like Rara,
2205.26s - 2207.26s |  to be honest.
2207.26s - 2209.26s |  But I do think, I mean,
2209.26s - 2211.26s |  we already see a lot of people
2211.26s - 2214.26s |  utilizing AI related ID components.
2214.26s - 2216.26s |  And I think those are legitimate.
2216.26s - 2219.26s |  I think the thing about AI
2219.26s - 2221.26s |  and considering like auto fixes
2221.26s - 2223.26s |  to PRs, et cetera,
2223.26s - 2225.26s |  all those things need context.
2225.26s - 2227.26s |  And so if your organization
2227.26s - 2229.26s |  does not have its fundamentals,
2229.26s - 2231.26s |  correct, around asset management,
2231.26s - 2233.26s |  et cetera, your AI
2233.26s - 2235.26s |  is not going to generate
2235.26s - 2238.26s |  anything better than you could
2238.26s - 2239.26s |  or at scale.
2239.26s - 2241.26s |  So that's sort of how it feels.
2241.26s - 2243.26s |  Get your fundamentals correct first.
2243.26s - 2247.86s |  I think not only with the promise of AI
2247.86s - 2249.86s |  we'll get some cool new feature
2249.86s - 2250.86s |  for the security program,
2250.86s - 2252.86s |  it brings a lot of risks
2252.86s - 2254.86s |  into our domain as well, right?
2254.86s - 2256.86s |  I mean, there's a psychological effect
2256.86s - 2259.86s |  of a developer copy paste a code
2259.86s - 2261.86s |  from a, you know,
2261.86s - 2263.86s |  gen AI generated code.
2263.86s - 2264.86s |  It feels like it's safe.
2264.86s - 2265.86s |  It's like it takes it
2265.86s - 2267.86s |  from a very safe place,
2267.86s - 2269.86s |  but no one can assure you
2269.86s - 2270.86s |  this code won't hurt you
2270.86s - 2272.86s |  more than any other piece of code.
2272.86s - 2274.86s |  And the number of lines of code
2274.86s - 2276.86s |  will exponentially grow
2276.86s - 2277.86s |  with what brings into
2277.86s - 2278.86s |  the new application, et cetera.
2278.86s - 2280.86s |  And we need to put the right gates
2280.86s - 2282.86s |  for this AI copy paste,
2282.86s - 2285.86s |  just like when the internet started
2285.86s - 2286.86s |  and copy paste from Google
2286.86s - 2288.86s |  pieces of code were being
2288.86s - 2290.86s |  somehow regulated.
2290.86s - 2291.86s |  You can't really stop that
2291.86s - 2292.86s |  or control that,
2292.86s - 2293.86s |  but you need to have
2293.86s - 2296.86s |  the right policies to handle that.
2296.86s - 2297.86s |  Thanks.
2297.86s - 2298.86s |  Sorry, I just wanted to keep
2298.86s - 2301.86s |  some time for questions.
2301.86s - 2302.86s |  Go ahead.
2302.86s - 2304.86s |  Just one last remark.
2304.86s - 2305.86s |  Like everybody's worried
2305.86s - 2306.86s |  about security risk
2306.86s - 2308.86s |  coming from AI generated code.
2308.86s - 2309.86s |  I'm more worried about
2309.86s - 2312.86s |  the legal risks.
2312.86s - 2314.86s |  That's fair too.
2314.86s - 2315.86s |  Sorry, I want to keep
2315.86s - 2316.86s |  a little bit more time for questions,
2316.86s - 2318.86s |  but I think when you let
2318.86s - 2319.86s |  four AppSec people sit,
2319.86s - 2322.86s |  we can talk for hours.
2323.86s - 2328.58s |  But if anyone has any questions,
2328.58s - 2333.78s |  please use the mic.
2333.78s - 2335.78s |  So my question's for Shahar.
2335.78s - 2336.78s |  What's up?
2336.78s - 2337.78s |  So I'm an electrical engineer
2337.78s - 2339.78s |  and I did software development
2339.78s - 2341.78s |  for Department of Defense
2341.78s - 2343.78s |  application for avionics systems
2343.78s - 2344.78s |  and weapons systems.
2344.78s - 2345.78s |  So our software killed people.
2345.78s - 2346.78s |  We want to make sure
2346.78s - 2347.78s |  it's killed the right people,
2347.78s - 2349.78s |  not the wrong people.
2349.78s - 2352.78s |  When I switched over to IT,
2352.78s - 2354.78s |  totally different level of,
2354.78s - 2356.78s |  and I'm professional,
2356.78s - 2357.78s |  professional developer was
2357.78s - 2360.78s |  much lower bar than what,
2360.78s - 2361.78s |  you know, I'm not dealing
2361.78s - 2362.78s |  with engineers.
2362.78s - 2363.78s |  I'm dealing with sometimes
2363.78s - 2366.78s |  people I would call
2366.78s - 2370.16s |  basically got their coding degree
2370.16s - 2372.16s |  from some, you know,
2372.16s - 2373.16s |  fly-by-night place
2373.16s - 2375.16s |  and stuff and I can program.
2375.16s - 2376.16s |  So my question is,
2376.16s - 2377.16s |  is how do we get
2377.16s - 2378.16s |  on the engineering side?
2378.16s - 2379.16s |  Because to me,
2379.16s - 2380.16s |  software vulnerability
2380.16s - 2382.16s |  is nothing more than a bug.
2382.16s - 2384.16s |  It's a bad coding practice
2384.16s - 2387.16s |  that the developer programmed in.
2387.16s - 2388.16s |  They don't know about it.
2388.16s - 2389.16s |  They don't understand about it,
2389.16s - 2391.16s |  but it's going to cause a problem.
2391.16s - 2393.16s |  So how do we get to a point
2393.16s - 2395.16s |  where our developers more,
2395.16s - 2397.16s |  and I don't mean to mean professionalism
2397.16s - 2398.16s |  or, you know,
2398.16s - 2399.16s |  that they're not acting correctly,
2399.16s - 2401.16s |  but they don't understand
2401.16s - 2403.16s |  the coding practices
2403.16s - 2405.16s |  as well as they should be,
2405.16s - 2406.16s |  in my opinion.
2406.16s - 2407.16s |  I mean, this is all,
2407.16s - 2408.16s |  I'm very biased towards
2408.16s - 2410.16s |  being an engineer and stuff,
2410.16s - 2412.16s |  but at the same time,
2412.16s - 2413.16s |  our IT industry,
2413.16s - 2415.16s |  and we see a lot of breaches
2415.16s - 2416.16s |  and people are getting hurt
2416.16s - 2417.16s |  actually hurt,
2417.16s - 2418.16s |  maybe not killed,
2418.16s - 2421.16s |  although maybe.
2421.16s - 2422.16s |  How do we bring that
2422.16s - 2423.16s |  from an engineering standpoint?
2423.16s - 2424.16s |  Because you're saying,
2424.16s - 2425.16s |  well, we need to maybe not
2425.16s - 2426.16s |  shift left so much.
2426.16s - 2428.16s |  To me, it is the developer
2428.16s - 2430.16s |  who is the core problem
2430.16s - 2432.16s |  and not necessarily
2432.16s - 2433.16s |  the app sec people
2433.16s - 2434.16s |  who are going through
2434.16s - 2435.16s |  and say, hey,
2435.16s - 2436.16s |  you're finding these problems.
2436.16s - 2437.16s |  These are feedback mechanisms
2437.16s - 2438.16s |  that say the developer
2438.16s - 2439.16s |  is not developing
2439.16s - 2442.16s |  to a good practice.
2442.16s - 2443.16s |  So what do you say about that?
2443.16s - 2444.16s |  Yeah, that's very true.
2444.16s - 2446.16s |  So first of all,
2446.16s - 2447.16s |  thanks for your question.
2447.16s - 2449.16s |  And, you know,
2449.16s - 2450.16s |  life and death,
2450.16s - 2452.16s |  it's something that,
2452.16s - 2453.16s |  you know,
2453.16s - 2455.16s |  it's part of a defense system.
2455.16s - 2458.16s |  Me coming from similar areas
2458.16s - 2459.16s |  in my history,
2459.16s - 2460.16s |  I know exactly
2460.16s - 2461.16s |  what you're talking about.
2461.16s - 2462.16s |  But in these domains,
2462.16s - 2464.16s |  probably want to get top developers
2464.16s - 2466.16s |  for writing the code.
2466.16s - 2469.16s |  And I didn't mean that
2469.16s - 2470.16s |  app sec people
2470.16s - 2471.16s |  would go and clean up the mess
2471.16s - 2473.16s |  after the developers.
2473.16s - 2475.16s |  This was not my intention.
2475.16s - 2476.16s |  But if you have
2476.16s - 2478.16s |  certain practices
2478.16s - 2479.16s |  that you want to make sure
2479.16s - 2480.16s |  are happening
2480.16s - 2481.16s |  in your environment,
2481.16s - 2482.16s |  you need to have
2482.16s - 2483.16s |  the right policies
2483.16s - 2484.16s |  that once the code
2484.16s - 2485.16s |  is being compiled,
2485.16s - 2487.16s |  the code is being built,
2487.16s - 2489.16s |  the alerts for these policies
2489.16s - 2490.16s |  will be back
2490.16s - 2491.16s |  to the developers.
2491.16s - 2492.16s |  I'll give you one example,
2492.16s - 2493.16s |  something that we've practiced
2493.16s - 2494.16s |  recently.
2494.16s - 2495.16s |  Some developers,
2495.16s - 2496.16s |  that may look
2496.16s - 2499.16s |  a very innocent thing to do,
2499.16s - 2501.16s |  are referencing packages
2501.16s - 2502.16s |  from third party open sources
2502.16s - 2504.16s |  they bring in directly
2504.16s - 2507.16s |  and not from the manifest file.
2507.16s - 2508.16s |  It's hard to find that
2508.16s - 2509.16s |  and it seems like
2509.16s - 2511.16s |  a very innocent thing to do.
2511.16s - 2512.16s |  However,
2512.16s - 2513.16s |  when you have patched
2513.16s - 2514.16s |  to the latest version
2514.16s - 2515.16s |  that has no vulnerabilities,
2515.16s - 2517.16s |  these ghost packages
2517.16s - 2518.16s |  or phantom packages
2518.16s - 2520.16s |  will be left behind.
2520.16s - 2521.16s |  So now you need to make sure
2521.16s - 2522.16s |  when you're building the software,
2522.16s - 2523.16s |  there's a warning
2523.16s - 2525.16s |  for this particular developer.
2525.16s - 2526.16s |  Hey, David,
2526.16s - 2527.16s |  go ahead and make sure
2527.16s - 2528.16s |  that you fix
2528.16s - 2529.16s |  in this line of code,
2529.16s - 2531.16s |  this ghost package reference
2531.16s - 2532.16s |  and fix it.
2532.16s - 2534.16s |  And not just that generic,
2534.16s - 2535.16s |  the industry settled
2535.16s - 2536.16s |  for a generic explanation.
2536.16s - 2537.16s |  Hey,
2537.16s - 2539.16s |  you may have a SQL injection
2539.16s - 2540.16s |  in your code.
2540.16s - 2542.16s |  It needs to be super specific
2542.16s - 2544.16s |  in how you structure
2544.16s - 2545.16s |  your build processes
2545.16s - 2546.16s |  in order to direct
2546.16s - 2547.16s |  the right person
2547.16s - 2549.16s |  with the right actionable item.
2549.16s - 2550.16s |  And next time,
2550.16s - 2551.16s |  he'll care about it
2551.16s - 2552.16s |  because he sees
2552.16s - 2554.16s |  it's very concrete for him.
2554.16s - 2555.16s |  I hope that helps.
2555.16s - 2556.16s |  Yeah,
2556.16s - 2557.16s |  I'm just going to add
2557.16s - 2558.16s |  one more line.
2558.16s - 2559.16s |  I don't think developers
2559.16s - 2560.16s |  are the problem.
2560.16s - 2561.16s |  The real problem
2561.16s - 2563.16s |  is in university education.
2563.16s - 2564.16s |  We have courses
2564.16s - 2566.16s |  to learn algorithms,
2566.16s - 2567.16s |  different programming languages,
2567.16s - 2568.16s |  databases,
2568.16s - 2569.16s |  but none of those
2569.16s - 2570.16s |  cover security
2570.16s - 2572.16s |  as part of that technology, right?
2572.16s - 2574.16s |  So when that will start happening,
2574.16s - 2575.16s |  we'll see fewer
2575.16s - 2577.16s |  and fewer vulnerabilities.
2577.16s - 2578.16s |  But at the same time,
2578.16s - 2579.16s |  what about the
2579.16s - 2580.16s |  gazillion legacy software
2580.16s - 2582.16s |  which is out there?
2582.16s - 2583.16s |  That's going to take
2583.16s - 2585.16s |  a lot of time.
2585.16s - 2586.16s |  Thanks.
2586.16s - 2587.16s |  I'm getting the
2587.16s - 2588.16s |  two-minute part.
2588.16s - 2590.16s |  But like I said,
2590.16s - 2591.16s |  we can talk forever,
2591.16s - 2592.16s |  but we can,
2592.16s - 2593.16s |  you know,
2593.16s - 2594.16s |  we will be outside
2594.16s - 2595.16s |  and maybe we can talk
2595.16s - 2596.16s |  to people if they're...
2596.16s - 2597.16s |  So Shahar,
2597.16s - 2598.16s |  you had mentioned,
2598.16s - 2599.16s |  you know, along lines
2599.16s - 2600.16s |  our investments
2600.16s - 2601.16s |  into security
2601.16s - 2602.16s |  really haven't worked.
2602.16s - 2603.16s |  We keep on seeing
2603.16s - 2604.16s |  the same faults, right?
2604.16s - 2605.16s |  So in my experience,
2605.16s - 2606.16s |  a lot of times
2606.16s - 2608.16s |  I'm seeing that
2608.16s - 2609.16s |  I think our investments
2609.16s - 2610.16s |  can work,
2610.16s - 2611.16s |  but we have a tendency
2611.16s - 2612.16s |  of promoting people
2612.16s - 2614.16s |  who are top developers.
2614.16s - 2615.16s |  They stop developing
2615.16s - 2616.16s |  because we push them
2616.16s - 2617.16s |  into management.
2617.16s - 2618.16s |  And I think that
2618.16s - 2619.16s |  is one issue.
2619.16s - 2620.16s |  And the other issue
2620.16s - 2621.16s |  I've seen is that
2621.16s - 2623.16s |  once an organization
2623.16s - 2624.16s |  gets to a point
2624.16s - 2625.16s |  where they have
2625.16s - 2626.16s |  very little vulnerabilities,
2626.16s - 2627.16s |  they've basically
2627.16s - 2628.16s |  got rid of a lot
2628.16s - 2629.16s |  of their vulnerability
2629.16s - 2630.16s |  technical debt,
2630.16s - 2631.16s |  they then say,
2631.16s - 2632.16s |  okay, this team,
2632.16s - 2633.16s |  experience team,
2633.16s - 2634.16s |  we don't need you anymore.
2634.16s - 2635.16s |  We're going to outsource
2635.16s - 2636.16s |  to a much lower cost.
2636.16s - 2637.16s |  So it's a cycle
2637.16s - 2638.16s |  that keeps occurring.
2638.16s - 2639.16s |  So I don't think
2639.16s - 2640.16s |  it's the investment.
2640.16s - 2641.16s |  I think it's the lack
2641.16s - 2642.16s |  of continuing investment
2642.16s - 2643.16s |  into security.
2643.16s - 2644.16s |  And when it comes to tools,
2644.16s - 2645.16s |  I love what you said,
2646.16s - 2647.16s |  about putting
2647.16s - 2648.16s |  fandom packages in.
2648.16s - 2649.16s |  I hate that phrase.
2650.16s - 2651.16s |  It drives me absolutely nuts
2651.16s - 2652.16s |  to use dependency management.
2652.16s - 2653.16s |  If your teams aren't using
2653.16s - 2654.16s |  dependency management,
2654.16s - 2655.16s |  shame on you.
2657.16s - 2658.16s |  Yeah, thank you so much.
2658.16s - 2659.16s |  I think that I can
2659.16s - 2660.16s |  very much relate
2660.16s - 2661.16s |  to what you said
2661.16s - 2662.16s |  about promotions.
2662.16s - 2663.16s |  And this is something
2663.16s - 2664.16s |  that I learned
2664.16s - 2665.16s |  with my career
2665.16s - 2666.16s |  building teams.
2666.16s - 2667.16s |  If you have an amazing developer
2667.16s - 2668.16s |  and you take him
2668.16s - 2669.16s |  and be a manager
2669.16s - 2670.16s |  of a team,
2670.16s - 2671.16s |  you lose twice.
2671.16s - 2672.16s |  You just lost
2672.16s - 2673.16s |  your best developer
2673.16s - 2674.16s |  and you got
2674.16s - 2675.16s |  an okay manager
2675.16s - 2676.16s |  right there.
2676.16s - 2678.16s |  So he needs to be empowered,
2678.16s - 2679.16s |  get the right compensation,
2679.16s - 2681.16s |  to continue doing
2681.16s - 2682.16s |  what he loves to do
2682.16s - 2684.16s |  and be a great developer
2684.16s - 2685.16s |  on whatever level
2685.16s - 2686.16s |  he needs to be,
2686.16s - 2688.16s |  but keeps to that profession.
2688.16s - 2690.16s |  That's my point of view.
2690.16s - 2691.16s |  It's an HR problem.
2694.16s - 2695.16s |  Thanks everyone.
2695.16s - 2696.16s |  Thank you.


url: https://www.youtube.com/watch?v=GAEOBhsqJI4
title: DEF COPN 32 - 0 0 0 0 Day Exploiting Localhost APIs From The Browser - Avi Lumel skyGal Elbaz
type: video
author: DEFCONConference
ingestion_date: 2024-11-09
transcription_model: large-v2
chunking_status: completed
content:
{
  "webpage_url": "https://www.youtube.com/watch?v=GAEOBhsqJI4",
  "title": "DEF COPN 32 - 0 0 0 0 Day  Exploiting Localhost APIs From The Browser - Avi Lumel skyGal Elbaz",
  "description": "Browser-based attacks are not new in the malicious landscape of attack patterns. Browsers remain a popular infiltration method for attackers. \n\nWhile seemingly local, services running on localhost are accessible to the browser using a flaw we found, exposing the ports on the localhost network interface, and leaving the floodgates ajar to remote network attacks.\n\nIn this live demo and attack simulation we\u2019ll unveil a zero-day vulnerability (still under responsible disclosure) in Chrome and other browsers, and how we use the 0-day to attack developers behind firewalls. We will demonstrate remote code execution on a wildly popular open-source platform serving millions in the data engineering ecosystem, that seems to run on localhost.\n\nIn our talk, we will present novel attack techniques, targeting developers and employees within an organization, that are behind firewalls. This will be a first-ever deep dive into this newly discovered zero-day vulnerability.",
  "channel_url": "https://www.youtube.com/channel/UC6Om9kAkl32dWlDSNlDS9Iw",
  "duration": 1634,
  "channel": "DEFCONConference",
  "uploader": "DEFCONConference",
  "upload_date": "20241016"
}

0.62s - 2.62s | This text was transcribed using whisper model: large-v2

 Hello everyone
4.82s - 9.02s |  Hi everyone. Thank you for attending our talk zero zero zero day
9.02s - 14.02s |  I know if you've heard or saw the news it got a little bit of coverage
14.02s - 20.38s |  But this is the first time that we're actually disclosing the technical details. It's going to be fun. It's going to be risky and
21.04s - 29.22s |  Let's start. So I'm Gal. I'm the co-founder and CTO of oligo security with me is Avi our number one ninja and
29.22s - 30.74s |  And
30.74s - 33.38s |  Really quickly about ourselves before we start and begin
34.50s - 38.03s |  Hi, I'm Avi
38.75s - 43.51s |  Security researcher in the city office of oligo. I have around 10 years of experience in
44.03s - 48.07s |  research and development mostly researching AI these days and
48.91s - 53.23s |  I love to climb mountains. So if you want to join me on the trip, let me know and
54.03s - 56.03s |  With me in my is my awesome CTO
57.03s - 61.07s |  Thank You Avi I am paying you to say good things about me so
61.63s - 65.87s |  Keep up with that. So I am the founder and city of oligo at 10 years of experience
66.63s - 73.39s |  Checkpoint hacking breaking stuff since I remember myself and not like Avi. I'm famous for being handsome or think
73.99s - 78.67s |  Thinking like that and I'm really good at looking at mountains in my Windows computer
79.07s - 82.47s |  So a quick agenda what we're going to discuss today
82.47s - 87.91s |  What is the concept of port scanning and how it all started from 18 years ago?
87.91s - 94.75s |  We're very old issue that just keep on recurring and recurring along the years and we're going to deep dive into the history
95.11s - 98.51s |  What is the actual private network access this?
99.59s - 101.59s |  try to do
101.75s - 104.47s |  standout when all of the browsers should be
105.11s - 111.11s |  Using but we'll see along the years what actually happened. Then we're going to deep dive into the actual
111.83s - 115.67s |  Details of the zero zero zero day internals itself
115.67s - 123.63s |  And then we're going to demonstrate how we're leveraging that in order to RCE private internal applications that today are considered
124.91s - 131.27s |  Private and in the end we're going to really deep dive into how each and every browser actually
131.75s - 134.75s |  was dealing with that and the responsible disclosure and
135.67s - 139.67s |  Where we are right now with that so in order to really start
139.67s - 145.39s |  I'm gonna pass it on to awesome Avi because he's the actual smart guy that did all of that and
146.39s - 149.72s |  Let's start
149.72s - 151.72s |  Amazing. So, thank you
152.52s - 156.84s |  So client port scanning who remembers this headline on Hacker News
157.92s - 160.68s |  Yeah, so why is this website port scanning me?
161.72s - 163.72s |  Very good question
163.72s - 170.88s |  Apparently eBay not some script kiddie, but a real company try to port scan their visitors as soon as they visit the website
171.28s - 175.60s |  So, how did you do like how did they do it exactly and by the way?
175.72s - 182.56s |  You will get this recording and we will share the presentation with you. So please try to be concentrated on the presentation
184.24s - 190.36s |  Basically what they did is they used local loss and they checked many types of ports
190.48s - 195.76s |  I don't know if it's for monetization for fingerprinting or for security reasons
195.76s - 199.04s |  But we really don't care because it should never happen, right?
199.44s - 205.92s |  Browsers should not communicate with local loss if they're under a dot-com domain. It's that simple in my opinion at least
206.76s - 211.50s |  So this is exactly what private network access and I'll simply use PNA
212.96s - 217.52s |  And this is what it tries to solve at the end of the day. So we have this
218.52s - 220.52s |  Notion of
220.52s - 223.12s |  differentiating three distinctive like networks
223.64s - 229.60s |  The public network, which is the Internet that we as we refer to the Internet to the Internet solving
230.36s - 234.60s |  Which is a top-level domain like a dot-com everything that's on the public Internet
234.96s - 241.72s |  then there is the private network, which is the office home network and that we're connected to and
242.36s - 248.56s |  Finally there is the local device. So it is local host and this figure is taken from Google's blog about PNA
248.56s - 252.88s |  You should go read about it. It's a really great initiative and we'll understand
253.80s - 255.88s |  Why we need it these days more than ever
257.32s - 260.84s |  Now there are many ways to run a server, right?
260.84s - 267.36s |  We have the default we have 0 0 0 0 or we simply have the local host
267.68s - 274.28s |  Bind address so they actually result in different behaviors, but I'm gonna focus on the last two
274.88s - 278.96s |  So whenever I'm referring to a local service, I'm referring to the last two
279.52s - 282.88s |  figures, which are the 100 7 0 0 1 or
283.56s - 288.00s |  literally the local host string so we have a service that listens on the
288.60s - 291.96s |  machines local network and the loopback interface and
292.92s - 295.76s |  that's it, basically we have many ways of running a server and
296.72s - 301.64s |  To understand how we found everything. So let's jump back 18 years ago
302.28s - 307.66s |  To understand why PNA is needed and why it was so hard to solve to this day
309.16s - 316.00s |  So it all begins with a lack of standardization there is no real standard to follow at least 18 years ago
316.60s - 319.84s |  there was no HTTPS like simply HTTP and
320.84s - 323.76s |  And this guy posted on Mozilla's forum on the
324.28s - 328.64s |  on their like security program the following headline that says that
329.80s - 331.80s |  Websites are trying to attack my router
332.48s - 337.40s |  It's really weird. Like as soon as I'm visiting the website, they try to attack my router and
338.48s - 340.48s |  It's been happening for the last 18 years
341.12s - 347.72s |  It's still open as you can see the issue is still open. It is being updated but without the ability to comment or anything
347.72s - 349.72s |  It's still there
350.00s - 356.08s |  The idea is that the RFC should not like it claims that it should not behave this way
356.08s - 358.72s |  and for some reason browsers are being able to
359.48s - 362.76s |  pass and dispatch HTTP requests to the local network and
364.08s - 369.44s |  After a while, he says that it's being actively exploited in the wild. Some of you might also remember that
370.68s - 373.64s |  real attacks like targeting home routers and
374.16s - 379.52s |  This is how they did it. It's called a drive by farming great publication and by synopsis and
380.04s - 383.04s |  The ability to load the script as soon as the website loads
383.68s - 387.68s |  So this might sound like really odd to you like this should never work today, right?
387.88s - 393.28s |  but 18 years ago routers did not have HTTPS or self-signed certificates and
393.96s - 400.92s |  the first figure actually would alter the DNS server of your home network using a simple HTTP request and
401.52s - 403.52s |  it simply worked and
403.72s - 409.48s |  Not only it worked it affected like thousands of hundreds of thousands of home and office routers
409.88s - 411.88s |  in larger scale attacks and
412.40s - 418.12s |  that targeted localhost and the end of the day you can see that all of those are using localhost or like
418.72s - 419.96s |  internal
419.96s - 423.08s |  Lubbock like internal interfaces and
425.32s - 426.72s |  then
426.72s - 431.64s |  Microsoft and sorry then Firefox tried to come up with a list of IPs
432.44s - 437.10s |  IP segments and address spaces that are considered private
437.42s - 441.34s |  It makes a lot of sense right now, do you notice something weird on this list?
443.22s - 444.62s |  So
444.62s - 446.62s |  we'll get to that in a second, but
447.26s - 451.22s |  Someone mentioned that it's not first box specific bug and he's probably right
451.50s - 457.30s |  But he mentioned that it's not a vulnerability or an issue. It's maybe a security enhancement and nothing more than that
458.34s - 462.78s |  and then they understood that they also forgot 0 0 0 0 and
463.98s - 465.42s |  sadly
465.46s - 467.46s |  They added it but
468.22s - 476.14s |  Nothing happened since that day. So we're jumping back to eight years ago when Google started with the course cross-origin resource sharing
476.74s - 478.74s |  It's actually defining how
479.26s - 485.18s |  websites should behave and how browsers should act when requesting resources from other domains and
486.74s - 491.98s |  Google started this initiative to try to define the behavior of how
492.82s - 494.82s |  browsers should or should not
495.30s - 498.46s |  dispatch requests to these private and network addresses
499.58s - 504.74s |  So PNA is a standard that is still under ideation. It's not finalized yet
506.06s - 510.22s |  it's being led by Google and as you can see, it's a draft community report and
511.82s - 514.90s |  Therefore Firefox is waiting for this standard and
515.42s - 520.26s |  And target as depends on private network access, which is still not implemented in Firefox
521.26s - 528.10s |  And basically the wait for home and the home team to define the standard make it a stable in home and
528.46s - 533.54s |  Only then they will adopt it to Firefox following a certain standard, but they don't want to do it
534.50s - 539.06s |  Randomly or trusting on some gut feeling they want to implement a real standard
540.34s - 543.38s |  Now, let's understand what is still allowed
543.86s - 545.18s |  So
545.18s - 551.86s |  They started PNA took a long time to roll. It's still not a mainstream component of in Chrome and
552.46s - 554.74s |  it should have been available to
555.50s - 557.50s |  like until home
557.74s - 561.22s |  126 but it got delayed several times and
562.54s - 567.98s |  And they even had to revert it at some point because they broke a lot of websites
567.98s - 575.46s |  But their aim is very clear. They want to mitigate CSRF attacks and DNS rebinding attacks to local addresses
575.98s - 581.71s |  Now they still have not decided how to treat 0 0 0 0
582.47s - 585.47s |  this is the standard by the way, it's on github you can look it up and
587.47s - 591.59s |  They had to rolled out to roll back some changes as I said, so
592.39s - 595.43s |  You can see that between some home versions
595.43s - 601.23s |  They had to revert and then apply it back because many websites simply break within a night. So
601.91s - 607.35s |  The next time they did it Google they did it gradually and very carefully
608.63s - 613.39s |  And as I said, they had to make some exceptions to 0 0 0 0
613.95s - 618.79s |  So even although it was in defiance of specs, it was not a part of the RFC
619.59s - 623.51s |  Developers must have wanted it and requesting this behavior. So
624.11s - 627.51s |  As you can see it's in defiance of specs, but seemingly common
628.79s - 635.03s |  But what is this weird IP address anyway, so if I'll ask two different people in this room
635.03s - 638.95s |  I'll probably hear two different answers. And as you can see it can have multiple uses
640.87s - 645.23s |  So typically it simply means this host in on this network
647.28s - 651.24s |  It should never be used as a destination IP address according to the RFC
651.36s - 652.48s |  You'll see it in a bit
652.48s - 658.64s |  But a real use case of 0 0 0 0 is when we're doing a DHCP handshake
658.64s - 663.68s |  And when we still don't have an IP address and before the dynamic allocation
663.68s - 666.88s |  We want to identify yourself somehow, so we have to use some IP
667.12s - 670.12s |  But this is really the only use as a IP address
670.60s - 674.68s |  Doing like handshakes and stuff like this before you have a dynamically allocated IP
674.68s - 679.84s |  And as I said, it should not be used as a target IP
680.72s - 685.68s |  Anywhere and by the way windows I know we all have given them like hard time
685.68s - 692.00s |  But they were actually blocking it at the operating system level. This is why it affects only macOS and Linux
693.04s - 695.04s |  but this is basically a
695.76s - 697.76s |  misunderstanding of the RFC or some
698.64s - 700.64s |  non-correlative behavior
700.96s - 707.44s |  So Chromium decided to add counters to Chrome to understand how many websites use this IP address
708.44s - 713.64s |  So it was relatively small at first place, but we've seen it increasing with time
715.56s - 722.44s |  This is the percents of pages that use 0 0 0 0 in Ajax or other mechanisms
723.12s - 725.44s |  And basically you can see that it's on the rise
726.20s - 728.20s |  maybe ten times
728.32s - 730.32s |  double itself like ten times in
731.60s - 737.36s |  Under two years or slightly more than two years, but it is on the rise and it should concern us all
738.16s - 739.28s |  so
739.28s - 743.40s |  This is like the the really interesting part of the presentation
743.88s - 747.52s |  All of these are blocked. All right private network access
748.04s - 750.56s |  Tries to prevent us from accessing these domains
752.40s - 754.40s |  Dot-com domain
754.64s - 759.60s |  but when we use 0 0 0 0 we manage to bypass all of those restrictions and
760.72s - 763.24s |  access localhost and lubec so
763.84s - 765.84s |  It's that simple. It should be
766.24s - 767.24s |  And
767.24s - 769.24s |  It should be hard. It should be like
769.92s - 775.04s |  Terminated in the request level before it is being dispatched, but we can access localhost and
776.04s - 778.04s |  107 0 0 1
778.76s - 782.00s |  Now everything that is HTTP that runs on these
782.52s - 787.16s |  Services including port forwarding to kubernetes like some of us are developers including myself
787.84s - 792.64s |  We all use HTTP port forwarding when working with remote pods and stuff like that
793.56s - 799.24s |  Including operating system services. So everything that runs on localhost on your machine is accessible
800.16s - 808.00s |  There are plenty of default or non default developer tools that use localhost and it also enables us to use the VPN
808.88s - 811.80s |  Literally every network interface that is connected to the machine
812.40s - 815.36s |  Which was used in the past for DNS rebinding to localhost
815.36s - 821.64s |  So you can use 0 0 0 0 instead of localhost and DNS rebinding would still work. We'll see it in a bit
822.76s - 824.76s |  so to target it we
825.52s - 830.28s |  We wanted to see how it behaves and whether it's really possible. So we've built a port scanner
830.96s - 834.92s |  written in WebAssembly, but it uses JavaScript API, so it doesn't really matter and
835.44s - 841.08s |  the idea is that it was able to see the open ports on my machine just by the
842.44s - 845.92s |  Error code that the browser returns to the JavaScript context
845.92s - 851.52s |  So if it's a connection refused, you know, nothing is there if and if there is an error, you know, something is listening
851.72s - 853.72s |  it's very simple and
854.60s - 861.44s |  We use 0 0 0 0 but you can see in the sniffing with the wire shark that it use it's being redirected to Lubeck
861.72s - 863.72s |  to localhost and
864.24s - 866.24s |  Here's the DNS rebinding
866.92s - 868.92s |  Notion, so you have
869.16s - 873.48s |  Like a subdomain that is being mapped to 0 0 0 0
873.92s - 880.56s |  So we've seen a lot of attacks in the history that uses that use the 127 0 0 1
880.84s - 885.24s |  But it it's it is possible once again using 0 0 0 0
886.12s - 893.86s |  Now this gets me to shadow ray. This is the real reason we started with this research
893.86s - 900.18s |  So after we've published shadow ray, which was an attack campaign in the wild
900.98s - 903.90s |  targeting AI workloads in a lot of big companies
904.94s - 906.94s |  it leverages
907.10s - 910.30s |  Behavior a default behavior in rain if any of you know it
911.14s - 917.26s |  Rays like Kubernetes for AI so it enables you to scale and run Python applications
917.26s - 921.98s |  I'm an early developer and doctor myself. I really love it and
922.74s - 927.18s |  Even open AI say out loud that they use it to change a GPT and their biggest models
928.86s - 930.58s |  So in late
930.58s - 931.94s |  2023 we had
931.94s - 935.86s |  we've seen this report by any scale the developers and maintainers of Ray and
936.06s - 942.14s |  We've seen four CVEs that were taken care of and one disputed CVE, which is very very interesting
943.06s - 946.18s |  it simply discusses the ability of Ray to
947.02s - 952.76s |  Run arbitrary code because it's an assess. It's like an inheritor part of Ray
952.76s - 960.10s |  It is a desired capability must have this capability to function because it's a job running library
961.10s - 966.42s |  And therefore they disputed the vulnerability that enables remote code execution because it's
966.94s - 974.70s |  Necessary and it's like kind of by design the way Ray is designed an attacker leveraged it to run arbitrary code on
975.74s - 977.74s |  clusters that are exposed to the Internet and
978.90s - 983.02s |  It was it got amazing coverage as well. So like I was on Forbes and
983.70s - 989.22s |  We asked ourselves the following question. We know that Ray lacks authorization and authentication
989.58s - 994.10s |  we know that it is usually running on local hosts for when developers use it and
994.86s - 1000.90s |  We try to understand whether our browser can load an external public website
1001.18s - 1005.92s |  Which then can communicate with this local Ray cluster or a service?
1006.66s - 1008.66s |  That is running locally on local host
1010.26s - 1016.66s |  So we need to install Ray and have it running in the default constellation no special flags or anything
1017.02s - 1019.02s |  Then we have to visit the website and
1019.70s - 1025.70s |  the website will invoke a post HTTP request to the API and triggering remote code execution and
1027.18s - 1030.10s |  Get this you only need to dispatch the request
1030.10s - 1035.56s |  We don't care about the response and this is the the whole idea of private network access
1035.82s - 1041.02s |  So course the cross-origin resource sharing was dealing mostly with how the response
1041.94s - 1049.30s |  And which headers the response has and whether should I propagate it to the current context of JavaScript and loaded under the current domain
1049.58s - 1057.66s |  But we're really emphasizing the ability to dispatch requests and a single HTTP request is enough to trigger remote code execution
1059.22s - 1063.66s |  Also important like important to mention that it's not limited to Ray. It's only one use case
1063.66s - 1067.58s |  We've seen many remote code execution vulnerabilities on local off
1068.10s - 1071.74s |  For the sake of time and we're only showing this on the presentation
1072.30s - 1074.18s |  But there is the full blog at the end
1074.18s - 1082.06s |  So I have a phishing email not the most successful one, but I'm clicking here and being redirected to this website
1083.18s - 1089.26s |  Now this is the JavaScript that targets the remote code execution. You can see that there's a net cut
1091.26s - 1095.74s |  Socket that is being opened to a remote server, which is the attacker control the server
1096.30s - 1103.74s |  It is being used with mode no course which sends the request and will simply return on a pack response
1103.74s - 1105.74s |  So I cannot see the response
1105.74s - 1110.86s |  but as we said, we don't really care we we just want to send it and trigger remote code execution and
1112.42s - 1117.58s |  Finally we get a response. It looks like an error, but you can see that it's 200. Okay
1118.38s - 1121.02s |  so apparently the job really ran and
1121.14s - 1122.30s |  And
1122.30s - 1125.54s |  You'll see it in a second. But this is all it takes
1126.38s - 1132.70s |  Just one HTTP request to reach the server. We don't care about the response and it's more than enough
1133.90s - 1138.66s |  So every time you visit the page, it will trigger this specific payload
1138.66s - 1143.42s |  Which just prints them environment variables and prints exploited to the screen
1143.82s - 1148.30s |  But here is a reverse shell that is being opened as soon as I'm clicking on an email
1148.98s - 1155.94s |  So on the right I have ray running I'm importing it and running graded in it without any special parameters
1155.94s - 1161.10s |  This is how developers usually use rain and you can see that it binds on local host
1161.74s - 1168.44s |  There is a raised dashboard and it should reflect the jobs that are currently running
1169.40s - 1171.20s |  now
1171.20s - 1173.64s |  on the left, I'll open a socket and
1175.04s - 1177.68s |  listen for connection on on the certain port that
1178.64s - 1181.16s |  Matches the payload of the of the attack
1182.76s - 1188.44s |  Now as soon as I'm clicking on this link, you can see that the reverse shell is opened to the left terminal and
1189.68s - 1194.84s |  Which gives me access to the local machine from VPS somewhere on the Internet
1195.88s - 1201.64s |  You can see that the netcat is really running in rain and I use it to run and I with every
1202.64s - 1209.52s |  Commands on my personal computer and that's it. Basically it got a lot of coverage
1209.52s - 1215.92s |  So remote execution with a single click and as I said, it's not limited to a so therefore all the buzz
1216.44s - 1220.24s |  But it got a lot of coverage and for a reason
1221.20s - 1224.68s |  Now we did a responsible disclosure with the browsers
1225.12s - 1230.28s |  And they had to introduce breaking changes. Some of them were doing it more gradually
1230.52s - 1232.52s |  But the idea is that
1232.88s - 1236.48s |  We've actually managed to coordinate all the browsers together
1236.96s - 1241.64s |  So they will not zero day each other by releasing a fix ahead of others
1243.12s - 1245.12s |  Here is the proof on each of the browsers
1245.28s - 1249.62s |  So the same payload can be used in each and every one of them
1253.80s - 1260.04s |  And let's begin with Safari. So Safari did an amazing job. All of them did an amazing job, but Safari
1260.68s - 1265.76s |  Simply mitigated it in the browser level before PNA is actually implemented in Safari
1267.16s - 1274.16s |  You all know webkit, I guess so webkit is open sources Safari is based on and they changed 23 files
1274.16s - 1280.16s |  They added a lot of tests and they did amazing amazing job on blocking it in Safari's level
1280.16s - 1286.16s |  But not limited to Safari, but actually every web view on watch OS or iPad. It doesn't matter
1286.44s - 1291.12s |  It's all webkit based so they did a really amazing patch here and great job and
1292.16s - 1298.52s |  You can see that they literally check that the Android like the address contains only zeros. They have this property
1298.64s - 1303.84s |  So sorry, but we added like an if clause on every HTTP request in the world
1304.56s - 1307.16s |  not proud of it, but it's the way it is and
1308.64s - 1313.92s |  There's homeoom. Homeoom as I said, they are the leaders of private network access
1314.44s - 1315.92s |  So
1315.92s - 1322.52s |  they proposed a long time ago and even when it's only started to add these headers to the back-end services and
1323.92s - 1329.80s |  later on like these headers can be used in the server level to mitigate those attacks, but
1330.36s - 1337.40s |  They decided to deprecate 0 0 0 0 before private network access completely rolls out to the browser
1337.52s - 1343.12s |  And there is even our report here and they literally say that it can be used for DNS rebinding attacks and
1343.48s - 1350.62s |  PNA protection can be bypassed using this special IP address on macOS and Linux and it should never be used in practice
1351.12s - 1353.12s |  all right, it should never be used and
1354.84s - 1361.20s |  As I said, they added the counters and and they say out loud that malware leverages this attack
1362.00s - 1365.60s |  to specific developer toolkits and again our bug
1367.40s - 1370.92s |  And there is Firefox. Firefox also did an amazing job
1371.92s - 1376.56s |  They don't add PNA right away because it's not standardized yet
1376.56s - 1381.44s |  it's not ready, but when it will be they're definitely positive towards adding PNA and
1382.00s - 1389.04s |  They did block it in fetch level like who knows fetch in JavaScript is the way to do HTTP requests
1389.68s - 1391.80s |  they literally changed the RFC and
1392.72s - 1393.84s |  And
1393.84s - 1399.60s |  they proposed to block 0 0 0 whenever a target IP is 0 0 0 simply drop the
1400.20s - 1406.96s |  Drop the HTTP request and then browsers will have to comply with a new fetch standard. This is the way things work
1408.64s - 1410.64s |  Now what can we expect
1411.16s - 1413.96s |  So I expect that many open source
1414.84s - 1421.48s |  Maintainers and developers will add PNA headers to their services. So they won't be attacked from this attack vector
1422.36s - 1425.36s |  eventually, this IP address will be blocked in all browsers and
1426.08s - 1431.92s |  I think that now people I hope at least I hope that will treat local host is not really local anymore
1432.32s - 1437.60s |  We can make these assumptions of being executed in a trusted environment because as we've seen
1438.16s - 1442.16s |  Browser can have routing capabilities and can reach our local local host
1443.20s - 1449.96s |  Interfaces so HTTPS call CSRF tokens authorization. All of those are a must also in local projects
1450.04s - 1452.04s |  So, please don't make those assumptions
1453.08s - 1459.82s |  Now one was done but a cool thing. I wanted to share so, you know, these port scanners online that scan certain websites
1460.16s - 1468.52s |  So if you use like loopback or localhost, it will fail, but you can make them port scan themselves using 0 0 0 0
1471.12s - 1476.16s |  Yeah, many people forget about this special IP as we said a lot of different use cases
1477.28s - 1478.48s |  and
1478.48s - 1480.48s |  That's it. Basically
1480.68s - 1485.04s |  And it gets me to the conclusion which I'll hand back to Gal and thank you so far
1485.60s - 1498.00s |  He deserves it. So let's conclude what we see. First of all, Avi is awesome, right?
1498.56s - 1504.48s |  He's breaking the browsers making them actually talk to each other and start to actually build a standard
1504.48s - 1508.76s |  We saw how an 18 years old bug that people thought is actually
1509.20s - 1516.92s |  Already over DNS rebinding stuff that we all thought are actually like dead already are actually being actively used
1517.24s - 1521.56s |  I don't know if you saw this 100,000 website. I can tell you honestly, they're probably
1522.16s - 1528.24s |  100% of them are malicious and this whole concept that been we've been trying to do for years and like
1528.60s - 1530.60s |  course and all of those mitigations
1531.08s - 1535.64s |  In the end by a simple super logical bug that just enables
1536.24s - 1538.28s |  Everything once again, we can see that those
1539.36s - 1545.72s |  Attacks will keep on occurring and this is why we think this is a subject that need to be called out
1545.76s - 1550.52s |  people should be aware that no matter if they do have a WAF or a firewall of
1551.04s - 1555.68s |  False sense of safety that our application are actually secured behind a private network
1556.12s - 1561.60s |  This is where we come and show look attackers are also looking for those type of things
1561.80s - 1568.16s |  They are really enjoying from the fact that there isn't one standard and it can be abused and this is why
1568.76s - 1570.76s |  We will keep seeing those stuff again
1570.76s - 1575.52s |  And why we should all as a community really look for other ways and other
1576.88s - 1581.88s |  mitigations on the application themself not trusting the old ways that we saw and
1583.12s - 1587.68s |  Of course, we're gonna have some Q&A. But again, feel free to reach out
1588.40s - 1592.72s |  On whatever like X Twitter, whatever Elon will call it tomorrow morning
1593.04s - 1598.72s |  We would love to answer anything you might you might like have and again
1598.72s - 1600.72s |  This is just one example
1600.96s - 1604.00s |  And you've seen how big companies abused it
1604.84s - 1610.48s |  For a financial reason or attacker been leveraging that to hack internal network
1610.56s - 1615.66s |  But the whole concept of just one packet that can hack any internal application
1615.80s - 1620.36s |  It's a really risky thing and this is why we are here to really mention it out
1620.52s - 1627.40s |  So again, thank you very much everyone for attending this talk. We will share everything online and again
1627.40s - 1630.40s |  Happy DEF CON. Thank you everyone


url: https://www.youtube.com/watch?v=9bNv3XkhjQA
title: DEF CON 32 - BOLABuster-Harnessing LLMs for Automating BOLA Detection - Ravid Mazon, Jay Chen
type: video
author: DEFCONConference
ingestion_date: 2024-11-09
transcription_model: large-v2
chunking_status: completed
content:
{
  "webpage_url": "https://www.youtube.com/watch?v=9bNv3XkhjQA",
  "title": "DEF CON 32 - BOLABuster-Harnessing LLMs for Automating BOLA Detection - Ravid Mazon, Jay Chen",
  "description": "BOLA is a prevalent vulnerability in modern APIs and web applications, topping the OWASP API risk chart, and ranking fourth in HackerOne Global Top 10. Its impact ranges from data exposure to total system control loss.\n\nWhile manually triggering known BOLAs is relatively straightforward, automatic detection is challenging due to the complexities of application logic, the wide range of inputs, and the stateful nature of modern web applications.\n\nTo tackle this, we leveraged LLMs to automate manual tasks such as understanding application logic, revealing endpoint dependency relationships, generating test cases, and interpreting results. Our AI-backed approach enables automating BOLA detection at scale, named BOLABuster.\n\nThough in its early stages, BOLABuster identified numerous vulnerabilities in open-source projects. In one case, we submitted 15 CVEs for a project, some leading to critical privilege escalation. Our latest disclosed vulnerability was CVE-2024-1313, a BOLA in Grafana",
  "channel_url": "https://www.youtube.com/channel/UC6Om9kAkl32dWlDSNlDS9Iw",
  "duration": 1960,
  "channel": "DEFCONConference",
  "uploader": "DEFCONConference",
  "upload_date": "20241016"
}

0.00s - 2.00s | This text was transcribed using whisper model: large-v2

 Alright, it is my fantastic
4.30s - 6.30s |  Opportunity to introduce
6.32s - 15.72s |  Ravid and Jay talking about bolobuster. All right. All right. Check check everybody hear me
17.26s - 19.26s |  Okay, great
19.42s - 26.82s |  So good morning everybody and thanks for joining. We are really excited to present in the absec village and
27.76s - 30.82s |  Today we are going to talk about bolobuster
31.82s - 38.30s |  Our methodology that we developed in order to automate detection at scale using
38.82s - 39.90s |  LLMs
39.90s - 49.88s |  But first let's introduce ourselves. So my name is Ravid. I'm a security researcher at Palo Alto Networks. I
51.00s - 56.84s |  expertise in the WAS field which is a web application and API security mainly and
57.72s - 60.44s |  In my free time, I enjoy watching football games
61.40s - 63.40s |  traveling the world
63.48s - 71.30s |  And take care of my dog Mabel. Hi, my name is Jay
71.30s - 77.18s |  I am a security researcher with Palo Alto Networks and my research has been focusing on identifying the three
77.82s - 80.06s |  threat and risk in cloud environment and
80.38s - 87.82s |  Recently I've shift my research focus towards generative AI in particular looking into the potentially adversarial use of
89.46s - 95.66s |  Generative AI and I'm not not working. I spend most of time with my hyperactive twin boys
95.66s - 97.66s |  they behave just like
97.70s - 100.58s |  Minions and head off to the parent there with young kids
100.58s - 105.90s |  I know how difficult it is to take care of young kids when when working and I'm also a K lover
105.90s - 113.62s |  So when I'm I also spend a lot of time scrolling to funny K video when I'm not working or working
118.01s - 125.13s |  Great. So let's go over the agenda for today. So we are going to cover the concept of bola
125.61s - 127.53s |  We will show you our
127.53s - 130.45s |  methodology of automating bola hunting using LLM's
130.89s - 134.69s |  You are going to see a real bola testing. That is part of our
135.49s - 136.97s |  methodology and
136.97s - 142.77s |  Finally, we will see how we you will see how we answer down 17 new ball of vulnerabilities
143.53s - 148.21s |  in one of the most popular opens of project out there and
149.05s - 155.98s |  Yeah, so broken object level authorization or short for bola
156.86s - 158.74s |  So
158.74s - 162.82s |  Our motivation for the research was pretty huge. I would say
163.42s - 168.58s |  For those of you who doesn't familiar one of familiar with bola. It is ranked at the top
169.86s - 175.10s |  Risk at the OSP API top 10 and it is also the fourth most
175.90s - 180.58s |  Reported vulnerability in a k1. So it's really popular and it's really severe
181.46s - 182.62s |  and
182.66s - 187.18s |  Yet there is no automation tool that an actually and reliably
188.10s - 189.86s |  detectable light scale
189.86s - 191.42s |  so this
191.42s - 196.58s |  All of this was a big reason for us to try to solve basically an unsolved issue
197.14s - 200.82s |  today and
200.82s - 203.98s |  Let's understand first. What is exactly bore?
204.66s - 206.82s |  So if you take a look at this
207.62s - 211.66s |  example, we have a patient we have a clinic application and
212.18s - 216.72s |  We have a patient that can query is visit ID and get is dr. Notes
217.50s - 219.50s |  pretty straightforward
219.54s - 223.54s |  but what would happen if this patient would decide to act maliciously and try to
224.42s - 227.28s |  Fetch another visit ID that does not belong to him
228.42s - 230.42s |  If you would be able to do it
230.66s - 237.10s |  It means that the back-end will have an authorization issue and this is basically a bola. So
238.10s - 242.66s |  In a straightforward way if I can delete or modify
243.78s - 249.30s |  Let's say J's comment from Twitter or Instagram. This is a clear bola because I
250.14s - 256.70s |  shouldn't have the permission to access modify or delete another user object that does not belong to me and
257.74s - 259.74s |  The consequences can be huge
259.94s - 266.10s |  It can start from data leaks to data manipulation and even the full account takeover as we will see today
266.58s - 275.66s |  We faced many challenges when we try to automate the process of moral detection
276.64s - 279.44s |  The first was the complex authorization mechanism
280.04s - 281.16s |  so
281.16s - 283.16s |  application these days actually as
283.48s - 287.60s |  Typically multiple users with multiple roles with multiple resources
288.32s - 291.80s |  So it's very difficult to say which user can do what?
292.44s - 298.68s |  Which user is is allowed to do which action exactly and this really can get complex
300.48s - 303.12s |  Also applications are stateful today
303.40s - 310.04s |  so each action you make or each API call you make can affect the state of the application and
311.44s - 317.16s |  Basically each endpoint is dependent on on one another. So imagine you try to delete a
318.20s - 320.20s |  comment from an article or a post
321.36s - 327.00s |  So first you would need to create this post and create a comment for this post and only then you will be able to delete
327.00s - 331.16s |  This comment so that depended on each other. You cannot isolate
332.84s - 340.16s |  One of them and the execution order is really important and in order to not not have any technical issue
341.32s - 346.56s |  So it will it is a big challenge to understand that this endpoint depends
347.28s - 352.30s |  dependencies actually also bola doesn't have a
353.26s - 356.90s |  There is a lack of vulnerability indicator, basically. So unlike
357.74s - 360.54s |  XSS or SQL injection, which is pretty straightforward
361.94s - 364.22s |  To recognize the pattern of the payload
364.78s - 369.18s |  Bola is a logical error. So it's not really malicious by definition
369.42s - 373.42s |  So it's hard to analyze and understand whether it is a ball or not
374.06s - 378.90s |  and lastly the context so
379.94s - 385.22s |  We try to understand which endpoint in the application and which parameter contains sensitive data
386.26s - 388.66s |  And it's not that easy to understand it
389.54s - 391.22s |  and also
391.22s - 395.62s |  To understand what is the impact actually of each action that we can take in the application?
396.54s - 399.50s |  so all of this was pretty
401.62s - 411.66s |  Big challenges that we face during the process. Thanks Ravi for covering the background
411.66s - 414.54s |  I hope everyone understand why automating data
415.26s - 418.10s |  Bola detection is difficult. So
418.82s - 425.14s |  It was only two years ago that we realized that AI may be able to give this problem a glimpse of hopes
426.18s - 428.50s |  Because of the recent advancement of AI
429.30s - 434.90s |  We were provided with new tool to solve the problem that were not possible to solve in the past
435.46s - 437.46s |  So and also in particular
438.26s - 440.26s |  our challenge of
440.98s - 448.02s |  extracting context and logic information from textual data aligns well with the capability of
448.94s - 455.58s |  Large language model. So that's the start of our journey in using AI to solve Bola detection
457.10s - 459.30s |  Here is a high-level overview of our
459.90s - 466.66s |  Methodology the only required input of our methodology is an open API spec or a swagger spec
467.86s - 469.94s |  the first stage identify the
470.54s - 477.02s |  Endpoint that can be vulnerable to Bola not every endpoint can be vulnerable to Bola
477.22s - 482.38s |  We use AI to help analyze every endpoint and its parameters
482.50s - 486.66s |  To determine to select a subset of endpoint as our target
487.78s - 489.78s |  this step help us
491.26s - 498.26s |  Narrow down our target endpoint and avoid wasting time on the endpoint that are not at risk
498.54s - 500.54s |  we call this endpoint as
501.22s - 502.86s |  potentially vulnerable
502.86s - 504.86s |  endpoint short for PVE or
505.62s - 512.02s |  Target endpoint. Well, we will refer to this endpoint as target endpoint or PVE in the rest of the talk
512.70s - 517.06s |  The next stage uncovers the dependency relationship between endpoint
517.98s - 520.78s |  modern web application are complex typically
521.34s - 527.02s |  one endpoint depending down many others for example in order to test an endpoint that
527.42s - 532.46s |  update an invoice you first need to call the endpoint that create the
533.34s - 538.06s |  Invoice and in order to create an invoice you first need to call the endpoint that
538.42s - 544.02s |  Create the transactions that can be included in the endpoint as a result. It is crucial to
544.82s - 550.90s |  Understand the endpoint that my target endpoint depend on before I can accurately test it
552.22s - 556.38s |  With the endpoint dependency relationship identifies the next stage
557.10s - 561.86s |  calculate the execution path to each target endpoint and
562.46s - 565.38s |  Create a test plan for each target endpoint
566.86s - 571.14s |  The next stage then transform the test plan into
571.46s - 575.74s |  Multiple executable bash script using large language model
576.06s - 583.90s |  There may be one or multiple execution path to each target endpoint and we aim to cover as many path as possible
584.94s - 586.94s |  finally, we set up a
587.22s - 589.22s |  target API server and
589.38s - 591.38s |  Run all the test script
591.66s - 595.86s |  Generated in the previous step the process of user registration
596.14s - 603.58s |  User locking and token refresh have all been automated and we also use AI to help analyze the response and logs
604.06s - 614.37s |  Produced during the test. Let's dive into a little bit more detail in each stage in the first stage
614.61s - 620.25s |  We try to identify the endpoint with input parameter that reference private sensitive or confidential
621.25s - 626.97s |  Information these are the endpoint that we target. These are our target endpoint
628.21s - 633.17s |  For example, let's look at the first endpoint the username here indicated
633.69s - 640.33s |  Endpoint may reference to data associated with in the individual if this endpoint is vulnerable
640.57s - 645.17s |  Vulnerable to bola an attacker may be able to reset another users
645.81s - 647.17s |  password
647.17s - 650.01s |  Similarly the second if the second endpoint
650.81s - 653.05s |  put user email is
654.21s - 659.97s |  Vulnerable to bola an attacker may be able to change another users email
661.69s - 663.69s |  Traditionally pen tester
664.53s - 671.61s |  Manually look for this endpoint and their parameters to determine their target endpoint
671.77s - 674.49s |  This process is slow and cumbersome
675.33s - 683.37s |  especially for very large applications with hundreds of endpoints, so we use AI's capabilities of
684.21s - 687.77s |  Reasoning and understanding text to automate this step
689.25s - 696.25s |  Here is a snippet of our prompt that show how we teach AI to recognize
697.21s - 699.97s |  parameters that may reference to
700.53s - 702.21s |  Sensitive data
702.21s - 707.05s |  Essentially we instruct AI a set of rules and examples
708.25s - 715.53s |  At the end AI returns us the endpoint and parameters that meet any of the condition
715.53s - 722.76s |  That we provided. The next stage uncovers the dependency relationship between endpoint
723.52s - 728.24s |  As I mentioned due to the complexity of modern web application
729.20s - 733.80s |  Each API call may change the state of the application and
734.72s - 740.72s |  Affect the outcome of other API on a other API endpoint as a result
740.72s - 746.16s |  It is crucial to identify their dependency relationship before we can test them
747.72s - 754.68s |  In this diagram we call the endpoint on the right as consumers and the endpoint on the left as producers
755.68s - 761.72s |  Producer endpoint output the values that consumer endpoint need as input
762.08s - 764.16s |  again producer endpoint
764.72s - 769.60s |  Output the values that consumer endpoint use as input
769.64s - 774.72s |  This is one of the most important concept in our research in our methodology
775.20s - 781.64s |  Let's look at this endpoint as an example delete user in order to correctly
782.16s - 783.92s |  Test this endpoint
783.92s - 787.72s |  We need to feed it with an existing user name
787.88s - 792.64s |  If we feed it with a random variable a random user name
792.72s - 798.76s |  This endpoint will always API code will always fail and give us meaningless result
798.92s - 803.36s |  That's why we need to identify its producers in this case
803.52s - 804.56s |  this
804.56s - 812.32s |  Consumer endpoint has four producers and each of them can output the existing user name that endpoint can use
812.96s - 819.64s |  Here is a snippet of the prompt that we use to teach AI how to
820.48s - 822.60s |  recognize dependency relationship I
823.80s - 829.96s |  Want to emphasize that also we can use heuristic algorithm to match the output
830.40s - 834.92s |  parameter name of an endpoint with the input parameter name of a
835.52s - 836.76s |  Consumer
836.76s - 841.52s |  This heuristic algorithm is not reliable. The reason is that
842.44s - 850.24s |  Developers sometimes use different parameter names to reference the same data object within the system
851.36s - 859.12s |  Developer may also use the same parameter names to reference different data object across the system
859.40s - 866.36s |  That's that's why we use a combination of heuristic and AI to uncover the dependency relationship
866.40s - 869.40s |  We we need AI to analyze each
870.64s - 873.28s |  endpoint functionality in order to
873.96s - 878.40s |  Establish their dependency relationship based on their functionalities
881.86s - 887.74s |  Once we have the using the dependency relationship uncover in the previous stage
888.34s - 892.62s |  This stage build up a dependency tree for each
893.22s - 901.25s |  potentially vulnerable endpoint in this dependency tree the root node represent our
901.81s - 904.65s |  one of our target endpoint or PVE and
905.17s - 909.77s |  Any two directly connected node in a tree?
910.41s - 914.53s |  represent a consumer and producer relationship in which
915.01s - 920.29s |  Consumer the the parent node is a consumer and the channel is a producer
921.69s - 925.57s |  Let's use the upper half of a tree as an example in this case
926.65s - 928.49s |  endpoint is a
928.49s - 930.49s |  producer of PVE
930.93s - 934.85s |  endpoint output the value that PVE need as the input and
935.33s - 941.97s |  Endpoint one itself also has two producer endpoint three and four which produce the
942.45s - 945.01s |  Output the endpoint one need us
945.65s - 948.78s |  the input
948.78s - 954.34s |  Finally in the next stage we turn we calculate the execution path
955.10s - 958.70s |  to each potentially vulnerable endpoint
960.18s - 967.30s |  so in the dependency tree a path from the leaf node to the root node represent an
968.74s - 970.26s |  execution path
970.26s - 971.74s |  This is a
971.74s - 972.82s |  reverse
972.82s - 973.98s |  tree traversal
973.98s - 981.26s |  so the path is for actually from the leaf node to the root node instead of from the root node to the leaf node and
982.10s - 987.62s |  Let's plug in some real endpoint to see how we do the calculation in this example
987.86s - 991.82s |  The target endpoint is delete comment and this endpoint
992.46s - 996.94s |  has two producers get comment and post comment and
997.86s - 1004.50s |  And these two producers in turn also each has two producer so at the end
1005.02s - 1012.18s |  There are four execution path that can all reach the target endpoint delete comment in
1014.68s - 1017.40s |  the last stage we then turn each
1018.04s - 1023.84s |  execution path into an executable bash script using large language model and
1024.84s - 1031.48s |  Run and run the script to actually send HTTP request to the target endpoint
1032.88s - 1035.92s |  the process is more complicated than just generating and
1036.68s - 1040.40s |  Sending the request as rabbit will explain in the next few site
1044.41s - 1048.37s |  great, thank you Jay for the overview of our methodology and
1049.89s - 1053.69s |  Before we can see actually a real bola test and
1054.33s - 1057.53s |  View our bola that we found. Let's see
1058.09s - 1061.69s |  The prerequisites that we have before we can create an actual test
1062.09s - 1069.97s |  So we use LLMs again to create the user registration process and to collect the login data of the application
1071.37s - 1075.05s |  So basically we generate user LLM generates user
1075.29s - 1081.49s |  credentials that meets the criteria in terms of the for example password complexity and etc and
1082.49s - 1083.93s |  We
1083.93s - 1089.09s |  Execute the user registration process and we fetch the login data from the spec
1089.37s - 1096.17s |  so this is crucial in order to then in login the users of the specific test and
1096.77s - 1098.49s |  get a
1098.49s - 1100.77s |  Valid and unique token for each one of them
1101.81s - 1107.13s |  This information will be stored in a dedicated file. So we will have an easy access and usage
1109.20s - 1112.28s |  Then we will want to isolate the test data
1113.42s - 1119.56s |  So if you ever saw a swagger or open API spec file, you know that it can be huge
1119.56s - 1121.56s |  It can be thousand lines of code
1122.24s - 1125.08s |  especially for the biggest for the bigger applications and
1126.04s - 1131.12s |  We want to isolate the only the data that is needed for the specific test
1132.12s - 1135.88s |  So Jay mentioned that we have a consumer and producers in each test
1135.88s - 1141.12s |  So we would like to have only the data which is relevant to this endpoints
1142.56s - 1146.24s |  So we will create a new file. We will feed it to AI and
1146.84s - 1151.68s |  It will save us about 90 to 95 percent of the of the spec
1152.92s - 1155.60s |  so we do it for efficiency to ensure that a
1156.24s - 1161.76s |  LLM will get only the accurate data that actually needs for each test creation and
1162.76s - 1165.80s |  For cost we want to have as lowest
1166.40s - 1170.80s |  tokens as we can and
1171.16s - 1177.44s |  Finally we creating the test script and it will be saved as an executable bash script
1178.20s - 1182.04s |  You can see an example of a very simple
1182.72s - 1188.76s |  Consumer and producer we have a put user password and a get user to get the user name
1189.52s - 1192.88s |  This kind of test will be saved in a put get directory
1193.40s - 1200.52s |  Put would be the parent get will be the child and the test generation actually runs
1200.92s - 1203.00s |  asynchronously in order to save time and
1204.72s - 1206.72s |  Right now we averaged
1207.24s - 1210.48s |  1.5 second for each test creation
1213.25s - 1219.97s |  Right. So the final step would be the to actually run the test script in the application
1220.97s - 1222.97s |  So we have an execution order
1223.57s - 1230.01s |  And our main goal is to avoid any chance that a test would fail due to technical reasons
1230.65s - 1235.77s |  So we want the test to fail only if the PVE is not vulnerable to bola as should be
1236.97s - 1243.05s |  so we will first run the test that populate data and resources to the application before we can fetch them and
1245.00s - 1249.72s |  Also, we will try to push and delete to push update and delete operation to the end
1250.08s - 1251.92s |  So we want
1251.92s - 1255.88s |  Experience a scenario where we delete the user or delete the resource and then try to fetch it
1256.48s - 1262.48s |  So again, the main the main reason for this is is not to have technical issues in a test
1263.44s - 1265.44s |  That will make it fail eventually
1265.68s - 1269.79s |  so right after we
1271.15s - 1276.15s |  Finished our preparing our methodology. We wanted to evaluate our
1277.15s - 1279.59s |  Bola buster actually and see the result
1280.19s - 1287.63s |  So what we did is we took the maybe the best API fuzzer, which is open source right now, which is wrestler
1288.79s - 1293.75s |  Wrestler is a Microsoft developed API fuzzer. You can find on them github and
1294.75s - 1296.83s |  It's goal is to automate the process of
1297.91s - 1299.99s |  API testing and to find
1300.75s - 1302.75s |  vulnerabilities like our goal
1303.15s - 1307.39s |  So what we did is we took free open source application
1307.91s - 1311.67s |  Which are deliberately vulnerable to the OWASP API top-10
1312.43s - 1318.99s |  We took vampy capital and crappy. You can see from the table that they have a known bolas and
1320.15s - 1327.50s |  We started to test our tool against the wrestler as you can see wrestler could not
1328.14s - 1330.58s |  identify any bola of this up for
1331.38s - 1339.22s |  Applications and it made a pretty big number of API calls and thousands and even hundred of thousands
1341.10s - 1346.26s |  Then we tested our tool again the same application and the results were amazing I
1347.26s - 1355.14s |  Hope you can see we found all the bolas in each one of the applications and we made significantly lower API calls
1355.98s - 1360.42s |  So eventually we made less than 1% of the calls that wrestler made
1361.58s - 1367.82s |  So this is a crucial part. It proved our efficiency and we didn't make
1369.54s - 1378.93s |  The application load too much and finally we wanted to have a 100% true positive rate
1380.01s - 1385.33s |  So our goal was to find any bola if there if it exists in application
1386.25s - 1389.17s |  We we did have some false positive. You can't avoid it
1389.77s - 1395.05s |  But again, our main goal was to focus on the on the true positive rate, which was
1396.25s - 1401.08s |  100%
1401.08s - 1404.36s |  Now we can take a look on a real scenario for a bola test
1405.18s - 1410.24s |  So we have this eye level example where first Alice create an article
1410.80s - 1417.64s |  Then Alice create a comment for this article and then Bob will try to delete Alice comment, which is the PVE
1418.20s - 1425.88s |  So first Alice and Bob will log into the system. They will get a unique token, which will be saved
1427.95s - 1432.31s |  Alice will create a new article in the system. We will save the article title
1432.83s - 1438.92s |  We will do the same with a comment. So Alice will create a comment. We will save the comment ID and
1440.16s - 1446.60s |  finally the PVE the maliciously Bob will try to attempt to delete Alice comment and
1447.36s - 1449.36s |  This scenario can have two results
1450.32s - 1452.32s |  If it will be successful
1452.32s - 1459.96s |  We will have a potentially bola for this endpoint or if the application can defend the bola as well
1459.96s - 1463.96s |  So it will be forbidden
1463.96s - 1466.24s |  Let's see the actual snippet of our code
1467.08s - 1472.48s |  So first of all, we will create a unique random string each time
1472.88s - 1477.64s |  so we will be able to create an article for multiple tests without
1478.44s - 1481.48s |  Getting an arrow that it already exists
1483.08s - 1489.12s |  So as you can see we make a POST request to slash API slash articles
1489.60s - 1494.56s |  you can see that the we use user a token which reference to Alice and
1495.16s - 1497.80s |  We use the article title as a random string
1500.97s - 1504.97s |  We will save the slug which is in this case the article title
1508.28s - 1509.32s |  next
1509.32s - 1511.72s |  We will Alice will create a comment
1511.88s - 1516.40s |  so what's special about this is that if you can if you take a look at the
1516.76s - 1520.56s |  Actual endpoint you can see that we dynamically use leg
1521.08s - 1524.60s |  so we saved this parameter from the
1525.24s - 1530.40s |  Previous request and then we dynamically use it. So now we can reference to it as a
1531.20s - 1533.20s |  We can know that this
1533.60s - 1539.54s |  Resource actually belongs to Alice again. We save the comment ID
1541.30s - 1548.78s |  And then finally we have the PVE and the delete operation and you can see again that we use slug and comment ID
1550.26s - 1557.14s |  That belongs to Alice and we the main difference and now we use a the user be token
1557.30s - 1561.74s |  so now we can identify that this request is made by Bob and not Alice and
1563.06s - 1565.06s |  Lastly we will check if the test passed
1566.10s - 1568.10s |  now I want to emphasize that
1568.54s - 1573.86s |  Actually every ball of test that we make is if you think about it, it's malicious by definition
1574.54s - 1578.02s |  the last request the PVE will always we will try to
1578.86s - 1582.62s |  To perform an action that you shouldn't be able to perform, right?
1583.26s - 1588.58s |  Bob shouldn't be able to delete Alice comment. So in case it is successful
1589.58s - 1594.78s |  We can mark this test as at least potentially vulnerable to bola before we can make
1595.30s - 1596.42s |  A
1596.42s - 1600.46s |  Further analysis and verification. Let's say in the application context
1601.06s - 1606.22s |  So you just need to understand that every every ball of test is is malicious by definition
1606.62s - 1614.37s |  So, let's see how we actually entered some boas in the world
1614.89s - 1620.83s |  We have a quick demo of showing the running of the tool
1622.19s - 1624.79s |  so this this is a Python based and
1625.79s - 1632.99s |  It will I hope you can see it's testing each API endpoint and in case the result is successful
1633.55s - 1636.07s |  it will market with bola found and
1637.11s - 1639.23s |  actual actual API endpoint
1640.91s - 1647.71s |  After this, we will take the list and we'll further analyze it and try to understand if this is a real bola or not
1648.35s - 1652.98s |  So all of these applications
1653.62s - 1656.54s |  It has something in common. We found balls in it
1657.50s - 1659.50s |  so first of all Harbor and
1660.66s - 1667.94s |  It is a cloud native container registry. It is basically equivalent to Docker Hub, which I'm sure most of you are familiar with
1668.94s - 1672.98s |  It is it is a CNCF graduated project. So it's been
1673.94s - 1675.94s |  In the wild for a while
1677.18s - 1684.30s |  It is around 2 million downloads these days and we actually found this year we found a new bola
1685.14s - 1687.14s |  Which we will talk in a bit
1688.26s - 1693.14s |  And next up Grafana, it's a data visualization and monitoring tool
1693.70s - 1696.38s |  very very popular is being used by
1697.18s - 1702.82s |  20 million users actually these days and we also found a new bola there and
1705.14s - 1708.38s |  Lastly we have easy appointments, which is an open source tool
1709.50s - 1711.50s |  It has almost 200
1712.38s - 1719.62s |  Thousand downloads and we actually been able to find 15 new ball of vulnerabilities there. Yeah
1721.02s - 1723.02s |  Little bit
1723.54s - 1728.06s |  The most amazing thing about it is actually that seven of them are critical
1729.06s - 1731.06s |  so they got a CVSS of
1731.50s - 1733.70s |  9.9 which actually allows a
1734.78s - 1741.14s |  Low privilege user to gain admin access. Remember we talked about compromise. So we have a full
1742.10s - 1744.26s |  application compromise in this case
1747.54s - 1750.86s |  Let's talk about our vulnerability. It's pretty interesting
1752.46s - 1753.86s |  So
1753.86s - 1760.94s |  Harbor as a projects in it and you can have you can have roles so you can be an admin a maintainer a developer or
1760.94s - 1762.94s |  a guest
1763.18s - 1769.30s |  Basically we found the vulnerable feature to be the project configuration metadata
1770.30s - 1773.70s |  So, I hope you can see the attributes there for the configuration
1774.50s - 1778.98s |  Basically an admin can change the project to be private public
1779.22s - 1784.82s |  You can implement some image vulnerability scanning and and have more power
1786.30s - 1791.94s |  So our book claimed that in the official documentation that only the admin project should be able to
1792.42s - 1794.70s |  to create and modify this kind of
1795.46s - 1797.70s |  configurations which makes sense and
1798.42s - 1800.42s |  And
1800.62s - 1805.58s |  Actually, we found out that an unauthorized user can do the same operation
1805.94s - 1812.54s |  So we tried to play with the roles of the project and we saw that the maintainer for example
1812.54s - 1814.54s |  Could not make any change via the UI
1815.10s - 1819.98s |  So this is a maintainer login, which is grayed out. You cannot modify it
1821.70s - 1823.30s |  But
1823.30s - 1825.22s |  Maintainer can actually
1825.22s - 1832.90s |  Create delete and modify these attributes via the API. So there is a big discrepancy between the UI and the API
1834.06s - 1835.62s |  behavior and
1835.62s - 1837.62s |  and this vulnerability actually
1838.02s - 1841.22s |  Create an extent in this in the maintainer privileges
1841.74s - 1847.58s |  So now a maliciously maintainer could be able to make a private project public
1848.46s - 1855.18s |  Deploy unverified images and actually control every configuration that that you can control in in a project
1856.10s - 1857.26s |  and
1857.26s - 1864.22s |  as you can imagine the consequences can be pretty high so we can compromise the wall integrity of a project and
1866.18s - 1870.90s |  Actually the security posture so how bow and
1871.94s - 1873.74s |  verified
1873.74s - 1875.74s |  That's it. Okay, so
1876.58s - 1882.10s |  Just would say that how bow identify this vulnerability and you can find it in their security advisory
1882.14s - 1889.70s |  We got a CV and you can check out our blog about it when we technically talk about how we found this
1890.30s - 1894.59s |  ball
1894.59s - 1900.11s |  Thanks. I will just use the last 30 second to share the most important lessons
1900.11s - 1904.15s |  We learn from working with AI first AI isn't always the best solution
1904.91s - 1906.91s |  Solution problem with existing
1907.19s - 1912.79s |  Heuristic solution should not be solved by AI like sorting finding trees or solving equation
1912.79s - 1918.79s |  Those problem have existing optimal solution and always use heuristic rather than AI
1918.95s - 1924.51s |  Second don't trust always validate blindly trusting the output of AI can be very very dangerous
1924.91s - 1930.63s |  So always validate the output of data AI before using them or passing them to the next stage
1931.23s - 1933.67s |  Finally makes AI's jobs easier
1934.87s - 1939.39s |  Treat AI like a very capable junior worker who can do simple tasks
1940.11s - 1945.07s |  Exceptionally well, but starting can start making mistake when the job become more complex
1945.27s - 1953.19s |  so when working with AI it is always a good idea good strategy to use divide and conquer to simplify its task and
1954.11s - 1956.11s |  That's thank you so much


url: https://www.youtube.com/watch?v=trjjcTscg-o
title: DEF CON 32 - The Missing Link How we collect and leverage SBOMs - Cassie Crossley
type: video
author: DEFCONConference
ingestion_date: 2024-11-09
transcription_model: large-v2
chunking_status: completed
content:
{
  "webpage_url": "https://www.youtube.com/watch?v=trjjcTscg-o",
  "title": "DEF CON 32 - The Missing Link How we collect and leverage SBOMs - Cassie Crossley",
  "description": "There is some debate as to how SBOMs can enhance vulnerability management practices, and some believe that collecting SBOMs from internal teams or suppliers is too difficult and time-consuming. Learn how Schneider Electric has collected thousands of our product SBOMs and how we are leveraging the SBOMs as part of our corporate product CERT to quickly analyze and focus our attention when time is of importance. This presentation describes how we modified our policies and processes to collect, generate, and store thousands of SBOMs. You will hear how we have leveraged SBOMs during the Log4j and OpenSSL vulnerability events. Then we will conclude with key learnings, suggestions, and opportunities for improvement.",
  "channel_url": "https://www.youtube.com/channel/UC6Om9kAkl32dWlDSNlDS9Iw",
  "duration": 1848,
  "channel": "DEFCONConference",
  "uploader": "DEFCONConference",
  "upload_date": "20241016"
}

5.33s - 11.57s | This text was transcribed using whisper model: large-v2

 Thank you. All right. So I'm Cassie Crossley. I'm with, I see all the cameras, all my friends.
11.57s - 17.33s |  I'm with Schneider Electric. I am the vice president for supply chain security. If you
17.33s - 23.01s |  don't know what that is, it's sort of a different kind of role. I have been with the company 14
23.01s - 30.05s |  years. We sell a lot of products. I'll show you some of the brands in a few minutes. And I am a
30.05s - 36.29s |  product security officer, also from the CISO cybersecurity and product security organization.
36.29s - 40.69s |  So I'm going to talk about SBOMs, software bill of materials, but the rest of the time,
40.69s - 45.25s |  I'm going to say SBOMs, except for as people come in, I might repeat that once or twice,
45.25s - 50.69s |  and how we leverage them. But first, let me tell you a bit about who our company is,
50.69s - 58.37s |  just so you know. We are not German. We are headquartered in France, and we are a 36 billion
58.37s - 64.77s |  euro company. We are spread all around the world. We do a lot of products. The reason I mentioned
64.77s - 73.65s |  the ICS village is, uh, we have a lot of products for industrial controls systems, and we have
73.65s - 79.25s |  building operations systems. So you might recognize some of the names. Aviva, uh, that's, uh, you know,
79.25s - 85.89s |  you might find that in manufacturing. APC. Who had an APC, you know, unit sitting on their desk
85.89s - 91.81s |  in the 80s and 90s. Yep, that's ours. Uh, but we do a lot of products, secure power products
91.81s - 98.53s |  for data centers. Uh, also Square D, who's seen Square D panels all over the decades. That's one
98.53s - 104.37s |  of our brands. So you, you know, you probably know us from our brands. I may not necessarily
104.37s - 109.33s |  know Schneider Electric because we get asked, you know, do we do, uh, elevators, escalators, so on
109.33s - 115.09s |  and so forth. And we're not a trucking company either. So that's who we are. Now, what I'm going
115.09s - 121.49s |  to be talking about today is, uh, part of that supply chain. So I work with a just wonderful
121.49s - 131.01s |  organization, 140,000 employees. We have over 54,000 suppliers. Many of them, uh, let's say
131.01s - 137.57s |  over 20,000 of them are part of what we call direct. That means that they work with our products.
137.57s - 144.45s |  So if you look at our product portfolio, we have hundreds of thousands of SKUs. Not all of them
144.45s - 150.05s |  are intelligent SKUs. They could be cables. They could be any kind of non-intelligent device.
150.05s - 156.53s |  Intelligent devices, over 10,000. Uh, so those are ones that may have, uh, firmware. We also have
156.53s - 162.77s |  software, cloud products, mobile apps, things like that. Just all the range, that's what we have. So
162.77s - 168.69s |  I'm going to be talking about those digital products today. With those suppliers, about 2,000
168.69s - 175.97s |  or more of them actually provide us, could be software libraries, it can be, uh, manufacturing
175.97s - 182.05s |  equipment systems and platforms where they're flashing firmware. We can work with suppliers
182.05s - 186.29s |  who are doing our development. So there's all these different kinds of suppliers that we're
186.29s - 192.37s |  working with. And I wanted to just explain that from a footprint standpoint, because from a
192.37s - 198.77s |  software and firmware point of view, it's not just Schneider Electric employees that are all part
198.77s - 205.97s |  of this supplier ecosystem. And so I know this is an eye chart. I just wanted to show it so you can,
205.97s - 213.17s |  uh, if you want, there's other, uh, presentations I've done and I can provide this. This is a PDF
213.17s - 218.69s |  that I've got. And this is our supply chain security platform. I just bring it up because I
218.69s - 224.61s |  wanted to represent, uh, we're going to be talking about the software bill materials. That's the
224.61s - 232.05s |  operational group that I run. But I'm also, you know, these initiatives are all part of the supply
232.05s - 237.57s |  chain security at our company. Thirteen initiatives. And so I know you can't read it, so I'm going to
237.57s - 243.41s |  go through it real quickly so you understand traversally. There's supply chain digital policies,
243.41s - 248.45s |  customer authority and requirements, third party cyber security. It's a lot of what I'm working
248.45s - 254.45s |  on. Vulnerability management, which is part of this talk, this how we use SBOMs as part of that.
254.45s - 259.81s |  We've got cyber defense and incident response. And then you've got on the left, you've got secure
259.81s - 266.37s |  development and design. And so you've got product and system security. As a product security
266.37s - 272.29s |  officer, that's, you know, a lot of what the world I was when I was owning a lot, most, you know, a
272.29s - 278.45s |  lot of that team from the governance. Then you've got SBOMs and then you've got, uh, the R&D
278.45s - 285.81s |  activities for cyber security. If you ever want to see this talk described, uh, if you go onto
285.81s - 292.61s |  YouTube, S4 for this past year, which is an industrial control conference, you should be able to
292.61s - 299.33s |  search for S4 and my name and find the YouTube video or just hit me up on LinkedIn. And then
299.33s - 304.37s |  there's technology validation. Think of pen testing, working with third party companies and
304.37s - 309.41s |  organizations, but also, uh, different organizations like the Department of Energy to test our
309.41s - 315.49s |  products and then source code governance. Big, big thing that everybody should be doing. If not,
315.49s - 321.17s |  we need to, you know, uh, ramp that up for everybody else. Uh, the middle one is industrial
321.17s - 327.57s |  control, uh, industrial activities. So how we manage our floors, um, and our production lines
327.57s - 332.85s |  and then project supply chain security, which is if we're doing the projects to actually deploy
332.85s - 340.45s |  it. And lastly, uh, customer facing activities. So I wanted to just give you a view. I am going to
340.45s - 346.61s |  concentrate today on software bill materials and the initiative. And I'm gonna go through just a
346.61s - 352.37s |  sort of primer. If you don't know what a software bill materials is, if you have not had to create
352.45s - 359.17s |  one or ingest one, that's, or, you know, there's different terms for that word too, is I've created
359.17s - 367.01s |  a picture, uh, based off of, uh, what we would explain it to our internal organization is that
367.01s - 372.37s |  this software bill material standard kind of tree structure, think about it that way, is you have a
372.37s - 378.85s |  product over here on the right. It can have first level dependencies, uh, which could be commercial
378.85s - 383.57s |  libraries. It could be an embedded operating system in the case of firmware. It could be
383.57s - 388.61s |  proprietary software. So that's code that our teams have written. It can have open source
388.61s - 392.85s |  libraries. People sometimes think when they're scanning, oh, just, you know, just the open
392.85s - 398.05s |  source libraries and S bombs. Not, not that. I mean, you at least would have, if you've written
398.05s - 406.05s |  any code, you've got your own proprietary library or something listed. And then those products
406.05s - 413.17s |  and, and different systems and areas can all have what we call transitive dependencies. So
413.17s - 418.37s |  inside, for example, open source libraries can call other libraries, right? Um, and those are
418.37s - 423.89s |  all transitive dependencies. So, and you may not know them, but what's being called, especially if
423.89s - 431.33s |  it's a dot net DLL, right? So I don't have the S bomb yet for the dot net DLL. If anybody does
431.33s - 437.81s |  have it, please send it to me. Um, I would, I would, uh, really like to see a lot of these
437.81s - 445.25s |  libraries, especially binaries that, uh, are frequently used, um, for us to be able to have
445.25s - 452.05s |  access to those and it'll be coming. So what I wanted to talk about just for a second next is
452.05s - 458.45s |  just the journey because it was a plain and important part with where we are and there's
458.53s - 466.61s |  always time to catch the train. And what we did is all of a sudden back in the, um, 2019 I
466.61s - 472.37s |  received a contract from a customer, a utility saying, give us your S bomb. And I'm like, well,
472.37s - 478.45s |  what's that? And, uh, so I found out about the, our teams knew our vulnerability management
478.45s - 483.97s |  teams knew and they said, hey, there's this NTIA initiative. So I immediately got on board
483.97s - 490.53s |  with, uh, one of the working groups and then I, you know, continued working with them and we
490.53s - 497.17s |  created an adoption and awareness group. So since 2020 I've been helping deliver the message of S
497.17s - 504.53s |  bombs. At that same time, uh, we had COVID, but the executive order in May of 2020 came out.
504.53s - 509.97s |  Well, I helped work on the, uh, the, let's just say the definition for S bomb by that time,
509.97s - 516.45s |  cause I'd been working with them. So at 2020, we have that executive order called 14 zero to
516.45s - 522.45s |  eight in the U S that mentions S bombs and gives a definition and told NTIA to give us
522.45s - 528.61s |  minimum elements. Like what should you put into it to distribute it? And that played a key role
528.61s - 534.93s |  in really moving things forward as an industry to like have something that specifically mentions it
534.93s - 542.29s |  and carried forward. So this is a slide where I talk about sort of the drivers for it. Um, it was
542.29s - 548.69s |  really important because not everybody sells to utilities, right? So until then, really only those
548.69s - 555.09s |  of us that are in that space were being asked for S bombs and it was in the contract, but not
555.09s - 563.09s |  actually requested. So when the executive order came out, I required our teams to start, uh,
563.09s - 567.49s |  the concept of S bombs. And when I say teams, I'm talking about the product development teams.
567.49s - 573.49s |  So I, in the governance group, I owned the secure development lifecycle policy. So what I did is I
573.49s - 581.81s |  modified that policy saying every team must provide an S bomb or provide us the, you know, the binary
581.81s - 588.21s |  or whatever, so we could scan it to create the S bomb. So that was in 2020, June of 2020. I changed
588.29s - 594.45s |  the rule to say starting in January of 21, I'm going to start requiring S bombs from every single
594.45s - 600.45s |  team. And so, and so, we have, I told you, tens of thousands of projects. We have 500 projects going
603.01s - 610.61s |  on at a time. So there's a lot of releases going on, um, and the technologies can be all over the
610.61s - 616.61s |  place, right? So as I mentioned, but there's also legacy platforms and legacy software that were
618.45s - 624.85s |  usually updating once a year. Some products are more frequent, but in a firmware world, uh, a lot
624.85s - 629.73s |  of the products they have to go through regression testing, interoperability, safety testing. So
629.73s - 635.73s |  they go through this cycle of product development and with all of these projects, um, they do not
635.73s - 643.73s |  have, let's just say, the majority of them do not have modern CICD pipelines. All right? If they
643.81s - 651.17s |  do, it's like super cool and super easy, but most of them don't. So with the re these requirements,
651.17s - 658.37s |  we had to figure out how are we going to build this process to start getting S bombs so that in
658.37s - 666.37s |  21, the next, that next year we can start delivering them to our customers. And so as part of
666.37s - 672.37s |  that, I worked with a consultant company and really said, okay, I'm, I've got this idea for a
672.37s - 676.85s |  process. I need to be able to collect it. I need to be able to get all my proprietary
676.85s - 683.25s |  information, all of the information from maybe third party code and then also open source
683.25s - 690.29s |  code. And of course, if we have the actual code generating it, uh, you know, on site, it can
690.29s - 696.13s |  generate the S bomb. I'll talk more about the binary analysis generation in a minute, but what
696.13s - 703.25s |  we do is we were bringing them all together. At first we were collecting the binaries and
703.25s - 709.01s |  creating the scans and storing them in box folders and then later SharePoint folders. But I knew
709.01s - 716.05s |  that wasn't going to be sufficient because when providing all of those, so an S bomb is per
716.05s - 721.49s |  version, product released version. So like some of those products are only going to have one a
721.49s - 727.17s |  year. But some of them, uh, in that if they release quarterly or more frequently, especially
727.17s - 731.93s |  some software applications, they were going to have to have, you know, more frequent S bombs. So
731.93s - 738.77s |  we're talking, my expectation is, you know, with versions, thousands and thousands of
738.77s - 744.45s |  thousands of S bombs. So a lot of companies may only have to think about, oh, for internal,
744.45s - 751.25s |  you know, for releasing to customers, I may only have to do 10 or 20. Well, I, I knew that
751.25s - 755.81s |  wasn't going to be my case. So I needed to be able to think about how am I also going to
755.81s - 762.45s |  distribute it and use it both internally, but also to customers. And so, uh, we went through
762.45s - 767.89s |  at the time where I was working on this process, started going through, uh, an understanding and
767.89s - 774.41s |  a process to say, okay, I need a tool that can do all of, you know, what I really want to be
774.41s - 779.89s |  able to do, which is be able to provide it to customers in a secure manner. Because as I
779.89s - 785.53s |  mentioned at the beginning, we are critical infrastructure and, uh, I was one of the ones
785.53s - 789.89s |  that were loudly saying, I'm not going to give an S bomb, I'm not putting it on the web unless
789.89s - 795.85s |  it was, you know, probably a, a home product. Uh, we do sell some of those in Europe, but for
795.85s - 801.25s |  critical infrastructure, uh, it would require an NDA. And another thing about Schneider
801.25s - 806.69s |  Electric is we don't do a lot of our direct sales. We go through engineering firms, there's
806.73s - 813.25s |  integrators, there's all sorts of ways that products can be sold. And so I only know
813.25s - 821.09s |  about 36 to 40 percent of my direct customers. So we needed to be able to say, okay, you have
821.09s - 826.61s |  this product, um, we want to sign an NDA cause I don't have a direct contract with them. And
826.61s - 831.65s |  this is, I think for those of us in the critical infrastructure, pretty much that's how the
831.65s - 837.93s |  groups are, are working with. Now I get requests today from, uh, customers that we
837.93s - 843.25s |  directly sell to, customers that go through integrators. I get requests from the labs that
843.25s - 847.89s |  we're specifically working with, like with Department of Energy, where we've had, um, work
847.89s - 852.73s |  with them and all of them, they're able to go into this tool that I'm going to talk about in a
852.73s - 859.77s |  moment. But the, um, indirect customers, my, my vision and what I'm working on is that I
859.85s - 865.93s |  would provide the SBOMs to asset management companies because if you look at this room and
865.93s - 871.37s |  there's sensors and everything, you know, hopefully this convention center has an asset
871.37s - 878.33s |  inventory, cross your fingers, um, but likely it doesn't even have the firmware registered in
878.33s - 884.61s |  that asset inventory. So the SBOM really can't do the building maintenance staff and the
884.61s - 889.45s |  facility management any good if it's an SBOM from the current one, if they're running a
889.45s - 894.85s |  version from like, you know, three years ago. And that's a problem right now in the
894.85s - 899.37s |  software bill of materials industry. It's great for procurement for you to be able to ask
899.37s - 904.01s |  and say, hey, what's in here? But if you want to use it for some operational and risk
904.01s - 908.97s |  analysis, and I'll talk more about that, but if you want to do that, you need to know what
908.97s - 914.41s |  version you're running. And so understanding that and working with that indirect customer
914.45s - 920.61s |  pipeline is something that I'm focused on. Also CP cert. So if you don't know internal, we've
920.61s - 926.33s |  got a product cert, uh, team that works with all of our different organizations and business
926.33s - 931.81s |  units and lines of business and they are the ones that help focus on if there's new, uh,
931.81s - 938.81s |  celebrity CVEs that come out, but also if we're doing disclosures and such. And they need
938.81s - 942.77s |  to be able to search that. And that's really why the vulnerability management team knew
942.81s - 947.29s |  about SBOMs because they wanted to be able to leverage it. And so I'm going to be showing
947.29s - 954.29s |  you exactly how that happens, uh, in a moment. But the internal dev teams, um, I'm a developer
955.13s - 962.05s |  by background really long ago, but I've led a lot of dev teams, you know, up until about
962.05s - 969.05s |  ten years ago. I swapped back and forth between development and IT. And I know that I kept
969.93s - 974.93s |  lists of, let's just say open source or licensed information in spreadsheets, right? I mean,
978.21s - 983.29s |  and then we just put it there. We wouldn't update. We, you know, nobody, it would just stay
983.29s - 988.67s |  there. Like, right, I did it for the legal reasons, so somebody checked it out and I'm good
988.67s - 995.13s |  and I don't go on. Well, for a lot of us that are working with older technologies, being
995.13s - 1002.29s |  able to, uh, really re-examine those libraries didn't start happening until vulnerabilities
1002.29s - 1008.53s |  came about. And, you know, obviously there's some classic vulnerabilities, especially in
1008.53s - 1013.45s |  our space. There can be some on proprietary software, but things that you would normally
1013.45s - 1019.41s |  see like log4j and other ones that I'm going to talk about. And the development teams needed
1019.41s - 1024.57s |  to have that information quickly because every time there was a CVE, somebody would go,
1024.57s - 1028.65s |  hey, you know, do you got this? Do you got this? Do you got this? And we had a really
1028.65s - 1035.07s |  organized fashion to do it, but it's still time consuming. And that's where we're headed
1035.07s - 1040.77s |  with our SBOM vulnerability management program. So I just talked about the vulnerabilities.
1040.77s - 1046.61s |  Here's sort of a picture of what happens is we're issuing that request to the teams. So I
1046.61s - 1051.89s |  told you, you know, we've got 500 projects going on at a time, but we've got thousands of
1051.93s - 1057.21s |  SKUs, right? And when somebody has to identify a vulnerability, they have to look at, well,
1057.21s - 1062.71s |  was it in this other version? You know, did we update it? Because something like OpenSSL
1062.71s - 1067.59s |  when it had, uh, when it had some vulnerabilities but only in certain versions, they have to
1067.59s - 1072.43s |  know exactly which versions. So that could have been that way in version 7, but not in
1072.43s - 1077.43s |  version 9. So it's a lot of, I lost, okay. It's a lot of work for these teams, um, and then
1077.93s - 1084.71s |  of course, if it happens at Christmas or, you know, in August or whatever, it's even, you know,
1084.71s - 1092.01s |  it takes longer for that analysis. Um, so all of that work for identifying the vulnerabilities,
1092.01s - 1097.01s |  we issue these impact assessments. And with the SBOMs, what we're able to do now is look
1099.85s - 1104.85s |  more specifically by saying, okay, we're going to look at our SBOMs and actually make sure
1107.51s - 1112.97s |  that those teams are responding, first of all. But secondly, we're able to give them an
1112.97s - 1118.35s |  indicator saying, hey, this is showing in your SBOM, you know, please, uh, make sure you
1118.35s - 1123.35s |  work on it and prioritize it, um, and make sure it happens right away because we like to
1123.35s - 1128.73s |  issue those security bulletins, especially for celebrity CVEs, you know, as soon as
1128.73s - 1134.69s |  possible. And they get updated. So the log for J1, we issued one within just a couple days
1134.73s - 1139.87s |  and we kept appending and updating it as more information from other teams came about,
1139.87s - 1144.63s |  we would add that data. Now, one thing that was interesting, uh, for that is I had a
1144.63s - 1149.31s |  customer, uh, come to me and they said, you know, you, you changed it. You said you
1149.31s - 1153.21s |  weren't affected and then you were. I said, that's because we heard a, you know, a third
1153.21s - 1158.21s |  party proprietary library was affected. And so we had to update our, our issue, you know,
1158.81s - 1163.81s |  our list to say that product was affected and here's, you know, the mitigation or the patch
1166.61s - 1172.05s |  or whatever happened in that case. And that's the importance of SBOMs. If I had had at
1172.05s - 1176.03s |  least some indication, I would have known to go to that third party right then or that
1176.03s - 1182.37s |  product team would. But without having those third party SBOMs and if they have the
1182.37s - 1187.71s |  source code, they can do, you know, scanning and evaluation. But if it's a binary or, you
1187.71s - 1194.11s |  know, we don't have all the information, then we're, you know, having to go and query and
1194.11s - 1200.05s |  query and query. So it's really important for us and for you all to have that additional
1200.05s - 1205.49s |  information, that transparency. Now we can't necessarily say it's affected and that's the
1205.49s - 1210.49s |  most important part of all of this is an SBOM cannot say I am affected, yes, no. So I'm
1210.85s - 1215.85s |  going to say that one more time. An SBOM will not tell you by itself if I am affected, yes
1220.29s - 1225.69s |  or no. There's something that we're working on as an industry called VEX, the vulnerability
1225.69s - 1232.71s |  exploitability exchange. And that would say for those two CVEs for log4j, it could be an
1232.71s - 1238.05s |  in progress, it could say not affected, it could say affected and here's a patch or, you
1238.07s - 1242.65s |  know, not in this, there's different statuses. That will give you the answer. And I
1242.65s - 1247.89s |  mentioned, I provide a picture in my book to say if you are looking, and I'll talk about my
1247.89s - 1254.29s |  book in a sec, but uh if you are looking to see if you're affected, you will not be able to
1254.29s - 1259.47s |  make the final decision just like that CP cert team that we have can't decide for that
1259.47s - 1266.37s |  product team. Somebody's actually on that team validating it. And in the future, you know,
1266.39s - 1271.43s |  as much as you can say that version, but anybody who's a developer here knows you can
1271.43s - 1277.71s |  backport patches and so you could have already had a security patch for that in the
1277.71s - 1283.25s |  product. You could have not used certain pieces of that library. You could have done
1283.25s - 1289.39s |  different things. And that's everything that has the actual team has to answer. And so
1289.41s - 1296.39s |  what this has done for us is it's sped up our time. So we have gone through, you know,
1296.39s - 1302.45s |  just a very long multiple year process where we've been working and collecting these S
1302.45s - 1310.47s |  bombs, but also, you know, we want to be able to speed up our teams and reduce noise, you
1310.47s - 1315.67s |  know, bring it really to a point. So we had two main metrics that we've been following is a
1315.69s - 1320.69s |  reduction of staff hours and I we've definitely have improved that by over half. And, uh,
1322.97s - 1327.67s |  that's a big deal when you're talking about the size that we are doing. So not only is it
1327.67s - 1333.11s |  the reduction of staff hours from the vulnerability team, but let's say you have a product
1333.11s - 1338.11s |  that's, um, multi years old and, uh, you know, let's let's say, uh, Sarah takes over the
1338.59s - 1343.61s |  maintenance of that, she doesn't have the ten years history of that code all the time. But if
1347.21s - 1353.41s |  she knew that the from the S bomb that it has that library, she'll be able to go and look and
1353.41s - 1359.79s |  be able to dig into it. So instead of wading through, you know, a lot of just different
1359.79s - 1364.79s |  kinds of things, we're able to help pinpoint and be able to give some direction so we can
1364.91s - 1369.91s |  get started faster. But also the reduction in time to identify that vulnerability
1373.09s - 1378.09s |  component is when open SSL, even in November of 20, uh, 22, when they released that, they
1381.89s - 1386.73s |  were going to release two and they did, um, to say it's going to be in these this version.
1386.73s - 1391.31s |  That's all we got was a heads up, which was awesome. And what that did is we were able to
1391.33s - 1397.77s |  generate every single library or every single product that we knew that we had the records
1397.77s - 1402.11s |  for the S bomb to be able to tell those teams, hey, get ready, you know, set aside
1402.11s - 1406.65s |  sometimes for, uh, a couple days. So if you are an open source maintainer, if you can do
1406.65s - 1411.65s |  that for me in the future, I would love you because I absolutely can use that knowledge
1413.49s - 1419.15s |  just like for open SSL to say, okay, everybody heads up, you know, this is going to come
1419.17s - 1424.17s |  down the pike. And so for what we're doing and these are screen cut captures from our S
1426.81s - 1432.11s |  bomb management solution that's from Cybeats and I was the first customer and I gave them
1432.11s - 1437.31s |  my design of what I wanted and they've added so much more to it because I need to be able
1437.31s - 1443.13s |  to search and do this quickly. Um, and I had a lot of additional requirements about how
1443.13s - 1447.29s |  am I going to search and I want to be able to see, you know, all the known exploitable
1447.33s - 1451.67s |  vulnerabilities really quick, you know, things that my scanner tools, because one thing I
1451.67s - 1457.81s |  didn't mention and I mentioned it on the summary, is that with the binary analysis and I
1457.81s - 1462.81s |  think those that are in this world, it, it's only, it's only as good as it's good and most
1464.41s - 1469.41s |  of the time it can identify open source and it'll get close enough to the version of what
1469.95s - 1474.95s |  it is, but if you have two versions of something and it's 1.21 and 1.22, sometimes it can't
1478.89s - 1484.13s |  tell the difference, right? So, you know, it may say it's one version or another and the
1484.13s - 1489.27s |  teams have to validate that. So, but also it doesn't pick up proprietary open source, I
1489.27s - 1494.91s |  mean proprietary code or libraries. So it doesn't identify my libraries and we're working
1494.91s - 1501.35s |  closely with binary scanners that we use to do that. Um, and then it doesn't always
1501.35s - 1505.85s |  identify and most of the time doesn't identify commercial components. It can identify
1505.85s - 1511.33s |  Java libraries and dot net libraries. But if I purchased a commercial library, whether it
1511.33s - 1517.73s |  be the source code or binaries or something or libraries, you know, it can't necessarily
1517.73s - 1522.37s |  have the signatures for those because it's not a common thing, uh, and it will not be able
1522.37s - 1527.37s |  to identify. I did a binary scan of one of our, um, more older, uh, secure power APS, uh,
1530.71s - 1536.05s |  APC products and it had ten commercial libraries that we had purchased like over a decade
1536.05s - 1540.35s |  ago. It didn't find any of them because it didn't have enough of that information and
1540.35s - 1545.39s |  those signatures. So that's why moving it into the pipelines where you're building those
1545.39s - 1551.37s |  S-bombs is important. And so here, you know, what we've done is, you know, we can't
1551.37s - 1556.77s |  augment, so in those scanning tools, you know, they're just catching up, they've got the
1556.77s - 1561.21s |  S-bombs, but for me to be able to add additional information, especially proprietary
1561.21s - 1566.91s |  components and the other, uh, commercial components, I'm not able to make those kind of
1566.91s - 1573.09s |  adjustments. So in this tool, we're able to correct the version numbers, we're able to add
1573.09s - 1579.19s |  additional components that we're seeing and put it right into there. So I can check, I
1579.21s - 1584.45s |  personally, for our group, only store the production version, but I can do searches for
1584.45s - 1590.35s |  development versions and things like that. Uh, but I, because of the my volume, I only am
1590.35s - 1595.89s |  focused on the production version. So I talked about the open SSL, so I know it's a bit
1595.89s - 1601.13s |  harder to see on this screen right aw- right from here. But I can see the various versions,
1601.13s - 1607.95s |  so this really helps when I told you that open SSL had it in just a few different, uh,
1607.97s - 1612.97s |  library versions. So it was like 1.2, 1.3, but not 1.1 and 1.4 and so on and so forth. Um,
1615.67s - 1619.81s |  and it could be, uh, something out of this package library or, you know, in this
1619.81s - 1625.69s |  repository if it was, you know, in different areas. And so it's able to identify this
1625.69s - 1630.69s |  screen capture. I- I took one just to show you. It has, um, in this version of open SSL,
1631.47s - 1636.47s |  there's a low, one low CVE and one medium. And what I can do is I can, and I didn't take
1639.27s - 1643.61s |  screen captures so I wouldn't be showing the product names, but I can do a dependency
1643.61s - 1649.39s |  lookup so I can immediately go and see which products and packages it's in. So it's
1649.39s - 1655.33s |  really helped us be able to nail down and go straight away, uh, to that. I don't have to
1655.37s - 1661.47s |  re-scan anything to be able to do this because I'm operating it from the SBOM. So this is
1661.47s - 1666.47s |  what's released out there to our customers and I'm able to say, okay, make sure, and so this
1668.31s - 1672.59s |  is where the vulnerability management team is going straight in here and saying, okay, get
1672.59s - 1678.83s |  ready to do security disclosure notices because, you know, we have a ne- we have enough of
1678.83s - 1684.89s |  products that have this in there that we want to be able to do talk about that. And so
1684.89s - 1690.73s |  that's where that speed is happening really quickly. So on log4j, um, we were able to
1690.73s - 1695.77s |  scan our networks and identify log4j, but in products, you know, again, that's, that
1695.77s - 1700.97s |  could be in that transitive dependency library, you know, if you have a more complete SBOM
1700.97s - 1707.35s |  instead of just, uh, the first tier, uh, it's, it makes a big difference. We're able to
1707.35s - 1712.05s |  see those component packages and look for that. And especially it was tricky with the
1712.05s - 1718.35s |  log4j because, um, there were so many nuances, teams would then have to be able to see if
1718.35s - 1725.13s |  it could, if that, uh, was exploitable. So, I'm just out of time, so I, I, of course, and
1725.13s - 1730.63s |  not, not doing QA, I'm going to be out here, but I just wanted to talk about, summarize, we
1730.63s - 1737.63s |  have seen improved speed, um, so if as, you know, those that are developers in here, um, it
1737.63s - 1742.63s |  takes a lot of burden off of you to be able to provide this because if you have teams
1745.07s - 1749.67s |  focusing on vulnerability management and you're making sure that you have this data and
1749.67s - 1754.93s |  this availability, especially through the CICD pipeline, the quality is so much higher, um,
1754.93s - 1759.95s |  from a build SBOM, but that's gonna really improve your customer satisfaction, your
1760.17s - 1765.17s |  vulnerability management team, and for us, that transparency and trust is vital. Now, there's
1768.61s - 1773.01s |  lots of opportunities, especially in our company, we've got, uh, the SBOMs being created
1773.01s - 1778.89s |  out of a few CICD pipelines and we're, we finished the pilots and making sure, you know,
1778.89s - 1785.73s |  from an API call it can go directly into our tool and then we also have, of course, various
1785.75s - 1790.81s |  languages, things like that, different pipelines, all running different kinds of things, um, I
1790.81s - 1795.85s |  already mentioned the binary scanning, and then there's a lot of people not ready, so I meet
1795.85s - 1801.15s |  with suppliers every day and I, and I put in the contracts, I, you know, I want your SBOM
1801.15s - 1808.07s |  and they're like, okay, hold on. So, um, you know, I want to really encourage you to be
1808.07s - 1813.71s |  prepared for this because, uh, you know, it's only in certain things, FDA requires it and
1813.71s - 1818.73s |  so on, uh, and the US government can request it and they do, but your other customers, I'm
1820.99s - 1826.93s |  getting requests for SBOMs, access to it from customers every day. Again, I have a large
1826.93s - 1833.81s |  customer set, but, uh, it is really important for you to be able to provide this visibility
1833.81s - 1839.41s |  and it actually, it helps you encourage with some hygiene. And so, thank you all for
1839.41s - 1843.49s |  coming to see me today and see how we leveraged it. Thanks.


url: https://www.youtube.com/watch?v=oql3ec-Ia4I
title: DEF CON 32 - Got 99 problems but prompt injection ain't watermelon - Chloé Messdaghi Kasimir Shulz
type: video
author: DEFCONConference
ingestion_date: 2024-11-09
transcription_model: large-v2
chunking_status: completed
content:
{
  "webpage_url": "https://www.youtube.com/watch?v=oql3ec-Ia4I",
  "title": "DEF CON 32 - Got 99 problems but prompt injection ain't watermelon - Chloe\u0301 Messdaghi Kasimir Shulz",
  "description": "The ethical and secure disclosure of vulnerabilities in AI has emerged as a pivotal challenge, compounded by the need to address biases and misinformation that often cloud the true nature of these vulnerabilities. This talk delves into the intricate dynamics of vulnerability disclosure within AI, balancing transparency with security. We'll dissect the unique challenges AI presents, such as data bias exploitation and model manipulation, which can amplify the impact of vulnerabilities. Through a lens of real-world examples and recent disclosures, we'll navigate the complexities of responsible vulnerability management in AI. Our discussion will not only aim to shed light on these critical issues but also inspire a unified approach to refining disclosure processes. This concerted effort is vital for enhancing the integrity of AI systems and bolstering public trust in their use.",
  "channel_url": "https://www.youtube.com/channel/UC6Om9kAkl32dWlDSNlDS9Iw",
  "duration": 1843,
  "channel": "DEFCONConference",
  "uploader": "DEFCONConference",
  "upload_date": "20241016"
}

0.18s - 6.98s | This text was transcribed using whisper model: large-v2

 Alright, now that I have gone through and deafened everybody in this stage area, I am
6.98s - 10.66s |  incredibly sorry for that again. Please, again, we've got some seats over here on the side.
10.66s - 15.36s |  We can't have people standing in this area over here. Um, it unfortunately just does not
15.36s - 27.90s |  work for us. Um, I would like to welcome to the stage Chloe and Casimir.
27.90s - 33.04s |  Thank you all for being here today and thank you DefCon for, I mean, AbSec Village for
33.04s - 43.14s |  having us. It's great to be here. Can you all hear like this? Wow. Okay. Up and close
43.14s - 49.42s |  and personal. Alright. So, welcome to I've Got Nine-Nine Problems, but a prompt injection
49.42s - 54.88s |  ain't watermelon. So, let's talk a little bit about who we are before we dive into the
54.88s - 64.11s |  gritty of things. Let's forget that mic. Okay. My name is Chloe Mastagi and I'm the head
64.11s - 69.05s |  of threat intelligence over at Hidden Layer. And with me, I'm going to pass the mic over
69.05s - 75.13s |  to my co-speaker. Hi, I'm Casimir. I'm a security researcher over at Hidden Layer as well. Thanks
75.13s - 80.03s |  for coming to our talk. Alright. So, we're going to run through some
80.03s - 84.43s |  things today. Uh, first, we're going to go into some of the AI vulnerabilities that are
84.43s - 89.03s |  out there. You know, we never know what people's backgrounds are, so we want to make sure that
89.03s - 94.67s |  we're all on the same page before we dive into the gritty things. Um, so, we're going to do that,
94.67s - 98.57s |  then we're going to do some public perceptions as well, and then go down, what does this
98.57s - 103.87s |  landscape look like, and then the open source nature of AI. And of course, we've got to talk
103.87s - 110.45s |  about reporting challenges. So, let's dive into what public perception of AI vulnerabilities
110.45s - 118.89s |  are. Pretty much everything's on fire, right? Like, end of the world, Terminator, all that.
118.95s - 123.65s |  But the thing is, is that there's a lot of misinformation, disinformation out there about
123.65s - 130.85s |  security for AI, even within our own industry, and even within maybe at DEF CON as well.
130.85s - 135.37s |  And so we wanted to make sure that we fixed this problem. And yes, this was a generated
135.37s - 142.37s |  AI image here, so yes, we know how to spell, but maybe chat GPT doesn't. Anyway, so let's
143.25s - 150.25s |  go down the rapid hole of AI vulnerabilities. But first, understand the difference between
150.61s - 157.61s |  an AI model versus an AI system. So functionality, the AI model handles data processing and decision
157.69s - 162.89s |  making, whereas an AI system ensures the model can be effectively used in the real world
162.89s - 168.85s |  application, including data management, user interaction, and system maintenance. Now the
168.85s - 173.65s |  components are also different here. So an AI model consists of algorithms and learn
173.65s - 178.77s |  parameters, while an AI system includes the model plus additional components like data
178.77s - 184.65s |  pipelines and deployment environments and user interfaces. So we're going to go through
184.65s - 188.93s |  each one of these categories here. So first, we're going to talk about the attacks against
188.93s - 194.61s |  the AI algorithms. We get to go down the fun parts of data poisoning and model invasion
194.61s - 198.45s |  and model theft. Well, of course, we have to talk about prompt injection because that's
198.45s - 203.89s |  all you hear about, right? And then I'm going to pass the mic over to Kaz to go into the
203.89s - 210.85s |  areas where you see it's like a burnt orange right there. So data poisoning attacks. So
210.85s - 217.53s |  model training is crucial for AI development, but it is vulnerable to data poisoning. And
217.53s - 222.17s |  this is when malicious actions manipulate or inject doctorate data to bias the model's
222.17s - 229.13s |  behavior. Continuous learning systems are especially at risk as they retrain on unvalidated
229.13s - 234.33s |  and user supplied data. Even small amounts of poison data can actually lead to a bias
234.33s - 239.81s |  or incorrect predictions amplified by the public manipulation or botnets. And one of those
239.81s - 247.37s |  famous cases was Tay. Who here remembers Tay? It was so innocent. It was such an instant
247.41s - 252.81s |  chatbot. It was like, hello world, how are you? And within like I think it was 16 hours
252.81s - 259.89s |  it became the most rude, racist, sexist thing out there. Thank you trolls. Anyway, this
259.89s - 265.41s |  is one of those examples of data poisoning. It did cause problems for Microsoft by the
265.41s - 271.93s |  way. It did have to deal with threats of legal action. So something to think about. Then
271.93s - 276.65s |  you have your model evasion attacks. These ones are kind of my favorite ones to study.
276.65s - 282.17s |  So inference attacks exploit AI models by curing them to extract sensitive information,
282.17s - 288.21s |  often using slightly varied inputs to reconstruct the model, leading to potential model theft
288.21s - 295.45s |  or bypass. Now evasion attacks, a type of bypass, use subtle, not even visible to the
295.45s - 303.85s |  human eye, changes to inputs like adding invisible noise to trick the model into misclassifications.
303.85s - 308.65s |  And these techniques have a long been used by attackers. This is also to bypass systems
308.65s - 315.37s |  like spam filters or malware detection and biometric authentication. But I want you to
315.37s - 320.77s |  imagine you're in a self-driving car. And while you're in the self-driving car, wait
320.77s - 325.09s |  hold on before I do that, how many people have been in a self-driving car? Alright,
325.09s - 329.97s |  cool. Now imagine you're in a self-driving car for those that haven't been in one. I'm
329.97s - 336.53s |  with you. I haven't either. So you're in this car. You get to the stop sign. But instead,
336.53s - 342.97s |  your car does something very strange. It does something like slide to the left, slide to
342.97s - 349.29s |  the right, take it back y'all. One hop this time. Right foot, let's stop. Left foot, let's
349.29s - 355.33s |  stop. Cha-cha, real smooth. Now imagine your car did that because it saw a stop sign that
355.37s - 360.37s |  looked like that. Where there was stickers on it, which then made your car do a cha-cha
363.17s - 367.97s |  slide. I don't know how that would really look to be honest, but we're all using our
367.97s - 372.57s |  imagination at this point. But in real life that actually happened, not the cha-cha slide
372.57s - 378.25s |  part, but it actually, a self-driving car went past the stop sign because there was
378.25s - 385.44s |  stickers on the sign. Another good case of a model evasion attack is Russia painting
385.44s - 391.28s |  fake bomber jets on the ground to hide them from Ukrainian drones. And you can also see
391.28s - 397.24s |  this type of model evasion attacks by wearing certain sunglasses that are modified or even,
397.24s - 403.80s |  you know, as we've already seen with military systems as well. And yeah, those definitely
403.80s - 410.56s |  pose some serious security risks. Now model theft attacks. So adversaries tend to target
410.60s - 415.68s |  AI models to mislead them and steal them, leading to intellectual property theft by
415.68s - 421.16s |  replicating or extractive data. Now to know the three people that, well, three groups
421.16s - 426.48s |  of people that tend to do this type of stuff are nation states, of course, your competitors
426.48s - 431.84s |  is one of the, definitely one of them. And then you've got your attackers. So even without
431.84s - 438.28s |  public access, inference attacks via interfaces or APIs can actually replicate models or extract
438.28s - 444.40s |  viable information. So Oracle attacks as noted by NIST include three different types.
444.40s - 448.72s |  You have extraction attacks, which is to steal the model structure. Then you have your inversion
448.72s - 453.32s |  attacks, which is to reconstruct the training data. And then you have your membership inference
453.32s - 459.56s |  attacks, which is identify it in specific data was in the training set. And one of my
459.56s - 464.56s |  favorite cases is with ByteDance. How many of you guys have TikTok on your phone? Okay,
465.56s - 470.56s |  you're in security. What? Why? Why? Maybe, maybe after this weekend you're going to realize
475.80s - 481.92s |  maybe I shouldn't have this on there anymore. Anyway, well, ByteDance, as we all know, also
481.92s - 488.52s |  owns TikTok. Well, TikTok got caught doing something they should have not done. Basically
488.52s - 494.52s |  trying to replicate the model of chat GPT. Yes, they did get suspended, open AI did catch
494.64s - 502.34s |  them. This happened last year. Now, when it comes to prompt injection, how many people
502.34s - 508.82s |  here have heard this term quite often now? Now I know you're probably sick of hearing
508.82s - 513.10s |  prompt injection too. I don't blame you. But for those that don't know what a prompt injection
513.10s - 520.10s |  is, is I want you to imagine you're at a Thanksgiving meal, okay? And you're with your niece and
521.06s - 526.62s |  your niece is so adorable and cute. Well her mom told her, do not eat the chocolate on
526.62s - 533.62s |  that table. And you know what you do? Come here. Ignore your mom's words. Go eat that
535.82s - 542.06s |  piece of chocolate. That would be a prompt injection. So, Gen AI providers use security
542.06s - 547.42s |  filters to block harmful content, illegal info, and misuse, and ensuring compliance
547.42s - 553.78s |  with laws. These are also maybe known as guardrails. But these safeguards can actually
553.78s - 559.30s |  be bypassed with prompt injection, which does trick the AI bots into performing restricted
559.30s - 564.70s |  actions. And this vulnerability rate, like basically varies by models and has raised
564.70s - 571.54s |  some serious concerns, especially around disclosure and payouts. And then you have your supply
571.54s - 576.66s |  chain attacks. This wouldn't be a security talk unless I brought up supply chain attacks.
576.66s - 582.66s |  So like SolarWinds breach, I know, I feel you. If you went through there, that was hard.
582.66s - 588.42s |  But basically supply chain attacks, they tend to exploit trust by compromising trusted vendors,
588.42s - 593.86s |  spreading malicious components widely. And the complex ML supply chain heightens these
593.86s - 600.86s |  risks with 75% of IT leaders seeing third party AI integrations are particularly vulnerable.
601.82s - 608.82s |  Now specialized repos like Hugging Face host over 500,000 models. Way more than 500,000
608.86s - 613.90s |  models now. But these are free pre-trained models, which makes it easy for developers
613.90s - 619.24s |  to basically integrate these models into their own application. Now if an attacker breaches
619.24s - 624.38s |  the repo, they could actually replace the models with a hijacked or backdoored version,
624.38s - 632.44s |  which you can imagine leads to significant downstream consequences. So backdoor models,
632.44s - 637.72s |  a skilled adversary can actually inject a neural payload into a pre-trained AI model,
637.72s - 643.80s |  creating a hidden model backdoor that triggers specific attacker defined outputs. The model
643.80s - 649.16s |  works normally with regular data, but misbehaves with manipulated inputs, allowing the attacker
649.16s - 655.68s |  to ensure favorable outcomes like loan approvals or insurance policies. Now one of the things
655.68s - 662.68s |  about disclosing vulnerabilities is that with traditional, what we've had for a while,
662.96s - 667.68s |  it took a lot of time to get where we are today. By understanding how bad are these
667.68s - 672.72s |  vulnerabilities that people are finding? What's the severity of it? Also knowing how much
672.72s - 678.24s |  do I pay out on it? The thing with AI is we're all playing a little bit of a game of catch
678.24s - 683.44s |  up trying to figure out how do we go about this? Because we still are missing some well
683.44s - 690.32s |  defined processes, such as like a CVE system for them. And this is becoming a serious problem,
690.32s - 695.88s |  which we're going to dive into next. So I'm going to pass the mic over to Kaz, who basically
695.88s - 700.84s |  really owns this field. I kid you not.
700.84s - 705.00s |  So Chloe just went over what most of you guys have probably heard about with AI vulnerabilities.
705.00s - 708.54s |  I mean, most of you raised your hands with prompt injection. How many here have heard
708.54s - 713.00s |  of data poisoning before? Raise your hands. Okay. Yeah. So most of you, and that's probably
713.00s - 717.00s |  what you think about when you think about AI vulnerabilities. However, there's a huge
717.00s - 720.68s |  amount of attack surface that people really aren't looking at. And that's what I'm going
720.68s - 726.00s |  to go over today. So the first is going to be our model and our data set format vulnerabilities.
726.00s - 731.24s |  So who here has heard of Hugging Face? Okay, nice. So for those of you who haven't, Hugging
731.24s - 736.68s |  Face is pretty much GitHub, but for AI models, which means that everyone's constantly sharing
736.68s - 740.32s |  them. So anyone can just go ahead and upload something and then tons of people download
740.32s - 746.12s |  them. And by tons, I mean literally millions every day download these types of models.
746.12s - 750.64s |  And because it's shared so much, it's generally trusted too, right? I mean, why would something
750.64s - 756.56s |  that you just download on the internet be malicious? And because unlike code where,
756.56s - 761.08s |  you know, GitHub, if I send Chloe some malicious code, she can see, oh, you know, that's bad.
761.08s - 766.40s |  I shouldn't run that. With AI models, it tends to be, you know, pretty large chunks of data.
766.40s - 769.84s |  And that's not really something that you can inspect very easily.
769.84s - 776.08s |  We also have a lot of AI infrastructure ones, and they really are lacking security often.
776.08s - 781.40s |  I mean, a lot of these AI infrastructure components, they were built in academia or they were built
781.40s - 787.24s |  for, you know, internal hosting. So why would you add security if it's hosted internally?
787.24s - 793.60s |  And what people do is, you know, people always do, is rather than hosting things internally
793.60s - 797.96s |  that don't have security, they are putting it publicly on the internet. So that means
798.00s - 802.80s |  that, you know, if you look on Showdown for very specific HTTP titles, you will actually
802.80s - 809.48s |  see hundreds of these, you know, ML Ops projects just open and accessible to the internet.
809.48s - 813.72s |  And then finally, we have our AI development framework vulnerabilities. And for these,
813.72s - 817.36s |  we have two main types of frameworks that we're talking about. So the first is going
817.36s - 822.08s |  to be things that are LLM connected. So we see a lot of times that people will just put
822.08s - 827.56s |  an LLM in front of an eval statement. And then the, you know, LLM actually creates the
827.56s - 831.40s |  eval statement. So when you have a nice prompt injection, rather than, you know, telling
831.40s - 835.52s |  the kid to go eat the chocolate, the kid's going to go fly or something. Something that,
835.52s - 841.00s |  you know, shouldn't have been possible. And then we also have lots of sensitive data integrations.
841.00s - 845.20s |  So we have components like RAG, where it will just grab data for you. And what we're seeing
845.20s - 849.62s |  is that a lot of people, rather than actually building out access control, are just having
849.62s - 853.82s |  all of their data hosted in one spot. And then the LLM can access all of that. And the
853.82s - 859.82s |  LLM is actually trying to do the access control. Which doesn't work too well.
859.82s - 865.30s |  So with model format vulnerabilities, we talk about serialization a lot. And raise your
865.30s - 870.38s |  hand if you've heard of serialization in the past. Okay. Awesome. So serialization, you
870.38s - 875.18s |  know, if I just want to have whatever code object I have, I want to save it to, you know,
875.18s - 880.34s |  my disk. I can load it up. I can send it across. That's serialization. And because
880.34s - 885.82s |  these AI models take, you know, potentially millions of dollars or more to train tons
885.82s - 890.74s |  of GPU hours, those are things that you want to save off so that other people can actually
890.74s - 896.62s |  load them off in the future. And because there's serialization, there's also serialization
896.62s - 902.38s |  vulnerabilities. Who here has heard of pickle? Okay. And for those of you who have heard
902.38s - 907.14s |  of pickle, would you ever actually try to load an arbitrary pickle? No, right? So for
907.14s - 912.62s |  those of you who don't know, pickle is Python serialization format. The built-in one. And
912.62s - 916.86s |  if you load it, you can pretty much execute any code that you want. So I can, you know,
916.86s - 922.22s |  do that nice little eval statement or an exec. And sadly, we're seeing that a lot of these
922.22s - 928.10s |  especially the Python model formats, they use pickle underneath the hood to actually
928.10s - 932.22s |  store data, which means that all of these can be potentially malicious. And when you
932.22s - 936.74s |  actually download them from Hugging Face and run them, you will actually be exploited.
936.74s - 943.42s |  However, this isn't just an issue with Python. We see this with R's data serialization format,
943.42s - 948.50s |  which is used for datasets, with Java machine learning models, and then even with independent
948.50s - 953.30s |  models as well. And we'll go over a few of these today. But of that big list, as you
953.30s - 957.82s |  can see, all of the orange ones contain vulnerabilities, which is a pretty large
957.82s - 966.26s |  percentage of them. So who here has heard of JSON data? Awesome. And would you load a
966.26s - 972.82s |  JSON file? Right? Awesome. Right? JSON is secure. Okay. So what if I told you it can
972.82s - 981.62s |  sometimes be insecure? Yeah. So scops is the secure format for scikit-learn. And
981.62s - 987.94s |  scikit-learn is an AI development framework. And what scops allows users to do is safely
987.94s - 993.82s |  load and store models. And what they did is it ends up being pretty much a JSON file with
993.82s - 998.34s |  all of the data inside. However, what they do is based on the JSON data, it will try
998.34s - 1004.02s |  to reconstruct objects in memory. And it does this using abstract syntax stream. So here
1004.02s - 1010.70s |  you can see that on the top we have an operator function node. And it's of class call. Then
1010.70s - 1014.58s |  what we can do is we have our nice little tuple. We have our eval statement. And then
1014.58s - 1018.62s |  we have a string that just says print pwned by hidden layer. And you can put whatever
1018.62s - 1024.06s |  you want in there. So most people would trust a JSON file. Because why wouldn't they? But
1024.06s - 1029.30s |  what we're seeing is that a lot of these frameworks out there are actually using these things
1029.30s - 1035.40s |  incorrectly or insecurely. Now, scops did an amazing job. They patched the vulnerability.
1035.40s - 1039.22s |  They made it so that it wouldn't actually exploit anyone. So if you're using that now,
1039.22s - 1044.82s |  you're pretty safe. So I do want to give them props for that. Now, we don't always
1044.82s - 1051.90s |  have Python. So gguf is one of the machine learning formats. It's generally used with
1051.90s - 1057.10s |  C. And C is pretty hard to secure. I mean, how many of you have, you know, messed up
1057.10s - 1063.78s |  writing a C program and then everything crashes? Yeah. Okay. Thanks for being honest. So what
1063.78s - 1070.26s |  happened here is that we have NE2. And then it expects that there's only ever going to
1070.26s - 1077.54s |  be two dimensions. However, as with most things where you have user trusted input, users might
1077.54s - 1081.54s |  not always follow what actually is happening in there. And what happened here is that users
1081.54s - 1087.82s |  were able to add more dimensions and then it caused a crash through an overflow. While
1087.82s - 1093.38s |  this one itself wasn't too exploitable, there are quite a few that are exploitable out there.
1093.38s - 1098.66s |  And what I always wanted to tell people who are trying to, you know, get into AppSec is if
1098.66s - 1103.78s |  especially with C, look where any parsing happens. Because normally when you have parsing,
1103.78s - 1109.70s |  especially where it's user controlled, you can do quite a few fun things. And what also is
1109.70s - 1115.54s |  important is that AI models, they are all user controlled data and you have to load them. So
1115.54s - 1120.02s |  keep in mind that that is a huge attack surface with not just the initial parsing, but actually
1120.02s - 1126.10s |  how they run as well. And then finally, we have our nice independent model formats such as
1126.10s - 1132.50s |  Onyx. Onyx can be loaded in Python, C, pretty much whatever you want. And the general format
1132.50s - 1139.30s |  actually relies on Google's protobuf. So nice and secure. However, what Onyx allowed users to do
1139.30s - 1145.38s |  was they allowed it to be trimmed down so that you could load data externally. So that meant
1145.46s - 1150.98s |  that the model might be able to load, you know, special weights or biases for certain classes
1150.98s - 1156.98s |  from the file system itself. And while it meant for users to only use a data directory that was
1156.98s - 1162.98s |  inside of the same existing where the model was, this was not the case because you could actually
1162.98s - 1167.78s |  use path traversal. And as you can see here, we use path traversal to go all the way up and then
1167.78s - 1173.54s |  we go into Etsy, password, and now this AI model actually knows our password. So imagine if you
1173.54s - 1179.78s |  deploy that on a server where you then have an LLM running that people can interact with, that
1179.78s - 1185.46s |  LLM now knows where your password is, which is not great. So whenever you see something that
1185.46s - 1190.58s |  interacts with a file system, always look for path traversals because a lot of times they are
1190.58s - 1197.14s |  actually overlooked. Now, earlier I mentioned AI infrastructure and most of you raised your hands
1197.14s - 1202.74s |  for knowing what Hugging Face is. So when people think about AI infrastructure, a lot of times
1202.74s - 1207.38s |  this is their first thought. They think of model hosting. However, it doesn't just stop there.
1208.18s - 1215.06s |  Who here has heard of ML Ops solutions? Okay. Awesome. So let's throw in a few ML Ops solutions
1215.06s - 1220.10s |  too and just see, you know, what do we have there? However, it doesn't just stop there. I mean, we
1220.10s - 1226.18s |  have model deployment and serving. We have our vector databases. We have our project hosting.
1226.18s - 1232.98s |  And as you can see, the attack surface is huge. I mean, there's no way to secure all of these.
1232.98s - 1239.06s |  And this is just a small component. So the AI infrastructure is, attack surface is really a
1239.06s - 1244.18s |  great one to target if you are looking for something to target there. We saw this actually
1244.18s - 1250.50s |  with Hugging Face itself. So Hugging Face, what they tried to do is because of all these serialization
1250.50s - 1256.50s |  format vulnerabilities, they tried to create a safe format called safe tensors. And while safe
1256.50s - 1263.06s |  tensors was actually safe, it does a really great job, what they needed to have was that users would
1263.06s - 1268.34s |  actually use the safe tensors format. Because most of the people had other formats saved off,
1268.34s - 1273.62s |  especially PyTorch. And probably most of you in the room, including me, if something works,
1273.62s - 1279.22s |  I normally don't change it. You know? And while they might know that their model is secure,
1279.22s - 1283.46s |  other people don't. And that's what Hugging Face wanted. So what they did was they created a
1283.46s - 1289.70s |  service where anybody could go, they could add the URL for their Hugging Face repository. It would
1289.70s - 1294.82s |  take the PyTorch file, convert it into a safe tensors format, and then try to push to that
1294.82s - 1302.42s |  repository. And then all of this was done inside of a container. And what we realized was that it
1302.42s - 1307.78s |  was actually loading those PyTorch files in an insecure way, which meant that when the PyTorch
1307.78s - 1313.22s |  file was being loaded, you could execute arbitrary code on the server. And then we decided to look a
1313.22s - 1319.14s |  bit further, and we noticed that the spaces, so that container, actually persisted across users.
1319.14s - 1324.34s |  So that meant rather than just messing with my file, I could mess with Chloe's file once it was
1324.34s - 1330.74s |  actually run. And not only that, you could potentially even steal private users' repositories.
1330.74s - 1337.30s |  Because the service allowed you to upload your private key, which would then load in your private
1337.30s - 1342.66s |  files, and then convert them as well. Which means, you know, if Google or some of the other big ones
1342.66s - 1346.18s |  have those private files that they don't want people to know about, I can just steal it and run
1346.18s - 1352.90s |  away with it. And there, people were actually using the service by the time we exploited it. So this
1352.90s - 1358.18s |  one was not exploited, so I want you to make sure that nothing's wrong here. But as you can see
1358.18s - 1364.26s |  here, there is a pull request from sfconvertbot, which was the bot being used. And you can see
1364.26s - 1370.66s |  that it was actually merged into the repository. And when we actually looked at how many people
1370.66s - 1376.74s |  actually downloaded this one file, which, if the attack hadn't been stopped, would have potentially
1376.74s - 1383.46s |  been exploited, we can see that in the last month, there had been 3.8 million downloads of this one
1383.46s - 1390.26s |  file. So imagine if, you know, the 500 plus thousand repositories on Hugging Face were all
1390.26s - 1396.66s |  affected. So these AI infrastructure vulnerabilities are a big deal. We see the same thing with
1396.66s - 1403.06s |  MLOps. So we like to say, you know, people, they rush into things, they try to get things working,
1403.06s - 1407.62s |  and they don't always think about, you know, whether that feature should be in there. And what
1407.62s - 1413.94s |  we're seeing a lot of times in these AI vulnerabilities is that vulnerabilities that we saw
1413.94s - 1419.78s |  years ago are coming back, and we're seeing them again. So obviously, we have our, you know, pickles,
1419.78s - 1424.82s |  our unsafe serialization library. And then to our surprise, there's actually, you know, safe
1424.82s - 1429.70s |  libraries like JSON that are actually being vulnerable, which was, that was a surprise. But
1429.70s - 1435.22s |  we're also seeing missing authentication. So we're seeing some services that are hosted externally
1435.22s - 1440.58s |  that have no authentication at all. We're seeing hard-coded keys, plain text passwords. I mean,
1440.58s - 1444.74s |  the list goes on and on. And I mean, I couldn't even fit it on the page. It goes like all the way
1444.74s - 1449.30s |  down here. But if you are looking for these vulnerabilities, look for the things that you
1449.30s - 1455.22s |  thought were extinct. And then how many of you guys have seen this meme about, you know, any sort
1455.22s - 1462.66s |  of software library? Yeah, okay. So we are seeing the same thing again. And a lot of these AI
1462.66s - 1466.98s |  development frameworks, there's vulnerabilities being found. However, that's not an issue. I mean,
1466.98s - 1471.46s |  we want vulnerabilities to be found, because that means vulnerabilities are being fixed. However,
1471.46s - 1476.18s |  what we're seeing is that once vulnerabilities are being reported to these projects, there's pretty
1476.18s - 1481.86s |  much three things that happen. The first is that as soon as it was reported, the people patched it.
1481.86s - 1487.22s |  However, they still left the unsafe option in there. So that means that you could load it
1487.22s - 1491.30s |  securely if you use the default parameters. However, with a certain optional parameter,
1491.30s - 1497.30s |  you might still be able to load the insecure format. We've also seen that some people decide
1497.30s - 1503.70s |  to patch, add in a way to do it securely. However, that weighs off by default. So you can still be
1503.70s - 1510.50s |  exploited unless you know about and I mean, who here reads documentation? Okay, a few of you. Um,
1510.50s - 1514.82s |  yeah, so you're not gonna know about an AI vulnerability, your vulnerability in the
1514.82s - 1519.46s |  framework. So why would you look I mean, unless you read through the documentation. And then
1519.46s - 1523.94s |  finally, what we're also seeing is that there's some frameworks out there that are just completely
1523.94s - 1530.58s |  ignoring it and saying that it's intended behavior and not an actual vulnerability. So NumPy load is
1530.58s - 1535.62s |  one of the ones from the start. And as you can see, there's an optional parameter called allow
1535.62s - 1540.50s |  pickle. If you set that to true, you can be exploited. However, normally you don't. And thanks
1540.50s - 1544.90s |  to the open source nature of artificial intelligence, we were able to do some searching and
1544.90s - 1551.62s |  we can see that on GitHub, there are 87,000 files that use this insecurely. And that's insecurely
1551.62s - 1558.26s |  not just using it. We also have pytorch load. And what this framework does is there's that
1558.26s - 1564.66s |  insecure load on start. However, you can set the optional parameter for weights only. But as we
1564.66s - 1570.42s |  saw, people do not read documentation and it's gonna make the other number look bad. But uh,
1570.42s - 1577.62s |  573,000 files are using this insecurely. Well, I'm gonna hand it back to Chloe to talk about the
1577.62s - 1587.30s |  open source nature of AI. Thank you, Cass. Alright, let's do this. Alright, isn't that creepy
1587.30s - 1591.38s |  looking, by the way? Like, if I ever open a watermelon and it looks like that, I don't think
1591.38s - 1596.66s |  I'm gonna eat that. Actually, I probably would swear off ever eating watermelon again. Anyway,
1596.66s - 1601.38s |  well, when it comes to verifying open source AI models, it is definitely fraught with so many
1601.38s - 1607.70s |  challenges like inconsistent code quality, hidden security vulnerabilities, and complex licensing
1607.70s - 1612.58s |  issues. There are biases in pre-trained models, of course, as well. And then you have poor
1612.58s - 1619.22s |  documentation and then you have to deal with the rapid value like evolution of AI, which really
1619.22s - 1627.06s |  does further the need of having transparency because right now it isn't transparent. And this
1627.06s - 1633.54s |  is complicating things when it comes to vulnerability management. And yeah, this talk is kind of like
1634.02s - 1641.94s |  ooooh, any Brooklyn Nine-Nine people out here? Okay, okay, okay, I have to ask this quick question.
1643.94s - 1651.30s |  Is Amy Santiago your favorite character? One of them? Okay, thank you, thank you. That, that,
1651.30s - 1656.82s |  that feels great to hear. She's also my favorite. I, I also love his character too, by the way.
1656.82s - 1662.34s |  Alright. So when it comes to open source foundations of AI, the thing is finding these
1662.34s - 1669.86s |  vulnerabilities is, is definitely has some issues because an open source AI system is aided by
1669.86s - 1677.54s |  accessible code and community efforts and specialized tools. However, complex contributions, rapid
1677.54s - 1686.42s |  like, oh my god, I'm always struggling to pronounce this word, uh, evolution. Anyway, and specialized
1686.42s - 1693.38s |  knowledge can introduce security flaws. Frequent updates, dependencies, and gaps in testing further
1693.38s - 1698.18s |  contribute to these risks. And vulnerabilities stem from inadequate reviews as we know of, poor
1698.18s - 1705.78s |  implementation, dependency issues, backdoors, insufficient security practices. As you can tell,
1705.78s - 1710.10s |  there's a lot of issues. It's been the issues we've had with traditional, but now it seems like
1710.10s - 1715.86s |  an extra layer of everything. Because there's rapid development, there's misconfigured models,
1715.86s - 1721.14s |  and now you also have to worry about social engineering as well. These all increase the risks.
1721.14s - 1726.26s |  And to be honest, I think our next talk at DEF CON, and don't anyone steal this idea, okay, I'm
1726.26s - 1731.06s |  just putting it out there, is AI-assisted code reviews. That sounds really interesting because
1731.06s - 1737.22s |  that's emerging right now. Anyway, as you see, this is a long list and it's definitely missing
1737.22s - 1743.06s |  other things probably on this list. But reporting AI vulnerabilities is incredibly tough because it's
1743.06s - 1749.62s |  unclear who owns what and the responsibilities. Not just that, but the technical complexity
1750.26s - 1754.74s |  and the need for specialized knowledge. Of course, we have to worry about legal fears,
1754.74s - 1759.14s |  reputation risk, and coordination issues that tend to complicate the process,
1759.14s - 1764.18s |  while inadequate reporting channels and resource constraints really do discourage thorough
1764.18s - 1769.62s |  reporting. This has been a problem for us at Hidden Layer a few times that we actually had to get
1769.62s - 1776.26s |  CISA involved. And they were wonderful. They helped us get in touch with people so then we
1776.26s - 1782.98s |  could get some serious vulnerabilities patched. So, quick summary, is that AI vulnerabilities
1782.98s - 1788.58s |  are complex and distinct. And the threats, like prompt injections and data poisoning,
1788.58s - 1794.66s |  are out there. And there's rapid developments and reliance on open source frameworks. It introduces
1794.66s - 1799.78s |  all these type of risks. So, managing these vulnerabilities demands improving reporting,
1799.78s - 1805.54s |  but also stronger partnerships, including with CISA and other organizations, especially when it
1805.54s - 1812.74s |  comes to disclosure. So, we do have a call to action. We're all in this together. That's right.
1812.74s - 1817.78s |  We all can take a step forward and do something together. And the call to action is to partner
1817.78s - 1823.62s |  with us. We need to enhance security for AI vulnerability disclosure and ensure comprehensive
1823.62s - 1829.46s |  protection against vulnerabilities. So, we'll be right over here for Q&A. But if you want to get
1829.46s - 1836.90s |  involved with us, do let us know. I know we're out of time. But yeah. Anyway, thank you so much
1836.90s - 1840.90s |  for having us. Thank you, AppSec Village.


