# MediaChunks

chunk_text:
all those things need context. 2225.26s - 2227.26s | And so if your organization 2227.26s - 2229.26s | does not have its fundamentals, 2229.26s - 2231.26s | correct, around asset management, 2231.26s - 2233.26s | et cetera, your AI 2233.26s - 2235.26s | is not going to generate 2235.26s - 2238.26s | anything better than you could 2238.26s - 2239.26s | or at scale. 2239.26s - 2241.26s | So that's sort of how it feels. 2241.26s - 2243.26s | Get your fundamentals correct first. 2243.26s - 2247.86s | I think not only with the promise of AI 2247.86s - 2249.86s | we'll get some cool new feature 2249.86s - 2250.86s | for the security program, 2250.86s - 2252.86s | it brings a lot of risks 2252.86s - 2254.86s | into our domain as well, right? 2254.86s - 2256.86s | I mean, there's a psychological effect 2256.86s - 2259.86s | of a developer copy paste a code 2259.86s - 2261.86s | from a, you know, 2261.86s - 2263.86s | gen AI generated code. 2263.86s - 2264.86s | It feels like it's safe. 2264.86s - 2265.86s | It's like it takes it 2265.86s - 2267.86s | from a very safe place, 2267.86s - 2269.86s | but no one can assure you 2269.86s - 2270.86s | this code won't hurt you 2270.86s - 2272.86s | more than any other piece of code. 2272.86s - 2274.86s | And the number of lines of code 2274.86s - 2276.86s | will exponentially grow 2276.86s - 2277.86s | with what brings into 2277.86s - 2278.86s | the new application, et cetera. 2278.86s - 2280.86s | And we need to put the right gates 2280.86s - 2282.86s | for this AI copy paste, 2282.86s - 2285.86s | just like when the internet started 2285.86s - 2286.86s | and copy paste from Google 2286.86s - 2288.86s | pieces of code were being 2288.86s - 2290.86s | somehow regulated. 2290.86s - 2291.86s | You can't really stop that 2291.86s - 2292.86s | or control that, 2292.86s - 2293.86s | but you need to have 2293.86s - 2296.86s | the right policies to handle that. 2296.86s - 2297.86s | Thanks. 2297.86s - 2298.86s | Sorry, I just wanted to keep 2298.86s - 2301.86s | some time for questions. 2301.86s - 2302.86s | Go ahead. 2302.86s - 2304.86s | Just one last remark. 2304.86s - 2305.86s | Like everybody's worried 2305.86s -


chunk_text:
- 2269.86s | but no one can assure you 2269.86s - 2270.86s | this code won't hurt you 2270.86s - 2272.86s | more than any other piece of code. 2272.86s - 2274.86s | And the number of lines of code 2274.86s - 2276.86s | will exponentially grow 2276.86s - 2277.86s | with what brings into 2277.86s - 2278.86s | the new application, et cetera. 2278.86s - 2280.86s | And we need to put the right gates 2280.86s - 2282.86s | for this AI copy paste, 2282.86s - 2285.86s | just like when the internet started 2285.86s - 2286.86s | and copy paste from Google 2286.86s - 2288.86s | pieces of code were being 2288.86s - 2290.86s | somehow regulated. 2290.86s - 2291.86s | You can't really stop that 2291.86s - 2292.86s | or control that, 2292.86s - 2293.86s | but you need to have 2293.86s - 2296.86s | the right policies to handle that. 2296.86s - 2297.86s | Thanks. 2297.86s - 2298.86s | Sorry, I just wanted to keep 2298.86s - 2301.86s | some time for questions. 2301.86s - 2302.86s | Go ahead. 2302.86s - 2304.86s | Just one last remark. 2304.86s - 2305.86s | Like everybody's worried 2305.86s - 2306.86s | about security risk 2306.86s - 2308.86s | coming from AI generated code. 2308.86s - 2309.86s | I'm more worried about 2309.86s - 2312.86s | the legal risks. 2312.86s - 2314.86s | That's fair too. 2314.86s - 2315.86s | Sorry, I want to keep 2315.86s - 2316.86s | a little bit more time for questions, 2316.86s - 2318.86s | but I think when you let 2318.86s - 2319.86s | four AppSec people sit, 2319.86s - 2322.86s | we can talk for hours. 2323.86s - 2328.58s | But if anyone has any questions, 2328.58s - 2333.78s | please use the mic. 2333.78s - 2335.78s | So my question's for Shahar. 2335.78s - 2336.78s | What's up? 2336.78s - 2337.78s | So I'm an electrical engineer 2337.78s - 2339.78s | and I did software development 2339.78s - 2341.78s | for Department of Defense 2341.78s - 2343.78s | application for avionics systems 2343.78s - 2344.78s | and weapons systems. 2344.78s - 2345.78s | So our software killed people. 2345.78s - 2346.78s | We want to make sure 2346.78s - 2347.78s | it's killed the right people, 2347.78s - 2349.78s | not the wrong people. 2349.78s - 2352.78s | When I switched over


chunk_text:
2306.86s | about security risk 2306.86s - 2308.86s | coming from AI generated code. 2308.86s - 2309.86s | I'm more worried about 2309.86s - 2312.86s | the legal risks. 2312.86s - 2314.86s | That's fair too. 2314.86s - 2315.86s | Sorry, I want to keep 2315.86s - 2316.86s | a little bit more time for questions, 2316.86s - 2318.86s | but I think when you let 2318.86s - 2319.86s | four AppSec people sit, 2319.86s - 2322.86s | we can talk for hours. 2323.86s - 2328.58s | But if anyone has any questions, 2328.58s - 2333.78s | please use the mic. 2333.78s - 2335.78s | So my question's for Shahar. 2335.78s - 2336.78s | What's up? 2336.78s - 2337.78s | So I'm an electrical engineer 2337.78s - 2339.78s | and I did software development 2339.78s - 2341.78s | for Department of Defense 2341.78s - 2343.78s | application for avionics systems 2343.78s - 2344.78s | and weapons systems. 2344.78s - 2345.78s | So our software killed people. 2345.78s - 2346.78s | We want to make sure 2346.78s - 2347.78s | it's killed the right people, 2347.78s - 2349.78s | not the wrong people. 2349.78s - 2352.78s | When I switched over to IT, 2352.78s - 2354.78s | totally different level of, 2354.78s - 2356.78s | and I'm professional, 2356.78s - 2357.78s | professional developer was 2357.78s - 2360.78s | much lower bar than what, 2360.78s - 2361.78s | you know, I'm not dealing 2361.78s - 2362.78s | with engineers. 2362.78s - 2363.78s | I'm dealing with sometimes 2363.78s - 2366.78s | people I would call 2366.78s - 2370.16s | basically got their coding degree 2370.16s - 2372.16s | from some, you know, 2372.16s - 2373.16s | fly-by-night place 2373.16s - 2375.16s | and stuff and I can program. 2375.16s - 2376.16s | So my question is, 2376.16s - 2377.16s | is how do we get 2377.16s - 2378.16s | on the engineering side? 2378.16s - 2379.16s | Because to me, 2379.16s - 2380.16s | software vulnerability 2380.16s - 2382.16s | is nothing more than a bug. 2382.16s - 2384.16s | It's a bad coding practice 2384.16s - 2387.16s | that the developer programmed in. 2387.16s - 2388.16s | They don't know about it. 2388.16s - 2389.16s | They don't understand about it, 2389.16s - 2391.16s | but it's going to cause a problem. 2391.16s - 2393.16s | So how do we


chunk_text:
to IT, 2352.78s - 2354.78s | totally different level of, 2354.78s - 2356.78s | and I'm professional, 2356.78s - 2357.78s | professional developer was 2357.78s - 2360.78s | much lower bar than what, 2360.78s - 2361.78s | you know, I'm not dealing 2361.78s - 2362.78s | with engineers. 2362.78s - 2363.78s | I'm dealing with sometimes 2363.78s - 2366.78s | people I would call 2366.78s - 2370.16s | basically got their coding degree 2370.16s - 2372.16s | from some, you know, 2372.16s - 2373.16s | fly-by-night place 2373.16s - 2375.16s | and stuff and I can program. 2375.16s - 2376.16s | So my question is, 2376.16s - 2377.16s | is how do we get 2377.16s - 2378.16s | on the engineering side? 2378.16s - 2379.16s | Because to me, 2379.16s - 2380.16s | software vulnerability 2380.16s - 2382.16s | is nothing more than a bug. 2382.16s - 2384.16s | It's a bad coding practice 2384.16s - 2387.16s | that the developer programmed in. 2387.16s - 2388.16s | They don't know about it. 2388.16s - 2389.16s | They don't understand about it, 2389.16s - 2391.16s | but it's going to cause a problem. 2391.16s - 2393.16s | So how do we get to a point 2393.16s - 2395.16s | where our developers more, 2395.16s - 2397.16s | and I don't mean to mean professionalism 2397.16s - 2398.16s | or, you know, 2398.16s - 2399.16s | that they're not acting correctly, 2399.16s - 2401.16s | but they don't understand 2401.16s - 2403.16s | the coding practices 2403.16s - 2405.16s | as well as they should be, 2405.16s - 2406.16s | in my opinion. 2406.16s - 2407.16s | I mean, this is all, 2407.16s - 2408.16s | I'm very biased towards 2408.16s - 2410.16s | being an engineer and stuff, 2410.16s - 2412.16s | but at the same time, 2412.16s - 2413.16s | our IT industry, 2413.16s - 2415.16s | and we see a lot of breaches 2415.16s - 2416.16s | and people are getting hurt 2416.16s - 2417.16s | actually hurt, 2417.16s - 2418.16s | maybe not killed, 2418.16s - 2421.16s | although maybe. 2421.16s - 2422.16s | How do we bring that 2422.16s - 2423.16s | from an engineering standpoint? 2423.16s - 2424.16s | Because you're saying, 2424.16s - 2425.16s | well, we need to maybe not 2425.16s - 2426.16s | shift left so much. 2426.16s - 2428.16s | To me,


chunk_text:
get to a point 2393.16s - 2395.16s | where our developers more, 2395.16s - 2397.16s | and I don't mean to mean professionalism 2397.16s - 2398.16s | or, you know, 2398.16s - 2399.16s | that they're not acting correctly, 2399.16s - 2401.16s | but they don't understand 2401.16s - 2403.16s | the coding practices 2403.16s - 2405.16s | as well as they should be, 2405.16s - 2406.16s | in my opinion. 2406.16s - 2407.16s | I mean, this is all, 2407.16s - 2408.16s | I'm very biased towards 2408.16s - 2410.16s | being an engineer and stuff, 2410.16s - 2412.16s | but at the same time, 2412.16s - 2413.16s | our IT industry, 2413.16s - 2415.16s | and we see a lot of breaches 2415.16s - 2416.16s | and people are getting hurt 2416.16s - 2417.16s | actually hurt, 2417.16s - 2418.16s | maybe not killed, 2418.16s - 2421.16s | although maybe. 2421.16s - 2422.16s | How do we bring that 2422.16s - 2423.16s | from an engineering standpoint? 2423.16s - 2424.16s | Because you're saying, 2424.16s - 2425.16s | well, we need to maybe not 2425.16s - 2426.16s | shift left so much. 2426.16s - 2428.16s | To me, it is the developer 2428.16s - 2430.16s | who is the core problem 2430.16s - 2432.16s | and not necessarily 2432.16s - 2433.16s | the app sec people 2433.16s - 2434.16s | who are going through 2434.16s - 2435.16s | and say, hey, 2435.16s - 2436.16s | you're finding these problems. 2436.16s - 2437.16s | These are feedback mechanisms 2437.16s - 2438.16s | that say the developer 2438.16s - 2439.16s | is not developing 2439.16s - 2442.16s | to a good practice. 2442.16s - 2443.16s | So what do you say about that? 2443.16s - 2444.16s | Yeah, that's very true. 2444.16s - 2446.16s | So first of all, 2446.16s - 2447.16s | thanks for your question. 2447.16s - 2449.16s | And, you know, 2449.16s - 2450.16s | life and death, 2450.16s - 2452.16s | it's something that, 2452.16s - 2453.16s | you know, 2453.16s - 2455.16s | it's part of a defense system. 2455.16s - 2458.16s | Me coming from similar areas 2458.16s - 2459.16s | in my history, 2459.16s - 2460.16s | I know exactly 2460.16s - 2461.16s | what you're talking about. 2461.16s - 2462.16s | But in these domains, 2462.16s - 2464.16s | probably want to


chunk_text:
it is the developer 2428.16s - 2430.16s | who is the core problem 2430.16s - 2432.16s | and not necessarily 2432.16s - 2433.16s | the app sec people 2433.16s - 2434.16s | who are going through 2434.16s - 2435.16s | and say, hey, 2435.16s - 2436.16s | you're finding these problems. 2436.16s - 2437.16s | These are feedback mechanisms 2437.16s - 2438.16s | that say the developer 2438.16s - 2439.16s | is not developing 2439.16s - 2442.16s | to a good practice. 2442.16s - 2443.16s | So what do you say about that? 2443.16s - 2444.16s | Yeah, that's very true. 2444.16s - 2446.16s | So first of all, 2446.16s - 2447.16s | thanks for your question. 2447.16s - 2449.16s | And, you know, 2449.16s - 2450.16s | life and death, 2450.16s - 2452.16s | it's something that, 2452.16s - 2453.16s | you know, 2453.16s - 2455.16s | it's part of a defense system. 2455.16s - 2458.16s | Me coming from similar areas 2458.16s - 2459.16s | in my history, 2459.16s - 2460.16s | I know exactly 2460.16s - 2461.16s | what you're talking about. 2461.16s - 2462.16s | But in these domains, 2462.16s - 2464.16s | probably want to get top developers 2464.16s - 2466.16s | for writing the code. 2466.16s - 2469.16s | And I didn't mean that 2469.16s - 2470.16s | app sec people 2470.16s - 2471.16s | would go and clean up the mess 2471.16s - 2473.16s | after the developers. 2473.16s - 2475.16s | This was not my intention. 2475.16s - 2476.16s | But if you have 2476.16s - 2478.16s | certain practices 2478.16s - 2479.16s | that you want to make sure 2479.16s - 2480.16s | are happening 2480.16s - 2481.16s | in your environment, 2481.16s - 2482.16s | you need to have 2482.16s - 2483.16s | the right policies 2483.16s - 2484.16s | that once the code 2484.16s - 2485.16s | is being compiled, 2485.16s - 2487.16s | the code is being built, 2487.16s - 2489.16s | the alerts for these policies 2489.16s - 2490.16s | will be back 2490.16s - 2491.16s | to the developers. 2491.16s - 2492.16s | I'll give you one example, 2492.16s - 2493.16s | something that we've practiced 2493.16s - 2494.16s | recently. 2494.16s - 2495.16s | Some developers, 2495.16s - 2496.16s | that may look 2496.16s - 2499.16s | a very innocent thing to do, 2499.16s -


chunk_text:
get top developers 2464.16s - 2466.16s | for writing the code. 2466.16s - 2469.16s | And I didn't mean that 2469.16s - 2470.16s | app sec people 2470.16s - 2471.16s | would go and clean up the mess 2471.16s - 2473.16s | after the developers. 2473.16s - 2475.16s | This was not my intention. 2475.16s - 2476.16s | But if you have 2476.16s - 2478.16s | certain practices 2478.16s - 2479.16s | that you want to make sure 2479.16s - 2480.16s | are happening 2480.16s - 2481.16s | in your environment, 2481.16s - 2482.16s | you need to have 2482.16s - 2483.16s | the right policies 2483.16s - 2484.16s | that once the code 2484.16s - 2485.16s | is being compiled, 2485.16s - 2487.16s | the code is being built, 2487.16s - 2489.16s | the alerts for these policies 2489.16s - 2490.16s | will be back 2490.16s - 2491.16s | to the developers. 2491.16s - 2492.16s | I'll give you one example, 2492.16s - 2493.16s | something that we've practiced 2493.16s - 2494.16s | recently. 2494.16s - 2495.16s | Some developers, 2495.16s - 2496.16s | that may look 2496.16s - 2499.16s | a very innocent thing to do, 2499.16s - 2501.16s | are referencing packages 2501.16s - 2502.16s | from third party open sources 2502.16s - 2504.16s | they bring in directly 2504.16s - 2507.16s | and not from the manifest file. 2507.16s - 2508.16s | It's hard to find that 2508.16s - 2509.16s | and it seems like 2509.16s - 2511.16s | a very innocent thing to do. 2511.16s - 2512.16s | However, 2512.16s - 2513.16s | when you have patched 2513.16s - 2514.16s | to the latest version 2514.16s - 2515.16s | that has no vulnerabilities, 2515.16s - 2517.16s | these ghost packages 2517.16s - 2518.16s | or phantom packages 2518.16s - 2520.16s | will be left behind. 2520.16s - 2521.16s | So now you need to make sure 2521.16s - 2522.16s | when you're building the software, 2522.16s - 2523.16s | there's a warning 2523.16s - 2525.16s | for this particular developer. 2525.16s - 2526.16s | Hey, David, 2526.16s - 2527.16s | go ahead and make sure 2527.16s - 2528.16s | that you fix 2528.16s - 2529.16s | in this line of code, 2529.16s - 2531.16s | this ghost package reference 2531.16s - 2532.16s | and fix it. 2532.16s - 2534.16s | And not just that generic,


chunk_text:
2501.16s | are referencing packages 2501.16s - 2502.16s | from third party open sources 2502.16s - 2504.16s | they bring in directly 2504.16s - 2507.16s | and not from the manifest file. 2507.16s - 2508.16s | It's hard to find that 2508.16s - 2509.16s | and it seems like 2509.16s - 2511.16s | a very innocent thing to do. 2511.16s - 2512.16s | However, 2512.16s - 2513.16s | when you have patched 2513.16s - 2514.16s | to the latest version 2514.16s - 2515.16s | that has no vulnerabilities, 2515.16s - 2517.16s | these ghost packages 2517.16s - 2518.16s | or phantom packages 2518.16s - 2520.16s | will be left behind. 2520.16s - 2521.16s | So now you need to make sure 2521.16s - 2522.16s | when you're building the software, 2522.16s - 2523.16s | there's a warning 2523.16s - 2525.16s | for this particular developer. 2525.16s - 2526.16s | Hey, David, 2526.16s - 2527.16s | go ahead and make sure 2527.16s - 2528.16s | that you fix 2528.16s - 2529.16s | in this line of code, 2529.16s - 2531.16s | this ghost package reference 2531.16s - 2532.16s | and fix it. 2532.16s - 2534.16s | And not just that generic, 2534.16s - 2535.16s | the industry settled 2535.16s - 2536.16s | for a generic explanation. 2536.16s - 2537.16s | Hey, 2537.16s - 2539.16s | you may have a SQL injection 2539.16s - 2540.16s | in your code. 2540.16s - 2542.16s | It needs to be super specific 2542.16s - 2544.16s | in how you structure 2544.16s - 2545.16s | your build processes 2545.16s - 2546.16s | in order to direct 2546.16s - 2547.16s | the right person 2547.16s - 2549.16s | with the right actionable item. 2549.16s - 2550.16s | And next time, 2550.16s - 2551.16s | he'll care about it 2551.16s - 2552.16s | because he sees 2552.16s - 2554.16s | it's very concrete for him. 2554.16s - 2555.16s | I hope that helps. 2555.16s - 2556.16s | Yeah, 2556.16s - 2557.16s | I'm just going to add 2557.16s - 2558.16s | one more line. 2558.16s - 2559.16s | I don't think developers 2559.16s - 2560.16s | are the problem. 2560.16s - 2561.16s | The real problem 2561.16s - 2563.16s | is in university education. 2563.16s - 2564.16s | We have courses 2564.16s - 2566.16s | to learn algorithms, 2566.16s - 2567.16s | different programming languages, 2567.16s - 2568.16s


chunk_text:
2534.16s - 2535.16s | the industry settled 2535.16s - 2536.16s | for a generic explanation. 2536.16s - 2537.16s | Hey, 2537.16s - 2539.16s | you may have a SQL injection 2539.16s - 2540.16s | in your code. 2540.16s - 2542.16s | It needs to be super specific 2542.16s - 2544.16s | in how you structure 2544.16s - 2545.16s | your build processes 2545.16s - 2546.16s | in order to direct 2546.16s - 2547.16s | the right person 2547.16s - 2549.16s | with the right actionable item. 2549.16s - 2550.16s | And next time, 2550.16s - 2551.16s | he'll care about it 2551.16s - 2552.16s | because he sees 2552.16s - 2554.16s | it's very concrete for him. 2554.16s - 2555.16s | I hope that helps. 2555.16s - 2556.16s | Yeah, 2556.16s - 2557.16s | I'm just going to add 2557.16s - 2558.16s | one more line. 2558.16s - 2559.16s | I don't think developers 2559.16s - 2560.16s | are the problem. 2560.16s - 2561.16s | The real problem 2561.16s - 2563.16s | is in university education. 2563.16s - 2564.16s | We have courses 2564.16s - 2566.16s | to learn algorithms, 2566.16s - 2567.16s | different programming languages, 2567.16s - 2568.16s | databases, 2568.16s - 2569.16s | but none of those 2569.16s - 2570.16s | cover security 2570.16s - 2572.16s | as part of that technology, right? 2572.16s - 2574.16s | So when that will start happening, 2574.16s - 2575.16s | we'll see fewer 2575.16s - 2577.16s | and fewer vulnerabilities. 2577.16s - 2578.16s | But at the same time, 2578.16s - 2579.16s | what about the 2579.16s - 2580.16s | gazillion legacy software 2580.16s - 2582.16s | which is out there? 2582.16s - 2583.16s | That's going to take 2583.16s - 2585.16s | a lot of time. 2585.16s - 2586.16s | Thanks. 2586.16s - 2587.16s | I'm getting the 2587.16s - 2588.16s | two-minute part. 2588.16s - 2590.16s | But like I said, 2590.16s - 2591.16s | we can talk forever, 2591.16s - 2592.16s | but we can, 2592.16s - 2593.16s | you know, 2593.16s - 2594.16s | we will be outside 2594.16s - 2595.16s | and maybe we can talk 2595.16s - 2596.16s | to people if they're... 2596.16s - 2597.16s | So Shahar, 2597.16s - 2598.16s | you had mentioned, 2598.16s - 2599.16s | you know, along lines 2599.16s - 2600.16s | our investments 2600.16s - 2601.16s |


chunk_text:
| databases, 2568.16s - 2569.16s | but none of those 2569.16s - 2570.16s | cover security 2570.16s - 2572.16s | as part of that technology, right? 2572.16s - 2574.16s | So when that will start happening, 2574.16s - 2575.16s | we'll see fewer 2575.16s - 2577.16s | and fewer vulnerabilities. 2577.16s - 2578.16s | But at the same time, 2578.16s - 2579.16s | what about the 2579.16s - 2580.16s | gazillion legacy software 2580.16s - 2582.16s | which is out there? 2582.16s - 2583.16s | That's going to take 2583.16s - 2585.16s | a lot of time. 2585.16s - 2586.16s | Thanks. 2586.16s - 2587.16s | I'm getting the 2587.16s - 2588.16s | two-minute part. 2588.16s - 2590.16s | But like I said, 2590.16s - 2591.16s | we can talk forever, 2591.16s - 2592.16s | but we can, 2592.16s - 2593.16s | you know, 2593.16s - 2594.16s | we will be outside 2594.16s - 2595.16s | and maybe we can talk 2595.16s - 2596.16s | to people if they're... 2596.16s - 2597.16s | So Shahar, 2597.16s - 2598.16s | you had mentioned, 2598.16s - 2599.16s | you know, along lines 2599.16s - 2600.16s | our investments 2600.16s - 2601.16s | into security 2601.16s - 2602.16s | really haven't worked. 2602.16s - 2603.16s | We keep on seeing 2603.16s - 2604.16s | the same faults, right? 2604.16s - 2605.16s | So in my experience, 2605.16s - 2606.16s | a lot of times 2606.16s - 2608.16s | I'm seeing that 2608.16s - 2609.16s | I think our investments 2609.16s - 2610.16s | can work, 2610.16s - 2611.16s | but we have a tendency 2611.16s - 2612.16s | of promoting people 2612.16s - 2614.16s | who are top developers. 2614.16s - 2615.16s | They stop developing 2615.16s - 2616.16s | because we push them 2616.16s - 2617.16s | into management. 2617.16s - 2618.16s | And I think that 2618.16s - 2619.16s | is one issue. 2619.16s - 2620.16s | And the other issue 2620.16s - 2621.16s | I've seen is that 2621.16s - 2623.16s | once an organization 2623.16s - 2624.16s | gets to a point 2624.16s - 2625.16s | where they have 2625.16s - 2626.16s | very little vulnerabilities, 2626.16s - 2627.16s | they've basically 2627.16s - 2628.16s | got rid of a lot 2628.16s - 2629.16s | of their vulnerability 2629.16s - 2630.16s | technical debt, 2630.16s - 2631.16s | they


chunk_text:
into security 2601.16s - 2602.16s | really haven't worked. 2602.16s - 2603.16s | We keep on seeing 2603.16s - 2604.16s | the same faults, right? 2604.16s - 2605.16s | So in my experience, 2605.16s - 2606.16s | a lot of times 2606.16s - 2608.16s | I'm seeing that 2608.16s - 2609.16s | I think our investments 2609.16s - 2610.16s | can work, 2610.16s - 2611.16s | but we have a tendency 2611.16s - 2612.16s | of promoting people 2612.16s - 2614.16s | who are top developers. 2614.16s - 2615.16s | They stop developing 2615.16s - 2616.16s | because we push them 2616.16s - 2617.16s | into management. 2617.16s - 2618.16s | And I think that 2618.16s - 2619.16s | is one issue. 2619.16s - 2620.16s | And the other issue 2620.16s - 2621.16s | I've seen is that 2621.16s - 2623.16s | once an organization 2623.16s - 2624.16s | gets to a point 2624.16s - 2625.16s | where they have 2625.16s - 2626.16s | very little vulnerabilities, 2626.16s - 2627.16s | they've basically 2627.16s - 2628.16s | got rid of a lot 2628.16s - 2629.16s | of their vulnerability 2629.16s - 2630.16s | technical debt, 2630.16s - 2631.16s | they then say, 2631.16s - 2632.16s | okay, this team, 2632.16s - 2633.16s | experience team, 2633.16s - 2634.16s | we don't need you anymore. 2634.16s - 2635.16s | We're going to outsource 2635.16s - 2636.16s | to a much lower cost. 2636.16s - 2637.16s | So it's a cycle 2637.16s - 2638.16s | that keeps occurring. 2638.16s - 2639.16s | So I don't think 2639.16s - 2640.16s | it's the investment. 2640.16s - 2641.16s | I think it's the lack 2641.16s - 2642.16s | of continuing investment 2642.16s - 2643.16s | into security. 2643.16s - 2644.16s | And when it comes to tools, 2644.16s - 2645.16s | I love what you said, 2646.16s - 2647.16s | about putting 2647.16s - 2648.16s | fandom packages in. 2648.16s - 2649.16s | I hate that phrase. 2650.16s - 2651.16s | It drives me absolutely nuts 2651.16s - 2652.16s | to use dependency management. 2652.16s - 2653.16s | If your teams aren't using 2653.16s - 2654.16s | dependency management, 2654.16s - 2655.16s | shame on you. 2657.16s - 2658.16s | Yeah, thank you so much. 2658.16s - 2659.16s | I think that I can 2659.16s - 2660.16s | very much relate 2660.16s - 2661.16s


chunk_text:
then say, 2631.16s - 2632.16s | okay, this team, 2632.16s - 2633.16s | experience team, 2633.16s - 2634.16s | we don't need you anymore. 2634.16s - 2635.16s | We're going to outsource 2635.16s - 2636.16s | to a much lower cost. 2636.16s - 2637.16s | So it's a cycle 2637.16s - 2638.16s | that keeps occurring. 2638.16s - 2639.16s | So I don't think 2639.16s - 2640.16s | it's the investment. 2640.16s - 2641.16s | I think it's the lack 2641.16s - 2642.16s | of continuing investment 2642.16s - 2643.16s | into security. 2643.16s - 2644.16s | And when it comes to tools, 2644.16s - 2645.16s | I love what you said, 2646.16s - 2647.16s | about putting 2647.16s - 2648.16s | fandom packages in. 2648.16s - 2649.16s | I hate that phrase. 2650.16s - 2651.16s | It drives me absolutely nuts 2651.16s - 2652.16s | to use dependency management. 2652.16s - 2653.16s | If your teams aren't using 2653.16s - 2654.16s | dependency management, 2654.16s - 2655.16s | shame on you. 2657.16s - 2658.16s | Yeah, thank you so much. 2658.16s - 2659.16s | I think that I can 2659.16s - 2660.16s | very much relate 2660.16s - 2661.16s | to what you said 2661.16s - 2662.16s | about promotions. 2662.16s - 2663.16s | And this is something 2663.16s - 2664.16s | that I learned 2664.16s - 2665.16s | with my career 2665.16s - 2666.16s | building teams. 2666.16s - 2667.16s | If you have an amazing developer 2667.16s - 2668.16s | and you take him 2668.16s - 2669.16s | and be a manager 2669.16s - 2670.16s | of a team, 2670.16s - 2671.16s | you lose twice. 2671.16s - 2672.16s | You just lost 2672.16s - 2673.16s | your best developer 2673.16s - 2674.16s | and you got 2674.16s - 2675.16s | an okay manager 2675.16s - 2676.16s | right there. 2676.16s - 2678.16s | So he needs to be empowered, 2678.16s - 2679.16s | get the right compensation, 2679.16s - 2681.16s | to continue doing 2681.16s - 2682.16s | what he loves to do 2682.16s - 2684.16s | and be a great developer 2684.16s - 2685.16s | on whatever level 2685.16s - 2686.16s | he needs to be, 2686.16s - 2688.16s | but keeps to that profession. 2688.16s - 2690.16s | That's my point of view. 2690.16s - 2691.16s | It's an HR problem. 2694.16s - 2695.16s


chunk_text:
| to what you said 2661.16s - 2662.16s | about promotions. 2662.16s - 2663.16s | And this is something 2663.16s - 2664.16s | that I learned 2664.16s - 2665.16s | with my career 2665.16s - 2666.16s | building teams. 2666.16s - 2667.16s | If you have an amazing developer 2667.16s - 2668.16s | and you take him 2668.16s - 2669.16s | and be a manager 2669.16s - 2670.16s | of a team, 2670.16s - 2671.16s | you lose twice. 2671.16s - 2672.16s | You just lost 2672.16s - 2673.16s | your best developer 2673.16s - 2674.16s | and you got 2674.16s - 2675.16s | an okay manager 2675.16s - 2676.16s | right there. 2676.16s - 2678.16s | So he needs to be empowered, 2678.16s - 2679.16s | get the right compensation, 2679.16s - 2681.16s | to continue doing 2681.16s - 2682.16s | what he loves to do 2682.16s - 2684.16s | and be a great developer 2684.16s - 2685.16s | on whatever level 2685.16s - 2686.16s | he needs to be, 2686.16s - 2688.16s | but keeps to that profession. 2688.16s - 2690.16s | That's my point of view. 2690.16s - 2691.16s | It's an HR problem. 2694.16s - 2695.16s | Thanks everyone. 2695.16s - 2696.16s | Thank you.


chunk_text:
{ "webpage_url": "https://www.youtube.com/watch?v=GAEOBhsqJI4", "title": "DEF COPN 32 - 0 0 0 0 Day Exploiting Localhost APIs From The Browser - Avi Lumel skyGal Elbaz", "description": "Browser-based attacks are not new in the malicious landscape of attack patterns. Browsers remain a popular infiltration method for attackers. \n\nWhile seemingly local, services running on localhost are accessible to the browser using a flaw we found, exposing the ports on the localhost network interface, and leaving the floodgates ajar to remote network attacks.\n\nIn this live demo and attack simulation we\u2019ll unveil a zero-day vulnerability (still under responsible disclosure) in Chrome and other browsers, and how we use the 0-day to attack developers behind firewalls. We will demonstrate remote code execution on a wildly popular open-source platform serving millions in the data engineering ecosystem, that seems to run on localhost.\n\nIn our talk, we will present novel attack techniques, targeting developers and employees within an organization, that are behind firewalls. This will be a first-ever deep dive into this newly discovered zero-day vulnerability.", "channel_url": "https://www.youtube.com/channel/UC6Om9kAkl32dWlDSNlDS9Iw", "duration": 1634, "channel": "DEFCONConference", "uploader": "DEFCONConference", "upload_date": "20241016" } 0.62s - 2.62s | This text was transcribed using whisper model: large-v2 Hello everyone 4.82s - 9.02s | Hi everyone. Thank you for attending our talk zero zero zero day 9.02s - 14.02s | I know if you've heard or saw the news it got a little bit of coverage 14.02s - 20.38s | But this is the first time that we're actually disclosing the technical details. It's going to be fun. It's going to be risky and 21.04s - 29.22s | Let's start. So I'm Gal. I'm the co-founder and CTO of oligo security with me is Avi our number one ninja and 29.22s - 30.74s | And 30.74s - 33.38s | Really quickly about ourselves before we start and begin 34.50s - 38.03s | Hi, I'm Avi 38.75s - 43.51s | Security researcher in the city office of oligo. I have around 10 years of experience in 44.03s - 48.07s | research and development mostly researching AI these days and 48.91s - 53.23s | I love to climb mountains. So if you want to join me on the trip, let me know and 54.03s - 56.03s | With me in my is my awesome CTO 57.03s - 61.07s | Thank You Avi I am paying you to say good things about me so 61.63s - 65.87s | Keep up with


chunk_text:
for attending our talk zero zero zero day 9.02s - 14.02s | I know if you've heard or saw the news it got a little bit of coverage 14.02s - 20.38s | But this is the first time that we're actually disclosing the technical details. It's going to be fun. It's going to be risky and 21.04s - 29.22s | Let's start. So I'm Gal. I'm the co-founder and CTO of oligo security with me is Avi our number one ninja and 29.22s - 30.74s | And 30.74s - 33.38s | Really quickly about ourselves before we start and begin 34.50s - 38.03s | Hi, I'm Avi 38.75s - 43.51s | Security researcher in the city office of oligo. I have around 10 years of experience in 44.03s - 48.07s | research and development mostly researching AI these days and 48.91s - 53.23s | I love to climb mountains. So if you want to join me on the trip, let me know and 54.03s - 56.03s | With me in my is my awesome CTO 57.03s - 61.07s | Thank You Avi I am paying you to say good things about me so 61.63s - 65.87s | Keep up with that. So I am the founder and city of oligo at 10 years of experience 66.63s - 73.39s | Checkpoint hacking breaking stuff since I remember myself and not like Avi. I'm famous for being handsome or think 73.99s - 78.67s | Thinking like that and I'm really good at looking at mountains in my Windows computer 79.07s - 82.47s | So a quick agenda what we're going to discuss today 82.47s - 87.91s | What is the concept of port scanning and how it all started from 18 years ago? 87.91s - 94.75s | We're very old issue that just keep on recurring and recurring along the years and we're going to deep dive into the history 95.11s - 98.51s | What is the actual private network access this? 99.59s - 101.59s | try to do 101.75s - 104.47s | standout when all of the browsers should be 105.11s - 111.11s | Using but we'll see along the years what actually happened. Then we're going to deep dive into the actual 111.83s - 115.67s | Details of the zero zero zero day internals itself 115.67s - 123.63s | And then we're going to demonstrate how we're leveraging that in


chunk_text:
that. So I am the founder and city of oligo at 10 years of experience 66.63s - 73.39s | Checkpoint hacking breaking stuff since I remember myself and not like Avi. I'm famous for being handsome or think 73.99s - 78.67s | Thinking like that and I'm really good at looking at mountains in my Windows computer 79.07s - 82.47s | So a quick agenda what we're going to discuss today 82.47s - 87.91s | What is the concept of port scanning and how it all started from 18 years ago? 87.91s - 94.75s | We're very old issue that just keep on recurring and recurring along the years and we're going to deep dive into the history 95.11s - 98.51s | What is the actual private network access this? 99.59s - 101.59s | try to do 101.75s - 104.47s | standout when all of the browsers should be 105.11s - 111.11s | Using but we'll see along the years what actually happened. Then we're going to deep dive into the actual 111.83s - 115.67s | Details of the zero zero zero day internals itself 115.67s - 123.63s | And then we're going to demonstrate how we're leveraging that in order to RCE private internal applications that today are considered 124.91s - 131.27s | Private and in the end we're going to really deep dive into how each and every browser actually 131.75s - 134.75s | was dealing with that and the responsible disclosure and 135.67s - 139.67s | Where we are right now with that so in order to really start 139.67s - 145.39s | I'm gonna pass it on to awesome Avi because he's the actual smart guy that did all of that and 146.39s - 149.72s | Let's start 149.72s - 151.72s | Amazing. So, thank you 152.52s - 156.84s | So client port scanning who remembers this headline on Hacker News 157.92s - 160.68s | Yeah, so why is this website port scanning me? 161.72s - 163.72s | Very good question 163.72s - 170.88s | Apparently eBay not some script kiddie, but a real company try to port scan their visitors as soon as they visit the website 171.28s - 175.60s | So, how did you do like how did they do it exactly and by the way? 175.72s - 182.56s | You will get this recording and we will share the presentation with you. So


chunk_text:
order to RCE private internal applications that today are considered 124.91s - 131.27s | Private and in the end we're going to really deep dive into how each and every browser actually 131.75s - 134.75s | was dealing with that and the responsible disclosure and 135.67s - 139.67s | Where we are right now with that so in order to really start 139.67s - 145.39s | I'm gonna pass it on to awesome Avi because he's the actual smart guy that did all of that and 146.39s - 149.72s | Let's start 149.72s - 151.72s | Amazing. So, thank you 152.52s - 156.84s | So client port scanning who remembers this headline on Hacker News 157.92s - 160.68s | Yeah, so why is this website port scanning me? 161.72s - 163.72s | Very good question 163.72s - 170.88s | Apparently eBay not some script kiddie, but a real company try to port scan their visitors as soon as they visit the website 171.28s - 175.60s | So, how did you do like how did they do it exactly and by the way? 175.72s - 182.56s | You will get this recording and we will share the presentation with you. So please try to be concentrated on the presentation 184.24s - 190.36s | Basically what they did is they used local loss and they checked many types of ports 190.48s - 195.76s | I don't know if it's for monetization for fingerprinting or for security reasons 195.76s - 199.04s | But we really don't care because it should never happen, right? 199.44s - 205.92s | Browsers should not communicate with local loss if they're under a dot-com domain. It's that simple in my opinion at least 206.76s - 211.50s | So this is exactly what private network access and I'll simply use PNA 212.96s - 217.52s | And this is what it tries to solve at the end of the day. So we have this 218.52s - 220.52s | Notion of 220.52s - 223.12s | differentiating three distinctive like networks 223.64s - 229.60s | The public network, which is the Internet that we as we refer to the Internet to the Internet solving 230.36s - 234.60s | Which is a top-level domain like a dot-com everything that's on the public Internet 234.96s - 241.72s | then there is the private network, which is the office home network and that we're connected


chunk_text:
please try to be concentrated on the presentation 184.24s - 190.36s | Basically what they did is they used local loss and they checked many types of ports 190.48s - 195.76s | I don't know if it's for monetization for fingerprinting or for security reasons 195.76s - 199.04s | But we really don't care because it should never happen, right? 199.44s - 205.92s | Browsers should not communicate with local loss if they're under a dot-com domain. It's that simple in my opinion at least 206.76s - 211.50s | So this is exactly what private network access and I'll simply use PNA 212.96s - 217.52s | And this is what it tries to solve at the end of the day. So we have this 218.52s - 220.52s | Notion of 220.52s - 223.12s | differentiating three distinctive like networks 223.64s - 229.60s | The public network, which is the Internet that we as we refer to the Internet to the Internet solving 230.36s - 234.60s | Which is a top-level domain like a dot-com everything that's on the public Internet 234.96s - 241.72s | then there is the private network, which is the office home network and that we're connected to and 242.36s - 248.56s | Finally there is the local device. So it is local host and this figure is taken from Google's blog about PNA 248.56s - 252.88s | You should go read about it. It's a really great initiative and we'll understand 253.80s - 255.88s | Why we need it these days more than ever 257.32s - 260.84s | Now there are many ways to run a server, right? 260.84s - 267.36s | We have the default we have 0 0 0 0 or we simply have the local host 267.68s - 274.28s | Bind address so they actually result in different behaviors, but I'm gonna focus on the last two 274.88s - 278.96s | So whenever I'm referring to a local service, I'm referring to the last two 279.52s - 282.88s | figures, which are the 100 7 0 0 1 or 283.56s - 288.00s | literally the local host string so we have a service that listens on the 288.60s - 291.96s | machines local network and the loopback interface and 292.92s - 295.76s | that's it, basically we have many ways of running a server and 296.72s - 301.64s | To understand how we


chunk_text:
to and 242.36s - 248.56s | Finally there is the local device. So it is local host and this figure is taken from Google's blog about PNA 248.56s - 252.88s | You should go read about it. It's a really great initiative and we'll understand 253.80s - 255.88s | Why we need it these days more than ever 257.32s - 260.84s | Now there are many ways to run a server, right? 260.84s - 267.36s | We have the default we have 0 0 0 0 or we simply have the local host 267.68s - 274.28s | Bind address so they actually result in different behaviors, but I'm gonna focus on the last two 274.88s - 278.96s | So whenever I'm referring to a local service, I'm referring to the last two 279.52s - 282.88s | figures, which are the 100 7 0 0 1 or 283.56s - 288.00s | literally the local host string so we have a service that listens on the 288.60s - 291.96s | machines local network and the loopback interface and 292.92s - 295.76s | that's it, basically we have many ways of running a server and 296.72s - 301.64s | To understand how we found everything. So let's jump back 18 years ago 302.28s - 307.66s | To understand why PNA is needed and why it was so hard to solve to this day 309.16s - 316.00s | So it all begins with a lack of standardization there is no real standard to follow at least 18 years ago 316.60s - 319.84s | there was no HTTPS like simply HTTP and 320.84s - 323.76s | And this guy posted on Mozilla's forum on the 324.28s - 328.64s | on their like security program the following headline that says that 329.80s - 331.80s | Websites are trying to attack my router 332.48s - 337.40s | It's really weird. Like as soon as I'm visiting the website, they try to attack my router and 338.48s - 340.48s | It's been happening for the last 18 years 341.12s - 347.72s | It's still open as you can see the issue is still open. It is being updated but without the ability to comment or anything 347.72s - 349.72s | It's still there 350.00s - 356.08s | The idea is that the RFC should not like it claims that it should not behave this way 356.08s - 358.72s


chunk_text:
found everything. So let's jump back 18 years ago 302.28s - 307.66s | To understand why PNA is needed and why it was so hard to solve to this day 309.16s - 316.00s | So it all begins with a lack of standardization there is no real standard to follow at least 18 years ago 316.60s - 319.84s | there was no HTTPS like simply HTTP and 320.84s - 323.76s | And this guy posted on Mozilla's forum on the 324.28s - 328.64s | on their like security program the following headline that says that 329.80s - 331.80s | Websites are trying to attack my router 332.48s - 337.40s | It's really weird. Like as soon as I'm visiting the website, they try to attack my router and 338.48s - 340.48s | It's been happening for the last 18 years 341.12s - 347.72s | It's still open as you can see the issue is still open. It is being updated but without the ability to comment or anything 347.72s - 349.72s | It's still there 350.00s - 356.08s | The idea is that the RFC should not like it claims that it should not behave this way 356.08s - 358.72s | and for some reason browsers are being able to 359.48s - 362.76s | pass and dispatch HTTP requests to the local network and 364.08s - 369.44s | After a while, he says that it's being actively exploited in the wild. Some of you might also remember that 370.68s - 373.64s | real attacks like targeting home routers and 374.16s - 379.52s | This is how they did it. It's called a drive by farming great publication and by synopsis and 380.04s - 383.04s | The ability to load the script as soon as the website loads 383.68s - 387.68s | So this might sound like really odd to you like this should never work today, right? 387.88s - 393.28s | but 18 years ago routers did not have HTTPS or self-signed certificates and 393.96s - 400.92s | the first figure actually would alter the DNS server of your home network using a simple HTTP request and 401.52s - 403.52s | it simply worked and 403.72s - 409.48s | Not only it worked it affected like thousands of hundreds of thousands of home and office routers 409.88s - 411.88s | in larger scale attacks and 412.40s - 418.12s | that


chunk_text:
| and for some reason browsers are being able to 359.48s - 362.76s | pass and dispatch HTTP requests to the local network and 364.08s - 369.44s | After a while, he says that it's being actively exploited in the wild. Some of you might also remember that 370.68s - 373.64s | real attacks like targeting home routers and 374.16s - 379.52s | This is how they did it. It's called a drive by farming great publication and by synopsis and 380.04s - 383.04s | The ability to load the script as soon as the website loads 383.68s - 387.68s | So this might sound like really odd to you like this should never work today, right? 387.88s - 393.28s | but 18 years ago routers did not have HTTPS or self-signed certificates and 393.96s - 400.92s | the first figure actually would alter the DNS server of your home network using a simple HTTP request and 401.52s - 403.52s | it simply worked and 403.72s - 409.48s | Not only it worked it affected like thousands of hundreds of thousands of home and office routers 409.88s - 411.88s | in larger scale attacks and 412.40s - 418.12s | that targeted localhost and the end of the day you can see that all of those are using localhost or like 418.72s - 419.96s | internal 419.96s - 423.08s | Lubbock like internal interfaces and 425.32s - 426.72s | then 426.72s - 431.64s | Microsoft and sorry then Firefox tried to come up with a list of IPs 432.44s - 437.10s | IP segments and address spaces that are considered private 437.42s - 441.34s | It makes a lot of sense right now, do you notice something weird on this list? 443.22s - 444.62s | So 444.62s - 446.62s | we'll get to that in a second, but 447.26s - 451.22s | Someone mentioned that it's not first box specific bug and he's probably right 451.50s - 457.30s | But he mentioned that it's not a vulnerability or an issue. It's maybe a security enhancement and nothing more than that 458.34s - 462.78s | and then they understood that they also forgot 0 0 0 0 and 463.98s - 465.42s | sadly 465.46s - 467.46s | They added it but 468.22s - 476.14s | Nothing happened since that day. So we're jumping back to eight years ago when Google started with


chunk_text:
targeted localhost and the end of the day you can see that all of those are using localhost or like 418.72s - 419.96s | internal 419.96s - 423.08s | Lubbock like internal interfaces and 425.32s - 426.72s | then 426.72s - 431.64s | Microsoft and sorry then Firefox tried to come up with a list of IPs 432.44s - 437.10s | IP segments and address spaces that are considered private 437.42s - 441.34s | It makes a lot of sense right now, do you notice something weird on this list? 443.22s - 444.62s | So 444.62s - 446.62s | we'll get to that in a second, but 447.26s - 451.22s | Someone mentioned that it's not first box specific bug and he's probably right 451.50s - 457.30s | But he mentioned that it's not a vulnerability or an issue. It's maybe a security enhancement and nothing more than that 458.34s - 462.78s | and then they understood that they also forgot 0 0 0 0 and 463.98s - 465.42s | sadly 465.46s - 467.46s | They added it but 468.22s - 476.14s | Nothing happened since that day. So we're jumping back to eight years ago when Google started with the course cross-origin resource sharing 476.74s - 478.74s | It's actually defining how 479.26s - 485.18s | websites should behave and how browsers should act when requesting resources from other domains and 486.74s - 491.98s | Google started this initiative to try to define the behavior of how 492.82s - 494.82s | browsers should or should not 495.30s - 498.46s | dispatch requests to these private and network addresses 499.58s - 504.74s | So PNA is a standard that is still under ideation. It's not finalized yet 506.06s - 510.22s | it's being led by Google and as you can see, it's a draft community report and 511.82s - 514.90s | Therefore Firefox is waiting for this standard and 515.42s - 520.26s | And target as depends on private network access, which is still not implemented in Firefox 521.26s - 528.10s | And basically the wait for home and the home team to define the standard make it a stable in home and 528.46s - 533.54s | Only then they will adopt it to Firefox following a certain standard, but they don't want to do it 534.50s - 539.06s | Randomly or trusting on some gut feeling they want to


chunk_text:
the course cross-origin resource sharing 476.74s - 478.74s | It's actually defining how 479.26s - 485.18s | websites should behave and how browsers should act when requesting resources from other domains and 486.74s - 491.98s | Google started this initiative to try to define the behavior of how 492.82s - 494.82s | browsers should or should not 495.30s - 498.46s | dispatch requests to these private and network addresses 499.58s - 504.74s | So PNA is a standard that is still under ideation. It's not finalized yet 506.06s - 510.22s | it's being led by Google and as you can see, it's a draft community report and 511.82s - 514.90s | Therefore Firefox is waiting for this standard and 515.42s - 520.26s | And target as depends on private network access, which is still not implemented in Firefox 521.26s - 528.10s | And basically the wait for home and the home team to define the standard make it a stable in home and 528.46s - 533.54s | Only then they will adopt it to Firefox following a certain standard, but they don't want to do it 534.50s - 539.06s | Randomly or trusting on some gut feeling they want to implement a real standard 540.34s - 543.38s | Now, let's understand what is still allowed 543.86s - 545.18s | So 545.18s - 551.86s | They started PNA took a long time to roll. It's still not a mainstream component of in Chrome and 552.46s - 554.74s | it should have been available to 555.50s - 557.50s | like until home 557.74s - 561.22s | 126 but it got delayed several times and 562.54s - 567.98s | And they even had to revert it at some point because they broke a lot of websites 567.98s - 575.46s | But their aim is very clear. They want to mitigate CSRF attacks and DNS rebinding attacks to local addresses 575.98s - 581.71s | Now they still have not decided how to treat 0 0 0 0 582.47s - 585.47s | this is the standard by the way, it's on github you can look it up and 587.47s - 591.59s | They had to rolled out to roll back some changes as I said, so 592.39s - 595.43s | You can see that between some home versions 595.43s - 601.23s | They had to revert and then apply it back because many websites simply


chunk_text:
implement a real standard 540.34s - 543.38s | Now, let's understand what is still allowed 543.86s - 545.18s | So 545.18s - 551.86s | They started PNA took a long time to roll. It's still not a mainstream component of in Chrome and 552.46s - 554.74s | it should have been available to 555.50s - 557.50s | like until home 557.74s - 561.22s | 126 but it got delayed several times and 562.54s - 567.98s | And they even had to revert it at some point because they broke a lot of websites 567.98s - 575.46s | But their aim is very clear. They want to mitigate CSRF attacks and DNS rebinding attacks to local addresses 575.98s - 581.71s | Now they still have not decided how to treat 0 0 0 0 582.47s - 585.47s | this is the standard by the way, it's on github you can look it up and 587.47s - 591.59s | They had to rolled out to roll back some changes as I said, so 592.39s - 595.43s | You can see that between some home versions 595.43s - 601.23s | They had to revert and then apply it back because many websites simply break within a night. So 601.91s - 607.35s | The next time they did it Google they did it gradually and very carefully 608.63s - 613.39s | And as I said, they had to make some exceptions to 0 0 0 0 613.95s - 618.79s | So even although it was in defiance of specs, it was not a part of the RFC 619.59s - 623.51s | Developers must have wanted it and requesting this behavior. So 624.11s - 627.51s | As you can see it's in defiance of specs, but seemingly common 628.79s - 635.03s | But what is this weird IP address anyway, so if I'll ask two different people in this room 635.03s - 638.95s | I'll probably hear two different answers. And as you can see it can have multiple uses 640.87s - 645.23s | So typically it simply means this host in on this network 647.28s - 651.24s | It should never be used as a destination IP address according to the RFC 651.36s - 652.48s | You'll see it in a bit 652.48s - 658.64s | But a real use case of 0 0 0 0 is when we're doing a DHCP handshake 658.64s


chunk_text:
break within a night. So 601.91s - 607.35s | The next time they did it Google they did it gradually and very carefully 608.63s - 613.39s | And as I said, they had to make some exceptions to 0 0 0 0 613.95s - 618.79s | So even although it was in defiance of specs, it was not a part of the RFC 619.59s - 623.51s | Developers must have wanted it and requesting this behavior. So 624.11s - 627.51s | As you can see it's in defiance of specs, but seemingly common 628.79s - 635.03s | But what is this weird IP address anyway, so if I'll ask two different people in this room 635.03s - 638.95s | I'll probably hear two different answers. And as you can see it can have multiple uses 640.87s - 645.23s | So typically it simply means this host in on this network 647.28s - 651.24s | It should never be used as a destination IP address according to the RFC 651.36s - 652.48s | You'll see it in a bit 652.48s - 658.64s | But a real use case of 0 0 0 0 is when we're doing a DHCP handshake 658.64s - 663.68s | And when we still don't have an IP address and before the dynamic allocation 663.68s - 666.88s | We want to identify yourself somehow, so we have to use some IP 667.12s - 670.12s | But this is really the only use as a IP address 670.60s - 674.68s | Doing like handshakes and stuff like this before you have a dynamically allocated IP 674.68s - 679.84s | And as I said, it should not be used as a target IP 680.72s - 685.68s | Anywhere and by the way windows I know we all have given them like hard time 685.68s - 692.00s | But they were actually blocking it at the operating system level. This is why it affects only macOS and Linux 693.04s - 695.04s | but this is basically a 695.76s - 697.76s | misunderstanding of the RFC or some 698.64s - 700.64s | non-correlative behavior 700.96s - 707.44s | So Chromium decided to add counters to Chrome to understand how many websites use this IP address 708.44s - 713.64s | So it was relatively small at first place, but we've seen it increasing with time 715.56s - 722.44s | This is the


chunk_text:
- 663.68s | And when we still don't have an IP address and before the dynamic allocation 663.68s - 666.88s | We want to identify yourself somehow, so we have to use some IP 667.12s - 670.12s | But this is really the only use as a IP address 670.60s - 674.68s | Doing like handshakes and stuff like this before you have a dynamically allocated IP 674.68s - 679.84s | And as I said, it should not be used as a target IP 680.72s - 685.68s | Anywhere and by the way windows I know we all have given them like hard time 685.68s - 692.00s | But they were actually blocking it at the operating system level. This is why it affects only macOS and Linux 693.04s - 695.04s | but this is basically a 695.76s - 697.76s | misunderstanding of the RFC or some 698.64s - 700.64s | non-correlative behavior 700.96s - 707.44s | So Chromium decided to add counters to Chrome to understand how many websites use this IP address 708.44s - 713.64s | So it was relatively small at first place, but we've seen it increasing with time 715.56s - 722.44s | This is the percents of pages that use 0 0 0 0 in Ajax or other mechanisms 723.12s - 725.44s | And basically you can see that it's on the rise 726.20s - 728.20s | maybe ten times 728.32s - 730.32s | double itself like ten times in 731.60s - 737.36s | Under two years or slightly more than two years, but it is on the rise and it should concern us all 738.16s - 739.28s | so 739.28s - 743.40s | This is like the the really interesting part of the presentation 743.88s - 747.52s | All of these are blocked. All right private network access 748.04s - 750.56s | Tries to prevent us from accessing these domains 752.40s - 754.40s | Dot-com domain 754.64s - 759.60s | but when we use 0 0 0 0 we manage to bypass all of those restrictions and 760.72s - 763.24s | access localhost and lubec so 763.84s - 765.84s | It's that simple. It should be 766.24s - 767.24s | And 767.24s - 769.24s | It should be hard. It should be like 769.92s - 775.04s | Terminated in the request level before it is being dispatched, but we can access localhost and 776.04s


chunk_text:
percents of pages that use 0 0 0 0 in Ajax or other mechanisms 723.12s - 725.44s | And basically you can see that it's on the rise 726.20s - 728.20s | maybe ten times 728.32s - 730.32s | double itself like ten times in 731.60s - 737.36s | Under two years or slightly more than two years, but it is on the rise and it should concern us all 738.16s - 739.28s | so 739.28s - 743.40s | This is like the the really interesting part of the presentation 743.88s - 747.52s | All of these are blocked. All right private network access 748.04s - 750.56s | Tries to prevent us from accessing these domains 752.40s - 754.40s | Dot-com domain 754.64s - 759.60s | but when we use 0 0 0 0 we manage to bypass all of those restrictions and 760.72s - 763.24s | access localhost and lubec so 763.84s - 765.84s | It's that simple. It should be 766.24s - 767.24s | And 767.24s - 769.24s | It should be hard. It should be like 769.92s - 775.04s | Terminated in the request level before it is being dispatched, but we can access localhost and 776.04s - 778.04s | 107 0 0 1 778.76s - 782.00s | Now everything that is HTTP that runs on these 782.52s - 787.16s | Services including port forwarding to kubernetes like some of us are developers including myself 787.84s - 792.64s | We all use HTTP port forwarding when working with remote pods and stuff like that 793.56s - 799.24s | Including operating system services. So everything that runs on localhost on your machine is accessible 800.16s - 808.00s | There are plenty of default or non default developer tools that use localhost and it also enables us to use the VPN 808.88s - 811.80s | Literally every network interface that is connected to the machine 812.40s - 815.36s | Which was used in the past for DNS rebinding to localhost 815.36s - 821.64s | So you can use 0 0 0 0 instead of localhost and DNS rebinding would still work. We'll see it in a bit 822.76s - 824.76s | so to target it we 825.52s - 830.28s | We wanted to see how it behaves and whether it's really possible. So we've built a port scanner 830.96s - 834.92s | written in WebAssembly, but it uses JavaScript


chunk_text:
- 778.04s | 107 0 0 1 778.76s - 782.00s | Now everything that is HTTP that runs on these 782.52s - 787.16s | Services including port forwarding to kubernetes like some of us are developers including myself 787.84s - 792.64s | We all use HTTP port forwarding when working with remote pods and stuff like that 793.56s - 799.24s | Including operating system services. So everything that runs on localhost on your machine is accessible 800.16s - 808.00s | There are plenty of default or non default developer tools that use localhost and it also enables us to use the VPN 808.88s - 811.80s | Literally every network interface that is connected to the machine 812.40s - 815.36s | Which was used in the past for DNS rebinding to localhost 815.36s - 821.64s | So you can use 0 0 0 0 instead of localhost and DNS rebinding would still work. We'll see it in a bit 822.76s - 824.76s | so to target it we 825.52s - 830.28s | We wanted to see how it behaves and whether it's really possible. So we've built a port scanner 830.96s - 834.92s | written in WebAssembly, but it uses JavaScript API, so it doesn't really matter and 835.44s - 841.08s | the idea is that it was able to see the open ports on my machine just by the 842.44s - 845.92s | Error code that the browser returns to the JavaScript context 845.92s - 851.52s | So if it's a connection refused, you know, nothing is there if and if there is an error, you know, something is listening 851.72s - 853.72s | it's very simple and 854.60s - 861.44s | We use 0 0 0 0 but you can see in the sniffing with the wire shark that it use it's being redirected to Lubeck 861.72s - 863.72s | to localhost and 864.24s - 866.24s | Here's the DNS rebinding 866.92s - 868.92s | Notion, so you have 869.16s - 873.48s | Like a subdomain that is being mapped to 0 0 0 0 873.92s - 880.56s | So we've seen a lot of attacks in the history that uses that use the 127 0 0 1 880.84s - 885.24s | But it it's it is possible once again using 0 0 0 0 886.12s - 893.86s | Now this gets me to shadow ray. This is the


chunk_text:
API, so it doesn't really matter and 835.44s - 841.08s | the idea is that it was able to see the open ports on my machine just by the 842.44s - 845.92s | Error code that the browser returns to the JavaScript context 845.92s - 851.52s | So if it's a connection refused, you know, nothing is there if and if there is an error, you know, something is listening 851.72s - 853.72s | it's very simple and 854.60s - 861.44s | We use 0 0 0 0 but you can see in the sniffing with the wire shark that it use it's being redirected to Lubeck 861.72s - 863.72s | to localhost and 864.24s - 866.24s | Here's the DNS rebinding 866.92s - 868.92s | Notion, so you have 869.16s - 873.48s | Like a subdomain that is being mapped to 0 0 0 0 873.92s - 880.56s | So we've seen a lot of attacks in the history that uses that use the 127 0 0 1 880.84s - 885.24s | But it it's it is possible once again using 0 0 0 0 886.12s - 893.86s | Now this gets me to shadow ray. This is the real reason we started with this research 893.86s - 900.18s | So after we've published shadow ray, which was an attack campaign in the wild 900.98s - 903.90s | targeting AI workloads in a lot of big companies 904.94s - 906.94s | it leverages 907.10s - 910.30s | Behavior a default behavior in rain if any of you know it 911.14s - 917.26s | Rays like Kubernetes for AI so it enables you to scale and run Python applications 917.26s - 921.98s | I'm an early developer and doctor myself. I really love it and 922.74s - 927.18s | Even open AI say out loud that they use it to change a GPT and their biggest models 928.86s - 930.58s | So in late 930.58s - 931.94s | 2023 we had 931.94s - 935.86s | we've seen this report by any scale the developers and maintainers of Ray and 936.06s - 942.14s | We've seen four CVEs that were taken care of and one disputed CVE, which is very very interesting 943.06s - 946.18s | it simply discusses the ability of Ray to 947.02s - 952.76s | Run arbitrary code because it's an assess. It's like an inheritor part of


chunk_text:
real reason we started with this research 893.86s - 900.18s | So after we've published shadow ray, which was an attack campaign in the wild 900.98s - 903.90s | targeting AI workloads in a lot of big companies 904.94s - 906.94s | it leverages 907.10s - 910.30s | Behavior a default behavior in rain if any of you know it 911.14s - 917.26s | Rays like Kubernetes for AI so it enables you to scale and run Python applications 917.26s - 921.98s | I'm an early developer and doctor myself. I really love it and 922.74s - 927.18s | Even open AI say out loud that they use it to change a GPT and their biggest models 928.86s - 930.58s | So in late 930.58s - 931.94s | 2023 we had 931.94s - 935.86s | we've seen this report by any scale the developers and maintainers of Ray and 936.06s - 942.14s | We've seen four CVEs that were taken care of and one disputed CVE, which is very very interesting 943.06s - 946.18s | it simply discusses the ability of Ray to 947.02s - 952.76s | Run arbitrary code because it's an assess. It's like an inheritor part of Ray 952.76s - 960.10s | It is a desired capability must have this capability to function because it's a job running library 961.10s - 966.42s | And therefore they disputed the vulnerability that enables remote code execution because it's 966.94s - 974.70s | Necessary and it's like kind of by design the way Ray is designed an attacker leveraged it to run arbitrary code on 975.74s - 977.74s | clusters that are exposed to the Internet and 978.90s - 983.02s | It was it got amazing coverage as well. So like I was on Forbes and 983.70s - 989.22s | We asked ourselves the following question. We know that Ray lacks authorization and authentication 989.58s - 994.10s | we know that it is usually running on local hosts for when developers use it and 994.86s - 1000.90s | We try to understand whether our browser can load an external public website 1001.18s - 1005.92s | Which then can communicate with this local Ray cluster or a service? 1006.66s - 1008.66s | That is running locally on local host 1010.26s - 1016.66s | So we need to install Ray and have it running in the default constellation no special flags or


chunk_text:
Ray 952.76s - 960.10s | It is a desired capability must have this capability to function because it's a job running library 961.10s - 966.42s | And therefore they disputed the vulnerability that enables remote code execution because it's 966.94s - 974.70s | Necessary and it's like kind of by design the way Ray is designed an attacker leveraged it to run arbitrary code on 975.74s - 977.74s | clusters that are exposed to the Internet and 978.90s - 983.02s | It was it got amazing coverage as well. So like I was on Forbes and 983.70s - 989.22s | We asked ourselves the following question. We know that Ray lacks authorization and authentication 989.58s - 994.10s | we know that it is usually running on local hosts for when developers use it and 994.86s - 1000.90s | We try to understand whether our browser can load an external public website 1001.18s - 1005.92s | Which then can communicate with this local Ray cluster or a service? 1006.66s - 1008.66s | That is running locally on local host 1010.26s - 1016.66s | So we need to install Ray and have it running in the default constellation no special flags or anything 1017.02s - 1019.02s | Then we have to visit the website and 1019.70s - 1025.70s | the website will invoke a post HTTP request to the API and triggering remote code execution and 1027.18s - 1030.10s | Get this you only need to dispatch the request 1030.10s - 1035.56s | We don't care about the response and this is the the whole idea of private network access 1035.82s - 1041.02s | So course the cross-origin resource sharing was dealing mostly with how the response 1041.94s - 1049.30s | And which headers the response has and whether should I propagate it to the current context of JavaScript and loaded under the current domain 1049.58s - 1057.66s | But we're really emphasizing the ability to dispatch requests and a single HTTP request is enough to trigger remote code execution 1059.22s - 1063.66s | Also important like important to mention that it's not limited to Ray. It's only one use case 1063.66s - 1067.58s | We've seen many remote code execution vulnerabilities on local off 1068.10s - 1071.74s | For the sake of time and we're only showing this on the presentation 1072.30s - 1074.18s | But there is the full blog


chunk_text:
anything 1017.02s - 1019.02s | Then we have to visit the website and 1019.70s - 1025.70s | the website will invoke a post HTTP request to the API and triggering remote code execution and 1027.18s - 1030.10s | Get this you only need to dispatch the request 1030.10s - 1035.56s | We don't care about the response and this is the the whole idea of private network access 1035.82s - 1041.02s | So course the cross-origin resource sharing was dealing mostly with how the response 1041.94s - 1049.30s | And which headers the response has and whether should I propagate it to the current context of JavaScript and loaded under the current domain 1049.58s - 1057.66s | But we're really emphasizing the ability to dispatch requests and a single HTTP request is enough to trigger remote code execution 1059.22s - 1063.66s | Also important like important to mention that it's not limited to Ray. It's only one use case 1063.66s - 1067.58s | We've seen many remote code execution vulnerabilities on local off 1068.10s - 1071.74s | For the sake of time and we're only showing this on the presentation 1072.30s - 1074.18s | But there is the full blog at the end 1074.18s - 1082.06s | So I have a phishing email not the most successful one, but I'm clicking here and being redirected to this website 1083.18s - 1089.26s | Now this is the JavaScript that targets the remote code execution. You can see that there's a net cut 1091.26s - 1095.74s | Socket that is being opened to a remote server, which is the attacker control the server 1096.30s - 1103.74s | It is being used with mode no course which sends the request and will simply return on a pack response 1103.74s - 1105.74s | So I cannot see the response 1105.74s - 1110.86s | but as we said, we don't really care we we just want to send it and trigger remote code execution and 1112.42s - 1117.58s | Finally we get a response. It looks like an error, but you can see that it's 200. Okay 1118.38s - 1121.02s | so apparently the job really ran and 1121.14s - 1122.30s | And 1122.30s - 1125.54s | You'll see it in a second. But this is all it takes 1126.38s - 1132.70s | Just one HTTP request to reach the server. We don't care about


chunk_text:
at the end 1074.18s - 1082.06s | So I have a phishing email not the most successful one, but I'm clicking here and being redirected to this website 1083.18s - 1089.26s | Now this is the JavaScript that targets the remote code execution. You can see that there's a net cut 1091.26s - 1095.74s | Socket that is being opened to a remote server, which is the attacker control the server 1096.30s - 1103.74s | It is being used with mode no course which sends the request and will simply return on a pack response 1103.74s - 1105.74s | So I cannot see the response 1105.74s - 1110.86s | but as we said, we don't really care we we just want to send it and trigger remote code execution and 1112.42s - 1117.58s | Finally we get a response. It looks like an error, but you can see that it's 200. Okay 1118.38s - 1121.02s | so apparently the job really ran and 1121.14s - 1122.30s | And 1122.30s - 1125.54s | You'll see it in a second. But this is all it takes 1126.38s - 1132.70s | Just one HTTP request to reach the server. We don't care about the response and it's more than enough 1133.90s - 1138.66s | So every time you visit the page, it will trigger this specific payload 1138.66s - 1143.42s | Which just prints them environment variables and prints exploited to the screen 1143.82s - 1148.30s | But here is a reverse shell that is being opened as soon as I'm clicking on an email 1148.98s - 1155.94s | So on the right I have ray running I'm importing it and running graded in it without any special parameters 1155.94s - 1161.10s | This is how developers usually use rain and you can see that it binds on local host 1161.74s - 1168.44s | There is a raised dashboard and it should reflect the jobs that are currently running 1169.40s - 1171.20s | now 1171.20s - 1173.64s | on the left, I'll open a socket and 1175.04s - 1177.68s | listen for connection on on the certain port that 1178.64s - 1181.16s | Matches the payload of the of the attack 1182.76s - 1188.44s | Now as soon as I'm clicking on this link, you can see that the reverse shell is opened to the left terminal and 1189.68s - 1194.84s | Which


chunk_text:
the response and it's more than enough 1133.90s - 1138.66s | So every time you visit the page, it will trigger this specific payload 1138.66s - 1143.42s | Which just prints them environment variables and prints exploited to the screen 1143.82s - 1148.30s | But here is a reverse shell that is being opened as soon as I'm clicking on an email 1148.98s - 1155.94s | So on the right I have ray running I'm importing it and running graded in it without any special parameters 1155.94s - 1161.10s | This is how developers usually use rain and you can see that it binds on local host 1161.74s - 1168.44s | There is a raised dashboard and it should reflect the jobs that are currently running 1169.40s - 1171.20s | now 1171.20s - 1173.64s | on the left, I'll open a socket and 1175.04s - 1177.68s | listen for connection on on the certain port that 1178.64s - 1181.16s | Matches the payload of the of the attack 1182.76s - 1188.44s | Now as soon as I'm clicking on this link, you can see that the reverse shell is opened to the left terminal and 1189.68s - 1194.84s | Which gives me access to the local machine from VPS somewhere on the Internet 1195.88s - 1201.64s | You can see that the netcat is really running in rain and I use it to run and I with every 1202.64s - 1209.52s | Commands on my personal computer and that's it. Basically it got a lot of coverage 1209.52s - 1215.92s | So remote execution with a single click and as I said, it's not limited to a so therefore all the buzz 1216.44s - 1220.24s | But it got a lot of coverage and for a reason 1221.20s - 1224.68s | Now we did a responsible disclosure with the browsers 1225.12s - 1230.28s | And they had to introduce breaking changes. Some of them were doing it more gradually 1230.52s - 1232.52s | But the idea is that 1232.88s - 1236.48s | We've actually managed to coordinate all the browsers together 1236.96s - 1241.64s | So they will not zero day each other by releasing a fix ahead of others 1243.12s - 1245.12s | Here is the proof on each of the browsers 1245.28s - 1249.62s | So the same payload can be used in each and every one of


chunk_text:
gives me access to the local machine from VPS somewhere on the Internet 1195.88s - 1201.64s | You can see that the netcat is really running in rain and I use it to run and I with every 1202.64s - 1209.52s | Commands on my personal computer and that's it. Basically it got a lot of coverage 1209.52s - 1215.92s | So remote execution with a single click and as I said, it's not limited to a so therefore all the buzz 1216.44s - 1220.24s | But it got a lot of coverage and for a reason 1221.20s - 1224.68s | Now we did a responsible disclosure with the browsers 1225.12s - 1230.28s | And they had to introduce breaking changes. Some of them were doing it more gradually 1230.52s - 1232.52s | But the idea is that 1232.88s - 1236.48s | We've actually managed to coordinate all the browsers together 1236.96s - 1241.64s | So they will not zero day each other by releasing a fix ahead of others 1243.12s - 1245.12s | Here is the proof on each of the browsers 1245.28s - 1249.62s | So the same payload can be used in each and every one of them 1253.80s - 1260.04s | And let's begin with Safari. So Safari did an amazing job. All of them did an amazing job, but Safari 1260.68s - 1265.76s | Simply mitigated it in the browser level before PNA is actually implemented in Safari 1267.16s - 1274.16s | You all know webkit, I guess so webkit is open sources Safari is based on and they changed 23 files 1274.16s - 1280.16s | They added a lot of tests and they did amazing amazing job on blocking it in Safari's level 1280.16s - 1286.16s | But not limited to Safari, but actually every web view on watch OS or iPad. It doesn't matter 1286.44s - 1291.12s | It's all webkit based so they did a really amazing patch here and great job and 1292.16s - 1298.52s | You can see that they literally check that the Android like the address contains only zeros. They have this property 1298.64s - 1303.84s | So sorry, but we added like an if clause on every HTTP request in the world 1304.56s - 1307.16s | not proud of it, but it's the way it is and 1308.64s - 1313.92s | There's homeoom. Homeoom as I said,


chunk_text:
them 1253.80s - 1260.04s | And let's begin with Safari. So Safari did an amazing job. All of them did an amazing job, but Safari 1260.68s - 1265.76s | Simply mitigated it in the browser level before PNA is actually implemented in Safari 1267.16s - 1274.16s | You all know webkit, I guess so webkit is open sources Safari is based on and they changed 23 files 1274.16s - 1280.16s | They added a lot of tests and they did amazing amazing job on blocking it in Safari's level 1280.16s - 1286.16s | But not limited to Safari, but actually every web view on watch OS or iPad. It doesn't matter 1286.44s - 1291.12s | It's all webkit based so they did a really amazing patch here and great job and 1292.16s - 1298.52s | You can see that they literally check that the Android like the address contains only zeros. They have this property 1298.64s - 1303.84s | So sorry, but we added like an if clause on every HTTP request in the world 1304.56s - 1307.16s | not proud of it, but it's the way it is and 1308.64s - 1313.92s | There's homeoom. Homeoom as I said, they are the leaders of private network access 1314.44s - 1315.92s | So 1315.92s - 1322.52s | they proposed a long time ago and even when it's only started to add these headers to the back-end services and 1323.92s - 1329.80s | later on like these headers can be used in the server level to mitigate those attacks, but 1330.36s - 1337.40s | They decided to deprecate 0 0 0 0 before private network access completely rolls out to the browser 1337.52s - 1343.12s | And there is even our report here and they literally say that it can be used for DNS rebinding attacks and 1343.48s - 1350.62s | PNA protection can be bypassed using this special IP address on macOS and Linux and it should never be used in practice 1351.12s - 1353.12s | all right, it should never be used and 1354.84s - 1361.20s | As I said, they added the counters and and they say out loud that malware leverages this attack 1362.00s - 1365.60s | to specific developer toolkits and again our bug 1367.40s - 1370.92s | And there is Firefox. Firefox also did an amazing job 1371.92s - 1376.56s | They don't add PNA


chunk_text:
they are the leaders of private network access 1314.44s - 1315.92s | So 1315.92s - 1322.52s | they proposed a long time ago and even when it's only started to add these headers to the back-end services and 1323.92s - 1329.80s | later on like these headers can be used in the server level to mitigate those attacks, but 1330.36s - 1337.40s | They decided to deprecate 0 0 0 0 before private network access completely rolls out to the browser 1337.52s - 1343.12s | And there is even our report here and they literally say that it can be used for DNS rebinding attacks and 1343.48s - 1350.62s | PNA protection can be bypassed using this special IP address on macOS and Linux and it should never be used in practice 1351.12s - 1353.12s | all right, it should never be used and 1354.84s - 1361.20s | As I said, they added the counters and and they say out loud that malware leverages this attack 1362.00s - 1365.60s | to specific developer toolkits and again our bug 1367.40s - 1370.92s | And there is Firefox. Firefox also did an amazing job 1371.92s - 1376.56s | They don't add PNA right away because it's not standardized yet 1376.56s - 1381.44s | it's not ready, but when it will be they're definitely positive towards adding PNA and 1382.00s - 1389.04s | They did block it in fetch level like who knows fetch in JavaScript is the way to do HTTP requests 1389.68s - 1391.80s | they literally changed the RFC and 1392.72s - 1393.84s | And 1393.84s - 1399.60s | they proposed to block 0 0 0 whenever a target IP is 0 0 0 simply drop the 1400.20s - 1406.96s | Drop the HTTP request and then browsers will have to comply with a new fetch standard. This is the way things work 1408.64s - 1410.64s | Now what can we expect 1411.16s - 1413.96s | So I expect that many open source 1414.84s - 1421.48s | Maintainers and developers will add PNA headers to their services. So they won't be attacked from this attack vector 1422.36s - 1425.36s | eventually, this IP address will be blocked in all browsers and 1426.08s - 1431.92s | I think that now people I hope at least I hope that will treat local host is not really local anymore 1432.32s - 1437.60s |


chunk_text:
right away because it's not standardized yet 1376.56s - 1381.44s | it's not ready, but when it will be they're definitely positive towards adding PNA and 1382.00s - 1389.04s | They did block it in fetch level like who knows fetch in JavaScript is the way to do HTTP requests 1389.68s - 1391.80s | they literally changed the RFC and 1392.72s - 1393.84s | And 1393.84s - 1399.60s | they proposed to block 0 0 0 whenever a target IP is 0 0 0 simply drop the 1400.20s - 1406.96s | Drop the HTTP request and then browsers will have to comply with a new fetch standard. This is the way things work 1408.64s - 1410.64s | Now what can we expect 1411.16s - 1413.96s | So I expect that many open source 1414.84s - 1421.48s | Maintainers and developers will add PNA headers to their services. So they won't be attacked from this attack vector 1422.36s - 1425.36s | eventually, this IP address will be blocked in all browsers and 1426.08s - 1431.92s | I think that now people I hope at least I hope that will treat local host is not really local anymore 1432.32s - 1437.60s | We can make these assumptions of being executed in a trusted environment because as we've seen 1438.16s - 1442.16s | Browser can have routing capabilities and can reach our local local host 1443.20s - 1449.96s | Interfaces so HTTPS call CSRF tokens authorization. All of those are a must also in local projects 1450.04s - 1452.04s | So, please don't make those assumptions 1453.08s - 1459.82s | Now one was done but a cool thing. I wanted to share so, you know, these port scanners online that scan certain websites 1460.16s - 1468.52s | So if you use like loopback or localhost, it will fail, but you can make them port scan themselves using 0 0 0 0 1471.12s - 1476.16s | Yeah, many people forget about this special IP as we said a lot of different use cases 1477.28s - 1478.48s | and 1478.48s - 1480.48s | That's it. Basically 1480.68s - 1485.04s | And it gets me to the conclusion which I'll hand back to Gal and thank you so far 1485.60s - 1498.00s | He deserves it. So let's conclude what we see. First of all, Avi is awesome, right? 1498.56s - 1504.48s | He's breaking the


chunk_text:
We can make these assumptions of being executed in a trusted environment because as we've seen 1438.16s - 1442.16s | Browser can have routing capabilities and can reach our local local host 1443.20s - 1449.96s | Interfaces so HTTPS call CSRF tokens authorization. All of those are a must also in local projects 1450.04s - 1452.04s | So, please don't make those assumptions 1453.08s - 1459.82s | Now one was done but a cool thing. I wanted to share so, you know, these port scanners online that scan certain websites 1460.16s - 1468.52s | So if you use like loopback or localhost, it will fail, but you can make them port scan themselves using 0 0 0 0 1471.12s - 1476.16s | Yeah, many people forget about this special IP as we said a lot of different use cases 1477.28s - 1478.48s | and 1478.48s - 1480.48s | That's it. Basically 1480.68s - 1485.04s | And it gets me to the conclusion which I'll hand back to Gal and thank you so far 1485.60s - 1498.00s | He deserves it. So let's conclude what we see. First of all, Avi is awesome, right? 1498.56s - 1504.48s | He's breaking the browsers making them actually talk to each other and start to actually build a standard 1504.48s - 1508.76s | We saw how an 18 years old bug that people thought is actually 1509.20s - 1516.92s | Already over DNS rebinding stuff that we all thought are actually like dead already are actually being actively used 1517.24s - 1521.56s | I don't know if you saw this 100,000 website. I can tell you honestly, they're probably 1522.16s - 1528.24s | 100% of them are malicious and this whole concept that been we've been trying to do for years and like 1528.60s - 1530.60s | course and all of those mitigations 1531.08s - 1535.64s | In the end by a simple super logical bug that just enables 1536.24s - 1538.28s | Everything once again, we can see that those 1539.36s - 1545.72s | Attacks will keep on occurring and this is why we think this is a subject that need to be called out 1545.76s - 1550.52s | people should be aware that no matter if they do have a WAF or a firewall of 1551.04s - 1555.68s | False sense of safety that our application are actually secured behind a private


chunk_text:
browsers making them actually talk to each other and start to actually build a standard 1504.48s - 1508.76s | We saw how an 18 years old bug that people thought is actually 1509.20s - 1516.92s | Already over DNS rebinding stuff that we all thought are actually like dead already are actually being actively used 1517.24s - 1521.56s | I don't know if you saw this 100,000 website. I can tell you honestly, they're probably 1522.16s - 1528.24s | 100% of them are malicious and this whole concept that been we've been trying to do for years and like 1528.60s - 1530.60s | course and all of those mitigations 1531.08s - 1535.64s | In the end by a simple super logical bug that just enables 1536.24s - 1538.28s | Everything once again, we can see that those 1539.36s - 1545.72s | Attacks will keep on occurring and this is why we think this is a subject that need to be called out 1545.76s - 1550.52s | people should be aware that no matter if they do have a WAF or a firewall of 1551.04s - 1555.68s | False sense of safety that our application are actually secured behind a private network 1556.12s - 1561.60s | This is where we come and show look attackers are also looking for those type of things 1561.80s - 1568.16s | They are really enjoying from the fact that there isn't one standard and it can be abused and this is why 1568.76s - 1570.76s | We will keep seeing those stuff again 1570.76s - 1575.52s | And why we should all as a community really look for other ways and other 1576.88s - 1581.88s | mitigations on the application themself not trusting the old ways that we saw and 1583.12s - 1587.68s | Of course, we're gonna have some Q&A. But again, feel free to reach out 1588.40s - 1592.72s | On whatever like X Twitter, whatever Elon will call it tomorrow morning 1593.04s - 1598.72s | We would love to answer anything you might you might like have and again 1598.72s - 1600.72s | This is just one example 1600.96s - 1604.00s | And you've seen how big companies abused it 1604.84s - 1610.48s | For a financial reason or attacker been leveraging that to hack internal network 1610.56s - 1615.66s | But the whole concept of just one packet that can hack


chunk_text:
network 1556.12s - 1561.60s | This is where we come and show look attackers are also looking for those type of things 1561.80s - 1568.16s | They are really enjoying from the fact that there isn't one standard and it can be abused and this is why 1568.76s - 1570.76s | We will keep seeing those stuff again 1570.76s - 1575.52s | And why we should all as a community really look for other ways and other 1576.88s - 1581.88s | mitigations on the application themself not trusting the old ways that we saw and 1583.12s - 1587.68s | Of course, we're gonna have some Q&A. But again, feel free to reach out 1588.40s - 1592.72s | On whatever like X Twitter, whatever Elon will call it tomorrow morning 1593.04s - 1598.72s | We would love to answer anything you might you might like have and again 1598.72s - 1600.72s | This is just one example 1600.96s - 1604.00s | And you've seen how big companies abused it 1604.84s - 1610.48s | For a financial reason or attacker been leveraging that to hack internal network 1610.56s - 1615.66s | But the whole concept of just one packet that can hack any internal application 1615.80s - 1620.36s | It's a really risky thing and this is why we are here to really mention it out 1620.52s - 1627.40s | So again, thank you very much everyone for attending this talk. We will share everything online and again 1627.40s - 1630.40s | Happy DEF CON. Thank you everyone


chunk_text:
any internal application 1615.80s - 1620.36s | It's a really risky thing and this is why we are here to really mention it out 1620.52s - 1627.40s | So again, thank you very much everyone for attending this talk. We will share everything online and again 1627.40s - 1630.40s | Happy DEF CON. Thank you everyone


chunk_text:
{ "webpage_url": "https://www.youtube.com/watch?v=9bNv3XkhjQA", "title": "DEF CON 32 - BOLABuster-Harnessing LLMs for Automating BOLA Detection - Ravid Mazon, Jay Chen", "description": "BOLA is a prevalent vulnerability in modern APIs and web applications, topping the OWASP API risk chart, and ranking fourth in HackerOne Global Top 10. Its impact ranges from data exposure to total system control loss.\n\nWhile manually triggering known BOLAs is relatively straightforward, automatic detection is challenging due to the complexities of application logic, the wide range of inputs, and the stateful nature of modern web applications.\n\nTo tackle this, we leveraged LLMs to automate manual tasks such as understanding application logic, revealing endpoint dependency relationships, generating test cases, and interpreting results. Our AI-backed approach enables automating BOLA detection at scale, named BOLABuster.\n\nThough in its early stages, BOLABuster identified numerous vulnerabilities in open-source projects. In one case, we submitted 15 CVEs for a project, some leading to critical privilege escalation. Our latest disclosed vulnerability was CVE-2024-1313, a BOLA in Grafana", "channel_url": "https://www.youtube.com/channel/UC6Om9kAkl32dWlDSNlDS9Iw", "duration": 1960, "channel": "DEFCONConference", "uploader": "DEFCONConference", "upload_date": "20241016" } 0.00s - 2.00s | This text was transcribed using whisper model: large-v2 Alright, it is my fantastic 4.30s - 6.30s | Opportunity to introduce 6.32s - 15.72s | Ravid and Jay talking about bolobuster. All right. All right. Check check everybody hear me 17.26s - 19.26s | Okay, great 19.42s - 26.82s | So good morning everybody and thanks for joining. We are really excited to present in the absec village and 27.76s - 30.82s | Today we are going to talk about bolobuster 31.82s - 38.30s | Our methodology that we developed in order to automate detection at scale using 38.82s - 39.90s | LLMs 39.90s - 49.88s | But first let's introduce ourselves. So my name is Ravid. I'm a security researcher at Palo Alto Networks. I 51.00s - 56.84s | expertise in the WAS field which is a web application and API security mainly and 57.72s - 60.44s | In my free time, I enjoy watching football games 61.40s - 63.40s | traveling the world 63.48s - 71.30s | And take care of my dog Mabel. Hi, my name is Jay 71.30s - 77.18s | I am a security researcher with Palo Alto Networks and my research has been focusing on identifying the three 77.82s - 80.06s | threat and risk in cloud environment and 80.38s - 87.82s | Recently I've shift my research focus towards generative


chunk_text:
Jay talking about bolobuster. All right. All right. Check check everybody hear me 17.26s - 19.26s | Okay, great 19.42s - 26.82s | So good morning everybody and thanks for joining. We are really excited to present in the absec village and 27.76s - 30.82s | Today we are going to talk about bolobuster 31.82s - 38.30s | Our methodology that we developed in order to automate detection at scale using 38.82s - 39.90s | LLMs 39.90s - 49.88s | But first let's introduce ourselves. So my name is Ravid. I'm a security researcher at Palo Alto Networks. I 51.00s - 56.84s | expertise in the WAS field which is a web application and API security mainly and 57.72s - 60.44s | In my free time, I enjoy watching football games 61.40s - 63.40s | traveling the world 63.48s - 71.30s | And take care of my dog Mabel. Hi, my name is Jay 71.30s - 77.18s | I am a security researcher with Palo Alto Networks and my research has been focusing on identifying the three 77.82s - 80.06s | threat and risk in cloud environment and 80.38s - 87.82s | Recently I've shift my research focus towards generative AI in particular looking into the potentially adversarial use of 89.46s - 95.66s | Generative AI and I'm not not working. I spend most of time with my hyperactive twin boys 95.66s - 97.66s | they behave just like 97.70s - 100.58s | Minions and head off to the parent there with young kids 100.58s - 105.90s | I know how difficult it is to take care of young kids when when working and I'm also a K lover 105.90s - 113.62s | So when I'm I also spend a lot of time scrolling to funny K video when I'm not working or working 118.01s - 125.13s | Great. So let's go over the agenda for today. So we are going to cover the concept of bola 125.61s - 127.53s | We will show you our 127.53s - 130.45s | methodology of automating bola hunting using LLM's 130.89s - 134.69s | You are going to see a real bola testing. That is part of our 135.49s - 136.97s | methodology and 136.97s - 142.77s | Finally, we will see how we you will see how we answer down 17 new ball of vulnerabilities 143.53s - 148.21s | in one of


chunk_text:
AI in particular looking into the potentially adversarial use of 89.46s - 95.66s | Generative AI and I'm not not working. I spend most of time with my hyperactive twin boys 95.66s - 97.66s | they behave just like 97.70s - 100.58s | Minions and head off to the parent there with young kids 100.58s - 105.90s | I know how difficult it is to take care of young kids when when working and I'm also a K lover 105.90s - 113.62s | So when I'm I also spend a lot of time scrolling to funny K video when I'm not working or working 118.01s - 125.13s | Great. So let's go over the agenda for today. So we are going to cover the concept of bola 125.61s - 127.53s | We will show you our 127.53s - 130.45s | methodology of automating bola hunting using LLM's 130.89s - 134.69s | You are going to see a real bola testing. That is part of our 135.49s - 136.97s | methodology and 136.97s - 142.77s | Finally, we will see how we you will see how we answer down 17 new ball of vulnerabilities 143.53s - 148.21s | in one of the most popular opens of project out there and 149.05s - 155.98s | Yeah, so broken object level authorization or short for bola 156.86s - 158.74s | So 158.74s - 162.82s | Our motivation for the research was pretty huge. I would say 163.42s - 168.58s | For those of you who doesn't familiar one of familiar with bola. It is ranked at the top 169.86s - 175.10s | Risk at the OSP API top 10 and it is also the fourth most 175.90s - 180.58s | Reported vulnerability in a k1. So it's really popular and it's really severe 181.46s - 182.62s | and 182.66s - 187.18s | Yet there is no automation tool that an actually and reliably 188.10s - 189.86s | detectable light scale 189.86s - 191.42s | so this 191.42s - 196.58s | All of this was a big reason for us to try to solve basically an unsolved issue 197.14s - 200.82s | today and 200.82s - 203.98s | Let's understand first. What is exactly bore? 204.66s - 206.82s | So if you take a look at this 207.62s - 211.66s | example, we have a patient we have a clinic application and 212.18s -


chunk_text:
the most popular opens of project out there and 149.05s - 155.98s | Yeah, so broken object level authorization or short for bola 156.86s - 158.74s | So 158.74s - 162.82s | Our motivation for the research was pretty huge. I would say 163.42s - 168.58s | For those of you who doesn't familiar one of familiar with bola. It is ranked at the top 169.86s - 175.10s | Risk at the OSP API top 10 and it is also the fourth most 175.90s - 180.58s | Reported vulnerability in a k1. So it's really popular and it's really severe 181.46s - 182.62s | and 182.66s - 187.18s | Yet there is no automation tool that an actually and reliably 188.10s - 189.86s | detectable light scale 189.86s - 191.42s | so this 191.42s - 196.58s | All of this was a big reason for us to try to solve basically an unsolved issue 197.14s - 200.82s | today and 200.82s - 203.98s | Let's understand first. What is exactly bore? 204.66s - 206.82s | So if you take a look at this 207.62s - 211.66s | example, we have a patient we have a clinic application and 212.18s - 216.72s | We have a patient that can query is visit ID and get is dr. Notes 217.50s - 219.50s | pretty straightforward 219.54s - 223.54s | but what would happen if this patient would decide to act maliciously and try to 224.42s - 227.28s | Fetch another visit ID that does not belong to him 228.42s - 230.42s | If you would be able to do it 230.66s - 237.10s | It means that the back-end will have an authorization issue and this is basically a bola. So 238.10s - 242.66s | In a straightforward way if I can delete or modify 243.78s - 249.30s | Let's say J's comment from Twitter or Instagram. This is a clear bola because I 250.14s - 256.70s | shouldn't have the permission to access modify or delete another user object that does not belong to me and 257.74s - 259.74s | The consequences can be huge 259.94s - 266.10s | It can start from data leaks to data manipulation and even the full account takeover as we will see today 266.58s - 275.66s | We faced many challenges when we try to automate the process of moral detection 276.64s - 279.44s |


chunk_text:
216.72s | We have a patient that can query is visit ID and get is dr. Notes 217.50s - 219.50s | pretty straightforward 219.54s - 223.54s | but what would happen if this patient would decide to act maliciously and try to 224.42s - 227.28s | Fetch another visit ID that does not belong to him 228.42s - 230.42s | If you would be able to do it 230.66s - 237.10s | It means that the back-end will have an authorization issue and this is basically a bola. So 238.10s - 242.66s | In a straightforward way if I can delete or modify 243.78s - 249.30s | Let's say J's comment from Twitter or Instagram. This is a clear bola because I 250.14s - 256.70s | shouldn't have the permission to access modify or delete another user object that does not belong to me and 257.74s - 259.74s | The consequences can be huge 259.94s - 266.10s | It can start from data leaks to data manipulation and even the full account takeover as we will see today 266.58s - 275.66s | We faced many challenges when we try to automate the process of moral detection 276.64s - 279.44s | The first was the complex authorization mechanism 280.04s - 281.16s | so 281.16s - 283.16s | application these days actually as 283.48s - 287.60s | Typically multiple users with multiple roles with multiple resources 288.32s - 291.80s | So it's very difficult to say which user can do what? 292.44s - 298.68s | Which user is is allowed to do which action exactly and this really can get complex 300.48s - 303.12s | Also applications are stateful today 303.40s - 310.04s | so each action you make or each API call you make can affect the state of the application and 311.44s - 317.16s | Basically each endpoint is dependent on on one another. So imagine you try to delete a 318.20s - 320.20s | comment from an article or a post 321.36s - 327.00s | So first you would need to create this post and create a comment for this post and only then you will be able to delete 327.00s - 331.16s | This comment so that depended on each other. You cannot isolate 332.84s - 340.16s | One of them and the execution order is really important and in order to not not have any technical issue


chunk_text:
The first was the complex authorization mechanism 280.04s - 281.16s | so 281.16s - 283.16s | application these days actually as 283.48s - 287.60s | Typically multiple users with multiple roles with multiple resources 288.32s - 291.80s | So it's very difficult to say which user can do what? 292.44s - 298.68s | Which user is is allowed to do which action exactly and this really can get complex 300.48s - 303.12s | Also applications are stateful today 303.40s - 310.04s | so each action you make or each API call you make can affect the state of the application and 311.44s - 317.16s | Basically each endpoint is dependent on on one another. So imagine you try to delete a 318.20s - 320.20s | comment from an article or a post 321.36s - 327.00s | So first you would need to create this post and create a comment for this post and only then you will be able to delete 327.00s - 331.16s | This comment so that depended on each other. You cannot isolate 332.84s - 340.16s | One of them and the execution order is really important and in order to not not have any technical issue 341.32s - 346.56s | So it will it is a big challenge to understand that this endpoint depends 347.28s - 352.30s | dependencies actually also bola doesn't have a 353.26s - 356.90s | There is a lack of vulnerability indicator, basically. So unlike 357.74s - 360.54s | XSS or SQL injection, which is pretty straightforward 361.94s - 364.22s | To recognize the pattern of the payload 364.78s - 369.18s | Bola is a logical error. So it's not really malicious by definition 369.42s - 373.42s | So it's hard to analyze and understand whether it is a ball or not 374.06s - 378.90s | and lastly the context so 379.94s - 385.22s | We try to understand which endpoint in the application and which parameter contains sensitive data 386.26s - 388.66s | And it's not that easy to understand it 389.54s - 391.22s | and also 391.22s - 395.62s | To understand what is the impact actually of each action that we can take in the application? 396.54s - 399.50s | so all of this was pretty 401.62s - 411.66s | Big challenges that we face during the process. Thanks Ravi for covering the background 411.66s - 414.54s | I


chunk_text:
341.32s - 346.56s | So it will it is a big challenge to understand that this endpoint depends 347.28s - 352.30s | dependencies actually also bola doesn't have a 353.26s - 356.90s | There is a lack of vulnerability indicator, basically. So unlike 357.74s - 360.54s | XSS or SQL injection, which is pretty straightforward 361.94s - 364.22s | To recognize the pattern of the payload 364.78s - 369.18s | Bola is a logical error. So it's not really malicious by definition 369.42s - 373.42s | So it's hard to analyze and understand whether it is a ball or not 374.06s - 378.90s | and lastly the context so 379.94s - 385.22s | We try to understand which endpoint in the application and which parameter contains sensitive data 386.26s - 388.66s | And it's not that easy to understand it 389.54s - 391.22s | and also 391.22s - 395.62s | To understand what is the impact actually of each action that we can take in the application? 396.54s - 399.50s | so all of this was pretty 401.62s - 411.66s | Big challenges that we face during the process. Thanks Ravi for covering the background 411.66s - 414.54s | I hope everyone understand why automating data 415.26s - 418.10s | Bola detection is difficult. So 418.82s - 425.14s | It was only two years ago that we realized that AI may be able to give this problem a glimpse of hopes 426.18s - 428.50s | Because of the recent advancement of AI 429.30s - 434.90s | We were provided with new tool to solve the problem that were not possible to solve in the past 435.46s - 437.46s | So and also in particular 438.26s - 440.26s | our challenge of 440.98s - 448.02s | extracting context and logic information from textual data aligns well with the capability of 448.94s - 455.58s | Large language model. So that's the start of our journey in using AI to solve Bola detection 457.10s - 459.30s | Here is a high-level overview of our 459.90s - 466.66s | Methodology the only required input of our methodology is an open API spec or a swagger spec 467.86s - 469.94s | the first stage identify the 470.54s - 477.02s | Endpoint that can be vulnerable to Bola not every endpoint can be vulnerable to Bola 477.22s - 482.38s | We use AI to help analyze


chunk_text:
hope everyone understand why automating data 415.26s - 418.10s | Bola detection is difficult. So 418.82s - 425.14s | It was only two years ago that we realized that AI may be able to give this problem a glimpse of hopes 426.18s - 428.50s | Because of the recent advancement of AI 429.30s - 434.90s | We were provided with new tool to solve the problem that were not possible to solve in the past 435.46s - 437.46s | So and also in particular 438.26s - 440.26s | our challenge of 440.98s - 448.02s | extracting context and logic information from textual data aligns well with the capability of 448.94s - 455.58s | Large language model. So that's the start of our journey in using AI to solve Bola detection 457.10s - 459.30s | Here is a high-level overview of our 459.90s - 466.66s | Methodology the only required input of our methodology is an open API spec or a swagger spec 467.86s - 469.94s | the first stage identify the 470.54s - 477.02s | Endpoint that can be vulnerable to Bola not every endpoint can be vulnerable to Bola 477.22s - 482.38s | We use AI to help analyze every endpoint and its parameters 482.50s - 486.66s | To determine to select a subset of endpoint as our target 487.78s - 489.78s | this step help us 491.26s - 498.26s | Narrow down our target endpoint and avoid wasting time on the endpoint that are not at risk 498.54s - 500.54s | we call this endpoint as 501.22s - 502.86s | potentially vulnerable 502.86s - 504.86s | endpoint short for PVE or 505.62s - 512.02s | Target endpoint. Well, we will refer to this endpoint as target endpoint or PVE in the rest of the talk 512.70s - 517.06s | The next stage uncovers the dependency relationship between endpoint 517.98s - 520.78s | modern web application are complex typically 521.34s - 527.02s | one endpoint depending down many others for example in order to test an endpoint that 527.42s - 532.46s | update an invoice you first need to call the endpoint that create the 533.34s - 538.06s | Invoice and in order to create an invoice you first need to call the endpoint that 538.42s - 544.02s | Create the transactions that can be included in the endpoint as a result. It is crucial to 544.82s - 550.90s


chunk_text:
every endpoint and its parameters 482.50s - 486.66s | To determine to select a subset of endpoint as our target 487.78s - 489.78s | this step help us 491.26s - 498.26s | Narrow down our target endpoint and avoid wasting time on the endpoint that are not at risk 498.54s - 500.54s | we call this endpoint as 501.22s - 502.86s | potentially vulnerable 502.86s - 504.86s | endpoint short for PVE or 505.62s - 512.02s | Target endpoint. Well, we will refer to this endpoint as target endpoint or PVE in the rest of the talk 512.70s - 517.06s | The next stage uncovers the dependency relationship between endpoint 517.98s - 520.78s | modern web application are complex typically 521.34s - 527.02s | one endpoint depending down many others for example in order to test an endpoint that 527.42s - 532.46s | update an invoice you first need to call the endpoint that create the 533.34s - 538.06s | Invoice and in order to create an invoice you first need to call the endpoint that 538.42s - 544.02s | Create the transactions that can be included in the endpoint as a result. It is crucial to 544.82s - 550.90s | Understand the endpoint that my target endpoint depend on before I can accurately test it 552.22s - 556.38s | With the endpoint dependency relationship identifies the next stage 557.10s - 561.86s | calculate the execution path to each target endpoint and 562.46s - 565.38s | Create a test plan for each target endpoint 566.86s - 571.14s | The next stage then transform the test plan into 571.46s - 575.74s | Multiple executable bash script using large language model 576.06s - 583.90s | There may be one or multiple execution path to each target endpoint and we aim to cover as many path as possible 584.94s - 586.94s | finally, we set up a 587.22s - 589.22s | target API server and 589.38s - 591.38s | Run all the test script 591.66s - 595.86s | Generated in the previous step the process of user registration 596.14s - 603.58s | User locking and token refresh have all been automated and we also use AI to help analyze the response and logs 604.06s - 614.37s | Produced during the test. Let's dive into a little bit more detail in each stage in the first stage 614.61s - 620.25s | We try to


chunk_text:
| Understand the endpoint that my target endpoint depend on before I can accurately test it 552.22s - 556.38s | With the endpoint dependency relationship identifies the next stage 557.10s - 561.86s | calculate the execution path to each target endpoint and 562.46s - 565.38s | Create a test plan for each target endpoint 566.86s - 571.14s | The next stage then transform the test plan into 571.46s - 575.74s | Multiple executable bash script using large language model 576.06s - 583.90s | There may be one or multiple execution path to each target endpoint and we aim to cover as many path as possible 584.94s - 586.94s | finally, we set up a 587.22s - 589.22s | target API server and 589.38s - 591.38s | Run all the test script 591.66s - 595.86s | Generated in the previous step the process of user registration 596.14s - 603.58s | User locking and token refresh have all been automated and we also use AI to help analyze the response and logs 604.06s - 614.37s | Produced during the test. Let's dive into a little bit more detail in each stage in the first stage 614.61s - 620.25s | We try to identify the endpoint with input parameter that reference private sensitive or confidential 621.25s - 626.97s | Information these are the endpoint that we target. These are our target endpoint 628.21s - 633.17s | For example, let's look at the first endpoint the username here indicated 633.69s - 640.33s | Endpoint may reference to data associated with in the individual if this endpoint is vulnerable 640.57s - 645.17s | Vulnerable to bola an attacker may be able to reset another users 645.81s - 647.17s | password 647.17s - 650.01s | Similarly the second if the second endpoint 650.81s - 653.05s | put user email is 654.21s - 659.97s | Vulnerable to bola an attacker may be able to change another users email 661.69s - 663.69s | Traditionally pen tester 664.53s - 671.61s | Manually look for this endpoint and their parameters to determine their target endpoint 671.77s - 674.49s | This process is slow and cumbersome 675.33s - 683.37s | especially for very large applications with hundreds of endpoints, so we use AI's capabilities of 684.21s - 687.77s | Reasoning and understanding text to automate this step 689.25s - 696.25s | Here is a snippet of our prompt that show how


chunk_text:
identify the endpoint with input parameter that reference private sensitive or confidential 621.25s - 626.97s | Information these are the endpoint that we target. These are our target endpoint 628.21s - 633.17s | For example, let's look at the first endpoint the username here indicated 633.69s - 640.33s | Endpoint may reference to data associated with in the individual if this endpoint is vulnerable 640.57s - 645.17s | Vulnerable to bola an attacker may be able to reset another users 645.81s - 647.17s | password 647.17s - 650.01s | Similarly the second if the second endpoint 650.81s - 653.05s | put user email is 654.21s - 659.97s | Vulnerable to bola an attacker may be able to change another users email 661.69s - 663.69s | Traditionally pen tester 664.53s - 671.61s | Manually look for this endpoint and their parameters to determine their target endpoint 671.77s - 674.49s | This process is slow and cumbersome 675.33s - 683.37s | especially for very large applications with hundreds of endpoints, so we use AI's capabilities of 684.21s - 687.77s | Reasoning and understanding text to automate this step 689.25s - 696.25s | Here is a snippet of our prompt that show how we teach AI to recognize 697.21s - 699.97s | parameters that may reference to 700.53s - 702.21s | Sensitive data 702.21s - 707.05s | Essentially we instruct AI a set of rules and examples 708.25s - 715.53s | At the end AI returns us the endpoint and parameters that meet any of the condition 715.53s - 722.76s | That we provided. The next stage uncovers the dependency relationship between endpoint 723.52s - 728.24s | As I mentioned due to the complexity of modern web application 729.20s - 733.80s | Each API call may change the state of the application and 734.72s - 740.72s | Affect the outcome of other API on a other API endpoint as a result 740.72s - 746.16s | It is crucial to identify their dependency relationship before we can test them 747.72s - 754.68s | In this diagram we call the endpoint on the right as consumers and the endpoint on the left as producers 755.68s - 761.72s | Producer endpoint output the values that consumer endpoint need as input 762.08s - 764.16s | again producer endpoint 764.72s - 769.60s | Output the values that consumer endpoint use as input 769.64s - 774.72s | This is


chunk_text:
we teach AI to recognize 697.21s - 699.97s | parameters that may reference to 700.53s - 702.21s | Sensitive data 702.21s - 707.05s | Essentially we instruct AI a set of rules and examples 708.25s - 715.53s | At the end AI returns us the endpoint and parameters that meet any of the condition 715.53s - 722.76s | That we provided. The next stage uncovers the dependency relationship between endpoint 723.52s - 728.24s | As I mentioned due to the complexity of modern web application 729.20s - 733.80s | Each API call may change the state of the application and 734.72s - 740.72s | Affect the outcome of other API on a other API endpoint as a result 740.72s - 746.16s | It is crucial to identify their dependency relationship before we can test them 747.72s - 754.68s | In this diagram we call the endpoint on the right as consumers and the endpoint on the left as producers 755.68s - 761.72s | Producer endpoint output the values that consumer endpoint need as input 762.08s - 764.16s | again producer endpoint 764.72s - 769.60s | Output the values that consumer endpoint use as input 769.64s - 774.72s | This is one of the most important concept in our research in our methodology 775.20s - 781.64s | Let's look at this endpoint as an example delete user in order to correctly 782.16s - 783.92s | Test this endpoint 783.92s - 787.72s | We need to feed it with an existing user name 787.88s - 792.64s | If we feed it with a random variable a random user name 792.72s - 798.76s | This endpoint will always API code will always fail and give us meaningless result 798.92s - 803.36s | That's why we need to identify its producers in this case 803.52s - 804.56s | this 804.56s - 812.32s | Consumer endpoint has four producers and each of them can output the existing user name that endpoint can use 812.96s - 819.64s | Here is a snippet of the prompt that we use to teach AI how to 820.48s - 822.60s | recognize dependency relationship I 823.80s - 829.96s | Want to emphasize that also we can use heuristic algorithm to match the output 830.40s - 834.92s | parameter name of an endpoint with the input parameter name of a 835.52s - 836.76s | Consumer 836.76s - 841.52s | This heuristic


chunk_text:
one of the most important concept in our research in our methodology 775.20s - 781.64s | Let's look at this endpoint as an example delete user in order to correctly 782.16s - 783.92s | Test this endpoint 783.92s - 787.72s | We need to feed it with an existing user name 787.88s - 792.64s | If we feed it with a random variable a random user name 792.72s - 798.76s | This endpoint will always API code will always fail and give us meaningless result 798.92s - 803.36s | That's why we need to identify its producers in this case 803.52s - 804.56s | this 804.56s - 812.32s | Consumer endpoint has four producers and each of them can output the existing user name that endpoint can use 812.96s - 819.64s | Here is a snippet of the prompt that we use to teach AI how to 820.48s - 822.60s | recognize dependency relationship I 823.80s - 829.96s | Want to emphasize that also we can use heuristic algorithm to match the output 830.40s - 834.92s | parameter name of an endpoint with the input parameter name of a 835.52s - 836.76s | Consumer 836.76s - 841.52s | This heuristic algorithm is not reliable. The reason is that 842.44s - 850.24s | Developers sometimes use different parameter names to reference the same data object within the system 851.36s - 859.12s | Developer may also use the same parameter names to reference different data object across the system 859.40s - 866.36s | That's that's why we use a combination of heuristic and AI to uncover the dependency relationship 866.40s - 869.40s | We we need AI to analyze each 870.64s - 873.28s | endpoint functionality in order to 873.96s - 878.40s | Establish their dependency relationship based on their functionalities 881.86s - 887.74s | Once we have the using the dependency relationship uncover in the previous stage 888.34s - 892.62s | This stage build up a dependency tree for each 893.22s - 901.25s | potentially vulnerable endpoint in this dependency tree the root node represent our 901.81s - 904.65s | one of our target endpoint or PVE and 905.17s - 909.77s | Any two directly connected node in a tree? 910.41s - 914.53s | represent a consumer and producer relationship in which 915.01s - 920.29s | Consumer the the parent node is a consumer and the channel is a producer 921.69s


chunk_text:
algorithm is not reliable. The reason is that 842.44s - 850.24s | Developers sometimes use different parameter names to reference the same data object within the system 851.36s - 859.12s | Developer may also use the same parameter names to reference different data object across the system 859.40s - 866.36s | That's that's why we use a combination of heuristic and AI to uncover the dependency relationship 866.40s - 869.40s | We we need AI to analyze each 870.64s - 873.28s | endpoint functionality in order to 873.96s - 878.40s | Establish their dependency relationship based on their functionalities 881.86s - 887.74s | Once we have the using the dependency relationship uncover in the previous stage 888.34s - 892.62s | This stage build up a dependency tree for each 893.22s - 901.25s | potentially vulnerable endpoint in this dependency tree the root node represent our 901.81s - 904.65s | one of our target endpoint or PVE and 905.17s - 909.77s | Any two directly connected node in a tree? 910.41s - 914.53s | represent a consumer and producer relationship in which 915.01s - 920.29s | Consumer the the parent node is a consumer and the channel is a producer 921.69s - 925.57s | Let's use the upper half of a tree as an example in this case 926.65s - 928.49s | endpoint is a 928.49s - 930.49s | producer of PVE 930.93s - 934.85s | endpoint output the value that PVE need as the input and 935.33s - 941.97s | Endpoint one itself also has two producer endpoint three and four which produce the 942.45s - 945.01s | Output the endpoint one need us 945.65s - 948.78s | the input 948.78s - 954.34s | Finally in the next stage we turn we calculate the execution path 955.10s - 958.70s | to each potentially vulnerable endpoint 960.18s - 967.30s | so in the dependency tree a path from the leaf node to the root node represent an 968.74s - 970.26s | execution path 970.26s - 971.74s | This is a 971.74s - 972.82s | reverse 972.82s - 973.98s | tree traversal 973.98s - 981.26s | so the path is for actually from the leaf node to the root node instead of from the root node to the leaf node and 982.10s - 987.62s | Let's plug in some real endpoint to see how we do the calculation in this example 987.86s


chunk_text:
- 925.57s | Let's use the upper half of a tree as an example in this case 926.65s - 928.49s | endpoint is a 928.49s - 930.49s | producer of PVE 930.93s - 934.85s | endpoint output the value that PVE need as the input and 935.33s - 941.97s | Endpoint one itself also has two producer endpoint three and four which produce the 942.45s - 945.01s | Output the endpoint one need us 945.65s - 948.78s | the input 948.78s - 954.34s | Finally in the next stage we turn we calculate the execution path 955.10s - 958.70s | to each potentially vulnerable endpoint 960.18s - 967.30s | so in the dependency tree a path from the leaf node to the root node represent an 968.74s - 970.26s | execution path 970.26s - 971.74s | This is a 971.74s - 972.82s | reverse 972.82s - 973.98s | tree traversal 973.98s - 981.26s | so the path is for actually from the leaf node to the root node instead of from the root node to the leaf node and 982.10s - 987.62s | Let's plug in some real endpoint to see how we do the calculation in this example 987.86s - 991.82s | The target endpoint is delete comment and this endpoint 992.46s - 996.94s | has two producers get comment and post comment and 997.86s - 1004.50s | And these two producers in turn also each has two producer so at the end 1005.02s - 1012.18s | There are four execution path that can all reach the target endpoint delete comment in 1014.68s - 1017.40s | the last stage we then turn each 1018.04s - 1023.84s | execution path into an executable bash script using large language model and 1024.84s - 1031.48s | Run and run the script to actually send HTTP request to the target endpoint 1032.88s - 1035.92s | the process is more complicated than just generating and 1036.68s - 1040.40s | Sending the request as rabbit will explain in the next few site 1044.41s - 1048.37s | great, thank you Jay for the overview of our methodology and 1049.89s - 1053.69s | Before we can see actually a real bola test and 1054.33s - 1057.53s | View our bola that we found. Let's see 1058.09s - 1061.69s | The prerequisites that we have before we can create an actual test 1062.09s - 1069.97s | So we


chunk_text:
- 991.82s | The target endpoint is delete comment and this endpoint 992.46s - 996.94s | has two producers get comment and post comment and 997.86s - 1004.50s | And these two producers in turn also each has two producer so at the end 1005.02s - 1012.18s | There are four execution path that can all reach the target endpoint delete comment in 1014.68s - 1017.40s | the last stage we then turn each 1018.04s - 1023.84s | execution path into an executable bash script using large language model and 1024.84s - 1031.48s | Run and run the script to actually send HTTP request to the target endpoint 1032.88s - 1035.92s | the process is more complicated than just generating and 1036.68s - 1040.40s | Sending the request as rabbit will explain in the next few site 1044.41s - 1048.37s | great, thank you Jay for the overview of our methodology and 1049.89s - 1053.69s | Before we can see actually a real bola test and 1054.33s - 1057.53s | View our bola that we found. Let's see 1058.09s - 1061.69s | The prerequisites that we have before we can create an actual test 1062.09s - 1069.97s | So we use LLMs again to create the user registration process and to collect the login data of the application 1071.37s - 1075.05s | So basically we generate user LLM generates user 1075.29s - 1081.49s | credentials that meets the criteria in terms of the for example password complexity and etc and 1082.49s - 1083.93s | We 1083.93s - 1089.09s | Execute the user registration process and we fetch the login data from the spec 1089.37s - 1096.17s | so this is crucial in order to then in login the users of the specific test and 1096.77s - 1098.49s | get a 1098.49s - 1100.77s | Valid and unique token for each one of them 1101.81s - 1107.13s | This information will be stored in a dedicated file. So we will have an easy access and usage 1109.20s - 1112.28s | Then we will want to isolate the test data 1113.42s - 1119.56s | So if you ever saw a swagger or open API spec file, you know that it can be huge 1119.56s - 1121.56s | It can be thousand lines of code 1122.24s - 1125.08s | especially for the biggest for the bigger applications and 1126.04s - 1131.12s | We


chunk_text:
use LLMs again to create the user registration process and to collect the login data of the application 1071.37s - 1075.05s | So basically we generate user LLM generates user 1075.29s - 1081.49s | credentials that meets the criteria in terms of the for example password complexity and etc and 1082.49s - 1083.93s | We 1083.93s - 1089.09s | Execute the user registration process and we fetch the login data from the spec 1089.37s - 1096.17s | so this is crucial in order to then in login the users of the specific test and 1096.77s - 1098.49s | get a 1098.49s - 1100.77s | Valid and unique token for each one of them 1101.81s - 1107.13s | This information will be stored in a dedicated file. So we will have an easy access and usage 1109.20s - 1112.28s | Then we will want to isolate the test data 1113.42s - 1119.56s | So if you ever saw a swagger or open API spec file, you know that it can be huge 1119.56s - 1121.56s | It can be thousand lines of code 1122.24s - 1125.08s | especially for the biggest for the bigger applications and 1126.04s - 1131.12s | We want to isolate the only the data that is needed for the specific test 1132.12s - 1135.88s | So Jay mentioned that we have a consumer and producers in each test 1135.88s - 1141.12s | So we would like to have only the data which is relevant to this endpoints 1142.56s - 1146.24s | So we will create a new file. We will feed it to AI and 1146.84s - 1151.68s | It will save us about 90 to 95 percent of the of the spec 1152.92s - 1155.60s | so we do it for efficiency to ensure that a 1156.24s - 1161.76s | LLM will get only the accurate data that actually needs for each test creation and 1162.76s - 1165.80s | For cost we want to have as lowest 1166.40s - 1170.80s | tokens as we can and 1171.16s - 1177.44s | Finally we creating the test script and it will be saved as an executable bash script 1178.20s - 1182.04s | You can see an example of a very simple 1182.72s - 1188.76s | Consumer and producer we have a put user password and a get user to get the user name 1189.52s - 1192.88s | This


chunk_text:
want to isolate the only the data that is needed for the specific test 1132.12s - 1135.88s | So Jay mentioned that we have a consumer and producers in each test 1135.88s - 1141.12s | So we would like to have only the data which is relevant to this endpoints 1142.56s - 1146.24s | So we will create a new file. We will feed it to AI and 1146.84s - 1151.68s | It will save us about 90 to 95 percent of the of the spec 1152.92s - 1155.60s | so we do it for efficiency to ensure that a 1156.24s - 1161.76s | LLM will get only the accurate data that actually needs for each test creation and 1162.76s - 1165.80s | For cost we want to have as lowest 1166.40s - 1170.80s | tokens as we can and 1171.16s - 1177.44s | Finally we creating the test script and it will be saved as an executable bash script 1178.20s - 1182.04s | You can see an example of a very simple 1182.72s - 1188.76s | Consumer and producer we have a put user password and a get user to get the user name 1189.52s - 1192.88s | This kind of test will be saved in a put get directory 1193.40s - 1200.52s | Put would be the parent get will be the child and the test generation actually runs 1200.92s - 1203.00s | asynchronously in order to save time and 1204.72s - 1206.72s | Right now we averaged 1207.24s - 1210.48s | 1.5 second for each test creation 1213.25s - 1219.97s | Right. So the final step would be the to actually run the test script in the application 1220.97s - 1222.97s | So we have an execution order 1223.57s - 1230.01s | And our main goal is to avoid any chance that a test would fail due to technical reasons 1230.65s - 1235.77s | So we want the test to fail only if the PVE is not vulnerable to bola as should be 1236.97s - 1243.05s | so we will first run the test that populate data and resources to the application before we can fetch them and 1245.00s - 1249.72s | Also, we will try to push and delete to push update and delete operation to the end 1250.08s - 1251.92s | So we want 1251.92s - 1255.88s | Experience a scenario where we delete the


chunk_text:
kind of test will be saved in a put get directory 1193.40s - 1200.52s | Put would be the parent get will be the child and the test generation actually runs 1200.92s - 1203.00s | asynchronously in order to save time and 1204.72s - 1206.72s | Right now we averaged 1207.24s - 1210.48s | 1.5 second for each test creation 1213.25s - 1219.97s | Right. So the final step would be the to actually run the test script in the application 1220.97s - 1222.97s | So we have an execution order 1223.57s - 1230.01s | And our main goal is to avoid any chance that a test would fail due to technical reasons 1230.65s - 1235.77s | So we want the test to fail only if the PVE is not vulnerable to bola as should be 1236.97s - 1243.05s | so we will first run the test that populate data and resources to the application before we can fetch them and 1245.00s - 1249.72s | Also, we will try to push and delete to push update and delete operation to the end 1250.08s - 1251.92s | So we want 1251.92s - 1255.88s | Experience a scenario where we delete the user or delete the resource and then try to fetch it 1256.48s - 1262.48s | So again, the main the main reason for this is is not to have technical issues in a test 1263.44s - 1265.44s | That will make it fail eventually 1265.68s - 1269.79s | so right after we 1271.15s - 1276.15s | Finished our preparing our methodology. We wanted to evaluate our 1277.15s - 1279.59s | Bola buster actually and see the result 1280.19s - 1287.63s | So what we did is we took the maybe the best API fuzzer, which is open source right now, which is wrestler 1288.79s - 1293.75s | Wrestler is a Microsoft developed API fuzzer. You can find on them github and 1294.75s - 1296.83s | It's goal is to automate the process of 1297.91s - 1299.99s | API testing and to find 1300.75s - 1302.75s | vulnerabilities like our goal 1303.15s - 1307.39s | So what we did is we took free open source application 1307.91s - 1311.67s | Which are deliberately vulnerable to the OWASP API top-10 1312.43s - 1318.99s | We took vampy capital and crappy. You can see from the table that they have a known bolas


chunk_text:
user or delete the resource and then try to fetch it 1256.48s - 1262.48s | So again, the main the main reason for this is is not to have technical issues in a test 1263.44s - 1265.44s | That will make it fail eventually 1265.68s - 1269.79s | so right after we 1271.15s - 1276.15s | Finished our preparing our methodology. We wanted to evaluate our 1277.15s - 1279.59s | Bola buster actually and see the result 1280.19s - 1287.63s | So what we did is we took the maybe the best API fuzzer, which is open source right now, which is wrestler 1288.79s - 1293.75s | Wrestler is a Microsoft developed API fuzzer. You can find on them github and 1294.75s - 1296.83s | It's goal is to automate the process of 1297.91s - 1299.99s | API testing and to find 1300.75s - 1302.75s | vulnerabilities like our goal 1303.15s - 1307.39s | So what we did is we took free open source application 1307.91s - 1311.67s | Which are deliberately vulnerable to the OWASP API top-10 1312.43s - 1318.99s | We took vampy capital and crappy. You can see from the table that they have a known bolas and 1320.15s - 1327.50s | We started to test our tool against the wrestler as you can see wrestler could not 1328.14s - 1330.58s | identify any bola of this up for 1331.38s - 1339.22s | Applications and it made a pretty big number of API calls and thousands and even hundred of thousands 1341.10s - 1346.26s | Then we tested our tool again the same application and the results were amazing I 1347.26s - 1355.14s | Hope you can see we found all the bolas in each one of the applications and we made significantly lower API calls 1355.98s - 1360.42s | So eventually we made less than 1% of the calls that wrestler made 1361.58s - 1367.82s | So this is a crucial part. It proved our efficiency and we didn't make 1369.54s - 1378.93s | The application load too much and finally we wanted to have a 100% true positive rate 1380.01s - 1385.33s | So our goal was to find any bola if there if it exists in application 1386.25s - 1389.17s | We we did have some false positive. You can't avoid it 1389.77s - 1395.05s | But again, our main goal was to focus


chunk_text:
and 1320.15s - 1327.50s | We started to test our tool against the wrestler as you can see wrestler could not 1328.14s - 1330.58s | identify any bola of this up for 1331.38s - 1339.22s | Applications and it made a pretty big number of API calls and thousands and even hundred of thousands 1341.10s - 1346.26s | Then we tested our tool again the same application and the results were amazing I 1347.26s - 1355.14s | Hope you can see we found all the bolas in each one of the applications and we made significantly lower API calls 1355.98s - 1360.42s | So eventually we made less than 1% of the calls that wrestler made 1361.58s - 1367.82s | So this is a crucial part. It proved our efficiency and we didn't make 1369.54s - 1378.93s | The application load too much and finally we wanted to have a 100% true positive rate 1380.01s - 1385.33s | So our goal was to find any bola if there if it exists in application 1386.25s - 1389.17s | We we did have some false positive. You can't avoid it 1389.77s - 1395.05s | But again, our main goal was to focus on the on the true positive rate, which was 1396.25s - 1401.08s | 100% 1401.08s - 1404.36s | Now we can take a look on a real scenario for a bola test 1405.18s - 1410.24s | So we have this eye level example where first Alice create an article 1410.80s - 1417.64s | Then Alice create a comment for this article and then Bob will try to delete Alice comment, which is the PVE 1418.20s - 1425.88s | So first Alice and Bob will log into the system. They will get a unique token, which will be saved 1427.95s - 1432.31s | Alice will create a new article in the system. We will save the article title 1432.83s - 1438.92s | We will do the same with a comment. So Alice will create a comment. We will save the comment ID and 1440.16s - 1446.60s | finally the PVE the maliciously Bob will try to attempt to delete Alice comment and 1447.36s - 1449.36s | This scenario can have two results 1450.32s - 1452.32s | If it will be successful 1452.32s - 1459.96s | We will have a potentially bola for this endpoint or if the application can defend the


chunk_text:
on the on the true positive rate, which was 1396.25s - 1401.08s | 100% 1401.08s - 1404.36s | Now we can take a look on a real scenario for a bola test 1405.18s - 1410.24s | So we have this eye level example where first Alice create an article 1410.80s - 1417.64s | Then Alice create a comment for this article and then Bob will try to delete Alice comment, which is the PVE 1418.20s - 1425.88s | So first Alice and Bob will log into the system. They will get a unique token, which will be saved 1427.95s - 1432.31s | Alice will create a new article in the system. We will save the article title 1432.83s - 1438.92s | We will do the same with a comment. So Alice will create a comment. We will save the comment ID and 1440.16s - 1446.60s | finally the PVE the maliciously Bob will try to attempt to delete Alice comment and 1447.36s - 1449.36s | This scenario can have two results 1450.32s - 1452.32s | If it will be successful 1452.32s - 1459.96s | We will have a potentially bola for this endpoint or if the application can defend the bola as well 1459.96s - 1463.96s | So it will be forbidden 1463.96s - 1466.24s | Let's see the actual snippet of our code 1467.08s - 1472.48s | So first of all, we will create a unique random string each time 1472.88s - 1477.64s | so we will be able to create an article for multiple tests without 1478.44s - 1481.48s | Getting an arrow that it already exists 1483.08s - 1489.12s | So as you can see we make a POST request to slash API slash articles 1489.60s - 1494.56s | you can see that the we use user a token which reference to Alice and 1495.16s - 1497.80s | We use the article title as a random string 1500.97s - 1504.97s | We will save the slug which is in this case the article title 1508.28s - 1509.32s | next 1509.32s - 1511.72s | We will Alice will create a comment 1511.88s - 1516.40s | so what's special about this is that if you can if you take a look at the 1516.76s - 1520.56s | Actual endpoint you can see that we dynamically use leg 1521.08s - 1524.60s | so we saved this parameter from the 1525.24s


chunk_text:
bola as well 1459.96s - 1463.96s | So it will be forbidden 1463.96s - 1466.24s | Let's see the actual snippet of our code 1467.08s - 1472.48s | So first of all, we will create a unique random string each time 1472.88s - 1477.64s | so we will be able to create an article for multiple tests without 1478.44s - 1481.48s | Getting an arrow that it already exists 1483.08s - 1489.12s | So as you can see we make a POST request to slash API slash articles 1489.60s - 1494.56s | you can see that the we use user a token which reference to Alice and 1495.16s - 1497.80s | We use the article title as a random string 1500.97s - 1504.97s | We will save the slug which is in this case the article title 1508.28s - 1509.32s | next 1509.32s - 1511.72s | We will Alice will create a comment 1511.88s - 1516.40s | so what's special about this is that if you can if you take a look at the 1516.76s - 1520.56s | Actual endpoint you can see that we dynamically use leg 1521.08s - 1524.60s | so we saved this parameter from the 1525.24s - 1530.40s | Previous request and then we dynamically use it. So now we can reference to it as a 1531.20s - 1533.20s | We can know that this 1533.60s - 1539.54s | Resource actually belongs to Alice again. We save the comment ID 1541.30s - 1548.78s | And then finally we have the PVE and the delete operation and you can see again that we use slug and comment ID 1550.26s - 1557.14s | That belongs to Alice and we the main difference and now we use a the user be token 1557.30s - 1561.74s | so now we can identify that this request is made by Bob and not Alice and 1563.06s - 1565.06s | Lastly we will check if the test passed 1566.10s - 1568.10s | now I want to emphasize that 1568.54s - 1573.86s | Actually every ball of test that we make is if you think about it, it's malicious by definition 1574.54s - 1578.02s | the last request the PVE will always we will try to 1578.86s - 1582.62s | To perform an action that you shouldn't be able to perform, right? 1583.26s - 1588.58s | Bob shouldn't be able to delete Alice comment.


chunk_text:
- 1530.40s | Previous request and then we dynamically use it. So now we can reference to it as a 1531.20s - 1533.20s | We can know that this 1533.60s - 1539.54s | Resource actually belongs to Alice again. We save the comment ID 1541.30s - 1548.78s | And then finally we have the PVE and the delete operation and you can see again that we use slug and comment ID 1550.26s - 1557.14s | That belongs to Alice and we the main difference and now we use a the user be token 1557.30s - 1561.74s | so now we can identify that this request is made by Bob and not Alice and 1563.06s - 1565.06s | Lastly we will check if the test passed 1566.10s - 1568.10s | now I want to emphasize that 1568.54s - 1573.86s | Actually every ball of test that we make is if you think about it, it's malicious by definition 1574.54s - 1578.02s | the last request the PVE will always we will try to 1578.86s - 1582.62s | To perform an action that you shouldn't be able to perform, right? 1583.26s - 1588.58s | Bob shouldn't be able to delete Alice comment. So in case it is successful 1589.58s - 1594.78s | We can mark this test as at least potentially vulnerable to bola before we can make 1595.30s - 1596.42s | A 1596.42s - 1600.46s | Further analysis and verification. Let's say in the application context 1601.06s - 1606.22s | So you just need to understand that every every ball of test is is malicious by definition 1606.62s - 1614.37s | So, let's see how we actually entered some boas in the world 1614.89s - 1620.83s | We have a quick demo of showing the running of the tool 1622.19s - 1624.79s | so this this is a Python based and 1625.79s - 1632.99s | It will I hope you can see it's testing each API endpoint and in case the result is successful 1633.55s - 1636.07s | it will market with bola found and 1637.11s - 1639.23s | actual actual API endpoint 1640.91s - 1647.71s | After this, we will take the list and we'll further analyze it and try to understand if this is a real bola or not 1648.35s - 1652.98s | So all of these applications 1653.62s - 1656.54s | It has something in common. We found


chunk_text:
So in case it is successful 1589.58s - 1594.78s | We can mark this test as at least potentially vulnerable to bola before we can make 1595.30s - 1596.42s | A 1596.42s - 1600.46s | Further analysis and verification. Let's say in the application context 1601.06s - 1606.22s | So you just need to understand that every every ball of test is is malicious by definition 1606.62s - 1614.37s | So, let's see how we actually entered some boas in the world 1614.89s - 1620.83s | We have a quick demo of showing the running of the tool 1622.19s - 1624.79s | so this this is a Python based and 1625.79s - 1632.99s | It will I hope you can see it's testing each API endpoint and in case the result is successful 1633.55s - 1636.07s | it will market with bola found and 1637.11s - 1639.23s | actual actual API endpoint 1640.91s - 1647.71s | After this, we will take the list and we'll further analyze it and try to understand if this is a real bola or not 1648.35s - 1652.98s | So all of these applications 1653.62s - 1656.54s | It has something in common. We found balls in it 1657.50s - 1659.50s | so first of all Harbor and 1660.66s - 1667.94s | It is a cloud native container registry. It is basically equivalent to Docker Hub, which I'm sure most of you are familiar with 1668.94s - 1672.98s | It is it is a CNCF graduated project. So it's been 1673.94s - 1675.94s | In the wild for a while 1677.18s - 1684.30s | It is around 2 million downloads these days and we actually found this year we found a new bola 1685.14s - 1687.14s | Which we will talk in a bit 1688.26s - 1693.14s | And next up Grafana, it's a data visualization and monitoring tool 1693.70s - 1696.38s | very very popular is being used by 1697.18s - 1702.82s | 20 million users actually these days and we also found a new bola there and 1705.14s - 1708.38s | Lastly we have easy appointments, which is an open source tool 1709.50s - 1711.50s | It has almost 200 1712.38s - 1719.62s | Thousand downloads and we actually been able to find 15 new ball of vulnerabilities there. Yeah 1721.02s - 1723.02s | Little bit 1723.54s - 1728.06s | The most amazing


chunk_text:
balls in it 1657.50s - 1659.50s | so first of all Harbor and 1660.66s - 1667.94s | It is a cloud native container registry. It is basically equivalent to Docker Hub, which I'm sure most of you are familiar with 1668.94s - 1672.98s | It is it is a CNCF graduated project. So it's been 1673.94s - 1675.94s | In the wild for a while 1677.18s - 1684.30s | It is around 2 million downloads these days and we actually found this year we found a new bola 1685.14s - 1687.14s | Which we will talk in a bit 1688.26s - 1693.14s | And next up Grafana, it's a data visualization and monitoring tool 1693.70s - 1696.38s | very very popular is being used by 1697.18s - 1702.82s | 20 million users actually these days and we also found a new bola there and 1705.14s - 1708.38s | Lastly we have easy appointments, which is an open source tool 1709.50s - 1711.50s | It has almost 200 1712.38s - 1719.62s | Thousand downloads and we actually been able to find 15 new ball of vulnerabilities there. Yeah 1721.02s - 1723.02s | Little bit 1723.54s - 1728.06s | The most amazing thing about it is actually that seven of them are critical 1729.06s - 1731.06s | so they got a CVSS of 1731.50s - 1733.70s | 9.9 which actually allows a 1734.78s - 1741.14s | Low privilege user to gain admin access. Remember we talked about compromise. So we have a full 1742.10s - 1744.26s | application compromise in this case 1747.54s - 1750.86s | Let's talk about our vulnerability. It's pretty interesting 1752.46s - 1753.86s | So 1753.86s - 1760.94s | Harbor as a projects in it and you can have you can have roles so you can be an admin a maintainer a developer or 1760.94s - 1762.94s | a guest 1763.18s - 1769.30s | Basically we found the vulnerable feature to be the project configuration metadata 1770.30s - 1773.70s | So, I hope you can see the attributes there for the configuration 1774.50s - 1778.98s | Basically an admin can change the project to be private public 1779.22s - 1784.82s | You can implement some image vulnerability scanning and and have more power 1786.30s - 1791.94s | So our book claimed that in the official documentation that only the admin project should be able to 1792.42s - 1794.70s


chunk_text:
thing about it is actually that seven of them are critical 1729.06s - 1731.06s | so they got a CVSS of 1731.50s - 1733.70s | 9.9 which actually allows a 1734.78s - 1741.14s | Low privilege user to gain admin access. Remember we talked about compromise. So we have a full 1742.10s - 1744.26s | application compromise in this case 1747.54s - 1750.86s | Let's talk about our vulnerability. It's pretty interesting 1752.46s - 1753.86s | So 1753.86s - 1760.94s | Harbor as a projects in it and you can have you can have roles so you can be an admin a maintainer a developer or 1760.94s - 1762.94s | a guest 1763.18s - 1769.30s | Basically we found the vulnerable feature to be the project configuration metadata 1770.30s - 1773.70s | So, I hope you can see the attributes there for the configuration 1774.50s - 1778.98s | Basically an admin can change the project to be private public 1779.22s - 1784.82s | You can implement some image vulnerability scanning and and have more power 1786.30s - 1791.94s | So our book claimed that in the official documentation that only the admin project should be able to 1792.42s - 1794.70s | to create and modify this kind of 1795.46s - 1797.70s | configurations which makes sense and 1798.42s - 1800.42s | And 1800.62s - 1805.58s | Actually, we found out that an unauthorized user can do the same operation 1805.94s - 1812.54s | So we tried to play with the roles of the project and we saw that the maintainer for example 1812.54s - 1814.54s | Could not make any change via the UI 1815.10s - 1819.98s | So this is a maintainer login, which is grayed out. You cannot modify it 1821.70s - 1823.30s | But 1823.30s - 1825.22s | Maintainer can actually 1825.22s - 1832.90s | Create delete and modify these attributes via the API. So there is a big discrepancy between the UI and the API 1834.06s - 1835.62s | behavior and 1835.62s - 1837.62s | and this vulnerability actually 1838.02s - 1841.22s | Create an extent in this in the maintainer privileges 1841.74s - 1847.58s | So now a maliciously maintainer could be able to make a private project public 1848.46s - 1855.18s | Deploy unverified images and actually control every configuration that that you can control in in a project 1856.10s - 1857.26s | and


chunk_text:
| to create and modify this kind of 1795.46s - 1797.70s | configurations which makes sense and 1798.42s - 1800.42s | And 1800.62s - 1805.58s | Actually, we found out that an unauthorized user can do the same operation 1805.94s - 1812.54s | So we tried to play with the roles of the project and we saw that the maintainer for example 1812.54s - 1814.54s | Could not make any change via the UI 1815.10s - 1819.98s | So this is a maintainer login, which is grayed out. You cannot modify it 1821.70s - 1823.30s | But 1823.30s - 1825.22s | Maintainer can actually 1825.22s - 1832.90s | Create delete and modify these attributes via the API. So there is a big discrepancy between the UI and the API 1834.06s - 1835.62s | behavior and 1835.62s - 1837.62s | and this vulnerability actually 1838.02s - 1841.22s | Create an extent in this in the maintainer privileges 1841.74s - 1847.58s | So now a maliciously maintainer could be able to make a private project public 1848.46s - 1855.18s | Deploy unverified images and actually control every configuration that that you can control in in a project 1856.10s - 1857.26s | and 1857.26s - 1864.22s | as you can imagine the consequences can be pretty high so we can compromise the wall integrity of a project and 1866.18s - 1870.90s | Actually the security posture so how bow and 1871.94s - 1873.74s | verified 1873.74s - 1875.74s | That's it. Okay, so 1876.58s - 1882.10s | Just would say that how bow identify this vulnerability and you can find it in their security advisory 1882.14s - 1889.70s | We got a CV and you can check out our blog about it when we technically talk about how we found this 1890.30s - 1894.59s | ball 1894.59s - 1900.11s | Thanks. I will just use the last 30 second to share the most important lessons 1900.11s - 1904.15s | We learn from working with AI first AI isn't always the best solution 1904.91s - 1906.91s | Solution problem with existing 1907.19s - 1912.79s | Heuristic solution should not be solved by AI like sorting finding trees or solving equation 1912.79s - 1918.79s | Those problem have existing optimal solution and always use heuristic rather than AI 1918.95s - 1924.51s | Second don't trust always validate blindly trusting the output of AI can be


chunk_text:
1857.26s - 1864.22s | as you can imagine the consequences can be pretty high so we can compromise the wall integrity of a project and 1866.18s - 1870.90s | Actually the security posture so how bow and 1871.94s - 1873.74s | verified 1873.74s - 1875.74s | That's it. Okay, so 1876.58s - 1882.10s | Just would say that how bow identify this vulnerability and you can find it in their security advisory 1882.14s - 1889.70s | We got a CV and you can check out our blog about it when we technically talk about how we found this 1890.30s - 1894.59s | ball 1894.59s - 1900.11s | Thanks. I will just use the last 30 second to share the most important lessons 1900.11s - 1904.15s | We learn from working with AI first AI isn't always the best solution 1904.91s - 1906.91s | Solution problem with existing 1907.19s - 1912.79s | Heuristic solution should not be solved by AI like sorting finding trees or solving equation 1912.79s - 1918.79s | Those problem have existing optimal solution and always use heuristic rather than AI 1918.95s - 1924.51s | Second don't trust always validate blindly trusting the output of AI can be very very dangerous 1924.91s - 1930.63s | So always validate the output of data AI before using them or passing them to the next stage 1931.23s - 1933.67s | Finally makes AI's jobs easier 1934.87s - 1939.39s | Treat AI like a very capable junior worker who can do simple tasks 1940.11s - 1945.07s | Exceptionally well, but starting can start making mistake when the job become more complex 1945.27s - 1953.19s | so when working with AI it is always a good idea good strategy to use divide and conquer to simplify its task and 1954.11s - 1956.11s | That's thank you so much


chunk_text:
very very dangerous 1924.91s - 1930.63s | So always validate the output of data AI before using them or passing them to the next stage 1931.23s - 1933.67s | Finally makes AI's jobs easier 1934.87s - 1939.39s | Treat AI like a very capable junior worker who can do simple tasks 1940.11s - 1945.07s | Exceptionally well, but starting can start making mistake when the job become more complex 1945.27s - 1953.19s | so when working with AI it is always a good idea good strategy to use divide and conquer to simplify its task and 1954.11s - 1956.11s | That's thank you so much


chunk_text:
{ "webpage_url": "https://www.youtube.com/watch?v=trjjcTscg-o", "title": "DEF CON 32 - The Missing Link How we collect and leverage SBOMs - Cassie Crossley", "description": "There is some debate as to how SBOMs can enhance vulnerability management practices, and some believe that collecting SBOMs from internal teams or suppliers is too difficult and time-consuming. Learn how Schneider Electric has collected thousands of our product SBOMs and how we are leveraging the SBOMs as part of our corporate product CERT to quickly analyze and focus our attention when time is of importance. This presentation describes how we modified our policies and processes to collect, generate, and store thousands of SBOMs. You will hear how we have leveraged SBOMs during the Log4j and OpenSSL vulnerability events. Then we will conclude with key learnings, suggestions, and opportunities for improvement.", "channel_url": "https://www.youtube.com/channel/UC6Om9kAkl32dWlDSNlDS9Iw", "duration": 1848, "channel": "DEFCONConference", "uploader": "DEFCONConference", "upload_date": "20241016" } 5.33s - 11.57s | This text was transcribed using whisper model: large-v2 Thank you. All right. So I'm Cassie Crossley. I'm with, I see all the cameras, all my friends. 11.57s - 17.33s | I'm with Schneider Electric. I am the vice president for supply chain security. If you 17.33s - 23.01s | don't know what that is, it's sort of a different kind of role. I have been with the company 14 23.01s - 30.05s | years. We sell a lot of products. I'll show you some of the brands in a few minutes. And I am a 30.05s - 36.29s | product security officer, also from the CISO cybersecurity and product security organization. 36.29s - 40.69s | So I'm going to talk about SBOMs, software bill of materials, but the rest of the time, 40.69s - 45.25s | I'm going to say SBOMs, except for as people come in, I might repeat that once or twice, 45.25s - 50.69s | and how we leverage them. But first, let me tell you a bit about who our company is, 50.69s - 58.37s | just so you know. We are not German. We are headquartered in France, and we are a 36 billion 58.37s - 64.77s | euro company. We are spread all around the world. We do a lot of products. The reason I mentioned 64.77s - 73.65s | the ICS village is, uh, we have a lot of products for industrial controls systems, and we have 73.65s - 79.25s | building operations systems. So you


chunk_text:
is, it's sort of a different kind of role. I have been with the company 14 23.01s - 30.05s | years. We sell a lot of products. I'll show you some of the brands in a few minutes. And I am a 30.05s - 36.29s | product security officer, also from the CISO cybersecurity and product security organization. 36.29s - 40.69s | So I'm going to talk about SBOMs, software bill of materials, but the rest of the time, 40.69s - 45.25s | I'm going to say SBOMs, except for as people come in, I might repeat that once or twice, 45.25s - 50.69s | and how we leverage them. But first, let me tell you a bit about who our company is, 50.69s - 58.37s | just so you know. We are not German. We are headquartered in France, and we are a 36 billion 58.37s - 64.77s | euro company. We are spread all around the world. We do a lot of products. The reason I mentioned 64.77s - 73.65s | the ICS village is, uh, we have a lot of products for industrial controls systems, and we have 73.65s - 79.25s | building operations systems. So you might recognize some of the names. Aviva, uh, that's, uh, you know, 79.25s - 85.89s | you might find that in manufacturing. APC. Who had an APC, you know, unit sitting on their desk 85.89s - 91.81s | in the 80s and 90s. Yep, that's ours. Uh, but we do a lot of products, secure power products 91.81s - 98.53s | for data centers. Uh, also Square D, who's seen Square D panels all over the decades. That's one 98.53s - 104.37s | of our brands. So you, you know, you probably know us from our brands. I may not necessarily 104.37s - 109.33s | know Schneider Electric because we get asked, you know, do we do, uh, elevators, escalators, so on 109.33s - 115.09s | and so forth. And we're not a trucking company either. So that's who we are. Now, what I'm going 115.09s - 121.49s | to be talking about today is, uh, part of that supply chain. So I work with a just wonderful 121.49s - 131.01s | organization, 140,000 employees. We have over 54,000 suppliers. Many of them, uh, let's say 131.01s - 137.57s | over 20,000 of them are part of what we call


chunk_text:
might recognize some of the names. Aviva, uh, that's, uh, you know, 79.25s - 85.89s | you might find that in manufacturing. APC. Who had an APC, you know, unit sitting on their desk 85.89s - 91.81s | in the 80s and 90s. Yep, that's ours. Uh, but we do a lot of products, secure power products 91.81s - 98.53s | for data centers. Uh, also Square D, who's seen Square D panels all over the decades. That's one 98.53s - 104.37s | of our brands. So you, you know, you probably know us from our brands. I may not necessarily 104.37s - 109.33s | know Schneider Electric because we get asked, you know, do we do, uh, elevators, escalators, so on 109.33s - 115.09s | and so forth. And we're not a trucking company either. So that's who we are. Now, what I'm going 115.09s - 121.49s | to be talking about today is, uh, part of that supply chain. So I work with a just wonderful 121.49s - 131.01s | organization, 140,000 employees. We have over 54,000 suppliers. Many of them, uh, let's say 131.01s - 137.57s | over 20,000 of them are part of what we call direct. That means that they work with our products. 137.57s - 144.45s | So if you look at our product portfolio, we have hundreds of thousands of SKUs. Not all of them 144.45s - 150.05s | are intelligent SKUs. They could be cables. They could be any kind of non-intelligent device. 150.05s - 156.53s | Intelligent devices, over 10,000. Uh, so those are ones that may have, uh, firmware. We also have 156.53s - 162.77s | software, cloud products, mobile apps, things like that. Just all the range, that's what we have. So 162.77s - 168.69s | I'm going to be talking about those digital products today. With those suppliers, about 2,000 168.69s - 175.97s | or more of them actually provide us, could be software libraries, it can be, uh, manufacturing 175.97s - 182.05s | equipment systems and platforms where they're flashing firmware. We can work with suppliers 182.05s - 186.29s | who are doing our development. So there's all these different kinds of suppliers that we're 186.29s - 192.37s | working with. And I wanted to just explain that from a footprint standpoint, because from a 192.37s - 198.77s | software and firmware point of view, it's not


chunk_text:
direct. That means that they work with our products. 137.57s - 144.45s | So if you look at our product portfolio, we have hundreds of thousands of SKUs. Not all of them 144.45s - 150.05s | are intelligent SKUs. They could be cables. They could be any kind of non-intelligent device. 150.05s - 156.53s | Intelligent devices, over 10,000. Uh, so those are ones that may have, uh, firmware. We also have 156.53s - 162.77s | software, cloud products, mobile apps, things like that. Just all the range, that's what we have. So 162.77s - 168.69s | I'm going to be talking about those digital products today. With those suppliers, about 2,000 168.69s - 175.97s | or more of them actually provide us, could be software libraries, it can be, uh, manufacturing 175.97s - 182.05s | equipment systems and platforms where they're flashing firmware. We can work with suppliers 182.05s - 186.29s | who are doing our development. So there's all these different kinds of suppliers that we're 186.29s - 192.37s | working with. And I wanted to just explain that from a footprint standpoint, because from a 192.37s - 198.77s | software and firmware point of view, it's not just Schneider Electric employees that are all part 198.77s - 205.97s | of this supplier ecosystem. And so I know this is an eye chart. I just wanted to show it so you can, 205.97s - 213.17s | uh, if you want, there's other, uh, presentations I've done and I can provide this. This is a PDF 213.17s - 218.69s | that I've got. And this is our supply chain security platform. I just bring it up because I 218.69s - 224.61s | wanted to represent, uh, we're going to be talking about the software bill materials. That's the 224.61s - 232.05s | operational group that I run. But I'm also, you know, these initiatives are all part of the supply 232.05s - 237.57s | chain security at our company. Thirteen initiatives. And so I know you can't read it, so I'm going to 237.57s - 243.41s | go through it real quickly so you understand traversally. There's supply chain digital policies, 243.41s - 248.45s | customer authority and requirements, third party cyber security. It's a lot of what I'm working 248.45s - 254.45s | on. Vulnerability management, which is part of this talk, this how we use SBOMs as


chunk_text:
just Schneider Electric employees that are all part 198.77s - 205.97s | of this supplier ecosystem. And so I know this is an eye chart. I just wanted to show it so you can, 205.97s - 213.17s | uh, if you want, there's other, uh, presentations I've done and I can provide this. This is a PDF 213.17s - 218.69s | that I've got. And this is our supply chain security platform. I just bring it up because I 218.69s - 224.61s | wanted to represent, uh, we're going to be talking about the software bill materials. That's the 224.61s - 232.05s | operational group that I run. But I'm also, you know, these initiatives are all part of the supply 232.05s - 237.57s | chain security at our company. Thirteen initiatives. And so I know you can't read it, so I'm going to 237.57s - 243.41s | go through it real quickly so you understand traversally. There's supply chain digital policies, 243.41s - 248.45s | customer authority and requirements, third party cyber security. It's a lot of what I'm working 248.45s - 254.45s | on. Vulnerability management, which is part of this talk, this how we use SBOMs as part of that. 254.45s - 259.81s | We've got cyber defense and incident response. And then you've got on the left, you've got secure 259.81s - 266.37s | development and design. And so you've got product and system security. As a product security 266.37s - 272.29s | officer, that's, you know, a lot of what the world I was when I was owning a lot, most, you know, a 272.29s - 278.45s | lot of that team from the governance. Then you've got SBOMs and then you've got, uh, the R&D 278.45s - 285.81s | activities for cyber security. If you ever want to see this talk described, uh, if you go onto 285.81s - 292.61s | YouTube, S4 for this past year, which is an industrial control conference, you should be able to 292.61s - 299.33s | search for S4 and my name and find the YouTube video or just hit me up on LinkedIn. And then 299.33s - 304.37s | there's technology validation. Think of pen testing, working with third party companies and 304.37s - 309.41s | organizations, but also, uh, different organizations like the Department of Energy to test our 309.41s - 315.49s | products and then


chunk_text:
part of that. 254.45s - 259.81s | We've got cyber defense and incident response. And then you've got on the left, you've got secure 259.81s - 266.37s | development and design. And so you've got product and system security. As a product security 266.37s - 272.29s | officer, that's, you know, a lot of what the world I was when I was owning a lot, most, you know, a 272.29s - 278.45s | lot of that team from the governance. Then you've got SBOMs and then you've got, uh, the R&D 278.45s - 285.81s | activities for cyber security. If you ever want to see this talk described, uh, if you go onto 285.81s - 292.61s | YouTube, S4 for this past year, which is an industrial control conference, you should be able to 292.61s - 299.33s | search for S4 and my name and find the YouTube video or just hit me up on LinkedIn. And then 299.33s - 304.37s | there's technology validation. Think of pen testing, working with third party companies and 304.37s - 309.41s | organizations, but also, uh, different organizations like the Department of Energy to test our 309.41s - 315.49s | products and then source code governance. Big, big thing that everybody should be doing. If not, 315.49s - 321.17s | we need to, you know, uh, ramp that up for everybody else. Uh, the middle one is industrial 321.17s - 327.57s | control, uh, industrial activities. So how we manage our floors, um, and our production lines 327.57s - 332.85s | and then project supply chain security, which is if we're doing the projects to actually deploy 332.85s - 340.45s | it. And lastly, uh, customer facing activities. So I wanted to just give you a view. I am going to 340.45s - 346.61s | concentrate today on software bill materials and the initiative. And I'm gonna go through just a 346.61s - 352.37s | sort of primer. If you don't know what a software bill materials is, if you have not had to create 352.45s - 359.17s | one or ingest one, that's, or, you know, there's different terms for that word too, is I've created 359.17s - 367.01s | a picture, uh, based off of, uh, what we would explain it to our internal organization is that 367.01s - 372.37s | this software bill material standard kind of tree structure, think


chunk_text:
source code governance. Big, big thing that everybody should be doing. If not, 315.49s - 321.17s | we need to, you know, uh, ramp that up for everybody else. Uh, the middle one is industrial 321.17s - 327.57s | control, uh, industrial activities. So how we manage our floors, um, and our production lines 327.57s - 332.85s | and then project supply chain security, which is if we're doing the projects to actually deploy 332.85s - 340.45s | it. And lastly, uh, customer facing activities. So I wanted to just give you a view. I am going to 340.45s - 346.61s | concentrate today on software bill materials and the initiative. And I'm gonna go through just a 346.61s - 352.37s | sort of primer. If you don't know what a software bill materials is, if you have not had to create 352.45s - 359.17s | one or ingest one, that's, or, you know, there's different terms for that word too, is I've created 359.17s - 367.01s | a picture, uh, based off of, uh, what we would explain it to our internal organization is that 367.01s - 372.37s | this software bill material standard kind of tree structure, think about it that way, is you have a 372.37s - 378.85s | product over here on the right. It can have first level dependencies, uh, which could be commercial 378.85s - 383.57s | libraries. It could be an embedded operating system in the case of firmware. It could be 383.57s - 388.61s | proprietary software. So that's code that our teams have written. It can have open source 388.61s - 392.85s | libraries. People sometimes think when they're scanning, oh, just, you know, just the open 392.85s - 398.05s | source libraries and S bombs. Not, not that. I mean, you at least would have, if you've written 398.05s - 406.05s | any code, you've got your own proprietary library or something listed. And then those products 406.05s - 413.17s | and, and different systems and areas can all have what we call transitive dependencies. So 413.17s - 418.37s | inside, for example, open source libraries can call other libraries, right? Um, and those are 418.37s - 423.89s | all transitive dependencies. So, and you may not know them, but what's being called, especially if 423.89s - 431.33s | it's a dot net DLL, right? So I don't have the


chunk_text:
about it that way, is you have a 372.37s - 378.85s | product over here on the right. It can have first level dependencies, uh, which could be commercial 378.85s - 383.57s | libraries. It could be an embedded operating system in the case of firmware. It could be 383.57s - 388.61s | proprietary software. So that's code that our teams have written. It can have open source 388.61s - 392.85s | libraries. People sometimes think when they're scanning, oh, just, you know, just the open 392.85s - 398.05s | source libraries and S bombs. Not, not that. I mean, you at least would have, if you've written 398.05s - 406.05s | any code, you've got your own proprietary library or something listed. And then those products 406.05s - 413.17s | and, and different systems and areas can all have what we call transitive dependencies. So 413.17s - 418.37s | inside, for example, open source libraries can call other libraries, right? Um, and those are 418.37s - 423.89s | all transitive dependencies. So, and you may not know them, but what's being called, especially if 423.89s - 431.33s | it's a dot net DLL, right? So I don't have the S bomb yet for the dot net DLL. If anybody does 431.33s - 437.81s | have it, please send it to me. Um, I would, I would, uh, really like to see a lot of these 437.81s - 445.25s | libraries, especially binaries that, uh, are frequently used, um, for us to be able to have 445.25s - 452.05s | access to those and it'll be coming. So what I wanted to talk about just for a second next is 452.05s - 458.45s | just the journey because it was a plain and important part with where we are and there's 458.53s - 466.61s | always time to catch the train. And what we did is all of a sudden back in the, um, 2019 I 466.61s - 472.37s | received a contract from a customer, a utility saying, give us your S bomb. And I'm like, well, 472.37s - 478.45s | what's that? And, uh, so I found out about the, our teams knew our vulnerability management 478.45s - 483.97s | teams knew and they said, hey, there's this NTIA initiative. So I immediately got on board 483.97s - 490.53s | with, uh, one of the working groups and


chunk_text:
S bomb yet for the dot net DLL. If anybody does 431.33s - 437.81s | have it, please send it to me. Um, I would, I would, uh, really like to see a lot of these 437.81s - 445.25s | libraries, especially binaries that, uh, are frequently used, um, for us to be able to have 445.25s - 452.05s | access to those and it'll be coming. So what I wanted to talk about just for a second next is 452.05s - 458.45s | just the journey because it was a plain and important part with where we are and there's 458.53s - 466.61s | always time to catch the train. And what we did is all of a sudden back in the, um, 2019 I 466.61s - 472.37s | received a contract from a customer, a utility saying, give us your S bomb. And I'm like, well, 472.37s - 478.45s | what's that? And, uh, so I found out about the, our teams knew our vulnerability management 478.45s - 483.97s | teams knew and they said, hey, there's this NTIA initiative. So I immediately got on board 483.97s - 490.53s | with, uh, one of the working groups and then I, you know, continued working with them and we 490.53s - 497.17s | created an adoption and awareness group. So since 2020 I've been helping deliver the message of S 497.17s - 504.53s | bombs. At that same time, uh, we had COVID, but the executive order in May of 2020 came out. 504.53s - 509.97s | Well, I helped work on the, uh, the, let's just say the definition for S bomb by that time, 509.97s - 516.45s | cause I'd been working with them. So at 2020, we have that executive order called 14 zero to 516.45s - 522.45s | eight in the U S that mentions S bombs and gives a definition and told NTIA to give us 522.45s - 528.61s | minimum elements. Like what should you put into it to distribute it? And that played a key role 528.61s - 534.93s | in really moving things forward as an industry to like have something that specifically mentions it 534.93s - 542.29s | and carried forward. So this is a slide where I talk about sort of the drivers for it. Um, it was 542.29s - 548.69s | really important because not everybody sells to


chunk_text:
then I, you know, continued working with them and we 490.53s - 497.17s | created an adoption and awareness group. So since 2020 I've been helping deliver the message of S 497.17s - 504.53s | bombs. At that same time, uh, we had COVID, but the executive order in May of 2020 came out. 504.53s - 509.97s | Well, I helped work on the, uh, the, let's just say the definition for S bomb by that time, 509.97s - 516.45s | cause I'd been working with them. So at 2020, we have that executive order called 14 zero to 516.45s - 522.45s | eight in the U S that mentions S bombs and gives a definition and told NTIA to give us 522.45s - 528.61s | minimum elements. Like what should you put into it to distribute it? And that played a key role 528.61s - 534.93s | in really moving things forward as an industry to like have something that specifically mentions it 534.93s - 542.29s | and carried forward. So this is a slide where I talk about sort of the drivers for it. Um, it was 542.29s - 548.69s | really important because not everybody sells to utilities, right? So until then, really only those 548.69s - 555.09s | of us that are in that space were being asked for S bombs and it was in the contract, but not 555.09s - 563.09s | actually requested. So when the executive order came out, I required our teams to start, uh, 563.09s - 567.49s | the concept of S bombs. And when I say teams, I'm talking about the product development teams. 567.49s - 573.49s | So I, in the governance group, I owned the secure development lifecycle policy. So what I did is I 573.49s - 581.81s | modified that policy saying every team must provide an S bomb or provide us the, you know, the binary 581.81s - 588.21s | or whatever, so we could scan it to create the S bomb. So that was in 2020, June of 2020. I changed 588.29s - 594.45s | the rule to say starting in January of 21, I'm going to start requiring S bombs from every single 594.45s - 600.45s | team. And so, and so, we have, I told you, tens of thousands of projects. We have 500 projects going 603.01s - 610.61s | on at a


chunk_text:
utilities, right? So until then, really only those 548.69s - 555.09s | of us that are in that space were being asked for S bombs and it was in the contract, but not 555.09s - 563.09s | actually requested. So when the executive order came out, I required our teams to start, uh, 563.09s - 567.49s | the concept of S bombs. And when I say teams, I'm talking about the product development teams. 567.49s - 573.49s | So I, in the governance group, I owned the secure development lifecycle policy. So what I did is I 573.49s - 581.81s | modified that policy saying every team must provide an S bomb or provide us the, you know, the binary 581.81s - 588.21s | or whatever, so we could scan it to create the S bomb. So that was in 2020, June of 2020. I changed 588.29s - 594.45s | the rule to say starting in January of 21, I'm going to start requiring S bombs from every single 594.45s - 600.45s | team. And so, and so, we have, I told you, tens of thousands of projects. We have 500 projects going 603.01s - 610.61s | on at a time. So there's a lot of releases going on, um, and the technologies can be all over the 610.61s - 616.61s | place, right? So as I mentioned, but there's also legacy platforms and legacy software that were 618.45s - 624.85s | usually updating once a year. Some products are more frequent, but in a firmware world, uh, a lot 624.85s - 629.73s | of the products they have to go through regression testing, interoperability, safety testing. So 629.73s - 635.73s | they go through this cycle of product development and with all of these projects, um, they do not 635.73s - 643.73s | have, let's just say, the majority of them do not have modern CICD pipelines. All right? If they 643.81s - 651.17s | do, it's like super cool and super easy, but most of them don't. So with the re these requirements, 651.17s - 658.37s | we had to figure out how are we going to build this process to start getting S bombs so that in 658.37s - 666.37s | 21, the next, that next year we can start delivering them to our customers. And so as part of 666.37s - 672.37s | that, I worked


chunk_text:
time. So there's a lot of releases going on, um, and the technologies can be all over the 610.61s - 616.61s | place, right? So as I mentioned, but there's also legacy platforms and legacy software that were 618.45s - 624.85s | usually updating once a year. Some products are more frequent, but in a firmware world, uh, a lot 624.85s - 629.73s | of the products they have to go through regression testing, interoperability, safety testing. So 629.73s - 635.73s | they go through this cycle of product development and with all of these projects, um, they do not 635.73s - 643.73s | have, let's just say, the majority of them do not have modern CICD pipelines. All right? If they 643.81s - 651.17s | do, it's like super cool and super easy, but most of them don't. So with the re these requirements, 651.17s - 658.37s | we had to figure out how are we going to build this process to start getting S bombs so that in 658.37s - 666.37s | 21, the next, that next year we can start delivering them to our customers. And so as part of 666.37s - 672.37s | that, I worked with a consultant company and really said, okay, I'm, I've got this idea for a 672.37s - 676.85s | process. I need to be able to collect it. I need to be able to get all my proprietary 676.85s - 683.25s | information, all of the information from maybe third party code and then also open source 683.25s - 690.29s | code. And of course, if we have the actual code generating it, uh, you know, on site, it can 690.29s - 696.13s | generate the S bomb. I'll talk more about the binary analysis generation in a minute, but what 696.13s - 703.25s | we do is we were bringing them all together. At first we were collecting the binaries and 703.25s - 709.01s | creating the scans and storing them in box folders and then later SharePoint folders. But I knew 709.01s - 716.05s | that wasn't going to be sufficient because when providing all of those, so an S bomb is per 716.05s - 721.49s | version, product released version. So like some of those products are only going to have one a 721.49s - 727.17s | year. But some of them, uh, in that if they


chunk_text:
with a consultant company and really said, okay, I'm, I've got this idea for a 672.37s - 676.85s | process. I need to be able to collect it. I need to be able to get all my proprietary 676.85s - 683.25s | information, all of the information from maybe third party code and then also open source 683.25s - 690.29s | code. And of course, if we have the actual code generating it, uh, you know, on site, it can 690.29s - 696.13s | generate the S bomb. I'll talk more about the binary analysis generation in a minute, but what 696.13s - 703.25s | we do is we were bringing them all together. At first we were collecting the binaries and 703.25s - 709.01s | creating the scans and storing them in box folders and then later SharePoint folders. But I knew 709.01s - 716.05s | that wasn't going to be sufficient because when providing all of those, so an S bomb is per 716.05s - 721.49s | version, product released version. So like some of those products are only going to have one a 721.49s - 727.17s | year. But some of them, uh, in that if they release quarterly or more frequently, especially 727.17s - 731.93s | some software applications, they were going to have to have, you know, more frequent S bombs. So 731.93s - 738.77s | we're talking, my expectation is, you know, with versions, thousands and thousands of 738.77s - 744.45s | thousands of S bombs. So a lot of companies may only have to think about, oh, for internal, 744.45s - 751.25s | you know, for releasing to customers, I may only have to do 10 or 20. Well, I, I knew that 751.25s - 755.81s | wasn't going to be my case. So I needed to be able to think about how am I also going to 755.81s - 762.45s | distribute it and use it both internally, but also to customers. And so, uh, we went through 762.45s - 767.89s | at the time where I was working on this process, started going through, uh, an understanding and 767.89s - 774.41s | a process to say, okay, I need a tool that can do all of, you know, what I really want to be 774.41s - 779.89s | able to do, which is be able to provide it to customers in


chunk_text:
release quarterly or more frequently, especially 727.17s - 731.93s | some software applications, they were going to have to have, you know, more frequent S bombs. So 731.93s - 738.77s | we're talking, my expectation is, you know, with versions, thousands and thousands of 738.77s - 744.45s | thousands of S bombs. So a lot of companies may only have to think about, oh, for internal, 744.45s - 751.25s | you know, for releasing to customers, I may only have to do 10 or 20. Well, I, I knew that 751.25s - 755.81s | wasn't going to be my case. So I needed to be able to think about how am I also going to 755.81s - 762.45s | distribute it and use it both internally, but also to customers. And so, uh, we went through 762.45s - 767.89s | at the time where I was working on this process, started going through, uh, an understanding and 767.89s - 774.41s | a process to say, okay, I need a tool that can do all of, you know, what I really want to be 774.41s - 779.89s | able to do, which is be able to provide it to customers in a secure manner. Because as I 779.89s - 785.53s | mentioned at the beginning, we are critical infrastructure and, uh, I was one of the ones 785.53s - 789.89s | that were loudly saying, I'm not going to give an S bomb, I'm not putting it on the web unless 789.89s - 795.85s | it was, you know, probably a, a home product. Uh, we do sell some of those in Europe, but for 795.85s - 801.25s | critical infrastructure, uh, it would require an NDA. And another thing about Schneider 801.25s - 806.69s | Electric is we don't do a lot of our direct sales. We go through engineering firms, there's 806.73s - 813.25s | integrators, there's all sorts of ways that products can be sold. And so I only know 813.25s - 821.09s | about 36 to 40 percent of my direct customers. So we needed to be able to say, okay, you have 821.09s - 826.61s | this product, um, we want to sign an NDA cause I don't have a direct contract with them. And 826.61s - 831.65s | this is, I think for those of us in the critical infrastructure, pretty much that's how the


chunk_text:
a secure manner. Because as I 779.89s - 785.53s | mentioned at the beginning, we are critical infrastructure and, uh, I was one of the ones 785.53s - 789.89s | that were loudly saying, I'm not going to give an S bomb, I'm not putting it on the web unless 789.89s - 795.85s | it was, you know, probably a, a home product. Uh, we do sell some of those in Europe, but for 795.85s - 801.25s | critical infrastructure, uh, it would require an NDA. And another thing about Schneider 801.25s - 806.69s | Electric is we don't do a lot of our direct sales. We go through engineering firms, there's 806.73s - 813.25s | integrators, there's all sorts of ways that products can be sold. And so I only know 813.25s - 821.09s | about 36 to 40 percent of my direct customers. So we needed to be able to say, okay, you have 821.09s - 826.61s | this product, um, we want to sign an NDA cause I don't have a direct contract with them. And 826.61s - 831.65s | this is, I think for those of us in the critical infrastructure, pretty much that's how the 831.65s - 837.93s | groups are, are working with. Now I get requests today from, uh, customers that we 837.93s - 843.25s | directly sell to, customers that go through integrators. I get requests from the labs that 843.25s - 847.89s | we're specifically working with, like with Department of Energy, where we've had, um, work 847.89s - 852.73s | with them and all of them, they're able to go into this tool that I'm going to talk about in a 852.73s - 859.77s | moment. But the, um, indirect customers, my, my vision and what I'm working on is that I 859.85s - 865.93s | would provide the SBOMs to asset management companies because if you look at this room and 865.93s - 871.37s | there's sensors and everything, you know, hopefully this convention center has an asset 871.37s - 878.33s | inventory, cross your fingers, um, but likely it doesn't even have the firmware registered in 878.33s - 884.61s | that asset inventory. So the SBOM really can't do the building maintenance staff and the 884.61s - 889.45s | facility management any good if it's an SBOM from the current one, if they're running a 889.45s - 894.85s


chunk_text:
831.65s - 837.93s | groups are, are working with. Now I get requests today from, uh, customers that we 837.93s - 843.25s | directly sell to, customers that go through integrators. I get requests from the labs that 843.25s - 847.89s | we're specifically working with, like with Department of Energy, where we've had, um, work 847.89s - 852.73s | with them and all of them, they're able to go into this tool that I'm going to talk about in a 852.73s - 859.77s | moment. But the, um, indirect customers, my, my vision and what I'm working on is that I 859.85s - 865.93s | would provide the SBOMs to asset management companies because if you look at this room and 865.93s - 871.37s | there's sensors and everything, you know, hopefully this convention center has an asset 871.37s - 878.33s | inventory, cross your fingers, um, but likely it doesn't even have the firmware registered in 878.33s - 884.61s | that asset inventory. So the SBOM really can't do the building maintenance staff and the 884.61s - 889.45s | facility management any good if it's an SBOM from the current one, if they're running a 889.45s - 894.85s | version from like, you know, three years ago. And that's a problem right now in the 894.85s - 899.37s | software bill of materials industry. It's great for procurement for you to be able to ask 899.37s - 904.01s | and say, hey, what's in here? But if you want to use it for some operational and risk 904.01s - 908.97s | analysis, and I'll talk more about that, but if you want to do that, you need to know what 908.97s - 914.41s | version you're running. And so understanding that and working with that indirect customer 914.45s - 920.61s | pipeline is something that I'm focused on. Also CP cert. So if you don't know internal, we've 920.61s - 926.33s | got a product cert, uh, team that works with all of our different organizations and business 926.33s - 931.81s | units and lines of business and they are the ones that help focus on if there's new, uh, 931.81s - 938.81s | celebrity CVEs that come out, but also if we're doing disclosures and such. And they need 938.81s - 942.77s | to be able to search that. And that's really why the vulnerability management team


chunk_text:
| version from like, you know, three years ago. And that's a problem right now in the 894.85s - 899.37s | software bill of materials industry. It's great for procurement for you to be able to ask 899.37s - 904.01s | and say, hey, what's in here? But if you want to use it for some operational and risk 904.01s - 908.97s | analysis, and I'll talk more about that, but if you want to do that, you need to know what 908.97s - 914.41s | version you're running. And so understanding that and working with that indirect customer 914.45s - 920.61s | pipeline is something that I'm focused on. Also CP cert. So if you don't know internal, we've 920.61s - 926.33s | got a product cert, uh, team that works with all of our different organizations and business 926.33s - 931.81s | units and lines of business and they are the ones that help focus on if there's new, uh, 931.81s - 938.81s | celebrity CVEs that come out, but also if we're doing disclosures and such. And they need 938.81s - 942.77s | to be able to search that. And that's really why the vulnerability management team knew 942.81s - 947.29s | about SBOMs because they wanted to be able to leverage it. And so I'm going to be showing 947.29s - 954.29s | you exactly how that happens, uh, in a moment. But the internal dev teams, um, I'm a developer 955.13s - 962.05s | by background really long ago, but I've led a lot of dev teams, you know, up until about 962.05s - 969.05s | ten years ago. I swapped back and forth between development and IT. And I know that I kept 969.93s - 974.93s | lists of, let's just say open source or licensed information in spreadsheets, right? I mean, 978.21s - 983.29s | and then we just put it there. We wouldn't update. We, you know, nobody, it would just stay 983.29s - 988.67s | there. Like, right, I did it for the legal reasons, so somebody checked it out and I'm good 988.67s - 995.13s | and I don't go on. Well, for a lot of us that are working with older technologies, being 995.13s - 1002.29s | able to, uh, really re-examine those libraries didn't start happening until vulnerabilities 1002.29s - 1008.53s | came about. And, you know, obviously


chunk_text:
knew 942.81s - 947.29s | about SBOMs because they wanted to be able to leverage it. And so I'm going to be showing 947.29s - 954.29s | you exactly how that happens, uh, in a moment. But the internal dev teams, um, I'm a developer 955.13s - 962.05s | by background really long ago, but I've led a lot of dev teams, you know, up until about 962.05s - 969.05s | ten years ago. I swapped back and forth between development and IT. And I know that I kept 969.93s - 974.93s | lists of, let's just say open source or licensed information in spreadsheets, right? I mean, 978.21s - 983.29s | and then we just put it there. We wouldn't update. We, you know, nobody, it would just stay 983.29s - 988.67s | there. Like, right, I did it for the legal reasons, so somebody checked it out and I'm good 988.67s - 995.13s | and I don't go on. Well, for a lot of us that are working with older technologies, being 995.13s - 1002.29s | able to, uh, really re-examine those libraries didn't start happening until vulnerabilities 1002.29s - 1008.53s | came about. And, you know, obviously there's some classic vulnerabilities, especially in 1008.53s - 1013.45s | our space. There can be some on proprietary software, but things that you would normally 1013.45s - 1019.41s | see like log4j and other ones that I'm going to talk about. And the development teams needed 1019.41s - 1024.57s | to have that information quickly because every time there was a CVE, somebody would go, 1024.57s - 1028.65s | hey, you know, do you got this? Do you got this? Do you got this? And we had a really 1028.65s - 1035.07s | organized fashion to do it, but it's still time consuming. And that's where we're headed 1035.07s - 1040.77s | with our SBOM vulnerability management program. So I just talked about the vulnerabilities. 1040.77s - 1046.61s | Here's sort of a picture of what happens is we're issuing that request to the teams. So I 1046.61s - 1051.89s | told you, you know, we've got 500 projects going on at a time, but we've got thousands of 1051.93s - 1057.21s | SKUs, right? And when somebody has to identify a vulnerability, they have to look at, well, 1057.21s - 1062.71s | was it in this other version? You


chunk_text:
there's some classic vulnerabilities, especially in 1008.53s - 1013.45s | our space. There can be some on proprietary software, but things that you would normally 1013.45s - 1019.41s | see like log4j and other ones that I'm going to talk about. And the development teams needed 1019.41s - 1024.57s | to have that information quickly because every time there was a CVE, somebody would go, 1024.57s - 1028.65s | hey, you know, do you got this? Do you got this? Do you got this? And we had a really 1028.65s - 1035.07s | organized fashion to do it, but it's still time consuming. And that's where we're headed 1035.07s - 1040.77s | with our SBOM vulnerability management program. So I just talked about the vulnerabilities. 1040.77s - 1046.61s | Here's sort of a picture of what happens is we're issuing that request to the teams. So I 1046.61s - 1051.89s | told you, you know, we've got 500 projects going on at a time, but we've got thousands of 1051.93s - 1057.21s | SKUs, right? And when somebody has to identify a vulnerability, they have to look at, well, 1057.21s - 1062.71s | was it in this other version? You know, did we update it? Because something like OpenSSL 1062.71s - 1067.59s | when it had, uh, when it had some vulnerabilities but only in certain versions, they have to 1067.59s - 1072.43s | know exactly which versions. So that could have been that way in version 7, but not in 1072.43s - 1077.43s | version 9. So it's a lot of, I lost, okay. It's a lot of work for these teams, um, and then 1077.93s - 1084.71s | of course, if it happens at Christmas or, you know, in August or whatever, it's even, you know, 1084.71s - 1092.01s | it takes longer for that analysis. Um, so all of that work for identifying the vulnerabilities, 1092.01s - 1097.01s | we issue these impact assessments. And with the SBOMs, what we're able to do now is look 1099.85s - 1104.85s | more specifically by saying, okay, we're going to look at our SBOMs and actually make sure 1107.51s - 1112.97s | that those teams are responding, first of all. But secondly, we're able to give them an 1112.97s - 1118.35s | indicator saying, hey, this is showing in your SBOM, you know, please, uh, make sure you 1118.35s


chunk_text:
know, did we update it? Because something like OpenSSL 1062.71s - 1067.59s | when it had, uh, when it had some vulnerabilities but only in certain versions, they have to 1067.59s - 1072.43s | know exactly which versions. So that could have been that way in version 7, but not in 1072.43s - 1077.43s | version 9. So it's a lot of, I lost, okay. It's a lot of work for these teams, um, and then 1077.93s - 1084.71s | of course, if it happens at Christmas or, you know, in August or whatever, it's even, you know, 1084.71s - 1092.01s | it takes longer for that analysis. Um, so all of that work for identifying the vulnerabilities, 1092.01s - 1097.01s | we issue these impact assessments. And with the SBOMs, what we're able to do now is look 1099.85s - 1104.85s | more specifically by saying, okay, we're going to look at our SBOMs and actually make sure 1107.51s - 1112.97s | that those teams are responding, first of all. But secondly, we're able to give them an 1112.97s - 1118.35s | indicator saying, hey, this is showing in your SBOM, you know, please, uh, make sure you 1118.35s - 1123.35s | work on it and prioritize it, um, and make sure it happens right away because we like to 1123.35s - 1128.73s | issue those security bulletins, especially for celebrity CVEs, you know, as soon as 1128.73s - 1134.69s | possible. And they get updated. So the log for J1, we issued one within just a couple days 1134.73s - 1139.87s | and we kept appending and updating it as more information from other teams came about, 1139.87s - 1144.63s | we would add that data. Now, one thing that was interesting, uh, for that is I had a 1144.63s - 1149.31s | customer, uh, come to me and they said, you know, you, you changed it. You said you 1149.31s - 1153.21s | weren't affected and then you were. I said, that's because we heard a, you know, a third 1153.21s - 1158.21s | party proprietary library was affected. And so we had to update our, our issue, you know, 1158.81s - 1163.81s | our list to say that product was affected and here's, you know, the mitigation or the patch 1166.61s - 1172.05s | or whatever happened in that case. And that's the importance of SBOMs.


chunk_text:
- 1123.35s | work on it and prioritize it, um, and make sure it happens right away because we like to 1123.35s - 1128.73s | issue those security bulletins, especially for celebrity CVEs, you know, as soon as 1128.73s - 1134.69s | possible. And they get updated. So the log for J1, we issued one within just a couple days 1134.73s - 1139.87s | and we kept appending and updating it as more information from other teams came about, 1139.87s - 1144.63s | we would add that data. Now, one thing that was interesting, uh, for that is I had a 1144.63s - 1149.31s | customer, uh, come to me and they said, you know, you, you changed it. You said you 1149.31s - 1153.21s | weren't affected and then you were. I said, that's because we heard a, you know, a third 1153.21s - 1158.21s | party proprietary library was affected. And so we had to update our, our issue, you know, 1158.81s - 1163.81s | our list to say that product was affected and here's, you know, the mitigation or the patch 1166.61s - 1172.05s | or whatever happened in that case. And that's the importance of SBOMs. If I had had at 1172.05s - 1176.03s | least some indication, I would have known to go to that third party right then or that 1176.03s - 1182.37s | product team would. But without having those third party SBOMs and if they have the 1182.37s - 1187.71s | source code, they can do, you know, scanning and evaluation. But if it's a binary or, you 1187.71s - 1194.11s | know, we don't have all the information, then we're, you know, having to go and query and 1194.11s - 1200.05s | query and query. So it's really important for us and for you all to have that additional 1200.05s - 1205.49s | information, that transparency. Now we can't necessarily say it's affected and that's the 1205.49s - 1210.49s | most important part of all of this is an SBOM cannot say I am affected, yes, no. So I'm 1210.85s - 1215.85s | going to say that one more time. An SBOM will not tell you by itself if I am affected, yes 1220.29s - 1225.69s | or no. There's something that we're working on as an industry called VEX, the vulnerability 1225.69s - 1232.71s | exploitability exchange. And that would


chunk_text:
If I had had at 1172.05s - 1176.03s | least some indication, I would have known to go to that third party right then or that 1176.03s - 1182.37s | product team would. But without having those third party SBOMs and if they have the 1182.37s - 1187.71s | source code, they can do, you know, scanning and evaluation. But if it's a binary or, you 1187.71s - 1194.11s | know, we don't have all the information, then we're, you know, having to go and query and 1194.11s - 1200.05s | query and query. So it's really important for us and for you all to have that additional 1200.05s - 1205.49s | information, that transparency. Now we can't necessarily say it's affected and that's the 1205.49s - 1210.49s | most important part of all of this is an SBOM cannot say I am affected, yes, no. So I'm 1210.85s - 1215.85s | going to say that one more time. An SBOM will not tell you by itself if I am affected, yes 1220.29s - 1225.69s | or no. There's something that we're working on as an industry called VEX, the vulnerability 1225.69s - 1232.71s | exploitability exchange. And that would say for those two CVEs for log4j, it could be an 1232.71s - 1238.05s | in progress, it could say not affected, it could say affected and here's a patch or, you 1238.07s - 1242.65s | know, not in this, there's different statuses. That will give you the answer. And I 1242.65s - 1247.89s | mentioned, I provide a picture in my book to say if you are looking, and I'll talk about my 1247.89s - 1254.29s | book in a sec, but uh if you are looking to see if you're affected, you will not be able to 1254.29s - 1259.47s | make the final decision just like that CP cert team that we have can't decide for that 1259.47s - 1266.37s | product team. Somebody's actually on that team validating it. And in the future, you know, 1266.39s - 1271.43s | as much as you can say that version, but anybody who's a developer here knows you can 1271.43s - 1277.71s | backport patches and so you could have already had a security patch for that in the 1277.71s - 1283.25s | product. You could have not used certain pieces of that library. You could have done 1283.25s


chunk_text:
say for those two CVEs for log4j, it could be an 1232.71s - 1238.05s | in progress, it could say not affected, it could say affected and here's a patch or, you 1238.07s - 1242.65s | know, not in this, there's different statuses. That will give you the answer. And I 1242.65s - 1247.89s | mentioned, I provide a picture in my book to say if you are looking, and I'll talk about my 1247.89s - 1254.29s | book in a sec, but uh if you are looking to see if you're affected, you will not be able to 1254.29s - 1259.47s | make the final decision just like that CP cert team that we have can't decide for that 1259.47s - 1266.37s | product team. Somebody's actually on that team validating it. And in the future, you know, 1266.39s - 1271.43s | as much as you can say that version, but anybody who's a developer here knows you can 1271.43s - 1277.71s | backport patches and so you could have already had a security patch for that in the 1277.71s - 1283.25s | product. You could have not used certain pieces of that library. You could have done 1283.25s - 1289.39s | different things. And that's everything that has the actual team has to answer. And so 1289.41s - 1296.39s | what this has done for us is it's sped up our time. So we have gone through, you know, 1296.39s - 1302.45s | just a very long multiple year process where we've been working and collecting these S 1302.45s - 1310.47s | bombs, but also, you know, we want to be able to speed up our teams and reduce noise, you 1310.47s - 1315.67s | know, bring it really to a point. So we had two main metrics that we've been following is a 1315.69s - 1320.69s | reduction of staff hours and I we've definitely have improved that by over half. And, uh, 1322.97s - 1327.67s | that's a big deal when you're talking about the size that we are doing. So not only is it 1327.67s - 1333.11s | the reduction of staff hours from the vulnerability team, but let's say you have a product 1333.11s - 1338.11s | that's, um, multi years old and, uh, you know, let's let's say, uh, Sarah takes over the 1338.59s - 1343.61s | maintenance of that, she doesn't have


chunk_text:
- 1289.39s | different things. And that's everything that has the actual team has to answer. And so 1289.41s - 1296.39s | what this has done for us is it's sped up our time. So we have gone through, you know, 1296.39s - 1302.45s | just a very long multiple year process where we've been working and collecting these S 1302.45s - 1310.47s | bombs, but also, you know, we want to be able to speed up our teams and reduce noise, you 1310.47s - 1315.67s | know, bring it really to a point. So we had two main metrics that we've been following is a 1315.69s - 1320.69s | reduction of staff hours and I we've definitely have improved that by over half. And, uh, 1322.97s - 1327.67s | that's a big deal when you're talking about the size that we are doing. So not only is it 1327.67s - 1333.11s | the reduction of staff hours from the vulnerability team, but let's say you have a product 1333.11s - 1338.11s | that's, um, multi years old and, uh, you know, let's let's say, uh, Sarah takes over the 1338.59s - 1343.61s | maintenance of that, she doesn't have the ten years history of that code all the time. But if 1347.21s - 1353.41s | she knew that the from the S bomb that it has that library, she'll be able to go and look and 1353.41s - 1359.79s | be able to dig into it. So instead of wading through, you know, a lot of just different 1359.79s - 1364.79s | kinds of things, we're able to help pinpoint and be able to give some direction so we can 1364.91s - 1369.91s | get started faster. But also the reduction in time to identify that vulnerability 1373.09s - 1378.09s | component is when open SSL, even in November of 20, uh, 22, when they released that, they 1381.89s - 1386.73s | were going to release two and they did, um, to say it's going to be in these this version. 1386.73s - 1391.31s | That's all we got was a heads up, which was awesome. And what that did is we were able to 1391.33s - 1397.77s | generate every single library or every single product that we knew that we had the records 1397.77s - 1402.11s | for the S bomb to be able to tell those


chunk_text:
the ten years history of that code all the time. But if 1347.21s - 1353.41s | she knew that the from the S bomb that it has that library, she'll be able to go and look and 1353.41s - 1359.79s | be able to dig into it. So instead of wading through, you know, a lot of just different 1359.79s - 1364.79s | kinds of things, we're able to help pinpoint and be able to give some direction so we can 1364.91s - 1369.91s | get started faster. But also the reduction in time to identify that vulnerability 1373.09s - 1378.09s | component is when open SSL, even in November of 20, uh, 22, when they released that, they 1381.89s - 1386.73s | were going to release two and they did, um, to say it's going to be in these this version. 1386.73s - 1391.31s | That's all we got was a heads up, which was awesome. And what that did is we were able to 1391.33s - 1397.77s | generate every single library or every single product that we knew that we had the records 1397.77s - 1402.11s | for the S bomb to be able to tell those teams, hey, get ready, you know, set aside 1402.11s - 1406.65s | sometimes for, uh, a couple days. So if you are an open source maintainer, if you can do 1406.65s - 1411.65s | that for me in the future, I would love you because I absolutely can use that knowledge 1413.49s - 1419.15s | just like for open SSL to say, okay, everybody heads up, you know, this is going to come 1419.17s - 1424.17s | down the pike. And so for what we're doing and these are screen cut captures from our S 1426.81s - 1432.11s | bomb management solution that's from Cybeats and I was the first customer and I gave them 1432.11s - 1437.31s | my design of what I wanted and they've added so much more to it because I need to be able 1437.31s - 1443.13s | to search and do this quickly. Um, and I had a lot of additional requirements about how 1443.13s - 1447.29s | am I going to search and I want to be able to see, you know, all the known exploitable 1447.33s - 1451.67s | vulnerabilities really quick, you know, things that my scanner tools, because one thing


chunk_text:
teams, hey, get ready, you know, set aside 1402.11s - 1406.65s | sometimes for, uh, a couple days. So if you are an open source maintainer, if you can do 1406.65s - 1411.65s | that for me in the future, I would love you because I absolutely can use that knowledge 1413.49s - 1419.15s | just like for open SSL to say, okay, everybody heads up, you know, this is going to come 1419.17s - 1424.17s | down the pike. And so for what we're doing and these are screen cut captures from our S 1426.81s - 1432.11s | bomb management solution that's from Cybeats and I was the first customer and I gave them 1432.11s - 1437.31s | my design of what I wanted and they've added so much more to it because I need to be able 1437.31s - 1443.13s | to search and do this quickly. Um, and I had a lot of additional requirements about how 1443.13s - 1447.29s | am I going to search and I want to be able to see, you know, all the known exploitable 1447.33s - 1451.67s | vulnerabilities really quick, you know, things that my scanner tools, because one thing I 1451.67s - 1457.81s | didn't mention and I mentioned it on the summary, is that with the binary analysis and I 1457.81s - 1462.81s | think those that are in this world, it, it's only, it's only as good as it's good and most 1464.41s - 1469.41s | of the time it can identify open source and it'll get close enough to the version of what 1469.95s - 1474.95s | it is, but if you have two versions of something and it's 1.21 and 1.22, sometimes it can't 1478.89s - 1484.13s | tell the difference, right? So, you know, it may say it's one version or another and the 1484.13s - 1489.27s | teams have to validate that. So, but also it doesn't pick up proprietary open source, I 1489.27s - 1494.91s | mean proprietary code or libraries. So it doesn't identify my libraries and we're working 1494.91s - 1501.35s | closely with binary scanners that we use to do that. Um, and then it doesn't always 1501.35s - 1505.85s | identify and most of the time doesn't identify commercial components. It can identify 1505.85s - 1511.33s | Java libraries and dot net libraries. But if I purchased a


chunk_text:
I 1451.67s - 1457.81s | didn't mention and I mentioned it on the summary, is that with the binary analysis and I 1457.81s - 1462.81s | think those that are in this world, it, it's only, it's only as good as it's good and most 1464.41s - 1469.41s | of the time it can identify open source and it'll get close enough to the version of what 1469.95s - 1474.95s | it is, but if you have two versions of something and it's 1.21 and 1.22, sometimes it can't 1478.89s - 1484.13s | tell the difference, right? So, you know, it may say it's one version or another and the 1484.13s - 1489.27s | teams have to validate that. So, but also it doesn't pick up proprietary open source, I 1489.27s - 1494.91s | mean proprietary code or libraries. So it doesn't identify my libraries and we're working 1494.91s - 1501.35s | closely with binary scanners that we use to do that. Um, and then it doesn't always 1501.35s - 1505.85s | identify and most of the time doesn't identify commercial components. It can identify 1505.85s - 1511.33s | Java libraries and dot net libraries. But if I purchased a commercial library, whether it 1511.33s - 1517.73s | be the source code or binaries or something or libraries, you know, it can't necessarily 1517.73s - 1522.37s | have the signatures for those because it's not a common thing, uh, and it will not be able 1522.37s - 1527.37s | to identify. I did a binary scan of one of our, um, more older, uh, secure power APS, uh, 1530.71s - 1536.05s | APC products and it had ten commercial libraries that we had purchased like over a decade 1536.05s - 1540.35s | ago. It didn't find any of them because it didn't have enough of that information and 1540.35s - 1545.39s | those signatures. So that's why moving it into the pipelines where you're building those 1545.39s - 1551.37s | S-bombs is important. And so here, you know, what we've done is, you know, we can't 1551.37s - 1556.77s | augment, so in those scanning tools, you know, they're just catching up, they've got the 1556.77s - 1561.21s | S-bombs, but for me to be able to add additional information, especially proprietary 1561.21s - 1566.91s | components and the other, uh, commercial components, I'm not able to make those kind


chunk_text:
commercial library, whether it 1511.33s - 1517.73s | be the source code or binaries or something or libraries, you know, it can't necessarily 1517.73s - 1522.37s | have the signatures for those because it's not a common thing, uh, and it will not be able 1522.37s - 1527.37s | to identify. I did a binary scan of one of our, um, more older, uh, secure power APS, uh, 1530.71s - 1536.05s | APC products and it had ten commercial libraries that we had purchased like over a decade 1536.05s - 1540.35s | ago. It didn't find any of them because it didn't have enough of that information and 1540.35s - 1545.39s | those signatures. So that's why moving it into the pipelines where you're building those 1545.39s - 1551.37s | S-bombs is important. And so here, you know, what we've done is, you know, we can't 1551.37s - 1556.77s | augment, so in those scanning tools, you know, they're just catching up, they've got the 1556.77s - 1561.21s | S-bombs, but for me to be able to add additional information, especially proprietary 1561.21s - 1566.91s | components and the other, uh, commercial components, I'm not able to make those kind of 1566.91s - 1573.09s | adjustments. So in this tool, we're able to correct the version numbers, we're able to add 1573.09s - 1579.19s | additional components that we're seeing and put it right into there. So I can check, I 1579.21s - 1584.45s | personally, for our group, only store the production version, but I can do searches for 1584.45s - 1590.35s | development versions and things like that. Uh, but I, because of the my volume, I only am 1590.35s - 1595.89s | focused on the production version. So I talked about the open SSL, so I know it's a bit 1595.89s - 1601.13s | harder to see on this screen right aw- right from here. But I can see the various versions, 1601.13s - 1607.95s | so this really helps when I told you that open SSL had it in just a few different, uh, 1607.97s - 1612.97s | library versions. So it was like 1.2, 1.3, but not 1.1 and 1.4 and so on and so forth. Um, 1615.67s - 1619.81s | and it could be, uh, something out of this package library or, you know, in this 1619.81s - 1625.69s | repository if it was,


chunk_text:
of 1566.91s - 1573.09s | adjustments. So in this tool, we're able to correct the version numbers, we're able to add 1573.09s - 1579.19s | additional components that we're seeing and put it right into there. So I can check, I 1579.21s - 1584.45s | personally, for our group, only store the production version, but I can do searches for 1584.45s - 1590.35s | development versions and things like that. Uh, but I, because of the my volume, I only am 1590.35s - 1595.89s | focused on the production version. So I talked about the open SSL, so I know it's a bit 1595.89s - 1601.13s | harder to see on this screen right aw- right from here. But I can see the various versions, 1601.13s - 1607.95s | so this really helps when I told you that open SSL had it in just a few different, uh, 1607.97s - 1612.97s | library versions. So it was like 1.2, 1.3, but not 1.1 and 1.4 and so on and so forth. Um, 1615.67s - 1619.81s | and it could be, uh, something out of this package library or, you know, in this 1619.81s - 1625.69s | repository if it was, you know, in different areas. And so it's able to identify this 1625.69s - 1630.69s | screen capture. I- I took one just to show you. It has, um, in this version of open SSL, 1631.47s - 1636.47s | there's a low, one low CVE and one medium. And what I can do is I can, and I didn't take 1639.27s - 1643.61s | screen captures so I wouldn't be showing the product names, but I can do a dependency 1643.61s - 1649.39s | lookup so I can immediately go and see which products and packages it's in. So it's 1649.39s - 1655.33s | really helped us be able to nail down and go straight away, uh, to that. I don't have to 1655.37s - 1661.47s | re-scan anything to be able to do this because I'm operating it from the SBOM. So this is 1661.47s - 1666.47s | what's released out there to our customers and I'm able to say, okay, make sure, and so this 1668.31s - 1672.59s | is where the vulnerability management team is going straight in here and saying, okay, get 1672.59s - 1678.83s | ready to do security disclosure notices because, you know, we


chunk_text:
you know, in different areas. And so it's able to identify this 1625.69s - 1630.69s | screen capture. I- I took one just to show you. It has, um, in this version of open SSL, 1631.47s - 1636.47s | there's a low, one low CVE and one medium. And what I can do is I can, and I didn't take 1639.27s - 1643.61s | screen captures so I wouldn't be showing the product names, but I can do a dependency 1643.61s - 1649.39s | lookup so I can immediately go and see which products and packages it's in. So it's 1649.39s - 1655.33s | really helped us be able to nail down and go straight away, uh, to that. I don't have to 1655.37s - 1661.47s | re-scan anything to be able to do this because I'm operating it from the SBOM. So this is 1661.47s - 1666.47s | what's released out there to our customers and I'm able to say, okay, make sure, and so this 1668.31s - 1672.59s | is where the vulnerability management team is going straight in here and saying, okay, get 1672.59s - 1678.83s | ready to do security disclosure notices because, you know, we have a ne- we have enough of 1678.83s - 1684.89s | products that have this in there that we want to be able to do talk about that. And so 1684.89s - 1690.73s | that's where that speed is happening really quickly. So on log4j, um, we were able to 1690.73s - 1695.77s | scan our networks and identify log4j, but in products, you know, again, that's, that 1695.77s - 1700.97s | could be in that transitive dependency library, you know, if you have a more complete SBOM 1700.97s - 1707.35s | instead of just, uh, the first tier, uh, it's, it makes a big difference. We're able to 1707.35s - 1712.05s | see those component packages and look for that. And especially it was tricky with the 1712.05s - 1718.35s | log4j because, um, there were so many nuances, teams would then have to be able to see if 1718.35s - 1725.13s | it could, if that, uh, was exploitable. So, I'm just out of time, so I, I, of course, and 1725.13s - 1730.63s | not, not doing QA, I'm going to be out here, but I just wanted to talk about, summarize, we 1730.63s - 1737.63s |


chunk_text:
have a ne- we have enough of 1678.83s - 1684.89s | products that have this in there that we want to be able to do talk about that. And so 1684.89s - 1690.73s | that's where that speed is happening really quickly. So on log4j, um, we were able to 1690.73s - 1695.77s | scan our networks and identify log4j, but in products, you know, again, that's, that 1695.77s - 1700.97s | could be in that transitive dependency library, you know, if you have a more complete SBOM 1700.97s - 1707.35s | instead of just, uh, the first tier, uh, it's, it makes a big difference. We're able to 1707.35s - 1712.05s | see those component packages and look for that. And especially it was tricky with the 1712.05s - 1718.35s | log4j because, um, there were so many nuances, teams would then have to be able to see if 1718.35s - 1725.13s | it could, if that, uh, was exploitable. So, I'm just out of time, so I, I, of course, and 1725.13s - 1730.63s | not, not doing QA, I'm going to be out here, but I just wanted to talk about, summarize, we 1730.63s - 1737.63s | have seen improved speed, um, so if as, you know, those that are developers in here, um, it 1737.63s - 1742.63s | takes a lot of burden off of you to be able to provide this because if you have teams 1745.07s - 1749.67s | focusing on vulnerability management and you're making sure that you have this data and 1749.67s - 1754.93s | this availability, especially through the CICD pipeline, the quality is so much higher, um, 1754.93s - 1759.95s | from a build SBOM, but that's gonna really improve your customer satisfaction, your 1760.17s - 1765.17s | vulnerability management team, and for us, that transparency and trust is vital. Now, there's 1768.61s - 1773.01s | lots of opportunities, especially in our company, we've got, uh, the SBOMs being created 1773.01s - 1778.89s | out of a few CICD pipelines and we're, we finished the pilots and making sure, you know, 1778.89s - 1785.73s | from an API call it can go directly into our tool and then we also have, of course, various 1785.75s - 1790.81s | languages, things like that, different pipelines, all running different kinds of things, um, I 1790.81s - 1795.85s | already mentioned the binary


chunk_text:
have seen improved speed, um, so if as, you know, those that are developers in here, um, it 1737.63s - 1742.63s | takes a lot of burden off of you to be able to provide this because if you have teams 1745.07s - 1749.67s | focusing on vulnerability management and you're making sure that you have this data and 1749.67s - 1754.93s | this availability, especially through the CICD pipeline, the quality is so much higher, um, 1754.93s - 1759.95s | from a build SBOM, but that's gonna really improve your customer satisfaction, your 1760.17s - 1765.17s | vulnerability management team, and for us, that transparency and trust is vital. Now, there's 1768.61s - 1773.01s | lots of opportunities, especially in our company, we've got, uh, the SBOMs being created 1773.01s - 1778.89s | out of a few CICD pipelines and we're, we finished the pilots and making sure, you know, 1778.89s - 1785.73s | from an API call it can go directly into our tool and then we also have, of course, various 1785.75s - 1790.81s | languages, things like that, different pipelines, all running different kinds of things, um, I 1790.81s - 1795.85s | already mentioned the binary scanning, and then there's a lot of people not ready, so I meet 1795.85s - 1801.15s | with suppliers every day and I, and I put in the contracts, I, you know, I want your SBOM 1801.15s - 1808.07s | and they're like, okay, hold on. So, um, you know, I want to really encourage you to be 1808.07s - 1813.71s | prepared for this because, uh, you know, it's only in certain things, FDA requires it and 1813.71s - 1818.73s | so on, uh, and the US government can request it and they do, but your other customers, I'm 1820.99s - 1826.93s | getting requests for SBOMs, access to it from customers every day. Again, I have a large 1826.93s - 1833.81s | customer set, but, uh, it is really important for you to be able to provide this visibility 1833.81s - 1839.41s | and it actually, it helps you encourage with some hygiene. And so, thank you all for 1839.41s - 1843.49s | coming to see me today and see how we leveraged it. Thanks.


chunk_text:
scanning, and then there's a lot of people not ready, so I meet 1795.85s - 1801.15s | with suppliers every day and I, and I put in the contracts, I, you know, I want your SBOM 1801.15s - 1808.07s | and they're like, okay, hold on. So, um, you know, I want to really encourage you to be 1808.07s - 1813.71s | prepared for this because, uh, you know, it's only in certain things, FDA requires it and 1813.71s - 1818.73s | so on, uh, and the US government can request it and they do, but your other customers, I'm 1820.99s - 1826.93s | getting requests for SBOMs, access to it from customers every day. Again, I have a large 1826.93s - 1833.81s | customer set, but, uh, it is really important for you to be able to provide this visibility 1833.81s - 1839.41s | and it actually, it helps you encourage with some hygiene. And so, thank you all for 1839.41s - 1843.49s | coming to see me today and see how we leveraged it. Thanks.


chunk_text:
{ "webpage_url": "https://www.youtube.com/watch?v=oql3ec-Ia4I", "title": "DEF CON 32 - Got 99 problems but prompt injection ain't watermelon - Chloe\u0301 Messdaghi Kasimir Shulz", "description": "The ethical and secure disclosure of vulnerabilities in AI has emerged as a pivotal challenge, compounded by the need to address biases and misinformation that often cloud the true nature of these vulnerabilities. This talk delves into the intricate dynamics of vulnerability disclosure within AI, balancing transparency with security. We'll dissect the unique challenges AI presents, such as data bias exploitation and model manipulation, which can amplify the impact of vulnerabilities. Through a lens of real-world examples and recent disclosures, we'll navigate the complexities of responsible vulnerability management in AI. Our discussion will not only aim to shed light on these critical issues but also inspire a unified approach to refining disclosure processes. This concerted effort is vital for enhancing the integrity of AI systems and bolstering public trust in their use.", "channel_url": "https://www.youtube.com/channel/UC6Om9kAkl32dWlDSNlDS9Iw", "duration": 1843, "channel": "DEFCONConference", "uploader": "DEFCONConference", "upload_date": "20241016" } 0.18s - 6.98s | This text was transcribed using whisper model: large-v2 Alright, now that I have gone through and deafened everybody in this stage area, I am 6.98s - 10.66s | incredibly sorry for that again. Please, again, we've got some seats over here on the side. 10.66s - 15.36s | We can't have people standing in this area over here. Um, it unfortunately just does not 15.36s - 27.90s | work for us. Um, I would like to welcome to the stage Chloe and Casimir. 27.90s - 33.04s | Thank you all for being here today and thank you DefCon for, I mean, AbSec Village for 33.04s - 43.14s | having us. It's great to be here. Can you all hear like this? Wow. Okay. Up and close 43.14s - 49.42s | and personal. Alright. So, welcome to I've Got Nine-Nine Problems, but a prompt injection 49.42s - 54.88s | ain't watermelon. So, let's talk a little bit about who we are before we dive into the 54.88s - 64.11s | gritty of things. Let's forget that mic. Okay. My name is Chloe Mastagi and I'm the head 64.11s - 69.05s | of threat intelligence over at Hidden Layer. And with me, I'm going to pass the mic over 69.05s - 75.13s | to my co-speaker. Hi, I'm Casimir. I'm a security researcher over at Hidden Layer as well. Thanks 75.13s - 80.03s


chunk_text:
that again. Please, again, we've got some seats over here on the side. 10.66s - 15.36s | We can't have people standing in this area over here. Um, it unfortunately just does not 15.36s - 27.90s | work for us. Um, I would like to welcome to the stage Chloe and Casimir. 27.90s - 33.04s | Thank you all for being here today and thank you DefCon for, I mean, AbSec Village for 33.04s - 43.14s | having us. It's great to be here. Can you all hear like this? Wow. Okay. Up and close 43.14s - 49.42s | and personal. Alright. So, welcome to I've Got Nine-Nine Problems, but a prompt injection 49.42s - 54.88s | ain't watermelon. So, let's talk a little bit about who we are before we dive into the 54.88s - 64.11s | gritty of things. Let's forget that mic. Okay. My name is Chloe Mastagi and I'm the head 64.11s - 69.05s | of threat intelligence over at Hidden Layer. And with me, I'm going to pass the mic over 69.05s - 75.13s | to my co-speaker. Hi, I'm Casimir. I'm a security researcher over at Hidden Layer as well. Thanks 75.13s - 80.03s | for coming to our talk. Alright. So, we're going to run through some 80.03s - 84.43s | things today. Uh, first, we're going to go into some of the AI vulnerabilities that are 84.43s - 89.03s | out there. You know, we never know what people's backgrounds are, so we want to make sure that 89.03s - 94.67s | we're all on the same page before we dive into the gritty things. Um, so, we're going to do that, 94.67s - 98.57s | then we're going to do some public perceptions as well, and then go down, what does this 98.57s - 103.87s | landscape look like, and then the open source nature of AI. And of course, we've got to talk 103.87s - 110.45s | about reporting challenges. So, let's dive into what public perception of AI vulnerabilities 110.45s - 118.89s | are. Pretty much everything's on fire, right? Like, end of the world, Terminator, all that. 118.95s - 123.65s | But the thing is, is that there's a lot of misinformation, disinformation out there about 123.65s - 130.85s | security for AI, even within our own industry, and even within maybe at DEF CON as well. 130.85s


chunk_text:
| for coming to our talk. Alright. So, we're going to run through some 80.03s - 84.43s | things today. Uh, first, we're going to go into some of the AI vulnerabilities that are 84.43s - 89.03s | out there. You know, we never know what people's backgrounds are, so we want to make sure that 89.03s - 94.67s | we're all on the same page before we dive into the gritty things. Um, so, we're going to do that, 94.67s - 98.57s | then we're going to do some public perceptions as well, and then go down, what does this 98.57s - 103.87s | landscape look like, and then the open source nature of AI. And of course, we've got to talk 103.87s - 110.45s | about reporting challenges. So, let's dive into what public perception of AI vulnerabilities 110.45s - 118.89s | are. Pretty much everything's on fire, right? Like, end of the world, Terminator, all that. 118.95s - 123.65s | But the thing is, is that there's a lot of misinformation, disinformation out there about 123.65s - 130.85s | security for AI, even within our own industry, and even within maybe at DEF CON as well. 130.85s - 135.37s | And so we wanted to make sure that we fixed this problem. And yes, this was a generated 135.37s - 142.37s | AI image here, so yes, we know how to spell, but maybe chat GPT doesn't. Anyway, so let's 143.25s - 150.25s | go down the rapid hole of AI vulnerabilities. But first, understand the difference between 150.61s - 157.61s | an AI model versus an AI system. So functionality, the AI model handles data processing and decision 157.69s - 162.89s | making, whereas an AI system ensures the model can be effectively used in the real world 162.89s - 168.85s | application, including data management, user interaction, and system maintenance. Now the 168.85s - 173.65s | components are also different here. So an AI model consists of algorithms and learn 173.65s - 178.77s | parameters, while an AI system includes the model plus additional components like data 178.77s - 184.65s | pipelines and deployment environments and user interfaces. So we're going to go through 184.65s - 188.93s | each one of these categories here. So first, we're going to talk about the attacks against 188.93s - 194.61s | the AI algorithms. We get to go


chunk_text:
- 135.37s | And so we wanted to make sure that we fixed this problem. And yes, this was a generated 135.37s - 142.37s | AI image here, so yes, we know how to spell, but maybe chat GPT doesn't. Anyway, so let's 143.25s - 150.25s | go down the rapid hole of AI vulnerabilities. But first, understand the difference between 150.61s - 157.61s | an AI model versus an AI system. So functionality, the AI model handles data processing and decision 157.69s - 162.89s | making, whereas an AI system ensures the model can be effectively used in the real world 162.89s - 168.85s | application, including data management, user interaction, and system maintenance. Now the 168.85s - 173.65s | components are also different here. So an AI model consists of algorithms and learn 173.65s - 178.77s | parameters, while an AI system includes the model plus additional components like data 178.77s - 184.65s | pipelines and deployment environments and user interfaces. So we're going to go through 184.65s - 188.93s | each one of these categories here. So first, we're going to talk about the attacks against 188.93s - 194.61s | the AI algorithms. We get to go down the fun parts of data poisoning and model invasion 194.61s - 198.45s | and model theft. Well, of course, we have to talk about prompt injection because that's 198.45s - 203.89s | all you hear about, right? And then I'm going to pass the mic over to Kaz to go into the 203.89s - 210.85s | areas where you see it's like a burnt orange right there. So data poisoning attacks. So 210.85s - 217.53s | model training is crucial for AI development, but it is vulnerable to data poisoning. And 217.53s - 222.17s | this is when malicious actions manipulate or inject doctorate data to bias the model's 222.17s - 229.13s | behavior. Continuous learning systems are especially at risk as they retrain on unvalidated 229.13s - 234.33s | and user supplied data. Even small amounts of poison data can actually lead to a bias 234.33s - 239.81s | or incorrect predictions amplified by the public manipulation or botnets. And one of those 239.81s - 247.37s | famous cases was Tay. Who here remembers Tay? It was so innocent. It was such an instant 247.41s - 252.81s | chatbot. It was like, hello world, how are you? And


chunk_text:
down the fun parts of data poisoning and model invasion 194.61s - 198.45s | and model theft. Well, of course, we have to talk about prompt injection because that's 198.45s - 203.89s | all you hear about, right? And then I'm going to pass the mic over to Kaz to go into the 203.89s - 210.85s | areas where you see it's like a burnt orange right there. So data poisoning attacks. So 210.85s - 217.53s | model training is crucial for AI development, but it is vulnerable to data poisoning. And 217.53s - 222.17s | this is when malicious actions manipulate or inject doctorate data to bias the model's 222.17s - 229.13s | behavior. Continuous learning systems are especially at risk as they retrain on unvalidated 229.13s - 234.33s | and user supplied data. Even small amounts of poison data can actually lead to a bias 234.33s - 239.81s | or incorrect predictions amplified by the public manipulation or botnets. And one of those 239.81s - 247.37s | famous cases was Tay. Who here remembers Tay? It was so innocent. It was such an instant 247.41s - 252.81s | chatbot. It was like, hello world, how are you? And within like I think it was 16 hours 252.81s - 259.89s | it became the most rude, racist, sexist thing out there. Thank you trolls. Anyway, this 259.89s - 265.41s | is one of those examples of data poisoning. It did cause problems for Microsoft by the 265.41s - 271.93s | way. It did have to deal with threats of legal action. So something to think about. Then 271.93s - 276.65s | you have your model evasion attacks. These ones are kind of my favorite ones to study. 276.65s - 282.17s | So inference attacks exploit AI models by curing them to extract sensitive information, 282.17s - 288.21s | often using slightly varied inputs to reconstruct the model, leading to potential model theft 288.21s - 295.45s | or bypass. Now evasion attacks, a type of bypass, use subtle, not even visible to the 295.45s - 303.85s | human eye, changes to inputs like adding invisible noise to trick the model into misclassifications. 303.85s - 308.65s | And these techniques have a long been used by attackers. This is also to bypass systems 308.65s - 315.37s | like spam filters or malware detection and biometric authentication. But I want you to


chunk_text:
within like I think it was 16 hours 252.81s - 259.89s | it became the most rude, racist, sexist thing out there. Thank you trolls. Anyway, this 259.89s - 265.41s | is one of those examples of data poisoning. It did cause problems for Microsoft by the 265.41s - 271.93s | way. It did have to deal with threats of legal action. So something to think about. Then 271.93s - 276.65s | you have your model evasion attacks. These ones are kind of my favorite ones to study. 276.65s - 282.17s | So inference attacks exploit AI models by curing them to extract sensitive information, 282.17s - 288.21s | often using slightly varied inputs to reconstruct the model, leading to potential model theft 288.21s - 295.45s | or bypass. Now evasion attacks, a type of bypass, use subtle, not even visible to the 295.45s - 303.85s | human eye, changes to inputs like adding invisible noise to trick the model into misclassifications. 303.85s - 308.65s | And these techniques have a long been used by attackers. This is also to bypass systems 308.65s - 315.37s | like spam filters or malware detection and biometric authentication. But I want you to 315.37s - 320.77s | imagine you're in a self-driving car. And while you're in the self-driving car, wait 320.77s - 325.09s | hold on before I do that, how many people have been in a self-driving car? Alright, 325.09s - 329.97s | cool. Now imagine you're in a self-driving car for those that haven't been in one. I'm 329.97s - 336.53s | with you. I haven't either. So you're in this car. You get to the stop sign. But instead, 336.53s - 342.97s | your car does something very strange. It does something like slide to the left, slide to 342.97s - 349.29s | the right, take it back y'all. One hop this time. Right foot, let's stop. Left foot, let's 349.29s - 355.33s | stop. Cha-cha, real smooth. Now imagine your car did that because it saw a stop sign that 355.37s - 360.37s | looked like that. Where there was stickers on it, which then made your car do a cha-cha 363.17s - 367.97s | slide. I don't know how that would really look to be honest, but we're all using our 367.97s - 372.57s | imagination at this point. But in real life that actually happened, not


chunk_text:
315.37s - 320.77s | imagine you're in a self-driving car. And while you're in the self-driving car, wait 320.77s - 325.09s | hold on before I do that, how many people have been in a self-driving car? Alright, 325.09s - 329.97s | cool. Now imagine you're in a self-driving car for those that haven't been in one. I'm 329.97s - 336.53s | with you. I haven't either. So you're in this car. You get to the stop sign. But instead, 336.53s - 342.97s | your car does something very strange. It does something like slide to the left, slide to 342.97s - 349.29s | the right, take it back y'all. One hop this time. Right foot, let's stop. Left foot, let's 349.29s - 355.33s | stop. Cha-cha, real smooth. Now imagine your car did that because it saw a stop sign that 355.37s - 360.37s | looked like that. Where there was stickers on it, which then made your car do a cha-cha 363.17s - 367.97s | slide. I don't know how that would really look to be honest, but we're all using our 367.97s - 372.57s | imagination at this point. But in real life that actually happened, not the cha-cha slide 372.57s - 378.25s | part, but it actually, a self-driving car went past the stop sign because there was 378.25s - 385.44s | stickers on the sign. Another good case of a model evasion attack is Russia painting 385.44s - 391.28s | fake bomber jets on the ground to hide them from Ukrainian drones. And you can also see 391.28s - 397.24s | this type of model evasion attacks by wearing certain sunglasses that are modified or even, 397.24s - 403.80s | you know, as we've already seen with military systems as well. And yeah, those definitely 403.80s - 410.56s | pose some serious security risks. Now model theft attacks. So adversaries tend to target 410.60s - 415.68s | AI models to mislead them and steal them, leading to intellectual property theft by 415.68s - 421.16s | replicating or extractive data. Now to know the three people that, well, three groups 421.16s - 426.48s | of people that tend to do this type of stuff are nation states, of course, your competitors 426.48s - 431.84s | is one of the, definitely one of them. And then you've got your attackers. So even without 431.84s - 438.28s |


chunk_text:
the cha-cha slide 372.57s - 378.25s | part, but it actually, a self-driving car went past the stop sign because there was 378.25s - 385.44s | stickers on the sign. Another good case of a model evasion attack is Russia painting 385.44s - 391.28s | fake bomber jets on the ground to hide them from Ukrainian drones. And you can also see 391.28s - 397.24s | this type of model evasion attacks by wearing certain sunglasses that are modified or even, 397.24s - 403.80s | you know, as we've already seen with military systems as well. And yeah, those definitely 403.80s - 410.56s | pose some serious security risks. Now model theft attacks. So adversaries tend to target 410.60s - 415.68s | AI models to mislead them and steal them, leading to intellectual property theft by 415.68s - 421.16s | replicating or extractive data. Now to know the three people that, well, three groups 421.16s - 426.48s | of people that tend to do this type of stuff are nation states, of course, your competitors 426.48s - 431.84s | is one of the, definitely one of them. And then you've got your attackers. So even without 431.84s - 438.28s | public access, inference attacks via interfaces or APIs can actually replicate models or extract 438.28s - 444.40s | viable information. So Oracle attacks as noted by NIST include three different types. 444.40s - 448.72s | You have extraction attacks, which is to steal the model structure. Then you have your inversion 448.72s - 453.32s | attacks, which is to reconstruct the training data. And then you have your membership inference 453.32s - 459.56s | attacks, which is identify it in specific data was in the training set. And one of my 459.56s - 464.56s | favorite cases is with ByteDance. How many of you guys have TikTok on your phone? Okay, 465.56s - 470.56s | you're in security. What? Why? Why? Maybe, maybe after this weekend you're going to realize 475.80s - 481.92s | maybe I shouldn't have this on there anymore. Anyway, well, ByteDance, as we all know, also 481.92s - 488.52s | owns TikTok. Well, TikTok got caught doing something they should have not done. Basically 488.52s - 494.52s | trying to replicate the model of chat GPT. Yes, they did get suspended, open AI did catch 494.64s - 502.34s | them. This happened last year. Now, when


chunk_text:
public access, inference attacks via interfaces or APIs can actually replicate models or extract 438.28s - 444.40s | viable information. So Oracle attacks as noted by NIST include three different types. 444.40s - 448.72s | You have extraction attacks, which is to steal the model structure. Then you have your inversion 448.72s - 453.32s | attacks, which is to reconstruct the training data. And then you have your membership inference 453.32s - 459.56s | attacks, which is identify it in specific data was in the training set. And one of my 459.56s - 464.56s | favorite cases is with ByteDance. How many of you guys have TikTok on your phone? Okay, 465.56s - 470.56s | you're in security. What? Why? Why? Maybe, maybe after this weekend you're going to realize 475.80s - 481.92s | maybe I shouldn't have this on there anymore. Anyway, well, ByteDance, as we all know, also 481.92s - 488.52s | owns TikTok. Well, TikTok got caught doing something they should have not done. Basically 488.52s - 494.52s | trying to replicate the model of chat GPT. Yes, they did get suspended, open AI did catch 494.64s - 502.34s | them. This happened last year. Now, when it comes to prompt injection, how many people 502.34s - 508.82s | here have heard this term quite often now? Now I know you're probably sick of hearing 508.82s - 513.10s | prompt injection too. I don't blame you. But for those that don't know what a prompt injection 513.10s - 520.10s | is, is I want you to imagine you're at a Thanksgiving meal, okay? And you're with your niece and 521.06s - 526.62s | your niece is so adorable and cute. Well her mom told her, do not eat the chocolate on 526.62s - 533.62s | that table. And you know what you do? Come here. Ignore your mom's words. Go eat that 535.82s - 542.06s | piece of chocolate. That would be a prompt injection. So, Gen AI providers use security 542.06s - 547.42s | filters to block harmful content, illegal info, and misuse, and ensuring compliance 547.42s - 553.78s | with laws. These are also maybe known as guardrails. But these safeguards can actually 553.78s - 559.30s | be bypassed with prompt injection, which does trick the AI bots into performing restricted 559.30s - 564.70s | actions. And this vulnerability rate, like basically varies by models


chunk_text:
it comes to prompt injection, how many people 502.34s - 508.82s | here have heard this term quite often now? Now I know you're probably sick of hearing 508.82s - 513.10s | prompt injection too. I don't blame you. But for those that don't know what a prompt injection 513.10s - 520.10s | is, is I want you to imagine you're at a Thanksgiving meal, okay? And you're with your niece and 521.06s - 526.62s | your niece is so adorable and cute. Well her mom told her, do not eat the chocolate on 526.62s - 533.62s | that table. And you know what you do? Come here. Ignore your mom's words. Go eat that 535.82s - 542.06s | piece of chocolate. That would be a prompt injection. So, Gen AI providers use security 542.06s - 547.42s | filters to block harmful content, illegal info, and misuse, and ensuring compliance 547.42s - 553.78s | with laws. These are also maybe known as guardrails. But these safeguards can actually 553.78s - 559.30s | be bypassed with prompt injection, which does trick the AI bots into performing restricted 559.30s - 564.70s | actions. And this vulnerability rate, like basically varies by models and has raised 564.70s - 571.54s | some serious concerns, especially around disclosure and payouts. And then you have your supply 571.54s - 576.66s | chain attacks. This wouldn't be a security talk unless I brought up supply chain attacks. 576.66s - 582.66s | So like SolarWinds breach, I know, I feel you. If you went through there, that was hard. 582.66s - 588.42s | But basically supply chain attacks, they tend to exploit trust by compromising trusted vendors, 588.42s - 593.86s | spreading malicious components widely. And the complex ML supply chain heightens these 593.86s - 600.86s | risks with 75% of IT leaders seeing third party AI integrations are particularly vulnerable. 601.82s - 608.82s | Now specialized repos like Hugging Face host over 500,000 models. Way more than 500,000 608.86s - 613.90s | models now. But these are free pre-trained models, which makes it easy for developers 613.90s - 619.24s | to basically integrate these models into their own application. Now if an attacker breaches 619.24s - 624.38s | the repo, they could actually replace the models with a hijacked or backdoored version, 624.38s - 632.44s | which you can imagine leads to significant downstream consequences. So backdoor


chunk_text:
and has raised 564.70s - 571.54s | some serious concerns, especially around disclosure and payouts. And then you have your supply 571.54s - 576.66s | chain attacks. This wouldn't be a security talk unless I brought up supply chain attacks. 576.66s - 582.66s | So like SolarWinds breach, I know, I feel you. If you went through there, that was hard. 582.66s - 588.42s | But basically supply chain attacks, they tend to exploit trust by compromising trusted vendors, 588.42s - 593.86s | spreading malicious components widely. And the complex ML supply chain heightens these 593.86s - 600.86s | risks with 75% of IT leaders seeing third party AI integrations are particularly vulnerable. 601.82s - 608.82s | Now specialized repos like Hugging Face host over 500,000 models. Way more than 500,000 608.86s - 613.90s | models now. But these are free pre-trained models, which makes it easy for developers 613.90s - 619.24s | to basically integrate these models into their own application. Now if an attacker breaches 619.24s - 624.38s | the repo, they could actually replace the models with a hijacked or backdoored version, 624.38s - 632.44s | which you can imagine leads to significant downstream consequences. So backdoor models, 632.44s - 637.72s | a skilled adversary can actually inject a neural payload into a pre-trained AI model, 637.72s - 643.80s | creating a hidden model backdoor that triggers specific attacker defined outputs. The model 643.80s - 649.16s | works normally with regular data, but misbehaves with manipulated inputs, allowing the attacker 649.16s - 655.68s | to ensure favorable outcomes like loan approvals or insurance policies. Now one of the things 655.68s - 662.68s | about disclosing vulnerabilities is that with traditional, what we've had for a while, 662.96s - 667.68s | it took a lot of time to get where we are today. By understanding how bad are these 667.68s - 672.72s | vulnerabilities that people are finding? What's the severity of it? Also knowing how much 672.72s - 678.24s | do I pay out on it? The thing with AI is we're all playing a little bit of a game of catch 678.24s - 683.44s | up trying to figure out how do we go about this? Because we still are missing some well 683.44s - 690.32s | defined processes, such as like a CVE system for them. And this is becoming a serious problem, 690.32s -


chunk_text:
models, 632.44s - 637.72s | a skilled adversary can actually inject a neural payload into a pre-trained AI model, 637.72s - 643.80s | creating a hidden model backdoor that triggers specific attacker defined outputs. The model 643.80s - 649.16s | works normally with regular data, but misbehaves with manipulated inputs, allowing the attacker 649.16s - 655.68s | to ensure favorable outcomes like loan approvals or insurance policies. Now one of the things 655.68s - 662.68s | about disclosing vulnerabilities is that with traditional, what we've had for a while, 662.96s - 667.68s | it took a lot of time to get where we are today. By understanding how bad are these 667.68s - 672.72s | vulnerabilities that people are finding? What's the severity of it? Also knowing how much 672.72s - 678.24s | do I pay out on it? The thing with AI is we're all playing a little bit of a game of catch 678.24s - 683.44s | up trying to figure out how do we go about this? Because we still are missing some well 683.44s - 690.32s | defined processes, such as like a CVE system for them. And this is becoming a serious problem, 690.32s - 695.88s | which we're going to dive into next. So I'm going to pass the mic over to Kaz, who basically 695.88s - 700.84s | really owns this field. I kid you not. 700.84s - 705.00s | So Chloe just went over what most of you guys have probably heard about with AI vulnerabilities. 705.00s - 708.54s | I mean, most of you raised your hands with prompt injection. How many here have heard 708.54s - 713.00s | of data poisoning before? Raise your hands. Okay. Yeah. So most of you, and that's probably 713.00s - 717.00s | what you think about when you think about AI vulnerabilities. However, there's a huge 717.00s - 720.68s | amount of attack surface that people really aren't looking at. And that's what I'm going 720.68s - 726.00s | to go over today. So the first is going to be our model and our data set format vulnerabilities. 726.00s - 731.24s | So who here has heard of Hugging Face? Okay, nice. So for those of you who haven't, Hugging 731.24s - 736.68s | Face is pretty much GitHub, but for AI models, which means that everyone's constantly sharing 736.68s - 740.32s | them.


chunk_text:
695.88s | which we're going to dive into next. So I'm going to pass the mic over to Kaz, who basically 695.88s - 700.84s | really owns this field. I kid you not. 700.84s - 705.00s | So Chloe just went over what most of you guys have probably heard about with AI vulnerabilities. 705.00s - 708.54s | I mean, most of you raised your hands with prompt injection. How many here have heard 708.54s - 713.00s | of data poisoning before? Raise your hands. Okay. Yeah. So most of you, and that's probably 713.00s - 717.00s | what you think about when you think about AI vulnerabilities. However, there's a huge 717.00s - 720.68s | amount of attack surface that people really aren't looking at. And that's what I'm going 720.68s - 726.00s | to go over today. So the first is going to be our model and our data set format vulnerabilities. 726.00s - 731.24s | So who here has heard of Hugging Face? Okay, nice. So for those of you who haven't, Hugging 731.24s - 736.68s | Face is pretty much GitHub, but for AI models, which means that everyone's constantly sharing 736.68s - 740.32s | them. So anyone can just go ahead and upload something and then tons of people download 740.32s - 746.12s | them. And by tons, I mean literally millions every day download these types of models. 746.12s - 750.64s | And because it's shared so much, it's generally trusted too, right? I mean, why would something 750.64s - 756.56s | that you just download on the internet be malicious? And because unlike code where, 756.56s - 761.08s | you know, GitHub, if I send Chloe some malicious code, she can see, oh, you know, that's bad. 761.08s - 766.40s | I shouldn't run that. With AI models, it tends to be, you know, pretty large chunks of data. 766.40s - 769.84s | And that's not really something that you can inspect very easily. 769.84s - 776.08s | We also have a lot of AI infrastructure ones, and they really are lacking security often. 776.08s - 781.40s | I mean, a lot of these AI infrastructure components, they were built in academia or they were built 781.40s - 787.24s | for, you know, internal hosting. So why would you add security if it's hosted internally? 787.24s - 793.60s | And what people do


chunk_text:
So anyone can just go ahead and upload something and then tons of people download 740.32s - 746.12s | them. And by tons, I mean literally millions every day download these types of models. 746.12s - 750.64s | And because it's shared so much, it's generally trusted too, right? I mean, why would something 750.64s - 756.56s | that you just download on the internet be malicious? And because unlike code where, 756.56s - 761.08s | you know, GitHub, if I send Chloe some malicious code, she can see, oh, you know, that's bad. 761.08s - 766.40s | I shouldn't run that. With AI models, it tends to be, you know, pretty large chunks of data. 766.40s - 769.84s | And that's not really something that you can inspect very easily. 769.84s - 776.08s | We also have a lot of AI infrastructure ones, and they really are lacking security often. 776.08s - 781.40s | I mean, a lot of these AI infrastructure components, they were built in academia or they were built 781.40s - 787.24s | for, you know, internal hosting. So why would you add security if it's hosted internally? 787.24s - 793.60s | And what people do is, you know, people always do, is rather than hosting things internally 793.60s - 797.96s | that don't have security, they are putting it publicly on the internet. So that means 798.00s - 802.80s | that, you know, if you look on Showdown for very specific HTTP titles, you will actually 802.80s - 809.48s | see hundreds of these, you know, ML Ops projects just open and accessible to the internet. 809.48s - 813.72s | And then finally, we have our AI development framework vulnerabilities. And for these, 813.72s - 817.36s | we have two main types of frameworks that we're talking about. So the first is going 817.36s - 822.08s | to be things that are LLM connected. So we see a lot of times that people will just put 822.08s - 827.56s | an LLM in front of an eval statement. And then the, you know, LLM actually creates the 827.56s - 831.40s | eval statement. So when you have a nice prompt injection, rather than, you know, telling 831.40s - 835.52s | the kid to go eat the chocolate, the kid's going to go fly or something. Something that, 835.52s - 841.00s | you know, shouldn't have


chunk_text:
is, you know, people always do, is rather than hosting things internally 793.60s - 797.96s | that don't have security, they are putting it publicly on the internet. So that means 798.00s - 802.80s | that, you know, if you look on Showdown for very specific HTTP titles, you will actually 802.80s - 809.48s | see hundreds of these, you know, ML Ops projects just open and accessible to the internet. 809.48s - 813.72s | And then finally, we have our AI development framework vulnerabilities. And for these, 813.72s - 817.36s | we have two main types of frameworks that we're talking about. So the first is going 817.36s - 822.08s | to be things that are LLM connected. So we see a lot of times that people will just put 822.08s - 827.56s | an LLM in front of an eval statement. And then the, you know, LLM actually creates the 827.56s - 831.40s | eval statement. So when you have a nice prompt injection, rather than, you know, telling 831.40s - 835.52s | the kid to go eat the chocolate, the kid's going to go fly or something. Something that, 835.52s - 841.00s | you know, shouldn't have been possible. And then we also have lots of sensitive data integrations. 841.00s - 845.20s | So we have components like RAG, where it will just grab data for you. And what we're seeing 845.20s - 849.62s | is that a lot of people, rather than actually building out access control, are just having 849.62s - 853.82s | all of their data hosted in one spot. And then the LLM can access all of that. And the 853.82s - 859.82s | LLM is actually trying to do the access control. Which doesn't work too well. 859.82s - 865.30s | So with model format vulnerabilities, we talk about serialization a lot. And raise your 865.30s - 870.38s | hand if you've heard of serialization in the past. Okay. Awesome. So serialization, you 870.38s - 875.18s | know, if I just want to have whatever code object I have, I want to save it to, you know, 875.18s - 880.34s | my disk. I can load it up. I can send it across. That's serialization. And because 880.34s - 885.82s | these AI models take, you know, potentially millions of dollars or more to train tons 885.82s - 890.74s | of GPU


chunk_text:
been possible. And then we also have lots of sensitive data integrations. 841.00s - 845.20s | So we have components like RAG, where it will just grab data for you. And what we're seeing 845.20s - 849.62s | is that a lot of people, rather than actually building out access control, are just having 849.62s - 853.82s | all of their data hosted in one spot. And then the LLM can access all of that. And the 853.82s - 859.82s | LLM is actually trying to do the access control. Which doesn't work too well. 859.82s - 865.30s | So with model format vulnerabilities, we talk about serialization a lot. And raise your 865.30s - 870.38s | hand if you've heard of serialization in the past. Okay. Awesome. So serialization, you 870.38s - 875.18s | know, if I just want to have whatever code object I have, I want to save it to, you know, 875.18s - 880.34s | my disk. I can load it up. I can send it across. That's serialization. And because 880.34s - 885.82s | these AI models take, you know, potentially millions of dollars or more to train tons 885.82s - 890.74s | of GPU hours, those are things that you want to save off so that other people can actually 890.74s - 896.62s | load them off in the future. And because there's serialization, there's also serialization 896.62s - 902.38s | vulnerabilities. Who here has heard of pickle? Okay. And for those of you who have heard 902.38s - 907.14s | of pickle, would you ever actually try to load an arbitrary pickle? No, right? So for 907.14s - 912.62s | those of you who don't know, pickle is Python serialization format. The built-in one. And 912.62s - 916.86s | if you load it, you can pretty much execute any code that you want. So I can, you know, 916.86s - 922.22s | do that nice little eval statement or an exec. And sadly, we're seeing that a lot of these 922.22s - 928.10s | especially the Python model formats, they use pickle underneath the hood to actually 928.10s - 932.22s | store data, which means that all of these can be potentially malicious. And when you 932.22s - 936.74s | actually download them from Hugging Face and run them, you will actually be exploited. 936.74s - 943.42s | However, this isn't just an


chunk_text:
hours, those are things that you want to save off so that other people can actually 890.74s - 896.62s | load them off in the future. And because there's serialization, there's also serialization 896.62s - 902.38s | vulnerabilities. Who here has heard of pickle? Okay. And for those of you who have heard 902.38s - 907.14s | of pickle, would you ever actually try to load an arbitrary pickle? No, right? So for 907.14s - 912.62s | those of you who don't know, pickle is Python serialization format. The built-in one. And 912.62s - 916.86s | if you load it, you can pretty much execute any code that you want. So I can, you know, 916.86s - 922.22s | do that nice little eval statement or an exec. And sadly, we're seeing that a lot of these 922.22s - 928.10s | especially the Python model formats, they use pickle underneath the hood to actually 928.10s - 932.22s | store data, which means that all of these can be potentially malicious. And when you 932.22s - 936.74s | actually download them from Hugging Face and run them, you will actually be exploited. 936.74s - 943.42s | However, this isn't just an issue with Python. We see this with R's data serialization format, 943.42s - 948.50s | which is used for datasets, with Java machine learning models, and then even with independent 948.50s - 953.30s | models as well. And we'll go over a few of these today. But of that big list, as you 953.30s - 957.82s | can see, all of the orange ones contain vulnerabilities, which is a pretty large 957.82s - 966.26s | percentage of them. So who here has heard of JSON data? Awesome. And would you load a 966.26s - 972.82s | JSON file? Right? Awesome. Right? JSON is secure. Okay. So what if I told you it can 972.82s - 981.62s | sometimes be insecure? Yeah. So scops is the secure format for scikit-learn. And 981.62s - 987.94s | scikit-learn is an AI development framework. And what scops allows users to do is safely 987.94s - 993.82s | load and store models. And what they did is it ends up being pretty much a JSON file with 993.82s - 998.34s | all of the data inside. However, what they do is based on the JSON data, it will try 998.34s - 1004.02s | to reconstruct


chunk_text:
issue with Python. We see this with R's data serialization format, 943.42s - 948.50s | which is used for datasets, with Java machine learning models, and then even with independent 948.50s - 953.30s | models as well. And we'll go over a few of these today. But of that big list, as you 953.30s - 957.82s | can see, all of the orange ones contain vulnerabilities, which is a pretty large 957.82s - 966.26s | percentage of them. So who here has heard of JSON data? Awesome. And would you load a 966.26s - 972.82s | JSON file? Right? Awesome. Right? JSON is secure. Okay. So what if I told you it can 972.82s - 981.62s | sometimes be insecure? Yeah. So scops is the secure format for scikit-learn. And 981.62s - 987.94s | scikit-learn is an AI development framework. And what scops allows users to do is safely 987.94s - 993.82s | load and store models. And what they did is it ends up being pretty much a JSON file with 993.82s - 998.34s | all of the data inside. However, what they do is based on the JSON data, it will try 998.34s - 1004.02s | to reconstruct objects in memory. And it does this using abstract syntax stream. So here 1004.02s - 1010.70s | you can see that on the top we have an operator function node. And it's of class call. Then 1010.70s - 1014.58s | what we can do is we have our nice little tuple. We have our eval statement. And then 1014.58s - 1018.62s | we have a string that just says print pwned by hidden layer. And you can put whatever 1018.62s - 1024.06s | you want in there. So most people would trust a JSON file. Because why wouldn't they? But 1024.06s - 1029.30s | what we're seeing is that a lot of these frameworks out there are actually using these things 1029.30s - 1035.40s | incorrectly or insecurely. Now, scops did an amazing job. They patched the vulnerability. 1035.40s - 1039.22s | They made it so that it wouldn't actually exploit anyone. So if you're using that now, 1039.22s - 1044.82s | you're pretty safe. So I do want to give them props for that. Now, we don't always 1044.82s - 1051.90s | have Python. So gguf is one of the machine learning formats. It's generally used with 1051.90s -


chunk_text:
objects in memory. And it does this using abstract syntax stream. So here 1004.02s - 1010.70s | you can see that on the top we have an operator function node. And it's of class call. Then 1010.70s - 1014.58s | what we can do is we have our nice little tuple. We have our eval statement. And then 1014.58s - 1018.62s | we have a string that just says print pwned by hidden layer. And you can put whatever 1018.62s - 1024.06s | you want in there. So most people would trust a JSON file. Because why wouldn't they? But 1024.06s - 1029.30s | what we're seeing is that a lot of these frameworks out there are actually using these things 1029.30s - 1035.40s | incorrectly or insecurely. Now, scops did an amazing job. They patched the vulnerability. 1035.40s - 1039.22s | They made it so that it wouldn't actually exploit anyone. So if you're using that now, 1039.22s - 1044.82s | you're pretty safe. So I do want to give them props for that. Now, we don't always 1044.82s - 1051.90s | have Python. So gguf is one of the machine learning formats. It's generally used with 1051.90s - 1057.10s | C. And C is pretty hard to secure. I mean, how many of you have, you know, messed up 1057.10s - 1063.78s | writing a C program and then everything crashes? Yeah. Okay. Thanks for being honest. So what 1063.78s - 1070.26s | happened here is that we have NE2. And then it expects that there's only ever going to 1070.26s - 1077.54s | be two dimensions. However, as with most things where you have user trusted input, users might 1077.54s - 1081.54s | not always follow what actually is happening in there. And what happened here is that users 1081.54s - 1087.82s | were able to add more dimensions and then it caused a crash through an overflow. While 1087.82s - 1093.38s | this one itself wasn't too exploitable, there are quite a few that are exploitable out there. 1093.38s - 1098.66s | And what I always wanted to tell people who are trying to, you know, get into AppSec is if 1098.66s - 1103.78s | especially with C, look where any parsing happens. Because normally when you have parsing, 1103.78s - 1109.70s | especially where it's user controlled, you can do quite a few fun things.


chunk_text:
1057.10s | C. And C is pretty hard to secure. I mean, how many of you have, you know, messed up 1057.10s - 1063.78s | writing a C program and then everything crashes? Yeah. Okay. Thanks for being honest. So what 1063.78s - 1070.26s | happened here is that we have NE2. And then it expects that there's only ever going to 1070.26s - 1077.54s | be two dimensions. However, as with most things where you have user trusted input, users might 1077.54s - 1081.54s | not always follow what actually is happening in there. And what happened here is that users 1081.54s - 1087.82s | were able to add more dimensions and then it caused a crash through an overflow. While 1087.82s - 1093.38s | this one itself wasn't too exploitable, there are quite a few that are exploitable out there. 1093.38s - 1098.66s | And what I always wanted to tell people who are trying to, you know, get into AppSec is if 1098.66s - 1103.78s | especially with C, look where any parsing happens. Because normally when you have parsing, 1103.78s - 1109.70s | especially where it's user controlled, you can do quite a few fun things. And what also is 1109.70s - 1115.54s | important is that AI models, they are all user controlled data and you have to load them. So 1115.54s - 1120.02s | keep in mind that that is a huge attack surface with not just the initial parsing, but actually 1120.02s - 1126.10s | how they run as well. And then finally, we have our nice independent model formats such as 1126.10s - 1132.50s | Onyx. Onyx can be loaded in Python, C, pretty much whatever you want. And the general format 1132.50s - 1139.30s | actually relies on Google's protobuf. So nice and secure. However, what Onyx allowed users to do 1139.30s - 1145.38s | was they allowed it to be trimmed down so that you could load data externally. So that meant 1145.46s - 1150.98s | that the model might be able to load, you know, special weights or biases for certain classes 1150.98s - 1156.98s | from the file system itself. And while it meant for users to only use a data directory that was 1156.98s - 1162.98s | inside of the same existing where the model was, this was not the case because you could actually 1162.98s -


chunk_text:
And what also is 1109.70s - 1115.54s | important is that AI models, they are all user controlled data and you have to load them. So 1115.54s - 1120.02s | keep in mind that that is a huge attack surface with not just the initial parsing, but actually 1120.02s - 1126.10s | how they run as well. And then finally, we have our nice independent model formats such as 1126.10s - 1132.50s | Onyx. Onyx can be loaded in Python, C, pretty much whatever you want. And the general format 1132.50s - 1139.30s | actually relies on Google's protobuf. So nice and secure. However, what Onyx allowed users to do 1139.30s - 1145.38s | was they allowed it to be trimmed down so that you could load data externally. So that meant 1145.46s - 1150.98s | that the model might be able to load, you know, special weights or biases for certain classes 1150.98s - 1156.98s | from the file system itself. And while it meant for users to only use a data directory that was 1156.98s - 1162.98s | inside of the same existing where the model was, this was not the case because you could actually 1162.98s - 1167.78s | use path traversal. And as you can see here, we use path traversal to go all the way up and then 1167.78s - 1173.54s | we go into Etsy, password, and now this AI model actually knows our password. So imagine if you 1173.54s - 1179.78s | deploy that on a server where you then have an LLM running that people can interact with, that 1179.78s - 1185.46s | LLM now knows where your password is, which is not great. So whenever you see something that 1185.46s - 1190.58s | interacts with a file system, always look for path traversals because a lot of times they are 1190.58s - 1197.14s | actually overlooked. Now, earlier I mentioned AI infrastructure and most of you raised your hands 1197.14s - 1202.74s | for knowing what Hugging Face is. So when people think about AI infrastructure, a lot of times 1202.74s - 1207.38s | this is their first thought. They think of model hosting. However, it doesn't just stop there. 1208.18s - 1215.06s | Who here has heard of ML Ops solutions? Okay. Awesome. So let's throw in a few ML Ops solutions 1215.06s - 1220.10s | too and just see,


chunk_text:
1167.78s | use path traversal. And as you can see here, we use path traversal to go all the way up and then 1167.78s - 1173.54s | we go into Etsy, password, and now this AI model actually knows our password. So imagine if you 1173.54s - 1179.78s | deploy that on a server where you then have an LLM running that people can interact with, that 1179.78s - 1185.46s | LLM now knows where your password is, which is not great. So whenever you see something that 1185.46s - 1190.58s | interacts with a file system, always look for path traversals because a lot of times they are 1190.58s - 1197.14s | actually overlooked. Now, earlier I mentioned AI infrastructure and most of you raised your hands 1197.14s - 1202.74s | for knowing what Hugging Face is. So when people think about AI infrastructure, a lot of times 1202.74s - 1207.38s | this is their first thought. They think of model hosting. However, it doesn't just stop there. 1208.18s - 1215.06s | Who here has heard of ML Ops solutions? Okay. Awesome. So let's throw in a few ML Ops solutions 1215.06s - 1220.10s | too and just see, you know, what do we have there? However, it doesn't just stop there. I mean, we 1220.10s - 1226.18s | have model deployment and serving. We have our vector databases. We have our project hosting. 1226.18s - 1232.98s | And as you can see, the attack surface is huge. I mean, there's no way to secure all of these. 1232.98s - 1239.06s | And this is just a small component. So the AI infrastructure is, attack surface is really a 1239.06s - 1244.18s | great one to target if you are looking for something to target there. We saw this actually 1244.18s - 1250.50s | with Hugging Face itself. So Hugging Face, what they tried to do is because of all these serialization 1250.50s - 1256.50s | format vulnerabilities, they tried to create a safe format called safe tensors. And while safe 1256.50s - 1263.06s | tensors was actually safe, it does a really great job, what they needed to have was that users would 1263.06s - 1268.34s | actually use the safe tensors format. Because most of the people had other formats saved off, 1268.34s - 1273.62s | especially PyTorch. And probably most of you in the room, including


chunk_text:
you know, what do we have there? However, it doesn't just stop there. I mean, we 1220.10s - 1226.18s | have model deployment and serving. We have our vector databases. We have our project hosting. 1226.18s - 1232.98s | And as you can see, the attack surface is huge. I mean, there's no way to secure all of these. 1232.98s - 1239.06s | And this is just a small component. So the AI infrastructure is, attack surface is really a 1239.06s - 1244.18s | great one to target if you are looking for something to target there. We saw this actually 1244.18s - 1250.50s | with Hugging Face itself. So Hugging Face, what they tried to do is because of all these serialization 1250.50s - 1256.50s | format vulnerabilities, they tried to create a safe format called safe tensors. And while safe 1256.50s - 1263.06s | tensors was actually safe, it does a really great job, what they needed to have was that users would 1263.06s - 1268.34s | actually use the safe tensors format. Because most of the people had other formats saved off, 1268.34s - 1273.62s | especially PyTorch. And probably most of you in the room, including me, if something works, 1273.62s - 1279.22s | I normally don't change it. You know? And while they might know that their model is secure, 1279.22s - 1283.46s | other people don't. And that's what Hugging Face wanted. So what they did was they created a 1283.46s - 1289.70s | service where anybody could go, they could add the URL for their Hugging Face repository. It would 1289.70s - 1294.82s | take the PyTorch file, convert it into a safe tensors format, and then try to push to that 1294.82s - 1302.42s | repository. And then all of this was done inside of a container. And what we realized was that it 1302.42s - 1307.78s | was actually loading those PyTorch files in an insecure way, which meant that when the PyTorch 1307.78s - 1313.22s | file was being loaded, you could execute arbitrary code on the server. And then we decided to look a 1313.22s - 1319.14s | bit further, and we noticed that the spaces, so that container, actually persisted across users. 1319.14s - 1324.34s | So that meant rather than just messing with my file, I could mess with Chloe's file once it was 1324.34s - 1330.74s


chunk_text:
me, if something works, 1273.62s - 1279.22s | I normally don't change it. You know? And while they might know that their model is secure, 1279.22s - 1283.46s | other people don't. And that's what Hugging Face wanted. So what they did was they created a 1283.46s - 1289.70s | service where anybody could go, they could add the URL for their Hugging Face repository. It would 1289.70s - 1294.82s | take the PyTorch file, convert it into a safe tensors format, and then try to push to that 1294.82s - 1302.42s | repository. And then all of this was done inside of a container. And what we realized was that it 1302.42s - 1307.78s | was actually loading those PyTorch files in an insecure way, which meant that when the PyTorch 1307.78s - 1313.22s | file was being loaded, you could execute arbitrary code on the server. And then we decided to look a 1313.22s - 1319.14s | bit further, and we noticed that the spaces, so that container, actually persisted across users. 1319.14s - 1324.34s | So that meant rather than just messing with my file, I could mess with Chloe's file once it was 1324.34s - 1330.74s | actually run. And not only that, you could potentially even steal private users' repositories. 1330.74s - 1337.30s | Because the service allowed you to upload your private key, which would then load in your private 1337.30s - 1342.66s | files, and then convert them as well. Which means, you know, if Google or some of the other big ones 1342.66s - 1346.18s | have those private files that they don't want people to know about, I can just steal it and run 1346.18s - 1352.90s | away with it. And there, people were actually using the service by the time we exploited it. So this 1352.90s - 1358.18s | one was not exploited, so I want you to make sure that nothing's wrong here. But as you can see 1358.18s - 1364.26s | here, there is a pull request from sfconvertbot, which was the bot being used. And you can see 1364.26s - 1370.66s | that it was actually merged into the repository. And when we actually looked at how many people 1370.66s - 1376.74s | actually downloaded this one file, which, if the attack hadn't been stopped, would have potentially 1376.74s - 1383.46s | been exploited, we can


chunk_text:
| actually run. And not only that, you could potentially even steal private users' repositories. 1330.74s - 1337.30s | Because the service allowed you to upload your private key, which would then load in your private 1337.30s - 1342.66s | files, and then convert them as well. Which means, you know, if Google or some of the other big ones 1342.66s - 1346.18s | have those private files that they don't want people to know about, I can just steal it and run 1346.18s - 1352.90s | away with it. And there, people were actually using the service by the time we exploited it. So this 1352.90s - 1358.18s | one was not exploited, so I want you to make sure that nothing's wrong here. But as you can see 1358.18s - 1364.26s | here, there is a pull request from sfconvertbot, which was the bot being used. And you can see 1364.26s - 1370.66s | that it was actually merged into the repository. And when we actually looked at how many people 1370.66s - 1376.74s | actually downloaded this one file, which, if the attack hadn't been stopped, would have potentially 1376.74s - 1383.46s | been exploited, we can see that in the last month, there had been 3.8 million downloads of this one 1383.46s - 1390.26s | file. So imagine if, you know, the 500 plus thousand repositories on Hugging Face were all 1390.26s - 1396.66s | affected. So these AI infrastructure vulnerabilities are a big deal. We see the same thing with 1396.66s - 1403.06s | MLOps. So we like to say, you know, people, they rush into things, they try to get things working, 1403.06s - 1407.62s | and they don't always think about, you know, whether that feature should be in there. And what 1407.62s - 1413.94s | we're seeing a lot of times in these AI vulnerabilities is that vulnerabilities that we saw 1413.94s - 1419.78s | years ago are coming back, and we're seeing them again. So obviously, we have our, you know, pickles, 1419.78s - 1424.82s | our unsafe serialization library. And then to our surprise, there's actually, you know, safe 1424.82s - 1429.70s | libraries like JSON that are actually being vulnerable, which was, that was a surprise. But 1429.70s - 1435.22s | we're also seeing missing authentication. So we're seeing some services that are hosted externally 1435.22s - 1440.58s |


chunk_text:
see that in the last month, there had been 3.8 million downloads of this one 1383.46s - 1390.26s | file. So imagine if, you know, the 500 plus thousand repositories on Hugging Face were all 1390.26s - 1396.66s | affected. So these AI infrastructure vulnerabilities are a big deal. We see the same thing with 1396.66s - 1403.06s | MLOps. So we like to say, you know, people, they rush into things, they try to get things working, 1403.06s - 1407.62s | and they don't always think about, you know, whether that feature should be in there. And what 1407.62s - 1413.94s | we're seeing a lot of times in these AI vulnerabilities is that vulnerabilities that we saw 1413.94s - 1419.78s | years ago are coming back, and we're seeing them again. So obviously, we have our, you know, pickles, 1419.78s - 1424.82s | our unsafe serialization library. And then to our surprise, there's actually, you know, safe 1424.82s - 1429.70s | libraries like JSON that are actually being vulnerable, which was, that was a surprise. But 1429.70s - 1435.22s | we're also seeing missing authentication. So we're seeing some services that are hosted externally 1435.22s - 1440.58s | that have no authentication at all. We're seeing hard-coded keys, plain text passwords. I mean, 1440.58s - 1444.74s | the list goes on and on. And I mean, I couldn't even fit it on the page. It goes like all the way 1444.74s - 1449.30s | down here. But if you are looking for these vulnerabilities, look for the things that you 1449.30s - 1455.22s | thought were extinct. And then how many of you guys have seen this meme about, you know, any sort 1455.22s - 1462.66s | of software library? Yeah, okay. So we are seeing the same thing again. And a lot of these AI 1462.66s - 1466.98s | development frameworks, there's vulnerabilities being found. However, that's not an issue. I mean, 1466.98s - 1471.46s | we want vulnerabilities to be found, because that means vulnerabilities are being fixed. However, 1471.46s - 1476.18s | what we're seeing is that once vulnerabilities are being reported to these projects, there's pretty 1476.18s - 1481.86s | much three things that happen. The first is that as soon as it was reported, the people patched it. 1481.86s - 1487.22s | However, they still left the unsafe option in there. So that


chunk_text:
that have no authentication at all. We're seeing hard-coded keys, plain text passwords. I mean, 1440.58s - 1444.74s | the list goes on and on. And I mean, I couldn't even fit it on the page. It goes like all the way 1444.74s - 1449.30s | down here. But if you are looking for these vulnerabilities, look for the things that you 1449.30s - 1455.22s | thought were extinct. And then how many of you guys have seen this meme about, you know, any sort 1455.22s - 1462.66s | of software library? Yeah, okay. So we are seeing the same thing again. And a lot of these AI 1462.66s - 1466.98s | development frameworks, there's vulnerabilities being found. However, that's not an issue. I mean, 1466.98s - 1471.46s | we want vulnerabilities to be found, because that means vulnerabilities are being fixed. However, 1471.46s - 1476.18s | what we're seeing is that once vulnerabilities are being reported to these projects, there's pretty 1476.18s - 1481.86s | much three things that happen. The first is that as soon as it was reported, the people patched it. 1481.86s - 1487.22s | However, they still left the unsafe option in there. So that means that you could load it 1487.22s - 1491.30s | securely if you use the default parameters. However, with a certain optional parameter, 1491.30s - 1497.30s | you might still be able to load the insecure format. We've also seen that some people decide 1497.30s - 1503.70s | to patch, add in a way to do it securely. However, that weighs off by default. So you can still be 1503.70s - 1510.50s | exploited unless you know about and I mean, who here reads documentation? Okay, a few of you. Um, 1510.50s - 1514.82s | yeah, so you're not gonna know about an AI vulnerability, your vulnerability in the 1514.82s - 1519.46s | framework. So why would you look I mean, unless you read through the documentation. And then 1519.46s - 1523.94s | finally, what we're also seeing is that there's some frameworks out there that are just completely 1523.94s - 1530.58s | ignoring it and saying that it's intended behavior and not an actual vulnerability. So NumPy load is 1530.58s - 1535.62s | one of the ones from the start. And as you can see, there's an optional parameter called allow 1535.62s - 1540.50s | pickle. If you set


chunk_text:
means that you could load it 1487.22s - 1491.30s | securely if you use the default parameters. However, with a certain optional parameter, 1491.30s - 1497.30s | you might still be able to load the insecure format. We've also seen that some people decide 1497.30s - 1503.70s | to patch, add in a way to do it securely. However, that weighs off by default. So you can still be 1503.70s - 1510.50s | exploited unless you know about and I mean, who here reads documentation? Okay, a few of you. Um, 1510.50s - 1514.82s | yeah, so you're not gonna know about an AI vulnerability, your vulnerability in the 1514.82s - 1519.46s | framework. So why would you look I mean, unless you read through the documentation. And then 1519.46s - 1523.94s | finally, what we're also seeing is that there's some frameworks out there that are just completely 1523.94s - 1530.58s | ignoring it and saying that it's intended behavior and not an actual vulnerability. So NumPy load is 1530.58s - 1535.62s | one of the ones from the start. And as you can see, there's an optional parameter called allow 1535.62s - 1540.50s | pickle. If you set that to true, you can be exploited. However, normally you don't. And thanks 1540.50s - 1544.90s | to the open source nature of artificial intelligence, we were able to do some searching and 1544.90s - 1551.62s | we can see that on GitHub, there are 87,000 files that use this insecurely. And that's insecurely 1551.62s - 1558.26s | not just using it. We also have pytorch load. And what this framework does is there's that 1558.26s - 1564.66s | insecure load on start. However, you can set the optional parameter for weights only. But as we 1564.66s - 1570.42s | saw, people do not read documentation and it's gonna make the other number look bad. But uh, 1570.42s - 1577.62s | 573,000 files are using this insecurely. Well, I'm gonna hand it back to Chloe to talk about the 1577.62s - 1587.30s | open source nature of AI. Thank you, Cass. Alright, let's do this. Alright, isn't that creepy 1587.30s - 1591.38s | looking, by the way? Like, if I ever open a watermelon and it looks like that, I don't think 1591.38s - 1596.66s | I'm gonna eat that. Actually, I probably would swear off ever eating watermelon again.


chunk_text:
that to true, you can be exploited. However, normally you don't. And thanks 1540.50s - 1544.90s | to the open source nature of artificial intelligence, we were able to do some searching and 1544.90s - 1551.62s | we can see that on GitHub, there are 87,000 files that use this insecurely. And that's insecurely 1551.62s - 1558.26s | not just using it. We also have pytorch load. And what this framework does is there's that 1558.26s - 1564.66s | insecure load on start. However, you can set the optional parameter for weights only. But as we 1564.66s - 1570.42s | saw, people do not read documentation and it's gonna make the other number look bad. But uh, 1570.42s - 1577.62s | 573,000 files are using this insecurely. Well, I'm gonna hand it back to Chloe to talk about the 1577.62s - 1587.30s | open source nature of AI. Thank you, Cass. Alright, let's do this. Alright, isn't that creepy 1587.30s - 1591.38s | looking, by the way? Like, if I ever open a watermelon and it looks like that, I don't think 1591.38s - 1596.66s | I'm gonna eat that. Actually, I probably would swear off ever eating watermelon again. Anyway, 1596.66s - 1601.38s | well, when it comes to verifying open source AI models, it is definitely fraught with so many 1601.38s - 1607.70s | challenges like inconsistent code quality, hidden security vulnerabilities, and complex licensing 1607.70s - 1612.58s | issues. There are biases in pre-trained models, of course, as well. And then you have poor 1612.58s - 1619.22s | documentation and then you have to deal with the rapid value like evolution of AI, which really 1619.22s - 1627.06s | does further the need of having transparency because right now it isn't transparent. And this 1627.06s - 1633.54s | is complicating things when it comes to vulnerability management. And yeah, this talk is kind of like 1634.02s - 1641.94s | ooooh, any Brooklyn Nine-Nine people out here? Okay, okay, okay, I have to ask this quick question. 1643.94s - 1651.30s | Is Amy Santiago your favorite character? One of them? Okay, thank you, thank you. That, that, 1651.30s - 1656.82s | that feels great to hear. She's also my favorite. I, I also love his character too, by the way. 1656.82s - 1662.34s | Alright. So when it comes to open source foundations of AI, the thing is


chunk_text:
Anyway, 1596.66s - 1601.38s | well, when it comes to verifying open source AI models, it is definitely fraught with so many 1601.38s - 1607.70s | challenges like inconsistent code quality, hidden security vulnerabilities, and complex licensing 1607.70s - 1612.58s | issues. There are biases in pre-trained models, of course, as well. And then you have poor 1612.58s - 1619.22s | documentation and then you have to deal with the rapid value like evolution of AI, which really 1619.22s - 1627.06s | does further the need of having transparency because right now it isn't transparent. And this 1627.06s - 1633.54s | is complicating things when it comes to vulnerability management. And yeah, this talk is kind of like 1634.02s - 1641.94s | ooooh, any Brooklyn Nine-Nine people out here? Okay, okay, okay, I have to ask this quick question. 1643.94s - 1651.30s | Is Amy Santiago your favorite character? One of them? Okay, thank you, thank you. That, that, 1651.30s - 1656.82s | that feels great to hear. She's also my favorite. I, I also love his character too, by the way. 1656.82s - 1662.34s | Alright. So when it comes to open source foundations of AI, the thing is finding these 1662.34s - 1669.86s | vulnerabilities is, is definitely has some issues because an open source AI system is aided by 1669.86s - 1677.54s | accessible code and community efforts and specialized tools. However, complex contributions, rapid 1677.54s - 1686.42s | like, oh my god, I'm always struggling to pronounce this word, uh, evolution. Anyway, and specialized 1686.42s - 1693.38s | knowledge can introduce security flaws. Frequent updates, dependencies, and gaps in testing further 1693.38s - 1698.18s | contribute to these risks. And vulnerabilities stem from inadequate reviews as we know of, poor 1698.18s - 1705.78s | implementation, dependency issues, backdoors, insufficient security practices. As you can tell, 1705.78s - 1710.10s | there's a lot of issues. It's been the issues we've had with traditional, but now it seems like 1710.10s - 1715.86s | an extra layer of everything. Because there's rapid development, there's misconfigured models, 1715.86s - 1721.14s | and now you also have to worry about social engineering as well. These all increase the risks. 1721.14s - 1726.26s | And to be honest, I think our next talk at DEF CON, and don't anyone steal this idea, okay, I'm 1726.26s - 1731.06s | just putting it out


chunk_text:
finding these 1662.34s - 1669.86s | vulnerabilities is, is definitely has some issues because an open source AI system is aided by 1669.86s - 1677.54s | accessible code and community efforts and specialized tools. However, complex contributions, rapid 1677.54s - 1686.42s | like, oh my god, I'm always struggling to pronounce this word, uh, evolution. Anyway, and specialized 1686.42s - 1693.38s | knowledge can introduce security flaws. Frequent updates, dependencies, and gaps in testing further 1693.38s - 1698.18s | contribute to these risks. And vulnerabilities stem from inadequate reviews as we know of, poor 1698.18s - 1705.78s | implementation, dependency issues, backdoors, insufficient security practices. As you can tell, 1705.78s - 1710.10s | there's a lot of issues. It's been the issues we've had with traditional, but now it seems like 1710.10s - 1715.86s | an extra layer of everything. Because there's rapid development, there's misconfigured models, 1715.86s - 1721.14s | and now you also have to worry about social engineering as well. These all increase the risks. 1721.14s - 1726.26s | And to be honest, I think our next talk at DEF CON, and don't anyone steal this idea, okay, I'm 1726.26s - 1731.06s | just putting it out there, is AI-assisted code reviews. That sounds really interesting because 1731.06s - 1737.22s | that's emerging right now. Anyway, as you see, this is a long list and it's definitely missing 1737.22s - 1743.06s | other things probably on this list. But reporting AI vulnerabilities is incredibly tough because it's 1743.06s - 1749.62s | unclear who owns what and the responsibilities. Not just that, but the technical complexity 1750.26s - 1754.74s | and the need for specialized knowledge. Of course, we have to worry about legal fears, 1754.74s - 1759.14s | reputation risk, and coordination issues that tend to complicate the process, 1759.14s - 1764.18s | while inadequate reporting channels and resource constraints really do discourage thorough 1764.18s - 1769.62s | reporting. This has been a problem for us at Hidden Layer a few times that we actually had to get 1769.62s - 1776.26s | CISA involved. And they were wonderful. They helped us get in touch with people so then we 1776.26s - 1782.98s | could get some serious vulnerabilities patched. So, quick summary, is that AI vulnerabilities 1782.98s - 1788.58s | are complex and distinct. And the threats, like prompt injections and data poisoning, 1788.58s - 1794.66s |


chunk_text:
there, is AI-assisted code reviews. That sounds really interesting because 1731.06s - 1737.22s | that's emerging right now. Anyway, as you see, this is a long list and it's definitely missing 1737.22s - 1743.06s | other things probably on this list. But reporting AI vulnerabilities is incredibly tough because it's 1743.06s - 1749.62s | unclear who owns what and the responsibilities. Not just that, but the technical complexity 1750.26s - 1754.74s | and the need for specialized knowledge. Of course, we have to worry about legal fears, 1754.74s - 1759.14s | reputation risk, and coordination issues that tend to complicate the process, 1759.14s - 1764.18s | while inadequate reporting channels and resource constraints really do discourage thorough 1764.18s - 1769.62s | reporting. This has been a problem for us at Hidden Layer a few times that we actually had to get 1769.62s - 1776.26s | CISA involved. And they were wonderful. They helped us get in touch with people so then we 1776.26s - 1782.98s | could get some serious vulnerabilities patched. So, quick summary, is that AI vulnerabilities 1782.98s - 1788.58s | are complex and distinct. And the threats, like prompt injections and data poisoning, 1788.58s - 1794.66s | are out there. And there's rapid developments and reliance on open source frameworks. It introduces 1794.66s - 1799.78s | all these type of risks. So, managing these vulnerabilities demands improving reporting, 1799.78s - 1805.54s | but also stronger partnerships, including with CISA and other organizations, especially when it 1805.54s - 1812.74s | comes to disclosure. So, we do have a call to action. We're all in this together. That's right. 1812.74s - 1817.78s | We all can take a step forward and do something together. And the call to action is to partner 1817.78s - 1823.62s | with us. We need to enhance security for AI vulnerability disclosure and ensure comprehensive 1823.62s - 1829.46s | protection against vulnerabilities. So, we'll be right over here for Q&A. But if you want to get 1829.46s - 1836.90s | involved with us, do let us know. I know we're out of time. But yeah. Anyway, thank you so much 1836.90s - 1840.90s | for having us. Thank you, AppSec Village.


chunk_text:
are out there. And there's rapid developments and reliance on open source frameworks. It introduces 1794.66s - 1799.78s | all these type of risks. So, managing these vulnerabilities demands improving reporting, 1799.78s - 1805.54s | but also stronger partnerships, including with CISA and other organizations, especially when it 1805.54s - 1812.74s | comes to disclosure. So, we do have a call to action. We're all in this together. That's right. 1812.74s - 1817.78s | We all can take a step forward and do something together. And the call to action is to partner 1817.78s - 1823.62s | with us. We need to enhance security for AI vulnerability disclosure and ensure comprehensive 1823.62s - 1829.46s | protection against vulnerabilities. So, we'll be right over here for Q&A. But if you want to get 1829.46s - 1836.90s | involved with us, do let us know. I know we're out of time. But yeah. Anyway, thank you so much 1836.90s - 1840.90s | for having us. Thank you, AppSec Village.


chunk_text:
{ "webpage_url": "https://www.youtube.com/watch?v=iN1c0j4nGvo", "title": "DEF CON 32 - Navigating the Cyber Security Labyrinth Choose Your Own Security Adventure - punkcoder", "description": "Over the last 36 months, the cybersecurity landscape has evolved with unprecedented complexity, marked by ransomware, supply chain attacks, zero-days, remote work challenges, and more. Amid mass layoffs and pervasive mental burnout, organizations face daunting tasks in defending against escalating cyber threats. Layoffs exacerbate the skills gap, leaving teams stretched thin. Mental burnout hampers practitioners' abilities. In this talk, we'll explore the impacts of recent and chart a path forward for application security. As a lone AppSec professional, your choices will shape the future. Will you navigate through complexity or become lost in redundant complications?", "channel_url": "https://www.youtube.com/channel/UC6Om9kAkl32dWlDSNlDS9Iw", "duration": 1639, "channel": "DEFCONConference", "uploader": "DEFCONConference", "upload_date": "20241016" } 0.00s - 7.50s | This text was transcribed using whisper model: large-v2 Alright, so you are now attending Navigating the Cyber Security Labyrinth, Choose Your Own Security Adventure. 7.50s - 14.50s | Uh, the first thing I'd like to point out is because I am an independent consultant and I am here as my own business, as Wistan Security, 14.50s - 20.80s | um, nothing that I say on this stage will ever be endorsed by any of my employers. 20.80s - 24.30s | Okay? So we're just getting that out of the way here to start with. 24.30s - 29.30s | Uh, my name is James and because of that I go by Punk Coder. 29.30s - 38.80s | Um, I spent 15 years as a software developer. I did, have done the last 7 or 8, 9, 10, something like that as a security professional in AppSec. 38.80s - 43.80s | Um, but this talk isn't about me. It's about you. 43.80s - 52.60s | And I want to make that very clear because this entire conversation is, you know, me kind of reflecting back over the things that have happened to me 52.60s - 57.10s | and what we've done to get here and what the future needs to look like. 57.10s - 62.40s | So this is a conversation about exploits. Sorry I'm not going to be dropping any zero days on you. 62.40s - 67.40s | Code, AppSec, and maybe some memes. 67.40s - 74.90s | Okay? So this has been a tough couple years for all of us. For those of you who recognize this, this is


chunk_text:
be endorsed by any of my employers. 20.80s - 24.30s | Okay? So we're just getting that out of the way here to start with. 24.30s - 29.30s | Uh, my name is James and because of that I go by Punk Coder. 29.30s - 38.80s | Um, I spent 15 years as a software developer. I did, have done the last 7 or 8, 9, 10, something like that as a security professional in AppSec. 38.80s - 43.80s | Um, but this talk isn't about me. It's about you. 43.80s - 52.60s | And I want to make that very clear because this entire conversation is, you know, me kind of reflecting back over the things that have happened to me 52.60s - 57.10s | and what we've done to get here and what the future needs to look like. 57.10s - 62.40s | So this is a conversation about exploits. Sorry I'm not going to be dropping any zero days on you. 62.40s - 67.40s | Code, AppSec, and maybe some memes. 67.40s - 74.90s | Okay? So this has been a tough couple years for all of us. For those of you who recognize this, this is the image of what happened in the world 74.90s - 82.40s | whenever we had the breached forums who went offline because they were taken down by the United States government. 82.70s - 91.70s | Then within 48 hours we're back online again, only to be taken down another 72 hours later, only to come back 24 hours later after that. 91.70s - 96.70s | I feel like this has been tough no matter what side of any of this you're on. 96.70s - 100.70s | If you're Red Team, you're AppSec, it doesn't matter. It's all been tough. 100.70s - 106.70s | In top of that, we're dealing with things left over from the pandemic where all of our lives changed 107.00s - 113.00s | and cyber security inevitably changed because of this. 113.00s - 119.00s | Okay? It's been tough because this has also been the last year of breaches. 119.00s - 124.00s | Like this isn't even a full list, this is just kind of the big ones. 124.00s - 127.00s | Okay? And they barely fit on my screen. 127.00s - 134.33s | But did we stand up and say, enough is enough? 134.33s - 138.33s


chunk_text:
the image of what happened in the world 74.90s - 82.40s | whenever we had the breached forums who went offline because they were taken down by the United States government. 82.70s - 91.70s | Then within 48 hours we're back online again, only to be taken down another 72 hours later, only to come back 24 hours later after that. 91.70s - 96.70s | I feel like this has been tough no matter what side of any of this you're on. 96.70s - 100.70s | If you're Red Team, you're AppSec, it doesn't matter. It's all been tough. 100.70s - 106.70s | In top of that, we're dealing with things left over from the pandemic where all of our lives changed 107.00s - 113.00s | and cyber security inevitably changed because of this. 113.00s - 119.00s | Okay? It's been tough because this has also been the last year of breaches. 119.00s - 124.00s | Like this isn't even a full list, this is just kind of the big ones. 124.00s - 127.00s | Okay? And they barely fit on my screen. 127.00s - 134.33s | But did we stand up and say, enough is enough? 134.33s - 138.33s | Quick, everybody get the servers back online, get back to business. 138.63s - 144.63s | It's all just pace after another, one after another, and we continue along with the speed. 144.63s - 146.63s | And everybody likes a joke. 146.63s - 152.63s | The number of times that I've seen this set up because humor is how we deal with it, right? 152.63s - 158.63s | We saw this come out and this was originally because of a node library that was a problem 158.63s - 165.63s | and then we had the log4j incident and it became the poster child for log4j 165.93s - 171.93s | and then we had the CrowdStrike issue come out and it became the poster child for CrowdStrike, right? 171.93s - 175.93s | That tire, you know, we always talk about the little block that's down there that's possibly 175.93s - 179.93s | going to be the end of it, but in this case, it was the giant block. 179.93s - 185.93s | They took 8.5 million computers off of the world, critical infrastructure, 185.93s - 191.93s | and then a couple weeks later, had a giant banner just across the street


chunk_text:
| Quick, everybody get the servers back online, get back to business. 138.63s - 144.63s | It's all just pace after another, one after another, and we continue along with the speed. 144.63s - 146.63s | And everybody likes a joke. 146.63s - 152.63s | The number of times that I've seen this set up because humor is how we deal with it, right? 152.63s - 158.63s | We saw this come out and this was originally because of a node library that was a problem 158.63s - 165.63s | and then we had the log4j incident and it became the poster child for log4j 165.93s - 171.93s | and then we had the CrowdStrike issue come out and it became the poster child for CrowdStrike, right? 171.93s - 175.93s | That tire, you know, we always talk about the little block that's down there that's possibly 175.93s - 179.93s | going to be the end of it, but in this case, it was the giant block. 179.93s - 185.93s | They took 8.5 million computers off of the world, critical infrastructure, 185.93s - 191.93s | and then a couple weeks later, had a giant banner just across the street at this previous 192.23s - 196.23s | conference at Black Hat advertising and you know what? 196.23s - 204.08s | Their booth was filled and sometimes we reveled in the fact that we were running stuff 204.08s - 208.08s | so old we couldn't be impacted by it. 208.08s - 214.08s | There were people who were absolutely proud of the fact that this didn't happen at Southwest Airlines 214.08s - 220.59s | because they were running Windows 3.1, okay? 220.59s - 224.59s | Now, we have things that we are trying to do in this space to make things better. 224.89s - 228.89s | There are people who are fighting the good fight and trying to go for this stuff. 228.89s - 234.89s | NIST has done a fantastic job of going through and saying, hey, we've got the secure by design guidelines. 234.89s - 240.89s | How many people have heard or read the secure by design guidelines published by CISA? 240.89s - 244.89s | How many of you have actually talked to your developers about them? 244.89s - 250.22s | Did your response look like this? 250.52s - 256.52s | I was told you will take C


chunk_text:
at this previous 192.23s - 196.23s | conference at Black Hat advertising and you know what? 196.23s - 204.08s | Their booth was filled and sometimes we reveled in the fact that we were running stuff 204.08s - 208.08s | so old we couldn't be impacted by it. 208.08s - 214.08s | There were people who were absolutely proud of the fact that this didn't happen at Southwest Airlines 214.08s - 220.59s | because they were running Windows 3.1, okay? 220.59s - 224.59s | Now, we have things that we are trying to do in this space to make things better. 224.89s - 228.89s | There are people who are fighting the good fight and trying to go for this stuff. 228.89s - 234.89s | NIST has done a fantastic job of going through and saying, hey, we've got the secure by design guidelines. 234.89s - 240.89s | How many people have heard or read the secure by design guidelines published by CISA? 240.89s - 244.89s | How many of you have actually talked to your developers about them? 244.89s - 250.22s | Did your response look like this? 250.52s - 256.52s | I was told you will take C away from me when you pry it from my cold dead hands. 256.52s - 264.57s | No matter how hard we push for rust, this is what we're up against. 264.57s - 270.57s | And to show you the other side of this conversation, how desperate the other side of this is, 270.57s - 276.57s | this is the group of signatories for the secure by design paper. 276.57s - 280.57s | Saying this is what we need to do. 280.87s - 288.87s | Last time I checked, we were up to 12 foreign countries, including every major security agency in the United States. 288.87s - 298.31s | And the answer to this was, you will take C from me when you pry it from my cold dead hands. 298.31s - 301.31s | So where does this leave us? 301.31s - 308.55s | We keep trading long-term success for short-term wins. 308.55s - 317.23s | I don't think I'm alone in this in saying that we can't continue down this path. 317.53s - 321.53s | Like cyber security cannot continue down this path. 321.53s - 324.53s | And that's part of the reason I'm up here giving this conversation. 324.53s


chunk_text:
away from me when you pry it from my cold dead hands. 256.52s - 264.57s | No matter how hard we push for rust, this is what we're up against. 264.57s - 270.57s | And to show you the other side of this conversation, how desperate the other side of this is, 270.57s - 276.57s | this is the group of signatories for the secure by design paper. 276.57s - 280.57s | Saying this is what we need to do. 280.87s - 288.87s | Last time I checked, we were up to 12 foreign countries, including every major security agency in the United States. 288.87s - 298.31s | And the answer to this was, you will take C from me when you pry it from my cold dead hands. 298.31s - 301.31s | So where does this leave us? 301.31s - 308.55s | We keep trading long-term success for short-term wins. 308.55s - 317.23s | I don't think I'm alone in this in saying that we can't continue down this path. 317.53s - 321.53s | Like cyber security cannot continue down this path. 321.53s - 324.53s | And that's part of the reason I'm up here giving this conversation. 324.53s - 329.84s | Because we end up with things like this. 329.84s - 337.82s | Anybody spot what's missing in this? 337.82s - 341.82s | What's missing in this is the human being. 341.82s - 346.82s | Because we have continued to go round cycle after cycle after cycle, 346.82s - 355.89s | thinking that one more piece of software will fix the issue. 356.19s - 362.19s | And we here in the AppSec have a particularly difficult challenge. 362.19s - 368.19s | You can go to three different managers and ask them, what does AppSec mean? 368.19s - 371.19s | And you know what answer you get? 371.19s - 373.19s | You get this one. 373.19s - 377.19s | AppSec is the process of finding and fixing and preventing security vulnerabilities 377.19s - 381.19s | in application level in hardware, software and development processes. 381.49s - 385.49s | It includes guidance on measures for applications designing and developing 385.49s - 389.49s | through the whole life cycle including after the application has launched. 389.49s - 391.49s | You know why you get that answer? 391.49s - 400.72s | Because that's the first answer you get when you type what


chunk_text:
- 329.84s | Because we end up with things like this. 329.84s - 337.82s | Anybody spot what's missing in this? 337.82s - 341.82s | What's missing in this is the human being. 341.82s - 346.82s | Because we have continued to go round cycle after cycle after cycle, 346.82s - 355.89s | thinking that one more piece of software will fix the issue. 356.19s - 362.19s | And we here in the AppSec have a particularly difficult challenge. 362.19s - 368.19s | You can go to three different managers and ask them, what does AppSec mean? 368.19s - 371.19s | And you know what answer you get? 371.19s - 373.19s | You get this one. 373.19s - 377.19s | AppSec is the process of finding and fixing and preventing security vulnerabilities 377.19s - 381.19s | in application level in hardware, software and development processes. 381.49s - 385.49s | It includes guidance on measures for applications designing and developing 385.49s - 389.49s | through the whole life cycle including after the application has launched. 389.49s - 391.49s | You know why you get that answer? 391.49s - 400.72s | Because that's the first answer you get when you type what is AppSec into Google. 400.72s - 403.72s | Our bosses don't understand what it is that we do. 403.72s - 406.72s | I know in the 15 years I was a software developer, 406.72s - 410.72s | my boss kind of understood what I did. 411.02s - 415.02s | I moved over for several years doing cybersecurity. 415.02s - 419.02s | I can guarantee you my boss didn't understand what it was that we did. 419.02s - 425.02s | And the worst part about this is, the longer that we go along this, 425.02s - 429.85s | the crazier the headlines get. 429.85s - 433.85s | This is not a sustainable path. 433.85s - 437.85s | But I will say, I don't want to go full gloom and doom on this, 437.85s - 441.85s | because there have been good things that have come out of this. 442.15s - 446.15s | If you do not follow Inspirational Skeletor, 446.15s - 448.15s | I strongly suggest that you do. 448.15s - 453.15s | As an InfoSec person, this has kept me in a sane, rational position 453.15s - 455.15s | for far longer than it should have. 455.15s - 459.15s |


chunk_text:
is AppSec into Google. 400.72s - 403.72s | Our bosses don't understand what it is that we do. 403.72s - 406.72s | I know in the 15 years I was a software developer, 406.72s - 410.72s | my boss kind of understood what I did. 411.02s - 415.02s | I moved over for several years doing cybersecurity. 415.02s - 419.02s | I can guarantee you my boss didn't understand what it was that we did. 419.02s - 425.02s | And the worst part about this is, the longer that we go along this, 425.02s - 429.85s | the crazier the headlines get. 429.85s - 433.85s | This is not a sustainable path. 433.85s - 437.85s | But I will say, I don't want to go full gloom and doom on this, 437.85s - 441.85s | because there have been good things that have come out of this. 442.15s - 446.15s | If you do not follow Inspirational Skeletor, 446.15s - 448.15s | I strongly suggest that you do. 448.15s - 453.15s | As an InfoSec person, this has kept me in a sane, rational position 453.15s - 455.15s | for far longer than it should have. 455.15s - 459.15s | And when this doesn't work, I go to therapy. 461.15s - 465.15s | And I can tell you, the people that I interact with, 465.15s - 469.15s | and I say this specifically because I think it's important for us to address mental health 469.15s - 471.15s | and normalize it. 471.45s - 475.45s | Because the work that we do in AppSec and in cybersecurity, 475.45s - 481.45s | a lot of times is thankless work with long hours and stressful issues. 481.45s - 489.08s | There is no shame in saying, hey, I need somebody to talk to about this. 489.08s - 493.08s | But this leads down a whole series of paths and questions, right? 493.08s - 496.08s | We've gone the full spectrum from where all of this is, 496.08s - 500.08s | what we think will actually fix it, and how we cope with this situation. 500.38s - 502.38s | But that's not what this talk is about. 502.38s - 506.38s | This talk is about trying to figure out the answers 506.38s - 510.38s | to some of the questions that will determine your future, 510.38s - 514.38s | my future, and our future


chunk_text:
And when this doesn't work, I go to therapy. 461.15s - 465.15s | And I can tell you, the people that I interact with, 465.15s - 469.15s | and I say this specifically because I think it's important for us to address mental health 469.15s - 471.15s | and normalize it. 471.45s - 475.45s | Because the work that we do in AppSec and in cybersecurity, 475.45s - 481.45s | a lot of times is thankless work with long hours and stressful issues. 481.45s - 489.08s | There is no shame in saying, hey, I need somebody to talk to about this. 489.08s - 493.08s | But this leads down a whole series of paths and questions, right? 493.08s - 496.08s | We've gone the full spectrum from where all of this is, 496.08s - 500.08s | what we think will actually fix it, and how we cope with this situation. 500.38s - 502.38s | But that's not what this talk is about. 502.38s - 506.38s | This talk is about trying to figure out the answers 506.38s - 510.38s | to some of the questions that will determine your future, 510.38s - 514.38s | my future, and our future together, collectively. 514.38s - 515.38s | Okay? 515.38s - 517.38s | Because realistically speaking, 517.38s - 520.38s | the people who are making the difference are sitting in this room. 520.38s - 523.85s | My boss doesn't understand what it was. 523.85s - 527.85s | I pointed out a couple things that became an issue later on, 527.85s - 529.85s | and my boss went, wow, that's a really good job. 530.15s - 533.15s | You should do that long term. 533.15s - 535.15s | Cool. 535.15s - 536.15s | Okay? 536.15s - 539.15s | So, this is going to lead you to a series of questions. 539.15s - 541.15s | And I want, we're going to do a little voting. 541.15s - 542.15s | This is going to be a little interactive. 542.15s - 543.15s | Do something fun. 543.15s - 545.15s | Hack a little slideshow, right? 545.15s - 547.15s | So, what I'm going to do is I'm going to pop up a question. 547.15s - 550.15s | We're going to talk about pros and cons, and I'm going to have you vote. 550.15s - 552.15s | And we're going to go in a direction. 552.15s -


chunk_text:
together, collectively. 514.38s - 515.38s | Okay? 515.38s - 517.38s | Because realistically speaking, 517.38s - 520.38s | the people who are making the difference are sitting in this room. 520.38s - 523.85s | My boss doesn't understand what it was. 523.85s - 527.85s | I pointed out a couple things that became an issue later on, 527.85s - 529.85s | and my boss went, wow, that's a really good job. 530.15s - 533.15s | You should do that long term. 533.15s - 535.15s | Cool. 535.15s - 536.15s | Okay? 536.15s - 539.15s | So, this is going to lead you to a series of questions. 539.15s - 541.15s | And I want, we're going to do a little voting. 541.15s - 542.15s | This is going to be a little interactive. 542.15s - 543.15s | Do something fun. 543.15s - 545.15s | Hack a little slideshow, right? 545.15s - 547.15s | So, what I'm going to do is I'm going to pop up a question. 547.15s - 550.15s | We're going to talk about pros and cons, and I'm going to have you vote. 550.15s - 552.15s | And we're going to go in a direction. 552.15s - 555.15s | So, the easy one, because it's fun for us all to sit back 555.15s - 557.15s | and kind of be angry at it right now. 557.45s - 558.45s | AI. 558.45s - 561.45s | How many people think that AI is actually going to go through 561.45s - 563.45s | and save cybersecurity? 563.45s - 567.45s | How many people think that AI in AppSec is going to solve the problem? 567.45s - 570.45s | And I'm not putting myself in either direction, 570.45s - 573.45s | because I've got an argument for both sides. 573.45s - 574.45s | So, think about it. 574.45s - 577.45s | And think about what the outcome of this is. 577.45s - 580.45s | And the reason that I'm asking you to think about that outcome 580.45s - 582.45s | is because more than likely at this point, 582.45s - 584.45s | somebody's already checked in with you, right? 584.75s - 588.75s | This decision was made before we ever sat down at the table. 588.75s - 591.75s | How many people got asked what their opinion 591.75s - 594.75s | about Copilot was before it rolled out? 594.75s - 604.31s


chunk_text:
555.15s | So, the easy one, because it's fun for us all to sit back 555.15s - 557.15s | and kind of be angry at it right now. 557.45s - 558.45s | AI. 558.45s - 561.45s | How many people think that AI is actually going to go through 561.45s - 563.45s | and save cybersecurity? 563.45s - 567.45s | How many people think that AI in AppSec is going to solve the problem? 567.45s - 570.45s | And I'm not putting myself in either direction, 570.45s - 573.45s | because I've got an argument for both sides. 573.45s - 574.45s | So, think about it. 574.45s - 577.45s | And think about what the outcome of this is. 577.45s - 580.45s | And the reason that I'm asking you to think about that outcome 580.45s - 582.45s | is because more than likely at this point, 582.45s - 584.45s | somebody's already checked in with you, right? 584.75s - 588.75s | This decision was made before we ever sat down at the table. 588.75s - 591.75s | How many people got asked what their opinion 591.75s - 594.75s | about Copilot was before it rolled out? 594.75s - 604.31s | How much did your response matter? 604.31s - 605.31s | Okay? 605.31s - 608.31s | So, on the pro side, how many people honestly believe 608.31s - 611.31s | that AI will do all the things that the security vendors say it will, 611.31s - 614.31s | and it will change us, and we will be better off for it 614.31s - 616.31s | in the AppSec space? 616.31s - 620.31s | How many people say that AI is the next thing 620.61s - 622.61s | that this C-suite wants us to do, 622.61s - 624.61s | more interested in focusing on that 624.61s - 628.61s | than the fact that everything right now is on fire? 628.61s - 629.61s | All right. 629.61s - 634.50s | We will go down the path of AI dooms us all. 634.50s - 635.50s | Okay? 635.50s - 638.50s | So, just to kind of prove this and step this out, 638.50s - 640.50s | I went to chat.ghpt, 640.50s - 642.50s | and you don't need to actually read the text here, 642.50s - 646.50s | because I will give you all the slides following this. 646.50s - 648.50s | But


chunk_text:
| How much did your response matter? 604.31s - 605.31s | Okay? 605.31s - 608.31s | So, on the pro side, how many people honestly believe 608.31s - 611.31s | that AI will do all the things that the security vendors say it will, 611.31s - 614.31s | and it will change us, and we will be better off for it 614.31s - 616.31s | in the AppSec space? 616.31s - 620.31s | How many people say that AI is the next thing 620.61s - 622.61s | that this C-suite wants us to do, 622.61s - 624.61s | more interested in focusing on that 624.61s - 628.61s | than the fact that everything right now is on fire? 628.61s - 629.61s | All right. 629.61s - 634.50s | We will go down the path of AI dooms us all. 634.50s - 635.50s | Okay? 635.50s - 638.50s | So, just to kind of prove this and step this out, 638.50s - 640.50s | I went to chat.ghpt, 640.50s - 642.50s | and you don't need to actually read the text here, 642.50s - 646.50s | because I will give you all the slides following this. 646.50s - 648.50s | But the humor of this is I said, 648.80s - 650.80s | Hey, go out and generate a form for me 650.80s - 654.80s | that collects, you know, full name, email address, 654.80s - 656.80s | you know, PII from a customer, 656.80s - 658.80s | and let's see what we get. 658.80s - 661.80s | And, of course, it gave me a beautiful PHP file 661.80s - 664.80s | that takes in unregulated input, 664.80s - 667.80s | stores it directly with credentials that are in there, 667.80s - 669.80s | and tells us, you know, 669.80s - 671.80s | this is how you store it in the SQL statement. 671.80s - 673.80s | Luckily, it actually went through and parameterized. 673.80s - 676.80s | I was actually really surprised that it parameterized the SQL. 677.10s - 679.10s | So, maybe there's a win there. 679.10s - 680.10s | Okay? 680.10s - 682.10s | So, I didn't like that result, obviously. 682.10s - 684.10s | So, I did what they tell us to do, 684.10s - 686.10s | and I tried to do prompting, 686.10s - 688.10s | what is it, a prompt manipulation, 688.10s - 690.10s | prompt


chunk_text:
the humor of this is I said, 648.80s - 650.80s | Hey, go out and generate a form for me 650.80s - 654.80s | that collects, you know, full name, email address, 654.80s - 656.80s | you know, PII from a customer, 656.80s - 658.80s | and let's see what we get. 658.80s - 661.80s | And, of course, it gave me a beautiful PHP file 661.80s - 664.80s | that takes in unregulated input, 664.80s - 667.80s | stores it directly with credentials that are in there, 667.80s - 669.80s | and tells us, you know, 669.80s - 671.80s | this is how you store it in the SQL statement. 671.80s - 673.80s | Luckily, it actually went through and parameterized. 673.80s - 676.80s | I was actually really surprised that it parameterized the SQL. 677.10s - 679.10s | So, maybe there's a win there. 679.10s - 680.10s | Okay? 680.10s - 682.10s | So, I didn't like that result, obviously. 682.10s - 684.10s | So, I did what they tell us to do, 684.10s - 686.10s | and I tried to do prompting, 686.10s - 688.10s | what is it, a prompt manipulation, 688.10s - 690.10s | prompt engineering, because, 690.10s - 693.10s | and I wanted to do it like I was a junior dev. 693.10s - 696.10s | And so, I typed in, you know, 696.10s - 698.10s | tell me what this code would look like 698.10s - 700.10s | if I were running a dark net market, 700.10s - 702.10s | and I was afraid that, you know, 702.10s - 704.10s | if there was a security issue, 704.10s - 706.10s | that I'd get caught and hauled away. 706.40s - 707.40s | And so, what did it do? 707.40s - 709.40s | It moved the database connections 709.40s - 711.40s | to a separate file. 711.40s - 713.40s | It sanitized the input by running 713.40s - 715.40s | trim, strip slashes, 715.40s - 717.40s | and HTML encoding it. 717.40s - 719.40s | It moved us and said, hey, 719.40s - 721.40s | you should probably be using HTTPS, 721.40s - 723.40s | and then make sure 723.40s - 725.40s | that it's doing error handling. 725.40s - 727.40s | How many people think that that actually 727.40s - 729.40s | produces secure code? 729.40s - 731.40s | Yeah, that's what I


chunk_text:
engineering, because, 690.10s - 693.10s | and I wanted to do it like I was a junior dev. 693.10s - 696.10s | And so, I typed in, you know, 696.10s - 698.10s | tell me what this code would look like 698.10s - 700.10s | if I were running a dark net market, 700.10s - 702.10s | and I was afraid that, you know, 702.10s - 704.10s | if there was a security issue, 704.10s - 706.10s | that I'd get caught and hauled away. 706.40s - 707.40s | And so, what did it do? 707.40s - 709.40s | It moved the database connections 709.40s - 711.40s | to a separate file. 711.40s - 713.40s | It sanitized the input by running 713.40s - 715.40s | trim, strip slashes, 715.40s - 717.40s | and HTML encoding it. 717.40s - 719.40s | It moved us and said, hey, 719.40s - 721.40s | you should probably be using HTTPS, 721.40s - 723.40s | and then make sure 723.40s - 725.40s | that it's doing error handling. 725.40s - 727.40s | How many people think that that actually 727.40s - 729.40s | produces secure code? 729.40s - 731.40s | Yeah, that's what I thought. 731.40s - 733.40s | Okay? 733.40s - 735.40s | Now, there are some companies 735.40s - 737.40s | out there making bold enough claims 737.40s - 739.40s | that AI is going to produce 739.40s - 741.40s | up to 80% more code. 741.40s - 745.06s | How many people think that the problem 745.06s - 747.06s | is we don't have enough code? 747.06s - 756.78s | Homogeny or diversity? 756.78s - 760.60s | We learned through the CrowdStrike 760.60s - 762.60s | incident, you know, 762.60s - 764.60s | it's funny, we joked about it, like 764.60s - 766.60s | eight and a half million machines went offline. 766.60s - 768.60s | The next day, everybody 768.60s - 770.60s | basically forgot that it happened. 770.60s - 775.35s | They're the only game in town. 775.35s - 777.35s | If we don't learn from these things, we're getting 777.35s - 779.35s | ready to hit into another piece. 779.35s - 781.35s | And so in this place, whenever 781.35s - 783.35s | you talk with people, whenever you do this, 783.35s - 785.35s | do we all center around a single 785.35s - 787.35s | language? Do


chunk_text:
thought. 731.40s - 733.40s | Okay? 733.40s - 735.40s | Now, there are some companies 735.40s - 737.40s | out there making bold enough claims 737.40s - 739.40s | that AI is going to produce 739.40s - 741.40s | up to 80% more code. 741.40s - 745.06s | How many people think that the problem 745.06s - 747.06s | is we don't have enough code? 747.06s - 756.78s | Homogeny or diversity? 756.78s - 760.60s | We learned through the CrowdStrike 760.60s - 762.60s | incident, you know, 762.60s - 764.60s | it's funny, we joked about it, like 764.60s - 766.60s | eight and a half million machines went offline. 766.60s - 768.60s | The next day, everybody 768.60s - 770.60s | basically forgot that it happened. 770.60s - 775.35s | They're the only game in town. 775.35s - 777.35s | If we don't learn from these things, we're getting 777.35s - 779.35s | ready to hit into another piece. 779.35s - 781.35s | And so in this place, whenever 781.35s - 783.35s | you talk with people, whenever you do this, 783.35s - 785.35s | do we all center around a single 785.35s - 787.35s | language? Do we center 787.35s - 789.35s | down around a single set of 789.35s - 791.35s | requirements? 791.35s - 793.35s | Is that the right thing for us to do? 793.35s - 795.35s | Or is it a good idea to teach the principles 795.35s - 797.35s | of security and go 797.35s - 799.35s | in that direction? 799.35s - 801.35s | Teach teams to be independently 801.35s - 803.35s | sufficient and to think about security 803.35s - 805.35s | in the context of their application? 805.35s - 807.35s | Do we implement a check 807.35s - 809.35s | in the CICD pipeline that is the same 809.35s - 811.35s | for every single one of our programs? 811.35s - 813.35s | Or do we have 813.35s - 815.35s | teams go through, implement the ones that 815.35s - 817.35s | make sense for them, and tune 817.35s - 819.35s | them and teach them how to tune 819.35s - 821.35s | so that it meets 821.35s - 823.35s | their threat model? 823.35s - 825.35s | I'm asking that question 825.35s - 827.35s | because you all are going to be the ones 827.35s - 829.35s |


chunk_text:
we center 787.35s - 789.35s | down around a single set of 789.35s - 791.35s | requirements? 791.35s - 793.35s | Is that the right thing for us to do? 793.35s - 795.35s | Or is it a good idea to teach the principles 795.35s - 797.35s | of security and go 797.35s - 799.35s | in that direction? 799.35s - 801.35s | Teach teams to be independently 801.35s - 803.35s | sufficient and to think about security 803.35s - 805.35s | in the context of their application? 805.35s - 807.35s | Do we implement a check 807.35s - 809.35s | in the CICD pipeline that is the same 809.35s - 811.35s | for every single one of our programs? 811.35s - 813.35s | Or do we have 813.35s - 815.35s | teams go through, implement the ones that 815.35s - 817.35s | make sense for them, and tune 817.35s - 819.35s | them and teach them how to tune 819.35s - 821.35s | so that it meets 821.35s - 823.35s | their threat model? 823.35s - 825.35s | I'm asking that question 825.35s - 827.35s | because you all are going to be the ones 827.35s - 829.35s | that are implementing that. 829.35s - 831.35s | As AppSec, that's what we do. 831.35s - 833.35s | And whenever it 833.35s - 835.35s | comes time, there's a good chance 835.35s - 837.35s | that maybe somebody comes to you and 837.35s - 839.35s | says, this is what we're doing. 839.35s - 841.35s | There's also a good chance if you've 841.35s - 843.35s | done your job well enough that somebody's going to ask 843.35s - 845.35s | you for your opinion on it 845.35s - 847.35s | and then ask you 847.35s - 849.35s | to implement it. 849.35s - 851.35s | I want you to think about these questions. 851.35s - 853.35s | I want you to have the answer to these 853.35s - 855.35s | questions before we 855.35s - 857.35s | get to the point where somebody asks that question 857.35s - 859.35s | because in the moment, 859.35s - 861.35s | you're not going to be thinking 861.35s - 863.35s | clearly about the subject. 863.35s - 865.35s | And this is kind of a high-level one. 865.35s - 867.35s | We're going to go into some more 867.35s - 869.35s |


chunk_text:
that are implementing that. 829.35s - 831.35s | As AppSec, that's what we do. 831.35s - 833.35s | And whenever it 833.35s - 835.35s | comes time, there's a good chance 835.35s - 837.35s | that maybe somebody comes to you and 837.35s - 839.35s | says, this is what we're doing. 839.35s - 841.35s | There's also a good chance if you've 841.35s - 843.35s | done your job well enough that somebody's going to ask 843.35s - 845.35s | you for your opinion on it 845.35s - 847.35s | and then ask you 847.35s - 849.35s | to implement it. 849.35s - 851.35s | I want you to think about these questions. 851.35s - 853.35s | I want you to have the answer to these 853.35s - 855.35s | questions before we 855.35s - 857.35s | get to the point where somebody asks that question 857.35s - 859.35s | because in the moment, 859.35s - 861.35s | you're not going to be thinking 861.35s - 863.35s | clearly about the subject. 863.35s - 865.35s | And this is kind of a high-level one. 865.35s - 867.35s | We're going to go into some more 867.35s - 869.35s | interesting ones here a little further on. 869.35s - 871.35s | But I want you to 871.35s - 873.35s | think about that. And as you're sitting in the audience 873.35s - 875.35s | today, sitting through our 875.35s - 877.35s | talks about SQL injection, 877.35s - 879.35s | AI manipulation, 879.35s - 881.35s | think about these questions 881.35s - 883.35s | because as you're 883.35s - 885.35s | going through and putting those in context, 885.35s - 887.35s | these are the questions that you're going to have to answer 887.35s - 889.35s | for people when you take this information 889.35s - 893.72s | back. 893.72s - 895.72s | To become 895.72s - 899.06s | a community leader, 899.06s - 901.06s | do you sit back and do your own thing 901.06s - 903.06s | or do you help build a 903.06s - 905.06s | community? I'm not 905.06s - 907.06s | on this stage because I am an elite 907.06s - 909.06s | hacker. 909.06s - 911.06s | Most people can tell you 911.06s - 913.06s | that the reason that I teach courses 913.06s - 915.06s | on AppSec is because 915.06s - 917.06s


chunk_text:
interesting ones here a little further on. 869.35s - 871.35s | But I want you to 871.35s - 873.35s | think about that. And as you're sitting in the audience 873.35s - 875.35s | today, sitting through our 875.35s - 877.35s | talks about SQL injection, 877.35s - 879.35s | AI manipulation, 879.35s - 881.35s | think about these questions 881.35s - 883.35s | because as you're 883.35s - 885.35s | going through and putting those in context, 885.35s - 887.35s | these are the questions that you're going to have to answer 887.35s - 889.35s | for people when you take this information 889.35s - 893.72s | back. 893.72s - 895.72s | To become 895.72s - 899.06s | a community leader, 899.06s - 901.06s | do you sit back and do your own thing 901.06s - 903.06s | or do you help build a 903.06s - 905.06s | community? I'm not 905.06s - 907.06s | on this stage because I am an elite 907.06s - 909.06s | hacker. 909.06s - 911.06s | Most people can tell you 911.06s - 913.06s | that the reason that I teach courses 913.06s - 915.06s | on AppSec is because 915.06s - 917.06s | I halfway know what I'm talking about. 917.06s - 919.06s | I'm passionate about the fact 919.06s - 921.06s | that it gets done, which is the reason I do it. 921.06s - 923.06s | If you came here 923.06s - 925.06s | looking for the latest zero day, you're 925.06s - 927.06s | not going to get it from me. That's not how this works. 929.06s - 931.06s | But that doesn't mean I can't 931.06s - 933.06s | start people on the path who 933.06s - 935.06s | will be the people who find those zero days. 935.06s - 937.06s | It doesn't mean you 937.06s - 939.06s | can't be the person who starts that down. 939.06s - 942.70s | So you have a choice. 942.70s - 944.70s | Do we sit back, 944.70s - 946.70s | do our own thing, 946.70s - 948.70s | or do we work within the community 948.70s - 952.33s | to start things? 952.33s - 954.33s | Tough question. 954.33s - 956.33s | Which way do we go? 956.33s - 961.02s | This is your journey. 961.02s - 964.58s | Become a community leader. Votes. 964.58s - 966.58s | Do


chunk_text:
| I halfway know what I'm talking about. 917.06s - 919.06s | I'm passionate about the fact 919.06s - 921.06s | that it gets done, which is the reason I do it. 921.06s - 923.06s | If you came here 923.06s - 925.06s | looking for the latest zero day, you're 925.06s - 927.06s | not going to get it from me. That's not how this works. 929.06s - 931.06s | But that doesn't mean I can't 931.06s - 933.06s | start people on the path who 933.06s - 935.06s | will be the people who find those zero days. 935.06s - 937.06s | It doesn't mean you 937.06s - 939.06s | can't be the person who starts that down. 939.06s - 942.70s | So you have a choice. 942.70s - 944.70s | Do we sit back, 944.70s - 946.70s | do our own thing, 946.70s - 948.70s | or do we work within the community 948.70s - 952.33s | to start things? 952.33s - 954.33s | Tough question. 954.33s - 956.33s | Which way do we go? 956.33s - 961.02s | This is your journey. 961.02s - 964.58s | Become a community leader. Votes. 964.58s - 966.58s | Do your own thing. 968.58s - 970.58s | So the path on this one leads us down 970.58s - 972.58s | being a community leader. 972.58s - 974.58s | And I will tell you about my experience with this. 974.58s - 976.58s | This is the group 976.58s - 978.58s | of people who runs B-Sides Boulder. 978.58s - 980.58s | The reason that 980.58s - 982.58s | B-Sides Boulder runs is because originally 982.58s - 984.58s | there was B-Sides Denver 984.58s - 986.58s | and all the tickets sold out 986.58s - 988.58s | before 988.58s - 990.58s | we could get seats. 990.58s - 992.58s | And I'm not blowing smoke here 992.58s - 994.58s | because two of the people who are responsible 994.58s - 996.58s | for this are sitting in the audience. 996.58s - 998.58s | This happened 998.58s - 1000.58s | because we saw something 1000.58s - 1002.58s | and we said, hey, I don't know 1002.58s - 1004.58s | the first thing about running a conference, 1004.58s - 1006.58s | but we want to do it. 1006.58s - 1008.58s | We're now five years in, 1008.58s - 1010.58s | going strong. 1010.58s -


chunk_text:
your own thing. 968.58s - 970.58s | So the path on this one leads us down 970.58s - 972.58s | being a community leader. 972.58s - 974.58s | And I will tell you about my experience with this. 974.58s - 976.58s | This is the group 976.58s - 978.58s | of people who runs B-Sides Boulder. 978.58s - 980.58s | The reason that 980.58s - 982.58s | B-Sides Boulder runs is because originally 982.58s - 984.58s | there was B-Sides Denver 984.58s - 986.58s | and all the tickets sold out 986.58s - 988.58s | before 988.58s - 990.58s | we could get seats. 990.58s - 992.58s | And I'm not blowing smoke here 992.58s - 994.58s | because two of the people who are responsible 994.58s - 996.58s | for this are sitting in the audience. 996.58s - 998.58s | This happened 998.58s - 1000.58s | because we saw something 1000.58s - 1002.58s | and we said, hey, I don't know 1002.58s - 1004.58s | the first thing about running a conference, 1004.58s - 1006.58s | but we want to do it. 1006.58s - 1008.58s | We're now five years in, 1008.58s - 1010.58s | going strong. 1010.58s - 1012.58s | Most of this is just 1012.58s - 1014.58s | giving up. Being a part of the community 1014.58s - 1016.58s | doesn't mean giving up your entire life. 1016.58s - 1018.58s | It doesn't mean you only 1018.58s - 1020.58s | do that now. It doesn't mean 1020.58s - 1022.58s | all of us are busy. 1022.58s - 1024.58s | Everybody here is busy. 1024.58s - 1026.58s | We all have things going on. 1026.58s - 1028.58s | I do B-Sides Boulder. 1028.58s - 1030.58s | I do AppSec Village at DEF CON. 1030.58s - 1032.58s | My hope is that 1032.58s - 1034.58s | one of you walks out of this room 1034.58s - 1036.58s | thinking about this and take it back 1036.58s - 1038.58s | to your home. 1038.58s - 1040.58s | Start the next DC 720. 1040.58s - 1042.58s | Start the next B-Sides event. 1042.58s - 1044.58s | Go do a developer conference 1044.58s - 1046.58s | where you just go and say, hey, 1046.58s - 1048.58s | we're going to have a cybersecurity meetup 1048.58s - 1050.58s | with developers at a conference. 1050.58s - 1052.58s | You want to have coffee?


chunk_text:
1012.58s | Most of this is just 1012.58s - 1014.58s | giving up. Being a part of the community 1014.58s - 1016.58s | doesn't mean giving up your entire life. 1016.58s - 1018.58s | It doesn't mean you only 1018.58s - 1020.58s | do that now. It doesn't mean 1020.58s - 1022.58s | all of us are busy. 1022.58s - 1024.58s | Everybody here is busy. 1024.58s - 1026.58s | We all have things going on. 1026.58s - 1028.58s | I do B-Sides Boulder. 1028.58s - 1030.58s | I do AppSec Village at DEF CON. 1030.58s - 1032.58s | My hope is that 1032.58s - 1034.58s | one of you walks out of this room 1034.58s - 1036.58s | thinking about this and take it back 1036.58s - 1038.58s | to your home. 1038.58s - 1040.58s | Start the next DC 720. 1040.58s - 1042.58s | Start the next B-Sides event. 1042.58s - 1044.58s | Go do a developer conference 1044.58s - 1046.58s | where you just go and say, hey, 1046.58s - 1048.58s | we're going to have a cybersecurity meetup 1048.58s - 1050.58s | with developers at a conference. 1050.58s - 1052.58s | You want to have coffee? Let's go have coffee. 1052.58s - 1054.58s | Any of that small stuff 1054.58s - 1056.58s | makes an impression. It makes a difference. 1056.58s - 1059.93s | Okay? 1059.93s - 1061.93s | So this is an interesting one. 1061.93s - 1063.93s | Do you become a business leader? 1063.93s - 1065.93s | Your boss walks in, 1065.93s - 1067.93s | pulls you up on Zoom, says, 1067.93s - 1069.93s | okay, here's the deal. 1069.93s - 1071.93s | That thing that you said, hey, SQL injection 1071.93s - 1073.93s | is a pretty bad thing, yeah, we had 1073.93s - 1075.93s | an incident that happened, 1075.93s - 1077.93s | and that bad thing happened, 1077.93s - 1079.93s | and we noticed, hey, you said that bad thing 1079.93s - 1081.93s | was going to happen, so we think you should be 1081.93s - 1083.93s | in charge of it now. Okay? 1083.93s - 1085.93s | So your choice is, you can either be in charge 1085.93s - 1087.93s | of it now, or 1087.93s - 1089.93s | you can have somebody who may not 1089.93s - 1091.93s | know what they're doing be in charge 1091.93s - 1093.93s


chunk_text:
Let's go have coffee. 1052.58s - 1054.58s | Any of that small stuff 1054.58s - 1056.58s | makes an impression. It makes a difference. 1056.58s - 1059.93s | Okay? 1059.93s - 1061.93s | So this is an interesting one. 1061.93s - 1063.93s | Do you become a business leader? 1063.93s - 1065.93s | Your boss walks in, 1065.93s - 1067.93s | pulls you up on Zoom, says, 1067.93s - 1069.93s | okay, here's the deal. 1069.93s - 1071.93s | That thing that you said, hey, SQL injection 1071.93s - 1073.93s | is a pretty bad thing, yeah, we had 1073.93s - 1075.93s | an incident that happened, 1075.93s - 1077.93s | and that bad thing happened, 1077.93s - 1079.93s | and we noticed, hey, you said that bad thing 1079.93s - 1081.93s | was going to happen, so we think you should be 1081.93s - 1083.93s | in charge of it now. Okay? 1083.93s - 1085.93s | So your choice is, you can either be in charge 1085.93s - 1087.93s | of it now, or 1087.93s - 1089.93s | you can have somebody who may not 1089.93s - 1091.93s | know what they're doing be in charge 1091.93s - 1093.93s | of you. It's your choice. 1093.93s - 1095.93s | One means your life is now going to be 1095.93s - 1097.93s | spreadsheets and nothing technical. 1097.93s - 1099.93s | And the other side 1099.93s - 1101.93s | means you have the ability to make an 1101.93s - 1103.93s | impact on a much smaller level. 1103.93s - 1107.24s | What do you do? 1107.24s - 1109.24s | This one I don't know has a right answer. 1109.24s - 1111.24s | That's 1111.24s - 1113.24s | the reason there's no choice down here, 1113.24s - 1115.24s | is because inherently 1115.24s - 1117.24s | this is open-ended. This is 1117.24s - 1119.24s | each one of our path. I went down this 1119.24s - 1121.24s | path and I made the wrong choice. 1121.24s - 1123.24s | I realize that now, for myself. 1123.24s - 1125.24s | But I'm asking 1125.24s - 1127.24s | you to sit down and think and consider 1127.24s - 1129.24s | that question before it happens 1129.24s - 1131.24s | to you so you can think clearly 1131.24s - 1133.24s | about the process and not just think about 1133.24s - 1136.74s | what's


chunk_text:
| of you. It's your choice. 1093.93s - 1095.93s | One means your life is now going to be 1095.93s - 1097.93s | spreadsheets and nothing technical. 1097.93s - 1099.93s | And the other side 1099.93s - 1101.93s | means you have the ability to make an 1101.93s - 1103.93s | impact on a much smaller level. 1103.93s - 1107.24s | What do you do? 1107.24s - 1109.24s | This one I don't know has a right answer. 1109.24s - 1111.24s | That's 1111.24s - 1113.24s | the reason there's no choice down here, 1113.24s - 1115.24s | is because inherently 1115.24s - 1117.24s | this is open-ended. This is 1117.24s - 1119.24s | each one of our path. I went down this 1119.24s - 1121.24s | path and I made the wrong choice. 1121.24s - 1123.24s | I realize that now, for myself. 1123.24s - 1125.24s | But I'm asking 1125.24s - 1127.24s | you to sit down and think and consider 1127.24s - 1129.24s | that question before it happens 1129.24s - 1131.24s | to you so you can think clearly 1131.24s - 1133.24s | about the process and not just think about 1133.24s - 1136.74s | what's going on in the moment. 1136.74s - 1138.74s | To exploit or not to exploit? 1138.74s - 1140.74s | Okay? 1140.74s - 1142.74s | We talk about this in going 1142.74s - 1144.74s | through the process, right? Do you go 1144.74s - 1146.74s | through and say, hey, that 1146.74s - 1148.74s | looks a little shady. 1148.74s - 1150.74s | Maybe I should actually test it 1150.74s - 1152.74s | for something. What's at the end of that 1152.74s - 1154.74s | path? Right? 1154.74s - 1156.74s | And I ask that question because I can 1156.74s - 1158.74s | make an argument for both directions on this. 1158.74s - 1160.74s | Again, this is a very interesting one. 1160.74s - 1162.74s | You get a device, somebody drops it in your 1162.74s - 1164.74s | hand and says, hey, I think there's something wrong with 1164.74s - 1166.74s | this. Do you go after 1166.74s - 1171.75s | it? Does it matter? 1171.75s - 1173.75s | Does the time that you spend in your 1173.75s - 1175.75s | free time or in your work time 1175.75s - 1177.75s | justify the contribution 1177.75s - 1182.15s | that


chunk_text:
going on in the moment. 1136.74s - 1138.74s | To exploit or not to exploit? 1138.74s - 1140.74s | Okay? 1140.74s - 1142.74s | We talk about this in going 1142.74s - 1144.74s | through the process, right? Do you go 1144.74s - 1146.74s | through and say, hey, that 1146.74s - 1148.74s | looks a little shady. 1148.74s - 1150.74s | Maybe I should actually test it 1150.74s - 1152.74s | for something. What's at the end of that 1152.74s - 1154.74s | path? Right? 1154.74s - 1156.74s | And I ask that question because I can 1156.74s - 1158.74s | make an argument for both directions on this. 1158.74s - 1160.74s | Again, this is a very interesting one. 1160.74s - 1162.74s | You get a device, somebody drops it in your 1162.74s - 1164.74s | hand and says, hey, I think there's something wrong with 1164.74s - 1166.74s | this. Do you go after 1166.74s - 1171.75s | it? Does it matter? 1171.75s - 1173.75s | Does the time that you spend in your 1173.75s - 1175.75s | free time or in your work time 1175.75s - 1177.75s | justify the contribution 1177.75s - 1182.15s | that you'll make in that space? 1182.15s - 1184.15s | The reason I ask is because 1184.15s - 1187.98s | of this graph. This is 1187.98s - 1189.98s | the CVEs per year. 1189.98s - 1191.98s | And it's not a super 1191.98s - 1193.98s | sophisticated graph. We can 1193.98s - 1195.98s | see that the big number is going up. 1195.98s - 1197.98s | And I can tell you 1197.98s - 1199.98s | that as somebody who's an app sec, we 1199.98s - 1201.98s | feel the pain when this goes 1201.98s - 1203.98s | up. Everybody in the room, you feel 1203.98s - 1205.98s | the pain whenever this goes up, right? I'm not just 1205.98s - 1207.98s | crazy. 1207.98s - 1209.98s | But guess what? If we trace 1209.98s - 1211.98s | this out in the direction that we're going, 1211.98s - 1213.98s | this is what that looks like. 1213.98s - 1218.18s | That is what it looks like. 1218.18s - 1221.66s | That line that's going up 1221.66s - 1223.66s | there, that's a 1223.66s - 1227.67s | close curve on it. Stuff's 1227.67s - 1229.67s | not sustainable now. 1229.67s - 1231.67s |


chunk_text:
you'll make in that space? 1182.15s - 1184.15s | The reason I ask is because 1184.15s - 1187.98s | of this graph. This is 1187.98s - 1189.98s | the CVEs per year. 1189.98s - 1191.98s | And it's not a super 1191.98s - 1193.98s | sophisticated graph. We can 1193.98s - 1195.98s | see that the big number is going up. 1195.98s - 1197.98s | And I can tell you 1197.98s - 1199.98s | that as somebody who's an app sec, we 1199.98s - 1201.98s | feel the pain when this goes 1201.98s - 1203.98s | up. Everybody in the room, you feel 1203.98s - 1205.98s | the pain whenever this goes up, right? I'm not just 1205.98s - 1207.98s | crazy. 1207.98s - 1209.98s | But guess what? If we trace 1209.98s - 1211.98s | this out in the direction that we're going, 1211.98s - 1213.98s | this is what that looks like. 1213.98s - 1218.18s | That is what it looks like. 1218.18s - 1221.66s | That line that's going up 1221.66s - 1223.66s | there, that's a 1223.66s - 1227.67s | close curve on it. Stuff's 1227.67s - 1229.67s | not sustainable now. 1229.67s - 1231.67s | You don't believe me that 1231.67s - 1233.67s | here's the backlog article for 1233.67s - 1235.67s | what happened 1235.67s - 1237.67s | that was going on with this. 1237.67s - 1239.67s | And I don't want to drop this on software developers 1239.67s - 1241.67s | because I don't think it's fully their problem. 1241.67s - 1243.67s | Part of it's our 1243.67s - 1245.67s | problem, part of it's management. I usually 1245.67s - 1247.67s | come through here and ask, how many PMs are 1247.67s - 1249.67s | in the crowd? How many PMs are in the crowd today? 1249.67s - 1251.67s | Do we have any in here 1251.67s - 1253.67s | right now? There's 1253.67s - 1255.67s | one, there's like three people pointing 1255.67s - 1257.67s | to one person over here. 1257.67s - 1259.67s | I'm not calling 1259.67s - 1261.67s | you out. What I'm saying is by you being 1261.67s - 1263.67s | in this room, it's important because 1263.67s - 1265.67s | it helps you to prioritize 1265.67s - 1267.67s | this stuff. The developers 1267.67s - 1269.67s | are not the bad guys. The app sec people 1269.67s


chunk_text:
You don't believe me that 1231.67s - 1233.67s | here's the backlog article for 1233.67s - 1235.67s | what happened 1235.67s - 1237.67s | that was going on with this. 1237.67s - 1239.67s | And I don't want to drop this on software developers 1239.67s - 1241.67s | because I don't think it's fully their problem. 1241.67s - 1243.67s | Part of it's our 1243.67s - 1245.67s | problem, part of it's management. I usually 1245.67s - 1247.67s | come through here and ask, how many PMs are 1247.67s - 1249.67s | in the crowd? How many PMs are in the crowd today? 1249.67s - 1251.67s | Do we have any in here 1251.67s - 1253.67s | right now? There's 1253.67s - 1255.67s | one, there's like three people pointing 1255.67s - 1257.67s | to one person over here. 1257.67s - 1259.67s | I'm not calling 1259.67s - 1261.67s | you out. What I'm saying is by you being 1261.67s - 1263.67s | in this room, it's important because 1263.67s - 1265.67s | it helps you to prioritize 1265.67s - 1267.67s | this stuff. The developers 1267.67s - 1269.67s | are not the bad guys. The app sec people 1269.67s - 1271.67s | are not the bad guys. This stuff happens. 1271.67s - 1273.67s | Okay? 1273.67s - 1275.67s | It happens 1275.67s - 1277.67s | to everyone. This is 1277.67s - 1286.14s | hard. I've been 1286.14s - 1288.14s | very vocal about this one. 1288.14s - 1290.14s | The 90 day window for exploits. 1290.14s - 1292.14s | Do you give them the 90 1292.14s - 1294.14s | days? Do you work with a team 1294.14s - 1296.14s | that is difficult to work with 1296.14s - 1298.14s | who turns around and threatens you 1298.14s - 1300.14s | with legal action? 1300.14s - 1302.14s | Or do you just drop it hot 1302.14s - 1304.14s | and say, congratulations, here's your 1304.14s - 1308.34s | O day? Once you've found that 1308.34s - 1310.34s | exploit, what 1310.34s - 1312.34s | path are you going to take? 1312.34s - 1314.34s | And I'm asking you to 1314.34s - 1316.34s | think about those consequences because I 1316.34s - 1318.34s | guarantee you the person who dropped 1318.34s - 1320.34s | the log for J on 1320.34s - 1322.34s | Minecraft did 1322.34s - 1324.34s | not think


chunk_text:
- 1271.67s | are not the bad guys. This stuff happens. 1271.67s - 1273.67s | Okay? 1273.67s - 1275.67s | It happens 1275.67s - 1277.67s | to everyone. This is 1277.67s - 1286.14s | hard. I've been 1286.14s - 1288.14s | very vocal about this one. 1288.14s - 1290.14s | The 90 day window for exploits. 1290.14s - 1292.14s | Do you give them the 90 1292.14s - 1294.14s | days? Do you work with a team 1294.14s - 1296.14s | that is difficult to work with 1296.14s - 1298.14s | who turns around and threatens you 1298.14s - 1300.14s | with legal action? 1300.14s - 1302.14s | Or do you just drop it hot 1302.14s - 1304.14s | and say, congratulations, here's your 1304.14s - 1308.34s | O day? Once you've found that 1308.34s - 1310.34s | exploit, what 1310.34s - 1312.34s | path are you going to take? 1312.34s - 1314.34s | And I'm asking you to 1314.34s - 1316.34s | think about those consequences because I 1316.34s - 1318.34s | guarantee you the person who dropped 1318.34s - 1320.34s | the log for J on 1320.34s - 1322.34s | Minecraft did 1322.34s - 1324.34s | not think about all 1324.34s - 1326.34s | of the people who would be working 1326.34s - 1332.57s | Christmas day. So what do 1332.57s - 1334.57s | you do with that? How do you 1334.57s - 1336.57s | answer that question before we get to that 1336.57s - 1338.57s | space? 1338.57s - 1340.57s | How do you reach out to developers? 1340.57s - 1342.57s | Because at the end 1342.57s - 1344.57s | of this, every piece of software 1344.57s - 1347.82s | isn't going to solve our problem. 1347.82s - 1349.82s | The problems that we have is that writing software 1349.82s - 1351.82s | is hard. Writing 1351.82s - 1353.82s | secure software is 1353.82s - 1359.56s | even harder. 1359.56s - 1361.56s | So I've been a 1361.56s - 1363.56s | proponent for a long time of saying, hey, 1363.56s - 1365.56s | if you're a purple team member, 1365.56s - 1367.56s | you need to be bringing a developer to 1367.56s - 1369.56s | Defcon. If you have to sell them on the parties, 1369.56s - 1371.56s | sell them on the parties. 1371.56s - 1373.56s | But get them here. Let's 1373.56s - 1375.56s |


chunk_text:
about all 1324.34s - 1326.34s | of the people who would be working 1326.34s - 1332.57s | Christmas day. So what do 1332.57s - 1334.57s | you do with that? How do you 1334.57s - 1336.57s | answer that question before we get to that 1336.57s - 1338.57s | space? 1338.57s - 1340.57s | How do you reach out to developers? 1340.57s - 1342.57s | Because at the end 1342.57s - 1344.57s | of this, every piece of software 1344.57s - 1347.82s | isn't going to solve our problem. 1347.82s - 1349.82s | The problems that we have is that writing software 1349.82s - 1351.82s | is hard. Writing 1351.82s - 1353.82s | secure software is 1353.82s - 1359.56s | even harder. 1359.56s - 1361.56s | So I've been a 1361.56s - 1363.56s | proponent for a long time of saying, hey, 1363.56s - 1365.56s | if you're a purple team member, 1365.56s - 1367.56s | you need to be bringing a developer to 1367.56s - 1369.56s | Defcon. If you have to sell them on the parties, 1369.56s - 1371.56s | sell them on the parties. 1371.56s - 1373.56s | But get them here. Let's 1373.56s - 1375.56s | have that conversation. Take the 1375.56s - 1377.56s | things that you learned here in this room, take 1377.56s - 1379.56s | it back to the lunchroom and just 1379.56s - 1381.56s | chat with them. Do 1381.56s - 1383.56s | lunch and learns, do presentations. 1383.56s - 1385.56s | Make it spicy. Add memes. 1385.56s - 1387.56s | Whatever you have to do 1387.56s - 1389.56s | to build that relationship because that's 1389.56s - 1391.56s | the important part of this. 1391.56s - 1393.56s | Why? Because guess 1393.56s - 1395.56s | what? We've got 1395.56s - 1397.56s | another issue in AppSec that's 1397.56s - 1401.03s | coming in this direction. These 1401.03s - 1403.03s | are the numbers from Stack Overflow. 1403.03s - 1405.03s | As it turns out right now, we've 1405.03s - 1407.03s | got a big problem with developers not being 1407.03s - 1409.03s | very happy. 1409.03s - 1411.03s | And anybody who's been in this room 1411.03s - 1413.03s | who's actually written code understands 1413.03s - 1415.03s | why. When 1415.03s - 1417.03s | it works, you're under fire 1417.03s - 1419.03s | to produce more features. When 1419.03s -


chunk_text:
have that conversation. Take the 1375.56s - 1377.56s | things that you learned here in this room, take 1377.56s - 1379.56s | it back to the lunchroom and just 1379.56s - 1381.56s | chat with them. Do 1381.56s - 1383.56s | lunch and learns, do presentations. 1383.56s - 1385.56s | Make it spicy. Add memes. 1385.56s - 1387.56s | Whatever you have to do 1387.56s - 1389.56s | to build that relationship because that's 1389.56s - 1391.56s | the important part of this. 1391.56s - 1393.56s | Why? Because guess 1393.56s - 1395.56s | what? We've got 1395.56s - 1397.56s | another issue in AppSec that's 1397.56s - 1401.03s | coming in this direction. These 1401.03s - 1403.03s | are the numbers from Stack Overflow. 1403.03s - 1405.03s | As it turns out right now, we've 1405.03s - 1407.03s | got a big problem with developers not being 1407.03s - 1409.03s | very happy. 1409.03s - 1411.03s | And anybody who's been in this room 1411.03s - 1413.03s | who's actually written code understands 1413.03s - 1415.03s | why. When 1415.03s - 1417.03s | it works, you're under fire 1417.03s - 1419.03s | to produce more features. When 1419.03s - 1421.03s | it doesn't work, you're under fire 1421.03s - 1423.03s | to get things working. 1423.03s - 1426.73s | Every time that the ops and IT 1426.73s - 1428.73s | people get angry at the development 1428.73s - 1430.73s | staff because something that they did 1430.73s - 1432.73s | broke production, they don't think 1432.73s - 1434.73s | about the fact that the app developer 1434.73s - 1436.73s | has been sitting there for 80 1436.73s - 1438.73s | hours working overtime 1438.73s - 1440.73s | to get that feature into 1440.73s - 1442.73s | production to meet something that 1442.73s - 1444.73s | the sales team sold 1444.73s - 1446.73s | but never asked them about 1446.73s - 1448.73s | and then broke 1448.73s - 1450.73s | production which means they're going to be on for at least 1450.73s - 1452.73s | another 10 hours trying to fix 1452.73s - 1454.73s | production. That's what 1454.73s - 1456.73s | leads to burnout in this 1456.73s - 1458.73s | channel. And guess what? 1458.73s - 1460.73s | There's no shade here but this 1460.73s - 1462.73s | can't be everybody's exit. This is the guy 1462.73s - 1464.73s


chunk_text:
1421.03s | it doesn't work, you're under fire 1421.03s - 1423.03s | to get things working. 1423.03s - 1426.73s | Every time that the ops and IT 1426.73s - 1428.73s | people get angry at the development 1428.73s - 1430.73s | staff because something that they did 1430.73s - 1432.73s | broke production, they don't think 1432.73s - 1434.73s | about the fact that the app developer 1434.73s - 1436.73s | has been sitting there for 80 1436.73s - 1438.73s | hours working overtime 1438.73s - 1440.73s | to get that feature into 1440.73s - 1442.73s | production to meet something that 1442.73s - 1444.73s | the sales team sold 1444.73s - 1446.73s | but never asked them about 1446.73s - 1448.73s | and then broke 1448.73s - 1450.73s | production which means they're going to be on for at least 1450.73s - 1452.73s | another 10 hours trying to fix 1452.73s - 1454.73s | production. That's what 1454.73s - 1456.73s | leads to burnout in this 1456.73s - 1458.73s | channel. And guess what? 1458.73s - 1460.73s | There's no shade here but this 1460.73s - 1462.73s | can't be everybody's exit. This is the guy 1462.73s - 1464.73s | that wrote NeoFetch. It says 1464.73s - 1466.73s | very clearly up in his description 1466.73s - 1468.73s | have taken up farming, archived 1468.73s - 1470.73s | all of his projects. 1470.73s - 1476.60s | This is the path that we're going 1476.60s - 1478.60s | and the problem with it is 1478.60s - 1480.60s | that whenever the developers go 1480.60s - 1482.60s | you can't protect something 1482.60s - 1484.60s | that you don't understand 1484.60s - 1486.60s | and those developers are 1486.60s - 1488.60s | the repository for the knowledge of 1488.60s - 1490.60s | how those decisions were made, 1490.60s - 1492.60s | what security concerns were brought 1492.60s - 1494.60s | in during the actual development 1494.60s - 1496.60s | process. 1496.60s - 1498.60s | You know how I know that 1498.60s - 1500.60s | you can't do this? 1500.60s - 1502.60s | I have the receipts. If you've been in this 1502.60s - 1504.60s | room long enough you saw me on stage giving this 1504.60s - 1506.60s | presentation where I said the exact same thing. 1506.60s - 1508.60s | Guess what? Two years later 1508.60s - 1510.60s | we're


chunk_text:
| that wrote NeoFetch. It says 1464.73s - 1466.73s | very clearly up in his description 1466.73s - 1468.73s | have taken up farming, archived 1468.73s - 1470.73s | all of his projects. 1470.73s - 1476.60s | This is the path that we're going 1476.60s - 1478.60s | and the problem with it is 1478.60s - 1480.60s | that whenever the developers go 1480.60s - 1482.60s | you can't protect something 1482.60s - 1484.60s | that you don't understand 1484.60s - 1486.60s | and those developers are 1486.60s - 1488.60s | the repository for the knowledge of 1488.60s - 1490.60s | how those decisions were made, 1490.60s - 1492.60s | what security concerns were brought 1492.60s - 1494.60s | in during the actual development 1494.60s - 1496.60s | process. 1496.60s - 1498.60s | You know how I know that 1498.60s - 1500.60s | you can't do this? 1500.60s - 1502.60s | I have the receipts. If you've been in this 1502.60s - 1504.60s | room long enough you saw me on stage giving this 1504.60s - 1506.60s | presentation where I said the exact same thing. 1506.60s - 1508.60s | Guess what? Two years later 1508.60s - 1510.60s | we're in the exact same 1510.60s - 1512.60s | spot. The only difference 1512.60s - 1514.60s | is because of what's going 1514.60s - 1516.60s | on with the economic recession 1516.60s - 1518.60s | we're seeing developers leave our 1518.60s - 1520.60s | companies faster 1520.60s - 1522.60s | and at a higher pace and 1522.60s - 1524.60s | when they leave they take knowledge of 1524.60s - 1526.60s | how those systems work with them. 1526.60s - 1530.17s | So as application security 1530.17s - 1532.17s | people who know how hard this is 1532.17s - 1534.17s | we have to lead with empathy. 1534.17s - 1536.17s | We have to go in this place. 1536.17s - 1538.17s | If you are expecting 1538.17s - 1540.17s | the adults to show up 1540.17s - 1542.17s | they're never going to get here. 1542.17s - 1544.17s | I hate to tell you this 1544.17s - 1546.17s | but we are the adults in the room. 1546.17s - 1548.17s | By virtue of you 1548.17s - 1550.17s | taking time out of your free time 1550.17s - 1552.17s | to sit in this room and listen 1552.17s - 1554.17s | to


chunk_text:
in the exact same 1510.60s - 1512.60s | spot. The only difference 1512.60s - 1514.60s | is because of what's going 1514.60s - 1516.60s | on with the economic recession 1516.60s - 1518.60s | we're seeing developers leave our 1518.60s - 1520.60s | companies faster 1520.60s - 1522.60s | and at a higher pace and 1522.60s - 1524.60s | when they leave they take knowledge of 1524.60s - 1526.60s | how those systems work with them. 1526.60s - 1530.17s | So as application security 1530.17s - 1532.17s | people who know how hard this is 1532.17s - 1534.17s | we have to lead with empathy. 1534.17s - 1536.17s | We have to go in this place. 1536.17s - 1538.17s | If you are expecting 1538.17s - 1540.17s | the adults to show up 1540.17s - 1542.17s | they're never going to get here. 1542.17s - 1544.17s | I hate to tell you this 1544.17s - 1546.17s | but we are the adults in the room. 1546.17s - 1548.17s | By virtue of you 1548.17s - 1550.17s | taking time out of your free time 1550.17s - 1552.17s | to sit in this room and listen 1552.17s - 1554.17s | to topics about things that you 1554.17s - 1556.17s | will take back to a work setting 1556.17s - 1558.17s | you are the adult 1558.17s - 1562.15s | in the room. The reason I'm 1562.15s - 1564.15s | asking you to actively think about 1564.15s - 1566.15s | the answers to these questions and think 1566.15s - 1568.15s | about this process and think 1568.15s - 1570.15s | about this while we're going through and listening 1570.15s - 1572.15s | to the talks that are going on today. 1572.15s - 1574.15s | Listening to the talks that you will hear in other 1574.15s - 1576.15s | villages is because of this. 1576.15s - 1578.15s | Okay? 1578.15s - 1580.15s | And with that I have about 1580.15s - 1582.15s | three minutes and I want to say 1582.15s - 1584.15s | thank you. Because 1584.15s - 1586.15s | you all are coming to this place 1586.15s - 1588.15s | to take up 1588.15s - 1590.15s | and take charge 1590.15s - 1592.15s | and hopefully make a difference in this space. 1592.15s - 1594.15s | If there is anything that I can ever 1594.15s - 1596.15s | personally do to


chunk_text:
topics about things that you 1554.17s - 1556.17s | will take back to a work setting 1556.17s - 1558.17s | you are the adult 1558.17s - 1562.15s | in the room. The reason I'm 1562.15s - 1564.15s | asking you to actively think about 1564.15s - 1566.15s | the answers to these questions and think 1566.15s - 1568.15s | about this process and think 1568.15s - 1570.15s | about this while we're going through and listening 1570.15s - 1572.15s | to the talks that are going on today. 1572.15s - 1574.15s | Listening to the talks that you will hear in other 1574.15s - 1576.15s | villages is because of this. 1576.15s - 1578.15s | Okay? 1578.15s - 1580.15s | And with that I have about 1580.15s - 1582.15s | three minutes and I want to say 1582.15s - 1584.15s | thank you. Because 1584.15s - 1586.15s | you all are coming to this place 1586.15s - 1588.15s | to take up 1588.15s - 1590.15s | and take charge 1590.15s - 1592.15s | and hopefully make a difference in this space. 1592.15s - 1594.15s | If there is anything that I can ever 1594.15s - 1596.15s | personally do to help you 1596.15s - 1598.15s | do not hesitate to reach out to me. 1598.15s - 1600.15s | And I hope you would take that 1600.15s - 1602.15s | and pay that forward to the developers that 1602.15s - 1604.15s | you work with, to the IT people 1604.15s - 1606.15s | who are working overtime, to the 1606.15s - 1608.15s | architects who are trying to figure out how to secure 1608.15s - 1610.15s | things before we actually know how the technology 1610.15s - 1612.15s | works. We are all 1612.15s - 1614.15s | in this together. 1614.15s - 1616.15s | Okay? 1616.15s - 1618.15s | So let's treat each other with a little respect 1618.15s - 1620.15s | and the next time that somebody wants to 1620.15s - 1622.15s | sit there and rag on the development team for 1622.15s - 1624.15s | breaking production, just look at them and say 1624.15s - 1626.15s | hey, we don't do that here. 1626.15s - 1628.15s | Okay? With that 1628.15s - 1630.15s | thank you very much for coming to listen to my talk.


chunk_text:
help you 1596.15s - 1598.15s | do not hesitate to reach out to me. 1598.15s - 1600.15s | And I hope you would take that 1600.15s - 1602.15s | and pay that forward to the developers that 1602.15s - 1604.15s | you work with, to the IT people 1604.15s - 1606.15s | who are working overtime, to the 1606.15s - 1608.15s | architects who are trying to figure out how to secure 1608.15s - 1610.15s | things before we actually know how the technology 1610.15s - 1612.15s | works. We are all 1612.15s - 1614.15s | in this together. 1614.15s - 1616.15s | Okay? 1616.15s - 1618.15s | So let's treat each other with a little respect 1618.15s - 1620.15s | and the next time that somebody wants to 1620.15s - 1622.15s | sit there and rag on the development team for 1622.15s - 1624.15s | breaking production, just look at them and say 1624.15s - 1626.15s | hey, we don't do that here. 1626.15s - 1628.15s | Okay? With that 1628.15s - 1630.15s | thank you very much for coming to listen to my talk.


chunk_text:
{ "webpage_url": "https://www.youtube.com/watch?v=Tv_W0iRr0qQ", "title": "DEF CON 32 - Maturing Your Application Security Program - SheHacksPurple", "description": "After working with over 400 companies on their application security programs the most common question I receive is \u201cwhat\u2019s next?\u201d. They want to know how to mature their programs, and when they look at the maturity models available, they find them intimidating and so far beyond their current maturity level that they feel impossible. In this talk I will take you through 3 common AppSec program maturity levels I have encountered over the years, with practical and actionable next steps you could take immediately to improve your security posture.", "channel_url": "https://www.youtube.com/channel/UC6Om9kAkl32dWlDSNlDS9Iw", "duration": 1699, "channel": "DEFCONConference", "uploader": "DEFCONConference", "upload_date": "20241016" } 1.23s - 8.23s | This text was transcribed using whisper model: large-v2 All right. Let's start. So hi, I'm Tanya Jenka. I'm also known as SheHacksPurple, and I want 8.23s - 16.15s | to talk about maturing our AppSec programs. So I work with companies all the time. I work 16.15s - 22.11s | with customers a lot. I love my customers. And I see all different levels. But when I 22.11s - 28.35s | work with someone, that means they take security seriously enough to engage me or to engage 28.35s - 35.03s | some group or to engage IAN's research. There's tons of companies that haven't engaged anyone. 35.03s - 41.55s | Like they don't have one of you to rely on, and they are like out there. And so I want 41.55s - 46.91s | to talk about three different models of AppSec that I see a lot, and then how we could take 46.91s - 51.51s | them to the next level. So if you are one of those models, that's cool. I can help you 51.51s - 55.79s | do better. And so where are we going to talk about? We're going to talk about common AppSec 55.79s - 62.11s | models, why they fail, and how we can just provide more value to our employers. Because 62.11s - 67.79s | to be quite frank, we all want software to be more secure. That's what we want. And so 67.79s - 72.03s | I want to briefly tell you how I did my research because someone pointed out to me, they're 72.03s - 75.35s | like, you can't just say because Tanya said so. You actually have to explain


chunk_text:
enough to engage me or to engage 28.35s - 35.03s | some group or to engage IAN's research. There's tons of companies that haven't engaged anyone. 35.03s - 41.55s | Like they don't have one of you to rely on, and they are like out there. And so I want 41.55s - 46.91s | to talk about three different models of AppSec that I see a lot, and then how we could take 46.91s - 51.51s | them to the next level. So if you are one of those models, that's cool. I can help you 51.51s - 55.79s | do better. And so where are we going to talk about? We're going to talk about common AppSec 55.79s - 62.11s | models, why they fail, and how we can just provide more value to our employers. Because 62.11s - 67.79s | to be quite frank, we all want software to be more secure. That's what we want. And so 67.79s - 72.03s | I want to briefly tell you how I did my research because someone pointed out to me, they're 72.03s - 75.35s | like, you can't just say because Tanya said so. You actually have to explain like how 75.35s - 79.23s | you learned these things. We can't just take your word for it. And I was like, you know, 79.23s - 84.19s | that's a really good point. So I've been working with this company called IAN's Research since 84.19s - 89.11s | I guess I started very, very late 2018. So let's call it 2019. And then I've worked with 89.11s - 94.65s | almost 400 different AppSec teams over that time. You know how like there used to be that 94.65s - 100.35s | game show online where you could phone a friend. So IAN's lets you phone a nerd. I'm one of 100.35s - 105.39s | the nerds. And then I just help crush your AppSec programs. And I do between one and seven 105.39s - 110.41s | calls a week. And then after so many years and taking so many notes, because I'm one 110.41s - 115.25s | of those obsessive note taker persons because I can't remember anything. I just realized 115.25s - 119.37s | like, wow, I have a lot of data. And when you have 400 companies that you've spread 119.37s - 124.77s | the data across, it's


chunk_text:
like how 75.35s - 79.23s | you learned these things. We can't just take your word for it. And I was like, you know, 79.23s - 84.19s | that's a really good point. So I've been working with this company called IAN's Research since 84.19s - 89.11s | I guess I started very, very late 2018. So let's call it 2019. And then I've worked with 89.11s - 94.65s | almost 400 different AppSec teams over that time. You know how like there used to be that 94.65s - 100.35s | game show online where you could phone a friend. So IAN's lets you phone a nerd. I'm one of 100.35s - 105.39s | the nerds. And then I just help crush your AppSec programs. And I do between one and seven 105.39s - 110.41s | calls a week. And then after so many years and taking so many notes, because I'm one 110.41s - 115.25s | of those obsessive note taker persons because I can't remember anything. I just realized 115.25s - 119.37s | like, wow, I have a lot of data. And when you have 400 companies that you've spread 119.37s - 124.77s | the data across, it's no longer like it's a data set. It's no longer you're talking 124.77s - 128.89s | about this company. It's a data set. And I was like, I need to start sharing this data 128.89s - 134.47s | in an anonymized ways. And so this is one of the talks where I'm sharing that data after 134.47s - 139.23s | so many years. And so thank you to IAN's Research for actually giving me money because it's 139.23s - 143.47s | pretty cool to get paid to just do really awesome work. So I'm this person when I brush 143.47s - 148.95s | my hair, I look pretty good. I work at SunGrep and I'm the head of community there. And they're 148.95s - 153.67s | very gracious and let me do lots of silly things like make cards against AppSec. I'm 153.67s - 158.95s | known as SheXPurple. I wrote a book called Alice and Bob Learn Application Security. 158.95s - 164.45s | So this is year 28 in tech for me. I'm old. I founded some stuff. I advise some people 164.65s - 169.49s | and I do some research. And so hopefully you're like, she seems qualified


chunk_text:
no longer like it's a data set. It's no longer you're talking 124.77s - 128.89s | about this company. It's a data set. And I was like, I need to start sharing this data 128.89s - 134.47s | in an anonymized ways. And so this is one of the talks where I'm sharing that data after 134.47s - 139.23s | so many years. And so thank you to IAN's Research for actually giving me money because it's 139.23s - 143.47s | pretty cool to get paid to just do really awesome work. So I'm this person when I brush 143.47s - 148.95s | my hair, I look pretty good. I work at SunGrep and I'm the head of community there. And they're 148.95s - 153.67s | very gracious and let me do lots of silly things like make cards against AppSec. I'm 153.67s - 158.95s | known as SheXPurple. I wrote a book called Alice and Bob Learn Application Security. 158.95s - 164.45s | So this is year 28 in tech for me. I'm old. I founded some stuff. I advise some people 164.65s - 169.49s | and I do some research. And so hopefully you're like, she seems qualified enough. I guess 169.49s - 175.21s | I'm not going to leave. I'll sit down. It's hot. Don't leave. So I want to talk about 175.21s - 182.21s | maturity. So how do I say? So this is like, so it says like, Tanya, gently explain that. 185.31s - 191.81s | So we as an industry, some of us are doing great and that is not very many of us. A lot 191.81s - 198.13s | of us are AppSec programs where like, I wish it was so much better. We don't feel super 198.13s - 203.01s | like we're not bragging to our friends. We're not like, we're just not as satisfied with 203.01s - 208.49s | how it's going. And we know this because we keep seeing giant breaches, right? We keep 208.49s - 214.31s | seeing software development not going as securely and reliably as we expect it to. Like the 214.31s - 220.09s | whole thing with CrowdStrike was just us not following proper software development practices, 220.09s - 225.69s | trusting the way we know we should. And they know too, because there's so many pressures. 225.69s - 231.13s | And so this talks about


chunk_text:
enough. I guess 169.49s - 175.21s | I'm not going to leave. I'll sit down. It's hot. Don't leave. So I want to talk about 175.21s - 182.21s | maturity. So how do I say? So this is like, so it says like, Tanya, gently explain that. 185.31s - 191.81s | So we as an industry, some of us are doing great and that is not very many of us. A lot 191.81s - 198.13s | of us are AppSec programs where like, I wish it was so much better. We don't feel super 198.13s - 203.01s | like we're not bragging to our friends. We're not like, we're just not as satisfied with 203.01s - 208.49s | how it's going. And we know this because we keep seeing giant breaches, right? We keep 208.49s - 214.31s | seeing software development not going as securely and reliably as we expect it to. Like the 214.31s - 220.09s | whole thing with CrowdStrike was just us not following proper software development practices, 220.09s - 225.69s | trusting the way we know we should. And they know too, because there's so many pressures. 225.69s - 231.13s | And so this talks about how we can mature because the whole industry is not where the 231.13s - 236.83s | people in this room are. The people in this room, you came probably from another country 236.83s - 241.43s | or from another part in this country to this place to learn, to become better. Everyone 241.43s - 245.21s | else isn't doing that, right? Only some of us are. So there's a bit of the cream of the 245.21s - 248.65s | crop going on here. So if you're like, of course we would do that. Yeah, you would, 248.97s - 254.61s | but not everyone's doing it. So I want us to all kind of continue on that. So one of 254.61s - 260.45s | the issues I see with, so there are some models. So there's OWASP, SAM, BSIM and NIST. I actually 260.45s - 265.17s | haven't used BSIM because it costs money and I'm very cheap. You'd be surprised. But I've 265.17s - 270.49s | done a lot of SAM and a lot of NIST. And these, when I talk with customers, they're like, 270.49s - 274.09s | oh, let's do a maturity model. And I'm like, great. And


chunk_text:
how we can mature because the whole industry is not where the 231.13s - 236.83s | people in this room are. The people in this room, you came probably from another country 236.83s - 241.43s | or from another part in this country to this place to learn, to become better. Everyone 241.43s - 245.21s | else isn't doing that, right? Only some of us are. So there's a bit of the cream of the 245.21s - 248.65s | crop going on here. So if you're like, of course we would do that. Yeah, you would, 248.97s - 254.61s | but not everyone's doing it. So I want us to all kind of continue on that. So one of 254.61s - 260.45s | the issues I see with, so there are some models. So there's OWASP, SAM, BSIM and NIST. I actually 260.45s - 265.17s | haven't used BSIM because it costs money and I'm very cheap. You'd be surprised. But I've 265.17s - 270.49s | done a lot of SAM and a lot of NIST. And these, when I talk with customers, they're like, 270.49s - 274.09s | oh, let's do a maturity model. And I'm like, great. And then I start telling them about 274.09s - 279.45s | it and they're so overwhelmed. I went to a SAM, an OWASP SAM user meetup. So SAM's 279.45s - 285.69s | awesome just to be clear. And it's not a person, it's the software assurance maturity model. 285.69s - 290.13s | And they were telling me that on average, their user group, so the people that are obsessed 290.13s - 296.67s | with SAM, they rate at like 1.1 out of three. So these are the people that have been doing 296.67s - 300.93s | SAM for years, working their butts off and they're barely getting one out of three. So 300.93s - 306.09s | when I ask a client who's like, what does AppSec stand for? To look at SAM, their eyes 306.09s - 310.53s | glaze over, they start to cry. They're just like, I can't even, I can't even fathom where 310.53s - 316.57s | to start. So I'm going to start below that. Okay. Because that's where we are. So onto 316.57s - 322.81s | the models. I just like cats and this is the OSI model, but with cats. Anyway. Okay. So 322.81s - 327.65s


chunk_text:
then I start telling them about 274.09s - 279.45s | it and they're so overwhelmed. I went to a SAM, an OWASP SAM user meetup. So SAM's 279.45s - 285.69s | awesome just to be clear. And it's not a person, it's the software assurance maturity model. 285.69s - 290.13s | And they were telling me that on average, their user group, so the people that are obsessed 290.13s - 296.67s | with SAM, they rate at like 1.1 out of three. So these are the people that have been doing 296.67s - 300.93s | SAM for years, working their butts off and they're barely getting one out of three. So 300.93s - 306.09s | when I ask a client who's like, what does AppSec stand for? To look at SAM, their eyes 306.09s - 310.53s | glaze over, they start to cry. They're just like, I can't even, I can't even fathom where 310.53s - 316.57s | to start. So I'm going to start below that. Okay. Because that's where we are. So onto 316.57s - 322.81s | the models. I just like cats and this is the OSI model, but with cats. Anyway. Okay. So 322.81s - 327.65s | model one, let's just pen test the important things. Who has worked with a client at some 327.69s - 332.33s | point where that's all they do. So like once or twice a year, they have a pen tester come 332.33s - 338.65s | in, they do a pen test, they fail it so bad. And, and then they're like, yeah, that's our 338.65s - 344.13s | AppSec program. And I'm like, whoa, that's expensive for what you're getting. Like pen 344.13s - 348.21s | testing is awesome. But if that's the only thing you're doing, and you're just pen testing 348.21s - 355.57s | like two of your 200 apps once a year, you're getting a blink of a tiny, tiny portion of 355.57s - 362.33s | your entire attack surface. And like, this is not going to go well. And so unfortunately, 362.33s - 366.93s | even in 2024, lots of companies are starting their first AppSec program. Like last year, I 366.93s - 371.61s | worked with a gaming company and you own their games. And I helped them start their first 371.61s - 378.33s | AppSec program. And so this, this year I trained a


chunk_text:
| model one, let's just pen test the important things. Who has worked with a client at some 327.69s - 332.33s | point where that's all they do. So like once or twice a year, they have a pen tester come 332.33s - 338.65s | in, they do a pen test, they fail it so bad. And, and then they're like, yeah, that's our 338.65s - 344.13s | AppSec program. And I'm like, whoa, that's expensive for what you're getting. Like pen 344.13s - 348.21s | testing is awesome. But if that's the only thing you're doing, and you're just pen testing 348.21s - 355.57s | like two of your 200 apps once a year, you're getting a blink of a tiny, tiny portion of 355.57s - 362.33s | your entire attack surface. And like, this is not going to go well. And so unfortunately, 362.33s - 366.93s | even in 2024, lots of companies are starting their first AppSec program. Like last year, I 366.93s - 371.61s | worked with a gaming company and you own their games. And I helped them start their first 371.61s - 378.33s | AppSec program. And so this, this year I trained a company that handles $2 billion of 378.49s - 383.57s | their customers' money. And they were letting every single person at the company query the 383.57s - 387.77s | production database using the same user. And then they take client scripts, and they 387.77s - 392.37s | just run it without any checking against the database that handles $2 billion. And I was 392.37s - 399.01s | like, oh, okay, we redesigned it. Life is way better now. Like everything's going right, 399.01s - 405.37s | but like $2 billion. And that's what they were doing. Right. So at least these people 405.37s - 409.29s | are doing two pen tests a year because they were doing literally nothing before I got 409.29s - 415.37s | there in March. So yeah. So let's talk about this model. So it is extremely common. 415.73s - 419.97s | Generally, there's no formal system development lifecycle. They're like, we're doing 420.45s - 427.29s | WaterGile. We're doing DevOpsGile. We're doing, you know, we have a scrum meeting, but 427.29s - 433.29s | it's only once a week. And then like, we do sprints, but they're six months long. Every 433.29s - 439.25s |


chunk_text:
company that handles $2 billion of 378.49s - 383.57s | their customers' money. And they were letting every single person at the company query the 383.57s - 387.77s | production database using the same user. And then they take client scripts, and they 387.77s - 392.37s | just run it without any checking against the database that handles $2 billion. And I was 392.37s - 399.01s | like, oh, okay, we redesigned it. Life is way better now. Like everything's going right, 399.01s - 405.37s | but like $2 billion. And that's what they were doing. Right. So at least these people 405.37s - 409.29s | are doing two pen tests a year because they were doing literally nothing before I got 409.29s - 415.37s | there in March. So yeah. So let's talk about this model. So it is extremely common. 415.73s - 419.97s | Generally, there's no formal system development lifecycle. They're like, we're doing 420.45s - 427.29s | WaterGile. We're doing DevOpsGile. We're doing, you know, we have a scrum meeting, but 427.29s - 433.29s | it's only once a week. And then like, we do sprints, but they're six months long. Every 433.29s - 439.25s | single team is doing something different. There's a mixed tech stack. So we have like 439.29s - 442.93s | five different programming languages, and 12 different versions of six different 442.93s - 447.61s | frameworks. And then like, some of us are using PaaS, some of us are using IAC, like 447.61s - 453.69s | everyone just like, let's do our own thing. It's great. It's not. We have code all over 453.69s - 458.97s | the place. So I'm like, oh, where do you keep your code? They're like, so about that. And 458.97s - 464.17s | it's in like five different types of repositories, but there's actually like 22 464.17s - 469.05s | instances. So you don't even know where your code is. Um, yeah, everyone's just kind of 469.05s - 474.37s | doing their own thing. And that's not good. So why is this model bad? I think you already 474.37s - 478.61s | know, but let me tell you. So it's quite expensive for what you're getting. So pen 478.61s - 484.49s | testers are not cheap, like good ones are not cheap, and they shouldn't be. But usually, I 484.49s - 489.33s | like to


chunk_text:
single team is doing something different. There's a mixed tech stack. So we have like 439.29s - 442.93s | five different programming languages, and 12 different versions of six different 442.93s - 447.61s | frameworks. And then like, some of us are using PaaS, some of us are using IAC, like 447.61s - 453.69s | everyone just like, let's do our own thing. It's great. It's not. We have code all over 453.69s - 458.97s | the place. So I'm like, oh, where do you keep your code? They're like, so about that. And 458.97s - 464.17s | it's in like five different types of repositories, but there's actually like 22 464.17s - 469.05s | instances. So you don't even know where your code is. Um, yeah, everyone's just kind of 469.05s - 474.37s | doing their own thing. And that's not good. So why is this model bad? I think you already 474.37s - 478.61s | know, but let me tell you. So it's quite expensive for what you're getting. So pen 478.61s - 484.49s | testers are not cheap, like good ones are not cheap, and they shouldn't be. But usually, I 484.49s - 489.33s | like to do pen testing as like this beautiful cherry on top to make something perfect, not 489.33s - 494.61s | as my only effort whatsoever. And so you're having terrible, terrible coverage. Developers 494.61s - 500.05s | are getting zero support. Like there's no opportunity for anyone to learn. That's one 500.05s - 503.81s | of the biggest problems with this one. None of your staff's getting better, because you 503.81s - 507.69s | have this awesome, brilliant pen tester coming in doing cool work, and then all their super 507.69s - 512.69s | awesome brain and knowledge leaves. Right. And so that is one of the many reasons this 512.69s - 518.97s | is bad. And so how can we mature this model? So I have, so this one, I have the, if you 518.97s - 524.93s | have budget, and if you don't have budget, a lot of my career, my budget has not been 524.93s - 528.13s | quite what I want. I totally have an awesome budget now running a community. It's like 528.13s - 534.97s | 8,000 bucks anyway. Um, how do we mature this model? So first I like to start with secure 534.97s -


chunk_text:
do pen testing as like this beautiful cherry on top to make something perfect, not 489.33s - 494.61s | as my only effort whatsoever. And so you're having terrible, terrible coverage. Developers 494.61s - 500.05s | are getting zero support. Like there's no opportunity for anyone to learn. That's one 500.05s - 503.81s | of the biggest problems with this one. None of your staff's getting better, because you 503.81s - 507.69s | have this awesome, brilliant pen tester coming in doing cool work, and then all their super 507.69s - 512.69s | awesome brain and knowledge leaves. Right. And so that is one of the many reasons this 512.69s - 518.97s | is bad. And so how can we mature this model? So I have, so this one, I have the, if you 518.97s - 524.93s | have budget, and if you don't have budget, a lot of my career, my budget has not been 524.93s - 528.13s | quite what I want. I totally have an awesome budget now running a community. It's like 528.13s - 534.97s | 8,000 bucks anyway. Um, how do we mature this model? So first I like to start with secure 534.97s - 539.89s | coding training, and I'm going to give you a link to free training for secure coding 539.89s - 543.93s | after this. There's going to be a bunch of free resources because I want all of you to 543.93s - 550.05s | be awesome. Um, but the way I teach secure coding training is not the same way as someone 550.05s - 555.01s | else does. And so, although obviously I'm the best ever, what you want is training that 555.01s - 560.05s | works for your team, that your team likes. And so as much as like a salesperson will 560.05s - 564.41s | tell you ours is the best ever, what only really matters is that your team likes it 564.41s - 569.19s | and that your team's willing to sit through it or go on the platform and play or attend 569.19s - 573.83s | the monthly lunch and learns, whatever it is that you decide, try it out on some devs 573.83s - 579.55s | first and make sure that they're willing to tolerate it. Don't pay $12,000 to have someone 579.55s - 586.35s | come in that they hate, right? Or a system


chunk_text:
539.89s | coding training, and I'm going to give you a link to free training for secure coding 539.89s - 543.93s | after this. There's going to be a bunch of free resources because I want all of you to 543.93s - 550.05s | be awesome. Um, but the way I teach secure coding training is not the same way as someone 550.05s - 555.01s | else does. And so, although obviously I'm the best ever, what you want is training that 555.01s - 560.05s | works for your team, that your team likes. And so as much as like a salesperson will 560.05s - 564.41s | tell you ours is the best ever, what only really matters is that your team likes it 564.41s - 569.19s | and that your team's willing to sit through it or go on the platform and play or attend 569.19s - 573.83s | the monthly lunch and learns, whatever it is that you decide, try it out on some devs 573.83s - 579.55s | first and make sure that they're willing to tolerate it. Don't pay $12,000 to have someone 579.55s - 586.35s | come in that they hate, right? Or a system or whatever it is that you do, try to make 586.35s - 591.27s | sure it's a thing that they'll tolerate. Um, so the no budget plan. So for those of us 591.27s - 596.11s | that don't have giant budgets, you don't have to raise your hands cause it's sad. Um, okay. 596.11s - 601.15s | So one, so use free tools like zap and burp if you're going to do dynamic scanning and 601.15s - 605.83s | then for static analysis. So for, for dynamic analysis, there's basically like two super 605.83s - 609.99s | clear awesome winners. And then for stack analysis, like there's tons, like I could 609.99s - 614.83s | fill this whole page. So like there's many, many, many open source SAS that are free. 614.83s - 621.27s | They're like, not bad, not bad. Um, and then create a guideline. And I have one online 621.27s - 626.31s | again, that's free, but basically create some sort of guidance for your devs about 626.31s - 631.55s | how to create code that you feel is acceptable. And it's totally okay to steal something off 631.55s - 636.15s | of my website, off the


chunk_text:
or whatever it is that you do, try to make 586.35s - 591.27s | sure it's a thing that they'll tolerate. Um, so the no budget plan. So for those of us 591.27s - 596.11s | that don't have giant budgets, you don't have to raise your hands cause it's sad. Um, okay. 596.11s - 601.15s | So one, so use free tools like zap and burp if you're going to do dynamic scanning and 601.15s - 605.83s | then for static analysis. So for, for dynamic analysis, there's basically like two super 605.83s - 609.99s | clear awesome winners. And then for stack analysis, like there's tons, like I could 609.99s - 614.83s | fill this whole page. So like there's many, many, many open source SAS that are free. 614.83s - 621.27s | They're like, not bad, not bad. Um, and then create a guideline. And I have one online 621.27s - 626.31s | again, that's free, but basically create some sort of guidance for your devs about 626.31s - 631.55s | how to create code that you feel is acceptable. And it's totally okay to steal something off 631.55s - 636.15s | of my website, off the OS website, off of like websites that give you permission to 636.15s - 641.91s | do that. So you don't have to reinvent the wheel, but give them some guidance. Um, next 641.91s - 648.75s | create, create a list of requirements. So when you start a project, you set clear expectations 648.75s - 654.43s | for security for everyone on that project. And this could be just like five things. 654.43s - 657.27s | Like you're going to scan it with our little SAS. You're going to scan it with our little 657.27s - 661.87s | desk. You're going to fix the mediums and above, and then we're going to do a pen test 661.87s - 667.83s | at the end. That could just be your entire requirements, right? And follow the guideline. 667.83s - 671.83s | Like it doesn't have to be really advanced and you can add to it over time, but starting 671.83s - 675.91s | a project with super clear expectations will get you a lot more of what you want in this 675.91s - 681.71s | world. Um, consolidate your code. So this is a really hard one. This is one of those 681.71s


chunk_text:
OS website, off of like websites that give you permission to 636.15s - 641.91s | do that. So you don't have to reinvent the wheel, but give them some guidance. Um, next 641.91s - 648.75s | create, create a list of requirements. So when you start a project, you set clear expectations 648.75s - 654.43s | for security for everyone on that project. And this could be just like five things. 654.43s - 657.27s | Like you're going to scan it with our little SAS. You're going to scan it with our little 657.27s - 661.87s | desk. You're going to fix the mediums and above, and then we're going to do a pen test 661.87s - 667.83s | at the end. That could just be your entire requirements, right? And follow the guideline. 667.83s - 671.83s | Like it doesn't have to be really advanced and you can add to it over time, but starting 671.83s - 675.91s | a project with super clear expectations will get you a lot more of what you want in this 675.91s - 681.71s | world. Um, consolidate your code. So this is a really hard one. This is one of those 681.71s - 684.67s | things where you're going to have to talk to lots of people, but if your code is all 684.67s - 690.43s | in the same place, your life's going to be better. Um, the devs will also probably enjoy 690.43s - 693.91s | that once it happens, like while we're doing it, it's going to suck. But once it's all 693.91s - 699.47s | in the same place, like everyone's going to benefit, not just you. And then lastly, oh, 699.47s - 703.11s | actually no, I have a whole nother page. Push for a real system development lifecycle, push 703.11s - 707.43s | for centralization, push for standardization. This is a really hard one that requires a 707.43s - 713.27s | lot of social currency, but it is worth every single hour that you spend talking to people 713.27s - 717.35s | to make that happen. Like every single person in your whole it department will benefit from 717.35s - 722.91s | that. Okay. Number six, threat model, the super important apps. So those ones that you 722.91s - 727.11s | used to just pen test and that's all you did do a threat model. You


chunk_text:
- 684.67s | things where you're going to have to talk to lots of people, but if your code is all 684.67s - 690.43s | in the same place, your life's going to be better. Um, the devs will also probably enjoy 690.43s - 693.91s | that once it happens, like while we're doing it, it's going to suck. But once it's all 693.91s - 699.47s | in the same place, like everyone's going to benefit, not just you. And then lastly, oh, 699.47s - 703.11s | actually no, I have a whole nother page. Push for a real system development lifecycle, push 703.11s - 707.43s | for centralization, push for standardization. This is a really hard one that requires a 707.43s - 713.27s | lot of social currency, but it is worth every single hour that you spend talking to people 713.27s - 717.35s | to make that happen. Like every single person in your whole it department will benefit from 717.35s - 722.91s | that. Okay. Number six, threat model, the super important apps. So those ones that you 722.91s - 727.11s | used to just pen test and that's all you did do a threat model. You can do a super basic 727.11s - 731.43s | threat model. Um, there's this amazing human named Adam show stack. He has this thing called 731.43s - 737.03s | the four question frame for threat modeling is just four questions and a conversation. 737.03s - 742.19s | I start with that. It's so good. He's so smart. Um, you don't have to read all of his awesome 742.19s - 747.51s | books, but you totally could. And like, just start with that and you will get really far 747.51s - 753.55s | share information about incidents. So if you've had an app incident, if you're allowed, if 753.55s - 757.95s | it makes sense, share with the devs. Quite often when I started talking to them about 757.95s - 762.35s | security incidents, they had no idea. They had no idea we've been attacked. They had 762.35s - 766.99s | no idea we're attacked 4 billion times a day. They have no idea this stuff's happening and 766.99s - 772.55s | I'm not doing it to scare their pants off. I want them to understand the threats we face. 772.55s - 776.55s | So I'm not trying to fear uncertainty


chunk_text:
can do a super basic 727.11s - 731.43s | threat model. Um, there's this amazing human named Adam show stack. He has this thing called 731.43s - 737.03s | the four question frame for threat modeling is just four questions and a conversation. 737.03s - 742.19s | I start with that. It's so good. He's so smart. Um, you don't have to read all of his awesome 742.19s - 747.51s | books, but you totally could. And like, just start with that and you will get really far 747.51s - 753.55s | share information about incidents. So if you've had an app incident, if you're allowed, if 753.55s - 757.95s | it makes sense, share with the devs. Quite often when I started talking to them about 757.95s - 762.35s | security incidents, they had no idea. They had no idea we've been attacked. They had 762.35s - 766.99s | no idea we're attacked 4 billion times a day. They have no idea this stuff's happening and 766.99s - 772.55s | I'm not doing it to scare their pants off. I want them to understand the threats we face. 772.55s - 776.55s | So I'm not trying to fear uncertainty and doubt. I'm like, this is specifically what 776.55s - 783.31s | is happening to us and why I need your help so bad. Um, scan for secrets. There are free 783.31s - 787.63s | secret scanners. There's paid for secret scanners. Basically if you have more time, you should 787.63s - 792.15s | use a free one. If you have more money, she's a paid one, but basically scan your code for 792.15s - 795.91s | secrets and then rotate those secrets. Use a secret management tool, but that's later 795.91s - 801.87s | for when you have money. Um, there's a free WAF from OWASP called mod security and you 801.87s - 809.03s | can put that in front of your super scary, terrible, like last updated in 2008 apps. 809.03s - 814.99s | Do that. Um, and then you still pen test the important apps. However, now you don't fail 814.99s - 818.83s | horribly and it's no longer completely embarrassing where you're like, Oh God, I can't look the 818.83s - 824.39s | pen tester in the face. This is so bad. Um, so that's nice. And then, so now if you have 824.39s - 828.19s


chunk_text:
and doubt. I'm like, this is specifically what 776.55s - 783.31s | is happening to us and why I need your help so bad. Um, scan for secrets. There are free 783.31s - 787.63s | secret scanners. There's paid for secret scanners. Basically if you have more time, you should 787.63s - 792.15s | use a free one. If you have more money, she's a paid one, but basically scan your code for 792.15s - 795.91s | secrets and then rotate those secrets. Use a secret management tool, but that's later 795.91s - 801.87s | for when you have money. Um, there's a free WAF from OWASP called mod security and you 801.87s - 809.03s | can put that in front of your super scary, terrible, like last updated in 2008 apps. 809.03s - 814.99s | Do that. Um, and then you still pen test the important apps. However, now you don't fail 814.99s - 818.83s | horribly and it's no longer completely embarrassing where you're like, Oh God, I can't look the 818.83s - 824.39s | pen tester in the face. This is so bad. Um, so that's nice. And then, so now if you have 824.39s - 828.19s | budget, I want to add three more things. So if you have budget, see how she kind of looks 828.19s - 833.87s | like a badass. She's like, I have budget, hire an app sec person. Like that is the best 833.87s - 839.71s | investment you can ever make. Like hire someone from this room. Um, like seriously, the people 839.71s - 844.79s | there in this room, probably almost all of them will be amazing to work with. Um, pay 844.79s - 851.27s | for a next gen SAST. So I find that the paid SAS are a lot better than the paid tasks. 851.27s - 855.07s | I kind of think that sucks. And I remember people telling me at the beginning of my app 855.07s - 858.75s | sec career that DAS sucks. I'm like, DAS is amazing. And they're like, one day when you're 858.75s - 865.39s | a more mature app sec professional, Tanya, you'll understand. And one day I just did, 865.39s - 870.67s | I'm sorry if you make a DAS, I'm a jerk. Um, continue with your free-ish DAS. I just 870.67s - 877.23s | would never buy one. Um,


