# MediaChunks

chunk_text:
| the first concern that comes to mind, 1758.84s - 1761.56s | which is like, wait, are they gonna instrument 1761.56s - 1763.60s | and know every single time 1763.60s - 1765.64s | that I don't make an absolutely perfect touchdown? 1765.64s - 1768.20s | And then is my job gonna be potentially at risk 1768.20s - 1770.24s | if I'm on the back half of that curve? 1770.24s - 1772.44s | So that's like the first thing that I wanna share 1772.44s - 1774.36s | that actually came to mind. 1774.36s - 1776.48s | To your point, sorry, could you just restate 1776.48s - 1808.16s | very concisely the final question that you asked? 1808.16s - 1809.44s | Evolving to, yeah. 1809.44s - 1811.64s | Okay, so my thoughts on this 1811.64s - 1815.12s | from a crew alerting perspective. 1815.12s - 1819.36s | Just really, really quick to bring us all up to speed. 1820.52s - 1821.92s | I think it's really important to acknowledge 1821.92s - 1824.80s | that one of the fundamental and most important things 1824.80s - 1827.32s | that a crew does obviously is decision-making 1827.32s - 1830.76s | and you want to preserve the pilot's ability 1830.76s - 1833.28s | to make decisions. 1833.36s - 1836.80s | There is a balance in terms of the amount of information 1836.80s - 1837.76s | that you give a pilot 1837.76s - 1841.04s | because all pilots are trained to manage 1841.04s - 1842.76s | basically their attention, their workload, 1842.76s - 1844.16s | they call this crew load. 1844.16s - 1847.36s | And when you overload cognitively the crew, 1847.36s - 1850.00s | the same thing happens like in a sock 1850.00s - 1852.20s | when you just throw so many alerts, 1852.20s - 1854.80s | you can kind of paralyze and affect the ability 1854.80s - 1856.44s | for them to accomplish their job. 1856.44s - 1858.52s | So there's a bunch of disciplines 1858.52s - 1862.28s | that come into this question and figuring out 1862.28s - 1865.44s | what do you share with the crew for what purpose, 1865.44s - 1868.48s | when, where and why and how you do so. 1868.48s - 1870.04s | There are human factors, 1870.04s - 1872.96s | there's like design UI UX elements to that, 1872.96s - 1875.72s | there's process, there's training implications. 1875.72s -


chunk_text:
the pilot's ability 1830.76s - 1833.28s | to make decisions. 1833.36s - 1836.80s | There is a balance in terms of the amount of information 1836.80s - 1837.76s | that you give a pilot 1837.76s - 1841.04s | because all pilots are trained to manage 1841.04s - 1842.76s | basically their attention, their workload, 1842.76s - 1844.16s | they call this crew load. 1844.16s - 1847.36s | And when you overload cognitively the crew, 1847.36s - 1850.00s | the same thing happens like in a sock 1850.00s - 1852.20s | when you just throw so many alerts, 1852.20s - 1854.80s | you can kind of paralyze and affect the ability 1854.80s - 1856.44s | for them to accomplish their job. 1856.44s - 1858.52s | So there's a bunch of disciplines 1858.52s - 1862.28s | that come into this question and figuring out 1862.28s - 1865.44s | what do you share with the crew for what purpose, 1865.44s - 1868.48s | when, where and why and how you do so. 1868.48s - 1870.04s | There are human factors, 1870.04s - 1872.96s | there's like design UI UX elements to that, 1872.96s - 1875.72s | there's process, there's training implications. 1875.72s - 1878.44s | And it's actually a really, really difficult, 1880.48s - 1882.28s | I think question to answer. 1882.28s - 1884.36s | So I don't know that anybody has that figured out 1884.36s - 1885.56s | first thing I want to say is like, 1885.56s - 1886.72s | I'm not going to have an answer to that, 1886.72s - 1888.80s | but I feel really strongly as a pilot 1888.80s - 1893.12s | that I would like to know before I go wheels up 1893.12s - 1897.60s | on the runway that I have a safe configuration 1897.60s - 1901.32s | of hardware and software on the plane first and foremost, 1901.32s - 1903.40s | because the situation is different 1903.40s - 1905.08s | when you are weight on wheels, 1905.08s - 1907.60s | you're on the ground as opposed to in flight. 1907.60s - 1912.60s | If I lose a system, I need to understand very quickly 1912.80s - 1915.28s | what other systems I can rely on 1915.28s - 1918.48s | to fly, navigate or communicate the plane. 1919.60s - 1923.28s | Pilots get indications, warnings, cautions and advisories 1923.28s - 1926.16s | to do that in the


chunk_text:
1878.44s | And it's actually a really, really difficult, 1880.48s - 1882.28s | I think question to answer. 1882.28s - 1884.36s | So I don't know that anybody has that figured out 1884.36s - 1885.56s | first thing I want to say is like, 1885.56s - 1886.72s | I'm not going to have an answer to that, 1886.72s - 1888.80s | but I feel really strongly as a pilot 1888.80s - 1893.12s | that I would like to know before I go wheels up 1893.12s - 1897.60s | on the runway that I have a safe configuration 1897.60s - 1901.32s | of hardware and software on the plane first and foremost, 1901.32s - 1903.40s | because the situation is different 1903.40s - 1905.08s | when you are weight on wheels, 1905.08s - 1907.60s | you're on the ground as opposed to in flight. 1907.60s - 1912.60s | If I lose a system, I need to understand very quickly 1912.80s - 1915.28s | what other systems I can rely on 1915.28s - 1918.48s | to fly, navigate or communicate the plane. 1919.60s - 1923.28s | Pilots get indications, warnings, cautions and advisories 1923.28s - 1926.16s | to do that in the event of systems losses, 1926.16s - 1929.28s | but we don't train or inform pilots to think today 1929.28s - 1932.64s | about the implications of either malicious software 1932.64s - 1934.28s | or software glitches or failures. 1934.28s - 1936.60s | I think the closest thing that comes to that 1937.92s - 1942.44s | relatably is like somebody didn't update logs on a plane 1942.44s - 1944.20s | and now I have like out of date charts 1944.20s - 1945.88s | or an out of date navigation database 1945.88s - 1946.72s | or something like that, 1946.72s - 1949.56s | and I can't do a specific type of procedure. 1949.56s - 1952.44s | I think that there is a component of training 1952.44s - 1955.64s | where we want pilots to understand 1955.64s - 1959.92s | that systems can be impacted in certain ways 1959.92s - 1962.36s | and it informs the way that they think 1962.36s - 1964.96s | about using redundant systems 1964.96s - 1968.24s | or more analog capabilities to perform functions. 1968.24s - 1971.68s | And I think some type of indication in the cockpit 1971.68s - 1973.68s | is important while we continue to


chunk_text:
event of systems losses, 1926.16s - 1929.28s | but we don't train or inform pilots to think today 1929.28s - 1932.64s | about the implications of either malicious software 1932.64s - 1934.28s | or software glitches or failures. 1934.28s - 1936.60s | I think the closest thing that comes to that 1937.92s - 1942.44s | relatably is like somebody didn't update logs on a plane 1942.44s - 1944.20s | and now I have like out of date charts 1944.20s - 1945.88s | or an out of date navigation database 1945.88s - 1946.72s | or something like that, 1946.72s - 1949.56s | and I can't do a specific type of procedure. 1949.56s - 1952.44s | I think that there is a component of training 1952.44s - 1955.64s | where we want pilots to understand 1955.64s - 1959.92s | that systems can be impacted in certain ways 1959.92s - 1962.36s | and it informs the way that they think 1962.36s - 1964.96s | about using redundant systems 1964.96s - 1968.24s | or more analog capabilities to perform functions. 1968.24s - 1971.68s | And I think some type of indication in the cockpit 1971.68s - 1973.68s | is important while we continue to advance 1974.08s - 1977.60s | the state of art of actually securing the technology itself. 1979.04s - 1981.52s | I don't know if that's a satisfying answer for you, 1981.52s - 1983.96s | but I see a head shaking. 2001.01s - 2001.85s | Yeah. 2004.09s - 2007.05s | Well, I don't think that we did, please, please. 2007.05s - 2010.17s | And also is real time a requirement 2010.17s - 2012.17s | or is it just getting all of the data 2012.17s - 2014.81s | and being able to protect a system at scale 2014.81s - 2017.41s | maybe with some delays in that data collection, 2017.41s - 2019.57s | the requirement, what are the drawbacks? 2019.57s - 2022.73s | Yeah, in my experience, basically the use of data 2022.73s - 2024.33s | basically is most effective when you need 2024.33s - 2026.65s | to make calibrations or changes to a system. 2026.65s - 2028.33s | So if everything's working great, 2029.21s - 2031.25s | I mean like it may be able to improve workflows, 2031.25s - 2032.85s | it may be able to improve the preciseness 2032.85s - 2033.85s | or accuracy of your work, 2033.85s - 2036.85s |


chunk_text:
advance 1974.08s - 1977.60s | the state of art of actually securing the technology itself. 1979.04s - 1981.52s | I don't know if that's a satisfying answer for you, 1981.52s - 1983.96s | but I see a head shaking. 2001.01s - 2001.85s | Yeah. 2004.09s - 2007.05s | Well, I don't think that we did, please, please. 2007.05s - 2010.17s | And also is real time a requirement 2010.17s - 2012.17s | or is it just getting all of the data 2012.17s - 2014.81s | and being able to protect a system at scale 2014.81s - 2017.41s | maybe with some delays in that data collection, 2017.41s - 2019.57s | the requirement, what are the drawbacks? 2019.57s - 2022.73s | Yeah, in my experience, basically the use of data 2022.73s - 2024.33s | basically is most effective when you need 2024.33s - 2026.65s | to make calibrations or changes to a system. 2026.65s - 2028.33s | So if everything's working great, 2029.21s - 2031.25s | I mean like it may be able to improve workflows, 2031.25s - 2032.85s | it may be able to improve the preciseness 2032.85s - 2033.85s | or accuracy of your work, 2033.85s - 2036.85s | but fundamentally if things are going great 2036.85s - 2037.73s | and you have no problem, 2037.73s - 2039.77s | then the data's not really important. 2039.77s - 2041.77s | The data becomes most useful under courses 2041.77s - 2045.29s | of basically need for calibration or change 2045.29s - 2046.65s | in current operating flows. 2046.65s - 2048.41s | So like for example, I could imagine 2048.41s - 2051.53s | if maybe there's a massive storm 2051.53s - 2053.73s | that requires basically a bunch of planes 2053.73s - 2056.61s | to land unexpectedly in different places, 2056.61s - 2058.93s | potentially the collection of data could inform 2058.93s - 2060.57s | or improve on decision making there. 2060.57s - 2063.17s | So that kind of translates to in the DoD world, 2063.17s - 2064.81s | we call that command and control, right? 2064.81s - 2067.01s | So some sort of centralized control 2067.01s - 2068.13s | or decentralized control, 2068.13s - 2069.81s | the collection of data under that circumstance 2069.81s - 2071.25s | would be useful. 2071.25s - 2074.21s | On the plane itself, I can think of a example 2074.21s - 2076.45s | which I'm gonna


chunk_text:
but fundamentally if things are going great 2036.85s - 2037.73s | and you have no problem, 2037.73s - 2039.77s | then the data's not really important. 2039.77s - 2041.77s | The data becomes most useful under courses 2041.77s - 2045.29s | of basically need for calibration or change 2045.29s - 2046.65s | in current operating flows. 2046.65s - 2048.41s | So like for example, I could imagine 2048.41s - 2051.53s | if maybe there's a massive storm 2051.53s - 2053.73s | that requires basically a bunch of planes 2053.73s - 2056.61s | to land unexpectedly in different places, 2056.61s - 2058.93s | potentially the collection of data could inform 2058.93s - 2060.57s | or improve on decision making there. 2060.57s - 2063.17s | So that kind of translates to in the DoD world, 2063.17s - 2064.81s | we call that command and control, right? 2064.81s - 2067.01s | So some sort of centralized control 2067.01s - 2068.13s | or decentralized control, 2068.13s - 2069.81s | the collection of data under that circumstance 2069.81s - 2071.25s | would be useful. 2071.25s - 2074.21s | On the plane itself, I can think of a example 2074.21s - 2076.45s | which I'm gonna be intentionally a little bit vague about, 2077.13s - 2080.25s | but basically if you were to lose, say, 2081.89s - 2086.33s | a sensor system that has multiple sensors 2086.33s - 2088.29s | and you were to lose, say, one of them, 2089.61s - 2092.05s | you could potentially rely on those others 2092.05s - 2094.53s | which may like to basically infer, 2094.53s - 2099.53s | to inference or predict what that missing sensor would see. 2100.93s - 2103.17s | And so under certain circumstances, that could be useful. 2103.17s - 2104.41s | So there's ways to basically produce 2104.41s - 2107.17s | some sort of emergent capability 2107.17s - 2111.49s | or extract some signal from other systems that are lost 2111.49s - 2113.09s | that are generally not relied on 2113.09s - 2115.77s | under general normal operating conditions. 2115.77s - 2118.37s | And so those are some ways in which we've found 2118.37s - 2121.53s | the use of data off an operational system 2121.53s - 2126.37s | to be really valuable under mission, like DoD mission. 2126.37s - 2128.37s | It may translate to civil aviation. 2128.37s - 2129.85s | Does that sort of


chunk_text:
be intentionally a little bit vague about, 2077.13s - 2080.25s | but basically if you were to lose, say, 2081.89s - 2086.33s | a sensor system that has multiple sensors 2086.33s - 2088.29s | and you were to lose, say, one of them, 2089.61s - 2092.05s | you could potentially rely on those others 2092.05s - 2094.53s | which may like to basically infer, 2094.53s - 2099.53s | to inference or predict what that missing sensor would see. 2100.93s - 2103.17s | And so under certain circumstances, that could be useful. 2103.17s - 2104.41s | So there's ways to basically produce 2104.41s - 2107.17s | some sort of emergent capability 2107.17s - 2111.49s | or extract some signal from other systems that are lost 2111.49s - 2113.09s | that are generally not relied on 2113.09s - 2115.77s | under general normal operating conditions. 2115.77s - 2118.37s | And so those are some ways in which we've found 2118.37s - 2121.53s | the use of data off an operational system 2121.53s - 2126.37s | to be really valuable under mission, like DoD mission. 2126.37s - 2128.37s | It may translate to civil aviation. 2128.37s - 2129.85s | Does that sort of answer your question? 2129.85s - 2133.01s | Give you some examples? 2133.01s - 2134.09s | Okay. 2134.29s - 2135.13s | I'll just add, 2137.41s - 2142.41s | I think that streaming real-time data is great, 2142.41s - 2145.81s | but there's a need to get all of the data 2145.81s - 2147.61s | and it doesn't necessarily happen 2149.13s - 2149.97s | when the plane is flying. 2149.97s - 2151.13s | And when I say all of the data, 2151.13s - 2153.29s | I'm thinking like full take data collection 2154.57s - 2157.57s | off the onboard avionics networks, right? 2157.57s - 2161.77s | So hearing 664, MIL-STD-1553, 2161.77s - 2163.93s | the inner LRU communications 2164.77s - 2167.81s | because if you were to think a little bit differently 2167.81s - 2170.29s | about the aircraft, that data allows you 2170.29s - 2173.05s | to treat each aircraft as a sensing system. 2173.05s - 2175.69s | And any data scientists will tell you 2175.69s - 2177.53s | that the more data that they have, 2177.53s - 2179.93s | obviously the better that they can train predictive, 2179.93s - 2182.13s | maintenance models, the better that you can train 2182.13s


chunk_text:
answer your question? 2129.85s - 2133.01s | Give you some examples? 2133.01s - 2134.09s | Okay. 2134.29s - 2135.13s | I'll just add, 2137.41s - 2142.41s | I think that streaming real-time data is great, 2142.41s - 2145.81s | but there's a need to get all of the data 2145.81s - 2147.61s | and it doesn't necessarily happen 2149.13s - 2149.97s | when the plane is flying. 2149.97s - 2151.13s | And when I say all of the data, 2151.13s - 2153.29s | I'm thinking like full take data collection 2154.57s - 2157.57s | off the onboard avionics networks, right? 2157.57s - 2161.77s | So hearing 664, MIL-STD-1553, 2161.77s - 2163.93s | the inner LRU communications 2164.77s - 2167.81s | because if you were to think a little bit differently 2167.81s - 2170.29s | about the aircraft, that data allows you 2170.29s - 2173.05s | to treat each aircraft as a sensing system. 2173.05s - 2175.69s | And any data scientists will tell you 2175.69s - 2177.53s | that the more data that they have, 2177.53s - 2179.93s | obviously the better that they can train predictive, 2179.93s - 2182.13s | maintenance models, the better that you can train 2182.13s - 2186.17s | and identify potential predictive indicators of threats, 2186.17s - 2189.49s | especially as certain adversaries in some cases 2189.49s - 2190.93s | have demonstrated over and over again, 2190.93s - 2195.73s | the ability to actually test emergent capabilities. 2195.73s - 2198.81s | Like you want to be able to identify that 2198.81s - 2201.93s | before an entire fleet is potentially impacted. 2201.93s - 2205.77s | So maybe this data is all collected, 2205.77s - 2209.93s | it's cached and then it's recovered like on the ground 2209.93s - 2211.45s | where it's less expensive to do so 2211.45s - 2213.09s | with some near wireless means 2213.09s - 2216.97s | as opposed to clogging up like an expensive SATCOM link. 2216.97s - 2220.41s | But you can imagine how useful that would be 2220.41s - 2222.13s | potentially for a security use case. 2222.13s - 2225.17s | I think the compelling economic reason to do that 2225.17s - 2227.29s | will certainly be like maintenance 2227.29s - 2229.69s | and continuous evaluation of like updates 2229.69s - 2231.65s | or feature identification initially, 2231.65s - 2234.65s | but certainly there's a safety 2234.65s - 2236.53s


chunk_text:
- 2186.17s | and identify potential predictive indicators of threats, 2186.17s - 2189.49s | especially as certain adversaries in some cases 2189.49s - 2190.93s | have demonstrated over and over again, 2190.93s - 2195.73s | the ability to actually test emergent capabilities. 2195.73s - 2198.81s | Like you want to be able to identify that 2198.81s - 2201.93s | before an entire fleet is potentially impacted. 2201.93s - 2205.77s | So maybe this data is all collected, 2205.77s - 2209.93s | it's cached and then it's recovered like on the ground 2209.93s - 2211.45s | where it's less expensive to do so 2211.45s - 2213.09s | with some near wireless means 2213.09s - 2216.97s | as opposed to clogging up like an expensive SATCOM link. 2216.97s - 2220.41s | But you can imagine how useful that would be 2220.41s - 2222.13s | potentially for a security use case. 2222.13s - 2225.17s | I think the compelling economic reason to do that 2225.17s - 2227.29s | will certainly be like maintenance 2227.29s - 2229.69s | and continuous evaluation of like updates 2229.69s - 2231.65s | or feature identification initially, 2231.65s - 2234.65s | but certainly there's a safety 2234.65s - 2236.53s | and a security element to that 2236.53s - 2239.57s | that I don't think is fully appreciated yet. 2239.57s - 2240.53s | Sir. 2240.53s - 2274.27s | So in the industry, is this problem going to get worse? 2274.27s - 2277.35s | Is this something, are there mitigations in place 2277.35s - 2278.19s | that are gonna stop this? 2279.60s - 2281.64s | I mean, kind of a crude example is 2281.64s - 2284.04s | if I see my level of pressure indicated about this, 2284.04s - 2285.40s | is that because my sensor failed 2285.40s - 2287.16s | or is that because the gravity's spinning things 2287.16s - 2289.32s | are gonna stop the way gravity spins? 2289.32s - 2291.32s | So that's a situation where an indicator 2291.32s - 2294.60s | is going to all of a sudden become much worse situation. 2294.60s - 2296.64s | So to me, I think that's probably, 2296.64s - 2298.16s | and we're a long ways from that. 2298.16s - 2299.80s | We're gonna have to gather a lot of this data 2299.80s - 2303.28s | to learn that we can now begin to identify 2303.28s - 2305.52s | this problem


chunk_text:
| and a security element to that 2236.53s - 2239.57s | that I don't think is fully appreciated yet. 2239.57s - 2240.53s | Sir. 2240.53s - 2274.27s | So in the industry, is this problem going to get worse? 2274.27s - 2277.35s | Is this something, are there mitigations in place 2277.35s - 2278.19s | that are gonna stop this? 2279.60s - 2281.64s | I mean, kind of a crude example is 2281.64s - 2284.04s | if I see my level of pressure indicated about this, 2284.04s - 2285.40s | is that because my sensor failed 2285.40s - 2287.16s | or is that because the gravity's spinning things 2287.16s - 2289.32s | are gonna stop the way gravity spins? 2289.32s - 2291.32s | So that's a situation where an indicator 2291.32s - 2294.60s | is going to all of a sudden become much worse situation. 2294.60s - 2296.64s | So to me, I think that's probably, 2296.64s - 2298.16s | and we're a long ways from that. 2298.16s - 2299.80s | We're gonna have to gather a lot of this data 2299.80s - 2303.28s | to learn that we can now begin to identify 2303.28s - 2305.52s | this problem is not contained. 2305.52s - 2309.24s | This problem is going to impact more flight systems 2310.04s - 2311.44s | and now the urgency of my decision-making 2311.44s - 2314.04s | is gonna be a fundamentally different thing. 2314.04s - 2316.08s | I think this is really exciting. 2316.08s - 2317.72s | The thing that I'm just concerned about 2317.72s - 2321.16s | is that we can't even do that on terrestrial systems 2321.16s - 2324.00s | without, we can't even do that on enterprise systems. 2324.00s - 2326.64s | I don't know how we're gonna do it on aircraft 2326.64s - 2330.60s | without folding our mental image of how we handle it. 2330.60s - 2332.76s | And maybe a fourth thing to consider too, 2332.76s - 2334.84s | and this one's totally up for debate, 2334.84s - 2337.84s | so contentious concepts here, but what about intent? 2339.96s - 2344.28s | Would it impact decision-making in the cockpit 2344.28s - 2347.92s | if you knew that systems were intentionally being affected 2347.92s - 2350.04s | or if it was an unintended thing, right? 2350.04s - 2352.64s | So if something was maintenance or if it was malicious. 2352.64s


chunk_text:
is not contained. 2305.52s - 2309.24s | This problem is going to impact more flight systems 2310.04s - 2311.44s | and now the urgency of my decision-making 2311.44s - 2314.04s | is gonna be a fundamentally different thing. 2314.04s - 2316.08s | I think this is really exciting. 2316.08s - 2317.72s | The thing that I'm just concerned about 2317.72s - 2321.16s | is that we can't even do that on terrestrial systems 2321.16s - 2324.00s | without, we can't even do that on enterprise systems. 2324.00s - 2326.64s | I don't know how we're gonna do it on aircraft 2326.64s - 2330.60s | without folding our mental image of how we handle it. 2330.60s - 2332.76s | And maybe a fourth thing to consider too, 2332.76s - 2334.84s | and this one's totally up for debate, 2334.84s - 2337.84s | so contentious concepts here, but what about intent? 2339.96s - 2344.28s | Would it impact decision-making in the cockpit 2344.28s - 2347.92s | if you knew that systems were intentionally being affected 2347.92s - 2350.04s | or if it was an unintended thing, right? 2350.04s - 2352.64s | So if something was maintenance or if it was malicious. 2352.64s - 2355.16s | So today, in the last year and a half, 2355.16s - 2356.60s | if you've been reading the news, 2356.60s - 2358.84s | there are thousands of flights every month 2358.84s - 2362.44s | that are impacted with GPS spoofing and jamming. 2362.44s - 2367.44s | And some MMRs, GPS receivers, when they get bad time 2367.72s - 2371.40s | from like a delayed GPS signal that's spoofing, 2371.40s - 2374.80s | it causes internal systems to fail and crash. 2374.80s - 2379.12s | This can, if you know that that is happening intentionally 2379.12s - 2384.12s | as opposed to just a maintenance anomaly, 2384.40s - 2386.24s | it can impact your decision-making 2386.24s - 2390.44s | on if you proceed, divert, whatever the case may be. 2391.36s - 2395.68s | That's an insight that I heard from some pilots recently 2395.72s - 2397.44s | that were sharing an example 2397.44s - 2399.08s | where coming back from the Middle East 2399.08s - 2404.08s | where they knew to expect some GPS interference, 2404.76s - 2409.16s | lost some systems, but they were expecting that. 2409.16s - 2412.04s | What they weren't expecting was that they


chunk_text:
- 2355.16s | So today, in the last year and a half, 2355.16s - 2356.60s | if you've been reading the news, 2356.60s - 2358.84s | there are thousands of flights every month 2358.84s - 2362.44s | that are impacted with GPS spoofing and jamming. 2362.44s - 2367.44s | And some MMRs, GPS receivers, when they get bad time 2367.72s - 2371.40s | from like a delayed GPS signal that's spoofing, 2371.40s - 2374.80s | it causes internal systems to fail and crash. 2374.80s - 2379.12s | This can, if you know that that is happening intentionally 2379.12s - 2384.12s | as opposed to just a maintenance anomaly, 2384.40s - 2386.24s | it can impact your decision-making 2386.24s - 2390.44s | on if you proceed, divert, whatever the case may be. 2391.36s - 2395.68s | That's an insight that I heard from some pilots recently 2395.72s - 2397.44s | that were sharing an example 2397.44s - 2399.08s | where coming back from the Middle East 2399.08s - 2404.08s | where they knew to expect some GPS interference, 2404.76s - 2409.16s | lost some systems, but they were expecting that. 2409.16s - 2412.04s | What they weren't expecting was that they would have to fly 2412.04s - 2414.04s | at a lower flight level over the Atlantic, 2414.04s - 2415.40s | and that impacted their fuel burn 2415.40s - 2417.48s | because when you fly at higher altitudes, 2417.48s - 2420.32s | you get different fuel burn efficiencies. 2420.32s - 2424.24s | Well, they had not expected that, 2424.24s - 2427.80s | and so when they reran all of their range calculations, 2427.80s - 2429.76s | they were within safety limits 2429.76s - 2431.60s | to make it to their final destination, 2431.60s - 2434.64s | but had to constantly weigh and consider a divert. 2435.48s - 2437.12s | Knowing that that was coming 2437.12s - 2439.32s | and knowing that that was a malicious activity 2439.32s - 2443.24s | was really impactful because they were expecting it 2443.24s - 2446.92s | and they were able to monitor and make changes. 2446.92s - 2449.64s | When the unexpected happens, sometimes it's useful 2449.64s - 2454.00s | and informs your decision-making if this is malicious. 2454.40s - 2457.84s | That's another element to the crew alert conversation 2457.84s - 2458.96s | that's extremely nuanced, 2458.96s - 2460.96s | and I don't think


chunk_text:
would have to fly 2412.04s - 2414.04s | at a lower flight level over the Atlantic, 2414.04s - 2415.40s | and that impacted their fuel burn 2415.40s - 2417.48s | because when you fly at higher altitudes, 2417.48s - 2420.32s | you get different fuel burn efficiencies. 2420.32s - 2424.24s | Well, they had not expected that, 2424.24s - 2427.80s | and so when they reran all of their range calculations, 2427.80s - 2429.76s | they were within safety limits 2429.76s - 2431.60s | to make it to their final destination, 2431.60s - 2434.64s | but had to constantly weigh and consider a divert. 2435.48s - 2437.12s | Knowing that that was coming 2437.12s - 2439.32s | and knowing that that was a malicious activity 2439.32s - 2443.24s | was really impactful because they were expecting it 2443.24s - 2446.92s | and they were able to monitor and make changes. 2446.92s - 2449.64s | When the unexpected happens, sometimes it's useful 2449.64s - 2454.00s | and informs your decision-making if this is malicious. 2454.40s - 2457.84s | That's another element to the crew alert conversation 2457.84s - 2458.96s | that's extremely nuanced, 2458.96s - 2460.96s | and I don't think we have any questions 2460.96s - 2462.40s | answered around that. 2462.40s - 2465.08s | But yeah, really, really tough stuff. 2465.08s - 2467.36s | As it relates to policy, 2467.36s - 2470.00s | I think getting the right frameworks, 2470.00s - 2471.88s | the right committees, and the right people in the room 2471.88s - 2475.24s | to wrestle and grapple with these concepts 2475.24s - 2478.28s | and see what we can learn from diverse experiences 2478.28s - 2481.88s | is probably the best way to arrive at the best answer, 2481.88s - 2483.80s | and that best answer's almost certainly 2483.80s - 2485.88s | going to evolve over time 2485.88s - 2488.12s | as threats evolve, as the technology evolves. 2489.00s - 2490.12s | In the meantime, a lot of people 2490.12s - 2492.68s | are just gonna be deeply unsatisfied with the status quo, 2492.68s - 2495.76s | so fantastic. 2495.76s - 2542.37s | Sir? 2542.37s - 2547.13s | Did you open that up? 2547.13s - 2549.45s | I think disclosure may be something 2549.45s - 2553.01s | like controlling disclosure processes 2554.57s - 2556.49s | is a potential way to mitigate some of that, 2556.49s


chunk_text:
we have any questions 2460.96s - 2462.40s | answered around that. 2462.40s - 2465.08s | But yeah, really, really tough stuff. 2465.08s - 2467.36s | As it relates to policy, 2467.36s - 2470.00s | I think getting the right frameworks, 2470.00s - 2471.88s | the right committees, and the right people in the room 2471.88s - 2475.24s | to wrestle and grapple with these concepts 2475.24s - 2478.28s | and see what we can learn from diverse experiences 2478.28s - 2481.88s | is probably the best way to arrive at the best answer, 2481.88s - 2483.80s | and that best answer's almost certainly 2483.80s - 2485.88s | going to evolve over time 2485.88s - 2488.12s | as threats evolve, as the technology evolves. 2489.00s - 2490.12s | In the meantime, a lot of people 2490.12s - 2492.68s | are just gonna be deeply unsatisfied with the status quo, 2492.68s - 2495.76s | so fantastic. 2495.76s - 2542.37s | Sir? 2542.37s - 2547.13s | Did you open that up? 2547.13s - 2549.45s | I think disclosure may be something 2549.45s - 2553.01s | like controlling disclosure processes 2554.57s - 2556.49s | is a potential way to mitigate some of that, 2556.49s - 2558.49s | but I see some audience reactions. 2558.49s - 2560.93s | Does anybody in the audience have thoughts and opinions? 2560.93s - 2561.77s | Sir? 2561.77s - 2578.63s | So you can see there being a lot of students 2578.63s - 2580.51s | who would be engaged in this. 2580.51s - 2589.40s | The thing that I understand where he's coming from, 2589.40s - 2593.36s | obscurity fails, and it fails in the worst ways 2593.36s - 2597.88s | because when you only rely on security by obscurity, 2597.88s - 2600.72s | you don't inform the defenders either. 2600.72s - 2604.52s | And so at least if we can find these bugs, 2604.52s - 2607.08s | we don't have to worry that an adversary has found it 2607.08s - 2608.28s | and just hasn't told anyone 2608.32s - 2610.20s | because they're waiting for that day 2610.20s - 2613.48s | that they're gonna, speaking of 9-11, 2613.48s - 2614.84s | they've got all these people out there 2614.84s - 2616.40s | ready to insert a thumb drive 2616.40s - 2619.76s | at all of these maintenance bases across the country. 2619.76s - 2622.32s | They don't even know


chunk_text:
- 2558.49s | but I see some audience reactions. 2558.49s - 2560.93s | Does anybody in the audience have thoughts and opinions? 2560.93s - 2561.77s | Sir? 2561.77s - 2578.63s | So you can see there being a lot of students 2578.63s - 2580.51s | who would be engaged in this. 2580.51s - 2589.40s | The thing that I understand where he's coming from, 2589.40s - 2593.36s | obscurity fails, and it fails in the worst ways 2593.36s - 2597.88s | because when you only rely on security by obscurity, 2597.88s - 2600.72s | you don't inform the defenders either. 2600.72s - 2604.52s | And so at least if we can find these bugs, 2604.52s - 2607.08s | we don't have to worry that an adversary has found it 2607.08s - 2608.28s | and just hasn't told anyone 2608.32s - 2610.20s | because they're waiting for that day 2610.20s - 2613.48s | that they're gonna, speaking of 9-11, 2613.48s - 2614.84s | they've got all these people out there 2614.84s - 2616.40s | ready to insert a thumb drive 2616.40s - 2619.76s | at all of these maintenance bases across the country. 2619.76s - 2622.32s | They don't even know why they're doing it. 2622.32s - 2625.94s | And then that day comes and we're at war. 2625.94s - 2627.22s | So just summarizing, right? 2627.22s - 2631.22s | Security, obscurity, the counterpoint to research. 2631.22s - 2632.66s | How do you control it from getting out? 2632.66s - 2634.58s | Especially important, just to highlight, 2634.58s - 2637.18s | I think you missed the, it's really expensive 2637.18s - 2639.54s | and takes a long time to fix these systems. 2640.42s - 2641.90s | To your point, yeah, 2641.90s - 2643.94s | so these are the two battling components. 2643.94s - 2645.38s | I saw an opinion in the back. 2645.38s - 2647.98s | Sir, did you have something to add to that? 2647.98s - 2680.93s | One thing I'll share is, 2680.93s - 2685.12s | I don't know if this is working or not, it's fine. 2685.12s - 2686.68s | Yeah, yeah. 2686.68s - 2690.72s | One thing I noticed at least in the Air Force 2690.72s - 2693.48s | which consists of Air Force and Space Force 2693.48s - 2696.12s | is that basically telemetry collection 2696.12s - 2699.68s | was natural off of space systems. 2699.68s


chunk_text:
why they're doing it. 2622.32s - 2625.94s | And then that day comes and we're at war. 2625.94s - 2627.22s | So just summarizing, right? 2627.22s - 2631.22s | Security, obscurity, the counterpoint to research. 2631.22s - 2632.66s | How do you control it from getting out? 2632.66s - 2634.58s | Especially important, just to highlight, 2634.58s - 2637.18s | I think you missed the, it's really expensive 2637.18s - 2639.54s | and takes a long time to fix these systems. 2640.42s - 2641.90s | To your point, yeah, 2641.90s - 2643.94s | so these are the two battling components. 2643.94s - 2645.38s | I saw an opinion in the back. 2645.38s - 2647.98s | Sir, did you have something to add to that? 2647.98s - 2680.93s | One thing I'll share is, 2680.93s - 2685.12s | I don't know if this is working or not, it's fine. 2685.12s - 2686.68s | Yeah, yeah. 2686.68s - 2690.72s | One thing I noticed at least in the Air Force 2690.72s - 2693.48s | which consists of Air Force and Space Force 2693.48s - 2696.12s | is that basically telemetry collection 2696.12s - 2699.68s | was natural off of space systems. 2699.68s - 2702.08s | And the connection I made to that was because 2702.08s - 2704.32s | they're designed, at least Space Force doesn't really, 2704.32s - 2706.84s | as of now, send astronauts up. 2706.84s - 2709.64s | And so those are autonomous systems. 2709.64s - 2714.64s | And so that's probably a very good test bed for, 2715.12s - 2719.48s | it's naturally collecting and it's naturally instrumented 2720.40s - 2722.08s | because it has to be. 2722.08s - 2725.32s | And it naturally also solves for that piping problem 2725.32s - 2728.48s | because generally, not always, 2728.48s - 2729.64s | because if you don't have the pipes, 2729.64s - 2731.32s | then it's gonna just need to be autonomous 2731.32s - 2732.44s | without the pipes, right? 2732.44s - 2735.24s | So yes, we did see that. 2735.24s - 2736.68s | One thing I thought of later, 2736.68s - 2739.20s | I wanna try to answer your question a little bit better 2739.20s - 2740.04s | that I thought was relevant. 2740.04s - 2741.40s | I think it's come up a bit, 2741.40s - 2742.92s | but may not have been emphasized 2742.92s - 2745.24s | is


chunk_text:
- 2702.08s | And the connection I made to that was because 2702.08s - 2704.32s | they're designed, at least Space Force doesn't really, 2704.32s - 2706.84s | as of now, send astronauts up. 2706.84s - 2709.64s | And so those are autonomous systems. 2709.64s - 2714.64s | And so that's probably a very good test bed for, 2715.12s - 2719.48s | it's naturally collecting and it's naturally instrumented 2720.40s - 2722.08s | because it has to be. 2722.08s - 2725.32s | And it naturally also solves for that piping problem 2725.32s - 2728.48s | because generally, not always, 2728.48s - 2729.64s | because if you don't have the pipes, 2729.64s - 2731.32s | then it's gonna just need to be autonomous 2731.32s - 2732.44s | without the pipes, right? 2732.44s - 2735.24s | So yes, we did see that. 2735.24s - 2736.68s | One thing I thought of later, 2736.68s - 2739.20s | I wanna try to answer your question a little bit better 2739.20s - 2740.04s | that I thought was relevant. 2740.04s - 2741.40s | I think it's come up a bit, 2741.40s - 2742.92s | but may not have been emphasized 2742.92s - 2745.24s | is the predictive maintenance piece 2745.24s - 2747.08s | of this data for humans. 2747.08s - 2751.08s | So in contrast to autonomous systems, 2751.08s - 2754.20s | like while you have human lives at stake, 2754.20s - 2757.24s | what we have seen, at least in the Air Force, 2757.24s - 2759.92s | is that the automatic collection of this data, 2760.04s - 2763.40s | even if harvested and leveraged later, 2763.40s - 2768.16s | we may be able to gain insights and start to see signal 2768.16s - 2772.76s | that certain functions of a mission system 2772.76s - 2774.92s | are not operating at spec, 2774.92s - 2777.64s | or maybe they are operating at spec, 2777.64s - 2780.64s | what we thought was spec 30 years ago, 2780.64s - 2782.76s | but maybe that spec needs to be changed. 2782.76s - 2784.96s | So like the collection of that data 2784.96s - 2788.16s | allows us to make these systems safer. 2788.16s - 2792.52s | Contrasting that against the concerns of vulnerability 2792.52s - 2796.08s | or exploitation is tough to understand. 2796.08s - 2799.48s | It's about, I mean, the typical kind of cost calculation 2799.48s


chunk_text:
the predictive maintenance piece 2745.24s - 2747.08s | of this data for humans. 2747.08s - 2751.08s | So in contrast to autonomous systems, 2751.08s - 2754.20s | like while you have human lives at stake, 2754.20s - 2757.24s | what we have seen, at least in the Air Force, 2757.24s - 2759.92s | is that the automatic collection of this data, 2760.04s - 2763.40s | even if harvested and leveraged later, 2763.40s - 2768.16s | we may be able to gain insights and start to see signal 2768.16s - 2772.76s | that certain functions of a mission system 2772.76s - 2774.92s | are not operating at spec, 2774.92s - 2777.64s | or maybe they are operating at spec, 2777.64s - 2780.64s | what we thought was spec 30 years ago, 2780.64s - 2782.76s | but maybe that spec needs to be changed. 2782.76s - 2784.96s | So like the collection of that data 2784.96s - 2788.16s | allows us to make these systems safer. 2788.16s - 2792.52s | Contrasting that against the concerns of vulnerability 2792.52s - 2796.08s | or exploitation is tough to understand. 2796.08s - 2799.48s | It's about, I mean, the typical kind of cost calculation 2799.48s - 2804.48s | is risk times the impact of that risk coming true, right? 2807.16s - 2811.00s | So I don't know which one to weigh between the two, 2811.00s - 2814.60s | but I do know that in terms of safety of systems, 2814.60s - 2815.60s | the collection of data 2815.60s - 2818.08s | can allow you to make those systems safer. 2818.84s - 2819.68s | It also could save money 2819.68s - 2821.48s | because if we don't have things breaking in the air, 2821.48s - 2823.12s | to your point of losing oil pressure, 2823.12s - 2825.52s | if we are able to mitigate them on the ground 2825.52s - 2828.48s | before you fly, well, it saves people 2828.48s - 2829.92s | and it also could save money. 2829.92s - 2831.08s | Those are some kind of thoughts 2831.08s - 2833.04s | that have emerged from this discussion. 2834.80s - 2836.00s | I have an observation, 2836.00s - 2840.16s | and that is that in the emergent eVTOL market, 2840.16s - 2842.04s | every single one of those designs 2842.04s - 2844.04s | that I've interacted with their design teams, 2844.04s - 2847.60s | they


chunk_text:
- 2804.48s | is risk times the impact of that risk coming true, right? 2807.16s - 2811.00s | So I don't know which one to weigh between the two, 2811.00s - 2814.60s | but I do know that in terms of safety of systems, 2814.60s - 2815.60s | the collection of data 2815.60s - 2818.08s | can allow you to make those systems safer. 2818.84s - 2819.68s | It also could save money 2819.68s - 2821.48s | because if we don't have things breaking in the air, 2821.48s - 2823.12s | to your point of losing oil pressure, 2823.12s - 2825.52s | if we are able to mitigate them on the ground 2825.52s - 2828.48s | before you fly, well, it saves people 2828.48s - 2829.92s | and it also could save money. 2829.92s - 2831.08s | Those are some kind of thoughts 2831.08s - 2833.04s | that have emerged from this discussion. 2834.80s - 2836.00s | I have an observation, 2836.00s - 2840.16s | and that is that in the emergent eVTOL market, 2840.16s - 2842.04s | every single one of those designs 2842.04s - 2844.04s | that I've interacted with their design teams, 2844.04s - 2847.60s | they are planning as a basic requirement, 2847.60s - 2849.72s | as a basic feature of the system, 2849.72s - 2851.64s | full take avionic data collection 2851.64s - 2853.96s | and significant telemetry instrumentation, 2853.96s - 2857.00s | like significantly more advanced 2857.00s - 2859.48s | than anything that we see in manned, 2859.48s - 2862.88s | both at the airline, GA level, et cetera. 2863.96s - 2867.20s | They're not just doing that for tests and evaluation. 2867.20s - 2870.44s | In DOD and in civil aviation, 2870.44s - 2872.00s | historically how we built systems 2872.00s - 2876.16s | is when you have a prototype test aircraft, 2876.16s - 2877.68s | we put all these orange wire 2877.68s - 2879.40s | and orange boxes, data collectors, 2879.40s - 2881.48s | and we set everything up, 2881.48s - 2885.00s | but then we rip all of that out for size weight, 2885.00s - 2887.92s | for swap savings when things go into production. 2887.92s - 2890.08s | And I think that those industries 2890.08s - 2890.92s | have certainly learned, 2890.92s - 2892.88s | hey, there's all of these unknown unknowns 2892.88s - 2894.08s | and we just want


chunk_text:
are planning as a basic requirement, 2847.60s - 2849.72s | as a basic feature of the system, 2849.72s - 2851.64s | full take avionic data collection 2851.64s - 2853.96s | and significant telemetry instrumentation, 2853.96s - 2857.00s | like significantly more advanced 2857.00s - 2859.48s | than anything that we see in manned, 2859.48s - 2862.88s | both at the airline, GA level, et cetera. 2863.96s - 2867.20s | They're not just doing that for tests and evaluation. 2867.20s - 2870.44s | In DOD and in civil aviation, 2870.44s - 2872.00s | historically how we built systems 2872.00s - 2876.16s | is when you have a prototype test aircraft, 2876.16s - 2877.68s | we put all these orange wire 2877.68s - 2879.40s | and orange boxes, data collectors, 2879.40s - 2881.48s | and we set everything up, 2881.48s - 2885.00s | but then we rip all of that out for size weight, 2885.00s - 2887.92s | for swap savings when things go into production. 2887.92s - 2890.08s | And I think that those industries 2890.08s - 2890.92s | have certainly learned, 2890.92s - 2892.88s | hey, there's all of these unknown unknowns 2892.88s - 2894.08s | and we just want all of the data 2894.08s - 2896.64s | because we're convinced that it will satisfy 2896.64s - 2898.28s | all of these needs in the future. 2898.28s - 2901.60s | The big question I think for operators right now, 2901.60s - 2902.92s | like airlines is, 2902.92s - 2905.56s | is it worth retrofitting legacy fleets 2905.56s - 2908.32s | with some type of capability to do that? 2908.32s - 2911.76s | Is there sufficient like safety 2911.76s - 2916.56s | and economic use case to underwrite that investment? 2916.56s - 2922.41s | And that's some of the debate that I hear right now. 2922.41s - 2971.12s | In the back, yes, sir. 2971.12s - 2975.96s | So I think answering the question to GPS interference 2977.00s - 2980.28s | is extremely system nuanced, obviously. 2981.72s - 2983.96s | There's been an interesting trend in aviation 2983.96s - 2985.76s | over the last 20 years 2986.56s - 2989.44s | where any systems engineers don't hold me 2989.44s - 2991.12s | to the strict definitions of this, 2991.12s - 2993.44s | but you've seen increasing coupling 2993.44s - 2997.44s | of inertial navigation to discipline that 2997.44s - 3000.68s | with


chunk_text:
all of the data 2894.08s - 2896.64s | because we're convinced that it will satisfy 2896.64s - 2898.28s | all of these needs in the future. 2898.28s - 2901.60s | The big question I think for operators right now, 2901.60s - 2902.92s | like airlines is, 2902.92s - 2905.56s | is it worth retrofitting legacy fleets 2905.56s - 2908.32s | with some type of capability to do that? 2908.32s - 2911.76s | Is there sufficient like safety 2911.76s - 2916.56s | and economic use case to underwrite that investment? 2916.56s - 2922.41s | And that's some of the debate that I hear right now. 2922.41s - 2971.12s | In the back, yes, sir. 2971.12s - 2975.96s | So I think answering the question to GPS interference 2977.00s - 2980.28s | is extremely system nuanced, obviously. 2981.72s - 2983.96s | There's been an interesting trend in aviation 2983.96s - 2985.76s | over the last 20 years 2986.56s - 2989.44s | where any systems engineers don't hold me 2989.44s - 2991.12s | to the strict definitions of this, 2991.12s - 2993.44s | but you've seen increasing coupling 2993.44s - 2997.44s | of inertial navigation to discipline that 2997.44s - 3000.68s | with external GNSS signaling. 3000.68s - 3003.20s | And when these systems are completely independent, 3003.20s - 3004.16s | that's fantastic. 3004.16s - 3007.20s | You lose one, you can roll over to the other. 3007.20s - 3009.44s | When they are highly coupled, 3009.44s - 3012.08s | obviously a problem with one flows into the other. 3012.08s - 3015.00s | And so it degrades the independence 3015.52s - 3017.04s | in the backup utility. 3017.04s - 3019.32s | We have seen that in a number of specific designs. 3019.32s - 3020.64s | I won't call out anybody, 3020.64s - 3023.16s | but a lot of really smart people right now 3023.16s - 3024.88s | are trying to implement fixes 3026.36s - 3028.04s | and separate these systems a little bit more 3028.04s - 3029.76s | because that seems like the natural thing. 3029.76s - 3031.52s | There were really good reasons, 3031.52s - 3034.36s | but I think, you know, 3034.36s - 3036.76s | like the industry learned that, 3036.76s - 3039.92s | boy, timing is really important 3039.92s - 3042.08s | and there's tremendous time sensitivity 3042.08s - 3043.88s | and synchronization on internal systems. 3043.92s - 3045.64s |


chunk_text:
external GNSS signaling. 3000.68s - 3003.20s | And when these systems are completely independent, 3003.20s - 3004.16s | that's fantastic. 3004.16s - 3007.20s | You lose one, you can roll over to the other. 3007.20s - 3009.44s | When they are highly coupled, 3009.44s - 3012.08s | obviously a problem with one flows into the other. 3012.08s - 3015.00s | And so it degrades the independence 3015.52s - 3017.04s | in the backup utility. 3017.04s - 3019.32s | We have seen that in a number of specific designs. 3019.32s - 3020.64s | I won't call out anybody, 3020.64s - 3023.16s | but a lot of really smart people right now 3023.16s - 3024.88s | are trying to implement fixes 3026.36s - 3028.04s | and separate these systems a little bit more 3028.04s - 3029.76s | because that seems like the natural thing. 3029.76s - 3031.52s | There were really good reasons, 3031.52s - 3034.36s | but I think, you know, 3034.36s - 3036.76s | like the industry learned that, 3036.76s - 3039.92s | boy, timing is really important 3039.92s - 3042.08s | and there's tremendous time sensitivity 3042.08s - 3043.88s | and synchronization on internal systems. 3043.92s - 3045.64s | And that wasn't like properly fuzzed 3045.64s - 3047.28s | or tested for interference 3047.28s - 3049.76s | because nobody really assumed that GPS spoofing 3049.76s - 3053.68s | would be such commonplace or directed against civil assets. 3053.68s - 3055.84s | And now we know that that's probably naive. 3056.92s - 3058.88s | So those are just some initial, 3058.88s - 3060.12s | I think thoughts that come to mind 3060.12s - 3063.00s | or at least perceptions from my point of view. 3063.00s - 3064.60s | There's really good reporting out 3064.60s - 3069.60s | on how pervasive GPS spoofing is. 3069.84s - 3072.40s | I would say spoofing is far more impactful 3072.40s - 3075.32s | on aviation systems in my learned experience 3075.32s - 3077.12s | the last two years than jamming. 3077.12s - 3078.48s | Jamming's kind of fine. 3078.48s - 3080.44s | Like systems are actually really resilient 3080.44s - 3085.00s | to not getting lock and not being able to provide that. 3085.00s - 3088.16s | They're designed with all kinds of redundancies and backups. 3088.16s - 3091.08s | It's that false data injection, 3091.08s - 3091.96s | if you think of spoofing,


chunk_text:
And that wasn't like properly fuzzed 3045.64s - 3047.28s | or tested for interference 3047.28s - 3049.76s | because nobody really assumed that GPS spoofing 3049.76s - 3053.68s | would be such commonplace or directed against civil assets. 3053.68s - 3055.84s | And now we know that that's probably naive. 3056.92s - 3058.88s | So those are just some initial, 3058.88s - 3060.12s | I think thoughts that come to mind 3060.12s - 3063.00s | or at least perceptions from my point of view. 3063.00s - 3064.60s | There's really good reporting out 3064.60s - 3069.60s | on how pervasive GPS spoofing is. 3069.84s - 3072.40s | I would say spoofing is far more impactful 3072.40s - 3075.32s | on aviation systems in my learned experience 3075.32s - 3077.12s | the last two years than jamming. 3077.12s - 3078.48s | Jamming's kind of fine. 3078.48s - 3080.44s | Like systems are actually really resilient 3080.44s - 3085.00s | to not getting lock and not being able to provide that. 3085.00s - 3088.16s | They're designed with all kinds of redundancies and backups. 3088.16s - 3091.08s | It's that false data injection, 3091.08s - 3091.96s | if you think of spoofing, 3091.96s - 3094.20s | like within the skin of the plane 3094.20s - 3096.32s | that maybe wasn't appropriately tested. 3098.24s - 3102.12s | The issue is that on certain fleets that are flying today, 3102.68s - 3106.36s | you have a subset of those 3106.36s - 3109.60s | where you can't just drop a software or firmware fix 3109.60s - 3113.08s | on certain LRUs to evolve this. 3113.08s - 3115.00s | And so people are trying to figure out 3115.00s - 3117.26s | like how do they mitigate those risks now? 3118.20s - 3120.24s | But yeah, there's some really good reporting on this already 3120.24s - 3121.20s | that's out in the news. 3121.20s - 3179.82s | I think we have time for one last question, sir. 3179.82s - 3184.10s | Yeah, I am a huge fan and fanatical advocate 3184.10s - 3185.78s | for we need to educate pilots 3185.78s - 3188.22s | to their software dependencies 3188.22s - 3189.94s | so that they really pay attention 3189.94s - 3192.46s | and understand that reverting back 3192.46s - 3196.58s | to secondary and tertiary navigation 3196.58s - 3200.02s | and flying processes that they're already taught,


chunk_text:
3091.96s - 3094.20s | like within the skin of the plane 3094.20s - 3096.32s | that maybe wasn't appropriately tested. 3098.24s - 3102.12s | The issue is that on certain fleets that are flying today, 3102.68s - 3106.36s | you have a subset of those 3106.36s - 3109.60s | where you can't just drop a software or firmware fix 3109.60s - 3113.08s | on certain LRUs to evolve this. 3113.08s - 3115.00s | And so people are trying to figure out 3115.00s - 3117.26s | like how do they mitigate those risks now? 3118.20s - 3120.24s | But yeah, there's some really good reporting on this already 3120.24s - 3121.20s | that's out in the news. 3121.20s - 3179.82s | I think we have time for one last question, sir. 3179.82s - 3184.10s | Yeah, I am a huge fan and fanatical advocate 3184.10s - 3185.78s | for we need to educate pilots 3185.78s - 3188.22s | to their software dependencies 3188.22s - 3189.94s | so that they really pay attention 3189.94s - 3192.46s | and understand that reverting back 3192.46s - 3196.58s | to secondary and tertiary navigation 3196.58s - 3200.02s | and flying processes that they're already taught, 3200.02s - 3202.02s | like keep those skills sharp, 3202.02s - 3204.42s | expect systems potentially to be lost 3204.42s - 3208.54s | and put a little bit of thought to like the fact that, 3208.54s - 3213.06s | yeah, there's a cyber dependency. 3213.34s - 3216.10s | I think that the training conversation 3216.10s - 3218.62s | is really gonna evolve in terms of what that means. 3218.62s - 3222.22s | But today the practical guidance is, 3222.22s - 3224.22s | make sure that you can use your backup 3224.22s - 3226.74s | like analog systems to navigate and fly. 3227.90s - 3229.18s | But what that means in the future, 3229.18s - 3231.82s | if there's some type of cyber crew indication, 3231.82s - 3233.62s | what emergency procedures they go to 3233.62s - 3234.62s | and how that all evolves, 3234.62s - 3236.58s | I'm really excited to see how that happens. 3236.58s - 3239.02s | So we have time for one more question, sir. 3257.56s - 3259.52s | I have not, that sounds really scary. 3261.24s - 3272.56s | Yeah, just, I haven't, no. 3272.56s - 3274.28s | That is very evil. 3274.28s - 3276.12s |


chunk_text:
3200.02s - 3202.02s | like keep those skills sharp, 3202.02s - 3204.42s | expect systems potentially to be lost 3204.42s - 3208.54s | and put a little bit of thought to like the fact that, 3208.54s - 3213.06s | yeah, there's a cyber dependency. 3213.34s - 3216.10s | I think that the training conversation 3216.10s - 3218.62s | is really gonna evolve in terms of what that means. 3218.62s - 3222.22s | But today the practical guidance is, 3222.22s - 3224.22s | make sure that you can use your backup 3224.22s - 3226.74s | like analog systems to navigate and fly. 3227.90s - 3229.18s | But what that means in the future, 3229.18s - 3231.82s | if there's some type of cyber crew indication, 3231.82s - 3233.62s | what emergency procedures they go to 3233.62s - 3234.62s | and how that all evolves, 3234.62s - 3236.58s | I'm really excited to see how that happens. 3236.58s - 3239.02s | So we have time for one more question, sir. 3257.56s - 3259.52s | I have not, that sounds really scary. 3261.24s - 3272.56s | Yeah, just, I haven't, no. 3272.56s - 3274.28s | That is very evil. 3274.28s - 3276.12s | Thank you for giving me another reason 3276.12s - 3277.36s | to stay up at night. 3277.36s - 3280.00s | Stewart, any last comments before we wrap it up? 3280.00s - 3281.32s | Hey, everybody, thank you so much. 3281.32s - 3282.92s | Really appreciate it, take care.


chunk_text:
Thank you for giving me another reason 3276.12s - 3277.36s | to stay up at night. 3277.36s - 3280.00s | Stewart, any last comments before we wrap it up? 3280.00s - 3281.32s | Hey, everybody, thank you so much. 3281.32s - 3282.92s | Really appreciate it, take care.


chunk_text:
{ "webpage_url": "https://www.youtube.com/watch?v=h8E6w3vVzWo", "title": "DEF CON 32 - Exposing Attacks Hiding in the Sheer Noise of False Positives - E Tahoun, L Hamida", "description": "In the complex landscape of modern cybersecurity, identifying coordinated attacks within massive volumes of security data is a formidable challenge. Security professionals often grapple with distinguishing these attacks from numerous false positives and isolated incidents. This talk will illuminate how data science can be harnessed to transform tons of heterogeneous events, logs, and alerts into a bunch of clusters, a few kill chains, and fewer actionable insights, with open-source models, and security knowledge encoding. Join us on a journey to enhance security operations efficacy and efficiency! No data science expertise is required!", "channel_url": "https://www.youtube.com/channel/UC6Om9kAkl32dWlDSNlDS9Iw", "duration": 3323, "channel": "DEFCONConference", "uploader": "DEFCONConference", "upload_date": "20241016" } 0.00s - 5.32s | This text was transcribed using whisper model: large-v2 I'm Ming from the Packet Hacking Village, and this last and final talk of the day is 5.32s - 10.28s | uh, sponsored by the Packet Hacking Village, and it is our pleasure to introduce to you 10.28s - 11.28s | Ez Tahoun! 11.28s - 12.28s | Let's just talk. 12.28s - 17.50s | Hello, hello. 17.50s - 18.50s | Hey, everybody. 18.50s - 19.50s | My name is Ez. 19.50s - 24.90s | I'm here to talk a little, a little, about AI, because I know everybody got baited here 24.90s - 28.70s | into AI, but the more the merrier, you know. 28.70s - 33.82s | I'll talk more about the use case, less about AI, but before we start, how is everybody's 33.82s - 34.82s | DevCon? 34.82s - 35.82s | Yeah? 35.82s - 36.82s | Alright. 36.82s - 38.78s | Hell yeah. 38.78s - 42.26s | Are you guys more on the blue team side? 42.26s - 43.26s | Anybody here on the blue team? 43.26s - 44.26s | Blue teamers? 44.26s - 45.26s | Holy moly. 45.26s - 46.26s | Alright. 46.26s - 47.26s | The whole, this side is blue team? 47.26s - 48.26s | This side is red team? 48.26s - 49.26s | Who's on this side? 49.26s - 50.26s | Are you guys red teamers, by any chance? 50.26s - 51.26s | Okay. 51.26s - 52.26s | What the hell is going on on this side? 52.26s - 53.26s | Are you guys, like, cloud engineers? 53.26s - 54.26s | Network engineers? 54.26s


chunk_text:
everybody. 18.50s - 19.50s | My name is Ez. 19.50s - 24.90s | I'm here to talk a little, a little, about AI, because I know everybody got baited here 24.90s - 28.70s | into AI, but the more the merrier, you know. 28.70s - 33.82s | I'll talk more about the use case, less about AI, but before we start, how is everybody's 33.82s - 34.82s | DevCon? 34.82s - 35.82s | Yeah? 35.82s - 36.82s | Alright. 36.82s - 38.78s | Hell yeah. 38.78s - 42.26s | Are you guys more on the blue team side? 42.26s - 43.26s | Anybody here on the blue team? 43.26s - 44.26s | Blue teamers? 44.26s - 45.26s | Holy moly. 45.26s - 46.26s | Alright. 46.26s - 47.26s | The whole, this side is blue team? 47.26s - 48.26s | This side is red team? 48.26s - 49.26s | Who's on this side? 49.26s - 50.26s | Are you guys red teamers, by any chance? 50.26s - 51.26s | Okay. 51.26s - 52.26s | What the hell is going on on this side? 52.26s - 53.26s | Are you guys, like, cloud engineers? 53.26s - 54.26s | Network engineers? 54.26s - 55.26s | Purple? 55.26s - 56.26s | Lots of purples. 56.26s - 57.26s | Data scientists? 57.26s - 58.26s | Lots of data scientists. 58.26s - 59.26s | Alright. 59.26s - 60.26s | Email security? 60.26s - 62.93s | Goddamn. 62.93s - 63.93s | Alright. 63.93s - 64.93s | That's pretty cool. 64.93s - 65.93s | Alright. 65.93s - 66.93s | So, a lot of blue teamers. 66.93s - 67.93s | A few data scientists made it in. 67.93s - 68.93s | So, thank you for, you know, coming into a talk that has AI in the title. 68.93s - 69.93s | I know that, that doesn't work. 69.93s - 70.93s | People usually get scared of that word. 70.93s - 71.93s | I remember at Black Hat, I got sold to while I was pissing in a urinal, and it was an AI. 71.93s - 72.93s | So, I was like, uh, you know, like, there's a lot of AI, and you don't know what to do 72.93s - 73.93s | with it. 73.93s - 74.93s | So, I was like, uh, you know, like, there's a lot of AI, and you don't know what to do 74.93s


chunk_text:
- 55.26s | Purple? 55.26s - 56.26s | Lots of purples. 56.26s - 57.26s | Data scientists? 57.26s - 58.26s | Lots of data scientists. 58.26s - 59.26s | Alright. 59.26s - 60.26s | Email security? 60.26s - 62.93s | Goddamn. 62.93s - 63.93s | Alright. 63.93s - 64.93s | That's pretty cool. 64.93s - 65.93s | Alright. 65.93s - 66.93s | So, a lot of blue teamers. 66.93s - 67.93s | A few data scientists made it in. 67.93s - 68.93s | So, thank you for, you know, coming into a talk that has AI in the title. 68.93s - 69.93s | I know that, that doesn't work. 69.93s - 70.93s | People usually get scared of that word. 70.93s - 71.93s | I remember at Black Hat, I got sold to while I was pissing in a urinal, and it was an AI. 71.93s - 72.93s | So, I was like, uh, you know, like, there's a lot of AI, and you don't know what to do 72.93s - 73.93s | with it. 73.93s - 74.93s | So, I was like, uh, you know, like, there's a lot of AI, and you don't know what to do 74.93s - 75.93s | with it. 75.93s - 76.93s | So, I was like, uh, you know, like, there's a lot of AI, and you don't know what to do 76.93s - 77.93s | with it. 77.93s - 79.93s | I know that, that doesn't work. 79.93s - 81.85s | People usually get scared of that word. 81.85s - 86.73s | I remember at Black Hat, I got sold to while I was pissing in a urinal, and it was an AI. 86.73s - 92.17s | So, I was like, uh, you know, like, there's a lot of AI, and you don't know what the hell 92.17s - 93.77s | works and what the hell doesn't. 93.77s - 97.73s | But the right framework that I will try to push on you today is essentially, talk about 97.73s - 102.29s | the use case, less about the tool, convince me about the use case first, and then we'll 102.29s - 103.79s | figure out how to do it. 103.79s - 104.79s | Right? 104.79s - 108.03s | So my talk is going to be all about context is all you need. 108.03s - 110.59s | I have a few practitioners here.


chunk_text:
- 75.93s | with it. 75.93s - 76.93s | So, I was like, uh, you know, like, there's a lot of AI, and you don't know what to do 76.93s - 77.93s | with it. 77.93s - 79.93s | I know that, that doesn't work. 79.93s - 81.85s | People usually get scared of that word. 81.85s - 86.73s | I remember at Black Hat, I got sold to while I was pissing in a urinal, and it was an AI. 86.73s - 92.17s | So, I was like, uh, you know, like, there's a lot of AI, and you don't know what the hell 92.17s - 93.77s | works and what the hell doesn't. 93.77s - 97.73s | But the right framework that I will try to push on you today is essentially, talk about 97.73s - 102.29s | the use case, less about the tool, convince me about the use case first, and then we'll 102.29s - 103.79s | figure out how to do it. 103.79s - 104.79s | Right? 104.79s - 108.03s | So my talk is going to be all about context is all you need. 108.03s - 110.59s | I have a few practitioners here. 110.59s - 113.07s | There's George Cizerou at Ruby. 113.07s - 116.23s | And there's Harry CIO at SAP AnnaSquare. 116.23s - 119.23s | They're essentially the people that I bring in whenever 119.23s - 122.23s | I have to get some validation on the problem. 122.23s - 125.75s | So I'll try to throw them a few curveballs every now and then. 125.75s - 127.71s | If you guys want to come up and do a quick intro, 127.71s - 128.63s | that's fine as well. 128.63s - 130.35s | You want to come up? 130.35s - 131.47s | Not feeling like it? 131.47s - 132.79s | Is everybody so tired at 5? 132.79s - 134.47s | Is that what's going on? 134.47s - 136.67s | You're drinking a Red Bull, and you're tired? 136.67s - 138.43s | What the hell is wrong with that Red Bull? 138.43s - 140.75s | Is it expired? 140.75s - 147.35s | There are stairs on this side. 147.35s - 147.85s | Go ahead. 147.85s - 148.35s | 100%. 148.35s - 150.31s | Go for it, Harry. 150.31s - 156.15s | Harry Sorori, NS2, SAP, supporting 156.15s - 159.19s | Department of Defense.


chunk_text:
110.59s - 113.07s | There's George Cizerou at Ruby. 113.07s - 116.23s | And there's Harry CIO at SAP AnnaSquare. 116.23s - 119.23s | They're essentially the people that I bring in whenever 119.23s - 122.23s | I have to get some validation on the problem. 122.23s - 125.75s | So I'll try to throw them a few curveballs every now and then. 125.75s - 127.71s | If you guys want to come up and do a quick intro, 127.71s - 128.63s | that's fine as well. 128.63s - 130.35s | You want to come up? 130.35s - 131.47s | Not feeling like it? 131.47s - 132.79s | Is everybody so tired at 5? 132.79s - 134.47s | Is that what's going on? 134.47s - 136.67s | You're drinking a Red Bull, and you're tired? 136.67s - 138.43s | What the hell is wrong with that Red Bull? 138.43s - 140.75s | Is it expired? 140.75s - 147.35s | There are stairs on this side. 147.35s - 147.85s | Go ahead. 147.85s - 148.35s | 100%. 148.35s - 150.31s | Go for it, Harry. 150.31s - 156.15s | Harry Sorori, NS2, SAP, supporting 156.15s - 159.19s | Department of Defense. 159.19s - 161.39s | It's about a $4.5 billion project 161.39s - 164.27s | that I'm heading over there. 164.27s - 166.87s | Lots of fun things, especially when 167.39s - 171.11s | dealing with defense and all that good stuff. 171.11s - 173.59s | There's a lot of moving parts there. 177.87s - 178.63s | Hi, everyone. 178.63s - 180.07s | My name is George Alcora. 180.07s - 183.91s | I'm the CISO for Ruby Life. 183.91s - 186.91s | I like to tell people that in my job, 186.91s - 189.87s | I work to help people get securely laid. 189.87s - 193.07s | I'm the CISO for, well, you've probably 193.07s - 195.11s | heard the brand, Ashley Madison. 195.11s - 196.67s | So we run online dating. 196.97s - 198.51s | Ashley Madison is our flagship brand. 198.51s - 200.35s | So that's where I work. 200.35s - 204.27s | In terms of this, well, when I started my cybersecurity 204.27s - 206.99s | career, I worked as a security operations analyst. 206.99s - 208.71s | And I dealt with ArcSight. 208.71s - 210.59s | If you guys remember ArcSight? 210.59s - 211.31s | Oh, yeah.


chunk_text:
159.19s - 161.39s | It's about a $4.5 billion project 161.39s - 164.27s | that I'm heading over there. 164.27s - 166.87s | Lots of fun things, especially when 167.39s - 171.11s | dealing with defense and all that good stuff. 171.11s - 173.59s | There's a lot of moving parts there. 177.87s - 178.63s | Hi, everyone. 178.63s - 180.07s | My name is George Alcora. 180.07s - 183.91s | I'm the CISO for Ruby Life. 183.91s - 186.91s | I like to tell people that in my job, 186.91s - 189.87s | I work to help people get securely laid. 189.87s - 193.07s | I'm the CISO for, well, you've probably 193.07s - 195.11s | heard the brand, Ashley Madison. 195.11s - 196.67s | So we run online dating. 196.97s - 198.51s | Ashley Madison is our flagship brand. 198.51s - 200.35s | So that's where I work. 200.35s - 204.27s | In terms of this, well, when I started my cybersecurity 204.27s - 206.99s | career, I worked as a security operations analyst. 206.99s - 208.71s | And I dealt with ArcSight. 208.71s - 210.59s | If you guys remember ArcSight? 210.59s - 211.31s | Oh, yeah. 211.31s - 212.83s | Remember ArcSight? 212.83s - 214.75s | So this is going to be a really relevant talk. 214.75s - 216.95s | And Ezeldine's been killing it this afternoon, guys. 216.95s - 217.79s | So please enjoy. 217.79s - 218.47s | 100%. 218.47s - 219.19s | I appreciate you. 219.19s - 219.67s | Please grab a seat. 219.67s - 220.83s | Can I make a disclaimer? 220.83s - 221.87s | Oh, more disclaimers. 221.87s - 223.41s | I don't know if the mics are working. 223.41s - 225.43s | Are the mics working, folks? 225.43s - 227.43s | If you may enable them, that would be fantastic. 227.43s - 232.39s | This way, they don't have to come back and forth. 232.39s - 233.55s | Disclaimer is all yours. 233.55s - 236.71s | I'm not a subscriber of Ashley Madison, 236.71s - 238.91s | so I could keep my job. 238.91s - 239.67s | Yes. 239.67s - 240.17s | Yes. 240.17s - 241.47s | I'll find your profile here. 241.47s - 242.11s | Right. 242.11s - 244.15s | None of us are in that breach, huh? 244.15s - 246.23s | OK. 246.23s - 247.91s | So the reason


chunk_text:
211.31s - 212.83s | Remember ArcSight? 212.83s - 214.75s | So this is going to be a really relevant talk. 214.75s - 216.95s | And Ezeldine's been killing it this afternoon, guys. 216.95s - 217.79s | So please enjoy. 217.79s - 218.47s | 100%. 218.47s - 219.19s | I appreciate you. 219.19s - 219.67s | Please grab a seat. 219.67s - 220.83s | Can I make a disclaimer? 220.83s - 221.87s | Oh, more disclaimers. 221.87s - 223.41s | I don't know if the mics are working. 223.41s - 225.43s | Are the mics working, folks? 225.43s - 227.43s | If you may enable them, that would be fantastic. 227.43s - 232.39s | This way, they don't have to come back and forth. 232.39s - 233.55s | Disclaimer is all yours. 233.55s - 236.71s | I'm not a subscriber of Ashley Madison, 236.71s - 238.91s | so I could keep my job. 238.91s - 239.67s | Yes. 239.67s - 240.17s | Yes. 240.17s - 241.47s | I'll find your profile here. 241.47s - 242.11s | Right. 242.11s - 244.15s | None of us are in that breach, huh? 244.15s - 246.23s | OK. 246.23s - 247.91s | So the reason I bring in these folks 247.91s - 249.91s | are essentially to talk more about the use case, 249.91s - 251.51s | like what the hell is going on. 251.51s - 254.67s | And usually, if I give them the mic, 254.67s - 256.51s | they start talking about incidents. 256.51s - 258.95s | And they tell you, oh, in this incident, 258.95s - 260.79s | we found so much shit. 260.79s - 262.27s | Here's how it started. 262.27s - 263.83s | Step one, we found this thing. 263.83s - 265.39s | Step two, there was this other thing. 265.39s - 267.71s | Step three, there was this other thing that we found out 267.71s - 268.87s | and we uncovered. 268.87s - 270.59s | And what they're essentially telling you 270.59s - 273.59s | is they have so much good investigators and their team 273.59s - 275.99s | and outstanding contextualization 275.99s - 278.03s | that these humans did. 278.03s - 280.95s | So context, apparently, is very important. 280.95s - 282.43s | And let's talk a little about that. 282.43s - 283.75s | You guys are all blue teamers, so I'm not 283.75s - 285.19s | going


chunk_text:
I bring in these folks 247.91s - 249.91s | are essentially to talk more about the use case, 249.91s - 251.51s | like what the hell is going on. 251.51s - 254.67s | And usually, if I give them the mic, 254.67s - 256.51s | they start talking about incidents. 256.51s - 258.95s | And they tell you, oh, in this incident, 258.95s - 260.79s | we found so much shit. 260.79s - 262.27s | Here's how it started. 262.27s - 263.83s | Step one, we found this thing. 263.83s - 265.39s | Step two, there was this other thing. 265.39s - 267.71s | Step three, there was this other thing that we found out 267.71s - 268.87s | and we uncovered. 268.87s - 270.59s | And what they're essentially telling you 270.59s - 273.59s | is they have so much good investigators and their team 273.59s - 275.99s | and outstanding contextualization 275.99s - 278.03s | that these humans did. 278.03s - 280.95s | So context, apparently, is very important. 280.95s - 282.43s | And let's talk a little about that. 282.43s - 283.75s | You guys are all blue teamers, so I'm not 283.75s - 285.19s | going to bore you with the bullshit. 285.19s - 287.19s | But here's a quick agenda. 287.19s - 288.87s | The agenda is, who the hell am I? 288.87s - 291.95s | And then some challenges, a lot of challenges, 291.95s - 293.35s | a little bit more. 293.35s - 295.55s | And we're going to talk a little about MITRE ATT&CK. 295.55s - 297.47s | Anybody knows what's MITRE ATT&CK? 297.47s - 299.07s | Oh, everybody knows. 299.07s - 300.35s | OK. 300.35s - 301.71s | I'll do a quick brief. 301.71s - 304.35s | And then we're going to talk about the what, the when, 304.35s - 305.19s | the who. 305.19s - 308.55s | When you do an investigation, what the hell happened? 308.55s - 309.91s | When did it happen? 309.91s - 310.87s | And who was involved? 310.87s - 312.27s | Tell me all the entities that are involved. 312.27s - 313.59s | Tell me what the hell happened. 313.63s - 315.23s | If you can do that with models, that's fantastic. 315.23s - 318.63s | If you can't, I mean, do it anyways. 318.63s - 320.59s | Do it the hard way, you know? 320.59s


chunk_text:
to bore you with the bullshit. 285.19s - 287.19s | But here's a quick agenda. 287.19s - 288.87s | The agenda is, who the hell am I? 288.87s - 291.95s | And then some challenges, a lot of challenges, 291.95s - 293.35s | a little bit more. 293.35s - 295.55s | And we're going to talk a little about MITRE ATT&CK. 295.55s - 297.47s | Anybody knows what's MITRE ATT&CK? 297.47s - 299.07s | Oh, everybody knows. 299.07s - 300.35s | OK. 300.35s - 301.71s | I'll do a quick brief. 301.71s - 304.35s | And then we're going to talk about the what, the when, 304.35s - 305.19s | the who. 305.19s - 308.55s | When you do an investigation, what the hell happened? 308.55s - 309.91s | When did it happen? 309.91s - 310.87s | And who was involved? 310.87s - 312.27s | Tell me all the entities that are involved. 312.27s - 313.59s | Tell me what the hell happened. 313.63s - 315.23s | If you can do that with models, that's fantastic. 315.23s - 318.63s | If you can't, I mean, do it anyways. 318.63s - 320.59s | Do it the hard way, you know? 320.59s - 321.87s | And then we're going to go through some examples. 321.87s - 325.35s | And I don't know why I put QA at the end. 325.35s - 328.71s | If you have questions, by all means, shout it out. 328.71s - 329.59s | I'm being serious. 329.59s - 330.83s | Shout it out. 330.83s - 333.31s | If you're so polite you cannot do that, just raise your hand. 333.31s - 334.67s | I will stop whatever the hell I'm saying, 334.67s - 335.91s | and I'll take your question. 335.91s - 337.11s | That's my promise. 337.11s - 337.99s | All right. 337.99s - 338.59s | Who the hell am I? 338.59s - 340.15s | I'm a data scientist. 340.15s - 342.11s | I used to be a data scientist at the SOC, 342.15s - 344.51s | Royal Bank of Canada, Hawaii Horn Cyber Defense, 344.51s - 346.03s | Forescout, a bunch of others. 346.03s - 347.99s | But essentially, I'm a nerd, 347.99s - 351.91s | and I do data science for the SOC. 351.91s - 353.39s | What's the problem with cybersecurity? 353.39s - 355.59s | Why does every year we have essentially 355.59s -


chunk_text:
- 321.87s | And then we're going to go through some examples. 321.87s - 325.35s | And I don't know why I put QA at the end. 325.35s - 328.71s | If you have questions, by all means, shout it out. 328.71s - 329.59s | I'm being serious. 329.59s - 330.83s | Shout it out. 330.83s - 333.31s | If you're so polite you cannot do that, just raise your hand. 333.31s - 334.67s | I will stop whatever the hell I'm saying, 334.67s - 335.91s | and I'll take your question. 335.91s - 337.11s | That's my promise. 337.11s - 337.99s | All right. 337.99s - 338.59s | Who the hell am I? 338.59s - 340.15s | I'm a data scientist. 340.15s - 342.11s | I used to be a data scientist at the SOC, 342.15s - 344.51s | Royal Bank of Canada, Hawaii Horn Cyber Defense, 344.51s - 346.03s | Forescout, a bunch of others. 346.03s - 347.99s | But essentially, I'm a nerd, 347.99s - 351.91s | and I do data science for the SOC. 351.91s - 353.39s | What's the problem with cybersecurity? 353.39s - 355.59s | Why does every year we have essentially 355.59s - 356.71s | a much bigger problem? 358.03s - 361.39s | I remember I did this similar talk about a year ago, 361.39s - 362.83s | and I was talking about Colonial Pipeline. 362.83s - 364.39s | This year I'm talking about CrowdStrike. 364.39s - 366.71s | Every year there's something that's way worse, right? 366.71s - 369.15s | So why do things keep on getting worse? 369.15s - 370.75s | And maybe CrowdStrike is not the best, 370.79s - 373.79s | but the other ones are not as popular. 373.79s - 376.63s | CrowdStrike seems to be popular this time around. 376.63s - 380.31s | But essentially, attacks seem to be a lot stealthier, right? 380.31s - 382.23s | And the folks here that work at Blue Team, 382.23s - 383.87s | they know the job is not very easy. 383.87s - 386.51s | They came up here because there was AI, 386.51s - 388.19s | and there was SOC, and they were like, fuck yeah. 388.19s - 390.11s | Somebody's gonna automate my job? 390.11s - 390.95s | Fuck yeah. 390.95s - 391.91s | Automate it, man. 391.91s - 393.91s | I wanna get another one. 393.91s - 395.87s


chunk_text:
356.71s | a much bigger problem? 358.03s - 361.39s | I remember I did this similar talk about a year ago, 361.39s - 362.83s | and I was talking about Colonial Pipeline. 362.83s - 364.39s | This year I'm talking about CrowdStrike. 364.39s - 366.71s | Every year there's something that's way worse, right? 366.71s - 369.15s | So why do things keep on getting worse? 369.15s - 370.75s | And maybe CrowdStrike is not the best, 370.79s - 373.79s | but the other ones are not as popular. 373.79s - 376.63s | CrowdStrike seems to be popular this time around. 376.63s - 380.31s | But essentially, attacks seem to be a lot stealthier, right? 380.31s - 382.23s | And the folks here that work at Blue Team, 382.23s - 383.87s | they know the job is not very easy. 383.87s - 386.51s | They came up here because there was AI, 386.51s - 388.19s | and there was SOC, and they were like, fuck yeah. 388.19s - 390.11s | Somebody's gonna automate my job? 390.11s - 390.95s | Fuck yeah. 390.95s - 391.91s | Automate it, man. 391.91s - 393.91s | I wanna get another one. 393.91s - 395.87s | Nobody really likes their job in the SOC 395.87s - 397.95s | because attacks are so fucking stealthy, 397.95s - 399.71s | and they grow stealthier every year. 399.71s - 400.59s | And guess what? 400.59s - 402.99s | You take the fall because you're the SOC, right? 404.15s - 406.11s | And because you're the SOC, you also have, 406.11s - 407.35s | I don't know how many alerts a day. 407.35s - 409.11s | I stopped updating that number, 409.11s - 410.51s | but that number keeps on increasing. 410.51s - 411.83s | But it doesn't matter, 411.83s - 412.87s | because at the end of the day as a human, 412.87s - 415.55s | you can only look at maybe like 20 tickets a day, right? 415.55s - 416.79s | 10 tickets. 416.79s - 419.07s | Anybody here looks at 20 or more? 420.31s - 421.15s | Yeah. 421.15s - 423.27s | There's one guy who's probably a fucking genius. 423.27s - 424.43s | He can look at more than 20. 424.43s - 428.35s | I actually blank out after five tickets. 428.35s - 429.51s | I used to work at the SOC. 429.51s - 431.99s


chunk_text:
| Nobody really likes their job in the SOC 395.87s - 397.95s | because attacks are so fucking stealthy, 397.95s - 399.71s | and they grow stealthier every year. 399.71s - 400.59s | And guess what? 400.59s - 402.99s | You take the fall because you're the SOC, right? 404.15s - 406.11s | And because you're the SOC, you also have, 406.11s - 407.35s | I don't know how many alerts a day. 407.35s - 409.11s | I stopped updating that number, 409.11s - 410.51s | but that number keeps on increasing. 410.51s - 411.83s | But it doesn't matter, 411.83s - 412.87s | because at the end of the day as a human, 412.87s - 415.55s | you can only look at maybe like 20 tickets a day, right? 415.55s - 416.79s | 10 tickets. 416.79s - 419.07s | Anybody here looks at 20 or more? 420.31s - 421.15s | Yeah. 421.15s - 423.27s | There's one guy who's probably a fucking genius. 423.27s - 424.43s | He can look at more than 20. 424.43s - 428.35s | I actually blank out after five tickets. 428.35s - 429.51s | I used to work at the SOC. 429.51s - 431.99s | Like after five tickets, everything sounds the same. 431.99s - 432.83s | You know, I just like, 432.83s - 436.03s | I wanna get by and my boss had this weird KPI of, 436.03s - 441.03s | you know, MTTD, MTTR, whatever bullshit. 441.03s - 443.27s | And it essentially just meant do it fast. 443.27s - 444.63s | No one cares. 444.63s - 446.55s | And they just did everything fast as you know, 446.55s - 449.59s | like false positive, false positive, false positive. 449.59s - 450.79s | Right? 450.79s - 451.67s | You don't want it fast, man. 451.67s - 453.15s | Sure, I'll get it to you. 453.15s - 455.59s | I'll get it in 20 seconds if I have to. 455.59s - 458.71s | But we have so many tools. 458.71s - 460.99s | I was telling you about Black Hat, terrible experience. 460.99s - 462.71s | There's like a shit ton of tools everywhere. 462.71s - 463.67s | Vendors are everywhere. 463.67s - 466.23s | Can't even find non-vendors anymore. 466.23s - 468.03s | But there's a lot of vendors. 468.03s - 469.27s | There's a lot of tools. 469.27s - 471.47s |


chunk_text:
| Like after five tickets, everything sounds the same. 431.99s - 432.83s | You know, I just like, 432.83s - 436.03s | I wanna get by and my boss had this weird KPI of, 436.03s - 441.03s | you know, MTTD, MTTR, whatever bullshit. 441.03s - 443.27s | And it essentially just meant do it fast. 443.27s - 444.63s | No one cares. 444.63s - 446.55s | And they just did everything fast as you know, 446.55s - 449.59s | like false positive, false positive, false positive. 449.59s - 450.79s | Right? 450.79s - 451.67s | You don't want it fast, man. 451.67s - 453.15s | Sure, I'll get it to you. 453.15s - 455.59s | I'll get it in 20 seconds if I have to. 455.59s - 458.71s | But we have so many tools. 458.71s - 460.99s | I was telling you about Black Hat, terrible experience. 460.99s - 462.71s | There's like a shit ton of tools everywhere. 462.71s - 463.67s | Vendors are everywhere. 463.67s - 466.23s | Can't even find non-vendors anymore. 466.23s - 468.03s | But there's a lot of vendors. 468.03s - 469.27s | There's a lot of tools. 469.27s - 471.47s | And somehow security is worse. 471.47s - 472.51s | How did that work? 472.51s - 473.35s | I don't know. 474.75s - 475.59s | Right? 486.12s - 488.00s | 100%, 100%. 488.00s - 489.28s | You're on top of it. 489.96s - 491.92s | Give this man a cookie, right? 491.92s - 493.72s | You're on top of it. 493.72s - 496.69s | It's a lot. 496.69s - 497.53s | Hell yeah. 498.49s - 501.57s | At 5 p.m. everybody wants a Red Bull, right? 501.57s - 503.49s | But yeah, we have a lot of tools. 503.49s - 504.41s | We have a lot of alerts. 504.41s - 505.45s | We have a lot of shit. 505.45s - 508.61s | And it's all up to not acquiring more. 508.61s - 511.61s | It's up to people and process, not technology anymore. 528.99s - 534.45s | Yeah, that's also presuming that over that three to four year 534.45s - 537.73s | cycle, you have the same staff and personnel manning your sock. 537.73s - 539.53s | Let's not forget about turnover, eh? 540.57s - 541.41s | Yeah. 547.19s - 548.99s | Dude, I'd fucking buy you a beer tonight. 548.99s


chunk_text:
And somehow security is worse. 471.47s - 472.51s | How did that work? 472.51s - 473.35s | I don't know. 474.75s - 475.59s | Right? 486.12s - 488.00s | 100%, 100%. 488.00s - 489.28s | You're on top of it. 489.96s - 491.92s | Give this man a cookie, right? 491.92s - 493.72s | You're on top of it. 493.72s - 496.69s | It's a lot. 496.69s - 497.53s | Hell yeah. 498.49s - 501.57s | At 5 p.m. everybody wants a Red Bull, right? 501.57s - 503.49s | But yeah, we have a lot of tools. 503.49s - 504.41s | We have a lot of alerts. 504.41s - 505.45s | We have a lot of shit. 505.45s - 508.61s | And it's all up to not acquiring more. 508.61s - 511.61s | It's up to people and process, not technology anymore. 528.99s - 534.45s | Yeah, that's also presuming that over that three to four year 534.45s - 537.73s | cycle, you have the same staff and personnel manning your sock. 537.73s - 539.53s | Let's not forget about turnover, eh? 540.57s - 541.41s | Yeah. 547.19s - 548.99s | Dude, I'd fucking buy you a beer tonight. 548.99s - 551.71s | Like, you know what's up and you speak it out. 551.71s - 552.91s | Like, I fucking love you, man. 552.91s - 554.15s | Like, that's what's up. 554.15s - 556.67s | So, hell yeah. 559.59s - 560.43s | Hell yeah. 560.43s - 561.75s | Yep, we need some of that. 561.75s - 562.59s | We need some of that. 562.59s - 564.55s | A lot of people are, you know. 564.55s - 565.39s | I mean. 577.60s - 580.08s | He was post-breach, though, yeah. 580.08s - 580.92s | He's post-breach. 580.92s - 584.64s | He came in to clean up, but dude, I was in fucking, 584.64s - 586.24s | I was in that fucking data set. 586.24s - 589.52s | I'm not, yep, yep. 589.52s - 591.96s | Well, yeah, the funny part is, I was actually, 591.96s - 594.96s | when that happened, I had started my career in cyber 594.96s - 596.80s | having left military SIGINT. 596.80s - 598.96s | So, I was working at a MSSP, 598.96s - 601.28s | which had a lot of Government of Canada clients. 601.28s - 603.76s | And so, the funny thing was in


chunk_text:
- 551.71s | Like, you know what's up and you speak it out. 551.71s - 552.91s | Like, I fucking love you, man. 552.91s - 554.15s | Like, that's what's up. 554.15s - 556.67s | So, hell yeah. 559.59s - 560.43s | Hell yeah. 560.43s - 561.75s | Yep, we need some of that. 561.75s - 562.59s | We need some of that. 562.59s - 564.55s | A lot of people are, you know. 564.55s - 565.39s | I mean. 577.60s - 580.08s | He was post-breach, though, yeah. 580.08s - 580.92s | He's post-breach. 580.92s - 584.64s | He came in to clean up, but dude, I was in fucking, 584.64s - 586.24s | I was in that fucking data set. 586.24s - 589.52s | I'm not, yep, yep. 589.52s - 591.96s | Well, yeah, the funny part is, I was actually, 591.96s - 594.96s | when that happened, I had started my career in cyber 594.96s - 596.80s | having left military SIGINT. 596.80s - 598.96s | So, I was working at a MSSP, 598.96s - 601.28s | which had a lot of Government of Canada clients. 601.28s - 603.76s | And so, the funny thing was in that breach list, 604.16s - 606.56s | there was a lot of at Canada dot C addresses. 606.56s - 608.52s | So, what a lot of people were doing, 608.52s - 610.00s | cheating on their spouses, 610.00s - 612.56s | were actually using their government email accounts, 612.56s - 614.56s | their government work email accounts, 614.56s - 617.76s | to get a platform or to get a profile on the site. 617.76s - 619.20s | But then when they got popped, 619.20s - 621.64s | not only was it an issue in terms of their spouse, 621.64s - 624.24s | but now their employer was pretty pissed off at them. 624.24s - 641.40s | And just saying, hey, that's Bitcoin. 641.40s - 642.24s | Yep. 642.24s - 643.76s | Well, there's been multiple. 643.76s - 646.47s | Multi-extortion. 646.47s - 647.95s | The funny thing is about that, 648.15s - 649.51s | I don't want to derail this talk, 649.51s - 652.67s | but like, the funny thing about that is to this day, 652.67s - 655.11s | I mean, they use that old list 655.11s - 657.11s | because Troy Hunt posted it, right? 657.11s - 660.43s


chunk_text:
that breach list, 604.16s - 606.56s | there was a lot of at Canada dot C addresses. 606.56s - 608.52s | So, what a lot of people were doing, 608.52s - 610.00s | cheating on their spouses, 610.00s - 612.56s | were actually using their government email accounts, 612.56s - 614.56s | their government work email accounts, 614.56s - 617.76s | to get a platform or to get a profile on the site. 617.76s - 619.20s | But then when they got popped, 619.20s - 621.64s | not only was it an issue in terms of their spouse, 621.64s - 624.24s | but now their employer was pretty pissed off at them. 624.24s - 641.40s | And just saying, hey, that's Bitcoin. 641.40s - 642.24s | Yep. 642.24s - 643.76s | Well, there's been multiple. 643.76s - 646.47s | Multi-extortion. 646.47s - 647.95s | The funny thing is about that, 648.15s - 649.51s | I don't want to derail this talk, 649.51s - 652.67s | but like, the funny thing about that is to this day, 652.67s - 655.11s | I mean, they use that old list 655.11s - 657.11s | because Troy Hunt posted it, right? 657.11s - 660.43s | So, everyone uses that list and tries to act like 660.43s - 663.39s | it's a new breach info or a new credential to pop off, 663.39s - 665.71s | but it's like still the 2015 credential. 665.71s - 668.71s | That person's moved on from the site well beyond, 668.71s - 672.11s | but they're still trying to run attempts on the old creds. 672.11s - 675.67s | So, I think it's just, it's a taboo thing. 676.67s - 677.75s | Everyone's been popped. 677.75s - 679.35s | SolarWinds been popped. 679.35s - 681.51s | Like British Airways has been popped. 681.51s - 683.23s | How many countless breaches have happened 683.23s - 684.91s | of major enterprises? 684.91s - 686.87s | Yeah, those are the ones we're talking about. 686.87s - 688.15s | Those are the ones we know about. 688.15s - 689.51s | Yeah. 689.51s - 691.11s | It's just cause it's got, 691.11s - 693.95s | it has to do with sex and everyone thinks it's taboo. 697.91s - 698.75s | Yep. 700.75s - 704.79s | Hell yeah. 704.79s - 706.39s | I appreciate your service, sir. 707.39s - 711.65s | But yeah, I mean,


chunk_text:
| So, everyone uses that list and tries to act like 660.43s - 663.39s | it's a new breach info or a new credential to pop off, 663.39s - 665.71s | but it's like still the 2015 credential. 665.71s - 668.71s | That person's moved on from the site well beyond, 668.71s - 672.11s | but they're still trying to run attempts on the old creds. 672.11s - 675.67s | So, I think it's just, it's a taboo thing. 676.67s - 677.75s | Everyone's been popped. 677.75s - 679.35s | SolarWinds been popped. 679.35s - 681.51s | Like British Airways has been popped. 681.51s - 683.23s | How many countless breaches have happened 683.23s - 684.91s | of major enterprises? 684.91s - 686.87s | Yeah, those are the ones we're talking about. 686.87s - 688.15s | Those are the ones we know about. 688.15s - 689.51s | Yeah. 689.51s - 691.11s | It's just cause it's got, 691.11s - 693.95s | it has to do with sex and everyone thinks it's taboo. 697.91s - 698.75s | Yep. 700.75s - 704.79s | Hell yeah. 704.79s - 706.39s | I appreciate your service, sir. 707.39s - 711.65s | But yeah, I mean, I can turn this into a SkyTalk 711.65s - 713.89s | and we can start ranting, you know, like, 713.89s - 715.93s | I can fucking rant. 715.93s - 716.77s | No, no, no. 716.77s - 717.61s | I'm- 717.61s - 719.33s | Is this turning into sock therapy? 719.33s - 720.81s | Sock therapy is valid too. 720.81s - 722.13s | Look up my fucking agenda, bro. 722.13s - 723.13s | It's like all SkyTalk. 723.13s - 724.17s | It's all ranting. 724.17s - 725.37s | Like the machine learning is like, 725.37s - 726.77s | you know, number 5.8. 728.77s - 729.93s | I'm all about the rant, bro. 729.93s - 732.21s | Like that's, it's the use case, right? 732.21s - 735.61s | It's like, what's broken and how do we fix it, right? 735.61s - 736.93s | But yeah. 737.53s - 738.77s | But yeah, you're on top of it. 738.77s - 742.13s | Like too many tools, too many rules, too many playbooks, 742.13s - 743.93s | too many whatever the fuck you want to call them. 743.93s - 746.65s | It's just searches, lookups, right? 746.65s - 749.53s | And who has to tune


chunk_text:
I can turn this into a SkyTalk 711.65s - 713.89s | and we can start ranting, you know, like, 713.89s - 715.93s | I can fucking rant. 715.93s - 716.77s | No, no, no. 716.77s - 717.61s | I'm- 717.61s - 719.33s | Is this turning into sock therapy? 719.33s - 720.81s | Sock therapy is valid too. 720.81s - 722.13s | Look up my fucking agenda, bro. 722.13s - 723.13s | It's like all SkyTalk. 723.13s - 724.17s | It's all ranting. 724.17s - 725.37s | Like the machine learning is like, 725.37s - 726.77s | you know, number 5.8. 728.77s - 729.93s | I'm all about the rant, bro. 729.93s - 732.21s | Like that's, it's the use case, right? 732.21s - 735.61s | It's like, what's broken and how do we fix it, right? 735.61s - 736.93s | But yeah. 737.53s - 738.77s | But yeah, you're on top of it. 738.77s - 742.13s | Like too many tools, too many rules, too many playbooks, 742.13s - 743.93s | too many whatever the fuck you want to call them. 743.93s - 746.65s | It's just searches, lookups, right? 746.65s - 749.53s | And who has to tune those? 749.53s - 750.77s | The people in the process. 750.77s - 752.53s | But we don't want to buy those. 752.53s - 753.49s | We want to buy the technology. 753.49s - 755.05s | So we want to buy the sexiest shit 755.05s - 756.97s | and we want to install the sexiest shit. 756.97s - 757.81s | Do we have the right people 757.81s - 759.01s | in process for that technology? 759.01s - 760.17s | Nope. 760.17s - 761.01s | Do we care? 761.01s - 761.85s | Nope. 761.85s - 764.69s | We care about AI because I'm a new leader 764.69s - 766.49s | and I want to put AI in my company 767.49s - 769.69s | and you guys deal with it until you stop dealing with it 769.69s - 771.89s | and the company's fucked and now I leave 771.89s - 773.77s | and they get the new guys, right? 774.89s - 777.49s | So it's just so bloated and ineffective 777.49s - 779.13s | for a variety of reasons. 779.13s - 781.05s | Most of these reasons are not technology. 781.05s - 783.61s | Most of these reasons are people in process. 783.61s


chunk_text:
those? 749.53s - 750.77s | The people in the process. 750.77s - 752.53s | But we don't want to buy those. 752.53s - 753.49s | We want to buy the technology. 753.49s - 755.05s | So we want to buy the sexiest shit 755.05s - 756.97s | and we want to install the sexiest shit. 756.97s - 757.81s | Do we have the right people 757.81s - 759.01s | in process for that technology? 759.01s - 760.17s | Nope. 760.17s - 761.01s | Do we care? 761.01s - 761.85s | Nope. 761.85s - 764.69s | We care about AI because I'm a new leader 764.69s - 766.49s | and I want to put AI in my company 767.49s - 769.69s | and you guys deal with it until you stop dealing with it 769.69s - 771.89s | and the company's fucked and now I leave 771.89s - 773.77s | and they get the new guys, right? 774.89s - 777.49s | So it's just so bloated and ineffective 777.49s - 779.13s | for a variety of reasons. 779.13s - 781.05s | Most of these reasons are not technology. 781.05s - 783.61s | Most of these reasons are people in process. 783.61s - 785.73s | And since I baited you all with AI, 785.73s - 788.37s | I will give you a lot more people in process today. 788.37s - 790.73s | So what the hell is wrong with our security? 790.73s - 794.09s | Why is it so less secure than what it was 50 years ago? 794.09s - 795.57s | Well, the naive way to think about this 795.65s - 797.33s | is it used to be so simple. 797.33s - 798.65s | It's not even apples to apples. 798.65s - 800.21s | It used to be so fucking simple. 800.21s - 802.45s | Now it's so fucking complicated, right? 802.45s - 804.25s | And to hack Stuxnet, 804.25s - 808.45s | you had a virus circulate the entire world for two years. 808.45s - 810.21s | Finally, someone picked it up on a USB, 810.21s - 812.53s | put it in the printer to print an assignment for their kid 812.53s - 813.97s | and boom, bada boom, right? 813.97s - 817.65s | But in reality, if you want to do Stuxnet today, 817.65s - 819.13s | there's like a smart grid. 819.13s - 820.81s | There's like people


chunk_text:
- 785.73s | And since I baited you all with AI, 785.73s - 788.37s | I will give you a lot more people in process today. 788.37s - 790.73s | So what the hell is wrong with our security? 790.73s - 794.09s | Why is it so less secure than what it was 50 years ago? 794.09s - 795.57s | Well, the naive way to think about this 795.65s - 797.33s | is it used to be so simple. 797.33s - 798.65s | It's not even apples to apples. 798.65s - 800.21s | It used to be so fucking simple. 800.21s - 802.45s | Now it's so fucking complicated, right? 802.45s - 804.25s | And to hack Stuxnet, 804.25s - 808.45s | you had a virus circulate the entire world for two years. 808.45s - 810.21s | Finally, someone picked it up on a USB, 810.21s - 812.53s | put it in the printer to print an assignment for their kid 812.53s - 813.97s | and boom, bada boom, right? 813.97s - 817.65s | But in reality, if you want to do Stuxnet today, 817.65s - 819.13s | there's like a smart grid. 819.13s - 820.81s | There's like people selling solar power 820.81s - 822.09s | back to the nuclear plant 822.09s - 824.05s | and some bullshit like that on the smart grid 824.09s - 826.65s | where technically everything is connected 826.65s - 830.25s | and the attack surface is like so much easier. 830.25s - 832.05s | What do you mean we have to wait for two years 832.05s - 833.65s | for someone to put it on a USB? 833.65s - 835.29s | Not anymore. 835.29s - 839.77s | So this is essentially why everybody here is so vibing 839.77s - 842.13s | when we are complaining 842.13s - 846.25s | because we are constantly waiting for that talk at DEF CON 846.25s - 848.01s | that will come and automate our jobs 848.01s - 850.61s | and take it away from us so that we can do something else. 850.61s - 851.61s | We don't want to be analysts. 851.61s - 852.65s | We want to be engineers. 852.65s - 853.61s | We want to be architects. 853.61s - 854.89s | We want to be cloud folks. 854.89s - 857.53s | Fuck it, we want to be compliance folks if we have to. 857.53s


chunk_text:
selling solar power 820.81s - 822.09s | back to the nuclear plant 822.09s - 824.05s | and some bullshit like that on the smart grid 824.09s - 826.65s | where technically everything is connected 826.65s - 830.25s | and the attack surface is like so much easier. 830.25s - 832.05s | What do you mean we have to wait for two years 832.05s - 833.65s | for someone to put it on a USB? 833.65s - 835.29s | Not anymore. 835.29s - 839.77s | So this is essentially why everybody here is so vibing 839.77s - 842.13s | when we are complaining 842.13s - 846.25s | because we are constantly waiting for that talk at DEF CON 846.25s - 848.01s | that will come and automate our jobs 848.01s - 850.61s | and take it away from us so that we can do something else. 850.61s - 851.61s | We don't want to be analysts. 851.61s - 852.65s | We want to be engineers. 852.65s - 853.61s | We want to be architects. 853.61s - 854.89s | We want to be cloud folks. 854.89s - 857.53s | Fuck it, we want to be compliance folks if we have to. 857.53s - 858.37s | Any... 859.89s - 861.45s | No hate on the compliance folks, 863.17s - 864.25s | but essentially like we can, 864.25s - 865.61s | I could be a freaking salesperson 865.61s - 866.81s | instead of being a SOC analyst. 866.81s - 868.69s | You know, like it's shit, 868.69s - 871.81s | but what does it take for a SOC analyst? 871.81s - 873.77s | What is actually so hard? 873.77s - 876.77s | And the expensive folks at SANS, 876.77s - 878.61s | they run a SOC survey 878.61s - 881.25s | and in the SOC survey for the last six, seven years, 881.25s - 882.41s | it's always been the same shit. 882.41s - 884.81s | People go and they vote on what's the thing 884.81s - 885.81s | they hate the most, 885.81s - 887.69s | what the thing essentially is 887.69s - 890.13s | we're going to all hang our trouble on. 890.13s - 892.01s | And it's always, always, always lack of 892.01s - 895.49s | quote, quote, beautiful, sweet correlation. 895.49s - 897.77s | What the hell is correlation? 897.77s - 899.49s | I don't know if someone knows what


chunk_text:
- 858.37s | Any... 859.89s - 861.45s | No hate on the compliance folks, 863.17s - 864.25s | but essentially like we can, 864.25s - 865.61s | I could be a freaking salesperson 865.61s - 866.81s | instead of being a SOC analyst. 866.81s - 868.69s | You know, like it's shit, 868.69s - 871.81s | but what does it take for a SOC analyst? 871.81s - 873.77s | What is actually so hard? 873.77s - 876.77s | And the expensive folks at SANS, 876.77s - 878.61s | they run a SOC survey 878.61s - 881.25s | and in the SOC survey for the last six, seven years, 881.25s - 882.41s | it's always been the same shit. 882.41s - 884.81s | People go and they vote on what's the thing 884.81s - 885.81s | they hate the most, 885.81s - 887.69s | what the thing essentially is 887.69s - 890.13s | we're going to all hang our trouble on. 890.13s - 892.01s | And it's always, always, always lack of 892.01s - 895.49s | quote, quote, beautiful, sweet correlation. 895.49s - 897.77s | What the hell is correlation? 897.77s - 899.49s | I don't know if someone knows what that is correlation, 899.49s - 900.65s | but essentially just says like, 900.65s - 902.33s | what that is relevant to what? 902.33s - 903.97s | And it's like so ambiguous 903.97s - 907.09s | that people can sell you what they call correlation searches 907.09s - 908.57s | if it's looking for like two things 908.57s - 909.97s | that it's going to put together 910.01s - 912.49s | if they fit a specific rule, right? 913.69s - 915.09s | But that's all good 916.37s - 918.61s | if you have all of your data in one place. 919.57s - 922.41s | How many people here have everything they need in the SIM? 924.65s - 925.97s | Yes. 925.97s - 927.37s | Nobody, right? 927.37s - 928.21s | Which one? 928.21s - 929.05s | Yes. 929.05s - 930.61s | Do you want the Splunk or the Elastic or the S3 bucket? 930.61s - 932.21s | Like which one do you want, bro? 933.53s - 934.37s | Damn. 934.37s - 935.21s | Okay. Which index? 935.21s - 936.05s | Which silo on it? 936.05s - 936.89s | Okay. 937.85s - 938.85s | Yeah. 938.85s - 941.55s | Yeah. 942.59s -


chunk_text:
that is correlation, 899.49s - 900.65s | but essentially just says like, 900.65s - 902.33s | what that is relevant to what? 902.33s - 903.97s | And it's like so ambiguous 903.97s - 907.09s | that people can sell you what they call correlation searches 907.09s - 908.57s | if it's looking for like two things 908.57s - 909.97s | that it's going to put together 910.01s - 912.49s | if they fit a specific rule, right? 913.69s - 915.09s | But that's all good 916.37s - 918.61s | if you have all of your data in one place. 919.57s - 922.41s | How many people here have everything they need in the SIM? 924.65s - 925.97s | Yes. 925.97s - 927.37s | Nobody, right? 927.37s - 928.21s | Which one? 928.21s - 929.05s | Yes. 929.05s - 930.61s | Do you want the Splunk or the Elastic or the S3 bucket? 930.61s - 932.21s | Like which one do you want, bro? 933.53s - 934.37s | Damn. 934.37s - 935.21s | Okay. Which index? 935.21s - 936.05s | Which silo on it? 936.05s - 936.89s | Okay. 937.85s - 938.85s | Yeah. 938.85s - 941.55s | Yeah. 942.59s - 943.43s | I have a sock for insider threat. 943.43s - 944.51s | I have a sock for this guy. 944.51s - 945.87s | I have a sock for the NERC SIP. 945.87s - 947.23s | I have a sock for whatever. 947.23s - 949.47s | I have a sock at the company we bought. 949.47s - 951.03s | It's not technically part of our company, 951.03s - 952.59s | but it's, you know, it's a sub-organization. 952.59s - 953.99s | It's like another entity that we bought. 953.99s - 956.15s | They have their own sock and they don't give us the shit. 956.15s - 958.15s | We have a sock that works part of a NOC. 958.15s - 959.87s | We have a sock that's called Data Fusion 959.87s - 961.07s | that the new leader made. 961.07s - 962.87s | You know, I know what's up. 962.87s - 967.51s | So if we don't have all the data in one place, 967.51s - 969.75s | can we put all the data in one place? 969.79s - 972.27s | Wouldn't that be a pretty nice process, right? 972.27s - 974.63s | Like let's get as much data in


chunk_text:
943.43s | I have a sock for insider threat. 943.43s - 944.51s | I have a sock for this guy. 944.51s - 945.87s | I have a sock for the NERC SIP. 945.87s - 947.23s | I have a sock for whatever. 947.23s - 949.47s | I have a sock at the company we bought. 949.47s - 951.03s | It's not technically part of our company, 951.03s - 952.59s | but it's, you know, it's a sub-organization. 952.59s - 953.99s | It's like another entity that we bought. 953.99s - 956.15s | They have their own sock and they don't give us the shit. 956.15s - 958.15s | We have a sock that works part of a NOC. 958.15s - 959.87s | We have a sock that's called Data Fusion 959.87s - 961.07s | that the new leader made. 961.07s - 962.87s | You know, I know what's up. 962.87s - 967.51s | So if we don't have all the data in one place, 967.51s - 969.75s | can we put all the data in one place? 969.79s - 972.27s | Wouldn't that be a pretty nice process, right? 972.27s - 974.63s | Like let's get as much data in one place. 974.63s - 976.15s | And then we start talking about money. 976.15s - 978.87s | Like, dude, it costs a shit ton to get it into one place. 978.87s - 981.35s | And by the way, to get it in one place, 981.35s - 983.23s | I have to put everything in one data model. 983.23s - 984.83s | I have to make sure I have the right, you know, 984.83s - 986.71s | parsers or whatnot. 986.71s - 987.83s | It's 2024. 988.71s - 991.07s | LLMs can actually make you, like, 991.07s - 993.39s | any kind of data model into any other data model. 993.39s - 994.75s | So we have an open source thing. 994.75s - 996.47s | If you go to GitHub Sapienta, 996.47s - 999.35s | we get anything in any data model put into one data model. 999.35s - 1000.91s | And it's also 2024. 1000.91s - 1002.43s | Storage is so fucking cheap. 1002.43s - 1004.79s | I don't know why do you have to put it into a SIM, 1004.79s - 1007.19s | put it into a fucking S3 bucket as a cold archive. 1007.19s - 1008.47s | It's


chunk_text:
one place. 974.63s - 976.15s | And then we start talking about money. 976.15s - 978.87s | Like, dude, it costs a shit ton to get it into one place. 978.87s - 981.35s | And by the way, to get it in one place, 981.35s - 983.23s | I have to put everything in one data model. 983.23s - 984.83s | I have to make sure I have the right, you know, 984.83s - 986.71s | parsers or whatnot. 986.71s - 987.83s | It's 2024. 988.71s - 991.07s | LLMs can actually make you, like, 991.07s - 993.39s | any kind of data model into any other data model. 993.39s - 994.75s | So we have an open source thing. 994.75s - 996.47s | If you go to GitHub Sapienta, 996.47s - 999.35s | we get anything in any data model put into one data model. 999.35s - 1000.91s | And it's also 2024. 1000.91s - 1002.43s | Storage is so fucking cheap. 1002.43s - 1004.79s | I don't know why do you have to put it into a SIM, 1004.79s - 1007.19s | put it into a fucking S3 bucket as a cold archive. 1007.19s - 1008.47s | It's cheap enough. 1008.47s - 1010.23s | If you can get it all there 1010.23s - 1012.83s | and you can fight the people to get it all there 1012.83s - 1015.07s | and you can call everybody and get the right data, 1015.07s - 1017.63s | you can have the dream kind of thing 1017.63s - 1019.27s | where you have everything in one S3. 1019.27s - 1021.63s | If you don't get the entire dream, that's fine. 1021.63s - 1023.31s | If you can get like 70% of that dream, 1023.31s - 1025.83s | get just like everything into like a cheap storage 1025.83s - 1027.79s | and a cloud, that'll be fantastic. 1027.79s - 1029.59s | Because if you have everything in one place, 1029.59s - 1031.71s | ideally in dreamland, 1032.71s - 1034.91s | and you can pull it off without wasting a lot of money 1034.91s - 1035.75s | or asking for budget, 1035.75s - 1036.75s | just put it in cloud, 1036.75s - 1039.31s | cost 20 cents a terabyte or something like that, right? 1039.31s - 1040.91s | On S3. 1040.91s - 1043.87s | You can essentially have almost everything you need 1043.87s -


chunk_text:
cheap enough. 1008.47s - 1010.23s | If you can get it all there 1010.23s - 1012.83s | and you can fight the people to get it all there 1012.83s - 1015.07s | and you can call everybody and get the right data, 1015.07s - 1017.63s | you can have the dream kind of thing 1017.63s - 1019.27s | where you have everything in one S3. 1019.27s - 1021.63s | If you don't get the entire dream, that's fine. 1021.63s - 1023.31s | If you can get like 70% of that dream, 1023.31s - 1025.83s | get just like everything into like a cheap storage 1025.83s - 1027.79s | and a cloud, that'll be fantastic. 1027.79s - 1029.59s | Because if you have everything in one place, 1029.59s - 1031.71s | ideally in dreamland, 1032.71s - 1034.91s | and you can pull it off without wasting a lot of money 1034.91s - 1035.75s | or asking for budget, 1035.75s - 1036.75s | just put it in cloud, 1036.75s - 1039.31s | cost 20 cents a terabyte or something like that, right? 1039.31s - 1040.91s | On S3. 1040.91s - 1043.87s | You can essentially have almost everything you need 1043.87s - 1047.27s | to run these kind of queries, right? 1047.27s - 1049.19s | Because what do we have in the SOC today? 1049.19s - 1050.39s | It's 2024. 1050.39s - 1053.87s | We have the same shit that we used to do before I was born, 1053.87s - 1055.31s | queries and playbooks. 1055.31s - 1056.83s | Why? 1056.83s - 1058.43s | We can trust that shit, right? 1058.43s - 1059.51s | We can trust it. 1059.51s - 1060.63s | It's explainable. 1060.63s - 1061.47s | It's adaptable. 1061.47s - 1062.39s | It's customizable. 1062.39s - 1063.23s | It's transparent. 1063.23s - 1067.01s | Yes. 1079.63s - 1080.47s | That's true. 1080.47s - 1081.31s | That's true. 1081.31s - 1082.15s | A lot of the insider threat data 1082.15s - 1083.55s | could not be given to the SOC. 1083.55s - 1084.63s | It's PIIs. 1084.63s - 1085.47s | Yeah. 1085.47s - 1087.27s | The risk compliance people 1087.27s - 1089.23s | are not gonna let you have your insider threat data there. 1089.23s - 1090.95s | If you have UEBA, they're not gonna let you have it. 1090.95s - 1092.79s | If you have endpoints or emails 1092.79s -


chunk_text:
1047.27s | to run these kind of queries, right? 1047.27s - 1049.19s | Because what do we have in the SOC today? 1049.19s - 1050.39s | It's 2024. 1050.39s - 1053.87s | We have the same shit that we used to do before I was born, 1053.87s - 1055.31s | queries and playbooks. 1055.31s - 1056.83s | Why? 1056.83s - 1058.43s | We can trust that shit, right? 1058.43s - 1059.51s | We can trust it. 1059.51s - 1060.63s | It's explainable. 1060.63s - 1061.47s | It's adaptable. 1061.47s - 1062.39s | It's customizable. 1062.39s - 1063.23s | It's transparent. 1063.23s - 1067.01s | Yes. 1079.63s - 1080.47s | That's true. 1080.47s - 1081.31s | That's true. 1081.31s - 1082.15s | A lot of the insider threat data 1082.15s - 1083.55s | could not be given to the SOC. 1083.55s - 1084.63s | It's PIIs. 1084.63s - 1085.47s | Yeah. 1085.47s - 1087.27s | The risk compliance people 1087.27s - 1089.23s | are not gonna let you have your insider threat data there. 1089.23s - 1090.95s | If you have UEBA, they're not gonna let you have it. 1090.95s - 1092.79s | If you have endpoints or emails 1092.79s - 1094.67s | or stuff from Outlook or whatnot, nope. 1094.67s - 1096.71s | You're not gonna have that in the SOC, no way. 1096.71s - 1098.83s | But if we can get as close as possible, 1099.75s - 1101.23s | we can get something there. 1101.23s - 1103.59s | And the point is we use queries and playbooks 1103.59s - 1106.19s | because I can fucking trust that shit. 1106.19s - 1107.07s | Like I trust it. 1107.07s - 1107.91s | I can see it. 1107.91s - 1108.75s | I can read the rule. 1108.75s - 1109.83s | Look at how nice it is. 1109.83s - 1110.91s | It's so open. 1110.91s - 1112.03s | It's so transparent. 1112.03s - 1113.51s | It's so nice. 1113.51s - 1115.35s | I can actually read that shit. 1115.35s - 1117.15s | I fully understand when it triggers. 1117.15s - 1120.23s | It triggers when it finds these kind of stuff in the query. 1120.23s - 1122.91s | Command, all kinds of commands in my web shell traffic 1122.91s - 1125.95s | or in my web traffic in general as a post, 1125.95s - 1126.83s | that's when it triggers. 1126.83s - 1127.99s


chunk_text:
1094.67s | or stuff from Outlook or whatnot, nope. 1094.67s - 1096.71s | You're not gonna have that in the SOC, no way. 1096.71s - 1098.83s | But if we can get as close as possible, 1099.75s - 1101.23s | we can get something there. 1101.23s - 1103.59s | And the point is we use queries and playbooks 1103.59s - 1106.19s | because I can fucking trust that shit. 1106.19s - 1107.07s | Like I trust it. 1107.07s - 1107.91s | I can see it. 1107.91s - 1108.75s | I can read the rule. 1108.75s - 1109.83s | Look at how nice it is. 1109.83s - 1110.91s | It's so open. 1110.91s - 1112.03s | It's so transparent. 1112.03s - 1113.51s | It's so nice. 1113.51s - 1115.35s | I can actually read that shit. 1115.35s - 1117.15s | I fully understand when it triggers. 1117.15s - 1120.23s | It triggers when it finds these kind of stuff in the query. 1120.23s - 1122.91s | Command, all kinds of commands in my web shell traffic 1122.91s - 1125.95s | or in my web traffic in general as a post, 1125.95s - 1126.83s | that's when it triggers. 1126.83s - 1127.99s | I completely understand that. 1128.03s - 1129.27s | Can you say that about an LLM? 1129.27s - 1130.47s | Hell no. 1130.47s - 1132.59s | How many people here are using anomaly detector? 1132.59s - 1134.27s | Anomaly detection, UEBA? 1135.51s - 1136.35s | Two people. 1136.35s - 1138.03s | It's 2024, right? 1138.03s - 1139.07s | Like five years ago, 1139.07s - 1142.19s | a shit ton of people put their money on UEBA, right? 1142.19s - 1143.23s | What's wrong with UEBA? 1143.23s - 1144.07s | Why don't you use it? 1144.07s - 1144.91s | Why do you use it? 1144.91s - 1155.92s | What do you hate about it if you use it? 1155.92s - 1157.16s | Oh. 1157.16s - 1158.00s | It works. 1158.00s - 1161.32s | If you say this is gonna give me everything I don't see 1161.32s - 1164.70s | and all that's going to be broken. 1164.70s - 1167.10s | I'm definitely buying you a beer tonight. 1167.14s - 1168.66s | Bye. 1168.66s - 1169.50s | Sorry, go ahead. 1170.82s - 1173.22s | Yeah, yeah, 100%, come up. 1173.22s - 1174.74s | Hell yeah, come up. 1174.74s - 1175.58s | Woo-hoo.


chunk_text:
| I completely understand that. 1128.03s - 1129.27s | Can you say that about an LLM? 1129.27s - 1130.47s | Hell no. 1130.47s - 1132.59s | How many people here are using anomaly detector? 1132.59s - 1134.27s | Anomaly detection, UEBA? 1135.51s - 1136.35s | Two people. 1136.35s - 1138.03s | It's 2024, right? 1138.03s - 1139.07s | Like five years ago, 1139.07s - 1142.19s | a shit ton of people put their money on UEBA, right? 1142.19s - 1143.23s | What's wrong with UEBA? 1143.23s - 1144.07s | Why don't you use it? 1144.07s - 1144.91s | Why do you use it? 1144.91s - 1155.92s | What do you hate about it if you use it? 1155.92s - 1157.16s | Oh. 1157.16s - 1158.00s | It works. 1158.00s - 1161.32s | If you say this is gonna give me everything I don't see 1161.32s - 1164.70s | and all that's going to be broken. 1164.70s - 1167.10s | I'm definitely buying you a beer tonight. 1167.14s - 1168.66s | Bye. 1168.66s - 1169.50s | Sorry, go ahead. 1170.82s - 1173.22s | Yeah, yeah, 100%, come up. 1173.22s - 1174.74s | Hell yeah, come up. 1174.74s - 1175.58s | Woo-hoo. 1178.68s - 1179.52s | Can you give us a quick intro? 1179.52s - 1181.00s | Can we get the third mic running? 1181.00s - 1181.84s | My dude. 1186.64s - 1187.68s | Thank you. 1187.68s - 1189.04s | Oh. 1189.04s - 1189.88s | All good. 1189.88s - 1190.72s | Sweet. 1190.72s - 1191.56s | Thank you for coming up. 1192.60s - 1194.04s | By popular choice. 1194.04s - 1195.56s | The people's choice panelist. 1196.88s - 1197.72s | My name's Walt. 1197.72s - 1199.36s | I'm a 15 year schmo. 1199.44s - 1204.44s | I'm in cyber defense working for the man. 1208.62s - 1209.82s | And- 1209.82s - 1211.02s | Uncle Sam? 1211.02s - 1211.86s | That man, yeah. 1213.30s - 1214.18s | I love that man. 1215.14s - 1220.14s | But, and I started as a sock analyst 1220.22s - 1224.26s | and I'm somewhere between senior sock analyst 1224.26s - 1227.98s | and the guy in the office who does ML 1227.98s - 1230.22s | and plays with the Splunk models 1230.74s - 1233.74s | and tries to build the dashboards. 1233.74s - 1234.58s | Oh yeah. 1235.54s - 1238.82s | That's, and yeah, I'm loud and


chunk_text:
1178.68s - 1179.52s | Can you give us a quick intro? 1179.52s - 1181.00s | Can we get the third mic running? 1181.00s - 1181.84s | My dude. 1186.64s - 1187.68s | Thank you. 1187.68s - 1189.04s | Oh. 1189.04s - 1189.88s | All good. 1189.88s - 1190.72s | Sweet. 1190.72s - 1191.56s | Thank you for coming up. 1192.60s - 1194.04s | By popular choice. 1194.04s - 1195.56s | The people's choice panelist. 1196.88s - 1197.72s | My name's Walt. 1197.72s - 1199.36s | I'm a 15 year schmo. 1199.44s - 1204.44s | I'm in cyber defense working for the man. 1208.62s - 1209.82s | And- 1209.82s - 1211.02s | Uncle Sam? 1211.02s - 1211.86s | That man, yeah. 1213.30s - 1214.18s | I love that man. 1215.14s - 1220.14s | But, and I started as a sock analyst 1220.22s - 1224.26s | and I'm somewhere between senior sock analyst 1224.26s - 1227.98s | and the guy in the office who does ML 1227.98s - 1230.22s | and plays with the Splunk models 1230.74s - 1233.74s | and tries to build the dashboards. 1233.74s - 1234.58s | Oh yeah. 1235.54s - 1238.82s | That's, and yeah, I'm loud and over opinionated. 1238.82s - 1242.82s | I mean, yeah, yeah. 1242.82s - 1244.50s | I hear you. 1244.50s - 1246.14s | There's a lot of stuff that you didn't say 1246.14s - 1247.58s | that, you know, I can feel it. 1247.58s - 1248.42s | I was in the sock. 1248.42s - 1249.78s | I was using Splunk. 1249.78s - 1252.70s | I used MLTK, DSDL, all that good shit, eh? 1252.70s - 1253.54s | What was that? 1253.54s - 1255.62s | MLTK, machine learning toolkit. 1255.62s - 1256.46s | Oh yes. 1256.46s - 1257.30s | Deep science. 1257.30s - 1259.18s | MLTK, I like it. 1259.18s - 1263.22s | It works nicely for something that's embedded. 1263.22s - 1264.98s | I haven't played with elastics 1264.98s - 1267.50s | embedded machine learning that much yet. 1268.90s - 1270.22s | I can do some stuff in scikit. 1270.22s - 1274.50s | I have done stuff in scikit-learn. 1274.50s - 1276.10s | I have a master's. 1276.10s - 1277.10s | Nice. 1277.10s - 1278.74s | I got it last year. 1278.74s - 1282.30s | Oh, all fresh with all the data science. 1282.30s - 1286.98s | Hey, and it's, I went


chunk_text:
over opinionated. 1238.82s - 1242.82s | I mean, yeah, yeah. 1242.82s - 1244.50s | I hear you. 1244.50s - 1246.14s | There's a lot of stuff that you didn't say 1246.14s - 1247.58s | that, you know, I can feel it. 1247.58s - 1248.42s | I was in the sock. 1248.42s - 1249.78s | I was using Splunk. 1249.78s - 1252.70s | I used MLTK, DSDL, all that good shit, eh? 1252.70s - 1253.54s | What was that? 1253.54s - 1255.62s | MLTK, machine learning toolkit. 1255.62s - 1256.46s | Oh yes. 1256.46s - 1257.30s | Deep science. 1257.30s - 1259.18s | MLTK, I like it. 1259.18s - 1263.22s | It works nicely for something that's embedded. 1263.22s - 1264.98s | I haven't played with elastics 1264.98s - 1267.50s | embedded machine learning that much yet. 1268.90s - 1270.22s | I can do some stuff in scikit. 1270.22s - 1274.50s | I have done stuff in scikit-learn. 1274.50s - 1276.10s | I have a master's. 1276.10s - 1277.10s | Nice. 1277.10s - 1278.74s | I got it last year. 1278.74s - 1282.30s | Oh, all fresh with all the data science. 1282.30s - 1286.98s | Hey, and it's, I went there because there was a need. 1286.98s - 1287.82s | Hell yeah. 1287.82s - 1290.38s | Somebody has to be able to sit there and say, 1290.38s - 1293.42s | the machine learning works for the use case 1293.42s - 1295.50s | and don't buy the black box. 1295.50s - 1296.50s | Hell yeah. 1296.50s - 1297.98s | That's my role in the office. 1297.98s - 1298.82s | Yes. 1298.82s - 1300.98s | If they ask me if this actually makes sense, 1300.98s - 1302.94s | then I figure out a way to use it. 1302.94s - 1303.78s | That's amazing. 1303.78s - 1304.62s | It's like taking my life 1304.62s - 1307.62s | and just making it a lot more full of experience 1307.62s - 1309.30s | kind of thing, you know? 1309.30s - 1311.30s | Like I worked like a couple of years in the sock 1311.30s - 1313.38s | and then I went into a PhD 1313.38s - 1315.78s | to solve some issues in the sock. 1315.78s - 1318.34s | You essentially worked a lot more in the sock, 1318.34s - 1319.66s | saw a lot more bullshit. 1319.66s - 1322.14s | So yeah, I'm going


chunk_text:
there because there was a need. 1286.98s - 1287.82s | Hell yeah. 1287.82s - 1290.38s | Somebody has to be able to sit there and say, 1290.38s - 1293.42s | the machine learning works for the use case 1293.42s - 1295.50s | and don't buy the black box. 1295.50s - 1296.50s | Hell yeah. 1296.50s - 1297.98s | That's my role in the office. 1297.98s - 1298.82s | Yes. 1298.82s - 1300.98s | If they ask me if this actually makes sense, 1300.98s - 1302.94s | then I figure out a way to use it. 1302.94s - 1303.78s | That's amazing. 1303.78s - 1304.62s | It's like taking my life 1304.62s - 1307.62s | and just making it a lot more full of experience 1307.62s - 1309.30s | kind of thing, you know? 1309.30s - 1311.30s | Like I worked like a couple of years in the sock 1311.30s - 1313.38s | and then I went into a PhD 1313.38s - 1315.78s | to solve some issues in the sock. 1315.78s - 1318.34s | You essentially worked a lot more in the sock, 1318.34s - 1319.66s | saw a lot more bullshit. 1319.66s - 1322.14s | So yeah, I'm going to refer a lot back to you. 1322.14s - 1323.58s | If you have anything to say, just- 1323.58s - 1324.42s | No worries. 1324.42s - 1325.26s | Boom, right away. 1325.26s - 1326.90s | I don't have to prompt you. 1326.90s - 1327.74s | I'll just go for it. 1327.74s - 1329.74s | Unfortunately, my daughter is now cringing. 1329.74s - 1330.58s | Oh yes. 1330.58s - 1333.90s | Right? 1333.90s - 1335.10s | Yeah, I mean, you made it to the con. 1335.10s - 1336.06s | What did you think? 1339.06s - 1339.90s | All right. 1339.90s - 1341.74s | So back to the discussion, 1341.74s - 1346.34s | we love rules because they're understandable. 1346.34s - 1347.38s | They're transparent. 1347.38s - 1348.34s | I can customize them. 1348.34s - 1349.90s | I can fucking give them feedback 1349.90s - 1351.46s | and they can do whatever I want them to do. 1351.46s - 1352.70s | I trust them. 1352.70s - 1355.30s | I will never get fired without understanding why. 1356.18s - 1357.02s | AI? 1357.02s - 1360.30s | I'll get fired and I will be like, why? 1361.46s - 1362.58s | Why did it not


chunk_text:
to refer a lot back to you. 1322.14s - 1323.58s | If you have anything to say, just- 1323.58s - 1324.42s | No worries. 1324.42s - 1325.26s | Boom, right away. 1325.26s - 1326.90s | I don't have to prompt you. 1326.90s - 1327.74s | I'll just go for it. 1327.74s - 1329.74s | Unfortunately, my daughter is now cringing. 1329.74s - 1330.58s | Oh yes. 1330.58s - 1333.90s | Right? 1333.90s - 1335.10s | Yeah, I mean, you made it to the con. 1335.10s - 1336.06s | What did you think? 1339.06s - 1339.90s | All right. 1339.90s - 1341.74s | So back to the discussion, 1341.74s - 1346.34s | we love rules because they're understandable. 1346.34s - 1347.38s | They're transparent. 1347.38s - 1348.34s | I can customize them. 1348.34s - 1349.90s | I can fucking give them feedback 1349.90s - 1351.46s | and they can do whatever I want them to do. 1351.46s - 1352.70s | I trust them. 1352.70s - 1355.30s | I will never get fired without understanding why. 1356.18s - 1357.02s | AI? 1357.02s - 1360.30s | I'll get fired and I will be like, why? 1361.46s - 1362.58s | Why did it not work? 1362.58s - 1365.22s | The vendor told me it will work, right? 1365.22s - 1367.82s | So anyways, rules are pretty nice. 1367.82s - 1370.22s | Have you guys ever seen investigative playbooks? 1371.18s - 1372.02s | Anybody know what that is? 1372.02s - 1373.30s | An investigative playbook? 1373.30s - 1374.14s | Okay. 1374.14s - 1376.66s | So an investigative playbook is a fancy way to say, 1376.66s - 1378.98s | you know, I have a bunch of searches 1378.98s - 1380.82s | that go back to back to back to back. 1380.82s - 1383.26s | And if they're within five minutes of each other, 1383.26s - 1385.74s | they come all together within a ticket. 1385.74s - 1386.58s | That's pretty much it. 1386.58s - 1390.18s | It's like security analytics stories from Splunk. 1390.18s - 1393.50s | In Splunk, they also now have the risk-based alerting. 1393.50s - 1396.94s | Essentially just means like if something is scored 1396.94s - 1398.10s | like five out of 10 risk, 1398.10s - 1399.50s | and another thing is five out of 10 risk, 1399.54s - 1400.70s | and they happen within five minutes, 1400.70s - 1402.50s | boom,


chunk_text:
work? 1362.58s - 1365.22s | The vendor told me it will work, right? 1365.22s - 1367.82s | So anyways, rules are pretty nice. 1367.82s - 1370.22s | Have you guys ever seen investigative playbooks? 1371.18s - 1372.02s | Anybody know what that is? 1372.02s - 1373.30s | An investigative playbook? 1373.30s - 1374.14s | Okay. 1374.14s - 1376.66s | So an investigative playbook is a fancy way to say, 1376.66s - 1378.98s | you know, I have a bunch of searches 1378.98s - 1380.82s | that go back to back to back to back. 1380.82s - 1383.26s | And if they're within five minutes of each other, 1383.26s - 1385.74s | they come all together within a ticket. 1385.74s - 1386.58s | That's pretty much it. 1386.58s - 1390.18s | It's like security analytics stories from Splunk. 1390.18s - 1393.50s | In Splunk, they also now have the risk-based alerting. 1393.50s - 1396.94s | Essentially just means like if something is scored 1396.94s - 1398.10s | like five out of 10 risk, 1398.10s - 1399.50s | and another thing is five out of 10 risk, 1399.54s - 1400.70s | and they happen within five minutes, 1400.70s - 1402.50s | boom, it's 10 out of 10 risk for both of them, 1402.50s - 1404.50s | put them together in a ticket and tell me. 1404.50s - 1405.90s | But essentially all of these nice things 1405.90s - 1408.94s | are rules on top of rules on top of rules. 1408.94s - 1412.98s | And every year, although it's 2020 fucking four, 1412.98s - 1417.82s | we get new ways to apply rules as if we're in the 60s. 1417.82s - 1420.98s | Why the hell are we not using any more like things 1420.98s - 1424.70s | other than rules and solid, narrow logic? 1424.70s - 1426.26s | Because we can not trust it. 1426.26s - 1427.10s | Okay. 1427.98s - 1429.90s | So what do we like so much about our rules? 1429.90s - 1431.02s | What makes us trust them? 1431.02s - 1433.86s | Well, I can also map them to MITRE ATT&CK techniques. 1433.86s - 1436.42s | Anybody here, I'll ask this question one more time 1436.42s - 1439.50s | just to, anybody here does not know what's MITRE ATT&CK, 1439.50s - 1441.14s | and I will go through it quickly. 1442.14s - 1443.66s | Okay, let's go


chunk_text:
it's 10 out of 10 risk for both of them, 1402.50s - 1404.50s | put them together in a ticket and tell me. 1404.50s - 1405.90s | But essentially all of these nice things 1405.90s - 1408.94s | are rules on top of rules on top of rules. 1408.94s - 1412.98s | And every year, although it's 2020 fucking four, 1412.98s - 1417.82s | we get new ways to apply rules as if we're in the 60s. 1417.82s - 1420.98s | Why the hell are we not using any more like things 1420.98s - 1424.70s | other than rules and solid, narrow logic? 1424.70s - 1426.26s | Because we can not trust it. 1426.26s - 1427.10s | Okay. 1427.98s - 1429.90s | So what do we like so much about our rules? 1429.90s - 1431.02s | What makes us trust them? 1431.02s - 1433.86s | Well, I can also map them to MITRE ATT&CK techniques. 1433.86s - 1436.42s | Anybody here, I'll ask this question one more time 1436.42s - 1439.50s | just to, anybody here does not know what's MITRE ATT&CK, 1439.50s - 1441.14s | and I will go through it quickly. 1442.14s - 1443.66s | Okay, let's go through it quickly. 1443.66s - 1447.54s | So MITRE ATT&CK is essentially a way to say what happened 1447.54s - 1450.10s | when we talk about the what, the who, the when. 1450.10s - 1454.54s | If I tell you, oh, this vendor says this thing happened, 1454.54s - 1455.98s | and you don't have a common reference 1456.22s - 1457.54s | for what the hell is this thing, 1457.54s - 1460.26s | you don't really fully understand what is it. 1460.26s - 1462.38s | And a lot of the people in this industry 1462.38s - 1465.90s | are so scared to ask or say that they don't know. 1465.90s - 1468.18s | So nobody actually figures out 1468.18s - 1469.38s | what the hell are they talking about. 1469.38s - 1471.74s | And they just keep on going and they never ask, 1471.74s - 1472.82s | dude, I didn't understand that. 1472.82s - 1474.30s | I don't know what's going on. 1474.30s - 1475.98s | And just keep on going, right? 1475.98s - 1478.74s | Everybody acts like, you know, they know it all. 1478.74s - 1480.18s | And nobody wants to be, you know, 1480.18s - 1482.30s |


chunk_text:
through it quickly. 1443.66s - 1447.54s | So MITRE ATT&CK is essentially a way to say what happened 1447.54s - 1450.10s | when we talk about the what, the who, the when. 1450.10s - 1454.54s | If I tell you, oh, this vendor says this thing happened, 1454.54s - 1455.98s | and you don't have a common reference 1456.22s - 1457.54s | for what the hell is this thing, 1457.54s - 1460.26s | you don't really fully understand what is it. 1460.26s - 1462.38s | And a lot of the people in this industry 1462.38s - 1465.90s | are so scared to ask or say that they don't know. 1465.90s - 1468.18s | So nobody actually figures out 1468.18s - 1469.38s | what the hell are they talking about. 1469.38s - 1471.74s | And they just keep on going and they never ask, 1471.74s - 1472.82s | dude, I didn't understand that. 1472.82s - 1474.30s | I don't know what's going on. 1474.30s - 1475.98s | And just keep on going, right? 1475.98s - 1478.74s | Everybody acts like, you know, they know it all. 1478.74s - 1480.18s | And nobody wants to be, you know, 1480.18s - 1482.30s | like the only person that asked the question. 1482.30s - 1484.86s | But anyways, MITRE ATT&CK is solving that 1484.86s - 1486.58s | as a big Wikipedia. 1486.58s - 1490.10s | And this Wikipedia is essentially full of techniques. 1490.10s - 1492.62s | And every technique has a procedural example. 1492.62s - 1493.86s | It tells you how to detect it, 1493.86s - 1495.62s | how to mitigate it and whatnot. 1495.62s - 1497.82s | And one neat thing about MITRE is essentially 1497.82s - 1501.26s | in MITRE ATT&CK, you can see that going left to right, 1501.26s - 1502.66s | you're actually going sequentially. 1502.66s - 1504.42s | So there is causality going left to right. 1504.42s - 1505.82s | Damn, I lost my light. 1505.82s - 1507.42s | Was that me by any chance? 1507.42s - 1508.50s | That wasn't me, right? 1509.62s - 1510.46s | No worries. 1510.46s - 1511.30s | You guys can still see me. 1511.30s - 1513.98s | I don't need a humongous light bulb in my face. 1513.98s - 1517.74s | But going left to right, like you start at initial access. 1517.74s - 1519.38s | So this is when I send


chunk_text:
like the only person that asked the question. 1482.30s - 1484.86s | But anyways, MITRE ATT&CK is solving that 1484.86s - 1486.58s | as a big Wikipedia. 1486.58s - 1490.10s | And this Wikipedia is essentially full of techniques. 1490.10s - 1492.62s | And every technique has a procedural example. 1492.62s - 1493.86s | It tells you how to detect it, 1493.86s - 1495.62s | how to mitigate it and whatnot. 1495.62s - 1497.82s | And one neat thing about MITRE is essentially 1497.82s - 1501.26s | in MITRE ATT&CK, you can see that going left to right, 1501.26s - 1502.66s | you're actually going sequentially. 1502.66s - 1504.42s | So there is causality going left to right. 1504.42s - 1505.82s | Damn, I lost my light. 1505.82s - 1507.42s | Was that me by any chance? 1507.42s - 1508.50s | That wasn't me, right? 1509.62s - 1510.46s | No worries. 1510.46s - 1511.30s | You guys can still see me. 1511.30s - 1513.98s | I don't need a humongous light bulb in my face. 1513.98s - 1517.74s | But going left to right, like you start at initial access. 1517.74s - 1519.38s | So this is when I send you like an email 1519.38s - 1521.62s | and tell you, yo, what's up? 1521.62s - 1524.02s | You won a hundred million Bitcoins. 1524.02s - 1524.86s | Click here. 1525.94s - 1528.46s | And then once you click here, a bunch of shit happens, 1528.46s - 1530.58s | including like execution, persistence, 1530.58s - 1533.90s | privilege escalation, you know, the whole thing 1533.90s - 1535.42s | until you end up at impact, right? 1535.42s - 1537.02s | You end up with some sort of like ransom. 1537.02s - 1538.46s | Now I'm extorting you. 1538.46s - 1539.50s | You end up, maybe it was like 1539.50s - 1540.66s | some sort of command and control. 1540.66s - 1542.90s | Now we have something that keeps on moving your mouse 1542.90s - 1544.90s | or something like that, right? 1544.90s - 1546.78s | Okay, so that's the fun stuff at MITRE. 1546.78s - 1549.38s | One more fun thing at MITRE. 1549.38s - 1553.02s | At the SOC, we are always talking about single alerts, 1553.02s - 1556.58s | single isolated, disjointed, siloed alerts. 1556.58s - 1559.94s | And these are not how people talk about incidents. 1559.94s -


chunk_text:
you like an email 1519.38s - 1521.62s | and tell you, yo, what's up? 1521.62s - 1524.02s | You won a hundred million Bitcoins. 1524.02s - 1524.86s | Click here. 1525.94s - 1528.46s | And then once you click here, a bunch of shit happens, 1528.46s - 1530.58s | including like execution, persistence, 1530.58s - 1533.90s | privilege escalation, you know, the whole thing 1533.90s - 1535.42s | until you end up at impact, right? 1535.42s - 1537.02s | You end up with some sort of like ransom. 1537.02s - 1538.46s | Now I'm extorting you. 1538.46s - 1539.50s | You end up, maybe it was like 1539.50s - 1540.66s | some sort of command and control. 1540.66s - 1542.90s | Now we have something that keeps on moving your mouse 1542.90s - 1544.90s | or something like that, right? 1544.90s - 1546.78s | Okay, so that's the fun stuff at MITRE. 1546.78s - 1549.38s | One more fun thing at MITRE. 1549.38s - 1553.02s | At the SOC, we are always talking about single alerts, 1553.02s - 1556.58s | single isolated, disjointed, siloed alerts. 1556.58s - 1559.94s | And these are not how people talk about incidents. 1559.94s - 1561.70s | If anybody here can talk about an incident, 1561.70s - 1562.62s | that would be nice. 1562.62s - 1563.82s | I don't mean to pick on you guys, 1563.82s - 1565.98s | but can you share a single incident 1565.98s - 1568.30s | that was like a full investigation, like end to end? 1568.30s - 1570.34s | I know George might share one. 1570.34s - 1571.54s | He kind of prepared one. 1572.54s - 1574.30s | Or if you can go for it as well. 1574.30s - 1576.18s | Whoever wants to go for like a full investigation, 1576.18s - 1578.06s | like some sort of an APT that you can share, 1578.06s - 1578.90s | some sort of like- 1578.90s - 1581.70s | Yeah, APTs I can't share. 1581.70s - 1582.54s | Right. 1582.54s - 1584.18s | I can't share APTs, sorry. 1584.18s - 1585.78s | The only thing I can share, 1585.78s - 1587.74s | and you kind of saw or heard earlier. 1587.74s - 1589.30s | Yeah, he's probably the only guy that will share 1589.30s - 1592.26s | because DOD, DOD, so, you know. 1592.26s - 1594.58s | This is fine because it happened


chunk_text:
1561.70s | If anybody here can talk about an incident, 1561.70s - 1562.62s | that would be nice. 1562.62s - 1563.82s | I don't mean to pick on you guys, 1563.82s - 1565.98s | but can you share a single incident 1565.98s - 1568.30s | that was like a full investigation, like end to end? 1568.30s - 1570.34s | I know George might share one. 1570.34s - 1571.54s | He kind of prepared one. 1572.54s - 1574.30s | Or if you can go for it as well. 1574.30s - 1576.18s | Whoever wants to go for like a full investigation, 1576.18s - 1578.06s | like some sort of an APT that you can share, 1578.06s - 1578.90s | some sort of like- 1578.90s - 1581.70s | Yeah, APTs I can't share. 1581.70s - 1582.54s | Right. 1582.54s - 1584.18s | I can't share APTs, sorry. 1584.18s - 1585.78s | The only thing I can share, 1585.78s - 1587.74s | and you kind of saw or heard earlier. 1587.74s - 1589.30s | Yeah, he's probably the only guy that will share 1589.30s - 1592.26s | because DOD, DOD, so, you know. 1592.26s - 1594.58s | This is fine because it happened years ago 1594.58s - 1597.46s | and we weren't like told we had to be classified about it, 1597.46s - 1600.86s | but like beyond the details of it. 1600.90s - 1604.66s | But basically we had stopped at my previous employer, 1604.66s - 1607.62s | a series of persistent DDoS attacks 1607.62s - 1609.54s | that the campaign ran for, 1609.54s - 1612.50s | I would say arguably about maybe almost two weeks. 1612.50s - 1614.30s | And every day we were getting progressively worse 1614.30s - 1615.86s | and worse attempts at us. 1615.86s - 1618.18s | We were getting like higher and higher ping rates. 1618.18s - 1621.98s | And so it was really crippling the latency 1621.98s - 1624.58s | of our entire organization, our ability to work. 1625.74s - 1628.94s | Eventually we were able to kind of change 1628.94s - 1629.82s | rate limiting rules. 1629.82s - 1631.54s | We were able to change certain configurations 1631.54s - 1633.14s | within the infrastructure that allowed us 1633.14s - 1636.34s | to retain traffic flow once again. 1636.34s - 1640.22s | And suddenly we found that the attacks had stopped 1640.22s - 1642.66s | and


chunk_text:
years ago 1594.58s - 1597.46s | and we weren't like told we had to be classified about it, 1597.46s - 1600.86s | but like beyond the details of it. 1600.90s - 1604.66s | But basically we had stopped at my previous employer, 1604.66s - 1607.62s | a series of persistent DDoS attacks 1607.62s - 1609.54s | that the campaign ran for, 1609.54s - 1612.50s | I would say arguably about maybe almost two weeks. 1612.50s - 1614.30s | And every day we were getting progressively worse 1614.30s - 1615.86s | and worse attempts at us. 1615.86s - 1618.18s | We were getting like higher and higher ping rates. 1618.18s - 1621.98s | And so it was really crippling the latency 1621.98s - 1624.58s | of our entire organization, our ability to work. 1625.74s - 1628.94s | Eventually we were able to kind of change 1628.94s - 1629.82s | rate limiting rules. 1629.82s - 1631.54s | We were able to change certain configurations 1631.54s - 1633.14s | within the infrastructure that allowed us 1633.14s - 1636.34s | to retain traffic flow once again. 1636.34s - 1640.22s | And suddenly we found that the attacks had stopped 1640.22s - 1642.66s | and we didn't really know what happened. 1644.30s - 1646.94s | We got a weird call, because I'm Canadian. 1646.94s - 1649.98s | We had a weird call about, I'd say 12 days later 1649.98s - 1652.14s | from our public safety department, 1652.14s - 1654.18s | which is kind of like our department 1654.18s - 1656.10s | that has all the intelligence agencies. 1656.10s - 1658.74s | And so folks from our CSIS, 1658.74s - 1662.34s | which is like our CIA, came up and told us, 1662.34s - 1663.90s | and they sat us down for a briefing 1663.90s - 1665.66s | with our senior executives, 1665.66s - 1669.66s | that we were actually, we were attacked by Fancy Bear. 1669.66s - 1672.90s | And we'd actually repelled the attack by Fancy Bear, 1672.90s - 1674.50s | which was fucking wild. 1677.18s - 1679.62s | But really what they were trying to do 1679.62s - 1681.54s | is they were trying to take down the ability 1681.54s - 1683.10s | of our organization to operate, 1683.10s - 1685.46s | because we were helping with research 1685.46s - 1687.62s | for missile targeting systems 1687.62s - 1689.98s | and other


chunk_text:
we didn't really know what happened. 1644.30s - 1646.94s | We got a weird call, because I'm Canadian. 1646.94s - 1649.98s | We had a weird call about, I'd say 12 days later 1649.98s - 1652.14s | from our public safety department, 1652.14s - 1654.18s | which is kind of like our department 1654.18s - 1656.10s | that has all the intelligence agencies. 1656.10s - 1658.74s | And so folks from our CSIS, 1658.74s - 1662.34s | which is like our CIA, came up and told us, 1662.34s - 1663.90s | and they sat us down for a briefing 1663.90s - 1665.66s | with our senior executives, 1665.66s - 1669.66s | that we were actually, we were attacked by Fancy Bear. 1669.66s - 1672.90s | And we'd actually repelled the attack by Fancy Bear, 1672.90s - 1674.50s | which was fucking wild. 1677.18s - 1679.62s | But really what they were trying to do 1679.62s - 1681.54s | is they were trying to take down the ability 1681.54s - 1683.10s | of our organization to operate, 1683.10s - 1685.46s | because we were helping with research 1685.46s - 1687.62s | for missile targeting systems 1687.62s - 1689.98s | and other kind of modern technology 1689.98s - 1693.10s | that the Canadian Armed Forces was developing. 1693.10s - 1695.58s | Unfortunately, they weren't, well, fortunately for us, 1695.58s - 1698.02s | but unfortunately for them, they weren't successful. 1698.02s - 1700.46s | But what was amazing to me was that for us, 1700.46s - 1702.18s | we just cut to the fundamentals 1702.18s - 1704.10s | of how to repel a DDoS attack, 1706.02s - 1708.06s | where we didn't know that we were fighting 1708.06s - 1709.78s | a nation state actor. 1709.78s - 1711.74s | We didn't know until after the fact, 1711.74s - 1714.02s | until the government came and told us, 1714.02s - 1716.86s | this is actually what you were up against. 1716.86s - 1718.98s | To this day, I'm still actually blown away by that, 1718.98s - 1720.42s | because we had no clue. 1720.42s - 1723.94s | To us, it just seemed like a really, really persistent 1723.94s - 1727.22s | APT group that was just a run-of-the-mill APT group, 1727.22s - 1731.66s | because it was really just a really annoying DDoS attack 1731.66s - 1734.30s | that didn't really have any special


chunk_text:
kind of modern technology 1689.98s - 1693.10s | that the Canadian Armed Forces was developing. 1693.10s - 1695.58s | Unfortunately, they weren't, well, fortunately for us, 1695.58s - 1698.02s | but unfortunately for them, they weren't successful. 1698.02s - 1700.46s | But what was amazing to me was that for us, 1700.46s - 1702.18s | we just cut to the fundamentals 1702.18s - 1704.10s | of how to repel a DDoS attack, 1706.02s - 1708.06s | where we didn't know that we were fighting 1708.06s - 1709.78s | a nation state actor. 1709.78s - 1711.74s | We didn't know until after the fact, 1711.74s - 1714.02s | until the government came and told us, 1714.02s - 1716.86s | this is actually what you were up against. 1716.86s - 1718.98s | To this day, I'm still actually blown away by that, 1718.98s - 1720.42s | because we had no clue. 1720.42s - 1723.94s | To us, it just seemed like a really, really persistent 1723.94s - 1727.22s | APT group that was just a run-of-the-mill APT group, 1727.22s - 1731.66s | because it was really just a really annoying DDoS attack 1731.66s - 1734.30s | that didn't really have any special parameters to it. 1734.30s - 1736.02s | So I found that kind of fascinating, 1736.02s - 1738.62s | and I don't really think that's a standard APT for them, 1738.62s - 1742.54s | because I've seen the old APT32 blogs 1742.54s - 1743.78s | come out all the time 1743.78s - 1746.70s | with some of the innovative social engineering tactics 1747.54s - 1748.46s | that they like to use. 1748.46s - 1749.90s | But I think on this one, 1749.90s - 1752.14s | they were just looking at who's part of the supply chain 1752.14s - 1755.02s | for this project, and let's attack every organization 1755.02s - 1757.22s | as part of that build for that construction. 1757.22s - 1760.42s | So it's kind of an interesting kind of outlook, 1760.42s - 1762.06s | but you have to look at, 1762.06s - 1764.66s | when you are attacked by a nation state 1764.66s - 1766.82s | or a large APT actor, 1766.82s - 1768.58s | you have to look at it from the human level 1768.58s - 1771.26s | as to why they are trying to attack your organization, 1771.26s - 1773.86s | and what are the crown


chunk_text:
parameters to it. 1734.30s - 1736.02s | So I found that kind of fascinating, 1736.02s - 1738.62s | and I don't really think that's a standard APT for them, 1738.62s - 1742.54s | because I've seen the old APT32 blogs 1742.54s - 1743.78s | come out all the time 1743.78s - 1746.70s | with some of the innovative social engineering tactics 1747.54s - 1748.46s | that they like to use. 1748.46s - 1749.90s | But I think on this one, 1749.90s - 1752.14s | they were just looking at who's part of the supply chain 1752.14s - 1755.02s | for this project, and let's attack every organization 1755.02s - 1757.22s | as part of that build for that construction. 1757.22s - 1760.42s | So it's kind of an interesting kind of outlook, 1760.42s - 1762.06s | but you have to look at, 1762.06s - 1764.66s | when you are attacked by a nation state 1764.66s - 1766.82s | or a large APT actor, 1766.82s - 1768.58s | you have to look at it from the human level 1768.58s - 1771.26s | as to why they are trying to attack your organization, 1771.26s - 1773.86s | and what are the crown jewels they're trying to go after. 1773.86s - 1775.78s | So if they're trying to just disable your ability 1775.82s - 1777.22s | to function and operate, 1777.22s - 1778.94s | that's legitimate means of compromise 1778.94s - 1780.30s | that they're trying to achieve. 1780.30s - 1782.98s | If they're trying to steal and exfiltrate data, 1782.98s - 1785.18s | that operation's probably gonna be more malicious, 1785.18s - 1788.06s | because they're gonna have to get inside your environment 1788.06s - 1790.14s | and then move laterally across, 1790.14s - 1791.78s | probably break access at some point 1791.78s - 1793.74s | or get some kind of privileged account, 1793.74s - 1796.30s | and then be able to exfiltrate the data 1796.30s - 1799.02s | to a C2 somewhere without you noticing. 1799.02s - 1801.46s | So oftentimes you're gonna deal with like, 1801.46s - 1802.30s | what's it called? 1807.30s - 1808.14s | Low and slow? 1808.14s - 1809.10s | Low and slow, there we go. 1809.10s - 1810.42s | Low and slow. 1810.42s - 1811.90s | By the way, for those of you 1811.90s - 1815.54s | who are playing in machine learning and anomaly detection,


chunk_text:
jewels they're trying to go after. 1773.86s - 1775.78s | So if they're trying to just disable your ability 1775.82s - 1777.22s | to function and operate, 1777.22s - 1778.94s | that's legitimate means of compromise 1778.94s - 1780.30s | that they're trying to achieve. 1780.30s - 1782.98s | If they're trying to steal and exfiltrate data, 1782.98s - 1785.18s | that operation's probably gonna be more malicious, 1785.18s - 1788.06s | because they're gonna have to get inside your environment 1788.06s - 1790.14s | and then move laterally across, 1790.14s - 1791.78s | probably break access at some point 1791.78s - 1793.74s | or get some kind of privileged account, 1793.74s - 1796.30s | and then be able to exfiltrate the data 1796.30s - 1799.02s | to a C2 somewhere without you noticing. 1799.02s - 1801.46s | So oftentimes you're gonna deal with like, 1801.46s - 1802.30s | what's it called? 1807.30s - 1808.14s | Low and slow? 1808.14s - 1809.10s | Low and slow, there we go. 1809.10s - 1810.42s | Low and slow. 1810.42s - 1811.90s | By the way, for those of you 1811.90s - 1815.54s | who are playing in machine learning and anomaly detection, 1815.54s - 1817.26s | anomaly detection is very good 1817.26s - 1819.66s | at giving you low and slow prospects. 1819.66s - 1820.50s | Yeah. 1820.50s - 1821.34s | Yeah. 1821.34s - 1823.62s | That's been the best use case I've got so far. 1823.62s - 1825.62s | But they can maintain your presence in your environment 1825.62s - 1827.10s | for months without you realizing it, 1827.10s - 1827.94s | if they do it correctly. 1827.94s - 1828.86s | If they do it correctly, 1828.86s - 1830.30s | but at the very least, 1830.30s - 1833.06s | when you're dealing with just trying to find odd traffic 1833.06s - 1835.82s | that you'll not spot, 1835.82s - 1839.50s | otherwise, that's the sort of anomaly 1839.50s - 1841.58s | that you are actually trying to leverage. 1842.42s - 1846.26s | But most of the people that I know that have UEBA still, 1846.26s - 1848.38s | but the people that don't have UEBA, 1848.38s - 1849.70s | they don't have the anomaly detection 1849.70s - 1853.26s | because they end up dialing it down so much 1853.26s - 1854.62s | in face of false positives, 1854.62s - 1857.06s |


chunk_text:
1815.54s - 1817.26s | anomaly detection is very good 1817.26s - 1819.66s | at giving you low and slow prospects. 1819.66s - 1820.50s | Yeah. 1820.50s - 1821.34s | Yeah. 1821.34s - 1823.62s | That's been the best use case I've got so far. 1823.62s - 1825.62s | But they can maintain your presence in your environment 1825.62s - 1827.10s | for months without you realizing it, 1827.10s - 1827.94s | if they do it correctly. 1827.94s - 1828.86s | If they do it correctly, 1828.86s - 1830.30s | but at the very least, 1830.30s - 1833.06s | when you're dealing with just trying to find odd traffic 1833.06s - 1835.82s | that you'll not spot, 1835.82s - 1839.50s | otherwise, that's the sort of anomaly 1839.50s - 1841.58s | that you are actually trying to leverage. 1842.42s - 1846.26s | But most of the people that I know that have UEBA still, 1846.26s - 1848.38s | but the people that don't have UEBA, 1848.38s - 1849.70s | they don't have the anomaly detection 1849.70s - 1853.26s | because they end up dialing it down so much 1853.26s - 1854.62s | in face of false positives, 1854.62s - 1857.06s | they miss the slow and stealthy stuff. 1857.06s - 1859.82s | They only wanna see the extra, extra, extra spicy stuff 1860.26s - 1861.98s | that the rule can get them. 1861.98s - 1864.34s | And that mindset's part of the problem. 1864.34s - 1865.42s | The other thing is, 1866.86s - 1870.18s | for the really loud, obvious stuff for the fault, 1870.18s - 1872.30s | which is generates tons of false positives, 1872.30s - 1876.34s | you should be tuning your inputs 1876.34s - 1879.62s | to your anomaly detection algorithm beforehand. 1881.46s - 1883.34s | If I know this, 1883.34s - 1886.34s | and your analysts should be able to also go through 1886.34s - 1889.30s | and say, right, here's the ones that are, 1889.30s - 1891.26s | this traffic I've seen elsewhere, 1892.14s - 1894.94s | it's loud, this isn't a good prospect because X, 1894.94s - 1896.98s | and then go to Y. 1896.98s - 1897.82s | Hell yeah. 1897.82s - 1900.82s | Yeah, I mean, it goes back to people on process. 1900.82s - 1903.62s | Like you have anomaly detection there for years, 1903.62s - 1905.94s | people don't use it because


chunk_text:
they miss the slow and stealthy stuff. 1857.06s - 1859.82s | They only wanna see the extra, extra, extra spicy stuff 1860.26s - 1861.98s | that the rule can get them. 1861.98s - 1864.34s | And that mindset's part of the problem. 1864.34s - 1865.42s | The other thing is, 1866.86s - 1870.18s | for the really loud, obvious stuff for the fault, 1870.18s - 1872.30s | which is generates tons of false positives, 1872.30s - 1876.34s | you should be tuning your inputs 1876.34s - 1879.62s | to your anomaly detection algorithm beforehand. 1881.46s - 1883.34s | If I know this, 1883.34s - 1886.34s | and your analysts should be able to also go through 1886.34s - 1889.30s | and say, right, here's the ones that are, 1889.30s - 1891.26s | this traffic I've seen elsewhere, 1892.14s - 1894.94s | it's loud, this isn't a good prospect because X, 1894.94s - 1896.98s | and then go to Y. 1896.98s - 1897.82s | Hell yeah. 1897.82s - 1900.82s | Yeah, I mean, it goes back to people on process. 1900.82s - 1903.62s | Like you have anomaly detection there for years, 1903.62s - 1905.94s | people don't use it because they don't tune it. 1905.94s - 1908.54s | And when they tune it, they tune it out. 1908.54s - 1911.14s | They literally just, it's too many false positives, 1911.14s - 1912.62s | let's just get rid of it. 1912.62s - 1915.42s | Because if I have all of the slow and stealthy appearing, 1915.42s - 1917.74s | I have all of the garbage appearing. 1917.74s - 1919.22s | So it's a trade-off. 1919.98s - 1921.38s | Do I wanna see all the slow and stealthy, 1921.38s - 1924.46s | or do I wanna not have a million fucking tickets every day? 1925.98s - 1929.46s | I think there's another issue of insider threats, right? 1929.46s - 1932.02s | Two of some of the biggest ones I've seen 1932.02s - 1936.02s | that I was a part of that organization 1936.02s - 1940.46s | were people that were either fired or about to quit, 1940.46s - 1942.82s | and they set up the environment. 1942.82s - 1945.10s | And if you don't have that detection, 1946.02s - 1949.66s | and someone's doing this setting up of the environment, 1949.66s - 1953.74s | and they're starting to have a bunch of


chunk_text:
they don't tune it. 1905.94s - 1908.54s | And when they tune it, they tune it out. 1908.54s - 1911.14s | They literally just, it's too many false positives, 1911.14s - 1912.62s | let's just get rid of it. 1912.62s - 1915.42s | Because if I have all of the slow and stealthy appearing, 1915.42s - 1917.74s | I have all of the garbage appearing. 1917.74s - 1919.22s | So it's a trade-off. 1919.98s - 1921.38s | Do I wanna see all the slow and stealthy, 1921.38s - 1924.46s | or do I wanna not have a million fucking tickets every day? 1925.98s - 1929.46s | I think there's another issue of insider threats, right? 1929.46s - 1932.02s | Two of some of the biggest ones I've seen 1932.02s - 1936.02s | that I was a part of that organization 1936.02s - 1940.46s | were people that were either fired or about to quit, 1940.46s - 1942.82s | and they set up the environment. 1942.82s - 1945.10s | And if you don't have that detection, 1946.02s - 1949.66s | and someone's doing this setting up of the environment, 1949.66s - 1953.74s | and they're starting to have a bunch of scripts execute 1953.74s - 1956.70s | to create accounts, and get into those accounts, 1956.70s - 1959.50s | and piggyback on that, and create scripts 1959.50s - 1960.74s | to start shutting things down, 1960.74s - 1963.46s | or moving data, whatever it is, or both, 1965.38s - 1967.46s | you should be able to detect those things 1967.46s - 1968.78s | before those people depart. 1968.78s - 1970.74s | That actually brings me up another point. 1970.74s - 1972.58s | You should really invest in DLP. 1973.58s - 1976.54s | Data loss prevention is a major issue, 1976.54s - 1978.86s | and particularly when it comes to insider threat. 1978.86s - 1980.10s | So I can give you an example. 1980.10s - 1981.18s | I won't say which employer, 1981.18s - 1983.66s | but let's say we had to let go of an HR person, 1983.66s - 1987.10s | and it was a sudden cut financially-based. 1987.10s - 1990.58s | Their performance was good, but we had to let them go. 1990.58s - 1992.94s | Oftentimes, when folks are put in that position, 1992.94s - 1994.46s | they're gonna try to smash and grab 1994.46s - 1996.14s | as


chunk_text:
scripts execute 1953.74s - 1956.70s | to create accounts, and get into those accounts, 1956.70s - 1959.50s | and piggyback on that, and create scripts 1959.50s - 1960.74s | to start shutting things down, 1960.74s - 1963.46s | or moving data, whatever it is, or both, 1965.38s - 1967.46s | you should be able to detect those things 1967.46s - 1968.78s | before those people depart. 1968.78s - 1970.74s | That actually brings me up another point. 1970.74s - 1972.58s | You should really invest in DLP. 1973.58s - 1976.54s | Data loss prevention is a major issue, 1976.54s - 1978.86s | and particularly when it comes to insider threat. 1978.86s - 1980.10s | So I can give you an example. 1980.10s - 1981.18s | I won't say which employer, 1981.18s - 1983.66s | but let's say we had to let go of an HR person, 1983.66s - 1987.10s | and it was a sudden cut financially-based. 1987.10s - 1990.58s | Their performance was good, but we had to let them go. 1990.58s - 1992.94s | Oftentimes, when folks are put in that position, 1992.94s - 1994.46s | they're gonna try to smash and grab 1994.46s - 1996.14s | as many files as possible, 1996.14s - 1997.98s | perhaps to spin off in a consultancy, 1997.98s - 1999.10s | or do whatever it is they're gonna do 1999.10s - 2000.94s | with their business after the fact. 2000.94s - 2002.82s | But obviously, corporate data, 2002.82s - 2004.62s | you can't just carry home with you. 2004.62s - 2006.74s | That's not the way it works. 2006.74s - 2009.54s | So I knew about a month in advance 2009.54s - 2011.74s | of when this person was gonna get let go, 2011.74s - 2015.46s | and so I kind of set up a surveillance 2015.46s - 2017.42s | with my SOC manager, and I was like, 2017.42s - 2019.14s | hey, can you watch this person? 2019.14s - 2021.46s | Can you specifically see what's happening 2021.46s - 2023.94s | in their endpoint over the next few weeks? 2023.94s - 2026.06s | So it was kind of an interesting process 2026.06s - 2028.40s | where they were gonna get notice 2028.40s - 2029.74s | that we were gonna let them go, 2029.74s - 2031.42s | and then there was a two-week buffer window 2031.42s - 2033.66s | for them to


chunk_text:
many files as possible, 1996.14s - 1997.98s | perhaps to spin off in a consultancy, 1997.98s - 1999.10s | or do whatever it is they're gonna do 1999.10s - 2000.94s | with their business after the fact. 2000.94s - 2002.82s | But obviously, corporate data, 2002.82s - 2004.62s | you can't just carry home with you. 2004.62s - 2006.74s | That's not the way it works. 2006.74s - 2009.54s | So I knew about a month in advance 2009.54s - 2011.74s | of when this person was gonna get let go, 2011.74s - 2015.46s | and so I kind of set up a surveillance 2015.46s - 2017.42s | with my SOC manager, and I was like, 2017.42s - 2019.14s | hey, can you watch this person? 2019.14s - 2021.46s | Can you specifically see what's happening 2021.46s - 2023.94s | in their endpoint over the next few weeks? 2023.94s - 2026.06s | So it was kind of an interesting process 2026.06s - 2028.40s | where they were gonna get notice 2028.40s - 2029.74s | that we were gonna let them go, 2029.74s - 2031.42s | and then there was a two-week buffer window 2031.42s - 2033.66s | for them to kind of hand off everything. 2033.66s - 2035.62s | And I suspected within that two weeks, 2035.62s - 2037.90s | they would attempt to do a big data exfil. 2037.90s - 2039.54s | And what happened, it was like, 2039.54s - 2042.00s | I think a day before we cut off 2042.00s - 2043.90s | all their accounts and everything, 2043.90s - 2045.88s | a massive data exfil. 2045.88s - 2048.26s | So we ended up getting alerted to what they were doing 2048.26s - 2050.50s | because we had our endpoint sensors 2050.50s - 2052.38s | detecting what was going on, 2052.38s - 2055.34s | that we had seen the files in the transfer specifically, 2055.34s - 2057.46s | so that we knew exactly by name 2057.46s - 2059.46s | which files had actually gone out. 2060.14s - 2061.66s | So we were able to call this person back, 2061.66s - 2064.42s | be like, hey, can you put those files on a USB 2064.42s - 2066.54s | and please confirm deleting them from your computer 2066.54s - 2070.24s | because they belong to the company. 2070.24s - 2073.42s | All right, so DLP is essentially a bunch of


chunk_text:
kind of hand off everything. 2033.66s - 2035.62s | And I suspected within that two weeks, 2035.62s - 2037.90s | they would attempt to do a big data exfil. 2037.90s - 2039.54s | And what happened, it was like, 2039.54s - 2042.00s | I think a day before we cut off 2042.00s - 2043.90s | all their accounts and everything, 2043.90s - 2045.88s | a massive data exfil. 2045.88s - 2048.26s | So we ended up getting alerted to what they were doing 2048.26s - 2050.50s | because we had our endpoint sensors 2050.50s - 2052.38s | detecting what was going on, 2052.38s - 2055.34s | that we had seen the files in the transfer specifically, 2055.34s - 2057.46s | so that we knew exactly by name 2057.46s - 2059.46s | which files had actually gone out. 2060.14s - 2061.66s | So we were able to call this person back, 2061.66s - 2064.42s | be like, hey, can you put those files on a USB 2064.42s - 2066.54s | and please confirm deleting them from your computer 2066.54s - 2070.24s | because they belong to the company. 2070.24s - 2073.42s | All right, so DLP is essentially a bunch of rules, right? 2073.42s - 2074.96s | More rules that we trust. 2074.96s - 2076.12s | It's not anomalous, right? 2076.12s - 2081.12s | Yeah, it's a, here we set up these sorts of files 2081.72s - 2084.60s | or these sorts of indicators. 2084.60s - 2086.72s | And one of the other things you absolutely need 2086.72s - 2088.86s | is somebody involved in the process 2088.86s - 2091.54s | to know what should be happening. 2091.54s - 2092.58s | It's the deep knowledge. 2092.58s - 2094.54s | You have to have somebody know what should be happening. 2094.54s - 2095.38s | Hell yeah. 2095.38s - 2096.90s | This is what's odd. 2096.90s - 2097.74s | Hell yeah. 2097.74s - 2102.74s | And have a clue as to when to really put that extra. 2104.34s - 2106.06s | Yeah, extra scrutiny kind of thing, you know? 2106.06s - 2106.90s | Extra scrutiny, yeah. 2106.90s - 2107.90s | And that's the context. 2107.90s - 2110.02s | Context was the boss told you, you know, 2110.02s - 2111.76s | like this guy's gonna do some shit. 2111.76s - 2114.18s | That's fucking amazing intel, thank you. 2114.18s - 2117.22s | Somebody's gonna


chunk_text:
rules, right? 2073.42s - 2074.96s | More rules that we trust. 2074.96s - 2076.12s | It's not anomalous, right? 2076.12s - 2081.12s | Yeah, it's a, here we set up these sorts of files 2081.72s - 2084.60s | or these sorts of indicators. 2084.60s - 2086.72s | And one of the other things you absolutely need 2086.72s - 2088.86s | is somebody involved in the process 2088.86s - 2091.54s | to know what should be happening. 2091.54s - 2092.58s | It's the deep knowledge. 2092.58s - 2094.54s | You have to have somebody know what should be happening. 2094.54s - 2095.38s | Hell yeah. 2095.38s - 2096.90s | This is what's odd. 2096.90s - 2097.74s | Hell yeah. 2097.74s - 2102.74s | And have a clue as to when to really put that extra. 2104.34s - 2106.06s | Yeah, extra scrutiny kind of thing, you know? 2106.06s - 2106.90s | Extra scrutiny, yeah. 2106.90s - 2107.90s | And that's the context. 2107.90s - 2110.02s | Context was the boss told you, you know, 2110.02s - 2111.76s | like this guy's gonna do some shit. 2111.76s - 2114.18s | That's fucking amazing intel, thank you. 2114.18s - 2117.22s | Somebody's gonna do some shit, I'll look there, right? 2117.26s - 2119.66s | But if I have anomaly detection, 2119.66s - 2121.58s | George was talking about that a little earlier as well. 2121.58s - 2123.26s | He was like, they can be there for months. 2123.26s - 2124.78s | What does that mean? 2124.78s - 2126.86s | Anomaly detection, my friends, 2126.86s - 2129.58s | needs some sort of a time for baselining. 2129.58s - 2131.18s | It baselines over a month. 2131.18s - 2133.46s | You have someone that does something fishy every year 2133.46s - 2135.22s | because they go and they pull financial metrics 2135.22s - 2136.58s | for the taxes or some bullshit. 2136.58s - 2138.78s | Every single year, you have a ticket. 2138.78s - 2140.50s | Every single year, it doesn't matter. 2140.50s - 2142.38s | And that's why UABA is not the most popular thing 2142.38s - 2143.66s | in the world, and it will never be 2143.66s - 2147.99s | because you have to tune the shit out of it, right? 2148.47s - 2152.07s | For us, we wanna stop fucking hyper-tuning everything. 2152.07s - 2153.75s | We wanna have regular lives. 2153.75s


chunk_text:
do some shit, I'll look there, right? 2117.26s - 2119.66s | But if I have anomaly detection, 2119.66s - 2121.58s | George was talking about that a little earlier as well. 2121.58s - 2123.26s | He was like, they can be there for months. 2123.26s - 2124.78s | What does that mean? 2124.78s - 2126.86s | Anomaly detection, my friends, 2126.86s - 2129.58s | needs some sort of a time for baselining. 2129.58s - 2131.18s | It baselines over a month. 2131.18s - 2133.46s | You have someone that does something fishy every year 2133.46s - 2135.22s | because they go and they pull financial metrics 2135.22s - 2136.58s | for the taxes or some bullshit. 2136.58s - 2138.78s | Every single year, you have a ticket. 2138.78s - 2140.50s | Every single year, it doesn't matter. 2140.50s - 2142.38s | And that's why UABA is not the most popular thing 2142.38s - 2143.66s | in the world, and it will never be 2143.66s - 2147.99s | because you have to tune the shit out of it, right? 2148.47s - 2152.07s | For us, we wanna stop fucking hyper-tuning everything. 2152.07s - 2153.75s | We wanna have regular lives. 2153.75s - 2157.67s | So that takes us to essentially attack flows. 2157.67s - 2160.99s | Attack flows are incidents that, you know, 2160.99s - 2162.87s | when I asked George to talk about an incident, 2162.87s - 2165.67s | he was like, there was a lot of DDoS, 2165.67s - 2169.23s | but DDoS usually happens all the way at the end. 2169.23s - 2173.19s | I know Chris usually has a lot of incidents to share, 2173.19s - 2176.47s | and sometimes they're all over the place. 2176.47s - 2177.91s | Sometimes they're only at the end. 2178.75s - 2181.59s | What happens is, if you only see stuff at the end, 2181.59s - 2183.63s | you just didn't see the rest. 2183.63s - 2185.27s | You just didn't put it together. 2185.27s - 2188.59s | And if you have seen some things but missed a few, 2188.59s - 2190.47s | you know why you missed them, 2190.47s - 2191.99s | because these guys were good, 2191.99s - 2194.95s | and they got your essentially, you know, 2194.95s - 2197.75s | like blind spots in coverage, blind spots in whatever. 2197.75s - 2199.75s | So most of the investigations,


chunk_text:
- 2157.67s | So that takes us to essentially attack flows. 2157.67s - 2160.99s | Attack flows are incidents that, you know, 2160.99s - 2162.87s | when I asked George to talk about an incident, 2162.87s - 2165.67s | he was like, there was a lot of DDoS, 2165.67s - 2169.23s | but DDoS usually happens all the way at the end. 2169.23s - 2173.19s | I know Chris usually has a lot of incidents to share, 2173.19s - 2176.47s | and sometimes they're all over the place. 2176.47s - 2177.91s | Sometimes they're only at the end. 2178.75s - 2181.59s | What happens is, if you only see stuff at the end, 2181.59s - 2183.63s | you just didn't see the rest. 2183.63s - 2185.27s | You just didn't put it together. 2185.27s - 2188.59s | And if you have seen some things but missed a few, 2188.59s - 2190.47s | you know why you missed them, 2190.47s - 2191.99s | because these guys were good, 2191.99s - 2194.95s | and they got your essentially, you know, 2194.95s - 2197.75s | like blind spots in coverage, blind spots in whatever. 2197.75s - 2199.75s | So most of the investigations, 2199.75s - 2201.27s | trying to put the whole story, 2201.27s - 2203.55s | like give me some sort of an attack flow. 2203.55s - 2205.91s | Anybody here familiar with a MITRE attack flow? 2206.87s - 2208.15s | God damn, only three people. 2208.15s - 2210.67s | What the hell is going on with MITRE? 2210.67s - 2213.27s | All right, so I'm gonna pitch MITRE for you. 2213.27s - 2215.87s | Here is what they call a MITRE attack flow. 2215.87s - 2217.19s | I can see strategically 2217.19s - 2220.27s | and mainly a little bit operationally 2220.27s - 2221.91s | what was happening at Equifax, for instance. 2221.91s - 2223.75s | So there was a vulnerability, exploit, whatever, 2223.75s - 2226.79s | you know, like web shell, yada, yada, yada, encrypted shell. 2226.79s - 2228.67s | There was like a lot of queries to the database. 2228.67s - 2231.79s | That's how Equifax got breached. 2231.79s - 2233.27s | And my data is there. 2233.27s - 2235.55s | If you can find my SIN number, my address, 2235.83s - 2238.59s | you can find a lot of information about me if you want. 2238.59s - 2239.63s


chunk_text:
2199.75s - 2201.27s | trying to put the whole story, 2201.27s - 2203.55s | like give me some sort of an attack flow. 2203.55s - 2205.91s | Anybody here familiar with a MITRE attack flow? 2206.87s - 2208.15s | God damn, only three people. 2208.15s - 2210.67s | What the hell is going on with MITRE? 2210.67s - 2213.27s | All right, so I'm gonna pitch MITRE for you. 2213.27s - 2215.87s | Here is what they call a MITRE attack flow. 2215.87s - 2217.19s | I can see strategically 2217.19s - 2220.27s | and mainly a little bit operationally 2220.27s - 2221.91s | what was happening at Equifax, for instance. 2221.91s - 2223.75s | So there was a vulnerability, exploit, whatever, 2223.75s - 2226.79s | you know, like web shell, yada, yada, yada, encrypted shell. 2226.79s - 2228.67s | There was like a lot of queries to the database. 2228.67s - 2231.79s | That's how Equifax got breached. 2231.79s - 2233.27s | And my data is there. 2233.27s - 2235.55s | If you can find my SIN number, my address, 2235.83s - 2238.59s | you can find a lot of information about me if you want. 2238.59s - 2239.63s | Download the data breach 2239.63s - 2242.47s | and please don't steal my identity, but I can't stop you. 2244.11s - 2246.31s | And God knows I spend a lot of time in the bank 2246.31s - 2248.71s | whenever I do something because of this, 2248.71s - 2250.71s | but can't really complain, you know. 2251.67s - 2252.99s | This is the attack flow. 2252.99s - 2255.55s | And they archived a bunch of stuff that they got 2255.55s - 2258.27s | and they exfiltrated it via C2, 2258.27s - 2260.59s | non-C2 protocol via multi-hub proxy and whatnot. 2260.59s - 2263.67s | If you want to watch some more of these, 2263.67s - 2267.63s | instead of APT PDF reports, you can have an APT flow 2267.63s - 2271.19s | and you can see the nice strategic movement there. 2271.19s - 2272.67s | But anyways, why the hell do we care about this? 2272.67s - 2273.95s | We care about this to tell you 2273.95s - 2277.55s | that we have been fed our entire lives in the sock, 2277.55s - 2279.87s | that an alert is important. 2279.87s - 2282.83s | There is something called a


chunk_text:
| Download the data breach 2239.63s - 2242.47s | and please don't steal my identity, but I can't stop you. 2244.11s - 2246.31s | And God knows I spend a lot of time in the bank 2246.31s - 2248.71s | whenever I do something because of this, 2248.71s - 2250.71s | but can't really complain, you know. 2251.67s - 2252.99s | This is the attack flow. 2252.99s - 2255.55s | And they archived a bunch of stuff that they got 2255.55s - 2258.27s | and they exfiltrated it via C2, 2258.27s - 2260.59s | non-C2 protocol via multi-hub proxy and whatnot. 2260.59s - 2263.67s | If you want to watch some more of these, 2263.67s - 2267.63s | instead of APT PDF reports, you can have an APT flow 2267.63s - 2271.19s | and you can see the nice strategic movement there. 2271.19s - 2272.67s | But anyways, why the hell do we care about this? 2272.67s - 2273.95s | We care about this to tell you 2273.95s - 2277.55s | that we have been fed our entire lives in the sock, 2277.55s - 2279.87s | that an alert is important. 2279.87s - 2282.83s | There is something called a notable alert. 2282.83s - 2284.51s | To hell with an alert. 2284.51s - 2285.75s | It's about the story. 2285.75s - 2287.79s | It's about show me the entire freaking flow. 2287.79s - 2289.27s | To hell with a single technique. 2289.27s - 2290.79s | Show me the other shit. 2290.79s - 2292.39s | That's the context. 2292.39s - 2295.15s | Show me a piece of intel that the boss tells me, 2295.15s - 2296.83s | somebody's going to pull some shit on you. 2296.83s - 2298.87s | Good. Now I know where to look. 2298.87s - 2301.03s | Now it's no more a single incident of data exfiltration. 2301.03s - 2303.95s | It's data exfiltration that I have good reasoning for. 2303.95s - 2306.15s | This person is disgruntled and so on and so forth. 2306.15s - 2308.39s | And these kinds of stories are very important. 2308.39s - 2310.83s | And that's what usually goes in an incident report. 2310.83s - 2312.11s | And that's what people like to talk about 2312.11s - 2313.35s | when they save the day. 2313.35s - 2315.95s | You don't save the day against a single incident. 2315.95s - 2317.55s | Sorry,


chunk_text:
notable alert. 2282.83s - 2284.51s | To hell with an alert. 2284.51s - 2285.75s | It's about the story. 2285.75s - 2287.79s | It's about show me the entire freaking flow. 2287.79s - 2289.27s | To hell with a single technique. 2289.27s - 2290.79s | Show me the other shit. 2290.79s - 2292.39s | That's the context. 2292.39s - 2295.15s | Show me a piece of intel that the boss tells me, 2295.15s - 2296.83s | somebody's going to pull some shit on you. 2296.83s - 2298.87s | Good. Now I know where to look. 2298.87s - 2301.03s | Now it's no more a single incident of data exfiltration. 2301.03s - 2303.95s | It's data exfiltration that I have good reasoning for. 2303.95s - 2306.15s | This person is disgruntled and so on and so forth. 2306.15s - 2308.39s | And these kinds of stories are very important. 2308.39s - 2310.83s | And that's what usually goes in an incident report. 2310.83s - 2312.11s | And that's what people like to talk about 2312.11s - 2313.35s | when they save the day. 2313.35s - 2315.95s | You don't save the day against a single incident. 2315.95s - 2317.55s | Sorry, against a single alert. 2317.55s - 2319.67s | You save the day against a full attack. 2319.67s - 2322.23s | It might be a full playbook that executes in 20 seconds. 2322.23s - 2324.23s | It might be an APT that's just been there 2324.23s - 2326.23s | for a couple of years, whatever it is. 2326.23s - 2327.19s | It's not one thing. 2327.19s - 2328.23s | It looks like this. 2329.15s - 2331.91s | So can we do this with data science? 2331.91s - 2333.91s | You know, I came here for data science. 2333.91s - 2334.75s | You baited me. 2334.75s - 2336.15s | And now you're feeding me a lot of bullshit 2336.15s - 2337.07s | about people process. 2337.07s - 2338.23s | Give me some technology, my dude, 2338.23s - 2339.39s | or I'm going to leave right now. 2339.39s - 2340.51s | Gotcha. Okay. 2340.51s - 2342.07s | Here's some technology. 2342.07s - 2345.19s | So we need some sort of finding 2345.19s - 2346.63s | mitered attack techniques for everything. 2346.63s - 2348.79s | Why? Because it's a what? 2348.79s - 2351.79s | If I hire unicorns like you guys, 2351.79s - 2354.15s


chunk_text:
against a single alert. 2317.55s - 2319.67s | You save the day against a full attack. 2319.67s - 2322.23s | It might be a full playbook that executes in 20 seconds. 2322.23s - 2324.23s | It might be an APT that's just been there 2324.23s - 2326.23s | for a couple of years, whatever it is. 2326.23s - 2327.19s | It's not one thing. 2327.19s - 2328.23s | It looks like this. 2329.15s - 2331.91s | So can we do this with data science? 2331.91s - 2333.91s | You know, I came here for data science. 2333.91s - 2334.75s | You baited me. 2334.75s - 2336.15s | And now you're feeding me a lot of bullshit 2336.15s - 2337.07s | about people process. 2337.07s - 2338.23s | Give me some technology, my dude, 2338.23s - 2339.39s | or I'm going to leave right now. 2339.39s - 2340.51s | Gotcha. Okay. 2340.51s - 2342.07s | Here's some technology. 2342.07s - 2345.19s | So we need some sort of finding 2345.19s - 2346.63s | mitered attack techniques for everything. 2346.63s - 2348.79s | Why? Because it's a what? 2348.79s - 2351.79s | If I hire unicorns like you guys, 2351.79s - 2354.15s | I don't need to detail everything for a what. 2354.15s - 2358.87s | But if I hire the 20 resumes that I got in Las Vegas, 2358.87s - 2360.91s | they're all juniors and they need jobs. 2360.91s - 2362.67s | And we in security like to say 2362.67s - 2364.23s | that we have a lack of talent. 2364.23s - 2365.59s | Fuck no. 2365.59s - 2367.55s | We have no talent shortage. 2367.55s - 2369.59s | We have unicorn shortage. 2369.59s - 2372.15s | I can not hire someone like you guys, 2372.15s - 2376.75s | but I can hire a shit ton of people that are juniors. 2376.75s - 2378.87s | And because I can't afford you within my budget, 2378.87s - 2380.59s | I just say, oh, I have a talent shortage. 2380.59s - 2382.91s | No, I just can't afford you. 2382.91s - 2384.03s | And you're too few. 2384.03s - 2387.27s | I can find a lot of juniors and beginners. 2387.27s - 2389.59s | So if I give them a mitered attack for everything, 2389.59s - 2390.87s | their life could be slightly better 2390.87s - 2392.39s | because now they know


chunk_text:
| I don't need to detail everything for a what. 2354.15s - 2358.87s | But if I hire the 20 resumes that I got in Las Vegas, 2358.87s - 2360.91s | they're all juniors and they need jobs. 2360.91s - 2362.67s | And we in security like to say 2362.67s - 2364.23s | that we have a lack of talent. 2364.23s - 2365.59s | Fuck no. 2365.59s - 2367.55s | We have no talent shortage. 2367.55s - 2369.59s | We have unicorn shortage. 2369.59s - 2372.15s | I can not hire someone like you guys, 2372.15s - 2376.75s | but I can hire a shit ton of people that are juniors. 2376.75s - 2378.87s | And because I can't afford you within my budget, 2378.87s - 2380.59s | I just say, oh, I have a talent shortage. 2380.59s - 2382.91s | No, I just can't afford you. 2382.91s - 2384.03s | And you're too few. 2384.03s - 2387.27s | I can find a lot of juniors and beginners. 2387.27s - 2389.59s | So if I give them a mitered attack for everything, 2389.59s - 2390.87s | their life could be slightly better 2390.87s - 2392.39s | because now they know the procedural examples, 2392.39s - 2393.35s | mitigation and detection. 2393.35s - 2394.27s | They have the description. 2394.27s - 2397.27s | They have a common language to talk about with me, 2397.27s - 2398.95s | especially when they see anything in the sim 2398.95s - 2400.67s | or in any other queue. 2400.67s - 2402.11s | So how do I get them everything? 2402.11s - 2403.15s | With mitered attack techniques? 2403.15s - 2406.75s | Well, you know, I can give them something that I get from, 2406.75s - 2409.59s | I don't know, sometimes my EDR has mitered attack, right? 2409.63s - 2412.31s | Anybody here has mitered attack from the EDR? 2412.31s - 2413.91s | Yeah, a bunch of us. 2413.91s - 2416.47s | So sometimes the NDR would give you some, you know, 2416.47s - 2419.27s | but most of the data does not have mitered attack. 2419.27s - 2422.11s | And the mitered attack that you have was for marketing. 2422.11s - 2423.19s | It was for dashboards. 2423.19s - 2426.39s | It was for sexy stuff at RSA and Black Hat. 2426.39s - 2428.51s | It was not to actually use it. 2428.51s -


chunk_text:
the procedural examples, 2392.39s - 2393.35s | mitigation and detection. 2393.35s - 2394.27s | They have the description. 2394.27s - 2397.27s | They have a common language to talk about with me, 2397.27s - 2398.95s | especially when they see anything in the sim 2398.95s - 2400.67s | or in any other queue. 2400.67s - 2402.11s | So how do I get them everything? 2402.11s - 2403.15s | With mitered attack techniques? 2403.15s - 2406.75s | Well, you know, I can give them something that I get from, 2406.75s - 2409.59s | I don't know, sometimes my EDR has mitered attack, right? 2409.63s - 2412.31s | Anybody here has mitered attack from the EDR? 2412.31s - 2413.91s | Yeah, a bunch of us. 2413.91s - 2416.47s | So sometimes the NDR would give you some, you know, 2416.47s - 2419.27s | but most of the data does not have mitered attack. 2419.27s - 2422.11s | And the mitered attack that you have was for marketing. 2422.11s - 2423.19s | It was for dashboards. 2423.19s - 2426.39s | It was for sexy stuff at RSA and Black Hat. 2426.39s - 2428.51s | It was not to actually use it. 2428.51s - 2430.35s | So ideally we have a mitered attack 2430.35s - 2432.71s | that we can actually use that is not crowdsourced 2432.71s - 2434.99s | where every single one was given by a different author. 2434.99s - 2435.83s | Ideally we have one thing 2435.83s - 2437.71s | that gives everything a mitered attack. 2437.71s - 2438.95s | How do I do that? 2439.03s - 2440.67s | I can just deed up everything in my sim 2440.67s - 2443.31s | and get all the different messages and signatures 2443.31s - 2444.35s | that would ever trigger. 2444.35s - 2446.83s | And I, you know, hand annotate them. 2446.83s - 2448.51s | That's the version 0.1. 2448.51s - 2451.59s | Version 0.2 is to use an LLM like ChatGPT 2451.59s - 2453.03s | and let ChatGPT do it. 2453.03s - 2456.83s | Like I can go to ChatGPT and I'd say, you know, 2456.83s - 2459.95s | ChatGPT, what technique is happening here and there? 2459.95s - 2461.43s | And it would tell me, you know, 2461.43s - 2466.43s | what mitered technique is run DLL 32 registry run key? 2467.43s - 2469.63s | You know, whatever. 2469.63s - 2471.95s


chunk_text:
2430.35s | So ideally we have a mitered attack 2430.35s - 2432.71s | that we can actually use that is not crowdsourced 2432.71s - 2434.99s | where every single one was given by a different author. 2434.99s - 2435.83s | Ideally we have one thing 2435.83s - 2437.71s | that gives everything a mitered attack. 2437.71s - 2438.95s | How do I do that? 2439.03s - 2440.67s | I can just deed up everything in my sim 2440.67s - 2443.31s | and get all the different messages and signatures 2443.31s - 2444.35s | that would ever trigger. 2444.35s - 2446.83s | And I, you know, hand annotate them. 2446.83s - 2448.51s | That's the version 0.1. 2448.51s - 2451.59s | Version 0.2 is to use an LLM like ChatGPT 2451.59s - 2453.03s | and let ChatGPT do it. 2453.03s - 2456.83s | Like I can go to ChatGPT and I'd say, you know, 2456.83s - 2459.95s | ChatGPT, what technique is happening here and there? 2459.95s - 2461.43s | And it would tell me, you know, 2461.43s - 2466.43s | what mitered technique is run DLL 32 registry run key? 2467.43s - 2469.63s | You know, whatever. 2469.63s - 2471.95s | It will tell me, it might not be good, 2471.95s - 2473.75s | but it could be consistent. 2473.75s - 2475.39s | Can ChatGPT be consistent? 2476.51s - 2479.23s | Depends, but if you really are after consistency, 2479.23s - 2484.23s | you go to a model that you can tune the API off. 2485.39s - 2486.95s | And if you tune the API, 2486.95s - 2489.07s | it would look something like this. 2489.07s - 2490.43s | You go to the settings on Hugging Chat. 2490.43s - 2491.51s | If you don't know about Hugging Chat, 2491.51s - 2495.51s | it's just a way to get LLMs tuned for your use cases. 2495.55s - 2497.47s | And you essentially choose any model 2497.47s - 2498.95s | and you can give it a lot of links 2498.95s - 2501.71s | for things that it should use, you know, 2501.71s - 2503.79s | to do OSINT or to do whatever you want. 2504.99s - 2508.51s | But essentially, let's create one together if you want. 2508.51s - 2510.27s | Let me figure out how do I create one. 2510.27s - 2511.71s | I can do this on my phone. 2511.71s


chunk_text:
| It will tell me, it might not be good, 2471.95s - 2473.75s | but it could be consistent. 2473.75s - 2475.39s | Can ChatGPT be consistent? 2476.51s - 2479.23s | Depends, but if you really are after consistency, 2479.23s - 2484.23s | you go to a model that you can tune the API off. 2485.39s - 2486.95s | And if you tune the API, 2486.95s - 2489.07s | it would look something like this. 2489.07s - 2490.43s | You go to the settings on Hugging Chat. 2490.43s - 2491.51s | If you don't know about Hugging Chat, 2491.51s - 2495.51s | it's just a way to get LLMs tuned for your use cases. 2495.55s - 2497.47s | And you essentially choose any model 2497.47s - 2498.95s | and you can give it a lot of links 2498.95s - 2501.71s | for things that it should use, you know, 2501.71s - 2503.79s | to do OSINT or to do whatever you want. 2504.99s - 2508.51s | But essentially, let's create one together if you want. 2508.51s - 2510.27s | Let me figure out how do I create one. 2510.27s - 2511.71s | I can do this on my phone. 2511.71s - 2514.07s | I don't have to, you know, walk away from here empty handed. 2514.07s - 2517.11s | I can walk away with a new assistant that I can create 2517.11s - 2519.51s | and I can call a TTP detector. 2519.51s - 2520.35s | And how do I do that? 2520.35s - 2523.71s | I close ChatGPT and I start any of these models. 2523.71s - 2525.75s | And the key here is temperature. 2525.75s - 2527.43s | Why is ChatGPT not consistent? 2527.43s - 2529.87s | It's creative, it hallucinates every now and then. 2529.87s - 2531.95s | That's why I don't know a single person in this room 2531.95s - 2535.55s | that uses an LLM for anything serious other than reporting 2535.55s - 2536.99s | or some sort of, you know, 2536.99s - 2539.51s | some sort of how do I write this query 2539.51s - 2540.91s | or something like that, right? 2540.91s - 2542.87s | But you don't really depend on it because it hallucinates. 2542.87s - 2544.79s | If you drive down the temperature all the way down, 2544.79s - 2546.95s | it will stop hallucinating and it will be consistent. 2546.95s


chunk_text:
- 2514.07s | I don't have to, you know, walk away from here empty handed. 2514.07s - 2517.11s | I can walk away with a new assistant that I can create 2517.11s - 2519.51s | and I can call a TTP detector. 2519.51s - 2520.35s | And how do I do that? 2520.35s - 2523.71s | I close ChatGPT and I start any of these models. 2523.71s - 2525.75s | And the key here is temperature. 2525.75s - 2527.43s | Why is ChatGPT not consistent? 2527.43s - 2529.87s | It's creative, it hallucinates every now and then. 2529.87s - 2531.95s | That's why I don't know a single person in this room 2531.95s - 2535.55s | that uses an LLM for anything serious other than reporting 2535.55s - 2536.99s | or some sort of, you know, 2536.99s - 2539.51s | some sort of how do I write this query 2539.51s - 2540.91s | or something like that, right? 2540.91s - 2542.87s | But you don't really depend on it because it hallucinates. 2542.87s - 2544.79s | If you drive down the temperature all the way down, 2544.79s - 2546.95s | it will stop hallucinating and it will be consistent. 2546.95s - 2548.47s | What is consistent or deterministic? 2548.47s - 2550.79s | It means same input, same output every single day. 2550.79s - 2552.19s | Same input, same output. 2552.23s - 2553.07s | And if I have that, 2553.07s - 2557.11s | I can technically use that for my classification. 2557.11s - 2560.27s | We're gonna talk a little bit more about that later. 2560.27s - 2562.47s | For now, let's talk about, okay, my dude, 2562.47s - 2564.31s | you need miter attacks for everything. 2564.31s - 2567.43s | Sure, give this guy all the miter attack he needs. 2567.43s - 2568.71s | Now what? 2568.71s - 2570.67s | Now the layer ones can actually understand what's going on 2570.67s - 2573.39s | when they open their sim and they see a million tickets 2573.39s - 2575.43s | for them to work on, a million notable alerts. 2575.43s - 2578.43s | However, how the hell do they work through the million? 2578.43s - 2579.87s | That's why we came here. 2579.87s - 2582.11s | How do we do data reduction? 2583.03s - 2584.87s | How do we go from a million notable alerts 2584.87s - 2587.19s | into, I


chunk_text:
- 2548.47s | What is consistent or deterministic? 2548.47s - 2550.79s | It means same input, same output every single day. 2550.79s - 2552.19s | Same input, same output. 2552.23s - 2553.07s | And if I have that, 2553.07s - 2557.11s | I can technically use that for my classification. 2557.11s - 2560.27s | We're gonna talk a little bit more about that later. 2560.27s - 2562.47s | For now, let's talk about, okay, my dude, 2562.47s - 2564.31s | you need miter attacks for everything. 2564.31s - 2567.43s | Sure, give this guy all the miter attack he needs. 2567.43s - 2568.71s | Now what? 2568.71s - 2570.67s | Now the layer ones can actually understand what's going on 2570.67s - 2573.39s | when they open their sim and they see a million tickets 2573.39s - 2575.43s | for them to work on, a million notable alerts. 2575.43s - 2578.43s | However, how the hell do they work through the million? 2578.43s - 2579.87s | That's why we came here. 2579.87s - 2582.11s | How do we do data reduction? 2583.03s - 2584.87s | How do we go from a million notable alerts 2584.87s - 2587.19s | into, I don't know, 10 tickets? 2587.19s - 2589.39s | Something that a human can actually do. 2589.39s - 2590.71s | Because if I have a million notable alerts, 2590.71s - 2593.03s | I'm only gonna tune my notable alerts 2593.03s - 2595.11s | to only have 20 extra, extra, extra spicy 2595.11s - 2597.71s | and I will never lay eyes on anything else. 2597.71s - 2600.71s | Anybody here has terabytes going into their sim? 2600.71s - 2603.19s | Like a day, terabytes? 2603.19s - 2604.43s | Fuck yeah, a lot of people. 2604.43s - 2605.59s | Okay, nice. 2605.59s - 2608.35s | So do you ever look at any of that data? 2608.35s - 2609.19s | No. 2609.19s - 2612.06s | Why do you collect it? 2612.06s - 2612.90s | Compliance. 2613.38s - 2614.34s | Thank you. 2614.34s - 2615.82s | We do not collect it because we need it. 2615.82s - 2617.98s | We collect it because the boss wants it. 2617.98s - 2622.30s | So essentially, what do we actually need? 2622.30s - 2627.10s | We need about 20 rules, 50 rules, 100 rules. 2627.10s - 2628.86s | 200 rules, that's about it. 2628.86s - 2630.70s |


chunk_text:
don't know, 10 tickets? 2587.19s - 2589.39s | Something that a human can actually do. 2589.39s - 2590.71s | Because if I have a million notable alerts, 2590.71s - 2593.03s | I'm only gonna tune my notable alerts 2593.03s - 2595.11s | to only have 20 extra, extra, extra spicy 2595.11s - 2597.71s | and I will never lay eyes on anything else. 2597.71s - 2600.71s | Anybody here has terabytes going into their sim? 2600.71s - 2603.19s | Like a day, terabytes? 2603.19s - 2604.43s | Fuck yeah, a lot of people. 2604.43s - 2605.59s | Okay, nice. 2605.59s - 2608.35s | So do you ever look at any of that data? 2608.35s - 2609.19s | No. 2609.19s - 2612.06s | Why do you collect it? 2612.06s - 2612.90s | Compliance. 2613.38s - 2614.34s | Thank you. 2614.34s - 2615.82s | We do not collect it because we need it. 2615.82s - 2617.98s | We collect it because the boss wants it. 2617.98s - 2622.30s | So essentially, what do we actually need? 2622.30s - 2627.10s | We need about 20 rules, 50 rules, 100 rules. 2627.10s - 2628.86s | 200 rules, that's about it. 2628.86s - 2630.70s | 200 rules, hopefully they don't trigger. 2630.70s - 2632.38s | When they trigger, I'm kind of worried. 2632.38s - 2633.42s | Do I ever investigate them? 2633.42s - 2636.50s | If I'm in the right mood, if I'm in the right day 2636.50s - 2639.34s | and one of them triggers, I will investigate them. 2639.34s - 2642.34s | But if you need me to do it super, super duper fast, 2642.42s - 2644.38s | I'll just close them right away. 2644.38s - 2646.70s | You want the entity D, I got you. 2646.70s - 2651.10s | So essentially, how do we find things that are relevant? 2651.10s - 2653.26s | Yes, 10 minutes to go. 2653.26s - 2654.82s | How do we find things that are relevant? 2654.82s - 2656.14s | I have to essentially investigate 2656.14s - 2657.90s | what are the questions to ask, right? 2657.90s - 2660.38s | So what is relevant between this alert and this alert? 2660.38s - 2661.86s | Well, they share an entity 2661.86s - 2663.50s | or maybe they don't share an exact entity. 2663.50s - 2664.78s | It's not a common entity. 2664.78s - 2666.66s | It's a similar entity.


chunk_text:
200 rules, hopefully they don't trigger. 2630.70s - 2632.38s | When they trigger, I'm kind of worried. 2632.38s - 2633.42s | Do I ever investigate them? 2633.42s - 2636.50s | If I'm in the right mood, if I'm in the right day 2636.50s - 2639.34s | and one of them triggers, I will investigate them. 2639.34s - 2642.34s | But if you need me to do it super, super duper fast, 2642.42s - 2644.38s | I'll just close them right away. 2644.38s - 2646.70s | You want the entity D, I got you. 2646.70s - 2651.10s | So essentially, how do we find things that are relevant? 2651.10s - 2653.26s | Yes, 10 minutes to go. 2653.26s - 2654.82s | How do we find things that are relevant? 2654.82s - 2656.14s | I have to essentially investigate 2656.14s - 2657.90s | what are the questions to ask, right? 2657.90s - 2660.38s | So what is relevant between this alert and this alert? 2660.38s - 2661.86s | Well, they share an entity 2661.86s - 2663.50s | or maybe they don't share an exact entity. 2663.50s - 2664.78s | It's not a common entity. 2664.78s - 2666.66s | It's a similar entity. 2666.66s - 2668.70s | Can I put that in the rule or a playbook? 2668.70s - 2671.42s | No, there is no similarity in the rules. 2671.42s - 2674.26s | You only have the exact same entity or not. 2674.26s - 2676.26s | These two entities are in the exact same business unit 2676.26s - 2677.10s | or not. 2677.10s - 2679.42s | They have the exact same operating system or not. 2679.42s - 2681.34s | Similar business units or operating systems 2681.34s - 2682.90s | or similar ports does not exist. 2682.90s - 2684.02s | There's no similarity. 2684.02s - 2686.18s | However, as a human, you're actually doing that. 2686.18s - 2687.58s | You're looking at similarity. 2687.58s - 2690.34s | You're looking at similar, even tickets historically, 2690.34s - 2692.86s | like there was something similar to this before 2692.86s - 2693.94s | and how would the people do it 2693.94s - 2696.14s | if I have historical data on my tickets? 2696.14s - 2698.62s | But essentially, I'm looking for event attributes. 2698.62s - 2700.26s | Like the EDR gives me a lot of attributes 2700.26s - 2702.10s | like parent process and whatnot, 2702.10s -


chunk_text:
2666.66s - 2668.70s | Can I put that in the rule or a playbook? 2668.70s - 2671.42s | No, there is no similarity in the rules. 2671.42s - 2674.26s | You only have the exact same entity or not. 2674.26s - 2676.26s | These two entities are in the exact same business unit 2676.26s - 2677.10s | or not. 2677.10s - 2679.42s | They have the exact same operating system or not. 2679.42s - 2681.34s | Similar business units or operating systems 2681.34s - 2682.90s | or similar ports does not exist. 2682.90s - 2684.02s | There's no similarity. 2684.02s - 2686.18s | However, as a human, you're actually doing that. 2686.18s - 2687.58s | You're looking at similarity. 2687.58s - 2690.34s | You're looking at similar, even tickets historically, 2690.34s - 2692.86s | like there was something similar to this before 2692.86s - 2693.94s | and how would the people do it 2693.94s - 2696.14s | if I have historical data on my tickets? 2696.14s - 2698.62s | But essentially, I'm looking for event attributes. 2698.62s - 2700.26s | Like the EDR gives me a lot of attributes 2700.26s - 2702.10s | like parent process and whatnot, 2702.10s - 2703.94s | registry keys and yada, yada, yada. 2703.94s - 2706.18s | NDR would give me like packet size and so on and so forth, 2706.18s - 2710.14s | protocol port and identity would give me a lot of events, 2710.14s - 2711.86s | you know, like what exactly happened in this? 2711.86s - 2712.98s | Was it like an account change? 2712.98s - 2714.74s | Was it, what the hell was it? 2714.74s - 2717.38s | But essentially, I'm looking at event attributes. 2717.38s - 2719.62s | I'm looking at entities attributes, right? 2719.62s - 2721.46s | And in my brain, I'm doing similarity. 2721.46s - 2722.74s | Nice. 2722.74s - 2725.66s | You were talking about attack detection models. 2725.66s - 2727.78s | You said I can do it with an LLM. 2727.78s - 2729.38s | Can you do any better, right? 2729.42s - 2730.26s | I can do a Hugging Chat. 2730.26s - 2731.50s | I can even do an Ollama. 2731.50s - 2733.98s | If you don't know what's an Ollama, 2733.98s - 2737.86s | you just go here to Ollama and it's a local LLM 2737.86s - 2739.74s | instead of hosted in Hugging Chat


chunk_text:
2703.94s | registry keys and yada, yada, yada. 2703.94s - 2706.18s | NDR would give me like packet size and so on and so forth, 2706.18s - 2710.14s | protocol port and identity would give me a lot of events, 2710.14s - 2711.86s | you know, like what exactly happened in this? 2711.86s - 2712.98s | Was it like an account change? 2712.98s - 2714.74s | Was it, what the hell was it? 2714.74s - 2717.38s | But essentially, I'm looking at event attributes. 2717.38s - 2719.62s | I'm looking at entities attributes, right? 2719.62s - 2721.46s | And in my brain, I'm doing similarity. 2721.46s - 2722.74s | Nice. 2722.74s - 2725.66s | You were talking about attack detection models. 2725.66s - 2727.78s | You said I can do it with an LLM. 2727.78s - 2729.38s | Can you do any better, right? 2729.42s - 2730.26s | I can do a Hugging Chat. 2730.26s - 2731.50s | I can even do an Ollama. 2731.50s - 2733.98s | If you don't know what's an Ollama, 2733.98s - 2737.86s | you just go here to Ollama and it's a local LLM 2737.86s - 2739.74s | instead of hosted in Hugging Chat 2739.74s - 2741.54s | or Chajipati or whatever the hell. 2741.54s - 2742.62s | You can just host it locally. 2742.62s - 2744.86s | You can do it in AWS with Bedrock, yada, yada, yada. 2744.86s - 2747.86s | But can I do better than just a vanilla LLM 2747.86s - 2749.86s | with the creativity turned all the way down, 2749.86s - 2750.98s | all the temperature all the way down? 2750.98s - 2751.86s | Yes, you can. 2751.86s - 2752.70s | We did. 2752.70s - 2756.18s | We put three years into this and we built a goddamn LLM 2756.18s - 2758.30s | that knows everything about cyber and it's open source 2758.30s - 2759.26s | and you can use it for free. 2760.14s - 2761.22s | You don't have to pay us ever. 2761.22s - 2762.94s | But essentially, how did we do that? 2762.94s - 2764.30s | We gave it a large unlabeled corpus. 2764.30s - 2765.74s | You know, all the stuff in your bookmarks, 2765.74s - 2767.90s | the books, the slides, the white papers, 2767.90s - 2768.98s | it's all the shit that you want to read 2768.98s - 2770.14s | but you


chunk_text:
2739.74s - 2741.54s | or Chajipati or whatever the hell. 2741.54s - 2742.62s | You can just host it locally. 2742.62s - 2744.86s | You can do it in AWS with Bedrock, yada, yada, yada. 2744.86s - 2747.86s | But can I do better than just a vanilla LLM 2747.86s - 2749.86s | with the creativity turned all the way down, 2749.86s - 2750.98s | all the temperature all the way down? 2750.98s - 2751.86s | Yes, you can. 2751.86s - 2752.70s | We did. 2752.70s - 2756.18s | We put three years into this and we built a goddamn LLM 2756.18s - 2758.30s | that knows everything about cyber and it's open source 2758.30s - 2759.26s | and you can use it for free. 2760.14s - 2761.22s | You don't have to pay us ever. 2761.22s - 2762.94s | But essentially, how did we do that? 2762.94s - 2764.30s | We gave it a large unlabeled corpus. 2764.30s - 2765.74s | You know, all the stuff in your bookmarks, 2765.74s - 2767.90s | the books, the slides, the white papers, 2767.90s - 2768.98s | it's all the shit that you want to read 2768.98s - 2770.14s | but you never will. 2770.14s - 2771.82s | You just put it in bookmarks forever. 2771.82s - 2773.98s | Or if you're one of these guys like me, 2773.98s - 2776.18s | you just open a shit ton of tabs 2776.18s - 2778.46s | and your browser can't take it anymore. 2779.86s - 2780.70s | Yep. 2782.18s - 2785.38s | So yeah, we just gave all of that to the LLM 2785.38s - 2787.42s | and I took them off my bookmarks 2787.42s - 2790.90s | and now I have an LLM that actually read them from scratch 2790.90s - 2794.38s | and that LLM, I gave a small labeled corpus to 2794.38s - 2797.06s | and I can talk a lot about the small labeled corpus 2797.06s - 2798.70s | but that's what took three years 2798.70s - 2800.50s | because I actually wanted a small labeled corpus 2800.50s - 2801.94s | of labels that I trust. 2801.94s - 2803.62s | Here's the alert, here's the mindset attack 2803.62s - 2805.38s | and you cannot get that. 2805.38s - 2807.10s | That is so fucking hard. 2807.10s - 2808.26s | Humans are not consistent. 2808.26s - 2810.34s | We have to


chunk_text:
never will. 2770.14s - 2771.82s | You just put it in bookmarks forever. 2771.82s - 2773.98s | Or if you're one of these guys like me, 2773.98s - 2776.18s | you just open a shit ton of tabs 2776.18s - 2778.46s | and your browser can't take it anymore. 2779.86s - 2780.70s | Yep. 2782.18s - 2785.38s | So yeah, we just gave all of that to the LLM 2785.38s - 2787.42s | and I took them off my bookmarks 2787.42s - 2790.90s | and now I have an LLM that actually read them from scratch 2790.90s - 2794.38s | and that LLM, I gave a small labeled corpus to 2794.38s - 2797.06s | and I can talk a lot about the small labeled corpus 2797.06s - 2798.70s | but that's what took three years 2798.70s - 2800.50s | because I actually wanted a small labeled corpus 2800.50s - 2801.94s | of labels that I trust. 2801.94s - 2803.62s | Here's the alert, here's the mindset attack 2803.62s - 2805.38s | and you cannot get that. 2805.38s - 2807.10s | That is so fucking hard. 2807.10s - 2808.26s | Humans are not consistent. 2808.26s - 2810.34s | We have to make one human sit down at the end 2810.34s - 2812.90s | and write regular expressions to keep themselves consistent 2812.90s - 2814.50s | when they attack something 2814.50s - 2817.18s | because every day they would attack something differently. 2817.94s - 2818.78s | It was so fucking hard. 2818.78s - 2819.62s | And then we had to like do data balance. 2819.62s - 2820.98s | Like this is so fucking hard. 2820.98s - 2823.34s | A small labeled corpus, that's impossible. 2823.34s - 2826.62s | But again, open source, already out there, use it. 2826.62s - 2828.98s | But this is how you end up with essentially 2828.98s - 2830.94s | like you have techniques for everything. 2830.94s - 2833.02s | Of course, it's actually probabilistic at the backend. 2833.02s - 2835.66s | Like I actually have a probability for every single class. 2835.66s - 2837.14s | But in this case, I was like, you know what? 2837.14s - 2839.10s | Anything above 99% show it to me. 2839.10s - 2839.94s | And that was it. 2839.94s - 2841.02s | And it was pretty decent. 2841.02s - 2843.14s | And even if it was not decent, I don't give


chunk_text:
make one human sit down at the end 2810.34s - 2812.90s | and write regular expressions to keep themselves consistent 2812.90s - 2814.50s | when they attack something 2814.50s - 2817.18s | because every day they would attack something differently. 2817.94s - 2818.78s | It was so fucking hard. 2818.78s - 2819.62s | And then we had to like do data balance. 2819.62s - 2820.98s | Like this is so fucking hard. 2820.98s - 2823.34s | A small labeled corpus, that's impossible. 2823.34s - 2826.62s | But again, open source, already out there, use it. 2826.62s - 2828.98s | But this is how you end up with essentially 2828.98s - 2830.94s | like you have techniques for everything. 2830.94s - 2833.02s | Of course, it's actually probabilistic at the backend. 2833.02s - 2835.66s | Like I actually have a probability for every single class. 2835.66s - 2837.14s | But in this case, I was like, you know what? 2837.14s - 2839.10s | Anything above 99% show it to me. 2839.10s - 2839.94s | And that was it. 2839.94s - 2841.02s | And it was pretty decent. 2841.02s - 2843.14s | And even if it was not decent, I don't give a shit. 2843.14s - 2844.86s | It's consistent and it's in everything. 2844.86s - 2846.74s | You have a PowerShell in my organization. 2847.30s - 2848.14s | I have a technique. 2848.14s - 2849.66s | You have a Windows event log in my organization. 2849.66s - 2850.62s | I have a technique. 2850.62s - 2852.66s | You have whatever the fuck that I get in my sim, 2852.66s - 2853.50s | I have a technique. 2853.50s - 2854.78s | Even if I don't get it in my sim, 2854.78s - 2856.70s | as long as it ends on my S3 bucket 2856.70s - 2858.98s | and it goes through this, I have a goddamn technique. 2858.98s - 2862.50s | And I can hire the 20 resumes that I got of juniors 2862.50s - 2863.86s | because they actually know what the hell is going on 2863.86s - 2864.70s | when they see something. 2864.70s - 2867.22s | You don't have to ask me, what is run DLL 32? 2867.22s - 2868.50s | I don't fucking give a shit. 2868.50s - 2871.50s | You have to just say, what is T1218? 2871.50s - 2872.34s | And you have to read about


chunk_text:
a shit. 2843.14s - 2844.86s | It's consistent and it's in everything. 2844.86s - 2846.74s | You have a PowerShell in my organization. 2847.30s - 2848.14s | I have a technique. 2848.14s - 2849.66s | You have a Windows event log in my organization. 2849.66s - 2850.62s | I have a technique. 2850.62s - 2852.66s | You have whatever the fuck that I get in my sim, 2852.66s - 2853.50s | I have a technique. 2853.50s - 2854.78s | Even if I don't get it in my sim, 2854.78s - 2856.70s | as long as it ends on my S3 bucket 2856.70s - 2858.98s | and it goes through this, I have a goddamn technique. 2858.98s - 2862.50s | And I can hire the 20 resumes that I got of juniors 2862.50s - 2863.86s | because they actually know what the hell is going on 2863.86s - 2864.70s | when they see something. 2864.70s - 2867.22s | You don't have to ask me, what is run DLL 32? 2867.22s - 2868.50s | I don't fucking give a shit. 2868.50s - 2871.50s | You have to just say, what is T1218? 2871.50s - 2872.34s | And you have to read about it. 2872.34s - 2873.74s | And you have to figure out how to mitigate 2873.74s - 2874.58s | and how to investigate about it. 2874.58s - 2876.58s | You can go to chat GPT and be like, 2877.42s - 2879.98s | T1218, what should I do? 2879.98s - 2880.82s | What are the next steps? 2880.82s - 2881.66s | What should I investigate? 2881.66s - 2882.82s | And it will tell you, right? 2882.82s - 2884.18s | It's a common language. 2884.18s - 2885.50s | So anyways, now that I have that, 2885.50s - 2887.34s | I wanna cluster events that are relevant. 2887.34s - 2889.14s | How do I cluster events that are relevant? 2889.14s - 2890.74s | Well, I can do it vanilla. 2890.74s - 2893.98s | And I have code here for essentially clustering. 2893.98s - 2897.22s | And I just zoom in on the sexy eye candy, 2897.22s - 2899.78s | but forget the eye candy for a second. 2899.78s - 2900.98s | This is just like eye candy that shows you 2900.98s - 2902.46s | like clusters of events and whatnot. 2902.46s - 2904.98s | But if I wanna do much better, 2905.02s - 2908.66s


chunk_text:
it. 2872.34s - 2873.74s | And you have to figure out how to mitigate 2873.74s - 2874.58s | and how to investigate about it. 2874.58s - 2876.58s | You can go to chat GPT and be like, 2877.42s - 2879.98s | T1218, what should I do? 2879.98s - 2880.82s | What are the next steps? 2880.82s - 2881.66s | What should I investigate? 2881.66s - 2882.82s | And it will tell you, right? 2882.82s - 2884.18s | It's a common language. 2884.18s - 2885.50s | So anyways, now that I have that, 2885.50s - 2887.34s | I wanna cluster events that are relevant. 2887.34s - 2889.14s | How do I cluster events that are relevant? 2889.14s - 2890.74s | Well, I can do it vanilla. 2890.74s - 2893.98s | And I have code here for essentially clustering. 2893.98s - 2897.22s | And I just zoom in on the sexy eye candy, 2897.22s - 2899.78s | but forget the eye candy for a second. 2899.78s - 2900.98s | This is just like eye candy that shows you 2900.98s - 2902.46s | like clusters of events and whatnot. 2902.46s - 2904.98s | But if I wanna do much better, 2905.02s - 2908.66s | and if I can actually hire a summer intern for data science, 2908.66s - 2911.06s | instead of just DIY it myself, 2911.06s - 2913.18s | I will tell them to use a knowledge graph. 2913.18s - 2914.38s | What the hell is a knowledge graph? 2914.38s - 2916.90s | Clustering is unsupervised, right? 2916.90s - 2917.98s | Clustering is unsupervised, 2917.98s - 2919.22s | which means that it actually doesn't know 2919.22s - 2920.66s | where the hell is it going. 2920.66s - 2921.74s | And it doesn't know what's right 2921.74s - 2923.50s | because you don't know what's right. 2923.50s - 2924.70s | You actually don't know what's right. 2924.70s - 2925.86s | And you don't know how to guide it, 2925.86s - 2927.66s | but you know how to think. 2927.66s - 2930.46s | You know how to investigate to a certain degree. 2930.46s - 2931.86s | Anybody here has documentation? 2931.86s - 2933.50s | If I join as a new SOC analyst, 2933.50s - 2935.14s | how to investigate something? 2936.14s - 2939.30s | One, okay, two, three, four, four. 2939.30s - 2941.22s | Okay, you guys are fantastic. 2941.22s - 2942.50s | Most


chunk_text:
| and if I can actually hire a summer intern for data science, 2908.66s - 2911.06s | instead of just DIY it myself, 2911.06s - 2913.18s | I will tell them to use a knowledge graph. 2913.18s - 2914.38s | What the hell is a knowledge graph? 2914.38s - 2916.90s | Clustering is unsupervised, right? 2916.90s - 2917.98s | Clustering is unsupervised, 2917.98s - 2919.22s | which means that it actually doesn't know 2919.22s - 2920.66s | where the hell is it going. 2920.66s - 2921.74s | And it doesn't know what's right 2921.74s - 2923.50s | because you don't know what's right. 2923.50s - 2924.70s | You actually don't know what's right. 2924.70s - 2925.86s | And you don't know how to guide it, 2925.86s - 2927.66s | but you know how to think. 2927.66s - 2930.46s | You know how to investigate to a certain degree. 2930.46s - 2931.86s | Anybody here has documentation? 2931.86s - 2933.50s | If I join as a new SOC analyst, 2933.50s - 2935.14s | how to investigate something? 2936.14s - 2939.30s | One, okay, two, three, four, four. 2939.30s - 2941.22s | Okay, you guys are fantastic. 2941.22s - 2942.50s | Most people don't, right? 2942.50s - 2945.14s | Do you have historical tickets by any chance? 2945.14s - 2947.14s | Historical tickets, historical investigations? 2947.14s - 2948.62s | Okay, about five, six people. 2948.62s - 2950.02s | Okay, nice. 2950.02s - 2952.06s | Most of us, we're the normies. 2952.06s - 2953.18s | We're the 99%. 2953.18s - 2954.50s | We don't have shit, right? 2954.50s - 2957.86s | So it's all stuck in the brain of a unicorn 2957.86s - 2958.78s | sitting in my SOC. 2958.78s - 2961.54s | And without this unicorn, everything fails, right? 2961.54s - 2962.38s | So I know how to think. 2962.50s - 2964.66s | That's a failure of your unicorn. 2964.66s - 2965.50s | Right. 2965.50s - 2967.50s | Because the first thing we should be doing 2967.50s - 2970.30s | when we pull this stuff is writing out, 2970.30s - 2974.62s | oh, here's a little FAQ of how I do what I do. 2974.62s - 2976.58s | And it might not work for you, 2976.58s - 2978.86s | but you can see how my brain's gonna work on it. 2978.86s - 2981.38s | And then you can adapt your brain 2981.38s


chunk_text:
people don't, right? 2942.50s - 2945.14s | Do you have historical tickets by any chance? 2945.14s - 2947.14s | Historical tickets, historical investigations? 2947.14s - 2948.62s | Okay, about five, six people. 2948.62s - 2950.02s | Okay, nice. 2950.02s - 2952.06s | Most of us, we're the normies. 2952.06s - 2953.18s | We're the 99%. 2953.18s - 2954.50s | We don't have shit, right? 2954.50s - 2957.86s | So it's all stuck in the brain of a unicorn 2957.86s - 2958.78s | sitting in my SOC. 2958.78s - 2961.54s | And without this unicorn, everything fails, right? 2961.54s - 2962.38s | So I know how to think. 2962.50s - 2964.66s | That's a failure of your unicorn. 2964.66s - 2965.50s | Right. 2965.50s - 2967.50s | Because the first thing we should be doing 2967.50s - 2970.30s | when we pull this stuff is writing out, 2970.30s - 2974.62s | oh, here's a little FAQ of how I do what I do. 2974.62s - 2976.58s | And it might not work for you, 2976.58s - 2978.86s | but you can see how my brain's gonna work on it. 2978.86s - 2981.38s | And then you can adapt your brain 2981.38s - 2983.78s | to the same goal at least. 2983.78s - 2987.14s | And oh, by the way, here's all the things I screwed up on 2987.14s - 2989.74s | so you don't make my mistakes. 2989.74s - 2990.58s | Yes. 2991.02s - 2992.86s | There's also SOPs too, right? 2992.86s - 2995.94s | I mean, if you document what people are supposed to do, 2995.94s - 2996.78s | then who reads them? 2996.78s - 2999.82s | Yeah, the SOPs, the FAQs, boilerplate. 2999.82s - 3003.46s | Oh, dear Lord, as much as you can boilerplate. 3003.46s - 3005.50s | Boilerplate your friend. 3006.94s - 3011.58s | Because if they want to spend hours crafting an IR 3011.58s - 3014.78s | and they've got 75% pre-written, 3015.66s - 3017.54s | put this here, put this here, 3017.54s - 3019.30s | make sure it all makes sense. 3019.30s - 3021.34s | Here's where you put the actual details 3021.34s - 3023.10s | of what happened in the narrative. 3023.10s - 3024.30s | And here's all the rest of the stuff 3024.30s - 3025.62s | that we just have to put in. 3025.62s - 3026.46s | Hell yeah. 3026.46s - 3028.42s |


chunk_text:
- 2983.78s | to the same goal at least. 2983.78s - 2987.14s | And oh, by the way, here's all the things I screwed up on 2987.14s - 2989.74s | so you don't make my mistakes. 2989.74s - 2990.58s | Yes. 2991.02s - 2992.86s | There's also SOPs too, right? 2992.86s - 2995.94s | I mean, if you document what people are supposed to do, 2995.94s - 2996.78s | then who reads them? 2996.78s - 2999.82s | Yeah, the SOPs, the FAQs, boilerplate. 2999.82s - 3003.46s | Oh, dear Lord, as much as you can boilerplate. 3003.46s - 3005.50s | Boilerplate your friend. 3006.94s - 3011.58s | Because if they want to spend hours crafting an IR 3011.58s - 3014.78s | and they've got 75% pre-written, 3015.66s - 3017.54s | put this here, put this here, 3017.54s - 3019.30s | make sure it all makes sense. 3019.30s - 3021.34s | Here's where you put the actual details 3021.34s - 3023.10s | of what happened in the narrative. 3023.10s - 3024.30s | And here's all the rest of the stuff 3024.30s - 3025.62s | that we just have to put in. 3025.62s - 3026.46s | Hell yeah. 3026.46s - 3028.42s | I mean, if you actually have some sort of documentation, 3028.42s - 3029.26s | that's fantastic. 3029.26s - 3031.18s | If you don't, you have to figure out 3031.18s - 3032.94s | how to put the knowledge out there. 3034.54s - 3037.34s | You can, you can, don't lose hope. 3037.34s - 3038.98s | The knowledge is essentially is gonna be encoded 3038.98s - 3040.30s | in some sort of data structure. 3040.30s - 3042.22s | And data structures are essentially pretty nice 3042.22s - 3044.58s | because they tell the clustering algorithm 3044.58s - 3045.90s | what to look for. 3045.90s - 3047.42s | Clustering, again, doesn't know what to do. 3047.42s - 3048.26s | It's unsupervised, 3048.26s - 3049.74s | but you can make it semi-supervised 3049.74s - 3051.66s | and cheat your way a little 3051.66s - 3052.94s | and just give it knowledge. 3052.94s - 3053.94s | Tell it, you know. 3053.94s - 3055.94s | In my investigations, I always look for entities. 3055.94s - 3056.78s | Okay. 3056.78s - 3060.18s | So the entity is a node and the event is a edge. 3060.18s - 3063.78s | And between as, as an entity and Google


chunk_text:
I mean, if you actually have some sort of documentation, 3028.42s - 3029.26s | that's fantastic. 3029.26s - 3031.18s | If you don't, you have to figure out 3031.18s - 3032.94s | how to put the knowledge out there. 3034.54s - 3037.34s | You can, you can, don't lose hope. 3037.34s - 3038.98s | The knowledge is essentially is gonna be encoded 3038.98s - 3040.30s | in some sort of data structure. 3040.30s - 3042.22s | And data structures are essentially pretty nice 3042.22s - 3044.58s | because they tell the clustering algorithm 3044.58s - 3045.90s | what to look for. 3045.90s - 3047.42s | Clustering, again, doesn't know what to do. 3047.42s - 3048.26s | It's unsupervised, 3048.26s - 3049.74s | but you can make it semi-supervised 3049.74s - 3051.66s | and cheat your way a little 3051.66s - 3052.94s | and just give it knowledge. 3052.94s - 3053.94s | Tell it, you know. 3053.94s - 3055.94s | In my investigations, I always look for entities. 3055.94s - 3056.78s | Okay. 3056.78s - 3060.18s | So the entity is a node and the event is a edge. 3060.18s - 3063.78s | And between as, as an entity and Google 3063.78s - 3066.34s | or Ashley Madison or whatever you want, 3066.34s - 3068.90s | like it's essentially another entity. 3068.90s - 3070.34s | And these are the events that happened. 3070.34s - 3072.42s | And these events, essentially, 3072.42s - 3074.22s | I want you to cluster them, 3074.22s - 3075.62s | thinking all the time in the cluster 3075.62s - 3077.22s | about what entities are involved. 3077.22s - 3079.82s | It's an entity-centric way. 3079.82s - 3081.42s | But again, do you have the right process 3081.42s - 3082.90s | to have the entity attributes? 3082.90s - 3084.38s | Do you have the right stuff? 3084.38s - 3085.50s | It's all a process. 3085.50s - 3086.86s | Because when you go to data science, 3086.86s - 3088.50s | it's about what data do you have, right? 3088.50s - 3090.74s | And also what knowledge do you have to put there? 3090.74s - 3092.78s | But given the right data, not the right data, 3092.78s - 3095.62s | given a good enough data and a good enough knowledge, 3095.62s - 3097.10s | you can actually get very useful clusters. 3097.10s - 3097.94s | And guess what? 3097.94s -


chunk_text:
3063.78s - 3066.34s | or Ashley Madison or whatever you want, 3066.34s - 3068.90s | like it's essentially another entity. 3068.90s - 3070.34s | And these are the events that happened. 3070.34s - 3072.42s | And these events, essentially, 3072.42s - 3074.22s | I want you to cluster them, 3074.22s - 3075.62s | thinking all the time in the cluster 3075.62s - 3077.22s | about what entities are involved. 3077.22s - 3079.82s | It's an entity-centric way. 3079.82s - 3081.42s | But again, do you have the right process 3081.42s - 3082.90s | to have the entity attributes? 3082.90s - 3084.38s | Do you have the right stuff? 3084.38s - 3085.50s | It's all a process. 3085.50s - 3086.86s | Because when you go to data science, 3086.86s - 3088.50s | it's about what data do you have, right? 3088.50s - 3090.74s | And also what knowledge do you have to put there? 3090.74s - 3092.78s | But given the right data, not the right data, 3092.78s - 3095.62s | given a good enough data and a good enough knowledge, 3095.62s - 3097.10s | you can actually get very useful clusters. 3097.10s - 3097.94s | And guess what? 3097.94s - 3099.38s | They compete with nothing. 3099.38s - 3101.78s | Today, chapter one, machine learning and clustering 3101.78s - 3103.58s | competes with absolutely nothing 3103.58s - 3105.58s | because you have a bunch of humans that sit in your sock 3105.58s - 3108.70s | and they do whatever, 3108.70s - 3112.22s | but you don't actually know how do they correlate things 3112.22s - 3113.74s | because you don't document them. 3113.74s - 3115.86s | Clustering, at least, is fully transparent. 3115.86s - 3117.06s | It's auditable. 3117.06s - 3118.90s | It's actually explainable. 3118.90s - 3121.22s | And you can track it down and you can give it feedback. 3121.22s - 3123.06s | And it's very easy to give feedback. 3123.06s - 3124.94s | But also, you can also chain things 3124.94s - 3126.06s | in a finite-state machine. 3126.06s - 3128.50s | And in data science, we call those Markovian chains, 3128.50s - 3130.98s | but Markovian models are another way 3130.98s - 3132.50s | to say finite-state machine. 3132.50s - 3134.06s | We just like to complicate things. 3134.10s - 3136.74s | But essentially, they say that if you find a few steps, 3136.74s - 3138.98s


chunk_text:
3099.38s | They compete with nothing. 3099.38s - 3101.78s | Today, chapter one, machine learning and clustering 3101.78s - 3103.58s | competes with absolutely nothing 3103.58s - 3105.58s | because you have a bunch of humans that sit in your sock 3105.58s - 3108.70s | and they do whatever, 3108.70s - 3112.22s | but you don't actually know how do they correlate things 3112.22s - 3113.74s | because you don't document them. 3113.74s - 3115.86s | Clustering, at least, is fully transparent. 3115.86s - 3117.06s | It's auditable. 3117.06s - 3118.90s | It's actually explainable. 3118.90s - 3121.22s | And you can track it down and you can give it feedback. 3121.22s - 3123.06s | And it's very easy to give feedback. 3123.06s - 3124.94s | But also, you can also chain things 3124.94s - 3126.06s | in a finite-state machine. 3126.06s - 3128.50s | And in data science, we call those Markovian chains, 3128.50s - 3130.98s | but Markovian models are another way 3130.98s - 3132.50s | to say finite-state machine. 3132.50s - 3134.06s | We just like to complicate things. 3134.10s - 3136.74s | But essentially, they say that if you find a few steps, 3136.74s - 3138.98s | even if it misses a few because of zero days or whatnot, 3138.98s - 3140.30s | it's fine, put them together. 3140.30s - 3143.26s | And what you end up with is 2,500 alerts 3143.26s - 3146.26s | given with 180,000 events and logs. 3146.26s - 3147.50s | And they give you eight clusters. 3147.50s - 3148.46s | In reality, in the sock, 3148.46s - 3151.74s | we only knew about one ticket or one cluster. 3151.74s - 3155.58s | And that one cluster had about 10 alerts. 3155.58s - 3157.66s | And 10 alerts are essentially a lot. 3157.66s - 3159.54s | I don't think anybody has a ticket with 10 alerts, 3159.54s - 3160.54s | but that was not enough 3160.54s - 3163.94s | because that exact cluster had 202 alerts. 3164.94s - 3166.74s | And of course, not all of these were good. 3166.74s - 3169.46s | Some of these were garbage, but we gave it feedback. 3169.46s - 3170.42s | And it was fantastic. 3170.42s - 3172.34s | And when I showed it to my sock people, 3172.34s - 3174.26s | it was like, okay, this is pretty good. 3174.26s - 3176.74s | What amazing


chunk_text:
| even if it misses a few because of zero days or whatnot, 3138.98s - 3140.30s | it's fine, put them together. 3140.30s - 3143.26s | And what you end up with is 2,500 alerts 3143.26s - 3146.26s | given with 180,000 events and logs. 3146.26s - 3147.50s | And they give you eight clusters. 3147.50s - 3148.46s | In reality, in the sock, 3148.46s - 3151.74s | we only knew about one ticket or one cluster. 3151.74s - 3155.58s | And that one cluster had about 10 alerts. 3155.58s - 3157.66s | And 10 alerts are essentially a lot. 3157.66s - 3159.54s | I don't think anybody has a ticket with 10 alerts, 3159.54s - 3160.54s | but that was not enough 3160.54s - 3163.94s | because that exact cluster had 202 alerts. 3164.94s - 3166.74s | And of course, not all of these were good. 3166.74s - 3169.46s | Some of these were garbage, but we gave it feedback. 3169.46s - 3170.42s | And it was fantastic. 3170.42s - 3172.34s | And when I showed it to my sock people, 3172.34s - 3174.26s | it was like, okay, this is pretty good. 3174.26s - 3176.74s | What amazing technology, what alien technology is this? 3176.74s - 3179.18s | And I was like, this is chapter one machine learning. 3179.18s - 3180.74s | This is clustering. 3180.74s - 3181.78s | And you can give it feedback. 3181.78s - 3183.38s | It's chapter one, fuck yeah. 3183.38s - 3186.26s | Like we're working with rules, we're working with stuff, 3186.26s - 3187.42s | and it's a big data problem. 3187.42s - 3189.02s | And we don't have a single big data solution 3189.02s - 3191.66s | because we're skeptical about anything 3191.66s - 3192.62s | that we don't understand. 3192.62s - 3193.78s | Clustering, you can understand 3194.06s - 3195.82s | down to the latest bit of it, 3195.82s - 3198.02s | and you can give it feedback and you should use it. 3198.02s - 3198.86s | And it's open source. 3198.86s - 3202.22s | So essentially, if you want to take something from this, 3202.22s - 3204.10s | put techniques in everything, that's the what. 3204.10s - 3207.50s | And for investigations, welcome to cybersecurity. 3207.50s - 3209.78s | No one actually knows how do we do investigations correctly. 3209.78s - 3211.30s | We don't have proper documentation. 3211.30s


chunk_text:
technology, what alien technology is this? 3176.74s - 3179.18s | And I was like, this is chapter one machine learning. 3179.18s - 3180.74s | This is clustering. 3180.74s - 3181.78s | And you can give it feedback. 3181.78s - 3183.38s | It's chapter one, fuck yeah. 3183.38s - 3186.26s | Like we're working with rules, we're working with stuff, 3186.26s - 3187.42s | and it's a big data problem. 3187.42s - 3189.02s | And we don't have a single big data solution 3189.02s - 3191.66s | because we're skeptical about anything 3191.66s - 3192.62s | that we don't understand. 3192.62s - 3193.78s | Clustering, you can understand 3194.06s - 3195.82s | down to the latest bit of it, 3195.82s - 3198.02s | and you can give it feedback and you should use it. 3198.02s - 3198.86s | And it's open source. 3198.86s - 3202.22s | So essentially, if you want to take something from this, 3202.22s - 3204.10s | put techniques in everything, that's the what. 3204.10s - 3207.50s | And for investigations, welcome to cybersecurity. 3207.50s - 3209.78s | No one actually knows how do we do investigations correctly. 3209.78s - 3211.30s | We don't have proper documentation. 3211.30s - 3213.38s | We don't have a single even resource. 3213.38s - 3215.02s | We don't have a course about how to investigate stuff 3215.02s - 3218.30s | in the SOC that is unified and people know about, yeah. 3218.30s - 3219.94s | But essentially, if you have a clustering algorithm, 3219.94s - 3222.54s | you have a way to unify how do you deal with this 3222.54s - 3225.74s | and it scales and it's not a GPU, it's not an LLM. 3225.74s - 3228.66s | It's $20 a month for a clustering algorithm 3228.66s - 3231.18s | to run through terabytes and terabytes of data every day 3231.18s - 3233.82s | and fully explainable and you can give it feedback. 3234.78s - 3237.38s | But yeah, here's essentially a quick example 3237.38s - 3239.22s | that is technical, so I didn't completely bait you. 3239.22s - 3241.62s | Here's an input and here's how you cluster it. 3241.62s - 3243.14s | You just say cluster zero has all of these 3243.14s - 3244.94s | and you can give that to an LLM and write the report. 3244.94s - 3246.58s | LLMs are really good at writing reports. 3246.58s


chunk_text:
- 3213.38s | We don't have a single even resource. 3213.38s - 3215.02s | We don't have a course about how to investigate stuff 3215.02s - 3218.30s | in the SOC that is unified and people know about, yeah. 3218.30s - 3219.94s | But essentially, if you have a clustering algorithm, 3219.94s - 3222.54s | you have a way to unify how do you deal with this 3222.54s - 3225.74s | and it scales and it's not a GPU, it's not an LLM. 3225.74s - 3228.66s | It's $20 a month for a clustering algorithm 3228.66s - 3231.18s | to run through terabytes and terabytes of data every day 3231.18s - 3233.82s | and fully explainable and you can give it feedback. 3234.78s - 3237.38s | But yeah, here's essentially a quick example 3237.38s - 3239.22s | that is technical, so I didn't completely bait you. 3239.22s - 3241.62s | Here's an input and here's how you cluster it. 3241.62s - 3243.14s | You just say cluster zero has all of these 3243.14s - 3244.94s | and you can give that to an LLM and write the report. 3244.94s - 3246.58s | LLMs are really good at writing reports. 3246.58s - 3248.06s | If I give this to an LLM, 3248.06s - 3250.82s | it will make me an amazing report I can give to my boss. 3250.82s - 3252.02s | But now when I look at a ticket, 3252.50s - 3254.30s | it doesn't have to be one alert, a notable alert. 3254.30s - 3256.42s | My ticket is one notable alert, 3256.42s - 3258.82s | 200 possibly relevant alerts 3258.82s - 3261.46s | and if I have historical tickets, 3261.46s - 3264.14s | here is three relevant tickets from history 3264.14s - 3266.86s | so that I know what happened historically. 3266.86s - 3271.02s | But yeah, that's pretty much a wrap. 3271.02s - 3274.34s | This is how we compare against essentially other methods. 3274.34s - 3276.82s | It's actually efficient because you don't do rules, 3276.82s - 3279.38s | you don't do searches and it's easier to adapt and maintain 3279.38s - 3280.30s | and it's actually standalone. 3280.30s - 3281.90s | So when your boss changes the SIM, 3282.78s - 3283.62s | you don't lose your work, it's standalone 3283.62s - 3285.62s | and it's a legacy you can leave behind you. 3285.62s - 3286.58s


chunk_text:
- 3248.06s | If I give this to an LLM, 3248.06s - 3250.82s | it will make me an amazing report I can give to my boss. 3250.82s - 3252.02s | But now when I look at a ticket, 3252.50s - 3254.30s | it doesn't have to be one alert, a notable alert. 3254.30s - 3256.42s | My ticket is one notable alert, 3256.42s - 3258.82s | 200 possibly relevant alerts 3258.82s - 3261.46s | and if I have historical tickets, 3261.46s - 3264.14s | here is three relevant tickets from history 3264.14s - 3266.86s | so that I know what happened historically. 3266.86s - 3271.02s | But yeah, that's pretty much a wrap. 3271.02s - 3274.34s | This is how we compare against essentially other methods. 3274.34s - 3276.82s | It's actually efficient because you don't do rules, 3276.82s - 3279.38s | you don't do searches and it's easier to adapt and maintain 3279.38s - 3280.30s | and it's actually standalone. 3280.30s - 3281.90s | So when your boss changes the SIM, 3282.78s - 3283.62s | you don't lose your work, it's standalone 3283.62s - 3285.62s | and it's a legacy you can leave behind you. 3285.62s - 3286.58s | Anyways, I appreciate you. 3286.58s - 3287.98s | Thank you for the panelists. 3287.98s - 3295.57s | Thank you guys for listening. 3295.57s - 3298.09s | I have a shit ton of stickers and I don't know, 3298.09s - 3300.41s | I can take a lot of questions outside as well. 3300.41s - 3301.25s | I have a shit ton of stickers 3301.25s - 3302.53s | and I'd love to connect with you. 3302.53s - 3305.33s | You guys probably should connect with these guys. 3305.33s - 3307.17s | There are innovative leaders. 3307.17s - 3309.17s | These are the kind of leaders that we have on camera 3309.17s - 3314.17s | saying they like juniors and they don't like MTTD. 3314.41s - 3317.21s | They don't like the old fashioned way of doing things. 3318.21s - 3320.09s | These are the kind of leaders you probably want to work with 3320.09s - 3321.17s | but I appreciate you.


chunk_text:
| Anyways, I appreciate you. 3286.58s - 3287.98s | Thank you for the panelists. 3287.98s - 3295.57s | Thank you guys for listening. 3295.57s - 3298.09s | I have a shit ton of stickers and I don't know, 3298.09s - 3300.41s | I can take a lot of questions outside as well. 3300.41s - 3301.25s | I have a shit ton of stickers 3301.25s - 3302.53s | and I'd love to connect with you. 3302.53s - 3305.33s | You guys probably should connect with these guys. 3305.33s - 3307.17s | There are innovative leaders. 3307.17s - 3309.17s | These are the kind of leaders that we have on camera 3309.17s - 3314.17s | saying they like juniors and they don't like MTTD. 3314.41s - 3317.21s | They don't like the old fashioned way of doing things. 3318.21s - 3320.09s | These are the kind of leaders you probably want to work with 3320.09s - 3321.17s | but I appreciate you.


chunk_text:
{ "webpage_url": "https://www.youtube.com/watch?v=jS3iNUfYtLA", "title": "DEF CON 32 - Data On Demand The challenges of building a privacy focused AI Device - Matt Domko", "description": "Building an AI product for the everyday person is challenging - doing it in a privacy focused way is nearly impossible without support from the right people. I'll walk through the techniques we're using at Rabbit to secure customer data and provide people a choice as to where their data goes.\n\nWe'll cover the pipelines that\n\nCollect and Manage customer identity after they login to a site\nLog, Anonymize, and Process customer voice interactions\nProvide \"just in time\" access to customer data for personalized RAG-like models\nAs a community, I think we're well within our rights to demand control over the data we provide to companies. This talk aims to provide engineers with a list of ideas on \"what right could look like\", and general attendees a list of things that are possible, so they know its ok to ask for them.", "channel_url": "https://www.youtube.com/channel/UC6Om9kAkl32dWlDSNlDS9Iw", "duration": 2447, "channel": "DEFCONConference", "uploader": "DEFCONConference", "upload_date": "20241016" } 0.00s - 6.08s | This text was transcribed using whisper model: large-v2 so hi everybody my name is Matt Domko I don't know who that was but they're my 6.08s - 10.92s | new best friend so you are at data on demand the challenges of building a 10.92s - 19.52s | privacy focused AI device and in reality like what I want to talk about is I am a 19.52s - 28.48s | consumer and also a supporter of the AI magical things and if if I have the one 28.48s - 32.56s | that I work on at work or if I have one from somebody else they're just certain 32.56s - 36.68s | things that I really want to make sure that they happen and that's kind of the 36.68s - 43.92s | the goal of this thing so I do have a job they pay me to do stuff these are my 43.92s - 51.04s | thoughts things in my brain that I wrote down in order to make a device safer but 51.04s - 58.08s | this is not the thoughts and ideas of my company my company a company I work at 58.08s - 64.20s | so I'm into 3d printing laser-cutting cosplay mostly steampunk stuff if you 64.20s - 67.80s | ever want to nerd out about any of that or


chunk_text:
my 6.08s - 10.92s | new best friend so you are at data on demand the challenges of building a 10.92s - 19.52s | privacy focused AI device and in reality like what I want to talk about is I am a 19.52s - 28.48s | consumer and also a supporter of the AI magical things and if if I have the one 28.48s - 32.56s | that I work on at work or if I have one from somebody else they're just certain 32.56s - 36.68s | things that I really want to make sure that they happen and that's kind of the 36.68s - 43.92s | the goal of this thing so I do have a job they pay me to do stuff these are my 43.92s - 51.04s | thoughts things in my brain that I wrote down in order to make a device safer but 51.04s - 58.08s | this is not the thoughts and ideas of my company my company a company I work at 58.08s - 64.20s | so I'm into 3d printing laser-cutting cosplay mostly steampunk stuff if you 64.20s - 67.80s | ever want to nerd out about any of that or just security education in general 67.80s - 73.40s | super super into it I'm also the director of fun at tines if anybody's 73.40s - 78.08s | familiar with tines it's an unofficial title and they don't pay me but I the 78.08s - 86.12s | director of fun here's the agenda will work through so the problem there's all 86.12s - 89.44s | these things and it's funny when I was creating these slides it was like okay 89.44s - 92.96s | what are the AI devices that exist and there were like three and then all of a 92.96s - 97.24s | sudden we had two new ones appear that are the exact same but different with 97.24s - 100.60s | the same name I don't know anyway the point is is I want a hardware device 100.60s - 106.40s | that makes AI easy for just the general consumer but I want to respect their 106.40s - 112.68s | privacy because it's the right thing to do and so it's it's kind of a challenge 112.88s - 116.32s | when you think about the difference between working in the consumer space 116.32s - 121.72s | where it's


chunk_text:
just security education in general 67.80s - 73.40s | super super into it I'm also the director of fun at tines if anybody's 73.40s - 78.08s | familiar with tines it's an unofficial title and they don't pay me but I the 78.08s - 86.12s | director of fun here's the agenda will work through so the problem there's all 86.12s - 89.44s | these things and it's funny when I was creating these slides it was like okay 89.44s - 92.96s | what are the AI devices that exist and there were like three and then all of a 92.96s - 97.24s | sudden we had two new ones appear that are the exact same but different with 97.24s - 100.60s | the same name I don't know anyway the point is is I want a hardware device 100.60s - 106.40s | that makes AI easy for just the general consumer but I want to respect their 106.40s - 112.68s | privacy because it's the right thing to do and so it's it's kind of a challenge 112.88s - 116.32s | when you think about the difference between working in the consumer space 116.32s - 121.72s | where it's like anyone in my family who's not into technology might actually 121.72s - 127.52s | purchase one of these versus the team that I work with at an organization and 127.52s - 131.60s | they already have an understanding of how they treat their data and where it's 131.60s - 137.76s | going so before we get into the pipelines that we we should be using to 137.76s - 143.72s | protect those that data I want to start off talking about some basic request 143.72s - 149.24s | workflows so when I think about doing something with any kind of AI it's 149.24s - 153.28s | really four different types of requests and there are two it's basically two 153.28s - 157.40s | with a little differences so the first one is just a general knowledge request 157.40s - 163.40s | I don't need to know anything about you I can just make this request the second 163.40s - 167.92s | one is a contextual knowledge request so I need to know some things about you 167.92s - 172.24s | but I don't need any super sensitive data generic actions requests so if I 172.24s - 176.16s | want


chunk_text:
like anyone in my family who's not into technology might actually 121.72s - 127.52s | purchase one of these versus the team that I work with at an organization and 127.52s - 131.60s | they already have an understanding of how they treat their data and where it's 131.60s - 137.76s | going so before we get into the pipelines that we we should be using to 137.76s - 143.72s | protect those that data I want to start off talking about some basic request 143.72s - 149.24s | workflows so when I think about doing something with any kind of AI it's 149.24s - 153.28s | really four different types of requests and there are two it's basically two 153.28s - 157.40s | with a little differences so the first one is just a general knowledge request 157.40s - 163.40s | I don't need to know anything about you I can just make this request the second 163.40s - 167.92s | one is a contextual knowledge request so I need to know some things about you 167.92s - 172.24s | but I don't need any super sensitive data generic actions requests so if I 172.24s - 176.16s | want to do something for you but it could be anybody it could be done for 176.16s - 179.84s | Matt it could be done for Steve it could be done for Jay I think that's who that 179.84s - 185.28s | was now that's what we would do but if I wanted to do something on behalf of you 185.28s - 189.48s | on your Spotify account on your Grubhub account whatever it is I need to be 189.48s - 196.60s | authenticated and so that's that fourth type that I like to think about so if we 196.60s - 202.88s | look at the general knowledge request it's it's fairly simple so I have my my 202.88s - 208.68s | AI walkie-talkie I push the button and I say whatever it is I'm gonna say what 208.68s - 214.92s | what's the weather out today it's gonna go up through the internet into my cloud 214.92s - 218.80s | provider there's a routing and storage and replay engine that sort of glues 218.80s - 223.12s | everything together but at its core we're really just doing speech to text 223.12s - 230.96s | text to speech and some LLM


chunk_text:
to do something for you but it could be anybody it could be done for 176.16s - 179.84s | Matt it could be done for Steve it could be done for Jay I think that's who that 179.84s - 185.28s | was now that's what we would do but if I wanted to do something on behalf of you 185.28s - 189.48s | on your Spotify account on your Grubhub account whatever it is I need to be 189.48s - 196.60s | authenticated and so that's that fourth type that I like to think about so if we 196.60s - 202.88s | look at the general knowledge request it's it's fairly simple so I have my my 202.88s - 208.68s | AI walkie-talkie I push the button and I say whatever it is I'm gonna say what 208.68s - 214.92s | what's the weather out today it's gonna go up through the internet into my cloud 214.92s - 218.80s | provider there's a routing and storage and replay engine that sort of glues 218.80s - 223.12s | everything together but at its core we're really just doing speech to text 223.12s - 230.96s | text to speech and some LLM calls to to figure out what the right answer is so 230.96s - 236.64s | that's a knowledge request when we're doing those those types of requests 236.64s - 241.56s | right what kind of data are we thinking about well if my request is tell me a 241.56s - 248.12s | cat about a fact about cats super easy if I say I've got a weird rash should I 248.16s - 253.64s | see a doctor a little bit more sensitive than tell me a fact about cats and so 253.64s - 257.60s | when I think about protecting this data it's really three different kinds I've 257.60s - 262.16s | got audio data that recording of my voice that needs to get converted into 262.16s - 267.12s | speech-to-text I've got the transcript data which is the result of that 267.12s - 271.88s | speech-to-text call and then I've got my response data which is where I took that 271.88s - 279.73s | that text ran it through an LLM and had the the LLM give me an answer when we 279.73s - 284.89s | think about generic action requests it's very very similar to what we just


chunk_text:
calls to to figure out what the right answer is so 230.96s - 236.64s | that's a knowledge request when we're doing those those types of requests 236.64s - 241.56s | right what kind of data are we thinking about well if my request is tell me a 241.56s - 248.12s | cat about a fact about cats super easy if I say I've got a weird rash should I 248.16s - 253.64s | see a doctor a little bit more sensitive than tell me a fact about cats and so 253.64s - 257.60s | when I think about protecting this data it's really three different kinds I've 257.60s - 262.16s | got audio data that recording of my voice that needs to get converted into 262.16s - 267.12s | speech-to-text I've got the transcript data which is the result of that 267.12s - 271.88s | speech-to-text call and then I've got my response data which is where I took that 271.88s - 279.73s | that text ran it through an LLM and had the the LLM give me an answer when we 279.73s - 284.89s | think about generic action requests it's very very similar to what we just 284.89s - 288.97s | saw the only difference is now we insert an actuator so an actuator is just a 288.97s - 294.77s | thing that can perform a task when you provide an input and so this is this is 294.77s - 302.99s | the thing that calls the Google weather API for you as far as the types of data 302.99s - 306.75s | that that we need and that we need to protect we've still got our audio 306.75s - 311.43s | transcript and response data but we also have some contextual data right things 311.43s - 317.71s | like your location things like your interests if I ask about if I ask if I 317.71s - 323.03s | ask my a walkie-talkie for five restaurants in the area that are good 323.03s - 329.03s | ideally if I absolutely do not like cheeseburgers it would not give me the 329.03s - 332.31s | greatest cheeseburger restaurant in the neighborhood because it knows and so 332.31s - 336.95s | that contextual data is also something super sensitive that we want to protect 336.95s - 342.83s | and the last piece is the authenticated action request exact same


chunk_text:
284.89s - 288.97s | saw the only difference is now we insert an actuator so an actuator is just a 288.97s - 294.77s | thing that can perform a task when you provide an input and so this is this is 294.77s - 302.99s | the thing that calls the Google weather API for you as far as the types of data 302.99s - 306.75s | that that we need and that we need to protect we've still got our audio 306.75s - 311.43s | transcript and response data but we also have some contextual data right things 311.43s - 317.71s | like your location things like your interests if I ask about if I ask if I 317.71s - 323.03s | ask my a walkie-talkie for five restaurants in the area that are good 323.03s - 329.03s | ideally if I absolutely do not like cheeseburgers it would not give me the 329.03s - 332.31s | greatest cheeseburger restaurant in the neighborhood because it knows and so 332.31s - 336.95s | that contextual data is also something super sensitive that we want to protect 336.95s - 342.83s | and the last piece is the authenticated action request exact same flow as 342.83s - 346.83s | everything else the only difference is now we have to store some sort of 346.83s - 352.47s | credential somewhere right I can't order you food from your uber eats account 352.47s - 358.55s | unless I have a way to tell uber hey Matt wants food from uber so whenever I 358.55s - 366.55s | think about that more examples call me a nude an uber order me some tacos did 366.55s - 371.63s | Jimmy do his homework same old same data as before but we have additional 371.63s - 376.23s | context we have the identity of the person that's asking for it we have 376.23s - 381.23s | additional context about who this Jimmy person is right some like familial 381.23s - 391.46s | connection and then authentication information so this is like the sticky 391.46s - 397.98s | point that I feel like always it's always a really fun discussion and so 397.98s - 401.54s | so yeah I don't want to just make authentication information something 401.54s - 406.90s | tiny on the slide I want to actually talk about it so if I think about how 406.90s


chunk_text:
flow as 342.83s - 346.83s | everything else the only difference is now we have to store some sort of 346.83s - 352.47s | credential somewhere right I can't order you food from your uber eats account 352.47s - 358.55s | unless I have a way to tell uber hey Matt wants food from uber so whenever I 358.55s - 366.55s | think about that more examples call me a nude an uber order me some tacos did 366.55s - 371.63s | Jimmy do his homework same old same data as before but we have additional 371.63s - 376.23s | context we have the identity of the person that's asking for it we have 376.23s - 381.23s | additional context about who this Jimmy person is right some like familial 381.23s - 391.46s | connection and then authentication information so this is like the sticky 391.46s - 397.98s | point that I feel like always it's always a really fun discussion and so 397.98s - 401.54s | so yeah I don't want to just make authentication information something 401.54s - 406.90s | tiny on the slide I want to actually talk about it so if I think about how 406.90s - 414.82s | can I authenticate a customer well or as a customer sorry I can do one of three 414.82s - 419.18s | things really I can save their username and password in a database and then 419.18s - 424.78s | anytime I need to do something as them I just log in as them that's cool right 424.78s - 430.78s | probably not I can have them log in and I can store their cookies so at least I 430.78s - 435.74s | can't create a new session I can only use that same session again and then the 435.74s - 442.54s | last option is I can just use OAuth the problem is is as security professionals 442.54s - 447.06s | it's really easy to just say go be a grown-up and use OAuth but whenever 447.06s - 450.66s | you're you're working with a business that has to make money you you actually 450.66s - 454.98s | have to think about these things and consider some of the trade-offs so we're 454.98s - 457.86s | going to talk about some of the trade-offs so if I just store the 457.86s - 463.90s | username and password


chunk_text:
- 414.82s | can I authenticate a customer well or as a customer sorry I can do one of three 414.82s - 419.18s | things really I can save their username and password in a database and then 419.18s - 424.78s | anytime I need to do something as them I just log in as them that's cool right 424.78s - 430.78s | probably not I can have them log in and I can store their cookies so at least I 430.78s - 435.74s | can't create a new session I can only use that same session again and then the 435.74s - 442.54s | last option is I can just use OAuth the problem is is as security professionals 442.54s - 447.06s | it's really easy to just say go be a grown-up and use OAuth but whenever 447.06s - 450.66s | you're you're working with a business that has to make money you you actually 450.66s - 454.98s | have to think about these things and consider some of the trade-offs so we're 454.98s - 457.86s | going to talk about some of the trade-offs so if I just store the 457.86s - 463.90s | username and password it's great we can log in whenever we want sorry yeah 463.90s - 468.34s | wherever we want whenever we want I don't actually have to store any session 468.34s - 471.66s | information because if if I get logged out I'll just log back in because I know 471.66s - 477.78s | their username and password downside I have their username and password and 477.78s - 485.46s | someone can steal it bonus we can log in whenever we want I covered that part 485.46s - 491.26s | already and then the downside to that is in order to actually revoke our access a 491.26s - 497.06s | customer has to change their password again another bonus we can do anything 497.06s - 501.18s | that the customer can do because we have their username and password but the 501.18s - 505.94s | downside we can do anything the customer can do including change their password 505.94s - 511.70s | so if we're thinking about trade-offs this one doesn't feel like the right 511.70s - 518.78s | thing to do so the next thing we have right session cookies so lots of pros 518.78s - 523.42s | here sessions


chunk_text:
it's great we can log in whenever we want sorry yeah 463.90s - 468.34s | wherever we want whenever we want I don't actually have to store any session 468.34s - 471.66s | information because if if I get logged out I'll just log back in because I know 471.66s - 477.78s | their username and password downside I have their username and password and 477.78s - 485.46s | someone can steal it bonus we can log in whenever we want I covered that part 485.46s - 491.26s | already and then the downside to that is in order to actually revoke our access a 491.26s - 497.06s | customer has to change their password again another bonus we can do anything 497.06s - 501.18s | that the customer can do because we have their username and password but the 501.18s - 505.94s | downside we can do anything the customer can do including change their password 505.94s - 511.70s | so if we're thinking about trade-offs this one doesn't feel like the right 511.70s - 518.78s | thing to do so the next thing we have right session cookies so lots of pros 518.78s - 523.42s | here sessions expire regularly the downside for the engineering team is 523.42s - 528.46s | sessions expire regularly I can have access to almost everything that a user 528.46s - 534.62s | can do if you've ever tried to go in and add someone to a repository in github 534.62s - 539.62s | like you can create a repository just I'm logged in but in order to add a 539.62s - 542.86s | person to that repository you have to step up you have to type in your 542.86s - 548.06s | password again you have to do an MFA push and so this is really great because 548.06s - 553.58s | if the provider supports it we can actually scope down what it is that 553.58s - 558.74s | we're able to do but the downside is the provider actually has to support that so 558.74s - 564.02s | while github might have that requirement the McDonald's mobile app maybe doesn't 565.02s - 572.18s | another nice thing on the engineering side is typically MFA requirements bot 572.18s - 577.18s | protection all of that happens at sign-in and before you actually get your 577.18s - 584.86s | session token for


chunk_text:
expire regularly the downside for the engineering team is 523.42s - 528.46s | sessions expire regularly I can have access to almost everything that a user 528.46s - 534.62s | can do if you've ever tried to go in and add someone to a repository in github 534.62s - 539.62s | like you can create a repository just I'm logged in but in order to add a 539.62s - 542.86s | person to that repository you have to step up you have to type in your 542.86s - 548.06s | password again you have to do an MFA push and so this is really great because 548.06s - 553.58s | if the provider supports it we can actually scope down what it is that 553.58s - 558.74s | we're able to do but the downside is the provider actually has to support that so 558.74s - 564.02s | while github might have that requirement the McDonald's mobile app maybe doesn't 565.02s - 572.18s | another nice thing on the engineering side is typically MFA requirements bot 572.18s - 577.18s | protection all of that happens at sign-in and before you actually get your 577.18s - 584.86s | session token for the site the downside is that it also doesn't usually generate 584.86s - 588.94s | a new login event so if you're like me and every time you get a message from 588.94s - 593.38s | Google that says hey you just logged in from somewhere new was that you and you 593.38s - 598.42s | get like super paranoid you're not gonna get those because most providers 598.42s - 601.06s | just say oh you've got a session token that already works so we're just gonna 601.06s - 609.90s | use that so that's it's not great a good thing users can can actually manually 609.90s - 613.94s | expire those sessions for themselves which is great right because if I go to 613.94s - 619.02s | gmail and I say where am I logged in and it says I'm logged into AWS I'm not 619.06s - 624.14s | logged into AWS I'm gonna click log out so that's great but again it requires 624.14s - 630.50s | the provider to actually create that opportunity for their customer and then 630.50s - 634.54s | the the other nice thing about this is it can be used on almost any


chunk_text:
the site the downside is that it also doesn't usually generate 584.86s - 588.94s | a new login event so if you're like me and every time you get a message from 588.94s - 593.38s | Google that says hey you just logged in from somewhere new was that you and you 593.38s - 598.42s | get like super paranoid you're not gonna get those because most providers 598.42s - 601.06s | just say oh you've got a session token that already works so we're just gonna 601.06s - 609.90s | use that so that's it's not great a good thing users can can actually manually 609.90s - 613.94s | expire those sessions for themselves which is great right because if I go to 613.94s - 619.02s | gmail and I say where am I logged in and it says I'm logged into AWS I'm not 619.06s - 624.14s | logged into AWS I'm gonna click log out so that's great but again it requires 624.14s - 630.50s | the provider to actually create that opportunity for their customer and then 630.50s - 634.54s | the the other nice thing about this is it can be used on almost any website 634.54s - 641.70s | right when we get to the OAuth piece the biggest limitation is if you look at all 641.70s - 646.62s | of the websites on the internet how many of them actually support OAuth if we're 646.62s - 653.22s | in the b2b world and we're focusing just on oh I want I want Datadog I want 653.22s - 658.18s | GitLab I want all of these like commercial enterprises absolutely 658.18s - 661.30s | there's OAuth because it's a thing that we've been hounding vendors about for 661.30s - 666.50s | the last I don't know 20 years but there's a benefit there and that is if 666.50s - 672.06s | they do it they get our business whereas if you're a local mechanic shop and you 672.06s - 677.62s | just want people to book spots online with you you're not incentivized to 677.62s - 683.14s | invest in OAuth and so how do we support the general consumer with a technology 683.14s - 688.78s | that doesn't exist everywhere the other thing that I think sort of makes OAuth 688.78s - 693.98s | tricky in situations like this is you have to


chunk_text:
website 634.54s - 641.70s | right when we get to the OAuth piece the biggest limitation is if you look at all 641.70s - 646.62s | of the websites on the internet how many of them actually support OAuth if we're 646.62s - 653.22s | in the b2b world and we're focusing just on oh I want I want Datadog I want 653.22s - 658.18s | GitLab I want all of these like commercial enterprises absolutely 658.18s - 661.30s | there's OAuth because it's a thing that we've been hounding vendors about for 661.30s - 666.50s | the last I don't know 20 years but there's a benefit there and that is if 666.50s - 672.06s | they do it they get our business whereas if you're a local mechanic shop and you 672.06s - 677.62s | just want people to book spots online with you you're not incentivized to 677.62s - 683.14s | invest in OAuth and so how do we support the general consumer with a technology 683.14s - 688.78s | that doesn't exist everywhere the other thing that I think sort of makes OAuth 688.78s - 693.98s | tricky in situations like this is you have to configure it for each site and 693.98s - 699.82s | so if there are a hundred million sites on the internet and I hire a 10x 699.90s - 705.66s | engineer who can knock out one integration every six hours okay so 705.66s - 710.82s | that's six hundred million did I say a hundred million sites I don't know but 710.82s - 714.02s | the the point is it's a very very long time to actually be able to reach the 714.02s - 720.26s | entire internet so we've done our pros and cons let's put them on a graph that 720.26s - 723.98s | graph saying low effort to high effort obviously the business likes that 723.98s - 728.66s | whenever we do things low effort not secure to more secure obviously as 728.70s - 732.18s | professionals we like to be over there on the far right hand side so we look at 732.18s - 735.54s | this graph and we say you know what there's just so much risk with username 735.54s - 739.58s | and password that's out and then we go to our CTO and we say hey I want to do 739.58s -


chunk_text:
configure it for each site and 693.98s - 699.82s | so if there are a hundred million sites on the internet and I hire a 10x 699.90s - 705.66s | engineer who can knock out one integration every six hours okay so 705.66s - 710.82s | that's six hundred million did I say a hundred million sites I don't know but 710.82s - 714.02s | the the point is it's a very very long time to actually be able to reach the 714.02s - 720.26s | entire internet so we've done our pros and cons let's put them on a graph that 720.26s - 723.98s | graph saying low effort to high effort obviously the business likes that 723.98s - 728.66s | whenever we do things low effort not secure to more secure obviously as 728.70s - 732.18s | professionals we like to be over there on the far right hand side so we look at 732.18s - 735.54s | this graph and we say you know what there's just so much risk with username 735.54s - 739.58s | and password that's out and then we go to our CTO and we say hey I want to do 739.58s - 744.26s | OAuth to the world we're gonna call get token every time we need this and I tell 744.26s - 748.02s | them it's gonna take a thousand years to get all of the sites that they want to 748.02s - 751.98s | cover done and I get told no you know what that's probably not and even as a 751.98s - 757.46s | practitioner right I realized that if I try to make the world's most secure 757.46s - 762.14s | company that's just us not making money with all the computers turned off and I 762.14s - 766.14s | I like having a place to work I like building cool things for consumers and 766.14s - 773.74s | so working with cookies for now is kind of like the way that I see us going so 773.74s - 781.34s | let's talk about that user context that authentication information so there's 781.34s - 785.74s | sort of like the naive way right where you can just do it make it work it's 785.74s - 789.98s | gonna work great and then there's more of a more secure way so first let's talk 789.98s - 794.38s | about


chunk_text:
744.26s | OAuth to the world we're gonna call get token every time we need this and I tell 744.26s - 748.02s | them it's gonna take a thousand years to get all of the sites that they want to 748.02s - 751.98s | cover done and I get told no you know what that's probably not and even as a 751.98s - 757.46s | practitioner right I realized that if I try to make the world's most secure 757.46s - 762.14s | company that's just us not making money with all the computers turned off and I 762.14s - 766.14s | I like having a place to work I like building cool things for consumers and 766.14s - 773.74s | so working with cookies for now is kind of like the way that I see us going so 773.74s - 781.34s | let's talk about that user context that authentication information so there's 781.34s - 785.74s | sort of like the naive way right where you can just do it make it work it's 785.74s - 789.98s | gonna work great and then there's more of a more secure way so first let's talk 789.98s - 794.38s | about the naive way so the first thing that I'll do is recognize oh the user 794.38s - 799.06s | wants to store a cookie I'll create context the intention controller is 799.06s - 803.14s | going to recognize that and say oh they want to store a cookie cool it's going 803.14s - 809.70s | to create a context container that container exists purely to to to allow 809.70s - 814.42s | the user to store those cookies so some companies do it in a web browser some 814.42s - 820.54s | companies do it with a Chrome extension lots of different ways that you could do 820.54s - 824.54s | it but the point is we're going to collect that context we're gonna write 824.54s - 829.86s | it into a database whenever it comes time to use that cookie again it all 829.86s - 834.98s | fits on one slide I recognize I want to use this context I say oh this is Matt's 834.98s - 840.06s | user account I'm gonna go ahead and go into the database grab Matt's cookie and 840.18s - 844.74s | then stuff it into an actuator that actuator now has a


chunk_text:
the naive way so the first thing that I'll do is recognize oh the user 794.38s - 799.06s | wants to store a cookie I'll create context the intention controller is 799.06s - 803.14s | going to recognize that and say oh they want to store a cookie cool it's going 803.14s - 809.70s | to create a context container that container exists purely to to to allow 809.70s - 814.42s | the user to store those cookies so some companies do it in a web browser some 814.42s - 820.54s | companies do it with a Chrome extension lots of different ways that you could do 820.54s - 824.54s | it but the point is we're going to collect that context we're gonna write 824.54s - 829.86s | it into a database whenever it comes time to use that cookie again it all 829.86s - 834.98s | fits on one slide I recognize I want to use this context I say oh this is Matt's 834.98s - 840.06s | user account I'm gonna go ahead and go into the database grab Matt's cookie and 840.18s - 844.74s | then stuff it into an actuator that actuator now has a login session for me 844.74s - 850.18s | and it can order the thing that I asked for it goes off into the third-party 850.18s - 855.42s | space and does the thing it's supposed to do what's the problem with this what 855.42s - 862.61s | makes you nervous stolen cookies I heard that in the front awesome yeah exactly I 862.61s - 868.25s | don't want to be the guy that has a database just full of cookies that other 868.25s - 873.65s | people can just come in and take from me that would be really really bad so 873.65s - 878.49s | let's think about a way that we could do this a little bit more securely so again 878.49s - 883.01s | I'll recognize I want to create some context I want to store that cookie the 883.01s - 886.53s | intention controller is going to recognize that very similar to before 886.53s - 891.81s | it's gonna create that that web browser so that the customer can log in but then 891.81s - 896.49s | notice we have a vault service down below and so once that cookies collected 896.49s - 901.33s | instead


chunk_text:
login session for me 844.74s - 850.18s | and it can order the thing that I asked for it goes off into the third-party 850.18s - 855.42s | space and does the thing it's supposed to do what's the problem with this what 855.42s - 862.61s | makes you nervous stolen cookies I heard that in the front awesome yeah exactly I 862.61s - 868.25s | don't want to be the guy that has a database just full of cookies that other 868.25s - 873.65s | people can just come in and take from me that would be really really bad so 873.65s - 878.49s | let's think about a way that we could do this a little bit more securely so again 878.49s - 883.01s | I'll recognize I want to create some context I want to store that cookie the 883.01s - 886.53s | intention controller is going to recognize that very similar to before 886.53s - 891.81s | it's gonna create that that web browser so that the customer can log in but then 891.81s - 896.49s | notice we have a vault service down below and so once that cookies collected 896.49s - 901.33s | instead of just stuffing it into a vault and then returning it to the 901.33s - 904.89s | database what we're actually gonna do is we're gonna stuff it into the vault but 904.89s - 909.33s | we're also going to add the identity of the customer it belongs to and so when 909.33s - 913.77s | we get to the next slide you'll see that we should actually say hey yes I want 913.77s - 918.37s | this cookie also Matt said I could have it two minutes ago and so by having like 918.37s - 924.57s | a short-lived JWT that the user provides with you provides to you your life just 924.57s - 928.41s | gets a lot better I can sleep a lot better at night knowing that I'm not 928.41s - 933.61s | storing cookies for a hundred thousand people I'm storing things that I'm 933.61s - 938.73s | storing encrypted blobs and you would have to actually have a request from the 938.73s - 945.28s | user in order to go get that cookie so when we get to using the context 945.28s - 949.44s | securely I kind of already covered this but we'll


chunk_text:
of just stuffing it into a vault and then returning it to the 901.33s - 904.89s | database what we're actually gonna do is we're gonna stuff it into the vault but 904.89s - 909.33s | we're also going to add the identity of the customer it belongs to and so when 909.33s - 913.77s | we get to the next slide you'll see that we should actually say hey yes I want 913.77s - 918.37s | this cookie also Matt said I could have it two minutes ago and so by having like 918.37s - 924.57s | a short-lived JWT that the user provides with you provides to you your life just 924.57s - 928.41s | gets a lot better I can sleep a lot better at night knowing that I'm not 928.41s - 933.61s | storing cookies for a hundred thousand people I'm storing things that I'm 933.61s - 938.73s | storing encrypted blobs and you would have to actually have a request from the 938.73s - 945.28s | user in order to go get that cookie so when we get to using the context 945.28s - 949.44s | securely I kind of already covered this but we'll walk through it again we 949.44s - 953.32s | recognize the user wants to take an authenticated action we get the object 953.32s - 959.12s | ID and not the actual object from the database we send that object ID and the 959.12s - 965.80s | users JWT to our vault or sorry to our context actuator the context 965.80s - 970.96s | actuator now is the thing that reaches out to the vault and gets that cookie 970.96s - 977.84s | and so this one tiny space in our network is the only place that with a 977.88s - 983.96s | request from the customer you can actually go get their cookie once once 983.96s - 988.92s | the actuator performs the action we tear it down we never use it again it was 988.92s - 993.56s | just that one thing and so the risk of someone coming in and stealing a token 993.56s - 998.60s | off of the the actuator gets a lot smaller right essentially we're looking 998.60s - 1005.00s | at like build system compromises that allow you to get in versus earlier it 1005.00s - 1012.64s | was just can someone access my


chunk_text:
walk through it again we 949.44s - 953.32s | recognize the user wants to take an authenticated action we get the object 953.32s - 959.12s | ID and not the actual object from the database we send that object ID and the 959.12s - 965.80s | users JWT to our vault or sorry to our context actuator the context 965.80s - 970.96s | actuator now is the thing that reaches out to the vault and gets that cookie 970.96s - 977.84s | and so this one tiny space in our network is the only place that with a 977.88s - 983.96s | request from the customer you can actually go get their cookie once once 983.96s - 988.92s | the actuator performs the action we tear it down we never use it again it was 988.92s - 993.56s | just that one thing and so the risk of someone coming in and stealing a token 993.56s - 998.60s | off of the the actuator gets a lot smaller right essentially we're looking 998.60s - 1005.00s | at like build system compromises that allow you to get in versus earlier it 1005.00s - 1012.64s | was just can someone access my database so it's a lot as far as securing the 1012.64s - 1018.48s | customer context so some do's and don'ts to sort of take away don't collect 1018.48s - 1027.08s | context that'll make you a target if you are only storing cookies for DoorDash 1027.08s - 1034.28s | Spotify and uber not a lot of people are gonna come after you right like like why 1034.28s - 1040.32s | do I want this cookie that is likely going to get caught there's much better 1040.32s - 1045.48s | things for me to spend my time on as an attacker so if we ever get to a world 1045.48s - 1050.84s | where somebody wants to use their AI walkie-talkie to spend money I don't 1050.84s - 1056.00s | want to store that financial instrument because that makes me a target and so 1056.00s - 1061.64s | allowing allowing vendors service providers to actually own that piece is 1061.64s - 1069.50s | super super important to me the second piece is don't reuse context or or yeah 1069.50s - 1073.76s | context containers or actuators across users so anytime I'm taking something 1073.76s - 1078.36s | sensitive from the


chunk_text:
database so it's a lot as far as securing the 1012.64s - 1018.48s | customer context so some do's and don'ts to sort of take away don't collect 1018.48s - 1027.08s | context that'll make you a target if you are only storing cookies for DoorDash 1027.08s - 1034.28s | Spotify and uber not a lot of people are gonna come after you right like like why 1034.28s - 1040.32s | do I want this cookie that is likely going to get caught there's much better 1040.32s - 1045.48s | things for me to spend my time on as an attacker so if we ever get to a world 1045.48s - 1050.84s | where somebody wants to use their AI walkie-talkie to spend money I don't 1050.84s - 1056.00s | want to store that financial instrument because that makes me a target and so 1056.00s - 1061.64s | allowing allowing vendors service providers to actually own that piece is 1061.64s - 1069.50s | super super important to me the second piece is don't reuse context or or yeah 1069.50s - 1073.76s | context containers or actuators across users so anytime I'm taking something 1073.76s - 1078.36s | sensitive from the customer it should be a one-off thing and then when it's done 1078.36s - 1083.24s | it should go away and then I don't ever have to worry about oh well I accidentally 1083.24s - 1090.52s | ordered $1,000 worth of tacos on Jay's uber eats account instead of on Matt's 1090.52s - 1099.84s | and now Jay has a ton of tacos to share with who I don't know so not doing those 1099.84s - 1105.44s | and then the the other don't big don't is don't trust that context as being 1105.44s - 1111.44s | true right it's really easy for for our engineers and developers to think oh 1111.44s - 1114.68s | yeah no this is a cookie that the customer gave us to order for them so 1114.68s - 1118.68s | that's good right no the answer is no this is data that came from the internet 1118.68s - 1121.40s | and we should just treat it like all the other data that came from the 1121.40s - 1128.72s | internet a couple of dues allow this vault access to only the that one thing 1128.72s - 1135.56s | that actually needs to go get it


chunk_text:
customer it should be a one-off thing and then when it's done 1078.36s - 1083.24s | it should go away and then I don't ever have to worry about oh well I accidentally 1083.24s - 1090.52s | ordered $1,000 worth of tacos on Jay's uber eats account instead of on Matt's 1090.52s - 1099.84s | and now Jay has a ton of tacos to share with who I don't know so not doing those 1099.84s - 1105.44s | and then the the other don't big don't is don't trust that context as being 1105.44s - 1111.44s | true right it's really easy for for our engineers and developers to think oh 1111.44s - 1114.68s | yeah no this is a cookie that the customer gave us to order for them so 1114.68s - 1118.68s | that's good right no the answer is no this is data that came from the internet 1118.68s - 1121.40s | and we should just treat it like all the other data that came from the 1121.40s - 1128.72s | internet a couple of dues allow this vault access to only the that one thing 1128.72s - 1135.56s | that actually needs to go get it this way if you do if you do have any issues 1135.56s - 1140.72s | with people trying or having the ability to connect to your vault service they 1140.72s - 1144.84s | can't actually do anything right because you have to be running in that one 1144.84s - 1150.64s | specific container in order to access those tokens and then the last piece 1150.64s - 1156.32s | which is kind of my favorite is correlation IDs so anytime I need to ask 1156.32s - 1162.88s | for a cookie from the vault I should be able to match that up to a request that 1162.88s - 1168.16s | came from a physical device and if I can't that means that somebody that I 1168.16s - 1171.56s | work with is going to have a bad day or I'm gonna have a bad day because 1171.56s - 1175.44s | somebody else is creating those requests and so I think that's a thing 1175.44s - 1179.56s | that we sort of owe to our customers when we build devices like this is a way 1179.56s - 1183.36s | to tell them not only are we doing our best to protect your


chunk_text:
this way if you do if you do have any issues 1135.56s - 1140.72s | with people trying or having the ability to connect to your vault service they 1140.72s - 1144.84s | can't actually do anything right because you have to be running in that one 1144.84s - 1150.64s | specific container in order to access those tokens and then the last piece 1150.64s - 1156.32s | which is kind of my favorite is correlation IDs so anytime I need to ask 1156.32s - 1162.88s | for a cookie from the vault I should be able to match that up to a request that 1162.88s - 1168.16s | came from a physical device and if I can't that means that somebody that I 1168.16s - 1171.56s | work with is going to have a bad day or I'm gonna have a bad day because 1171.56s - 1175.44s | somebody else is creating those requests and so I think that's a thing 1175.44s - 1179.56s | that we sort of owe to our customers when we build devices like this is a way 1179.56s - 1183.36s | to tell them not only are we doing our best to protect your authentication 1183.36s - 1188.24s | information from random people on the internet but we also recognize your data 1188.24s - 1192.60s | is important to you we are not you so we shouldn't have access to it either and 1192.60s - 1199.52s | so by by introducing those correlation IDs you can do really cool things like 1199.52s - 1204.68s | requiring that correlation ID to exist before the request gets routed into the 1204.68s - 1208.80s | vault or you can just set up automated alerting so that you know if it ever 1208.80s - 1216.92s | happens you at least know about it so do's and don'ts for context so the next 1216.92s - 1222.96s | piece securing customer conversations this one was really really fun to think 1222.96s - 1229.04s | about so if I take out my AI walkie-talkie and I say what's my 1229.04s - 1234.00s | favorite color there's a couple of types of data that happen there right we still 1234.00s - 1239.44s | have our same speech-to-text request and response we still have the LLM request 1239.44s - 1245.24s | and response and the the text-to-speech back as well so essentially I


chunk_text:
authentication 1183.36s - 1188.24s | information from random people on the internet but we also recognize your data 1188.24s - 1192.60s | is important to you we are not you so we shouldn't have access to it either and 1192.60s - 1199.52s | so by by introducing those correlation IDs you can do really cool things like 1199.52s - 1204.68s | requiring that correlation ID to exist before the request gets routed into the 1204.68s - 1208.80s | vault or you can just set up automated alerting so that you know if it ever 1208.80s - 1216.92s | happens you at least know about it so do's and don'ts for context so the next 1216.92s - 1222.96s | piece securing customer conversations this one was really really fun to think 1222.96s - 1229.04s | about so if I take out my AI walkie-talkie and I say what's my 1229.04s - 1234.00s | favorite color there's a couple of types of data that happen there right we still 1234.00s - 1239.44s | have our same speech-to-text request and response we still have the LLM request 1239.44s - 1245.24s | and response and the the text-to-speech back as well so essentially I have audio 1245.24s - 1250.72s | data and I have text data that I need to worry about as far as what type of 1250.72s - 1254.60s | binary data it might be an audio recording but it also might be a picture 1254.60s - 1259.76s | right maybe your your magic AI walkie-talkie takes magic photos and when 1259.76s - 1266.36s | it takes the photo it converts it into like some cool 8-bit format I probably 1266.36s - 1270.28s | don't want random people on the internet or even the company that makes that AI 1270.28s - 1273.88s | walkie-talkie to have access to those pictures right because I'm a general 1273.88s - 1279.68s | consumer why do people need access to that same thing goes with with all of 1279.68s - 1282.84s | the the text-to-speech that we just talked about right we have a transcript 1282.84s - 1287.44s | of what the user said we have an answer to their question and we have what we 1287.44s - 1293.76s | want the device to echo back so how do we store that well really really easy 1293.76s - 1300.24s | way is I have my magic


chunk_text:
have audio 1245.24s - 1250.72s | data and I have text data that I need to worry about as far as what type of 1250.72s - 1254.60s | binary data it might be an audio recording but it also might be a picture 1254.60s - 1259.76s | right maybe your your magic AI walkie-talkie takes magic photos and when 1259.76s - 1266.36s | it takes the photo it converts it into like some cool 8-bit format I probably 1266.36s - 1270.28s | don't want random people on the internet or even the company that makes that AI 1270.28s - 1273.88s | walkie-talkie to have access to those pictures right because I'm a general 1273.88s - 1279.68s | consumer why do people need access to that same thing goes with with all of 1279.68s - 1282.84s | the the text-to-speech that we just talked about right we have a transcript 1282.84s - 1287.44s | of what the user said we have an answer to their question and we have what we 1287.44s - 1293.76s | want the device to echo back so how do we store that well really really easy 1293.76s - 1300.24s | way is I have my magic AI gadget it sends a request to that super awesome 1300.24s - 1306.00s | router that will take everything that I wrote and then just store it in a 1306.00s - 1311.64s | database if it's binary data it'll just store it in s3 in the database I just 1311.64s - 1316.60s | put a pointer to s3 life's great this is super easy right again we're storing 1316.60s - 1322.96s | sensitive things about people in a database I don't want to be like going 1322.96s - 1328.52s | to sleep at night knowing that that happens so what do we do to make life a 1328.52s - 1336.56s | little bit safer so instead of just receiving that data from the the magical 1336.56s - 1342.84s | device and then writing it all down again we'll do the same thing with with 1342.84s - 1349.72s | our vault engine and so whenever we receive the data we'll actually reach 1349.72s - 1355.00s | out to the vault engine and say can I have a kind of got two really cool 1355.00s - 1360.40s | options here so you could have the the service send that data to the


chunk_text:
AI gadget it sends a request to that super awesome 1300.24s - 1306.00s | router that will take everything that I wrote and then just store it in a 1306.00s - 1311.64s | database if it's binary data it'll just store it in s3 in the database I just 1311.64s - 1316.60s | put a pointer to s3 life's great this is super easy right again we're storing 1316.60s - 1322.96s | sensitive things about people in a database I don't want to be like going 1322.96s - 1328.52s | to sleep at night knowing that that happens so what do we do to make life a 1328.52s - 1336.56s | little bit safer so instead of just receiving that data from the the magical 1336.56s - 1342.84s | device and then writing it all down again we'll do the same thing with with 1342.84s - 1349.72s | our vault engine and so whenever we receive the data we'll actually reach 1349.72s - 1355.00s | out to the vault engine and say can I have a kind of got two really cool 1355.00s - 1360.40s | options here so you could have the the service send that data to the vault 1360.40s - 1365.40s | engine have it encrypt it and then return back to you the the ciphertext 1365.40s - 1369.16s | and now you can just write that now in the future you just have to take that 1369.16s - 1372.48s | ciphertext and say vault engine can you give me this data in plain text please 1372.48s - 1377.12s | the other option that you have is you can request the customers storage key 1377.12s - 1380.60s | and you can say please give me the key for Matt Domko I have a bunch of stuff I 1380.60s - 1387.64s | want to write in in both cases I'm not storing things that I have access to I'm 1387.64s - 1392.04s | storing things that I only have access to for that two-minute window whenever 1392.04s - 1402.58s | Matt Domko pushes the button so I've got some do's and don'ts for securing those 1402.58s - 1408.66s | conversations so don't please don't use built-in types for these things right 1408.66s - 1413.82s | the my worst day ever is whenever someone says oh yeah no we're just 1413.82s - 1416.90s | sending this JSON and there's


chunk_text:
vault 1360.40s - 1365.40s | engine have it encrypt it and then return back to you the the ciphertext 1365.40s - 1369.16s | and now you can just write that now in the future you just have to take that 1369.16s - 1372.48s | ciphertext and say vault engine can you give me this data in plain text please 1372.48s - 1377.12s | the other option that you have is you can request the customers storage key 1377.12s - 1380.60s | and you can say please give me the key for Matt Domko I have a bunch of stuff I 1380.60s - 1387.64s | want to write in in both cases I'm not storing things that I have access to I'm 1387.64s - 1392.04s | storing things that I only have access to for that two-minute window whenever 1392.04s - 1402.58s | Matt Domko pushes the button so I've got some do's and don'ts for securing those 1402.58s - 1408.66s | conversations so don't please don't use built-in types for these things right 1408.66s - 1413.82s | the my worst day ever is whenever someone says oh yeah no we're just 1413.82s - 1416.90s | sending this JSON and there's just like a string there and I look at it I'm like 1416.90s - 1420.70s | okay cool it's just string no big deal and that string is actually like six 1420.70s - 1424.98s | hours of me watching Gilmore Girls because I started a meeting 1424.98s - 1429.14s | transcription during it so rather than taking these things and just going with 1429.14s - 1433.50s | the defaults let's create a custom object use a strongly typed language 1433.50s - 1437.70s | that allows you to say okay this this is actually a type of sensitive data and 1437.70s - 1443.42s | the only way that it's allowed to be logged is either through that vault 1443.42s - 1451.22s | controller interaction or through a secure logger so yeah don't please one 1451.22s - 1459.74s | second so the other thing that definitely in the business-to-business 1459.74s - 1466.46s | space it happens and it's not great but it could be worse right essentially if I 1466.46s - 1470.34s | need to do analytics on data and I have the data in a place well let me just go 1470.34s - 1474.70s | grab that data and do analytics


chunk_text:
just like a string there and I look at it I'm like 1416.90s - 1420.70s | okay cool it's just string no big deal and that string is actually like six 1420.70s - 1424.98s | hours of me watching Gilmore Girls because I started a meeting 1424.98s - 1429.14s | transcription during it so rather than taking these things and just going with 1429.14s - 1433.50s | the defaults let's create a custom object use a strongly typed language 1433.50s - 1437.70s | that allows you to say okay this this is actually a type of sensitive data and 1437.70s - 1443.42s | the only way that it's allowed to be logged is either through that vault 1443.42s - 1451.22s | controller interaction or through a secure logger so yeah don't please one 1451.22s - 1459.74s | second so the other thing that definitely in the business-to-business 1459.74s - 1466.46s | space it happens and it's not great but it could be worse right essentially if I 1466.46s - 1470.34s | need to do analytics on data and I have the data in a place well let me just go 1470.34s - 1474.70s | grab that data and do analytics it's one thing to do it whenever it's how many 1474.70s - 1479.82s | times did my employees click on a phishing link it's another thing all 1479.82s - 1487.94s | together for it to be a recording of me and my kid talking about something so 1487.94s - 1494.46s | don't let it happen right don't don't share data between production and 1494.46s - 1498.98s | anything else if you're not using that data to provide a direct value to the 1498.98s - 1504.86s | user you're essentially stealing their data right now and I not a fan and then 1504.86s - 1510.10s | the last one don't allow new log types without a privacy and security review 1510.10s - 1516.26s | and so this is a thing that I've sort of started harping on a lot more work which 1516.26s - 1520.74s | is just like if if you want to do something new and it has to do with 1520.74s - 1526.26s | customer data we owe it to our customers to spend 15 minutes drawing out what 1526.26s - 1529.22s | we're going to do identifying the data that we're going to write down and 1529.22s


chunk_text:
it's one thing to do it whenever it's how many 1474.70s - 1479.82s | times did my employees click on a phishing link it's another thing all 1479.82s - 1487.94s | together for it to be a recording of me and my kid talking about something so 1487.94s - 1494.46s | don't let it happen right don't don't share data between production and 1494.46s - 1498.98s | anything else if you're not using that data to provide a direct value to the 1498.98s - 1504.86s | user you're essentially stealing their data right now and I not a fan and then 1504.86s - 1510.10s | the last one don't allow new log types without a privacy and security review 1510.10s - 1516.26s | and so this is a thing that I've sort of started harping on a lot more work which 1516.26s - 1520.74s | is just like if if you want to do something new and it has to do with 1520.74s - 1526.26s | customer data we owe it to our customers to spend 15 minutes drawing out what 1526.26s - 1529.22s | we're going to do identifying the data that we're going to write down and 1529.22s - 1533.90s | talking about whether or not that is a good thing to do with our customers data 1533.90s - 1542.50s | so yeah don't do that do move encryption to client-side where possible so the way 1542.50s - 1550.62s | that it is in this diagram that I just showed you as the service provider you 1550.62s - 1556.02s | can just grab a JWT and go get the data wouldn't it be great if the only person 1556.02s - 1560.14s | that had your data was like you in your web browser when you get to go view your 1560.14s - 1565.02s | pictures that would be great that's a technology that exists it's not a 1565.02s - 1569.98s | technology that exists as far as I'm aware that works really well with AI use 1569.98s - 1573.54s | cases where we want to get your permission to use your data to make 1573.54s - 1579.46s | things better and so that's that's a tough it's a tough road to climb but I 1579.46s - 1585.06s | think I think that there's it's it's a goal to move towards require a short 1585.06s - 1591.30s | lift


chunk_text:
- 1533.90s | talking about whether or not that is a good thing to do with our customers data 1533.90s - 1542.50s | so yeah don't do that do move encryption to client-side where possible so the way 1542.50s - 1550.62s | that it is in this diagram that I just showed you as the service provider you 1550.62s - 1556.02s | can just grab a JWT and go get the data wouldn't it be great if the only person 1556.02s - 1560.14s | that had your data was like you in your web browser when you get to go view your 1560.14s - 1565.02s | pictures that would be great that's a technology that exists it's not a 1565.02s - 1569.98s | technology that exists as far as I'm aware that works really well with AI use 1569.98s - 1573.54s | cases where we want to get your permission to use your data to make 1573.54s - 1579.46s | things better and so that's that's a tough it's a tough road to climb but I 1579.46s - 1585.06s | think I think that there's it's it's a goal to move towards require a short 1585.06s - 1591.30s | lift JWT like this is really really easy to do you just when your user wants 1591.30s - 1595.10s | to do something make them send you a token that says that they want to do it 1595.10s - 1599.74s | and have that expire quickly if you look into the different types of vault 1599.74s - 1605.30s | providers that exist I know of at least one that that does that and you can give 1605.30s - 1609.90s | them a JWT and they will only give you the data back if that JWT is signed by 1609.90s - 1614.02s | the customer the person who owns the data I sleep a lot better at night 1614.02s - 1619.14s | knowing that I have that and then the last piece if you've ever worked at an 1619.14s - 1625.14s | AI company you probably have like a chat room where it's just AI news and people 1625.14s - 1628.42s | are excited about new models and new things that are coming out and 1628.42s - 1634.98s | anthropic has this AWS has this open AI has this and they want to try it out the 1634.98s - 1640.42s | problem


chunk_text:
JWT like this is really really easy to do you just when your user wants 1591.30s - 1595.10s | to do something make them send you a token that says that they want to do it 1595.10s - 1599.74s | and have that expire quickly if you look into the different types of vault 1599.74s - 1605.30s | providers that exist I know of at least one that that does that and you can give 1605.30s - 1609.90s | them a JWT and they will only give you the data back if that JWT is signed by 1609.90s - 1614.02s | the customer the person who owns the data I sleep a lot better at night 1614.02s - 1619.14s | knowing that I have that and then the last piece if you've ever worked at an 1619.14s - 1625.14s | AI company you probably have like a chat room where it's just AI news and people 1625.14s - 1628.42s | are excited about new models and new things that are coming out and 1628.42s - 1634.98s | anthropic has this AWS has this open AI has this and they want to try it out the 1634.98s - 1640.42s | problem is unless you have like like a commercial agreement with them a lot of 1640.42s - 1645.50s | times you're just sort of giving them free rein to do whatever they want with 1645.50s - 1651.62s | your customer data and so one of the things that that I've worked really hard 1651.62s - 1655.86s | on at the last few jobs has been making sure that we we actually make those 1655.86s - 1660.98s | thoughtful decisions we say okay this type of data maybe we're okay with with 1660.98s - 1665.42s | that company using it for whatever they want but this other type of data super 1665.42s - 1670.30s | super important to us and we're not gonna allow it to to go to those levels 1670.30s - 1677.58s | of LLMs essentially pay more to protect our customers I think it's worth it so 1677.58s - 1684.26s | the last one is privacy focused logging so if you're an engineer and you're 1684.26s - 1688.18s | coming up with logs for this new service you built this is probably how you're 1688.18s - 1690.54s | gonna do your logs you're just gonna take everything that you


chunk_text:
is unless you have like like a commercial agreement with them a lot of 1640.42s - 1645.50s | times you're just sort of giving them free rein to do whatever they want with 1645.50s - 1651.62s | your customer data and so one of the things that that I've worked really hard 1651.62s - 1655.86s | on at the last few jobs has been making sure that we we actually make those 1655.86s - 1660.98s | thoughtful decisions we say okay this type of data maybe we're okay with with 1660.98s - 1665.42s | that company using it for whatever they want but this other type of data super 1665.42s - 1670.30s | super important to us and we're not gonna allow it to to go to those levels 1670.30s - 1677.58s | of LLMs essentially pay more to protect our customers I think it's worth it so 1677.58s - 1684.26s | the last one is privacy focused logging so if you're an engineer and you're 1684.26s - 1688.18s | coming up with logs for this new service you built this is probably how you're 1688.18s - 1690.54s | gonna do your logs you're just gonna take everything that you know you're 1690.54s - 1694.86s | like I might need this someday ship it to data doc and then one of your SRE 1694.86s - 1698.54s | managers is gonna be like hey so your data dog bills kind of high can you cut 1698.54s - 1701.78s | it back you're like no I need all of this need every single piece of data 1701.78s - 1707.78s | that comes in or I can't do my job okay that's that's a way but again we're 1707.78s - 1711.90s | talking about consumers we're not talking about business-to-business 1711.90s - 1719.86s | operations and so if I'm a consumer and and I have to worry about every single 1719.86s - 1724.22s | thing being stored on my behalf I start to get nervous and so what I like to do 1724.22s - 1731.10s | in these cases is require like a custom logging handler engineers are already 1731.10s - 1734.78s | creating these to make their lives easier all you have to do as the 1734.78s - 1739.10s | security person to say hey so right now it looks like you're taking everything 1739.10s - 1742.06s | you're formatting it and


chunk_text:
know you're 1690.54s - 1694.86s | like I might need this someday ship it to data doc and then one of your SRE 1694.86s - 1698.54s | managers is gonna be like hey so your data dog bills kind of high can you cut 1698.54s - 1701.78s | it back you're like no I need all of this need every single piece of data 1701.78s - 1707.78s | that comes in or I can't do my job okay that's that's a way but again we're 1707.78s - 1711.90s | talking about consumers we're not talking about business-to-business 1711.90s - 1719.86s | operations and so if I'm a consumer and and I have to worry about every single 1719.86s - 1724.22s | thing being stored on my behalf I start to get nervous and so what I like to do 1724.22s - 1731.10s | in these cases is require like a custom logging handler engineers are already 1731.10s - 1734.78s | creating these to make their lives easier all you have to do as the 1734.78s - 1739.10s | security person to say hey so right now it looks like you're taking everything 1739.10s - 1742.06s | you're formatting it and then you're you're shipping it out what I would love 1742.06s - 1746.94s | for you to do is instead explicitly declare these are the the fields that 1746.94s - 1751.26s | we're going to log and by forcing them to explicitly declare the fields that 1751.26s - 1755.82s | they're going to log you don't have to worry about oh I just updated the code 1755.82s - 1761.90s | to log this extra field also it's everyone's birthday and social security 1761.90s - 1765.62s | number that's okay right you don't have to worry about that because the logger 1765.62s - 1770.34s | won't allow it and now instead of monitoring your entire code base for a 1770.34s - 1775.66s | change that might log something in a wrong way you only have to monitor that 1775.66s - 1782.38s | that base privacy logger so it's definitely worth investment this code is 1782.38s - 1789.10s | from chachi PT write me a privacy focused log handler in Python it might 1789.10s - 1794.06s | work I don't I don't know but the the point is it's like it's not super 1794.06s - 1801.38s | difficult and your engineers are


chunk_text:
then you're you're shipping it out what I would love 1742.06s - 1746.94s | for you to do is instead explicitly declare these are the the fields that 1746.94s - 1751.26s | we're going to log and by forcing them to explicitly declare the fields that 1751.26s - 1755.82s | they're going to log you don't have to worry about oh I just updated the code 1755.82s - 1761.90s | to log this extra field also it's everyone's birthday and social security 1761.90s - 1765.62s | number that's okay right you don't have to worry about that because the logger 1765.62s - 1770.34s | won't allow it and now instead of monitoring your entire code base for a 1770.34s - 1775.66s | change that might log something in a wrong way you only have to monitor that 1775.66s - 1782.38s | that base privacy logger so it's definitely worth investment this code is 1782.38s - 1789.10s | from chachi PT write me a privacy focused log handler in Python it might 1789.10s - 1794.06s | work I don't I don't know but the the point is it's like it's not super 1794.06s - 1801.38s | difficult and your engineers are likely already doing it anyway so some some 1801.38s - 1809.06s | highlights on privacy focus logging prefer metrics over like actual counts 1809.06s - 1812.86s | and group by statements whenever you're thinking about things you don't need to 1812.86s - 1817.18s | log every single request in order to know how many requests happened right 1817.18s - 1823.94s | and so if you don't need to store the data don't a thing that I've been big 1823.94s - 1830.66s | about for a very long time has been the fact that business metrics and that type 1830.66s - 1835.18s | of work needs to be a feature request of the product this is not something 1835.18s - 1839.06s | where oh I've got a really strong program manager they're gonna go into 1839.06s - 1842.66s | data dog and create these dashboards that are gonna tell them like what our 1842.66s - 1847.42s | most popular feature is it's a very very easy shortcut but it's also a very very 1847.42s - 1852.34s | easy easy way to lose control of the way that you're managing the privacy of your 1852.34s - 1859.10s | customer data and so


chunk_text:
likely already doing it anyway so some some 1801.38s - 1809.06s | highlights on privacy focus logging prefer metrics over like actual counts 1809.06s - 1812.86s | and group by statements whenever you're thinking about things you don't need to 1812.86s - 1817.18s | log every single request in order to know how many requests happened right 1817.18s - 1823.94s | and so if you don't need to store the data don't a thing that I've been big 1823.94s - 1830.66s | about for a very long time has been the fact that business metrics and that type 1830.66s - 1835.18s | of work needs to be a feature request of the product this is not something 1835.18s - 1839.06s | where oh I've got a really strong program manager they're gonna go into 1839.06s - 1842.66s | data dog and create these dashboards that are gonna tell them like what our 1842.66s - 1847.42s | most popular feature is it's a very very easy shortcut but it's also a very very 1847.42s - 1852.34s | easy easy way to lose control of the way that you're managing the privacy of your 1852.34s - 1859.10s | customer data and so rather than getting into that when we do a like a product 1859.10s - 1865.62s | design doc right let's explicitly call out I need these metrics and now it's 1865.62s - 1868.70s | part of the product you don't have to go look through logs because you've already 1868.70s - 1874.14s | decided I'm going to use counters instead engineering logs for engineering 1874.14s - 1880.66s | things we talked about using custom logging classes anonymize where possible 1880.66s - 1887.82s | sudo anonymize everywhere else so I might need to know which user I'm 1887.82s - 1891.98s | dealing with whenever I'm solving an engineering challenge I don't need to 1891.98s - 1897.70s | know that users email I don't need to know that users address I don't need to 1897.70s - 1903.66s | know any of those things I just need to be to be able to say user 1 2 3 of a 1903.66s - 1908.26s | thousand and so this is where when we do logging rather than logging all of the 1908.26s - 1912.54s | information we have about that that customer let's just log a sudo 1912.54s - 1917.14s | anonymized user ID


chunk_text:
rather than getting into that when we do a like a product 1859.10s - 1865.62s | design doc right let's explicitly call out I need these metrics and now it's 1865.62s - 1868.70s | part of the product you don't have to go look through logs because you've already 1868.70s - 1874.14s | decided I'm going to use counters instead engineering logs for engineering 1874.14s - 1880.66s | things we talked about using custom logging classes anonymize where possible 1880.66s - 1887.82s | sudo anonymize everywhere else so I might need to know which user I'm 1887.82s - 1891.98s | dealing with whenever I'm solving an engineering challenge I don't need to 1891.98s - 1897.70s | know that users email I don't need to know that users address I don't need to 1897.70s - 1903.66s | know any of those things I just need to be to be able to say user 1 2 3 of a 1903.66s - 1908.26s | thousand and so this is where when we do logging rather than logging all of the 1908.26s - 1912.54s | information we have about that that customer let's just log a sudo 1912.54s - 1917.14s | anonymized user ID it doesn't provide access to anything but it lets us sort 1917.14s - 1923.54s | of work through the streams and track down when bugs happen set an extremely 1923.54s - 1928.10s | short time limit for your your engineering logs if you need something 1928.10s - 1932.54s | for longer than a week two weeks that should probably live somewhere else 1932.54s - 1938.10s | probably doesn't need to be in Datadog your CFO is gonna thank you and so are 1938.10s - 1941.26s | your customers because you're not storing their personal data somewhere 1941.26s - 1948.14s | that you don't need to enforce just-in-time logging so I think one of 1948.14s - 1953.50s | the coolest and scariest things about joining an engineering team at Facebook 1953.50s - 1957.50s | is like the first two days when they're like oh yeah and by the way if you look 1957.50s - 1961.30s | at anyone that you work with's data we're gonna fire you and I'm just like I 1961.30s - 1964.30s | don't even want to accidentally do that you're paying me really well I get free 1964.30s - 1970.92s | pizza like why would I mess


chunk_text:
it doesn't provide access to anything but it lets us sort 1917.14s - 1923.54s | of work through the streams and track down when bugs happen set an extremely 1923.54s - 1928.10s | short time limit for your your engineering logs if you need something 1928.10s - 1932.54s | for longer than a week two weeks that should probably live somewhere else 1932.54s - 1938.10s | probably doesn't need to be in Datadog your CFO is gonna thank you and so are 1938.10s - 1941.26s | your customers because you're not storing their personal data somewhere 1941.26s - 1948.14s | that you don't need to enforce just-in-time logging so I think one of 1948.14s - 1953.50s | the coolest and scariest things about joining an engineering team at Facebook 1953.50s - 1957.50s | is like the first two days when they're like oh yeah and by the way if you look 1957.50s - 1961.30s | at anyone that you work with's data we're gonna fire you and I'm just like I 1961.30s - 1964.30s | don't even want to accidentally do that you're paying me really well I get free 1964.30s - 1970.92s | pizza like why would I mess that up and the way that they enforce that is if you 1970.92s - 1975.06s | need access to something and you don't have it by default you just go find the 1975.06s - 1980.12s | table and say I would like access to this I need it for this long and this is 1980.12s - 1985.96s | my business purpose behind it and as long as it's not like your ex-partners 1985.96s - 1994.36s | page it'll just grant you access and log it and I feel like it's way better to 1994.36s - 1998.76s | have that then oh yeah just everybody in engineering has access to everything all 1998.88s - 2003.44s | the time so that that just-in-time access gives us again a little bit more 2003.44s - 2007.84s | safety and a little bit better of a way to recognize when something's going off 2007.84s - 2013.44s | the rails all of the do's and don'ts that we talked about in in the other two 2013.44s - 2020.68s | sort of apply to privacy focused logging as well all right the last one privacy 2020.68s - 2026.32s | focused analytics so if I'm got a feature


chunk_text:
that up and the way that they enforce that is if you 1970.92s - 1975.06s | need access to something and you don't have it by default you just go find the 1975.06s - 1980.12s | table and say I would like access to this I need it for this long and this is 1980.12s - 1985.96s | my business purpose behind it and as long as it's not like your ex-partners 1985.96s - 1994.36s | page it'll just grant you access and log it and I feel like it's way better to 1994.36s - 1998.76s | have that then oh yeah just everybody in engineering has access to everything all 1998.88s - 2003.44s | the time so that that just-in-time access gives us again a little bit more 2003.44s - 2007.84s | safety and a little bit better of a way to recognize when something's going off 2007.84s - 2013.44s | the rails all of the do's and don'ts that we talked about in in the other two 2013.44s - 2020.68s | sort of apply to privacy focused logging as well all right the last one privacy 2020.68s - 2026.32s | focused analytics so if I'm got a feature that's out there it's doing 2026.32s - 2029.88s | great things but I want to make it faster but I don't understand what 2029.88s - 2034.48s | requests are happening and so I need to like actually see these requests in 2034.48s - 2041.32s | order to make the product work better well 18 year old Matt Domko goes into 2041.32s - 2046.96s | the routing engine and he just adds a new branch of code for 2a that says yeah 2046.96s - 2051.32s | so anytime somebody makes a request I want you to go do the thing that you 2051.32s - 2057.12s | always do but also lock a copy of it to my analytics storage and then once that 2057.12s - 2060.84s | table is super full or that bucket has all the data I need I can go in and do 2060.84s - 2065.88s | some sort of analytics process on it the problem with that is in order to solve 2065.88s - 2070.88s | whatever challenge I'm facing again I probably don't need to know the 2070.88s - 2075.72s | customers name I don't need to know all of the things that they did I


chunk_text:
that's out there it's doing 2026.32s - 2029.88s | great things but I want to make it faster but I don't understand what 2029.88s - 2034.48s | requests are happening and so I need to like actually see these requests in 2034.48s - 2041.32s | order to make the product work better well 18 year old Matt Domko goes into 2041.32s - 2046.96s | the routing engine and he just adds a new branch of code for 2a that says yeah 2046.96s - 2051.32s | so anytime somebody makes a request I want you to go do the thing that you 2051.32s - 2057.12s | always do but also lock a copy of it to my analytics storage and then once that 2057.12s - 2060.84s | table is super full or that bucket has all the data I need I can go in and do 2060.84s - 2065.88s | some sort of analytics process on it the problem with that is in order to solve 2065.88s - 2070.88s | whatever challenge I'm facing again I probably don't need to know the 2070.88s - 2075.72s | customers name I don't need to know all of the things that they did I probably 2075.72s - 2081.96s | just need to know what's the length of their request and what time was it when 2081.96s - 2086.72s | they made the request well whatever the thing is that I need it's probably not 2086.72s - 2094.88s | everything so the way that the way that I've thought about handling this we have 2094.88s - 2104.36s | our our new analytics logger and again we still insert it into 2a but instead 2104.36s - 2108.80s | of having it directly write data outside of my little nice fancy 2108.80s - 2112.52s | production bubble where data never leaves I read it to a privacy filtering 2112.52s - 2117.08s | account and so that privacy filtering account whenever it receives new data it 2117.08s - 2122.24s | reaches in and it says oh I'm expecting that Matt needs to know the locations of 2122.24s - 2130.08s | every failed uber request and so what that privacy filtering lambda will do is 2130.08s - 2136.84s | it will just extract the location and the time that that it happened because I 2136.84s - 2141.44s | don't need to know anything else about that conversation or what's


chunk_text:
probably 2075.72s - 2081.96s | just need to know what's the length of their request and what time was it when 2081.96s - 2086.72s | they made the request well whatever the thing is that I need it's probably not 2086.72s - 2094.88s | everything so the way that the way that I've thought about handling this we have 2094.88s - 2104.36s | our our new analytics logger and again we still insert it into 2a but instead 2104.36s - 2108.80s | of having it directly write data outside of my little nice fancy 2108.80s - 2112.52s | production bubble where data never leaves I read it to a privacy filtering 2112.52s - 2117.08s | account and so that privacy filtering account whenever it receives new data it 2117.08s - 2122.24s | reaches in and it says oh I'm expecting that Matt needs to know the locations of 2122.24s - 2130.08s | every failed uber request and so what that privacy filtering lambda will do is 2130.08s - 2136.84s | it will just extract the location and the time that that it happened because I 2136.84s - 2141.44s | don't need to know anything else about that conversation or what's going on 2141.44s - 2146.48s | except for that one single line so why would I allow myself to copy the entire 2146.48s - 2151.56s | thing out so I create this process where in order to get data out of production I 2151.56s - 2155.32s | ensure that it's anonymized I ensure that I only have the data that I need 2155.32s - 2161.12s | once that data has been cleaned we've ran it through something like Amazon 2161.12s - 2165.92s | Macy to check for PII and things like that then we'll go ahead and copy it 2165.92s - 2171.68s | from the privacy account into the analytics account analytics is already 2171.68s - 2177.28s | expensive and so like this additional process that we're introducing in order 2177.28s - 2188.83s | to protect our customers is totally worth it and so we've got a summary of 2188.83s - 2194.51s | the the things that that have made my life easier in protecting customers 2194.51s - 2201.99s | privacy the first thing simple data classification guide anyone that was 2201.99s - 2205.15s | ever in the military they've got like those colors and when you see colors 2205.15s


chunk_text:
going on 2141.44s - 2146.48s | except for that one single line so why would I allow myself to copy the entire 2146.48s - 2151.56s | thing out so I create this process where in order to get data out of production I 2151.56s - 2155.32s | ensure that it's anonymized I ensure that I only have the data that I need 2155.32s - 2161.12s | once that data has been cleaned we've ran it through something like Amazon 2161.12s - 2165.92s | Macy to check for PII and things like that then we'll go ahead and copy it 2165.92s - 2171.68s | from the privacy account into the analytics account analytics is already 2171.68s - 2177.28s | expensive and so like this additional process that we're introducing in order 2177.28s - 2188.83s | to protect our customers is totally worth it and so we've got a summary of 2188.83s - 2194.51s | the the things that that have made my life easier in protecting customers 2194.51s - 2201.99s | privacy the first thing simple data classification guide anyone that was 2201.99s - 2205.15s | ever in the military they've got like those colors and when you see colors 2205.15s - 2208.79s | you're like oh this is sensitive I shouldn't do anything wrong with this oh 2208.79s - 2214.27s | this is green I think I can do whatever I want with this but let me check having 2214.27s - 2219.75s | some sort of simple guide where your engineers know if data is labeled as red 2219.75s - 2224.95s | that means it can't leave leave prod if data is labeled as yellow it means it 2224.95s - 2230.79s | can go from prod into the privacy account into an analytics account if 2230.79s - 2236.99s | data is labeled as green it's on the website so who cares but coming up with 2236.99s - 2242.27s | those shortcuts for your engineers it's gonna make your life a lot easier build 2242.27s - 2246.59s | that culture of design reviews show up to them I know that our engineers build 2246.59s - 2251.55s | a lot of really cool stuff all the time and that means I'm gonna be at more 2251.55s - 2255.55s | meetings but again it's worth it you're learning more about the product you you 2255.55s - 2259.11s | have a chance to actually


chunk_text:
- 2208.79s | you're like oh this is sensitive I shouldn't do anything wrong with this oh 2208.79s - 2214.27s | this is green I think I can do whatever I want with this but let me check having 2214.27s - 2219.75s | some sort of simple guide where your engineers know if data is labeled as red 2219.75s - 2224.95s | that means it can't leave leave prod if data is labeled as yellow it means it 2224.95s - 2230.79s | can go from prod into the privacy account into an analytics account if 2230.79s - 2236.99s | data is labeled as green it's on the website so who cares but coming up with 2236.99s - 2242.27s | those shortcuts for your engineers it's gonna make your life a lot easier build 2242.27s - 2246.59s | that culture of design reviews show up to them I know that our engineers build 2246.59s - 2251.55s | a lot of really cool stuff all the time and that means I'm gonna be at more 2251.55s - 2255.55s | meetings but again it's worth it you're learning more about the product you you 2255.55s - 2259.11s | have a chance to actually step in and say well what is this data and why do 2259.11s - 2265.51s | you actually need to store it or move it or read it so build that culture show up 2265.51s - 2272.47s | to those things service control policies so these exist I know in AWS 2272.47s - 2276.47s | and Azure does GCP have something like this where you can say you're not allowed 2276.47s - 2282.07s | to break this rule no matter what I don't care who you are yeah okay and so 2282.07s - 2285.79s | set that up if you have a production account where all of your day-to-day 2285.79s - 2290.27s | data exists make it so you can't mess it up no data is allowed to leave 2290.27s - 2296.19s | production except for to go to this singular privacy account it works fairly 2296.19s - 2299.87s | well and it keeps you out of trouble when you're traveling or at a conference 2299.87s - 2305.91s | talking about privacy the two-step filtering process that we talked about I 2305.91s - 2310.35s | feel like that's the only way that we can truly ensure we're not accidentally 2310.35s


chunk_text:
step in and say well what is this data and why do 2259.11s - 2265.51s | you actually need to store it or move it or read it so build that culture show up 2265.51s - 2272.47s | to those things service control policies so these exist I know in AWS 2272.47s - 2276.47s | and Azure does GCP have something like this where you can say you're not allowed 2276.47s - 2282.07s | to break this rule no matter what I don't care who you are yeah okay and so 2282.07s - 2285.79s | set that up if you have a production account where all of your day-to-day 2285.79s - 2290.27s | data exists make it so you can't mess it up no data is allowed to leave 2290.27s - 2296.19s | production except for to go to this singular privacy account it works fairly 2296.19s - 2299.87s | well and it keeps you out of trouble when you're traveling or at a conference 2299.87s - 2305.91s | talking about privacy the two-step filtering process that we talked about I 2305.91s - 2310.35s | feel like that's the only way that we can truly ensure we're not accidentally 2310.35s - 2315.55s | moving data somewhere we shouldn't and so think about how to implement something 2315.55s - 2319.91s | like that there's tons of services and all the different cloud providers that 2319.91s - 2326.27s | will parse your data and tell you yes or no this can this can go or this yes 2326.27s - 2334.11s | or no this has sensitive data in it I just learned about s3 bucket filtering 2334.11s - 2341.83s | that I don't remember the exact name but anyway you can create a new s3 access 2341.83s - 2346.43s | point we'll go with that you can create a new s3 access point and say through 2346.43s - 2351.71s | this access point you can only read filtered data which kind of takes your 2351.71s - 2354.75s | hands away from having to do anything as long as you only let your developers 2354.75s - 2365.46s | read through that singular access point oh yes almost done so a couple of 2365.46s - 2369.34s | guidelines that that I've set with the teams that I've worked with in the past 2369.34s - 2374.90s | this is this is the one that you


chunk_text:
- 2315.55s | moving data somewhere we shouldn't and so think about how to implement something 2315.55s - 2319.91s | like that there's tons of services and all the different cloud providers that 2319.91s - 2326.27s | will parse your data and tell you yes or no this can this can go or this yes 2326.27s - 2334.11s | or no this has sensitive data in it I just learned about s3 bucket filtering 2334.11s - 2341.83s | that I don't remember the exact name but anyway you can create a new s3 access 2341.83s - 2346.43s | point we'll go with that you can create a new s3 access point and say through 2346.43s - 2351.71s | this access point you can only read filtered data which kind of takes your 2351.71s - 2354.75s | hands away from having to do anything as long as you only let your developers 2354.75s - 2365.46s | read through that singular access point oh yes almost done so a couple of 2365.46s - 2369.34s | guidelines that that I've set with the teams that I've worked with in the past 2369.34s - 2374.90s | this is this is the one that you screenshot and and send to everyone and 2374.90s - 2380.66s | it's really just four questions are you logging a new type of data we should 2380.66s - 2383.98s | talk are you moving data outside abroad we 2383.98s - 2389.46s | should talk are you connecting to broad databases we should talk are you 2389.46s - 2395.18s | bringing in a new tool or vendor we should talk I think like those are 2395.18s - 2401.06s | fairly easy to do in practice and it's worth the extra time that that we have 2401.06s - 2411.98s | to invest that's it what time is it 442 I've time for questions if anybody has 2411.98s - 2420.15s | any if not I'm gonna get off the stage and chug a bunch of water Oh contact 2420.15s - 2428.35s | page so hashtag cyber on the Twitter and if like you're worried about your 2428.35s - 2443.74s | rabbit r1 security at rabbit tech don't that guy right there cool well thank 2443.74s - 2447.14s | you everybody have a great time


chunk_text:
screenshot and and send to everyone and 2374.90s - 2380.66s | it's really just four questions are you logging a new type of data we should 2380.66s - 2383.98s | talk are you moving data outside abroad we 2383.98s - 2389.46s | should talk are you connecting to broad databases we should talk are you 2389.46s - 2395.18s | bringing in a new tool or vendor we should talk I think like those are 2395.18s - 2401.06s | fairly easy to do in practice and it's worth the extra time that that we have 2401.06s - 2411.98s | to invest that's it what time is it 442 I've time for questions if anybody has 2411.98s - 2420.15s | any if not I'm gonna get off the stage and chug a bunch of water Oh contact 2420.15s - 2428.35s | page so hashtag cyber on the Twitter and if like you're worried about your 2428.35s - 2443.74s | rabbit r1 security at rabbit tech don't that guy right there cool well thank 2443.74s - 2447.14s | you everybody have a great time


chunk_text:
{ "webpage_url": "https://www.youtube.com/watch?v=fs0QR7fPnJg", "title": "DEF CON 32 - D0N0H4RM Cyber STEM Storytime - Panel", "description": "All-star panel discussing Cyber STEM", "channel_url": "https://www.youtube.com/channel/UC6Om9kAkl32dWlDSNlDS9Iw", "duration": 3261, "channel": "DEFCONConference", "uploader": "DEFCONConference", "upload_date": "20241016" } 0.00s - 9.92s | This text was transcribed using whisper model: large-v2 All right, Defcon. How tired are you of people asking you how you're doing? Yeah? Cheer really 9.92s - 17.23s | loudly if you're really tired of that. The other thing I'm trying to do is get people 17.23s - 24.71s | out there to realize how good a time you guys are having in here. So I'm going to say, um, 24.71s - 30.23s | no one doesn't like cake, right? I mean, cake is universally. Can we all cheer for cake? 30.23s - 38.95s | Yeah! Everyone out there, come in here. All right, folks. Welcome to, uh, another creator 38.95s - 44.79s | stage talk. We have some killer folks here to talk to you about some very pressing issues. 44.79s - 48.39s | And I know we started out with cake, which is going to be the high mark. We're going 48.39s - 52.11s | to be talking about healthcare, cybersecurity, medical devices, whistleblowing, and a bunch 52.11s - 56.43s | of other really cool stuff. In some sense, it can be a little bit of a downer. So I don't 56.43s - 59.51s | have any trigger warnings or anything for our panel, but the only thing I'll ask is 59.51s - 62.87s | for the audience, uh, we'll have an opportunity for a little Q and a at the end, unless we 62.87s - 68.23s | go really over, uh, but just be respectful of the community and other folks. Uh, because 68.23s - 72.19s | sometimes this can be a really personal thing as we're going to talk about. I think the 72.19s - 76.75s | first thing we're going to do is actually go down the line and do introductions. And 76.75s - 80.35s | before I do that, can you guys hear us? Okay. I know we're competing over here. You guys 80.35s - 84.27s | are good. All right. So let's go ahead and, uh, let's go right down the line, Matt on 84.27s - 89.23s | and go to introduce yourself, tell the crowd what you do. And, uh, we'll get going on to 89.23s -


chunk_text:
whistleblowing, and a bunch 52.11s - 56.43s | of other really cool stuff. In some sense, it can be a little bit of a downer. So I don't 56.43s - 59.51s | have any trigger warnings or anything for our panel, but the only thing I'll ask is 59.51s - 62.87s | for the audience, uh, we'll have an opportunity for a little Q and a at the end, unless we 62.87s - 68.23s | go really over, uh, but just be respectful of the community and other folks. Uh, because 68.23s - 72.19s | sometimes this can be a really personal thing as we're going to talk about. I think the 72.19s - 76.75s | first thing we're going to do is actually go down the line and do introductions. And 76.75s - 80.35s | before I do that, can you guys hear us? Okay. I know we're competing over here. You guys 80.35s - 84.27s | are good. All right. So let's go ahead and, uh, let's go right down the line, Matt on 84.27s - 89.23s | and go to introduce yourself, tell the crowd what you do. And, uh, we'll get going on to 89.23s - 94.87s | the real good questions. All right. After you, sounds good. Um, I'm Matt Hazlett. I'm 94.87s - 102.27s | the chief regulatory officer at MedSec currently. I most recently left the FDA a couple of weeks 102.27s - 108.23s | ago. Um, I was at the FDA for eight and a half years. Um, started getting very involved 108.23s - 115.51s | in the cybersecurity policy, um, and implementation efforts. So I was very involved in the pre-market 115.51s - 122.59s | cybersecurity guidance, writing a significant portion of that, um, and overseer oversaw 122.59s - 128.31s | our implementation efforts. So the reviewer resources training of how we were going to 128.31s - 131.83s | review cybersecurity, given our new statutory authorities, 131.83s - 143.90s | Hi everyone. I am Erica Chung. I'm currently the executive director of a nonprofit called 143.90s - 149.46s | ethics and entrepreneurship. Uh, my interest in the safety of medical devices started 149.46s - 156.62s | off at my very first job. When I was 22 years old, I was a lab associate working for a biomedical 156.62s - 162.06s | company called Theranos, which ended up being a $9 billion


chunk_text:
94.87s | the real good questions. All right. After you, sounds good. Um, I'm Matt Hazlett. I'm 94.87s - 102.27s | the chief regulatory officer at MedSec currently. I most recently left the FDA a couple of weeks 102.27s - 108.23s | ago. Um, I was at the FDA for eight and a half years. Um, started getting very involved 108.23s - 115.51s | in the cybersecurity policy, um, and implementation efforts. So I was very involved in the pre-market 115.51s - 122.59s | cybersecurity guidance, writing a significant portion of that, um, and overseer oversaw 122.59s - 128.31s | our implementation efforts. So the reviewer resources training of how we were going to 128.31s - 131.83s | review cybersecurity, given our new statutory authorities, 131.83s - 143.90s | Hi everyone. I am Erica Chung. I'm currently the executive director of a nonprofit called 143.90s - 149.46s | ethics and entrepreneurship. Uh, my interest in the safety of medical devices started 149.46s - 156.62s | off at my very first job. When I was 22 years old, I was a lab associate working for a biomedical 156.62s - 162.06s | company called Theranos, which ended up being a $9 billion fraud scandal, which I was one 162.06s - 167.14s | of the key whistleblowers who reported the company to regulators that stopped them from 167.14s - 175.94s | utilizing a faulty device on patients. And, uh, I am also involved, uh, as a board member 175.94s - 180.86s | of whistleblowers of America, which we provide mental health services and peer support for 180.86s - 186.78s | whistleblowers across the United States. And also a advisor to the signals network, which 186.78s - 194.54s | is a agency that helps whistleblowers with legal advice, uh, safe housing, uh, funding 194.54s - 200.14s | job placement in the event that, um, they are in need of those services. 200.14s - 202.34s | Give it up for Erica. 202.34s - 210.18s | Hey everybody. Um, I'm Andrew Carney. Uh, I'm a program manager at a relatively new 210.18s - 216.82s | federal agency ARPA-H where I work on healthcare cybersecurity, uh, patient data privacy. Um, 216.82s - 221.22s | I'm also the program manager for the AI cyber challenge at DARPA. Uh, so if you haven't 221.22s - 225.90s | already, please come visit us next door, uh, and spend some time


chunk_text:
fraud scandal, which I was one 162.06s - 167.14s | of the key whistleblowers who reported the company to regulators that stopped them from 167.14s - 175.94s | utilizing a faulty device on patients. And, uh, I am also involved, uh, as a board member 175.94s - 180.86s | of whistleblowers of America, which we provide mental health services and peer support for 180.86s - 186.78s | whistleblowers across the United States. And also a advisor to the signals network, which 186.78s - 194.54s | is a agency that helps whistleblowers with legal advice, uh, safe housing, uh, funding 194.54s - 200.14s | job placement in the event that, um, they are in need of those services. 200.14s - 202.34s | Give it up for Erica. 202.34s - 210.18s | Hey everybody. Um, I'm Andrew Carney. Uh, I'm a program manager at a relatively new 210.18s - 216.82s | federal agency ARPA-H where I work on healthcare cybersecurity, uh, patient data privacy. Um, 216.82s - 221.22s | I'm also the program manager for the AI cyber challenge at DARPA. Uh, so if you haven't 221.22s - 225.90s | already, please come visit us next door, uh, and spend some time there, have some fun. 225.90s - 231.10s | And before all of that, um, I spent most of the last 15 years as a vulnerability researcher 231.10s - 239.58s | and a painful CTF player. Um, and I am now semi-retired from that fabulous pastime. Um, 239.58s - 243.50s | please go check out DEF CON CTF as well. If you haven't, the folks put on an amazing game. 243.50s - 249.54s | Um, and I'm super excited to be, uh, so the last year, uh, plus in healthcare, uh, looking 249.54s - 253.94s | at the health cybersecurity of healthcare and, um, coming from a very offensive kind 253.94s - 259.14s | of mindset, uh, just seeing what we can, seeing what we can do from a defensive kind of position 259.14s - 263.74s | and, and really, um, hoping to supercharge kind of our efforts to safeguard, uh, this, 263.74s - 264.74s | this sector. 264.74s - 265.74s | Give it up, Andrew. 265.74s - 277.38s | I'm the deputy director of CISA. Hopefully you've heard of us, some ways you've performed. 277.38s - 282.54s | Um, and, uh, my earlier in my career, I was actually a


chunk_text:
there, have some fun. 225.90s - 231.10s | And before all of that, um, I spent most of the last 15 years as a vulnerability researcher 231.10s - 239.58s | and a painful CTF player. Um, and I am now semi-retired from that fabulous pastime. Um, 239.58s - 243.50s | please go check out DEF CON CTF as well. If you haven't, the folks put on an amazing game. 243.50s - 249.54s | Um, and I'm super excited to be, uh, so the last year, uh, plus in healthcare, uh, looking 249.54s - 253.94s | at the health cybersecurity of healthcare and, um, coming from a very offensive kind 253.94s - 259.14s | of mindset, uh, just seeing what we can, seeing what we can do from a defensive kind of position 259.14s - 263.74s | and, and really, um, hoping to supercharge kind of our efforts to safeguard, uh, this, 263.74s - 264.74s | this sector. 264.74s - 265.74s | Give it up, Andrew. 265.74s - 277.38s | I'm the deputy director of CISA. Hopefully you've heard of us, some ways you've performed. 277.38s - 282.54s | Um, and, uh, my earlier in my career, I was actually a hospital administrator and a flight 282.54s - 286.94s | medic. So I started my career in healthcare. So it's great to be here today to merge cybersecurity 286.94s - 290.82s | and healthcare together and talk about what we can all do, uh, to continue to make the 290.82s - 297.26s | sector more safe. Thanks. 297.26s - 301.02s | All right. So we're gonna, yeah, this is give it up for the whole panel. Try to beat them 301.02s - 309.30s | out next door. All right. I think it's important to level set a little bit, but I want to take 309.30s - 313.54s | a poll. The audience real quick, raise your hand. If you work in healthcare, some adjacent 313.54s - 318.30s | field, raise your hand. There's no one in this audience. This is all brand new for you. 318.30s - 323.14s | I take it. Just kidding. It's like everyone, it's like my high school reunion out there. 323.14s - 327.38s | All right. I do think it's important for us to level set a little bit. And we're gonna 327.38s - 331.38s | talk about threat land landscape a bit. I have some,


chunk_text:
hospital administrator and a flight 282.54s - 286.94s | medic. So I started my career in healthcare. So it's great to be here today to merge cybersecurity 286.94s - 290.82s | and healthcare together and talk about what we can all do, uh, to continue to make the 290.82s - 297.26s | sector more safe. Thanks. 297.26s - 301.02s | All right. So we're gonna, yeah, this is give it up for the whole panel. Try to beat them 301.02s - 309.30s | out next door. All right. I think it's important to level set a little bit, but I want to take 309.30s - 313.54s | a poll. The audience real quick, raise your hand. If you work in healthcare, some adjacent 313.54s - 318.30s | field, raise your hand. There's no one in this audience. This is all brand new for you. 318.30s - 323.14s | I take it. Just kidding. It's like everyone, it's like my high school reunion out there. 323.14s - 327.38s | All right. I do think it's important for us to level set a little bit. And we're gonna 327.38s - 331.38s | talk about threat land landscape a bit. I have some, a little bit of a couple of slides 331.38s - 337.18s | at the end, but a deputy director, you could tell us a little bit more than about the threat 337.18s - 343.58s | landscape and what you're seeing kind of nationally about healthcare ransomware because, and just 343.58s - 347.86s | healthcare threats, generally speaking, just give us a little bit of a one-on-one and what 347.86s - 348.86s | do you see? 348.86s - 352.42s | Thanks for sure. I think the big thing we're trying to message out to folks as we look 352.42s - 356.90s | at the healthcare sector, we're seeing an increase in the volume frequency and complexity 356.90s - 361.70s | of attacks against the healthcare sector, not just in the United States, but globally. 361.70s - 367.04s | I think when we think about healthcare, if we go back to kinetic warfare going back centuries, 367.04s - 372.14s | we never attacked hospitals, right? We always protected healthcare providers. We always 372.14s - 376.42s | protected the tents with the red cross on it. We never attacked those institutions, 376.42s - 380.50s | but today we're seeing attacks against healthcare organizations


chunk_text:
a little bit of a couple of slides 331.38s - 337.18s | at the end, but a deputy director, you could tell us a little bit more than about the threat 337.18s - 343.58s | landscape and what you're seeing kind of nationally about healthcare ransomware because, and just 343.58s - 347.86s | healthcare threats, generally speaking, just give us a little bit of a one-on-one and what 347.86s - 348.86s | do you see? 348.86s - 352.42s | Thanks for sure. I think the big thing we're trying to message out to folks as we look 352.42s - 356.90s | at the healthcare sector, we're seeing an increase in the volume frequency and complexity 356.90s - 361.70s | of attacks against the healthcare sector, not just in the United States, but globally. 361.70s - 367.04s | I think when we think about healthcare, if we go back to kinetic warfare going back centuries, 367.04s - 372.14s | we never attacked hospitals, right? We always protected healthcare providers. We always 372.14s - 376.42s | protected the tents with the red cross on it. We never attacked those institutions, 376.42s - 380.50s | but today we're seeing attacks against healthcare organizations throughout the country. This 380.50s - 384.66s | is not just about urban America. We're seeing attacks against rural America as well, where 384.66s - 389.14s | arguably healthcare systems are more critical in those communities, given the access to 389.14s - 393.34s | care and the access to backup care is much further away and the consequence of patient 393.34s - 398.50s | safety is even greater. As we look at a lot of, everyone has published a top three or 398.50s - 404.98s | a top five list of ransomware attacks or sector impacts. Healthcare continues to be in that 404.98s - 409.78s | top three and top five list of every list that I see as sectors that are being attacked 409.78s - 413.34s | by our adversaries. And we're not just talking about nation states. We're not just talking 413.34s - 417.30s | about China, Russia, Iran, North Korea. We're seeing cyber terrorists and cyber criminal 417.30s - 422.38s | organizations also attacking healthcare. Why they foresee money there. They know that, 422.38s - 427.54s | I think healthcare facilities are kind of one-stop shops. You have access to PII, you 427.54s - 430.94s | have


chunk_text:
throughout the country. This 380.50s - 384.66s | is not just about urban America. We're seeing attacks against rural America as well, where 384.66s - 389.14s | arguably healthcare systems are more critical in those communities, given the access to 389.14s - 393.34s | care and the access to backup care is much further away and the consequence of patient 393.34s - 398.50s | safety is even greater. As we look at a lot of, everyone has published a top three or 398.50s - 404.98s | a top five list of ransomware attacks or sector impacts. Healthcare continues to be in that 404.98s - 409.78s | top three and top five list of every list that I see as sectors that are being attacked 409.78s - 413.34s | by our adversaries. And we're not just talking about nation states. We're not just talking 413.34s - 417.30s | about China, Russia, Iran, North Korea. We're seeing cyber terrorists and cyber criminal 417.30s - 422.38s | organizations also attacking healthcare. Why they foresee money there. They know that, 422.38s - 427.54s | I think healthcare facilities are kind of one-stop shops. You have access to PII, you 427.54s - 430.94s | have access to billing information, you have patient information, you have information 430.94s - 435.06s | that essentially people don't want out in the public space that you can get a hold of. 435.06s - 439.02s | It really is that one-stop shop and our adversaries know this. So we're continuing to see this 439.02s - 442.94s | increase in attacks against healthcare, which is why it's even more critical today to make 442.94s - 446.62s | sure that we're building resilience across the healthcare space. And when we talk about 446.62s - 452.34s | healthcare, it's not just hospitals, it's that entire spectrum that's needed for healthcare 452.34s - 458.58s | to be delivered. So it includes medical manufacturing, distribution of pharma, labs, blood, biotech, 458.58s - 464.10s | health IT, insurers, payers, HMOs, that entire spectrum that's needed for that clinical care 464.10s - 467.26s | to be delivered. And we want to make sure that we're continuing to build resilience 467.26s - 470.06s | across the board. 470.06s - 474.02s | That is awesome. And I have a couple slides to now take a little bit of another audience 474.02s - 481.02s | poll here. Raise your hand


chunk_text:
access to billing information, you have patient information, you have information 430.94s - 435.06s | that essentially people don't want out in the public space that you can get a hold of. 435.06s - 439.02s | It really is that one-stop shop and our adversaries know this. So we're continuing to see this 439.02s - 442.94s | increase in attacks against healthcare, which is why it's even more critical today to make 442.94s - 446.62s | sure that we're building resilience across the healthcare space. And when we talk about 446.62s - 452.34s | healthcare, it's not just hospitals, it's that entire spectrum that's needed for healthcare 452.34s - 458.58s | to be delivered. So it includes medical manufacturing, distribution of pharma, labs, blood, biotech, 458.58s - 464.10s | health IT, insurers, payers, HMOs, that entire spectrum that's needed for that clinical care 464.10s - 467.26s | to be delivered. And we want to make sure that we're continuing to build resilience 467.26s - 470.06s | across the board. 470.06s - 474.02s | That is awesome. And I have a couple slides to now take a little bit of another audience 474.02s - 481.02s | poll here. Raise your hand if you've ever had your medical records leak or be breached. 481.02s - 487.22s | Raise your hand. All right. That number is growing every time. I'm going to just take 487.22s - 491.82s | a gut pull here for anyone brave enough to raise their hand. Has anyone had their clinical 491.82s - 497.10s | care impacted by some type of cyber attack? Maybe your appointment was canceled. I'm seeing 497.10s - 501.74s | a couple of hands. Maybe you couldn't go to the hospital that was nearby. You heard 501.74s - 506.54s | about that? Now there's a couple hands. That's going to increase because what are we seeing 506.54s - 514.34s | now? You know, 2024 has been a banner year for healthcare ransomware. And we're seeing 514.34s - 519.26s | big attacks and some not ransomware things like CrowdStrike. But what I wanted to show 519.26s - 524.10s | you guys is for those of you who don't really work in this very often, now it's not just 524.10s - 531.58s | about privacy of data. It's not just about the financial threat. It's also about threats 531.58s - 540.78s | to clinical care and human


chunk_text:
if you've ever had your medical records leak or be breached. 481.02s - 487.22s | Raise your hand. All right. That number is growing every time. I'm going to just take 487.22s - 491.82s | a gut pull here for anyone brave enough to raise their hand. Has anyone had their clinical 491.82s - 497.10s | care impacted by some type of cyber attack? Maybe your appointment was canceled. I'm seeing 497.10s - 501.74s | a couple of hands. Maybe you couldn't go to the hospital that was nearby. You heard 501.74s - 506.54s | about that? Now there's a couple hands. That's going to increase because what are we seeing 506.54s - 514.34s | now? You know, 2024 has been a banner year for healthcare ransomware. And we're seeing 514.34s - 519.26s | big attacks and some not ransomware things like CrowdStrike. But what I wanted to show 519.26s - 524.10s | you guys is for those of you who don't really work in this very often, now it's not just 524.10s - 531.58s | about privacy of data. It's not just about the financial threat. It's also about threats 531.58s - 540.78s | to clinical care and human life. And so here's some social media I grabbed based on one of 540.78s - 545.18s | these attacks this year about people that were working. These are doctor and nursing 545.18s - 551.10s | forums on Reddit and other places where they basically talk about their experience working 551.10s - 556.38s | at some of these hospitals during a ransomware attack. And so in true DEF CON fashion it's 556.38s - 561.90s | stuff like largest hospital system in the country is under attack. No one knows where 561.90s - 569.22s | the forms are. We got a separate sign out. Nursing staff was trying to communicate what 569.22s - 573.74s | medications patients are on. Had to do stuff from memory. We were just told the internet 573.74s - 577.46s | may be out for days. We're still on a previous owner's network and they're down in multiple 577.46s - 581.58s | states. And then there's a nursing comment here. Vermont here, they never taught me how 581.58s - 588.86s | to use paper chart in nursing school. What about this one? We brought a trauma patient 588.86s - 593.46s | to an Ascension hospital and the trauma


chunk_text:
life. And so here's some social media I grabbed based on one of 540.78s - 545.18s | these attacks this year about people that were working. These are doctor and nursing 545.18s - 551.10s | forums on Reddit and other places where they basically talk about their experience working 551.10s - 556.38s | at some of these hospitals during a ransomware attack. And so in true DEF CON fashion it's 556.38s - 561.90s | stuff like largest hospital system in the country is under attack. No one knows where 561.90s - 569.22s | the forms are. We got a separate sign out. Nursing staff was trying to communicate what 569.22s - 573.74s | medications patients are on. Had to do stuff from memory. We were just told the internet 573.74s - 577.46s | may be out for days. We're still on a previous owner's network and they're down in multiple 577.46s - 581.58s | states. And then there's a nursing comment here. Vermont here, they never taught me how 581.58s - 588.86s | to use paper chart in nursing school. What about this one? We brought a trauma patient 588.86s - 593.46s | to an Ascension hospital and the trauma team was freaking out because half the team didn't 593.46s - 598.50s | get the page or notification. They're down doing paper charts, handwriting orders and 598.50s - 607.30s | doing consults over their personal phones. The top comment says they couldn't get Tylenol 607.30s - 618.54s | for a patient that needed it for two hours. We also learned a lot about critical connected 618.54s - 623.42s | infrastructure this last year with things like CrowdStrike where it wasn't even an attack 623.42s - 628.38s | but we saw huge amounts of hospitals go down as a consequence. And this is a couple social 628.38s - 633.82s | media grabs I pulled for the CrowdStrike issue. I swear to you I pulled this image on the 633.82s - 639.46s | left here right off of Reddit. So when the hospital went down, they took all their patients 639.46s - 643.14s | and had to write them on a white board where they were, what their name was and what the 643.14s - 650.98s | current issue was. On the same white board below they drew this picture. The ICU telemetry 650.98s - 656.40s | monitors are down. They said


chunk_text:
team was freaking out because half the team didn't 593.46s - 598.50s | get the page or notification. They're down doing paper charts, handwriting orders and 598.50s - 607.30s | doing consults over their personal phones. The top comment says they couldn't get Tylenol 607.30s - 618.54s | for a patient that needed it for two hours. We also learned a lot about critical connected 618.54s - 623.42s | infrastructure this last year with things like CrowdStrike where it wasn't even an attack 623.42s - 628.38s | but we saw huge amounts of hospitals go down as a consequence. And this is a couple social 628.38s - 633.82s | media grabs I pulled for the CrowdStrike issue. I swear to you I pulled this image on the 633.82s - 639.46s | left here right off of Reddit. So when the hospital went down, they took all their patients 639.46s - 643.14s | and had to write them on a white board where they were, what their name was and what the 643.14s - 650.98s | current issue was. On the same white board below they drew this picture. The ICU telemetry 650.98s - 656.40s | monitors are down. They said that the ones in the NICU weren't operational. They're unable 656.40s - 660.86s | to upload labs. How many of you folks think this type of stuff is really important? The 660.86s - 665.02s | ability to do laboratory tests and imaging and do things in the electronic health record. 665.02s - 670.62s | Raise your hand. How many of you feel like doctors and nurses can't be good at the jobs 670.62s - 674.70s | of taking care of their sick patients without these things? Raise your hand. You are correct. 674.70s - 680.70s | And so we're going to go into kind of the first question at the tip of this little period 680.70s - 686.46s | here. Cyber attacks degrade, delay, disrupt and decay the digital systems that power timely 686.46s - 690.94s | life-saving medical care. And for this we're going to go into this Q&A and try to get 690.94s - 694.30s | some amazing insights from the folks here. But I also remind you there's an opportunity 694.30s - 699.68s | at the end to ask some questions yourself. So the first question has to do with a lot 699.68s - 705.62s | of


chunk_text:
that the ones in the NICU weren't operational. They're unable 656.40s - 660.86s | to upload labs. How many of you folks think this type of stuff is really important? The 660.86s - 665.02s | ability to do laboratory tests and imaging and do things in the electronic health record. 665.02s - 670.62s | Raise your hand. How many of you feel like doctors and nurses can't be good at the jobs 670.62s - 674.70s | of taking care of their sick patients without these things? Raise your hand. You are correct. 674.70s - 680.70s | And so we're going to go into kind of the first question at the tip of this little period 680.70s - 686.46s | here. Cyber attacks degrade, delay, disrupt and decay the digital systems that power timely 686.46s - 690.94s | life-saving medical care. And for this we're going to go into this Q&A and try to get 690.94s - 694.30s | some amazing insights from the folks here. But I also remind you there's an opportunity 694.30s - 699.68s | at the end to ask some questions yourself. So the first question has to do with a lot 699.68s - 705.62s | of what we continue to learn more about every year is this interdependency. So after 2008, 705.62s - 711.30s | the financial crisis, the nation went under a painstaking process to try to better understand 711.30s - 715.78s | and map the dependencies of the financial sector. To understand if this bank fails, 715.78s - 721.78s | what will happen to the rest of it? In 2024, we've seen a bunch of attacks, change, ascension 721.78s - 726.62s | and then Krauszak was an attack. But the question I have for the first part of this panel is 726.62s - 731.74s | should the U.S. undergo a similar mapping exercise, right? Where we understand where 731.74s - 736.74s | the critical interdependencies, third party risk are of healthcare delivery organizations 736.74s - 741.66s | in this country because to my knowledge, no one knows what's going to happen. No one knows 741.66s - 746.38s | where the linchpins of linchpins are. We have a pile of them. So the first question 746.38s - 750.46s | is what do you all think about mapping the sector dependencies and what we should do 750.46s - 753.42s | about that? And this is not


chunk_text:
what we continue to learn more about every year is this interdependency. So after 2008, 705.62s - 711.30s | the financial crisis, the nation went under a painstaking process to try to better understand 711.30s - 715.78s | and map the dependencies of the financial sector. To understand if this bank fails, 715.78s - 721.78s | what will happen to the rest of it? In 2024, we've seen a bunch of attacks, change, ascension 721.78s - 726.62s | and then Krauszak was an attack. But the question I have for the first part of this panel is 726.62s - 731.74s | should the U.S. undergo a similar mapping exercise, right? Where we understand where 731.74s - 736.74s | the critical interdependencies, third party risk are of healthcare delivery organizations 736.74s - 741.66s | in this country because to my knowledge, no one knows what's going to happen. No one knows 741.66s - 746.38s | where the linchpins of linchpins are. We have a pile of them. So the first question 746.38s - 750.46s | is what do you all think about mapping the sector dependencies and what we should do 750.46s - 753.42s | about that? And this is not a new thing and we've been doing it for a little bit but let's 753.42s - 755.26s | learn about it now. What's going on? 755.26s - 761.26s | So I'm happy to go first. I'm a huge fan of mapping out the sector, something I tried 761.26s - 766.86s | to do actually when I was at HHS back in 2009. I think the challenge is it's getting that 766.86s - 771.58s | visibility and I think we've made some progress. CISA right now is trying to look at, can I 771.58s - 774.78s | ask any consequence analysis and part of that is really understanding, mapping out 774.78s - 779.62s | the sectors, understanding where those critical nodes are. I think so we are much further 779.62s - 783.78s | ahead than where we were 20 years ago but I still offer we're still nowhere near where 783.78s - 792.14s | we need to be to truly understand those interdependencies within a sector, much less between sectors. 792.14s - 797.74s | I think the other part of it is is getting visibility to understanding the actual software 797.74s - 801.26s | platforms and the other challenges. So it's


chunk_text:
a new thing and we've been doing it for a little bit but let's 753.42s - 755.26s | learn about it now. What's going on? 755.26s - 761.26s | So I'm happy to go first. I'm a huge fan of mapping out the sector, something I tried 761.26s - 766.86s | to do actually when I was at HHS back in 2009. I think the challenge is it's getting that 766.86s - 771.58s | visibility and I think we've made some progress. CISA right now is trying to look at, can I 771.58s - 774.78s | ask any consequence analysis and part of that is really understanding, mapping out 774.78s - 779.62s | the sectors, understanding where those critical nodes are. I think so we are much further 779.62s - 783.78s | ahead than where we were 20 years ago but I still offer we're still nowhere near where 783.78s - 792.14s | we need to be to truly understand those interdependencies within a sector, much less between sectors. 792.14s - 797.74s | I think the other part of it is is getting visibility to understanding the actual software 797.74s - 801.26s | platforms and the other challenges. So it's one thing to say that the connections exist 801.26s - 806.62s | between this type of financial system and healthcare and agriculture and those types 806.62s - 811.18s | of entities but to get to that next level to understand the market space, to understand 811.18s - 815.22s | that is just a massive undertaking. We've done, there's pockets of it. There's a great 815.22s - 820.46s | report from 2013 that looked at working through the Department of Commerce that looked at 820.46s - 825.26s | understanding global supply chain issues for pharmaceuticals and some medical devices. 825.26s - 829.34s | So there's pockets of this type of information out there but we don't have the ability yet 829.38s - 834.34s | to kind of coalesce it. We've launched a recent IT tool called Star that we're trying to use 834.34s - 838.22s | to really understand some of that but it's still at a very macro level. There's a lot 838.22s - 842.02s | more that needs to get done. A lot of that information is proprietary so getting access 842.02s - 846.34s | to it is somewhat of a challenge and then frankly the question I always


chunk_text:
one thing to say that the connections exist 801.26s - 806.62s | between this type of financial system and healthcare and agriculture and those types 806.62s - 811.18s | of entities but to get to that next level to understand the market space, to understand 811.18s - 815.22s | that is just a massive undertaking. We've done, there's pockets of it. There's a great 815.22s - 820.46s | report from 2013 that looked at working through the Department of Commerce that looked at 820.46s - 825.26s | understanding global supply chain issues for pharmaceuticals and some medical devices. 825.26s - 829.34s | So there's pockets of this type of information out there but we don't have the ability yet 829.38s - 834.34s | to kind of coalesce it. We've launched a recent IT tool called Star that we're trying to use 834.34s - 838.22s | to really understand some of that but it's still at a very macro level. There's a lot 838.22s - 842.02s | more that needs to get done. A lot of that information is proprietary so getting access 842.02s - 846.34s | to it is somewhat of a challenge and then frankly the question I always ask when we 846.34s - 850.26s | think about how the government should be asking questions and getting information, the next 850.26s - 854.06s | question I always ask is what are we going to do with it? I never want to be in a government 854.06s - 858.90s | agency where we're asking for information that we can't actually use to have some return 858.90s - 862.78s | on investment and value back to the people providing information and understanding what 862.78s - 866.62s | that looks like is another piece that we're trying to figure out. How do we get information 866.62s - 871.54s | and how do we ensure there's an adequate ROI back to those information sharers that can 871.54s - 874.58s | help them increase their cyber defense posture? 874.58s - 881.58s | Just kind of building on that, I mean the, it's really interesting talking to different 882.10s - 888.86s | stakeholders in both industry and government about their introspection into kind of the 888.86s - 894.70s | interdependencies and then the acknowledgement that there may be common good kind of upstream 894.70s - 899.06s | but perhaps a reticence to actually provide the


chunk_text:
ask when we 846.34s - 850.26s | think about how the government should be asking questions and getting information, the next 850.26s - 854.06s | question I always ask is what are we going to do with it? I never want to be in a government 854.06s - 858.90s | agency where we're asking for information that we can't actually use to have some return 858.90s - 862.78s | on investment and value back to the people providing information and understanding what 862.78s - 866.62s | that looks like is another piece that we're trying to figure out. How do we get information 866.62s - 871.54s | and how do we ensure there's an adequate ROI back to those information sharers that can 871.54s - 874.58s | help them increase their cyber defense posture? 874.58s - 881.58s | Just kind of building on that, I mean the, it's really interesting talking to different 882.10s - 888.86s | stakeholders in both industry and government about their introspection into kind of the 888.86s - 894.70s | interdependencies and then the acknowledgement that there may be common good kind of upstream 894.70s - 899.06s | but perhaps a reticence to actually provide the information that would enable that common 899.06s - 906.06s | good to manifest and so there we've, you know, talking to folks over the last year and sort 906.34s - 911.02s | of trying to identify like how do we get a lot of value from investment of public kind 911.02s - 915.30s | of public interest of taxpayer dollars from a research perspective, from a kind of transition 915.30s - 920.50s | or ops perspective, how do we find those really high value points to insert, inject 920.50s - 927.10s | kind of like support and it requires collaboration at a pretty large scale but it doesn't necessarily 927.10s - 931.66s | require exposure like for, you know, the folks that are, that we would need to kind of talk 931.66s - 935.98s | to that we are talking to, you know, folks are getting braver and that's a very good 935.98s - 941.68s | sign and sort of recognizing that this is, you know, we all win together or we all lose. 941.68s - 945.04s | So I think we're making progress but we could definitely make progress faster. 945.04s - 952.04s | I'd say on the medical


chunk_text:
information that would enable that common 899.06s - 906.06s | good to manifest and so there we've, you know, talking to folks over the last year and sort 906.34s - 911.02s | of trying to identify like how do we get a lot of value from investment of public kind 911.02s - 915.30s | of public interest of taxpayer dollars from a research perspective, from a kind of transition 915.30s - 920.50s | or ops perspective, how do we find those really high value points to insert, inject 920.50s - 927.10s | kind of like support and it requires collaboration at a pretty large scale but it doesn't necessarily 927.10s - 931.66s | require exposure like for, you know, the folks that are, that we would need to kind of talk 931.66s - 935.98s | to that we are talking to, you know, folks are getting braver and that's a very good 935.98s - 941.68s | sign and sort of recognizing that this is, you know, we all win together or we all lose. 941.68s - 945.04s | So I think we're making progress but we could definitely make progress faster. 945.04s - 952.04s | I'd say on the medical device side it's incredibly interesting to think about how to scope such 955.48s - 961.56s | an exercise because you're going to have differences, the medical device landscape is incredibly 961.56s - 968.56s | complex when you're ranging from implants to bedside equipment, capital equipment, software 969.56s - 976.08s | functions that only operate on a mobile device or in the cloud, then looking at what's actually 976.08s - 983.08s | used in different facilities and whether there's differentiation between similar devices from 983.20s - 990.20s | other manufacturers. So if you have one issue from one device type, do they have a backup 990.44s - 997.44s | that can still have some continuity of care? We've also seen a lot of discussions around 999.32s - 1005.32s | hospital CISOs trying to figure out how to respond to ransomware and what devices are 1005.32s - 1011.68s | relying on network or external connections because the initial response is to pull that 1011.68s - 1018.68s | network down and retract in order to better defend the larger ecosystem. But we've seen 1019.92s - 1026.16s | instances where all of a sudden the product security people are running to those CISOs 1026.16s


chunk_text:
device side it's incredibly interesting to think about how to scope such 955.48s - 961.56s | an exercise because you're going to have differences, the medical device landscape is incredibly 961.56s - 968.56s | complex when you're ranging from implants to bedside equipment, capital equipment, software 969.56s - 976.08s | functions that only operate on a mobile device or in the cloud, then looking at what's actually 976.08s - 983.08s | used in different facilities and whether there's differentiation between similar devices from 983.20s - 990.20s | other manufacturers. So if you have one issue from one device type, do they have a backup 990.44s - 997.44s | that can still have some continuity of care? We've also seen a lot of discussions around 999.32s - 1005.32s | hospital CISOs trying to figure out how to respond to ransomware and what devices are 1005.32s - 1011.68s | relying on network or external connections because the initial response is to pull that 1011.68s - 1018.68s | network down and retract in order to better defend the larger ecosystem. But we've seen 1019.92s - 1026.16s | instances where all of a sudden the product security people are running to those CISOs 1026.16s - 1032.56s | or they flip the switch to tell them that if they do that they have surgeries in progress 1032.56s - 1038.76s | that would be interrupted because they're losing that live access. So how do we maintain 1038.76s - 1045.36s | that continuity of care in response and how do we know what would be impacted based off 1045.36s - 1050.80s | of different actions that could be taken to resolve some of these issues? So the mapping 1050.96s - 1056.80s | can get incredibly complex. So then it's the how do you approach it and how do you make 1056.80s - 1063.80s | it useful for the facilities that are using it as well as useful for government regulators 1064.60s - 1076.20s | to help in response activities as well. Kind of a terrifying thought. So many times when 1076.20s - 1081.00s | I think about these types of problems, I'm always like, surely someone has done this 1081.04s - 1086.64s | and surely we have really great data on this. And so the thought of that we don't really 1086.64s - 1093.00s | even understand these interdependencies at any deep level and certainly don't have


chunk_text:
- 1032.56s | or they flip the switch to tell them that if they do that they have surgeries in progress 1032.56s - 1038.76s | that would be interrupted because they're losing that live access. So how do we maintain 1038.76s - 1045.36s | that continuity of care in response and how do we know what would be impacted based off 1045.36s - 1050.80s | of different actions that could be taken to resolve some of these issues? So the mapping 1050.96s - 1056.80s | can get incredibly complex. So then it's the how do you approach it and how do you make 1056.80s - 1063.80s | it useful for the facilities that are using it as well as useful for government regulators 1064.60s - 1076.20s | to help in response activities as well. Kind of a terrifying thought. So many times when 1076.20s - 1081.00s | I think about these types of problems, I'm always like, surely someone has done this 1081.04s - 1086.64s | and surely we have really great data on this. And so the thought of that we don't really 1086.64s - 1093.00s | even understand these interdependencies at any deep level and certainly don't have visibility 1093.00s - 1097.36s | over the entirety of the problem makes me think about how many other CrowdStrike-like 1097.36s - 1101.40s | issues we're going to have. Or like what's going to happen when, we'll talk about this 1101.40s - 1105.24s | in a little bit, when there's only really three or four electronic health record vendors 1105.24s - 1110.60s | in the whole country and we don't understand the technical plumbing behind it and understand 1110.60s - 1114.00s | what's really going to happen when something goes down. All right, we're going to go to 1114.00s - 1119.84s | question two here. All right, this question does not apply to anyone in this room, I guarantee 1119.84s - 1125.34s | it. Just kidding, it's everyone. Hackers, cyber security professionals employed at health 1125.34s - 1130.04s | systems, medical device manufacturers are often not considered whistleblower protections 1130.04s - 1137.16s | unless they see criminal activity. But we hear time and time again from folks in these 1137.16s - 1142.64s | spaces that they see problems and they can't fix them. And they're increasingly worried 1142.64s - 1148.72s | about what would happen when these issues are not fixed.


chunk_text:
visibility 1093.00s - 1097.36s | over the entirety of the problem makes me think about how many other CrowdStrike-like 1097.36s - 1101.40s | issues we're going to have. Or like what's going to happen when, we'll talk about this 1101.40s - 1105.24s | in a little bit, when there's only really three or four electronic health record vendors 1105.24s - 1110.60s | in the whole country and we don't understand the technical plumbing behind it and understand 1110.60s - 1114.00s | what's really going to happen when something goes down. All right, we're going to go to 1114.00s - 1119.84s | question two here. All right, this question does not apply to anyone in this room, I guarantee 1119.84s - 1125.34s | it. Just kidding, it's everyone. Hackers, cyber security professionals employed at health 1125.34s - 1130.04s | systems, medical device manufacturers are often not considered whistleblower protections 1130.04s - 1137.16s | unless they see criminal activity. But we hear time and time again from folks in these 1137.16s - 1142.64s | spaces that they see problems and they can't fix them. And they're increasingly worried 1142.64s - 1148.72s | about what would happen when these issues are not fixed. So my question to the panel 1148.72s - 1156.12s | is really just talking about should we expand protections to folks in technical healthcare 1156.12s - 1161.32s | roles or you could say technical roles more broadly that offer some protections when they 1161.32s - 1166.84s | don't necessarily cross the line of criminality but they believe in good faith that there 1166.84s - 1175.24s | is going to be some issue before the ransomware hits. So maybe also take it away. 1175.24s - 1180.48s | So I think when these whistleblower complaints sort of come about, you have to understand 1180.48s - 1185.44s | where they exist. Is it happening internally in the company? And is there a concern that 1185.44s - 1190.20s | you can address with the internal controls that exist within a company? And a lot of 1190.20s - 1196.04s | that is do you have the leadership buy-in? Do you have the board buy-in to take cyber 1196.04s - 1201.16s | security concerns seriously? And how do you sort of implement that culture within that 1201.16s - 1208.68s | organization to have them say this is a top critical issue in order to maintain good


chunk_text:
So my question to the panel 1148.72s - 1156.12s | is really just talking about should we expand protections to folks in technical healthcare 1156.12s - 1161.32s | roles or you could say technical roles more broadly that offer some protections when they 1161.32s - 1166.84s | don't necessarily cross the line of criminality but they believe in good faith that there 1166.84s - 1175.24s | is going to be some issue before the ransomware hits. So maybe also take it away. 1175.24s - 1180.48s | So I think when these whistleblower complaints sort of come about, you have to understand 1180.48s - 1185.44s | where they exist. Is it happening internally in the company? And is there a concern that 1185.44s - 1190.20s | you can address with the internal controls that exist within a company? And a lot of 1190.20s - 1196.04s | that is do you have the leadership buy-in? Do you have the board buy-in to take cyber 1196.04s - 1201.16s | security concerns seriously? And how do you sort of implement that culture within that 1201.16s - 1208.68s | organization to have them say this is a top critical issue in order to maintain good business 1208.68s - 1214.80s | practices, in order to maintain patient safety by the examples that you have seen. And so 1214.80s - 1220.96s | there's a lot of work to be done in terms of what can the infrastructure within an organization 1220.96s - 1225.28s | look like so the people that are working with these technologies can action on them 1225.28s - 1231.20s | appropriately. And then there's the sort of external whistleblower. I've tried to basically 1231.20s - 1236.08s | work within my organization. It doesn't seem like these issues are going to be addressed. 1236.08s - 1242.62s | So what do I do now? Who do I turn to in order to prevent some major crisis from happening? 1242.62s - 1248.28s | And I will say if there are lines that are crossed, if say a company does not make a 1248.36s - 1254.44s | disclosure about a hack that has happened, then there are ways in which you can report to the 1254.44s - 1259.00s | SEC that they didn't make the appropriate disclosures for the information that was 1259.00s - 1265.96s | granted. And you are covered under federal whistleblower protections in those cases.


chunk_text:
business 1208.68s - 1214.80s | practices, in order to maintain patient safety by the examples that you have seen. And so 1214.80s - 1220.96s | there's a lot of work to be done in terms of what can the infrastructure within an organization 1220.96s - 1225.28s | look like so the people that are working with these technologies can action on them 1225.28s - 1231.20s | appropriately. And then there's the sort of external whistleblower. I've tried to basically 1231.20s - 1236.08s | work within my organization. It doesn't seem like these issues are going to be addressed. 1236.08s - 1242.62s | So what do I do now? Who do I turn to in order to prevent some major crisis from happening? 1242.62s - 1248.28s | And I will say if there are lines that are crossed, if say a company does not make a 1248.36s - 1254.44s | disclosure about a hack that has happened, then there are ways in which you can report to the 1254.44s - 1259.00s | SEC that they didn't make the appropriate disclosures for the information that was 1259.00s - 1265.96s | granted. And you are covered under federal whistleblower protections in those cases. 1265.96s - 1270.76s | And then there are also several different whistleblower protections that exist at a state 1270.76s - 1276.84s | level. But it has to sort of pay attention to what those agencies are. And then it also depends 1276.84s - 1283.16s | what type of regulatory agency. Are you talking to the HHS? Are you talking to the SEC? Who are 1283.16s - 1289.16s | the appropriate parties involved that can help guide you through the process of getting 1289.16s - 1295.24s | the concern addressed? Whether it's patient information that shouldn't have been disclosed 1295.24s - 1299.96s | that is disclosed, whether it's a ransomware attack that instead of sort of addressing that 1299.96s - 1308.04s | and submitting that to the FBI or someone else that that happened. And with those, there should 1308.04s - 1313.88s | be expansion of those protections for some of these people. But it's also about kind of turning 1313.88s - 1318.92s | back to the internal controls. What most of these people who are reporting these concerns are 1318.92s - 1325.24s | concerned about are, can I maintain my anonymity? If I can't maintain my anonymity, will I be


chunk_text:
1265.96s - 1270.76s | And then there are also several different whistleblower protections that exist at a state 1270.76s - 1276.84s | level. But it has to sort of pay attention to what those agencies are. And then it also depends 1276.84s - 1283.16s | what type of regulatory agency. Are you talking to the HHS? Are you talking to the SEC? Who are 1283.16s - 1289.16s | the appropriate parties involved that can help guide you through the process of getting 1289.16s - 1295.24s | the concern addressed? Whether it's patient information that shouldn't have been disclosed 1295.24s - 1299.96s | that is disclosed, whether it's a ransomware attack that instead of sort of addressing that 1299.96s - 1308.04s | and submitting that to the FBI or someone else that that happened. And with those, there should 1308.04s - 1313.88s | be expansion of those protections for some of these people. But it's also about kind of turning 1313.88s - 1318.92s | back to the internal controls. What most of these people who are reporting these concerns are 1318.92s - 1325.24s | concerned about are, can I maintain my anonymity? If I can't maintain my anonymity, will I be able 1325.24s - 1331.72s | to keep my job? Am I going to face any type of retaliation? Is there going to be litigation 1331.72s - 1338.04s | involved? Is there going to be public backlash? Am I going to get blacklisted from my career? 1338.04s - 1343.08s | And so I think that's some of the things fundamentally and structurally, both in terms of 1343.08s - 1349.32s | like how we treat people who report concerns, in addition to the sort of external infrastructure 1349.32s - 1355.40s | that we need to take into consideration, considering we're seeing so many people speak up 1355.40s - 1359.16s | about the fact that they're seeing these attacks, but they don't really know what to do because 1359.16s - 1364.52s | there's not necessarily enough external pressure to address them just yet. So I think those are 1364.52s - 1370.04s | some of the fundamentals to pay attention to when it comes to working with people who are 1370.04s - 1374.92s | concerned about these issues, but the regulatory landscape isn't as clear cut yet. 1375.00s - 1380.68s | Raise your hand if you've ever worked for a healthcare organization, you had


chunk_text:
able 1325.24s - 1331.72s | to keep my job? Am I going to face any type of retaliation? Is there going to be litigation 1331.72s - 1338.04s | involved? Is there going to be public backlash? Am I going to get blacklisted from my career? 1338.04s - 1343.08s | And so I think that's some of the things fundamentally and structurally, both in terms of 1343.08s - 1349.32s | like how we treat people who report concerns, in addition to the sort of external infrastructure 1349.32s - 1355.40s | that we need to take into consideration, considering we're seeing so many people speak up 1355.40s - 1359.16s | about the fact that they're seeing these attacks, but they don't really know what to do because 1359.16s - 1364.52s | there's not necessarily enough external pressure to address them just yet. So I think those are 1364.52s - 1370.04s | some of the fundamentals to pay attention to when it comes to working with people who are 1370.04s - 1374.92s | concerned about these issues, but the regulatory landscape isn't as clear cut yet. 1375.00s - 1380.68s | Raise your hand if you've ever worked for a healthcare organization, you had some real 1380.68s - 1384.92s | concerns about the security, you raised them a bunch of times and then nothing happened and 1384.92s - 1393.56s | you found a new job. Raise your hand. Yep. That's a common story. There should be more hands up. 1393.56s - 1402.76s | I've talked to you guys at bars. Anything else? Any comments? I'm going to pull a little bit of 1402.76s - 1406.44s | a thread out of that if that's all right. I was just amazed at the work that you do to support 1406.44s - 1411.16s | whistleblowers. And one of the things I learned is how much you do to support the mental health 1411.16s - 1415.96s | and wellbeing of these whistleblowers as they go through incredibly stressful situations. 1415.96s - 1420.68s | I imagine there are folks in this audience that feel similarly, those stress and pressure. And 1420.68s - 1425.32s | if we could just take a moment, if you have any advice, suggestions, anything that folks out there 1425.32s - 1431.56s | that might feel some of those constraints that they could access. Yeah. I always tell people 1431.56s - 1437.80s | whistleblowing is a


chunk_text:
some real 1380.68s - 1384.92s | concerns about the security, you raised them a bunch of times and then nothing happened and 1384.92s - 1393.56s | you found a new job. Raise your hand. Yep. That's a common story. There should be more hands up. 1393.56s - 1402.76s | I've talked to you guys at bars. Anything else? Any comments? I'm going to pull a little bit of 1402.76s - 1406.44s | a thread out of that if that's all right. I was just amazed at the work that you do to support 1406.44s - 1411.16s | whistleblowers. And one of the things I learned is how much you do to support the mental health 1411.16s - 1415.96s | and wellbeing of these whistleblowers as they go through incredibly stressful situations. 1415.96s - 1420.68s | I imagine there are folks in this audience that feel similarly, those stress and pressure. And 1420.68s - 1425.32s | if we could just take a moment, if you have any advice, suggestions, anything that folks out there 1425.32s - 1431.56s | that might feel some of those constraints that they could access. Yeah. I always tell people 1431.56s - 1437.80s | whistleblowing is a last resort. You shouldn't really rely on like one David against a Goliath 1437.80s - 1442.44s | for some of these issues. So the best way to help whistleblowers is usually to 1443.16s - 1449.40s | figure out how to build the infrastructure to address concerns early because it's an incredibly 1449.40s - 1454.44s | stressful process. I haven't met a single person that this hasn't deeply affected their families, 1455.08s - 1459.80s | their mental health, their careers, their professions. It's an incredibly stressful 1459.80s - 1468.36s | process that is very difficult to negotiate and navigate. So people who ever witnessed someone 1468.36s - 1472.68s | in that position, the more support infrastructure you can build around those people, the better 1472.68s - 1477.96s | they're going to fare. And then if they are, if you are in a position that you're reporting a 1477.96s - 1483.64s | concern, the biggest piece of advice is try to find a lawyer, a lawyer, you can work on contingency 1483.64s - 1489.16s | and not just the lawyer that your company provides because you need someone who's going to protect 1489.24s - 1494.60s | your own interest and really show


chunk_text:
last resort. You shouldn't really rely on like one David against a Goliath 1437.80s - 1442.44s | for some of these issues. So the best way to help whistleblowers is usually to 1443.16s - 1449.40s | figure out how to build the infrastructure to address concerns early because it's an incredibly 1449.40s - 1454.44s | stressful process. I haven't met a single person that this hasn't deeply affected their families, 1455.08s - 1459.80s | their mental health, their careers, their professions. It's an incredibly stressful 1459.80s - 1468.36s | process that is very difficult to negotiate and navigate. So people who ever witnessed someone 1468.36s - 1472.68s | in that position, the more support infrastructure you can build around those people, the better 1472.68s - 1477.96s | they're going to fare. And then if they are, if you are in a position that you're reporting a 1477.96s - 1483.64s | concern, the biggest piece of advice is try to find a lawyer, a lawyer, you can work on contingency 1483.64s - 1489.16s | and not just the lawyer that your company provides because you need someone who's going to protect 1489.24s - 1494.60s | your own interest and really show you the landscape of how you can appropriately protect yourself. 1495.24s - 1501.32s | And then additionally to that, you know, there are lots of advocacy agencies, non-profits that are 1501.32s - 1507.40s | here to help and to support people who are going through that tough process, whether it's 1507.40s - 1512.84s | Whistleblowers of America, whether it's different regulatory agencies that have a Budsman's programs 1512.84s - 1518.04s | that are meant to really support the people who are coming forward. But make sure you have a 1518.04s - 1523.56s | good support network of the right people through it because it's really critical for people in that 1523.56s - 1531.00s | position. Give it up. That's really great stuff. All right, we're going to shift back gears to some 1531.00s - 1537.88s | medical devices. All right. There are no legacy medical devices in hospitals anymore. We fixed 1537.88s - 1545.00s | the problem, right, Matt? You fix it? No. Oh, wow. I don't have trillions of dollars. 1545.88s - 1551.24s | They're often cited as a major source of risk for hospitals. They have antiquated operating 1551.24s - 1557.08s | systems, unpatched remote vulnerabilities. They're often


chunk_text:
you the landscape of how you can appropriately protect yourself. 1495.24s - 1501.32s | And then additionally to that, you know, there are lots of advocacy agencies, non-profits that are 1501.32s - 1507.40s | here to help and to support people who are going through that tough process, whether it's 1507.40s - 1512.84s | Whistleblowers of America, whether it's different regulatory agencies that have a Budsman's programs 1512.84s - 1518.04s | that are meant to really support the people who are coming forward. But make sure you have a 1518.04s - 1523.56s | good support network of the right people through it because it's really critical for people in that 1523.56s - 1531.00s | position. Give it up. That's really great stuff. All right, we're going to shift back gears to some 1531.00s - 1537.88s | medical devices. All right. There are no legacy medical devices in hospitals anymore. We fixed 1537.88s - 1545.00s | the problem, right, Matt? You fix it? No. Oh, wow. I don't have trillions of dollars. 1545.88s - 1551.24s | They're often cited as a major source of risk for hospitals. They have antiquated operating 1551.24s - 1557.08s | systems, unpatched remote vulnerabilities. They're often deployed on super flat networks. 1557.88s - 1563.32s | And this is not just a problem in rural America or poorly resourced hospitals. I mean, we see 1563.32s - 1570.36s | similar types of really concerning implementations, deployments in larger hospitals as well, some of 1570.84s - 1576.44s | which you might actually get care at. The question I have for here is talking about upgrade. And I 1576.44s - 1581.56s | think this is going to be a really important thing to talk about. How can we take this problem of 1581.56s - 1589.40s | patching legacy medical devices that should have, many people thought would have been solved by now, 1589.40s - 1593.00s | like 20 years ago. You think people would have been like, oh, we're not going to be talking 1593.00s - 1598.12s | about this at DEF CON 32 anymore. So clearly what we're doing isn't working. What's some new 1598.12s - 1602.84s | approaches we can get at, Andrew, that I think would be a good opportunity for us to get into? 1603.48s - 1610.12s | You're very kind, Christian. You're very kind. So yeah, the legacy medical device problem, 1610.12s - 1614.68s |


chunk_text:
deployed on super flat networks. 1557.88s - 1563.32s | And this is not just a problem in rural America or poorly resourced hospitals. I mean, we see 1563.32s - 1570.36s | similar types of really concerning implementations, deployments in larger hospitals as well, some of 1570.84s - 1576.44s | which you might actually get care at. The question I have for here is talking about upgrade. And I 1576.44s - 1581.56s | think this is going to be a really important thing to talk about. How can we take this problem of 1581.56s - 1589.40s | patching legacy medical devices that should have, many people thought would have been solved by now, 1589.40s - 1593.00s | like 20 years ago. You think people would have been like, oh, we're not going to be talking 1593.00s - 1598.12s | about this at DEF CON 32 anymore. So clearly what we're doing isn't working. What's some new 1598.12s - 1602.84s | approaches we can get at, Andrew, that I think would be a good opportunity for us to get into? 1603.48s - 1610.12s | You're very kind, Christian. You're very kind. So yeah, the legacy medical device problem, 1610.12s - 1614.68s | or the legacy device problem generally is especially problematic in healthcare because 1614.68s - 1621.32s | of how many devices we use to provide care, to provide high quality care. And so getting the 1621.32s - 1626.92s | most life out of a system that's still performant, right, there's a lot of value in that. But if the 1626.92s - 1631.08s | security pieces of that, if the security or connectivity or interoperability elements of that 1631.64s - 1635.24s | device, despite its ability to provide care, maintaining, you know, being sufficiently 1635.24s - 1640.60s | performant, if it can't, maybe not, I would have, if you would ask me 15 years ago to 1641.72s - 1645.72s | develop a device that would be safe from today's threats, I'd be like, nope, can't do that. 1646.68s - 1649.40s | So, you know, how we think about this space. 1652.92s - 1653.96s | Exactly, exactly. 1655.56s - 1657.24s | Yeah, they're clapping for you. 1657.24s - 1659.32s | Very nice. DEF CON next, everybody. 1661.80s - 1667.00s | So I think, so you have this large complex ecosystem with lots of different vendors, 1667.00s - 1670.84s | lots of different stakeholders,


chunk_text:
or the legacy device problem generally is especially problematic in healthcare because 1614.68s - 1621.32s | of how many devices we use to provide care, to provide high quality care. And so getting the 1621.32s - 1626.92s | most life out of a system that's still performant, right, there's a lot of value in that. But if the 1626.92s - 1631.08s | security pieces of that, if the security or connectivity or interoperability elements of that 1631.64s - 1635.24s | device, despite its ability to provide care, maintaining, you know, being sufficiently 1635.24s - 1640.60s | performant, if it can't, maybe not, I would have, if you would ask me 15 years ago to 1641.72s - 1645.72s | develop a device that would be safe from today's threats, I'd be like, nope, can't do that. 1646.68s - 1649.40s | So, you know, how we think about this space. 1652.92s - 1653.96s | Exactly, exactly. 1655.56s - 1657.24s | Yeah, they're clapping for you. 1657.24s - 1659.32s | Very nice. DEF CON next, everybody. 1661.80s - 1667.00s | So I think, so you have this large complex ecosystem with lots of different vendors, 1667.00s - 1670.84s | lots of different stakeholders, you have providers that are doing everything, 1670.84s - 1677.96s | their focus should remain on care. I have a small aneurysm, every time someone tells me 1677.96s - 1684.60s | that better training for clinicians will solve hospital cybersecurity, I flip the table, rage, 1684.60s - 1689.64s | quit and leave immediately. Ideally, we want to get to a place where clinical staff don't think 1689.64s - 1695.00s | about the cybersecurity of the systems they're dealing with. That's the dream. But we still have 1695.00s - 1700.44s | this very complex, diverse tech ecosystem to deal with. And so how we do that, and how we do that 1700.44s - 1704.28s | in a way that is context sensitive to each different clinical environment, to each different 1704.28s - 1709.48s | department, that's not something that I've seen addressed in industry, kind of like enterprise 1709.48s - 1714.60s | tooling, like does not hit the it's not looking at that problem space, because it's really hard. 1714.60s - 1718.52s | And there's not necessarily like the upside, I think, to them from from an investment standpoint. 1719.16s - 1724.60s | So at ARPA-H, we are looking at how


chunk_text:
you have providers that are doing everything, 1670.84s - 1677.96s | their focus should remain on care. I have a small aneurysm, every time someone tells me 1677.96s - 1684.60s | that better training for clinicians will solve hospital cybersecurity, I flip the table, rage, 1684.60s - 1689.64s | quit and leave immediately. Ideally, we want to get to a place where clinical staff don't think 1689.64s - 1695.00s | about the cybersecurity of the systems they're dealing with. That's the dream. But we still have 1695.00s - 1700.44s | this very complex, diverse tech ecosystem to deal with. And so how we do that, and how we do that 1700.44s - 1704.28s | in a way that is context sensitive to each different clinical environment, to each different 1704.28s - 1709.48s | department, that's not something that I've seen addressed in industry, kind of like enterprise 1709.48s - 1714.60s | tooling, like does not hit the it's not looking at that problem space, because it's really hard. 1714.60s - 1718.52s | And there's not necessarily like the upside, I think, to them from from an investment standpoint. 1719.16s - 1724.60s | So at ARPA-H, we are looking at how we can invest and create the next generation of tools 1724.60s - 1729.88s | to address that space, so that all of the hospitals, all the clinical environments in the 1729.88s - 1737.40s | US can have like more performant care of their own kind of healthcare technology ecosystems. So we 1737.40s - 1741.32s | can understand what's there kind of going back to this mapping question from earlier, you can't 1741.32s - 1746.76s | defend what you can't find what you can't see. So how do we understand the environment? And then 1746.76s - 1752.68s | how do we empower the IT staff, even if they're only one or two folks to engage with and manage 1752.68s - 1758.60s | that environment in a super effective way? When they're dealing with, you know, hundreds or 1758.60s - 1762.52s | thousands of devices across hundreds, potentially 10s to hundreds of vendors, like the configuration 1762.52s - 1770.76s | management alone is a nightmare. So it's just incredibly kind of challenging stack of problems, 1770.76s - 1775.56s | on top of the fact that if you make a mistake, or if your devices are out of commission, 1775.56s - 1779.80s |


chunk_text:
we can invest and create the next generation of tools 1724.60s - 1729.88s | to address that space, so that all of the hospitals, all the clinical environments in the 1729.88s - 1737.40s | US can have like more performant care of their own kind of healthcare technology ecosystems. So we 1737.40s - 1741.32s | can understand what's there kind of going back to this mapping question from earlier, you can't 1741.32s - 1746.76s | defend what you can't find what you can't see. So how do we understand the environment? And then 1746.76s - 1752.68s | how do we empower the IT staff, even if they're only one or two folks to engage with and manage 1752.68s - 1758.60s | that environment in a super effective way? When they're dealing with, you know, hundreds or 1758.60s - 1762.52s | thousands of devices across hundreds, potentially 10s to hundreds of vendors, like the configuration 1762.52s - 1770.76s | management alone is a nightmare. So it's just incredibly kind of challenging stack of problems, 1770.76s - 1775.56s | on top of the fact that if you make a mistake, or if your devices are out of commission, 1775.56s - 1779.80s | like you increase the mortality of your patients every time you increase the delay to care. 1780.28s - 1784.84s | And so, you know, we don't really have a choice but to do better in that space. 1784.84s - 1789.00s | And so I'm really excited to see we've had a lot of engagement across healthcare providers, 1789.00s - 1793.40s | across industry, you know, folks coming to the table with, you know, hey, we kind of had this 1793.40s - 1797.32s | idea for this space. Let's let's see if we can make it work. Let's see if we can, you know, 1797.32s - 1802.68s | make a huge dent in this in this challenge. So yeah, I'm very excited to kind of see what we 1802.68s - 1808.92s | can do, because it really is a tremendous, like, just technical challenge. But I think I think 1809.24s - 1811.56s | obviously, it's worth solving. Hell, yeah. 1813.96s - 1815.64s | DEF CON next, everybody. DEF CON next. 1816.84s - 1820.60s | All right, Matt, do you want to pick up a little bit where that left off from the legacy perspective 1820.60s - 1825.96s | from, you know,


chunk_text:
like you increase the mortality of your patients every time you increase the delay to care. 1780.28s - 1784.84s | And so, you know, we don't really have a choice but to do better in that space. 1784.84s - 1789.00s | And so I'm really excited to see we've had a lot of engagement across healthcare providers, 1789.00s - 1793.40s | across industry, you know, folks coming to the table with, you know, hey, we kind of had this 1793.40s - 1797.32s | idea for this space. Let's let's see if we can make it work. Let's see if we can, you know, 1797.32s - 1802.68s | make a huge dent in this in this challenge. So yeah, I'm very excited to kind of see what we 1802.68s - 1808.92s | can do, because it really is a tremendous, like, just technical challenge. But I think I think 1809.24s - 1811.56s | obviously, it's worth solving. Hell, yeah. 1813.96s - 1815.64s | DEF CON next, everybody. DEF CON next. 1816.84s - 1820.60s | All right, Matt, do you want to pick up a little bit where that left off from the legacy perspective 1820.60s - 1825.96s | from, you know, not now as current FDA, but maybe in the past, right? Tell us a little bit about 1825.96s - 1829.16s | what your guys's evolution of thinking has been around it around this. 1830.04s - 1838.04s | Yeah. A lot of our effort has started in the healthcare sector coordinating council and in 1838.04s - 1844.76s | the International Medical Device Regulators Forum, or FDA involvement, not our anymore. 1846.76s - 1853.64s | It's vocal memory from eight and a half years there. It's hard to fight. But a lot of the 1853.64s - 1859.00s | effort has been very collaborative, trying to get the healthcare perspective, the medical device 1859.00s - 1866.28s | manufacturer perspective, as well as the regulator involvement. The legacy device problem is not 1866.28s - 1874.20s | going away anytime soon, especially since FDA's statutory authority for cybersecurity just took 1874.20s - 1884.04s | effect last year. It's going to take some time for those better devices to fully help prevent 1884.04s - 1892.76s | some of the onset of the legacy device problem. A lot of what we see in response to some of the 1892.76s - 1899.24s | issues to maintain continuity of


chunk_text:
not now as current FDA, but maybe in the past, right? Tell us a little bit about 1825.96s - 1829.16s | what your guys's evolution of thinking has been around it around this. 1830.04s - 1838.04s | Yeah. A lot of our effort has started in the healthcare sector coordinating council and in 1838.04s - 1844.76s | the International Medical Device Regulators Forum, or FDA involvement, not our anymore. 1846.76s - 1853.64s | It's vocal memory from eight and a half years there. It's hard to fight. But a lot of the 1853.64s - 1859.00s | effort has been very collaborative, trying to get the healthcare perspective, the medical device 1859.00s - 1866.28s | manufacturer perspective, as well as the regulator involvement. The legacy device problem is not 1866.28s - 1874.20s | going away anytime soon, especially since FDA's statutory authority for cybersecurity just took 1874.20s - 1884.04s | effect last year. It's going to take some time for those better devices to fully help prevent 1884.04s - 1892.76s | some of the onset of the legacy device problem. A lot of what we see in response to some of the 1892.76s - 1899.24s | issues to maintain continuity of care is, okay, well, we'll just make it not connected to the 1899.24s - 1906.52s | Ethernet, and that removes a lot of our risk. But as we start to see devices evolve and additional 1906.52s - 1912.92s | functionality being added, once you start doing that, you can start impacting some of the intended 1912.92s - 1918.12s | use of that medical device and some of the core functionality that that device is providing. 1918.12s - 1923.48s | So there's a greater cost to some of those mitigations that were available on older 1923.48s - 1929.32s | products may no longer be available on some of the more recent products that are going out. So 1929.32s - 1938.04s | it's a complex issue. It's also a regulated environment for updating and patching devices. 1938.04s - 1946.44s | So making sure that addressing some of those challenges to make sure that the device is not 1946.44s - 1954.60s | impacted by some of these updates to address some of the vulnerabilities, it's going to take 1954.60s - 1961.72s | a kind of large collaborative effort to make sure that all of the nuances, the challenges, 1961.72s - 1968.20s | and


