Analyzing sysmon events with graph

Hello colleagues, this is an example I promised answering this tweet. I used https://github.com/SwiftOnSecurity/sysmon-configthis sysmon config to capture activities happening on my system. Unfortunately it did not capture a lot of network-related activities, perhaps I need to change it to extend network-level filters. But on the other hand it captured a lot of process level activities, so in this example i’d like to try to graph process creation events.
So first thing to do in this case is create a graph object

$g = New-Graph -Type BidirectionalGraph

And now we can fill in the graph with some data right from the event log. It may take few seconds until all events will be processed

Get-WinEvent -LogName Microsoft-Windows-Sysmon/Operational | ? {$_.id -eq 1} |
    % { if ($_.properties[3]) {Add-Edge -From $_.Properties[-2].value -To $_.properties[3].value -Graph $g}} |
    Out-Null

And then we just display the graph

Show-GraphLayout -Graph $g

This is how it looks like

Analyzing sysmon events with graph

Using graphs to analyze Windows Firewall logs

Hello colleagues, lets talk about how we can use graphs to look inside of communications happening in our environments in an easy way. First of all we need to have some data to analyze. Lets gather some. It is pretty simple – just use this article and enable Windows Firewall Logging. I usually put the logs into a separate folder, just for easy access. Here is how it looks like on my system:

FWLog

As you can see it is structured, so it is good idea to parse it as objects. Lets do it, and it may look like the following:

$f = gc 'C:\Logs\FWLogs\pfirewall_public - Copy.log'
$regex = '^(?<datetime>\d{4,4}-\d{2,2}-\d{2,2}\s\d{2}:\d{2}:\d{2})\s(?<action>\w+)\s(?<protocol>\w+)\s(?<srcip>\b(?:\d{1,3}\.){3}\d{1,3}\b)\s(?<dstip>\b(?:\d{1,3}\.){3}\d{1,3}\b)\s(?<srcport>\d{1,5})\s(?<dstport>\d{1,5})\s(?<size>\d+|-)\s(?<tcpflags>\d+|-)\s(?<tcpsyn>\d+|-)\s(?<tcpack>\d+|-)\s(?<tcpwin>\d+|-)\s(?<icmptype>\d+|-)\s(?<icmpcode>\d+|-)\s(?<info>\d+|-)\s(?<path>.+)$'


$log =
$f | % {
    $_ -match $regex | Out-Null
    if ($Matches) {
    [PSCustomObject]@{
        action   = $Matches.action
        srcip    = [ipaddress]$Matches.srcip
        dstport  = $Matches.dstport
        tcpflags = $Matches.tcpflags
        dstip    = [ipaddress]$Matches.dstip
        info     = $Matches.info
        size     = $Matches.size
        protocol = $Matches.protocol
        tcpack   = $Matches.tcpac
        srcport  = $Matches.srcport
        tcpsyn   = $Matches.tcpsyn
        datetime = [datetime]$Matches.datetime
        icmptype = $Matches.icmptype
        tcpwin   = $Matches.tcpwin
        icmpcode = $Matches.icmpcode
        path     = $Matches.path
    }
    }
}

The $regex variable here contains a long regular expression which hopefully is going to parse our file onto objects – one per line. We use -match operator in the pipeline to apply this expression to each line of the file and suppress output by piping it to Out-Null. This is not the fastest way of parsing files but to me one of the easiest ones. If the there is a match $Matches variable gets populated. Here we want to do the trick. First we fill in the hashtable with the fields we would like to put into our new object and then convert this hashtable to an object. One thing to pay attention to is we convert datetime field into [datetime] type to be able to use filtering and sorting capabilities later on. The same we do with ip addresses. So at the end we’ve got objects and they look like this:

fwlog3

Looks great so far, but what is next? First of all objects we’ve got are just edges of our graph. So what we can do now to convert the set of edges to a set of vertices along with their edges? This is really easy, lets just add them

$g = new-graph -Type BidirectionalGraph

$log | ? {$_.srcip -and $_.dstip} | % {
    Add-Edge -From $_.srcip -To $_.dstip -Graph $g | out-null
}

So here we create a graph and add vertices. Source and destination IPs being converted into string representations and added to the graph, and the library itself takes care about duplicated entries. So at the end we have a $g variable containing the graph. Now we can easily display it by issuing

Show-GraphLayout -Graph $g

which will display something like this
fwlog4
It does not look beautiful for me as it basically shows only a single log from my own laptop. But if we had multiple logs from some environment we would be able to see communications happened inside and outside the environment.
What else can we do here? For example we can try to filter the set of log data we parsed and display the smaller subset of data, for instance like this

$d = ($log | sort datetime -Descending | select -First 1).datetime.addhours(-1)
$twoHrsLog = $log.Where({$_.datetime -gt $d})

$g1 = new-graph -Type BidirectionalGraph

$twoHrsLog | ? {$_.srcip -and $_.dstip} | % {
    Add-Edge -From $_.srcip -To $_.dstip -Graph $g1 | out-null
}
Show-GraphLayout -Graph $g1

where we just filter the log to see just communications happened for the last hour. We could also filter by IPs or by degree of out or in edges

$g2 = new-graph -Type BidirectionalGraph
$x = $g.Vertices.Where({$g.OutDegree($_) -gt 0})

$x | where {$_ -ne '192.168.0.107'} | % {$e = $g.InEdges($_); if ($e) {$e | % {add-edge -from $_.source -to $_.target -Graph $g2}}}
$x | where {$_ -ne '192.168.0.107'} | % {$e = $g.OutEdges($_); if ($e) {$e | % {add-edge -from $_.source -to $_.target -Graph $g2}}}

Show-GraphLayout -Graph $g2

Complete set of commands is below

#file and regular expression
$f = gc 'C:\Logs\FWLogs\pfirewall_public - Copy.log'
$regex = '^(?<datetime>\d{4,4}-\d{2,2}-\d{2,2}\s\d{2}:\d{2}:\d{2})\s(?<action>\w+)\s(?<protocol>\w+)\s(?<srcip>\b(?:\d{1,3}\.){3}\d{1,3}\b)\s(?<dstip>\b(?:\d{1,3}\.){3}\d{1,3}\b)\s(?<srcport>\d{1,5})\s(?<dstport>\d{1,5})\s(?<size>\d+|-)\s(?<tcpflags>\d+|-)\s(?<tcpsyn>\d+|-)\s(?<tcpack>\d+|-)\s(?<tcpwin>\d+|-)\s(?<icmptype>\d+|-)\s(?<icmpcode>\d+|-)\s(?<info>\d+|-)\s(?<path>.+)$'

#parsing
$log = 
$f | % {
    $_ -match $regex | Out-Null
    if ($Matches) {
    [PSCustomObject]@{
        action   = $Matches.action
        srcip    = [ipaddress]$Matches.srcip
        dstport  = $Matches.dstport
        tcpflags = $Matches.tcpflags
        dstip    = [ipaddress]$Matches.dstip
        info     = $Matches.info
        size     = $Matches.size
        protocol = $Matches.protocol
        tcpack   = $Matches.tcpac
        srcport  = $Matches.srcport
        tcpsyn   = $Matches.tcpsyn
        datetime = [datetime]$Matches.datetime
        icmptype = $Matches.icmptype
        tcpwin   = $Matches.tcpwin
        icmpcode = $Matches.icmpcode
        path     = $Matches.path
    }
    }
}

 #whole graph   
$g = new-graph -Type BidirectionalGraph

$log | ? {$_.srcip -and $_.dstip} | % {
    Add-Edge -From $_.srcip -To $_.dstip -Graph $g | out-null
}

Show-GraphLayout -Graph $g

#subset of log records filterd by time
$d = ($log | sort datetime -Descending | select -First 1).datetime.addhours(-1)
$twoHrsLog = $log.Where({$_.datetime -gt $d})

$g1 = new-graph -Type BidirectionalGraph

$twoHrsLog | ? {$_.srcip -and $_.dstip} | % {
    Add-Edge -From $_.srcip -To $_.dstip -Graph $g1 | out-null
}
Show-GraphLayout -Graph $g1

#subset of log records filterd by degree of edges
$g2 = new-graph -Type BidirectionalGraph
$x = $g.Vertices.Where({$g.OutDegree($_) -gt 0})


$x | where {$_ -ne '192.168.0.107'} | % {$e = $g.InEdges($_); if ($e) {$e | % {add-edge -from $_.source -to $_.target -Graph $g2}}}
$x | where {$_ -ne '192.168.0.107'} | % {$e = $g.OutEdges($_); if ($e) {$e | % {add-edge -from $_.source -to $_.target -Graph $g2}}}

Show-GraphLayout -Graph $g2
Using graphs to analyze Windows Firewall logs

Let’s ‘graph’ it

So, after some time my  friends and I released the first version on a PSQuickGraph module. It is also available on GitHub. It is  to be able to analyze dependencies in a scripted way.

Let’s give it a try. First you need to do is to install it. To do it, just start PowerShell and issue the following command, which¬† installs¬† the module from PS¬† gallery


Install-Module PSQuickGraph -Scope CurrentUser

Next thing you need to do is to install  graphviz. It is a command line tool which takes the text file in a special format and makes a picture from it. You can take it from here and simply unzip in into a folder. In my case I use c:\temp\graphviz folder. Once it is there you can do some magic.

First –¬† start PowerShell and say:


Import-Module PSQuickGraph

Now just open this link and copy the contents to the PowerShell window. And in a second  you will see a your first graph made using PowerShell.

You can do a lot more with it,

Enjoy!

Let’s ‘graph’ it

Visio 2016 search is not working

Actually I have no idea how this works, probably I need to investigate deeper, but now I have not enough time tor that. So the issue was with the search, like here or here. Search simply was not working. I run win10 + Visio 2016 pro, all x64. I downloaded Azure and AWS  stencils, put them  to My Shapes folder under Documents and It sis not work.

I spent huge amount of time poking around, I was rebuilding indexes, restarting services, copying files. I even looked into the index with PowerShell. Files were there but Indexing did not work.

And as last resort I just copied and pasted all the files inside of the same directory:
C:\Program Files\Microsoft Office\root\Office16\Visio Content\1033. And suddenly it worked!

So the workaround was to:

  1. Copy all stencil files you need to a C:\Program Files\Microsoft Office\root\Office16\Visio Content\1033
  2. Select them all, copy (CTRL+C) and paste (CTRL+V) them once again in the same directory

And they immediately appear in the search! Don’t ask me why.

visioFilesCopied

visioFilesCopied2

Visio 2016 search is not working