APT Detection
Full Recipe¶
Shared by: Ryan Wright
This APT (Advanced Persistent Threat) detection recipe ingests EDR (Endpoint Detection and Response) and network traffic logs, while monitoring for an IoB (Indicator of Behavior) that matches malicious data exfiltration patterns.
APT Recipe
1 | |
Scenario¶
In this scenario, a malicious Excel macro collects personal data and stores it in a temporary file. The APT process ntclean infiltrated the system previously through an SSH exploit, and now reads from that temporary file and exfiltrates data from the network hiding it as an HTTP GET request before deleting the temporary file to cover its tracks.
Using a standing query, the recipe monitors for covert interprocess communication using a file to pass data. When that pattern is matched, with a network SEND event, we have our smoking gun and a URL is logged linking to the Quine Exploration UI with the full activity and context for investigation.
The source of the SSH exploit that planted the APT and the destination for exfiltrated data utilize the same IP address.
Sample Data¶
Download the sample data to the same directory where Quine will be run.
endpoint.json- https://recipes.quine.io/apt-detection/endpoint-jsonnetwork.json- https://recipes.quine.io/apt-detection/network-json
How it Works¶
The recipe reads observations from the two sample data files using ingest streams to manifest a graph in Quine. A separate ingest stream is configured to process each file, each containing Cypher that parses the observations, manifests nodes, and relates them to each other in the graph.
INGEST-1 processes the endpoints.json file:
- type: FileIngest
path: endpoint.json
format:
type: CypherJson
query: >-
MATCH (proc), (event), (object)
WHERE id(proc) = idFrom($that.pid)
AND id(event) = idFrom($that)
AND id(object) = idFrom($that.object)
SET proc.id = $that.pid,
proc: Process,
event.type = $that.event_type,
event: EndpointEvent,
event.time = $that.time,
object.data = $that.object
CREATE (proc)-[:EVENT]->(event)-[:EVENT]->(object)
{
"type": "FileIngest",
"path": "endpoint.json",
"format": {
"type": "CypherJson",
"query": "MATCH (proc), (event), (object) WHERE id(proc) = idFrom($that.pid) AND id(event) = idFrom($that) AND id(object) = idFrom($that.object) SET proc.id = $that.pid, proc: Process, event.type = $that.event_type, event: EndpointEvent, event.time = $that.time, object.data = $that.object CREATE (proc)-[:EVENT]->(event)-[:EVENT]->(object)"
}
}
INGEST-2 processes the network.json file:
- type: FileIngest
path: network.json
format:
type: CypherJson
query: >-
MATCH (src), (dst), (event)
WHERE id(src) = idFrom($that.src_ip+":"+$that.src_port)
AND id(dst) = idFrom($that.dst_ip+":"+$that.dst_port)
AND id(event) = idFrom('network_event', $that)
SET src.ip = $that.src_ip+":"+$that.src_port,
src: IP,
dst.ip = $that.dst_ip+":"+$that.dst_port,
dst: IP,
event.proto = $that.proto,
event.time = $that.time,
event.detail = $that.detail,
event: NetTraffic
CREATE (src)-[:NET_TRAFFIC]->(event)-[:NET_TRAFFIC]->(dst)
[
{
"type": "FileIngest",
"path": "network.json",
"format": {
"type": "CypherJson",
"query": "MATCH (src), (dst), (event) WHERE id(src) = idFrom($that.src_ip+\":\"+$that.src_port)\n AND id(dst) = idFrom($that.dst_ip+\":\"+$that.dst_port)\n AND id(event) = idFrom('network_event', $that)\n\nSET src.ip = $that.src_ip+\":\"+$that.src_port,\n src: IP,\n dst.ip = $that.dst_ip+\":\"+$that.dst_port,\n dst: IP,\n event.proto = $that.proto,\n event.time = $that.time,\n event.detail = $that.detail,\n event: NetTraffic\n\nCREATE (src)-[:NET_TRAFFIC]->(event)-[:NET_TRAFFIC]->(dst)"
}
}
]
A standing query is configured to detect a WRITE->READ->SEND->DELETE pattern that is typical for this type of exflitration event.
- pattern:
type: Cypher
query: >-
MATCH (e1)-[:EVENT]->(f)<-[:EVENT]-(e2),
(f)<-[:EVENT]-(e3)<-[:EVENT]-(p2)-[:EVENT]->(e4)
WHERE e1.type = "WRITE"
AND e2.type = "READ"
AND e3.type = "DELETE"
AND e4.type = "SEND"
RETURN DISTINCT id(f) as fileId
[
{
"pattern": {
"type": "Cypher",
"query": "MATCH (e1)-[:EVENT]->(f)<-[:EVENT]-(e2), \n (f)<-[:EVENT]-(e3)<-[:EVENT]-(p2)-[:EVENT]->(e4)\nWHERE e1.type = \"WRITE\"\n AND e2.type = \"READ\"\n AND e3.type = \"DELETE\"\n AND e4.type = \"SEND\"\nRETURN DISTINCT id(f) as fileId"
},
"outputs": {
"stolen-data": {
"type": "CypherQuery",
"query": "MATCH (p1)-[:EVENT]->(e1)-[:EVENT]->(f)<-[:EVENT]-(e2)<-[:EVENT]-(p2), \n (f)<-[:EVENT]-(e3)<-[:EVENT]-(p2)-[:EVENT]->(e4)-[:EVENT]->(ip)\nWHERE id(f) = $that.data.fileId\n AND e1.type = \"WRITE\"\n AND e2.type = \"READ\"\n AND e3.type = \"DELETE\"\n AND e4.type = \"SEND\"\n AND e1.time < e2.time\n AND e2.time < e3.time\n AND e2.time < e4.time\n\nCREATE (e1)-[:NEXT]->(e2)-[:NEXT]->(e4)-[:NEXT]->(e3)\nWITH e1, e2, e3, e4, p1, p2, f, ip, \"http://localhost:8080/#MATCH\" + text.urlencode(\" (e1),(e2),(e3),(e4),(p1),(p2),(f),(ip) WHERE id(p1)='\"+strId(p1)+\"' AND id(e1)='\"+strId(e1)+\"' AND id(f)='\"+strId(f)+\"' AND id(e2)='\"+strId(e2)+\"' AND id(p2)='\"+strId(p2)+\"' AND id(e3)='\"+strId(e3)+\"' AND id(e4)='\"+strId(e4)+\"' AND id(ip)='\"+strId(ip)+\"' RETURN e1, e2, e3, e4, p1, p2, f, ip\") as URL RETURN URL",
"andThen": {
"type": "PrintToStandardOut"
}
}
}
}
]
Once Quine detects the pattern, the event is sent to a standing query output for additional processing and action.
outputs:
stolen-data:
type: CypherQuery
query: >-
MATCH (p1)-[:EVENT]->(e1)-[:EVENT]->(f)<-[:EVENT]-(e2)<-[:EVENT]-(p2),
(f)<-[:EVENT]-(e3)<-[:EVENT]-(p2)-[:EVENT]->(e4)-[:EVENT]->(ip)
WHERE id(f) = $that.data.fileId
AND e1.type = "WRITE"
AND e2.type = "READ"
AND e3.type = "DELETE"
AND e4.type = "SEND"
AND e1.time < e2.time
AND e2.time < e3.time
AND e2.time < e4.time
CREATE (e1)-[:NEXT]->(e2)-[:NEXT]->(e4)-[:NEXT]->(e3)
WITH e1, e2, e3, e4, p1, p2, f, ip, "http://localhost:8080/#MATCH" + text.urlencode(" (e1),(e2),(e3),(e4),(p1),(p2),(f),(ip) WHERE id(p1)='"+strId(p1)+"' AND id(e1)='"+strId(e1)+"' AND id(f)='"+strId(f)+"' AND id(e2)='"+strId(e2)+"' AND id(p2)='"+strId(p2)+"' AND id(e3)='"+strId(e3)+"' AND id(e4)='"+strId(e4)+"' AND id(ip)='"+strId(ip)+"' RETURN e1, e2, e3, e4, p1, p2, f, ip") as URL
RETURN URL
andThen:
type: PrintToStandardOut
The result once the pattern is detected is to output a link to the console that an analyst can use to review the event further within Quine's Exploration UI.
1 | |
Running the Recipe¶
❯ java -jar quine-1.8.2.jar -r apt-detection.yaml
Graph is ready
Running Recipe: APT Detection
Using 5 node appearances
Using 14 quick queries
Running Standing Query STANDING-1
Running Ingest Stream INGEST-1
Running Ingest Stream INGEST-2
Quine web server available at http://localhost:8080
Summary¶
When the standing query detects the WRITE->READ->SEND->DELETE pattern, it will output a link to the console that can be copied and pasted into a browser to explore the event in the Quine Exploration UI. Copy and paste the URL section of the match JSON from your console into your browser.
The nodes will be jumbled together when you first open the graph. Arrange the nodes to look similar to the image below before you start exploring.
The recipe includes a number of quick queries to assist in exploring the event data. Right click on a node to bring up the quick query interface.
Tip
Quick Queries are available by right clicking on a node.
| Quick Query | Node Type | Description |
|---|---|---|
| Adjacent Nodes | All | Display the nodes that are adjacent to this node. |
| Refresh | All | Refresh the content stored in a node |
| Local Properties | All | Display the properties stored by the node |
| Files Read | Process | Load nodes representing the files read by this process |
| Files Written | Process | Load nodes representing files written by this process |
| Read By | data | Load nodes that read data from this node |
| Written By | data | Load nodes that wrote to this node |
| Received Data | Process | Where did this process receive data from |
| Sent Data | Process | Where did this process send data to |
| Started By | Process | What started this process |
| Started Other Process | Process | What process did this process start |
| Network Send | IP | Where did this IP node send data |
| Network Receive | IP | Whre did this IP node receive data from |
| Network Communication | IP | What other nodes did this IP node communicate with |
Use the quick queries to explore the graph and uncover the timeline behind the entire event.