GitXplorerGitXplorer
n

ioc-extractor

public
55 stars
12 forks
4 issues

Commits

List of commits on branch master.
Verified
2eb89b6c1e5e69cbc4a3ec20c42e9f83c4eb252e

Merge pull request #344 from ninoseki/vitest-eslint-plugin

nninoseki committed 2 months ago
Unverified
3f76211ff26be033103ab8e77fd090babad8d93a

refactor: introduce @vitest/eslint-plugin

nninoseki committed 2 months ago
Verified
7df8993d5a0b788549120ca59eb75ee8afa20004

Merge pull request #343 from ninoseki/use-punycode.js

nninoseki committed 2 months ago
Unverified
59e350c96604bd8890ec7442dc4586008221e7dc

refactor: use punycode.js

nninoseki committed 2 months ago
Verified
c69c13b48f375eac6991fe70daceeb3620b70167

Merge pull request #342 from ninoseki/set-vitest-globals

nninoseki committed 2 months ago
Unverified
84abae6ed27acd2c1904c71dd32ecf58fd227ff0

fix: set vitest/globals as a type

nninoseki committed 2 months ago

README

The README file for this repository.

IoC extractor

npm version Node.js CI CodeFactor Coverage Status Documentation

IoC extractor is an npm package for extracting common IoC (Indicator of Compromise) from a block of text.

Note: the package is highly influenced by cacador.

Installation

npm install -g ioc-extractor
# or if you want to use ioc-extractor as a library in your JS/TS project
npm install ioc-extractor

Usage

As a CLI

$ ioc-extractor --help
Usage: ioc-extractor [options]

Options:
  --no-strict            Disable strict option
  --no-refang            Disable refang option
  --no-sort              Disable sort option
  -p, --punycode         Enable punycode option
  -o, --only <types...>  Show only specific IoC types
  -h, --help             display help for command
$ echo "1.1.1.1 8.8.8.8 example.com" | ioc-extractor | jq
{
  "asns": [],
  "btcs": [],
  "cves": [],
  "domains": [
    "example.com"
  ],
  "emails": [],
  "eths": [],
  "gaPubIDs": [],
  "gaTrackIDs": [],
  "ipv4s": [
    "1.1.1.1",
    "8.8.8.8"
  ],
  "ipv6s": [],
  "macAddresses": [],
  "md5s": [],
  "sha1s": [],
  "sha256s": [],
  "sha512s": [],
  "ssdeeps": [],
  "urls": [],
  "xmrs": []
}

$ echo "1.1.1.1 8.8.8.8" | ioc-extractor --only ipv4s | jq
{
  "ipv4s": [
    "1.1.1.1",
    "8.8.8.8"
  ]
}

As a Library

import { extractIOC } from "ioc-extractor";

const input = "1.1.1[.]1 google(.)com f6f8179ac71eaabff12b8c024342109b";
const ioc = extractIOC(input);
console.log(ioc.md5s);
// => ['f6f8179ac71eaabff12b8c024342109b']
console.log(ioc.ipv4s);
// => ['1.1.1.1']
console.log(ioc.domains);
// => ['google.com']

extractIOC takes the following options:

If you want to extract a specific type of IoC, you can use an extract function by IoC type.

import {
  refang,
  extractDomains,
  extractIPv4s,
  extractMD5s,
} from "ioc-extractor";

const input = "1.1.1[.]1 google(.)com f6f8179ac71eaabff12b8c024342109b";
const refanged = refang(input);
// => 1.1.1.1 google.com f6f8179ac71eaabff12b8c024342109b

const ipv4s = extractIPv4s(refanged);
// => ['1.1.1.1']

const domains = extractDomains(refanged);
// => ['google.com']

const md5s = extractMD5s(refanged);
// => ['f6f8179ac71eaabff12b8c024342109b']

Network related extract functions (e.g. extractDomains) can take the following options:

See docs for more details.

Alternatively, if you want to extract a list of specific IoC types at once, you can use partialExtractIOC.

import { partialExtractIOC } from "ioc-extractor";

const input = "1.1.1[.]1 google(.)com f6f8179ac71eaabff12b8c024342109b";
const ioc = partialExtractIOC(input, ["ipv4s", "domains"]);
console.log(ioc);
// => {"ipv4s":["1.1.1.1"],"domains":["google.com"]}

IoC Types

This package supports the following IoCs:

  • Hashes: MD5, SHA1, SHA256, SHA512, SSDEEP
  • Networks: domain, email, IPv4, IPv6, URL, ASN
  • Hardwares: MAC address
  • Utilities: CVE (CVE ID)
  • Cryptocurrencies: BTC (BTC address), ETH (ETH address), XMR (XMR address)
  • Trackers: GA track ID (Google Analytics tracking ID), GA pub ID (Google Adsense Publisher ID)

Refang Techniques

For Networks IoCs, the following refang techniques are supported:

Techniques Defanged Refanged
. in spaces 1.1.1 . 1 1.1.1.1
. in brackets, parentheses, etc. 1.1.1[.]1 1.1.1.1
dot in brackets, parentheses, etc. example[dot]com example.com
Back slash before . example\.com example.com
/ in brackets, parentheses, etc. http://example.com[/]path http://example.com/path
:// in brackets, parentheses, etc. http[://]example.com http://example.com
: in brackets, parentheses, etc. http[:]//example.com http://example.com
@ in brackets, parentheses, etc. test[@]example.com test@example.com
at in brackets, parentheses, etc. test[at]example.com test@example.com
hxxp hxxps://example.com https://example.com
Partial 1.1.1[.1 1.1.1.1
Any combination hxxps[:]//test\.example[.)com[/]path https://test.example.com/path

Options

strict

Whether to do strict TLD matching or not. Defaults to true.

refang

Whether to do refang or not. Defaults to false.

punycode

Whether to do Punycode conversion or not. Defaults to false.

sort

Whether to sort values or not. Defaults to true.

Alternatives