feeds2stix

Overview

This repository contains processors that take data from threat intelligence feeds and convert them it into STIX 2.1 objects.

It also includes GitHub actions to upload the data from the feeds to the Cyber Threat Exchange.

Before you begin...

You can access all of the data generated by processors in this repository on the Cyber Threat Exchange.

Structure of this repository

The key parts of this repository are structured as follows;

.
├── processors/
│   ├── feed1/
│   │	├── README.md # describes the mapping of the feed.py files
│   │   └── feed.py
│   └── feed2/
│    	├── README.md
│    	└── feed.py 
└── bundles/ # will only exist after one script has been run
    ├── feed1/
    │	└── feed1
    │		└── bundle.json # multiple bundles can be produced for a single feed
    └── feed2
    		└── bundle.json

The processors directory contains the scripts that generate the data. These scripts output the data they create into the bundles directory (this directory will only exist once you run one of the processor scripts).

Setup

Installing the script;

# clone the latest code
git clone https://github.com/muchdogesec/feeds2stix
# create a venv
cd feeds2stix
python3 -m venv feeds2stix-venv
source feeds2stix-venv/bin/activate
# install requirements
pip3 install -r requirements.txt
export PYTHONPATH=.

You can then run each script as shown in that feeds README.md file.

Github actions note

Cyber Threat Exchange has a maximum upload file size.

To avoid issues with large files, the Github actions split the bundles in 10mb chunks.

Useful supporting tools

• This repository contains Open Source freely usable Threat Intel feeds that can be used without additional requirements. Contains multiple types such as IP, URL, CVE and Hash.

Support

Minimal support provided via the dogesec community.

License

Apache 2.0.