Building a Vulnerability Data Source With Fastapi

Now that I am working in vulnerability management, I often have to use data from various sources to rank and prioritize vulnerabilities. One of those is the Known Exploited Vulnerabilities (KEV) list from the Cybersecurity and Infrastructure Security Agency (CISA). This list tracks vulnerabilities that are confirmed by CISA to have been exploited in the wild.

I built a project that pulls the KEV data and serves it through a FastAPI application so I can use it with other vulnerability management automation.

I first used the requests Python library to fetch the KEV data from the CISA website in JSON format.

# URL for the CISA KEV list
url = "https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json"

# Download the JSON data
response = requests.get(url)
if response.status_code != 200:
    print("Failed to download the KEV list.")
    exit(1)

kev_data = response.json()

For storing the downloaded data, I chose SQLite - a lightweight, file-based database system. This involved learning how to create a new table in SQLite and how to insert data into it. I also learned about ensuring that the operations are successful by using transactions and committing the changes to the database.

# Connect to SQLite database (it will create the file if it doesn't exist)
conn = sqlite3.connect("kev_list.db")
cursor = conn.cursor()

# Create a table to store the KEV list
cursor.execute("""
CREATE TABLE IF NOT EXISTS vulnerabilities (
    cveID TEXT PRIMARY KEY,
    vendorProject TEXT,
    product TEXT,
    vulnerabilityName TEXT,
    dateAdded TEXT,
    shortDescription TEXT,
    requiredAction TEXT,
    dueDate TEXT,
    notes TEXT
)
""")

# Insert the KEV data into the table
for item in kev_data['vulnerabilities']:
    cursor.execute("""
    INSERT INTO vulnerabilities (
        cveID,
        vendorProject,
        product,
        vulnerabilityName,
        dateAdded,
        shortDescription,
        requiredAction,
        dueDate,
        notes)
    VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
    """, (
        item['cveID'],
        item['vendorProject'],
        item['product'],
        item['vulnerabilityName'],
        item['dateAdded'],
        item['shortDescription'],
        item['requiredAction'],
        item['dueDate'],
        item.get('notes', '')  # Assuming 'notes' might be optional
    ))

# Commit the changes and close the connection
conn.commit()
conn.close()

To serve the file using a REST API, I used FastAPI, a modern web framework for building APIs with Python. Even if you’re not interested in building web apps with Python, the project’s documentation is worth admiring. I got up and running in minutes.

from fastapi import FastAPI, HTTPException
app = FastAPI()

@app.get("/vulnerability/{cve_id}", response_model=Vulnerability)
def read_vulnerability(cve_id: str):
    vulnerability = get_vulnerability_by_cve_id(cve_id)
    if vulnerability is None:
        raise HTTPException(status_code=404, detail="Vulnerability not found")
    
    return vulnerability

Finally I learned how to define a Pydantic model, which provides validation of the incoming data, serialization, and documentation out-of-the-box.

from pydantic import BaseModel

class Vulnerability(BaseModel):
    cveID: str
    vendorProject: str
    product: str
    vulnerabilityName: str
    dateAdded: str
    shortDescription: str
    requiredAction: str
    dueDate: str
    notes: str = None  # Assuming 'notes' might be optional

This project was a fantastic opportunity to learn about a number of Python’s libraries, FastAPI’s capabilities, and SQLite’s simplicity.