Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Nikto boefje that scans for outdated software #3409

Open
wants to merge 42 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
16726b1
Made base format for new boefje
Souf149 Jul 31, 2024
bc3a7c8
Fixed issue of nikto boefje not contacting boefje api
Souf149 Aug 7, 2024
09882a6
Added `oci_adapter.js` to better mirror the python's implementation
Souf149 Aug 13, 2024
022ac4e
generic cve normalier
noamblitz Aug 19, 2024
7572d1e
switch to generic cve normalizer
noamblitz Aug 19, 2024
a1765f1
also output orignal boefje output
noamblitz Aug 19, 2024
6f84904
update docs on generic cve normalizer
noamblitz Aug 19, 2024
0ce1f35
Merge branch 'main' into generic-cve-normalizer
noamblitz Aug 20, 2024
0e057bb
Cleaned up code
Souf149 Aug 22, 2024
50e85ae
Added normalizer to kat_nikto boefje
Souf149 Aug 22, 2024
6af750e
Update boefjes/boefjes/plugins/kat_cve_normalizer/normalizer.json
noamblitz Aug 22, 2024
7b62c25
Update boefjes/boefjes/plugins/kat_cve_normalizer/normalizer.json
noamblitz Aug 22, 2024
45c000f
Update docs/source/developer_documentation/development_tutorial/creat…
noamblitz Aug 22, 2024
ac6ca62
Update boefjes/boefjes/plugins/kat_cve_2023_34039/main.py
noamblitz Aug 22, 2024
5ae981e
allow comma seperated list of all findingtypes
noamblitz Aug 22, 2024
60570dd
Merge branch 'main' into generic-cve-normalizer
noamblitz Aug 22, 2024
09ea726
Added new OOI that has information about outdated software
Souf149 Aug 23, 2024
1322be6
use direct mapping
noamblitz Aug 23, 2024
d57565a
Rewrote `oci_adapter.js` to work like `oci_adapter.py` does
Souf149 Aug 23, 2024
0712f4d
Fixed example url
Souf149 Aug 23, 2024
b3a3252
Made base format for new boefje
Souf149 Jul 31, 2024
f8d3294
Fixed issue of nikto boefje not contacting boefje api
Souf149 Aug 7, 2024
4d790c7
Added `oci_adapter.js` to better mirror the python's implementation
Souf149 Aug 13, 2024
d4f92b7
Cleaned up code
Souf149 Aug 22, 2024
6d39582
Added normalizer to kat_nikto boefje
Souf149 Aug 22, 2024
723fd55
Added new OOI that has information about outdated software
Souf149 Aug 23, 2024
13ca4de
Rewrote `oci_adapter.js` to work like `oci_adapter.py` does
Souf149 Aug 23, 2024
cb8d136
Fixed example url
Souf149 Aug 23, 2024
0e74dac
Merge branch 'generic-cve-normalizer' into `feature/nikto-boefje`
Souf149 Aug 23, 2024
2d96eff
cleaned up dockerfiles to remove comments
Souf149 Aug 23, 2024
b050735
Removed new OOI type and sent the nikto message straight to the finding
Souf149 Aug 27, 2024
9954e62
Removed unused variable
Souf149 Aug 27, 2024
0ace647
Added new finding type and use this one instead for nikto boefje
Souf149 Aug 27, 2024
6858fc7
Merge branch 'main' into feature/nikto-boefje
Souf149 Aug 28, 2024
56df40a
Merge branch 'main' into feature/nikto-boefje
Souf149 Aug 28, 2024
491247d
Fixed merging mistake
Souf149 Aug 28, 2024
ef25331
Updated dev's cat'
Souf149 Aug 28, 2024
c6e90f1
Added tests for nikto normalizer
Souf149 Aug 28, 2024
c6647da
Refactored code
Souf149 Aug 28, 2024
0c169ba
Made nikto boefje run on `HostnameHTTPURL`
Souf149 Aug 30, 2024
346e170
Added env variables to nikto boefje
Souf149 Sep 4, 2024
8d5cba0
Moved config file creation to a seperate function
Souf149 Sep 4, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion boefjes/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ images: # Build the images for the containerized boefjes
# docker build -f images/base.Dockerfile -t ghcr.io/minvws/openkat/dns-records --build-arg BOEFJE_PATH=./boefjes/plugins/kat_dns .
docker build -f ./boefjes/plugins/kat_dnssec/boefje.Dockerfile -t ghcr.io/minvws/openkat/dns-sec:latest .
docker build -f ./boefjes/plugins/kat_nmap_tcp/boefje.Dockerfile -t ghcr.io/minvws/openkat/nmap:latest .

docker build -f ./boefjes/plugins/kat_nikto/boefje.Dockerfile -t openkat/nikto .

##
##|------------------------------------------------------------------------|
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -486,5 +486,11 @@
"source": "https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers",
"impact": "Nonstandard headers may not be supported by all browsers and may not provide the security that is expected.",
"recommendation": "Remove the nonstandard headers from the response."
},
"KAT-OUTDATED-SOFTWARE": {
"description": "A newer version of existing software has been found.",
"risk": "recommendation",
"impact": "Depending on what software is outdated this can be critical.",
"recommendation": "Inspect the software version, determine if additional measures need to be taken and install updates to reduce the attack surface."
}
}
Empty file.
13 changes: 13 additions & 0 deletions boefjes/boefjes/plugins/kat_nikto/boefje.Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
FROM perl:5.40

WORKDIR /app
RUN apt update
RUN apt install -y git
RUN apt install -y nodejs

RUN git clone https://github.com/sullo/nikto

ARG BOEFJE_PATH=./boefjes/plugins/kat_nikto
COPY $BOEFJE_PATH ./

ENTRYPOINT [ "node", "./" ]
18 changes: 18 additions & 0 deletions boefjes/boefjes/plugins/kat_nikto/boefje.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
{
"id": "nikto",
"name": "Nikto",
"description": "Uses Nikto",
"consumes": [
"HostnameHTTPURL"
],
"environment_keys": [
"PROXYHOST",
"PROXYPORT",
"PROXYUSER",
"PROXYPASS",
Comment on lines +9 to +12
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See earlier remark

"USERAGENT"
],
"scan_level": 3,
"oci_image": "openkat/nikto",
"oci_arguments": []
}
Binary file added boefjes/boefjes/plugins/kat_nikto/cover.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
15 changes: 15 additions & 0 deletions boefjes/boefjes/plugins/kat_nikto/description.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Nikto

Nikto 2.5 is an Open Source (GPL) web server scanner which performs comprehensive tests against web servers for multiple items, including over 7,000 potentially dangerous files/programs, checks for outdated versions of over 1250 servers, and version specific problems on over 270 servers. It also checks for server configuration items such as the presence of multiple index files, HTTP server options, and will attempt to identify installed web servers and software. Scan items and plugins are frequently updated and can be automatically updated.

(taken from [CIRT.net](https://cirt.net/Nikto2))

### Input OOIs

Nikto expects an IpAddress or a Hostname OOI.

### Output OOIs

Nikto outputs the following the found outdated software and a finding explaining that the software is outdated.

**Cat name**: Kitty
98 changes: 98 additions & 0 deletions boefjes/boefjes/plugins/kat_nikto/main.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
import fs from "node:fs";
import { execSync } from "node:child_process";

/**
* @param {string} scheme
* @returns {string}
*/
function get_config_content(scheme) {
const IS_USING_PROXY = !!process.env.PROXYHOST;

// Setup config file
try {
let config_contents =
"PROMPTS=no\nUPDATES=no\nCLIOPTS=-404code=301,302,307,308 -o ./output.json";

if (scheme == "https") config_contents += " -ssl";
if (IS_USING_PROXY) config_contents += " -useproxy";
config_contents += "\n";

if (IS_USING_PROXY) {
const PROXY_HOST = process.env.PROXYHOST;
const PROXY_PORT = process.env.PROXYPORT || "8080";
const PROXY_USER = process.env.PROXYUSER || "";
const PROXY_PASS = process.env.PROXYPASS || "";

config_contents += `PROXYHOST=${PROXY_HOST}\n`;
config_contents += `PROXYPORT=${PROXY_PORT}\n`;
config_contents += `PROXYUSER=${PROXY_USER}\n`;
config_contents += `PROXYPASS=${PROXY_PASS}\n`;
}

if (process.env.USERAGENT)
config_contents += `USERAGENT=${process.env.USERAGENT}\n`;

return config_contents;
} catch (e) {
throw new Error("Something went wrong writing to the config file.\n" + e);
}
}

/**
* @param {Object} boefje_meta Information about the task
* @param {Object} boefje_meta.arguments
* @param {Object} boefje_meta.arguments.input
* @param {string} boefje_meta.arguments.input.object_type
* @param {"http" | "https"} boefje_meta.arguments.input.scheme
* @param {number} boefje_meta.arguments.input.port
* @param {Object} boefje_meta.arguments.input.netloc
* @param {string} boefje_meta.arguments.input.netloc.name
* @returns {(string | string[])[][]}
*/
export default function (boefje_meta) {
// Depending on what OOI triggered this task, the hostname / address will be in a different location
const hostname = boefje_meta.arguments.input.netloc.name;

const config_contents = get_config_content(
boefje_meta.arguments.input.scheme,
);
fs.writeFileSync("./nikto.conf", config_contents);

// Running nikto and outputting to a file
try {
execSync(`./nikto/program/nikto.pl -h ${hostname} -config ./nikto.conf`, {
stdio: "inherit",
});
} catch (e) {
throw new Error(
"Something went wrong running the nikto command.\n" +
e +
"\n" +
config_contents,
);
}

const raws = [];

// Reading the file created by nikto
try {
var file_contents = fs.readFileSync("./output.json").toString();
raws.push([["boefje/nikto-output"], file_contents]);
} catch (e) {
throw new Error(
"Something went wrong reading the file from the nikto command.\n" + e,
);
}

// Looking if outdated software has been found
try {
const data = JSON.parse(file_contents);
for (const vulnerability of data["vulnerabilities"])
if (vulnerability["id"].startsWith("6"))
raws.push([["openkat/finding"], "KAT-OUTDATED-SOFTWARE"]);
} catch (e) {
console.error(e);
}

return raws;
}
38 changes: 38 additions & 0 deletions boefjes/boefjes/plugins/kat_nikto/normalize.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
import json
from collections.abc import Iterable

from boefjes.job_models import NormalizerOutput
from octopoes.models import Reference
from octopoes.models.ooi.findings import Finding, KATFindingType
from octopoes.models.ooi.software import Software, SoftwareInstance


def scan_outdated_software(data: dict, ooi_ref: Reference) -> Iterable[NormalizerOutput]:
for scan in data:
for vulnerability in scan["vulnerabilities"]:
# If the scanned vulnerability has to do with outdated software
if vulnerability["id"].startswith("6"):
# Example of `vulnerability["msg"]`
# @SOFTWARE/@RUNNING_VER appears to be outdated (current is at least @CURRENT_VER)
software_name, found_version = vulnerability["msg"].split()[0].split("/")

software = Software(name=software_name, version=found_version)
software_instance = SoftwareInstance(ooi=ooi_ref, software=software.reference)
yield software
yield software_instance

finding_type = KATFindingType(id="KAT-OUTDATED-SOFTWARE")
yield finding_type
yield Finding(
finding_type=finding_type.reference,
ooi=software_instance.reference,
description=vulnerability["msg"],
)


def run(input_ooi: dict, raw: bytes) -> Iterable[NormalizerOutput]:
data = json.loads(raw)

ooi_ref = Reference.from_str(input_ooi["primary_key"])

yield from scan_outdated_software(data, ooi_ref)
10 changes: 10 additions & 0 deletions boefjes/boefjes/plugins/kat_nikto/normalizer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"id": "kat_nikto_normalize",
"consumes": [
"boefje/nikto-output"
],
"produces": [
"Software",
"SoftwareInstance"
]
}
64 changes: 64 additions & 0 deletions boefjes/boefjes/plugins/kat_nikto/oci_adapter.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
import { execSync } from "node:child_process";
import run from "./main.js";

/**
* @param {string} inp The string input to base64
* @returns {string}
*/
function b64encode(inp) {
return Buffer.from(inp).toString("base64");
}

function main() {
const input_url = process.argv[process.argv.length - 1];

// Getting the boefje input
try {
var boefje_input = JSON.parse(
execSync(`curl --request GET --url ${input_url}`).toString(),
);
} catch (error) {
console.error(`Getting boefje input went wrong with URL: ${input_url}`);
throw new Error(error);
}

Object.assign(process.env, boefje_input["boefje_meta"]["environment"]);

let out = undefined;
let output_url = boefje_input.output_url;
try {
// Getting the raw files
const raws = run(boefje_input.boefje_meta);
out = {
status: "COMPLETED",
files: raws.map((x) => ({
content: b64encode(x[1]),
tags: x[0],
})),
};
} catch (error) {
out = {
status: "FAILED",
files: [
{
content: b64encode("Boefje caught an error: " + error.message),
tags: ["error/boefje"],
},
],
};
}

// Example command
/*
curl --request POST \
--url http://boefje:8000/api/v0/tasks/7342e8dd-b945-4185-aaec-787205b7b664 \
--header 'Content-Type: application/json' \
--data '{"status":"COMPLETED","files":[{"content":"BASE_64_ENCODED_CONTENT","tags":[]}]}'
*/
const out_json = JSON.stringify(out);
const cmd = `curl --request POST --url ${output_url} --header "Content-Type: application/json" --data '${out_json}'`;

execSync(cmd);
}

main();
43 changes: 43 additions & 0 deletions boefjes/boefjes/plugins/kat_nikto/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 10 additions & 0 deletions boefjes/boefjes/plugins/kat_nikto/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{
"name": "kat-nikto",
"version": "1.0.0",
"type": "module",
"main": "oci_adapter.js",
"author": "cynalytics",
"dependencies": {
"@types/node": "^22.1.0"
}
}
32 changes: 32 additions & 0 deletions boefjes/boefjes/plugins/kat_nikto/schema.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
{
"title": "Arguments",
"type": "object",
"properties": {
"PROXYHOST": {
"title": "PROXYHOST",
"maxLength": 256,
"type": "string"
},
"PROXYPORT": {
"title": "PROXYPORT",
"maxLength": 256,
"type": "string"
},
"PROXYUSER": {
"title": "PROXYUSER",
"maxLength": 256,
"type": "string"
},
"PROXYPASS": {
"title": "PROXYPASS",
"maxLength": 256,
"type": "string"
},
"USERAGENT": {
"title": "USERAGENT",
"maxLength": 256,
"type": "string"
}
Comment on lines +5 to +29
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps you could use the HTTP_PROXY (and optionally HTTPS_PROXY) for this? This is a common proxy URI configuration. This will save a few variables and make the form less complex (given that it has a good example, e.g. http://user:[email protected]:3128/

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are config variables required for Nikto. If we would make the boefje's schema less complex, then it would move the complexity to the code of the boefje.

I believe since the user chooses to activate the Nikto boefje, they would know that Nikto requires these variables which would make them self-explaining.

},
"required": []
}
23 changes: 23 additions & 0 deletions boefjes/tests/examples/raw/nikto-example.com.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
[
{
"host": "example.com",
"ip": "178.128.108.228",
"port": "80",
"banner": "",
"vulnerabilities": [
{
"id": "600575",
"method": "HEAD",
"url": "/",
"msg": "nginx/1.18.0 appears to be outdated (current is at least 1.25.3)."
},
{
"id": "999103",
"references": "https://www.netsparker.com/web-vulnerability-scanner/vulnerabilities/missing-content-type-header/",
"method": "GET",
"url": "/",
"msg": "The X-Content-Type-Options header is not set. This could allow the user agent to render the content of the site in a different fashion to the MIME type."
}
]
}
]
Loading
Loading