Can't get Suricata to work offline with PCAP

[amrogers3]

Can't get Suricata to work, been researching and messing with the conf file for two days. I have been wanting to get experience with network forensics using the NETRESEC tutorials and running pcaps through Suricata using -r option.

suricata -r /opt/samples/zeus-sample-1.pcap -c /etc/nsm/sans-virtual-machine-eth1/suricata.yaml

Here are the multiple errors I am getting. Googling just leads me to dead ends. I tried using --list-runmodes and that didnt work either.

18/2/2018 -- 13:20:00 - <Error> - [ERRCODE: SC_ERR_DUPLICATE_SIG(176)] - Duplicate signature "alert tcp $EXTERNAL_NET any -> $HOME_NET $HTTP_PORTS (msg:"SERVER-WEBAPP Cisco DDR2200 ADSL gateway command injection attempt"; flow:to_server,established; content:"waitPingqry"; fast_patternnly; http_uri; content:"pingAddr="; nocase; http_uri; content:"%26"; http_raw_uri; pcre:"/[?&]pingAddr=[&]*?%26/Ii"; metadataolicy max-detect-ips drop, policy security-ips drop, ruleset community, service http; reference:url,nvd.nist.gov/vuln/detail/CVE-2017-11588; reference:url,seclists.org/fulldisclosure/2017/Jul/26; classtype:web-application-attack; sid:44006; rev:2"

18/2/2018 -- 13:20:00 - <Error> - [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET any -> $HOME_NET $HTTP_PORTS (msg:"SERVER-WEBAPP Cisco DDR2200 ADSL gateway command injection attempt"; flow:to_server,established; content:"waitPingqry"; fast_patternnly; http_uri; content:"pingAddr="; nocase; http_uri; content:"%26"; http_raw_uri; pcre:"/[?&]pingAddr=[&]*?%26/Ii"; metadataolicy max-detect-ips drop, policy security-ips drop, ruleset community, service http; reference:url,nvd.nist.gov/vuln/detail/CVE-2017-11588; reference:url,seclists.org/fulldisclosure/2017/Jul/26; classtype:web-application-attack; sid:44006; rev:2" from file /etc/nsm/rules/downloaded.rules at line

35748 18/2/2018 -- 13:20:00 - <Error> - [ERRCODE: SC_ERR_DUPLICATE_SIG(176)] - Duplicate signature "alert tcp $EXTERNAL_NET any -> $HOME_NET $HTTP_PORTS (msg:"SERVER-WEBAPP Cisco DDR2200 ADSL gateway command injection attempt"; flow:to_server,established; content:"waitPingqry"; fast_patternnly; http_uri; content:"pingAddr="; nocase; http_client_body; pcre:"/(|&)pingAddr=[&]*?([\x60\x3b\x7c]|[\x3c\x3e\x24]\x28|%60|%3b|%7c|%26|%3c%28|%3e%28|%24%28)/Pim"; metadataolicy max-detect-ips drop, policy security-ips drop, ruleset community, service http; reference:url,nvd.nist.gov/vuln/detail/CVE-2017-11588; reference:url,seclists.org/fulldisclosure/2017/Jul/26; classtype:web-application-attack; sid:44007; rev:2"

18/2/2018 -- 13:20:00 - <Error> - [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET any -> $HOME_NET $HTTP_PORTS (msg:"SERVER-WEBAPP Cisco DDR2200 ADSL gateway command injection attempt"; flow:to_server,established; content:"waitPingqry"; fast_patternnly; http_uri; content:"pingAddr="; nocase; http_client_body; pcre:"/(|&)pingAddr=[&]*?([\x60\x3b\x7c]|[\x3c\x3e\x24]\x28|%60|%3b|%7c|%26|%3c%28|%3e%28|%24%28)/Pim"; metadataolicy max-detect-ips drop, policy security-ips drop, ruleset community, service http; reference:url,nvd.nist.gov/vuln/detail/CVE-2017-11588; reference:url,seclists.org/fulldisclosure/2017/Jul/26; classtype:web-application-attack; sid:44007; rev:2" from file /etc/nsm/rules/downloaded.rules at line 35749

18/2/2018 -- 13:20:00 - <Error> - [ERRCODE: SC_ERR_DUPLICATE_SIG(176)] - Duplicate signature "alert tcp $EXTERNAL_NET any -> $HOME_NET $HTTP_PORTS (msg:"SERVER-WEBAPP Cisco DDR2200 ADSL gateway command injection attempt"; flow:to_server,established; content:"waitPingqry"; fast_patternnly; http_uri; content:"pingAddr"; nocase; http_client_body; content:"Content-Disposition"; nocase; http_client_body; pcre:"/name\s=\s[\x22\x27]?pingAddr((?!--).)?[\r\n]{2,}((?!--).)?([\x60\x3b\x7c\x26]|[\x3c\x3e\x24]\x28)/Psim"; metadataolicy max-detect-ips drop, policy security-ips drop, ruleset community, service http; reference:url,seclists.org/fulldisclosure/2017/Jul/26; classtype:web-application-attack; sid:44008; rev:2"

18/2/2018 -- 13:20:00 - <Error> - [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET any -> $HOME_NET $HTTP_PORTS (msg:"SERVER-WEBAPP Cisco DDR2200 ADSL gateway command injection attempt"; flow:to_server,established; content:"waitPingqry"; fast_patternnly; http_uri; content:"pingAddr"; nocase; http_client_body; content:"Content-Disposition"; nocase; http_client_body; pcre:"/name\s=\s[\x22\x27]?pingAddr((?!--).)?[\r\n]{2,}((?!--).)?([\x60\x3b\x7c\x26]|[\x3c\x3e\x24]\x28)/Psim"; metadataolicy max-detect-ips drop, policy security-ips drop, ruleset community, service http; reference:url,seclists.org/fulldisclosure/2017/Jul/26; classtype:web-application-attack; sid:44008; rev:2" from file /etc/nsm/rules/downloaded.rules at line 35750

18/2/2018 -- 13:20:00 - <Error> - [ERRCODE: SC_ERR_DUPLICATE_SIG(176)] - Duplicate signature "alert tcp $EXTERNAL_NET any -> $HOME_NET $HTTP_PORTS (msg:"SERVER-WEBAPP Asus RT-AC88U deleteOfflineClients memory corruption attempt"; flow:to_server,established; urilen:>64; content:"/deleteOfflineClient.cgi"; fast_patternnly; http_uri; content:"delete_offline_client="; http_uri; pcre:"/[?&]delete_offline_client=[&]{14}/Ui"; metadataolicy max-detect-ips drop, policy security-ips drop, ruleset community, service http; reference:cve,2017-12754; classtype:attempted-admin; sid:45412; rev:3"

18/2/2018 -- 13:20:00 - <Error> - [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET any -> $HOME_NET $HTTP_PORTS (msg:"SERVER-WEBAPP Asus RT-AC88U deleteOfflineClients memory corruption attempt"; flow:to_server,established; urilen:>64; content:"/deleteOfflineClient.cgi"; fast_patternnly; http_uri; content:"delete_offline_client="; http_uri; pcre:"/[?&]delete_offline_client=[&]{14}/Ui"; metadataolicy max-detect-ips drop, policy security-ips drop, ruleset community, service http; reference:cve,2017-12754; classtype:attempted-admin; sid:45412; rev:3" from file /etc/nsm/rules/downloaded.rules at line 35950

18/2/2018 -- 13:20:00 - <Error> - [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - previous keyword has a fast_patternnly; set. Can't have relative keywords around a fast_pattern only content

18/2/2018 -- 13:20:00 - <Error> - [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET any -> $HOME_NET $HTTP_PORTS (msg:"SERVER-WEBAPP DotNetNuke DNNPersonalization remote code execution attempt"; flow:to_server,established; content:"DNNPersonalization"; fast_patternnly; content:"DNNPersonalization"; http_cookie; content:"System.Data.Services.Internal.ExpandedWrapper"; within:100; http_cookie; content:"System.Windows.Data.ObjectDataProvider"; within:200; http_cookie; metadataolicy max-detect-ips drop, policy security-ips drop, service http; reference:cve,2017-9822; reference:url,www.dnnsoftware.com/community/security/security-center; classtype:attempted-admin; sid:45414; rev:2" from file /etc/nsm/rules/downloaded.rules at line 35952

18/2/2018 -- 13:20:00 - <Error> - [ERRCODE: SC_ERR_DUPLICATE_SIG(176)] - Duplicate signature "alert tcp $EXTERNAL_NET any -> $HOME_NET $HTTP_PORTS (msg:"SQL generic convert injection attempt - GET parameter"; flow:to_server,established; content:"convert|28|"; fast_patternnly; http_uri; metadataolicy max-detect-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.securiteam.com/securityreviews/5DP0N1P76E.html; classtype:web-application-attack; sid:26925; rev:2"

18/2/2018 -- 13:20:00 - <Error> - [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET any -> $HOME_NET $HTTP_PORTS (msg:"SQL generic convert injection attempt - GET parameter"; flow:to_server,established; content:"convert|28|"; fast_patternnly; http_uri; metadataolicy max-detect-ips drop, policy security-ips drop, ruleset community, service http; reference:url,www.securiteam.com/securityreviews/5DP0N1P76E.html; classtype:web-application-attack; sid:26925; rev:2" from file /etc/nsm/rules/downloaded.rules at line 36143

18/2/2018 -- 13:20:00 - <Error> - [ERRCODE: SC_ERR_DUPLICATE_SIG(176)] - Duplicate signature "alert tcp any any -> any $HTTP_PORTS (msg:"SQL use of sleep function in HTTP header - likely SQL injection attempt"; flow:established,to_server; content:"User-Agent|3A| "; http_header; content:"sleep("; within:200; fast_pattern; http_header; pcre:"/User-Agent\x3A\x20[\r\n]*sleep\x28/H"; metadataolicy balanced-ips drop, policy max-detect-ips drop, policy security-ips drop, ruleset community, service http; reference:url,blog.cloudflare.com/the-sleepy-user-agent/; classtype:web-application-attack; sid:38993; rev:8"

18/2/2018 -- 13:20:00 - <Error> - [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp any any -> any $HTTP_PORTS (msg:"SQL use of sleep function in HTTP header - likely SQL injection attempt"; flow:established,to_server; content:"User-Agent|3A| "; http_header; content:"sleep("; within:200; fast_pattern; http_header; pcre:"/User-Agent\x3A\x20[\r\n]*sleep\x28/H"; metadataolicy balanced-ips drop, policy max-detect-ips drop, policy security-ips drop, ruleset community, service http; reference:url,blog.cloudflare.com/the-sleepy-user-agent/; classtype:web-application-attack; sid:38993; rev:8" from file /etc/nsm/rules/downloaded.rules at line 36161

18/2/2018 -- 13:20:00 - <Error> - [ERRCODE: SC_ERR_DUPLICATE_SIG(176)] - Duplicate signature "alert tcp $SQL_SERVERS 1433 -> $EXTERNAL_NET any (msg:"SQL sa login failed"; flow:to_client,established; content:"Login failed for user 'sa'"; fast_patternnly; metadataolicy balanced-ips drop, policy connectivity-ips drop, policy security-ips drop, ruleset community; reference:bugtraq,4797; reference:cve,2000-1209; reference:nessus,10673; classtype:unsuccessful-user; sid:688; rev:16"

18/2/2018 -- 13:20:00 - <Error> - [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $SQL_SERVERS 1433 -> $EXTERNAL_NET any (msg:"SQL sa login failed"; flow:to_client,established; content:"Login failed for user 'sa'"; fast_patternnly; metadataolicy balanced-ips drop, policy connectivity-ips drop, policy security-ips drop, ruleset community; reference:bugtraq,4797; reference:cve,2000-1209; reference:nessus,10673; classtype:unsuccessful-user; sid:688; rev:16" from file /etc/nsm/rules/downloaded.rules at line 36177

18/2/2018 -- 13:20:06 - <Error> - [ERRCODE: SC_ERR_FOPEN(44)] - failed to open /var/log/nsm//snort.unified2.1518960006: Permission denied

18/2/2018 -- 13:20:06 - <Warning> - [ERRCODE: SC_ERR_INVALID_ARGUMENT(13)] - output module setup failed

18/2/2018 -- 13:20:06 - <Error> - [ERRCODE: SC_ERR_FOPEN(44)] - Error opening file: "/var/log/nsm//stats.log": Permission denied 18/2/2018 -- 13:20:06 - <Warning> - [ERRCODE: SC_ERR_INVALID_ARGUMENT(13)] - output module setup failed

18/2/2018 -- 13:20:06 - <Warning> - [ERRCODE: SC_WARN_NO_STATS_LOGGERS(261)] - stats are enabled but no loggers are active

18/2/2018 -- 13:20:06 - <Error> - [ERRCODE: SC_ERR_RUNMODE(187)] - The custom type "workers" doesn't exist for this runmode type "PCAP_FILE". Please use --list-runmodes to see available custom types for this runmode

 

[Track Drew]

Sorry, been meaning to reply to this since Sunday - just haven't had the time.

Couple things:
-You have duplicate signatures - either in the same rules file or duplicated rules files. You can try commenting out all the *.rules files in your suricata.yaml file (under the "rule-files:" section), then enable one at a time until you find the problem.
-The last errors related to permission denied is what's likely your user not having write permissions to the /var/log/ dirs. The quick/dirty workaround is to run as root - not really best practice but should be fine for your testing.

Next time you post something like this you can use the Insert -> Code in the post menu:

so you don't end up with all the smiles - ;)

 

[amrogers3]

Hi Drew, thanks for the reply, it is much appreciated sir. Least now I have a place to start troubleshooting.

I probably won't have time to try that until this weekend but I will do it and let you know. I am at wits end with the Security Onion install. (This was a SC install, probably forgot to mention that) Never had so much trouble, spent all day Sat and part of Sun trying to fix to no avail.

I thought the whole point of SC is that it is pretty much plug and play, no? I haven't found these type of issues online and it doesn't seem these are normal errors you would see in a SC install. I tried even downloading and reinstalling an older version. Still have the same problems.

Also thanks for the "insert code" tip. I will do that next time I need to post up log messages.

 

[amrogers3]

Hi Drew, ok I commented out the rules file and ran it with sudo. I got this. First error is expected as there is no rules files in the yaml. Going to comment out the workers in yaml and that should fix the problem.

So got the number of errors down but still getting a bunch of "39" invalid signature codes.

[ERRCODE: SC_ERR_INVALID_SIGNATURE(39)]

1/3/2018 -- 04:28:38 - <Error> - [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - pcre with /R (relative) needs preceeding match in the same buffer

1/3/2018 -- 04:28:38 - <Error> - [ERRCODE: SC_ERR_INVALID_SIGNATURE(39)] - error parsing signature "alert tcp $EXTERNAL_NET any -> $HTTP_SERVERS $HTTP_PORTS (msg:"ET WEB_SPECIFIC_APPS Pandora FMS SQLi"; flow:to_server,established; content:"POST"; http_method; content:"/pandora_console/mobile/index.php"; http_uri; content:"action=login"; http_client_body; fast_pattern; content:"user="; http_client_body; distance:0; pcre:"/[^&]*(?:S(?:HOW.+(?:C(?:HARACTER.+SET|UR(DATE|TIME))|(?:VARI|T)ABLES)|ELECT.+(?:FROM|USER))|U(?:NION.+SELEC|PDATE.+SE)T|DELETE.+FROM|INSERT.+INTO|\/\*.+\*\/|EXEC)/Ri"; reference:url,www.rapid7.com/db/modules/exploit/linux/http/pandora_fms_sqli; classtype:attempted-admin; sid:2019903; rev:1; metadata:created_at 2014_12_09, updated_at 2014_12_09" from file /etc/nsm/rules/downloaded.rules at line 26620

1/3/2018 -- 04:28:38 - <Warning> - [ERRCODE: SC_WARN_POOR_RULE(276)] - rule 2001219: SYN-only to port(s) 22:22 w/o direction specified, disabling for toclient direction

From googling best I can tell, it may be a problem with the signature file. So downloaded new rule set (emerging.rules.tar.gz) from here: https://rules.emergingthreats.net/open/suricata-4.0/
Problem is I don't see where to update and download the "downloaded.rules" file. I think this may be part of my problem too.
yaml file:

default-rule-path: /etc/nsm/rules/
rule-files:
# - local.rules
- downloaded.rules

The "downloaded.rules" location is in the yaml is /etc/nsm/rules/
The "rules" folder with all the rules files is /etc/nsm/rules/rules

Since I am struggling with security onion, I did an install of Suricata in Windows and got it working. The output is eve.json. Any chance you might know what program I need to use to open the eve file?