using tshark and want to parse output

Discussion in 'Networking & Security' started by amrogers3, Aug 15, 2018.

  1. amrogers3

    amrogers3 Gawd

    Messages:
    550
    Joined:
    Nov 7, 2010
    Using tshark to analyze pcap file.

    tshark -n -r capture.pcap -Y 'http.request.uri.query contains "some search string"' -T fields -e ip.src -e http.request.uri.query

    OUTPUT:
    r=4&f=3&s=400:585&query=this+is+what+I need&hl=en&gl=us&c=33&d=http%3A%2F%2Fwww.jskklok.com&b=1&j=google.ip.c.j_8Ejasdassw2fDt5TLAg_3783538590_2&a=ID4​

    I would like to pipe the tshark command to parse the output string. Looked into AWK but that appears to need a delimiter.

    Is there a way to pipe the tshark command to a parser and only output the query= results?
     
  2. Mega6

    Mega6 Gawd

    Messages:
    988
    Joined:
    Aug 13, 2017
    grep, cut and awk should give you everything you need.
     
    FNtastic likes this.
  3. NoOther

    NoOther [H]ardness Supreme

    Messages:
    6,675
    Joined:
    May 14, 2008
    It seems your output has a delimiter "&".

    Why not try:


    tshark -n -r capture.pcap -Y 'http.request.uri.query contains "some search string"' -T fields -e ip.src -e http.request.uri.query | awk -F"&" '{print $4}'

    or similar
     
  4. amrogers3

    amrogers3 Gawd

    Messages:
    550
    Joined:
    Nov 7, 2010
    Thanks for the reply. This would be most elegant but there are multiple &'s in various positions so doesn't always grab the correct one.
     
  5. amrogers3

    amrogers3 Gawd

    Messages:
    550
    Joined:
    Nov 7, 2010

    board is awesome as usual. Thanks for the replies.

    Ok, may not be the most efficient but this works.

    tshark -n -r capture.pcap -Y 'http.request.uri.query contains "q="' -T fields -e ip.src -e http.request.uri.query | awk '{print $1,$2}' | awk -F"q=" '{print $2}' | cut -d'&' -f1


    Have another problem. I need to print out the source IP, however, it is being cut off by awk. Is there a way to retain the 1st field?

    This is orginal output:

    Code:
    field #1                field #2
    192.168.1.2            r=4&f=3&s=400:585&q=this+is+what+I+need&hl=en&gl=us&c=33&d=http%3A%2F%2Fwww.jskklok.com&b=1&j=google.ip.c.j_8Ejasdassw2fDt5TLAg_3783538590_2&a=ID4
     
    Last edited: Aug 15, 2018
  6. NoOther

    NoOther [H]ardness Supreme

    Messages:
    6,675
    Joined:
    May 14, 2008
    According to that it seems you are cutting to show the first field that is delimited by '&'. If there is a tab between the first field and the second field, you could try using ' ' as the delimiter and that should give you the first field with cut.

    Although the problem is that in your second awk pipe you are only selecting items in the second field.

    Basically you are saying take the output of tshark and send it to awk, print out the first 2 fields, then send that to awk again and only give me the specified output in the second field, then send that to cut and only print out the first new field delimited by &

    You probably want to use && to combine some of your statements

    Maybe this:

    tshark -n -r capture.pcap -Y 'http.request.uri.query contains "q="' -T fields -e ip.src -e http.request.uri.query | awk '{print $1}' && awk -F"q=" '{print $2}' | cut -d'&' -f1

    Really the best option here is probably to write a python script to get the accuracy you want in your output.
     
    Last edited: Aug 15, 2018
  7. amrogers3

    amrogers3 Gawd

    Messages:
    550
    Joined:
    Nov 7, 2010
    && doesnt appear to work

    | awk '{print $1}' && awk '{print $2}' only prints the 1st field. Any suggestions on how to get this guys to output $1 and $2 fiel
     
  8. Frank Bello

    Frank Bello n00bie

    Messages:
    12
    Joined:
    Dec 27, 2016
    Are you still working on this ? I have some ideas that might help but would need to see what the sample output looks like before the script changes it.
     
  9. PaulTech

    PaulTech [H]Lite

    Messages:
    83
    Joined:
    Jan 20, 2004
    Is the parser something you have written? If so you could easily parse the .pcap binary format with a library like libpcap.
    This would allow you to extract the exact information without screen-scraping the possible invalid results.