Format this output with Bash?

[c.d.e]

This is a cross post from Programming because of no replies.
Only thing that has changed is the log file is now /dev/ttyu1 and I cat from /dev/ptyu1 instead of tailing the log file. This completely removes constant harddrive writing and should solve one huge problem I was looking at (Damnsmalllinux based OS from flash. Too many writes to a flash drive = bad), though OSX gui does not handle named pipes at all, so I used a psuedo-terminal. DSL should should handle named pipes from gui well enough, or I will use a peusdo-terminal pair there as well.

---------------

I am trying to take the output of a GUI program (VLC message buffer saved to a log file) and format it for command line display.

The file is saved to ~/Logs/vlc-log.txt.
The output to the log is filled with alot of pointless info, but I broke down the useful info (this is the output of streaming radio connections, local files are different, but that's a problem for another day)

main debug: `http://radio2-us.digitalgunfire.com:9000/' gives access `http' demux `' path `radio2-us.digitalgunfire.com:9000/'
main debug: creating demux: access='http' demux='' path='radio2-us.digitalgunfire.com:9000/'
main debug: creating access 'http' path='radio2-us.digitalgunfire.com:9000/'
access_http debug: http: server='radio2-us.digitalgunfire.com' port=9000 file='/
main debug: net: connecting to radio2-us.digitalgunfire.com port 9000
macosx debug: input has changed, refreshing interface
access_http debug: Icy-Name: [ DigitalGunfire.com ] - Long Range, Hard Hitting!
access_http debug: Meta-Info: icy-url: http://www.digitalgunfire.com
main debug: creating demux: access='http' demux='mp3' path='radio2-us.digitalgunfire.com:9000/'
main debug: meta information:
main debug:   - 'Title' = '[ DigitalGunfire.com ] - Long Range, Hard Hitting!'
main debug: `http://radio2-us.digitalgunfire.com:9000/' successfully opened
access_http debug: New Title=God Module - Ever After
main debug: meta information:
main debug:   - 'Title' = '[ DigitalGunfire.com ] - Long Range, Hard Hitting!'
main debug:   - 'Now Playing' = 'God Module - Ever After'
macosx debug: input has stopped, refreshing interface

The macosx debug lines tell me when the file/stream was opened and closed (Play, Pause, Stop, change song), which might tie in to getting this to output the correct info depending on streaming radio or local file/mp3. But less on that for now.


The parts I want for now are:
access_http debug: Icy-Name: [ DigitalGunfire.com ] - Long Range, Hard Hitting!
access_http debug: Meta-Info: icy-url: http://www.digitalgunfire.com
access_http debug: New Title=God Module - Ever After

Of course, the Stream name, url, and title change depending on the stream and song being played.
I do not want the
main debug: - 'Title' = '[ DigitalGunfire.com ] - Long Range, Hard Hitting!'
main debug: - 'Now Playing' = 'God Module - Ever After'
because the main debug- title line is also used for mp3's and gets sent on every new "Now Playing" so any output I get has the stream's name added repetitively.
Now Playing also has lots of single quotes that I figure would be harder to strip out for formatting the info.

What I currently have is
tail -n15 -f ~/Logs/vlc-log.txt | fgrep -e "New Title" -e "Icy-Name" -e "icy-url"

Giving me an output of

access_http debug: Icy-Name: [ DigitalGunfire.com ] - Long Range, Hard Hitting!
access_http debug: Meta-Info: icy-url: http://www.digitalgunfire.com
access_http debug: New Title=Hocico - Ladykiller (In Cold Blood)

I had to resort to fgrep for the same reason I can not figure out how to format the output in the first place. With tail + (f)grep, I cannot pipe it to a new instance of grep or any other program and get output.

Now, what I want to do is strip "access_http debug: " from each line of output, and change "Icy-Name: ", "Meta-Info: icy-url: ", and "New Title=" to "Stream: ", "@ ", and "Now Playing: "

I have tried using sed, cut, and a grep using -v and -o (invert matching and Show only the part matching pattern, which don't seem to work together). But because I can't get any output from tail piped into grep piped into ANYTHING else, I'm stuck.

Currently, I am trying to do this using just bash and standard commands, but want to expand this program. As I have mentioned, I also need to do this to VLC's mp3 output, but if I get help with this, I can do that one (Same commands, different grepped patterns). I would also like to get this program to recognize the difference between the mp3 and streaming radio output and display only the correct type of formatted output. (As well as recognizing id3v1 and id3v2 tags to avoid duplicate entries). Past this, it would turn into a GUI program, etc.

If a mod feels this belongs in the OS->Linux forum, by all means, but I figured that this would fall under programming since bash is a programming language
Oh, and I am on OSX if it matters.

 

[Boomslang]

You might be able to use "tr" to accomplish all those needs - tr translates between character sets, so you can either prune a string of characters completely, or replace a specific string with another specific string.

 

[c.d.e]

You might be able to use "tr" to accomplish all those needs - tr translates between character sets, so you can either prune a string of characters completely, or replace a specific string with another specific string.

Thanks for the input (haha, get it? Input, commandline? I won't be quiting my day job), but I was able to solve it with a bit of googling. Grepping out to pipes results in a large buffer, so it would be a long time before anything gets outputed, so it needs to be ran with the --line-buffered option when outputting to pipes, so each line fills the buffer and gets sent out.

Here is my solved command

c.d.e:~ cat /dev/ptyu1 | fgrep --line-buffered -e "New Title" -e "Icy-Name" -e "icy-url" | sed -e /"access_http debug: "/s/// -e '/Icy-Name/i\
--
' -e /"Meta-Info: icy-url"/s//"Icy-Url"/ -e /"New Title="/s//"Now Playing: "/

I cat the master pseudo-terminal to read the non-saving log (which when I port this to DSL on a flash drive, will allow me to not kill the flash drive with pointless writes). I'm using the pseudo-terminal instead of a named pipe because OSX does not handle named pipes graciously in the GUI (Finder and VLC crash). DSl should handle named-pipes easily, if not pseudo-terminal there as well.

The catting is piped out to fgrep so I only have to run one instance of grep instead of 3, with one -e for each expression to be matched, with --line-buffered so grep would work with pipes.

The grepped lines are then passed to sed like normal, for the formatting. Sed removes all instances of 'access_http debug: ", then adds a new line, followed by -- followed by a new line before every instance of "Icy-Name". This is so I can browse the terminal buffer and quickly see when I switched to a new station. Note the single quotes around

'/Icy-Name/i\
--
'

which are needed because of the two line returns needed for this to work. Next, Sed replaces "Meta-Info: icy-url" with Icy-Url, so that the website url line looks like the station name line. Finally, sed changes "New Title=" to "Now Playing: ", changing = to : so it matches the other two lines, as well as adding a space for the same reason. Output now looks like:

Icy-Name: Evil Robot Radio
Icy-Url: http://www.evilrobotradio.com
Now Playing: FGFC820 - Victim
--
Icy-Name: Evil Robot Radio
Icy-Url: http://www.evilrobotradio.com
Now Playing: FGFC820 - Victim
Now Playing: Android Lust - Cherished Agony [Gridlock Mix]
Now Playing: The Retrosic - Bastard [The Retrosic Version]
Now Playing: Ministry - Destruction
Now Playing: Alexziq - Black Manta
Now Playing: Hate Dept. - Fiend

This is most of what I wanted to do, but it can be expanded. To keep just this info, I could pipe the sed output to tee, saving the playlist to a actual log (Smaller sized log, and ~3~5 minutes between actual writes, a life saver for flash drives) while still outputting to stdin (which can be useful for outputting this info to a nokia lcd I am wiring to a thinclient/DamnSmallLinux based streaming radio player I am building). Also, since the terminal is using a mono-sized font, I can add spaces infront of Icy-Name and Icy-Url so that the ": " on each line are lined up, hence aesthetically pleasing.

Just as a future note, I might want to take the "Now Playing" line, which tends to have a standard delimiter " - " and trying to make it look like:

Now Playing:
     Artist:  
     Title:

Also, turning it into a gui app, have it display the three lines, changing info when appropriate, and having a drop down list showing all previously played songs, with an option of clicking a song on the list then pressing a button to get just that song added to a "Music to download" textfile (which I am currently doing by copying the track name and artist, and then echo "artist- track" >> music.txt"

I will ALSO work on being a little bit less LONG-WINDED, oh-vey.

 

[Boomslang]

Very nice, very nice. I had my mind wrapped around making a dedicated web radio client a while back, and I was using mplayer to handle the streams. I had considered mpd also, but wasn't sure on how to take the icyinfo and make it visible in some way. Two things were a sticking point for me - external display and station selection control.

Because it was based on a mini-itx motherboard, I wanted to keep things as small as possible, as cheap as possible, and as simple as possible. In order to view the currently playing track, I was considering using a cheap-as-possible 2-line LCD screen to show the name of the current station and the icyinfo in "Artist - Track" format. I also considered using a USB-attached numpad to control my station selections, with each number mapped to a different web-radio station, and +/- used for volume, * to start and stop playback, etc.

I never followed through with any of this because I didn't want to spend more money ... Especially considering there's a lot of politics tainting the web radio scene now, and I was unsure of how long the streams would keep streaming. Blah.

 

[c.d.e]

Very nice, very nice. I had my mind wrapped around making a dedicated web radio client a while back, and I was using mplayer to handle the streams. I had considered mpd also, but wasn't sure on how to take the icyinfo and make it visible in some way. Two things were a sticking point for me - external display and station selection control.

Because it was based on a mini-itx motherboard, I wanted to keep things as small as possible, as cheap as possible, and as simple as possible. In order to view the currently playing track, I was considering using a cheap-as-possible 2-line LCD screen to show the name of the current station and the icyinfo in "Artist - Track" format. I also considered using a USB-attached numpad to control my station selections, with each number mapped to a different web-radio station, and +/- used for volume, * to start and stop playback, etc.

I never followed through with any of this because I didn't want to spend more money ... Especially considering there's a lot of politics tainting the web radio scene now, and I was unsure of how long the streams would keep streaming. Blah.

Nokia cellphone lcd or HD44780 based lcd connected to a parallel port, and a lirc remote module or xmms (http://wejp.k.vu/electronics/xmmsremote/) type serial port controller. The nokia screen is free from a old phone and the remote really only needs a couple of resistors/caps. Cheaper then the usb-numpad.

We have very similar goals/project
I have recently start messing with metal crafting, and have some aluminum flashing that I will (along with milk based plastic making) to mold a pair of speakers, the nokia screen, and the four buttons to a neo-ware thinclient that has two serial ports. Since my streaming radio player is also the basis of a project for school, I might have pics up in a few weeks.

 

[Boomslang]

Nokia cellphone lcd or HD44780 based lcd connected to a parallel port, and a lirc remote module or xmms (http://wejp.k.vu/electronics/xmmsremote/) type serial port controller. The nokia screen is free from a old phone and the remote really only needs a couple of resistors/caps. Cheaper then the usb-numpad.

We have very similar goals/project
I have recently start messing with metal crafting, and have some aluminum flashing that I will (along with milk based plastic making) to mold a pair of speakers, the nokia screen, and the four buttons to a neo-ware thinclient that has two serial ports. Since my streaming radio player is also the basis of a project for school, I might have pics up in a few weeks.

Well, now that you mention it, I do have a Nokia 1100 that I pulled out of a dumpster, and I also have a USB IR dongle + remote control that came with my desktop's motherboard. I also have the LCD and touchscreen layer from a Palm M100, as well as a complete Palm Zire 71. While I expect the Palms to be hard to work with, do you think that Nokia 1100 screen could be used? Are you following a guide? This might be a neat thing to interface with any Linux computer, let alone a dedicated web radio client.

Your project sounds awesome. Just the kind of custom stuff I love to hear about. Can't wait for pictures.

 

[c.d.e]

Well, now that you mention it, I do have a Nokia 1100 that I pulled out of a dumpster, and I also have a USB IR dongle + remote control that came with my desktop's motherboard. I also have the LCD and touchscreen layer from a Palm M100, as well as a complete Palm Zire 71. While I expect the Palms to be hard to work with, do you think that Nokia 1100 screen could be used? Are you following a guide? This might be a neat thing to interface with any Linux computer, let alone a dedicated web radio client.

Your project sounds awesome. Just the kind of custom stuff I love to hear about. Can't wait for pictures.

Just a bit of an update. Got it working, but still need to custom compile things. My test machine is a x86 Transmeta 1ghz thinclient with 256mb ram and 256mb ide flashdisk (booted off of a 512mb usb flashdrive for now), a hp t55xx something. The final will either be a hp t5510, 800mhz Transmeta x86 with 128 + ram and 32mb flashdisk (Trying to squeeze DamnSmallLinux + vlc + needed vlc libs onto a compressed cloop of 30 megs. Geez, 15 megs have never been so hard to cut) or a neoware c200, 1ghz Via C3, 128+ ram, 32mb ide flashdisk. For the Via, I would need to compile for i586 instead of i686 or i486. (Damn emulated x86 >_<)

On the software side, mpg321 + streaming info patch, and mpg123 standard both have crappy Remote Client interfaces, and arn't able to log to file. Also tried other players, but found that someone had gotten vlc .8.6b compiled and packed for DSL. Downloaded it, plus the gtk2-core, libwxgtk, and multimedia libs (~17 megs uncompressed >:[ ), but running it with the http interface (Had to download the http files, .5megs, but need to be in the folder you call vlc from, ie pwd) and the logger interface allows me to work with it just like I had set up for in OSX (I need to cat the ptyu1 first or the logger won't unsuspend, and Ctrl-C to kill vlc won't work unless I press it three time. The pty/tty pair work exactly like a named pipe.) I just want to recompile vlc without any of the video codecs or gui interface, so I can skip the 4 meg gtk core and the 5 meg libWXgtk (wx is the language the current gui interface is in), as well as half (or more) of the video codecs.

(In the future, I can pass my SRP [StreamingRadioPlayer] distro to a better ram/hd spaced computer and readd the video codecs so I can stream videos through it + vlc)

Now to add the lcd + lcdproc to handle it, and maybe the serial port buttons + vlc's hotkey interface (four buttons sending keyboard keycodes + universal hotkeys)

Or, instead (or along with) the lcd, I can output the streaming info to nc, and read it like I currently am from my computer.


So, to control, I have SSH, Vlc's HTTP, and external Buttons (Maybe VLC's remote client, but I don't think so).
To Display info, I have VLc's HTTP, nokia lcd, and NC (And ssh, but thats overkill).
To play music, I have VLC (and ffmpeg through it) loading a local m3u playlist of streaming radio and nfs-client from another server/local load from usb drive, (and possibly mt-daap/firefly/ammarock if I want to load/share daap shares (iTunes sharing).)

I might also try to get S4/S4Bios/Swsusp to work when I get a bigger flashdisk/cf-to-ide adaptor.