• Some users have recently had their accounts hijacked. It seems that the now defunct EVGA forums might have compromised your password there and seems many are using the same PW here. We would suggest you UPDATE YOUR PASSWORD and TURN ON 2FA for your account here to further secure it. None of the compromised accounts had 2FA turned on.
    Once you have enabled 2FA, your account will be updated soon to show a badge, letting other members know that you use 2FA to protect your account. This should be beneficial for everyone that uses FSFT.

Computer Builder

E

Eibach

Limp Gawd
Joined
Feb 4, 2007
Messages
230
I just had this idea.

Okay so I always make spreadsheets for building my new computer builds, and I compare prices from different websites. And I have to keep going to those websites to see if there is an update on the price or whatever or if there is a sale or something. Well what if I created a program that did all that for me. You could fill in the part name or something or add links and it would go check those links and automatically update the prices in the program and you would be able to see if there is anything going on with out having to go to all the websites....

I had a question, is it possible to retrieve information from a website using .net. I have seen other places do it like pricegrabber but im not sure if they have access to the actual database of the website because they are non profit or what.

let me know what you think

I did find this but i havnt looked into it that much yet

http://www.codeplex.com/htmlagilitypack

I also know if the website has a web service then i can grab it with .NET and parse it into and XML document but i dont think all the websites i want are going to have that... any ideas??
 
Good Lawd, How often do you build new computers? Or is this from a business standpoint?
 
What you're talking about it typically done with screen scraping. That is, you fetch the page that you're interested in, the raw HTML, (in .NET it looks like you can use System.Net.WebClient and then scan the markup, either the entire page, using regular expressions or using some other method to find what you need.

Screen scraping will break fairly often as it is entirely dependent on the appearance and structure of the site itself and not the underlying data. Changes often occur in the frontend which will make your parsing routines unable to get the data you need.
 
isn't this the kind of thing Web Services exist for?
 
unhappymage, IIRC, did something similar but just for hard drives only:
http://will.incorrige.us/price-checker.cgi?

Unfortunately search is disable so I can't bring up the original thread where unhappymage introduced the parser. He also posted the source code in that thread as well.
 
You can do it with excel somewhat, i've only tried with 2003 but you can import an external datasource as a webpage and depending on the website you should be able to get the item you want, i would assume 2007 has alot better capability for this. I used to have hard drives prices out to show trends on pricing 3-5 years back. I think if you play with excel enough you can get it so it just grabs the price, otherwise the custom parser would work fine, if you use java you can just use the httpclient library and hitting the webpages are pretty straight forward and parsing them isn't hard if you do it right your parser shouldn't break too often, i've done this with several applications. I"d try excel first though
 
oplin said:
You can do it with excel somewhat, i've only tried with 2003 but you can import an external datasource as a webpage and depending on the website you should be able to get the item you want, i would assume 2007 has alot better capability for this. I used to have hard drives prices out to show trends on pricing 3-5 years back. I think if you play with excel enough you can get it so it just grabs the price, otherwise the custom parser would work fine, if you use java you can just use the httpclient library and hitting the webpages are pretty straight forward and parsing them isn't hard if you do it right your parser shouldn't break too often, i've done this with several applications. I"d try excel first though
Click to expand...

that is a very good idea i will check out execl thank you
 
Hypothetical:

If I had to solve this problem, using newegg.com as an example. I'd use the item number to grab the web page, not even worry about parsing the HTML elements and just consume the content line by line using regex to find:

<h3 class="zmp">*</h3>

which is where the price shows up in newegg product pages. Then just parse that line individually.

This is just off the top of my head btw, so I apologize in advance if I left a gaping whole somewhere in the though process. :D
 
So, just because....I went and implemented my above. I have it compiled in a jar file here (source included, comments sparse. Certainly not a case study in Java best practices :p).

Its got a one second wait between queries so you don't spam Newegg.

Standard disclaimer:
Software is provided as-is, with no guarantees of functionality. In fact, if the inputs are screwed up it's quite possible the program will fail less than gracefully (i.e. you'll have to kill it in Task Manager.)

>>Download<<

You'll have to run it from the command line using "java -jar", and of course you'll have to have a JRE installed.

Examples below.

Code: 
>java -jar NeweggItemGrabber.jar -help

Usage: java NeweggItemGrabber [-i inputfile][-o outputfile][Newegg Item Number]
Options:
        -i
                Use to set a file as input. Any whitespace will be used as a delimiter.
                Note that if this option is set any Newegg Item Number arguments are ignored.
        -o
                Use to set a file for output instead of console output.
        -help
                Shows this help text.

>java -jar NeweggItemGrabber.jar N82E16833101500
APPLE MB053LL/A   AirPort Extreme Base Station - Retail
$169.00
 
it didnt seem to work, also i am not that familier with Java =(, im more of a C fan boy lol(c++,c#) anyway if you could send me the source code i think it would be esier to figure it out thanks
 
That java app was pretty simple, all he was doing was downloading the web page and just searching the huge string (html code) for the html tag that contains the price.

Basically

Grab the webpage (C or C++ should make it pretty easy)
Get the content as a string
Search for <h3 class="zmp">
Parse out price
Finished,

Or C++ has dom classes where you could probably just do a .getClassByName and get the contents otherwise.
 
The source was included in the .jar file. You can unpack it with any standard compression utility (I know 7zip works for sure).
 
You must log in or register to reply here.
Back
Top