Looking for a good web scraping program

new

« Prev
1
Next »

dsummoner

Ascriber of Jahbulon. Vae Victus.

Posts: 137

Looking for a good web scraping program Nov 3, 2013 13:19:29 GMT -5

Quote

Post by dsummoner on Nov 3, 2013 13:19:29 GMT -5

I am in the process of working on a project (just for the fun of it) for which a web scraping program would be a boon. In short, I have a starting search page that contains over 10K documents in PDF format (the search was generated by keyword search). I would like a program that can download all of the PDF files on the starting search page and then sequentially for each additional search page. I am not averse to having to learn a bit of coding in either Ruby or HTML5 in order to write my own program to achieve such a function but would rather not spend the time and would instead like something 'ready made' for this purpose.

Socialism is a philosophy of failure, the creed of ignorance, and the gospel of envy, its inherent virtue is the equal sharing of misery (Winston Churchill). But how is this legal plunder to be identified? Quite simply. See if the law takes from some persons what belongs to them, and gives it to other persons to whom it does not belong. See if the law benefits one citizen at the expense of another by doing what the citizen himself cannot do without committing a crime (Frederic Bastiat).

Bartman

Still Working,,,,,,Dammit!

Posts: 77

Looking for a good web scraping program Sept 25, 2015 22:18:29 GMT -5

Quote

Post by Bartman on Sept 25, 2015 22:18:29 GMT -5

I downloaded a Site Grabber or something that said it would Rip any Images from a given URL but I never tried it. PDFs I don't know about grabbing. Think it was called Site Ripper or something like that. I'd hunt around on the shareware/freeware sites.

I have as much Useless Knowledge as the next guy!

The mission of this board is to keep together a community of people who originated from the old Baltimore Sun's SunSpot message board, and to welcome new friends to join us.