I'm a Digital Operations manager for a college news site, and as part of a new project, I'm trying to generate a corpus of all article content we've released.
Approaches I've considered thus far:
- Using a web scraper to retrieve all post content
- Using an XML parser to parse the site's post XML dump
What's the best methodology to do this? Any specific libraries you've found helpful?
I have admin privileges if that helps.