Harvesting the Web: Using PowerShell to Scrape Screens, Exploit Web Services and Save Time

NTK 2016
Stopnja 300
Datum sreda
18. maj 2016 13:15

For many IT pros, our days start with the same activity: surfing to catch up. We skim our Twitter feeds, view and perhaps download The Picture of the Day from some amazing photo site, check the weather, peek at eBay to see if that prized item is selling under $100 yet, or whatever. That's great, but what isn't so great is the fact that most of us are gathering those data the same way we've been doing it for it for twenty years -- by clicking around in a web browser. Hey, it's the 21st century, and all the cool kids are automating things… so why aren't you using PowerShell's tools to accomplish your daily web data-gathering tasks, whether personal- or business-related? PowerShell includes a number of little-known but essential web-harvesting cmdlets, including Invoke-WebRequest, New-WebServiceProxy, Invoke-RestMethod, and Select-XML, to name just a few. Before they can help you automate your web data gathering and interpretation, though, you'll need some background in screen scraping, SOAP versus RESTful web services, XPath queries, OAuth authentication, managing JSON data and the like. That could take a pretty long time, unless you attend this session created and delivered by best-selling tech author and speaker Mark Minasi. This fast-paced, fun session interweaves brief, "just enough information and no more" explanations of the underlying technologies with illustrative, real-world examples. Attend this session and you too can be The Master of Web Automation!

Mark Minasi is a best-selling author, popular technology columnist, commentator, keynote speaker, and IT consultant. He first got the chance to play with a computer at a university class in 1973. At that time, he learned two things: •First, computers are neat. (People still said "neat" back in 1973. Hey, it was back in the 20th Century.) •Second, many technical people are very nice folks, but they can sure put you to sleep in an instant while explaining technical things. Mark transformed those two insights into a career making computers and networking easier and more fun to understand. He's done that by writing over a thousand computer columns, several dozen best-selling technical books, and explaining operating system and networking planning, installation, maintenance and repair to crowds from two to two thousand. An independent voice hailed as "Favorite Technical Author" by CertCities four times out of four, Mark has the unusual ability to take even the most technical topics, filter out the hype and explain them in plain English. Perhaps that's why when TechTarget hired him to deliver a webcast on PC tuning, he drew three times as many attendees as any of their previous webcasts, crashing Yahoo's servers, and why he's been hired to deliver keynote addresses at hundreds of techie conferences around the world. Mark is probably best known for his Mastering Windows Server and Complete PC Upgrade and Maintenance Guide books, both of which have seen more than 12 editions and sold over a million copies. An audience member at a recent talk remarked that he believed that Mark could "do a talk on watching paint dry that would be so good that people would be motivated to go home and paint a wall just to experience the joy of drying paint." While this has led to many very tempting offers from the likes of Sherwin-Williams and Behr, he's decided to stay with his first love... technology. Mark's humorous, provocative and yet informative style makes him a favorite of audiences around the world. Mark's firm, MR&D, is based in Pungo, a town in Virginia's Tidewater area which is distinguished by having one and only one traffic light.