The Python Podcast.__init__

Simplified Data Extraction And Analysis For Current Events With Newspaper

Listen Later
Summary

News media is an important source of information for understanding the context of the world. To make it easier to access and process the contents of news sites Lucas Ou-Yang built the Newspaper library that aids in automatic retrieval of articles and prepare it for analysis. In this episode he shares how the project got started, how it is implemented, and how you can get started with it today. He also discusses how recent improvements in the utility and ease of use of deep learning libraries open new possibilities for future iterations of the project.

Announcements
  • Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
  • When you’re ready to launch your next app or want to try a project you hear about on the show, you’ll need somewhere to deploy it, so take a look at our friends over at Linode. With the launch of their managed Kubernetes platform it’s easy to get started with the next generation of deployment and scaling, powered by the battle tested Linode platform, including simple pricing, node balancers, 40Gbit networking, dedicated CPU and GPU instances, and worldwide data centers. Go to pythonpodcast.com/linode and get a $60 credit to try out a Kubernetes cluster of your own. And don’t forget to thank them for their continued support of this show!
  • This portion of Python Podcast is brought to you by Datadog. Do you have an app in production that is slower than you like? Is its performance all over the place (sometimes fast, sometimes slow)? Do you know why? With Datadog, you will. You can troubleshoot your app’s performance with Datadog’s end-to-end tracing and in one click correlate those Python traces with related logs and metrics. Use their detailed flame graphs to identify bottlenecks and latency in that app of yours. Start tracking the performance of your apps with a free trial at pythonpodcast.com/datadog. If you sign up for a trial and install the agent, Datadog will send you a free t-shirt.
  • You listen to this show to learn and stay up to date with the ways that Python is being used, including the latest in machine learning and data analysis. For more opportunities to stay up to date, gain new skills, and learn from your peers there are a growing number of virtual events that you can attend from the comfort and safety of your home. Go to pythonpodcast.com/conferences to check out the upcoming events being offered by our partners and get registered today!
  • Your host as usual is Tobias Macey and today I’m interviewing Lucas Ou-Yang about Newspaper, a framework for easily extracting and processing online articles.
    • Interview
    • Introductions
    • How did you get introduced to Python?
    • Can you start by describing what the Newspaper project is and your motivations for creating it?
    • What are the main use cases that Newspaper is built for?
      • What are some libraries or tools that Newspaper might replace?
      • What are the common structures in news sites that allow you to abstract across them for content extraction?
        • What are some ways of determining whether a site will be a good candidate for using with Newspaper?
        • Can you talk through the developer workflow of someone using Newspaper?
          • What are some of the other libraries or tools that are commonly used alongside Newspaper?
          • How is Newspaper implemented?
            • How has the design of he project evolved since you first began working on it?
            • What are some of the most complex or challenging aspects of building an automated article extraction tool?
            • What are some of the most interesting, unexpected, or innovative projects that you have seen built with Newspaper?
            • What keeps you interested in the ongoing support and maintenance of the project?
            • What do you have planned for the future of Newspaper?
              • Keep In Touch
              • LinkedIn
              • @LucasOuYang on Twitter
              • Website
              • codelucas on GitHub
                • Picks
                • Tobias
                  • Million Bazillion Podcast
                  • Lucas
                    • Hackers and Painters: Big Ideas from the Computer Age by Paul Graham
                      • Closing Announcements
                      • Thank you for listening! Don’t forget to check out our other show, the Data Engineering Podcast for the latest on modern data management.
                      • Visit the site to subscribe to the show, sign up for the mailing list, and read the show notes.
                      • If you’ve learned something or tried out a project from the show then tell us about it! Email [email protected]) with your story.
                      • To help other people find the show please leave a review on iTunes and tell your friends and co-workers
                      • Join the community in the new Zulip chat workspace at pythonpodcast.com/chat
                        • Links
                        • Newspaper
                        • Los Angeles
                        • Reddit
                        • Django
                        • NLP == Natural Language Processing
                        • Web Scraping
                          • Podcast Episode
                          • Requests
                          • Wintria
                          • Python Goose
                          • Diffbot
                          • Heuristics
                          • Stop Words
                          • RSS
                          • SpaCy
                            • Podcast Episode
                            • Gensim
                              • Podcast Episode
                              • PyTorch
                                • Podcast Episode
                                • NLTK
                                • LXML
                                • Beautiful Soup

                                  • The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA

                                  ...more
                                  View all episodesView all episodes
                                  Download on the App Store
                                  The Python Podcast.__init__By Tobias Macey
                                  • 4.4
                                  • 4.4
                                  • 4.4
                                  • 4.4
                                  • 4.4

                                  4.4

                                  100 ratings

                                  More shows like The Python Podcast.__init__

                                  View all
                                  TED Talks Daily by TED

                                  TED Talks Daily

                                  11,280 Listeners

                                  6 Minute English by BBC Radio

                                  6 Minute English

                                  1,779 Listeners

                                  The Changelog: Software Development, Open Source by Changelog Media

                                  The Changelog: Software Development, Open Source

                                  285 Listeners

                                  Data Skeptic by Kyle Polich

                                  Data Skeptic

                                  474 Listeners

                                  Talk Python To Me by Michael Kennedy

                                  Talk Python To Me

                                  585 Listeners

                                  Software Engineering Daily by Software Engineering Daily

                                  Software Engineering Daily

                                  630 Listeners

                                  The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) by Sam Charrington

                                  The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)

                                  429 Listeners

                                  Super Data Science: ML & AI Podcast with Jon Krohn by Jon Krohn

                                  Super Data Science: ML & AI Podcast with Jon Krohn

                                  295 Listeners

                                  Python Bytes by Michael Kennedy and Brian Okken

                                  Python Bytes

                                  212 Listeners

                                  Syntax - Tasty Web Development Treats by Wes Bos & Scott Tolinski - Full Stack JavaScript Web Developers

                                  Syntax - Tasty Web Development Treats

                                  984 Listeners

                                  DataFramed by DataCamp

                                  DataFramed

                                  267 Listeners

                                  Practical AI by Practical AI LLC

                                  Practical AI

                                  196 Listeners

                                  The Real Python Podcast by Real Python

                                  The Real Python Podcast

                                  136 Listeners

                                  Last Week in AI by Skynet Today

                                  Last Week in AI

                                  275 Listeners

                                  Latent Space: The AI Engineer Podcast by swyx + Alessio

                                  Latent Space: The AI Engineer Podcast

                                  64 Listeners