Python BeautifulSoup: Extract all the text from a given web page

Last update on December 21 2024 07:24:25 (UTC/GMT +8 hours)

Write a Python program to extract all the text from a given web page.

Sample Solution:

Python Code:

import requests
from bs4 import BeautifulSoup
url = 'https://www.python.org/'
reqs = requests.get(url)
soup = BeautifulSoup(reqs.text, 'lxml')
print("Text from the said page:")
print(soup.get_text())

Sample Output:

Text from the said page:
Welcome to Python.org
     {
       "@context": "https://schema.org",
       "@type": "WebSite",
       "url": "https://www.python.org/",
       "potentialAction": {
         "@type": "SearchAction",
         "target": "https://www.python.org/search/?q={search_term_string}",
         "query-input": "required name=search_term_string"
       }
     }
    

    var _gaq = _gaq || [];
    _gaq.push(['_setAccount', 'UA-39055973-1']);
    _gaq.push(['_trackPageview']);

    (function() {
        var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
        ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
        var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
    })();
    
Notice: While Javascript is not essential for this website, your interaction with the content will be limited. Please turn Javascript on for the full experience. 

...........

▲ Back to Top

Help & General Contact
Diversity Initiatives
Submit Website Bug

Status 

Copyright ©2001-2019.
                             Python Software Foundation
                            Legal Statements
                             Privacy Policy
                             Powered by Heroku

window.jQuery || document.write('<script src="/static/js/libs/jquery-1.8.2.min.js"><\/script>')

Python Code Editor:

Have another way to solve this solution? Contribute your code (and comments) through Disqus.

Previous: Write a Python program to a list of all the h1, h2, h3 tags from the webpage python.org.
Next: Write a Python program to print the names of all HTML tags of a given web page going through the document tree.