# Find all article titles
titles = soup.find_all('span', class_='titleline')
for idx, title in enumerate(titles):
    print(f"{idx + 1}. {title.text.strip()}")
    if idx == 4:
        break

```

This code demonstrates BeautifulSoup4’s straightforward approach to web scraping. It fetches the Hacker News homepage using the requests library, then creates a BeautifulSoup object to parse the HTML. The `find_all()` method searches for `<span>` elements with the class `"titleline"`, which contain article titles. The code loops through the first 5 titles, printing each one with its index number. The `strip()` method removes any extra whitespace around the titles.

The output shows real article titles from Hacker News, demonstrating how BS4 can easily extract specific content from a webpage:

```out
1. The GTA III port for the Dreamcast has been released (gitlab.com/skmp)
2. Arnis: Generate Cities in Minecraft from OpenStreetMap (github.com/louis-e)
3. Things we learned about LLMs in 2024 (simonwillison.net)
4. Journey from Entrepreneur to Employee (akshay.co)
5. Systems ideas that sound good but almost never work (learningbyshipping.com)

```

While BS4 excels at handling static websites, it does have limitations. It can’t process JavaScript-generated content, which many modern websites use. It also doesn’t handle tasks like managing multiple requests or storing data. However, these limitations are often outweighed by its gentle learning curve and excellent documentation, making it an ideal starting point for anyone new to web scraping.

Key Features:

- Simple, intuitive API for parsing HTML/XML
- Powerful searching and filtering methods
- Forgiving HTML parser that can handle messy code
- Extensive documentation with clear examples
- Small memory footprint
- Compatible with multiple parsers ( `lxml`, `html5lib`)

## 3\. Understanding Scrapy

![Scrapy architecture diagram showing components like spiders, engine, scheduler and pipelines](https://www.firecrawl.dev/images/blog/bs4_scrapy/scrapy_architecture.jpeg)

Source: [Scrapy documentation](https://docs.scrapy.org/en/latest/topics/architecture.html).

Scrapy takes a different approach to web scraping by providing a complete framework rather than just a parsing library. Think of it as a Swiss Army knife for web scraping – it includes everything you need to crawl websites, process data, and handle common scraping challenges all in one package. While this makes it more powerful than BS4, it also means there’s more to learn before you can get started.

Here’s a basic example of how Scrapy works:

```python