# Extract website data using LLMs

![Extract website data using LLMs image](https://www.firecrawl.dev/images/blog/g2.png)

## Setup

Install our python dependencies, including groq and firecrawl-py.

```bash
pip install groq firecrawl-py

```

## Getting your Groq and Firecrawl API Keys

To use Groq and Firecrawl, you will need to get your API keys. You can get your Groq API key from [here](https://groq.com/) and your Firecrawl API key from [here](https://firecrawl.dev/).

## Load website with Firecrawl

To be able to get all the data from a website page and make sure it is in the cleanest format, we will use [Firecrawl](https://firecrawl.dev/). It handles by-passing JS-blocked websites, extracting the main content, and outputting in a LLM-readable format for increased accuracy.

Here is how we will scrape a website url using Firecrawl. We will also set a `pageOptions` for only extracting the main content ( `onlyMainContent: True`) of the website page - excluding the navs, footers, etc.

```python
from firecrawl import FirecrawlApp  # Importing the FireCrawlLoader

url = "https://about.fb.com/news/2024/04/introducing-our-open-mixed-reality-ecosystem/"

firecrawl = FirecrawlApp(
    api_key="fc-YOUR_FIRECRAWL_API_KEY",
)
page_content = firecrawl.scrape_url(url=url,  # Target URL to crawl
    params={
        "pageOptions":{
            "onlyMainContent": True # Ignore navs, footers, etc.
        }
    })
print(page_content)

```

Perfect, now we have clean data from the website - ready to be fed to the LLM for data extraction.

## Extraction and Generation

Now that we have the website data, let’s use Groq to pull out the information we need. We’ll use Groq Llama 3 model in JSON mode and pick out certain fields from the page content.

We are using LLama 3 8b model for this example. Feel free to use bigger models for improved results.

```python
import json
from groq import Groq

client = Groq(
    api_key="gsk_YOUR_GROQ_API_KEY",  # Note: Replace 'API_KEY' with your actual Groq API key
)