# Crunchbase Company Data Scraper

A Streamlit app that scrapes company information and funding data from Crunchbase.

## Features

- Bulk scraping of company profiles
- AI-powered data summarization
- CSV export functionality
- Clean, user-friendly interface

## Setup

1. Clone the repository
2. Install dependencies: `pip install -r requirements.txt`
3. Set up environment variables in `.env`:
   - `FIRECRAWL_API_KEY`
   - `ANTHROPIC_API_KEY`
4. Run the app: `streamlit run src/app.py`

## Usage

1. Enter company names (one per line) or upload a text file
2. Click "Start Scraping"
3. View AI-generated insights
4. Download results as CSV

## License

MIT

```

7. **Deploy the app**

After pushing all changes to GitHub, go back to Streamlit Cloud and:

1. Click “Deploy”

2. Wait for the build process to complete

3. Your app will be live at `https://share.streamlit.io/yourusername/company-data-scraper/main`

4. **Monitor and maintain**


After deployment:

- Check the app logs in Streamlit Cloud for any issues
- Monitor API usage and rate limits
- Update dependencies periodically
- Test the app regularly with different inputs

The deployed app will automatically update whenever you push changes to your GitHub repository. Streamlit Cloud provides free hosting for public repositories, making it an excellent choice for sharing your scraper with others.

## Conclusion

In this tutorial, we’ve built a powerful web application that combines the capabilities of Firecrawl and Claude to extract and analyze company data at scale. By leveraging Firecrawl’s AI-powered scraping and Claude’s natural language processing, we’ve created a tool that not only gathers raw data but also provides meaningful insights about companies and their funding landscapes. The Streamlit interface makes the tool accessible to users of all technical levels, while features like bulk processing and CSV export enable efficient data collection workflows.

### Limitations and Considerations

- Rate limiting: Implement appropriate delays between requests
- Data accuracy: Always verify scraped data against official sources
- API costs: Monitor usage to stay within budget
- Maintenance: Website structure changes may require updates

### Next Steps

Consider these enhancements for your implementation:

- Add data validation and cleaning
- Implement request caching
- Add data visualizations
- Include historical data tracking
- Implement error retry logic

### Resources

- [Firecrawl Documentation](https://firecrawl.dev/docs)
- [Claude API Documentation](https://docs.anthropic.com/claude/docs)
- [Streamlit Deployment Guide](https://docs.streamlit.io/streamlit-cloud)
- [Firecrawl’s scrape endpoint](https://www.firecrawl.dev/blog/mastering-firecrawl-scrape-endpoint)

Article updated recently

[🔥](https://www.firecrawl.dev/)

## Ready to _Build?_

Start scraping web data for your AI apps today.

No credit card needed.

Get Started

## About the Author

[![Bex Tuychiev image](https://www.firecrawl.dev/bex.jpg)\\
Bex Tuychiev@bextuychiev](https://x.com/bextuychiev)

Bex is a Top 10 AI writer on Medium and a Kaggle Master with over 15k followers. He loves writing detailed guides, tutorials, and notebooks on complex data science and machine learning topics

### More articles by Bex Tuychiev

[Building an Automated Price Tracking Tool\\
\\
Build an automated e-commerce price tracker in Python. Learn web scraping, price monitoring, and automated alerts using Firecrawl, Streamlit, PostgreSQL.](https://www.firecrawl.dev/blog/automated-price-tracking-tutorial-python) [Web Scraping Automation: How to Run Scrapers on a Schedule\\
\\
Learn how to automate web scraping in Python using free tools like schedule, asyncio, cron jobs and GitHub Actions. This comprehensive guide covers local and cloud-based scheduling methods to run scrapers reliably in 2025.](https://www.firecrawl.dev/blog/automated-web-scraping-free-2025) [Automated Data Collection - A Comprehensive Guide\\
\\
Learn how to build robust automated data collection systems using modern tools and best practices. This guide covers everything from selecting the right tools to implementing scalable collection pipelines.](https://www.firecrawl.dev/blog/automated-data-collection-guide) [BeautifulSoup4 vs. Scrapy - A Comprehensive Comparison for Web Scraping in Python\\
\\
Learn the key differences between BeautifulSoup4 and Scrapy for web scraping in Python. Compare their features, performance, and use cases to choose the right tool for your web scraping needs.](https://www.firecrawl.dev/blog/beautifulsoup4-vs-scrapy-comparison) [How to Build an Automated Competitor Price Monitoring System with Python\\
\\
Learn how to build an automated competitor price monitoring system in Python that tracks prices across e-commerce sites, provides real-time comparisons, and maintains price history using Firecrawl, Streamlit, and GitHub Actions.](https://www.firecrawl.dev/blog/automated-competitor-price-scraping) [Scraping Company Data and Funding Information in Bulk With Firecrawl and Claude\\
\\
Learn how to build a web scraper in Python that gathers company details, funding rounds, and investor information from public sources like Crunchbase using Firecrawl and Claude for automated data collection and analysis.](https://www.firecrawl.dev/blog/crunchbase-scraping-with-firecrawl-claude) [How to Create Custom Instruction Datasets for LLM Fine-tuning\\
\\
Learn how to build high-quality instruction datasets for fine-tuning large language models (LLMs). This guide covers when to create custom datasets, best practices for data collection and curation, and a practical example of building a code documentation dataset.](https://www.firecrawl.dev/blog/custom-instruction-datasets-llm-fine-tuning) [Data Enrichment: A Complete Guide to Enhancing Your Data Quality\\
\\
Learn how to enrich your data quality with a comprehensive guide covering data enrichment tools, best practices, and real-world examples. Discover how to leverage modern solutions like Firecrawl to automate data collection, validation, and integration for better business insights.](https://www.firecrawl.dev/blog/complete-guide-to-data-enrichment)

## Automated Price Tracking
Introducing /extract - Get web data with a prompt [Try now](https://www.firecrawl.dev/extract)

Dec 9, 2024

•

[![Bex Tuychiev image](https://www.firecrawl.dev/bex.jpg)Bex Tuychiev](https://x.com/bextuychiev)