We are looking for a skilled freelancer to design and develop a web scraping solution that extracts data from a set of public tender and procurement websites. The extracted data must be structured, deduplicated, and delivered in Excel and CSV formats for internal analysis and reporting. The ideal candidate has proven experience in web scraping and data extraction, ideally in technical or industrial sectors (experience with the space industry is a strong plus). About the service we're looking for: - Large-scale web data extraction - Automated ETL workflows and data pipelines - Machine Learning is a plus, not a requirement About the scraping in more details: - It should not require any payments to websites for running - Output formats: CSV and Excel (.xlsx) - We would need a website dashboard interface to: ○ Run the scrapers on demand ○ Schedule recurring runs ○ Be able to add new sources easily (config-driven) - Detect duplicates across sources (same reference, same URL, or similar info on title, buyer & deadline) - Track changes: if deadline/budget/status changes, mark as updated with a tag - Each run of the script produces the following sheets: ○ “All tenders” ○ “New since last run” ○ “Updated since last run” The system should support both manual execution and scheduled runs. Deliverables: - Source list integrated (5 to 10 sources to start) - Extraction pipeline + normalized data model - Deduplication and scraping improvement - CSV and Excel outputs - Web-based interface - Deployment instructions & knowledge transfer More detailed information about target sources, keywords, filtering logic, and final deliverables will be shared with shortlisted candidates. Important note: Due to security, confidentiality, and sensitive business considerations, we intentionally keep some technical and business details high-level at this stage. Additional information will be shared once we confirm alignment and expertise. If you have a keen eye for detail and can work with various data structures, we want to hear from you!