Data Collection Ethics

What We Collect

Our crawler monitors publicly accessible web pages—pricing pages, product pages, blog posts, press releases, job listings, and similar content that is freely available to any internet user without authentication.

We never:

• Access password-protected, login-gated, or paywalled content
• Scrape personal data, employee records, or private communications
• Circumvent CAPTCHA or other access-control mechanisms
• Access unpublished, confidential, or proprietary information

How We Collect

robots.txt Compliance

We respect robots.txt directives. If a website instructs crawlers not to access certain paths, we honour those instructions. We identify ourselves via a descriptive user-agent string.

Rate Limiting

We crawl at conservative rates to avoid placing undue load on target servers. Crawl frequency is limited and configurable—by default we do not crawl any given page more than once per day, and often much less frequently.

No Excessive Scraping

We only collect pages that are reasonably necessary to provide competitive intelligence. We do not bulk-download entire websites or build general-purpose web archives.

Legal Basis for Collection

Collecting and processing publicly available information for business intelligence purposes is lawful under EU and UK data protection law where:

• The information was manifestly made public by the data subject
• Processing is based on legitimate interests (business intelligence)
• Processing does not override the rights of individuals

We do not collect or process special category personal data and actively exclude personally identifiable information from our competitive intelligence pipeline.

Your Competitors’ Data

Data collected about competitor companies is used solely to generate insights for the customer who configured that competitor. We do not aggregate competitor data across customers or build shared intelligence databases.

Opt-Out for Website Owners

If you operate a website and wish your content excluded from 3RA’s monitoring, you may:

• Add a Disallow directive in your robots.txt for our user-agent
• Email privacy@3ra.io with your domain and we will add it to our exclusion list within 5 business days

Changes to This Policy

We may update this policy to reflect changes in our practices or applicable law. Material updates will be communicated via email.

Contact

Questions about our data collection practices? Contact us at privacy@3ra.io.