Provide opt-out after the fact. To operate in good faith, manual opt-out requests should also be respected after the fact too. Because the dataset format allows users to bypass and override opt-out client side, not honoring request makes LAION look complicit in infringement.
@alexjc
-
Database Storage for LAION-5B: Removing Illegal Content Safely
By
–
Use a database storage method not version control. Storing the LAION-5B dataset on revision control means nothing can be removed (e.g. private data or illegal content) without leaving a trace or nuking history. Using a database storage + wrapper API would avoid this problem.
-
Dataset sourcing and ethical image attribution in AI
By
–
Have the dataset to include sources. Each image was referenced by a specific page, and this original root URL should be included as a field. By "hot linking" files directly that could be on CDNs, it bypasses usual access mechanisms & prevents reliably checking opt-out status.
-
Copyright Compliance: LAION Must Prove File Deletion Records
By
–
Publish code & logs upfront show files were deleted. Complying with Copyright exceptions means the burden of proof is on LAION to show they qualify. They're required to delete the files so keeping records of this (or publishing them upfront) could help avoid court.
-
img2dataset User-Agent Spoofing Ethics and Web Protection
By
–
Give img2dataset a correct User Agent name. The img2dataset tool also by default presents itself with a User-Agent field defined as Mozilla Firefox only. This spoofing circumvents copy-protection mechanisms built-in to web servers, and can be considered a form of fraud.
-
LAION Should Disclose User Agent Strings and IP Addresses
By
–
Announce user agent and IP addresses upfront. It's industry-standard for search engines to disclose their User Agent strings before starting operations, and also a list of IP addresses doing the crawling. LAION could have done this in a public manner upfront.
-
LAION Dataset Compliance: Robots.txt Verification Ethics
By
–
Check opt-out themselves! LAION relies on the fact that CommonCrawl allegedly checked opt-out, but the legal defense "our upstream provider is compliant so we don't need to be" isn't strong. Instead, LAION could have checked robots.txt themselves when creating the dataset.
-
Software Tool Recommendations for Robots.txt Compliance Checking
By
–
Recommend software that checks robots.txt. One of LAION's members built img2dataset, which does not check robots.txt. It would be a stronger legal argument if this recommended tool did check at least robots.txt now, even if better forms of opt-out are added later.
-
LAION-5B Legal Compliance: Dataset Challenges and Solutions
By
–
Legal uncertainty is all around the AI industry, in particular the LAION-5B dataset. Despite the RELAION update, there are still many problems to be addressed… (My peer review follows.) Here's what LAION could have done to make their dataset more legal and more certain!
-
Sonnet vs GPT-4: Performance and Speed Comparison Analysis
By
–
Sonnet isn't even setup as a default model, I had to add it. I don't think it's Sonnet 3.5 quality: for me it doesn't solve problems any more consistently than with GPT 4. Sonnet is quite a bit faster though, maybe it's UX/UI and no longer hitting limits or waiting as much…