Give img2dataset a correct User Agent name. The img2dataset tool also by default presents itself with a User-Agent field defined as Mozilla Firefox only. This spoofing circumvents copy-protection mechanisms built-in to web servers, and can be considered a form of fraud.
CODE
-
Software Tool Recommendations for Robots.txt Compliance Checking
By
–
Recommend software that checks robots.txt. One of LAION's members built img2dataset, which does not check robots.txt. It would be a stronger legal argument if this recommended tool did check at least robots.txt now, even if better forms of opt-out are added later.
-
Fine-tuned o1 Achieves Gold Medal in IOI Programming Competition
By
–
screenshot of eval results in the tweet above and more in the blog post, but worth especially noting: a fine-tuned version of o1 scored at the 49th percentile in the IOI under competition conditions! and got gold with 10k submissions per problem.
-
Irony of a 20-second game code feeling slow
By
–
Imagine having an entire video game coded for you in 20 seconds and thinking “dude this is taking forever”
-
SQL UI Generation Implementation on LangGraph Cloud
By
–
sql + gen ui – oh my! great work from @DhruvAtreja1 implementing this on LangGraph cloud
-
AssemblyAI Speech-to-Text API for Voice-Driven AI Products
By
–
Weekly Spotlight: Want to build voice-driven AI products? AssemblyAI’s Speech-to-Text API can help. Transcribe accurately, generate real-time captions, and extract insights. Get $50 in free credits! #VoiceTech #SpeechRecognition
-
API Builder GPT: Generate APIs and Setup Automations Easily
By
–
GPT of the Week: Need to create & connect APIs easily? The API Builder OpenAPI Schema GPT is perfect for generating APIs, setting up automations, and troubleshooting code. #APIDevelopment #Automation
-
Staging as Data Transformation in ETL Pipelines
By
–
Can we say Staging is just fancy way to describe data transformation in ETL pipeline?
-
JinaAI Reader-LM extracts markdown from HTML webpages
By
–
Extracting your HTML webpages to markdown is now possible end-to-end with a simple LLM! @JinaAI_ just released Reader-LM, that handles the whole pipeline of extracting markdown from HTML webpages. A while ago, they had released a completely code-based deterministic program
