Pretty amazing that we can get such good results even without batchnorm — and not using a proper resnet init like Fixup either! Maybe I should try lsuv with batchnorm-free resnet sometime…
INNOVATION
-
Active Learning: The Year of Breakthrough for Machine Learning Systems
By
–
Active learning can enable machine learning systems to: 1) Adapt to changing conditions 2) Learn from fewer labels 3) Keep a human-in-the-loop for most difficult examples 4) Achieve higher performance. AL has been around for decades, 2023 might finally be the takeoff year.
-
Generative AI to Revolutionize Active Learning Through Uncertainty
By
–
With recent advances in generative AI, Active Learning is primed for a major breakthrough. When an algorithm is unsure of correct label for some part of encoding space, it can actively generate from that section to get input from a human.
-
AI Avatar App Successfully Launches on iOS Platform
By
–
Very interesting post on someone who DID go into iOS with an AI avatar app
-
Reforming Academic Metrics: Beyond Peer-Review Point Systems
By
–
I guess I'm glossing over the 2nd use of peer-reviewed publications though, which is the point-scoring system widely used for academic career development. Maybe that should be replaced with something more personal and less metric-driven.
-
OpenAI’s Point-E Generates 3D Point Clouds From Text Prompts
By
–
OpenAI’s Point·E: Generating 3D Point Clouds From Complex Prompts in Minutes on a Single GPU https://
syncedreview.com/2022/12/27/ope
nais-pointe-generating-3d-point-clouds-from-complex-prompts-in-minutes-on-a-single-gpu/
… -
AI Summer 2022: Year’s Biggest News and Insights
By
–
Despite the cold weather, we’re in the middle of a red hot AI summer. Read my thoughts — and a roundup of this year’s biggest AI news — in The Batch!
-
Post-Copilot Directions from Anshul and Codeium Team
By
–
see also the proposed post-Copilot directions from @_anshulr and the @codeiumdev team on last week's post
-
Embeddings as Versioned Documentation for Machine-Readable Code
By
–
my take: Docs for humans, Embeddings for machines Write samples -> get embeddings -> ship with each API/SDK version composable give users 99% accurate codegen can't version foundation models, but can version embeddings If you liked @types
, you'll *LOVE* @embeddings! -
Beyond Copilot: The Next Evolution in AI Developer Experience
By
–
The Big Question in AI-driven developer experience: What's after Copilot? Everyone is convinced this is "just" the beginning but can't articulate what the next killer form factor will be. Increasingly convinced next step change won't be from optimal stuffing context windows