… which is a bit unintuitive because one would expect that part of the promise of client libraries would be insulating their users from breaking changes made to the JSON APIs that they wrap I have been burned enough times now: I don't trust vendor client libraries not to break
SOFTWARE
-
Python Library Publishing Tutorial Validation Success
By
–
Nice validation here that my short tutorial on building and publishing a Python library to PyPI works as intended!
-
PyTorch Compile Tutorial: Optimizing Machine Learning Code
By
–
https://
pytorch.org/tutorials/inte
rmediate/torch_compile_tutorial.html
… -
llm.c Optimization: Matching PyTorch Performance After Bug Fix
By
–
Highly amusing update, ~18 hours later: llm.c is now down to 26.2ms/iteration, exactly matching PyTorch (tf32 forward pass). We discovered a bug where we incorrectly called cuBLAS in fp32 mathmode . And ademeure contributed a more optimized softmax kernel for very long rows
-
ONNX WebAssembly Trade-offs in AI Model Deployment
By
–
i’m very skeptical that that’s a better decision than just using onnx wasm or whatever else is available
-
AI Tool Building Websites Better Than Intern-Level Work
By
–
I have been using it for all sorts of tasks where it builds little websites or tools. It works better than an RA at first pass for most things. Intern-level work, not experienced expert.
-
Value Creation Lies in the Application Layer
By
–
the value is in the app layer the value is in the app layer the value is in the app layer the value is in the app layer the value is in the app layer the value is in the app layer the value is in the app layer the value is in the app layer the value is in the app layer
-
Mixtral Training Challenges: Memory Issues and Model Recovery
By
–
Training Bixtral, which is expensive + slow so mistakes are costly. The run finally finished, but then we got a sigkill -9 error (I believe it's a memory issue). I thought I lost the model, but I went into the output directory anyway, and luckily, it was there.
-
Apple Silicon GPU Programming Requires Different Libraries and Approach
By
–
Apple silicon is a whole different thing but yes it also has a GPU, and the code can also be targeted to it in a similar way, but all of the libraries and details change (Metal instead of CUDA) so it’s a whole separate effort.
-
Debugging Systems: Adding Permanent Metrics and Logging
By
–
When debugging an opaque problem in a system, a nice strategy is to just focus on adding permanent (rather than temporary) metrics or logging to make the problem obvious. Often makes it easy to solve not just the specific issue but also a whole class of potential related issues.