Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Hmm.. this might make it feasible to build something like a command line program where you can optionally just specify the arguments in natural language. Although I know people will object to including an extra 14 MB and the computation for "parsing" and it could be pretty bad if everyone started doing that.

But it's really interesting to me that that may be possible now. You can include a fine-tuned model that understands how to use your program.

E.g. `> toolcli what can you do` runs `toolcli --help summary`, `toolcli add tom to teamfutz group` = `toolcli --gadd teamfutz tom`

 help



I wonder if it will make sense to have a central natural language parser of sorts managed by the OS that includes a model like this?

So all command line programs can hook into this model at runtime, and you can have adaptors for fine-tuning etc.


Apple already have this working on iOS - as discussed in this recent post.

https://unix.foo/posts/local-ai-needs-to-be-norm/


Yes I assume some kind of LLM system will eventually make its way into OS package registries and eventually there will be a standard of some sort. Already OpenAI's API is the de facto standard available from all providers. But who knows what port it's on. Maybe implement mDNS/DNS-SD or SSDP, and if not there it could just go through default ports like the Ollama one and check for a certain endpoint.

So Needle is trained for INT4, what you see in the playground is INT4, only 14MB, same challenge though.

Oh gotcha. Fixed my comment.

sounds loke a job for llamafile! https://github.com/mozilla-ai/llamafile



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: