Hello ! I am looking for a self-hosted or self-hostable nlp (like llama or chatgpt) but very very small one that could work with as low as ~300 mb of ram. It needs to have an API. I plan to integrate it into my dashboard project you might know, homarr. I’d like to make some kind of assistant to directly help within the app by using its integrations capabilities.
The tool needs to be self-hosted so that users won’t leak the queries to anyone. A freemium service that you can either self-host or pay for would also work.
It does not need to have a huge knowledge base (doesn’t need to know a good lobster recipe) , just to be able understand basic language inputs and in turn I will make it communicate with the key parts of the app
I apologize if this is not worded properly as I am fairly new to the world of LLMs.
I do a bunch of AI stuff, but you won’t get chatgpt quality from anything else. It requires a massive amount of storage, memory and processing hardware- millions of dollars in hardware alone. Not sure what you’re trying to do exactly, but that model is insane to attempt reproduction in any part