Ability to run local models
Some small models are capable to run locally on phones. This would be great for privacy and also as a free alternative for the more powerful paid LLMs.
-
Tony Stark
commented
I would also like to have the option to run Lumo locally as well. I'll admit, with Proton's privacy focus in the cloud version, it is more of a theoretical excercise for me. However, to take privacy as far as I can go, no cloud connection is optimal. Also, since Lumo is maintained by Proton, there would be more implicit trust in the model itself as opposed to a Meta/Google managed model. (No reason not to trust them ;) )
Even if there isn't anything suspect with the other local models now, I'm sure I share the philosophy of zero trust; giving any company the least amount of privilege. You never know what they will try; knowingly or unknowingly through bad actors. I'm now looking side-eyed at my Nvidia chips lol
-
David
commented
I agree at least add the ability maybe many users will consider it and that way proton can save some cloud resources.
-
Baud
commented
You can already do this very easily using ollama. It has a nice gui. Note that only powerful computers (e.g 32gb macbook or PC with an expensive NVIDIA GPU) can run actually useful LLMs. Cloud computing is puch more efficient.