LLMs running on a dedicated card: The final frontier as hacker makes it so
Consider your wish for an AI digital assistant that runs locally and offline officially granted. Not by a major industry player, naturally – your personal data is too enticing – but by a guy on GitHub who built one to run on a Raspberry Pi., to do"anything a standard LLM can do in a conversational setting," all without needing to rely on, or share data with, the outside world.
And he managed to get it all up and running on a Raspberry Pi 5 connected to a USB microphone, a speaker and a camera.that he sees Pi-CARD's purpose as being an offline assistant that is to be able to access files on a local machine."So while not requiring Wi-Fi, can still tell you info like what you wrote in your journal two weeks ago," Kasmanoff cited as an example.
Of course running locally means it can be a bit slow, and lacks a lot of data that more cloud-centric LLMs have access to."The system is designed to be a fun project that can be a somewhat helpful AI assistant," Kasmanoff said, adding that there are"a lot of improvements to be made." "I wish," Kasmanoff told us when asked if Star Trek star Patrick Stewart's voice is available for his Pi counterpart."I am using a generic dictation system, which while not exciting, gets the job done."There are a couple familiar LLMs running under Pi-CARD's hood - or C++ versions of them, at least.to do its speech and image recognition, respectively.
Canada Latest News, Canada Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
Meet the long serving staff at a Belfast train station as it closes its doorsThe closure of Great Victoria Street station is to make way for Belfast Grand Central Station which will open this autumn
Read more »
Raspberry Pi adds more memory to the Compute Module 4SCompute Module 5 still on track for later this year
Read more »
Got an old Raspberry Pi spare? Try RISC OS. It is, literally, something elseRISC OS Open 5.30 arrives – with Raspberry Pi Wi-Fi support
Read more »
Raspberry Pi prepares to boot up a London listingThe little computer that could gets ready for an IPO
Read more »
Is the long awaited Raspberry Pi flotation about to happen?Reports suggest an end of May IPO and a valuation of up to half a billion
Read more »
Taproom serving up three-day festival of craft beer, live music and food at Walton SummitHopworks Taproom is hosting SpringFest Pic: Hopworks Brew Co. / Facebook A Walton Summit taproom is serving up a three-day festival of craft beer, live musi
Read more »