I think Raspberry lost the magic of the older Pis, they lost that sense of purpose. They basically created a niche with the first Pis, now they're just jumping into segments that others created and are already filled to the brim with perhaps even more qualified competition.
Are they seeing a worthwhile niche for the tinkerers (or businesses?) who want to run local LLMs with middling performance but still need full set of GPIOs in a small package? Maybe. But maybe this is just Raspberry jumping on the bandwagon.
I don't blame them for looking to expand into new segments, the business needs to survive. But these efforts just look a bit aimless to me. I "blame" them for not having another "Raspberry Pi moment".
P.S. I can maybe see Frigate and similar solutions driving the adoption for these, like they boosted Coral TPU sales. Not sure if that's enough of a push to make it successful. The hat just doesn't have any of the unique value proposition that kickstarted the Raspberry wave.
Yep. RPi foundation lost the plot a long time ago. The original RPi was in a league of its own when it launched since nothing like it existed and it was cheap.
But now if I want some low power linux PC replacement with display output, for the price of the latest RPi 5, I can buy on the used market a ~2018 laptop with a 15W quad core CPU, 8GB RAM, 256 NVME and 1080p IPS display, that's orders of magnitude more capable. And if I want a battery powered embedded ARM device for GPIO over WIFI, I can get an ESP32 clone, that's orders of magnitude cheaper.
Now RPi at sticker price is only good for commercial users since it's still cheaper than the dedicated industrial embedded boards, which I think is the new market the RPI company caters to. I haven't seen any embedded product company that hasn't incorporate RPis in its products they ship, or at least in their lab/dev/testing stage, so if you can sell your entire production stock to industrial users who will pay top dollar, why bother making less money selling to consumers, just thank them for all the fish. Jensen Huang would approve.
I still use Pis in my 3d printers. Laptop would be too big, and a ESP could not run the software. "China clone" might work, but the nice part of the pi is the images available. It just works™
I'm also currently building a small device with 5" touchscreen that can control a midi fx padle of mine. It's just so easy to find images, code and documentation on how to use the GPIO pins.
Might be niche, but that is just what the Pi excels at. It's a board for tinkers and it works.
Yeah, Pi 5 2gb is ~20% more expensive compared to pi3b on release, factoring in inflation (Both in including VAT and local prices)
It's 10 bucks more. ¯\_(ツ)_/¯ Still half the price that I see intel NUCs for sale. Which of course are way more capable. But still, I don't mind the price that much.
I could go with a cheaper alternative, but then AFAIK you might have to fiddle with images, kernel and documentation. For me that is worth 10 bucks.
>Yeah, Pi 5 2gb is ~20% more expensive compared to pi3b on release, factoring in inflation (Both in including VAT and local prices)
I don't really care how it compares to past models or inflation to justify its price tag. I was just comparing to to what you can buy on the used market today for the same price and it gets absolutely dunked on in the value proposition by notebooks.
60 Euros for 2GB and 100 for 8GB models is kind of a ripoff if you don't really need it for a specific niche use case.
Not just laptops but the used enterprise micro PCs from Dell, HP, and Lenovo. All the same small form factor with very low TDP You can have up to 32 or 64 GB RAMs depending on the CPU, dual
or even triple disks if you want a NAS etc.
yeah, depends on what the used market looks like where you live. Here I see way more laptops for sale for cheap than those enterprise thin clients.
And the thin clients when they are for sale tend to have their SSDs ripped out by IT for data security, so then it's a hassle to go out and buy and extra SSD, compared to just buying a used laptop that already comes with display , keyboard, etc.
>I can buy on the used market a ~2018 laptop with a 15W quad core CPU, 8GB RAM, 256 NVME and 1080p IPS display, that's orders of magnitude more capable..
But it won't be as reliable, mostly motherboards won't last long.
Don't know what your source is for that, but that's not my experience, and i've had dozens of laptops through my hands due to my hobby.
The ticking timebomb lemons with reliability or design issues, will just die in the first 2-4 years like clockwork, but if they've already survived 6+ years without any faults, they'll most likely be reliable from then on as well.
The Raspberry Pi probably still has the advantage of an actually robust firmware/software ecosystem? The problem with SBCs has always been that the software situation is awful. That was the Raspberry Pi's real innovation: Raspbian and a commitment to openness.
Nah, they released products better suited to what people were already using Pis for.
The Picos are great for the smaller stuff, new Pis are great for bigger stuff, and old Pis and Zeros are still available. They've innovated around their segment.
The AI stuff is just an expression of that. People are doing AI on Pi5s and this is just a way to make that better.
Not everything needs to be for everyone. I think this is super cool - I run a local transcription tool on my laptop, and the idea of miniaturising it is super cool.
I wouldn't dare suggest that. The RPi was never for everyone yet it turned out it was for many. It was small but powerful for the size, it was low power, it was extremely flexible, it had great software support, and last but not least, it was dirt cheap. There was nothing like that on the market.
They need to target a "minimum viable audience" with a unique value proposition otherwise they'll just Rube-Goldberg themselves into irrelevance. This hat is a convoluted way to change the parameters of an existing compromise and turn it into a different but equally difficult compromise. Worse performance, better efficiency, adds cost, and it doesn't differentiate itself from the competing Hailo-10H-based products that work with any system not just RPi (e.g. ASUS UGen300 USB AI Accelerator).
> the idea of miniaturising
If you aren't ditching the laptop you aren't miniaturizing, just splitting into discrete specialized components.
This is the problem with this gen of “external AI boards” floating around. 8, 16, even 24 is not really enough to run much useful, and even then (ie. offloading to disk) they're so impractically slow.
Forget running a serious foundation model, or any kind of realtime thing.
The blunt reality is fast high memory GPU systems you actually need to self host are really really expensive.
These devices are more optics and dreams (“itd be great if…”) than practical hacker toys.
In the UK I've never seen the hailo hats (which are quite old BTW) advertised for LLMs. The presented usecase has been object detection from lots of video cameras in realtime.
They seem very fast and I certainly want to use that kind of thing in my house and garden - spotting when foxes and cats arrive and dig up my compost pit, or if people come over when I'm away to water the plants etc.
[edit: I've just seen the updated version in Pimonori and it does claim usefulness for LLMs but also for VLMs and I suspect this is the best way to use it].
can't wait to not be able to buy it, and also for it to be more expensive than a mini-computer
I buy a raspberry pi because I need a small workhorse - I understand adding RAM for local LLMs, but it would be like a raspberry pi with a GPU, why do i need it when a normal mini machine will have more ram, more compute capacity and better specs for cheaper?
Is there any usefulness with the small large language models, outside perhaps embeddings and learning?
I fail to see the use-case on a Pi. For learning you can have access to much better hardware for cheaper. Perhaps you can use it as a slow and expensive embedding machine, but why?
As an edge computing enthusiast, this feels like a meaningful leap for the Raspberry Pi ecosystem. Having a low-power inference accelerator baked into the platform opens up a lot of practical local AI use cases without dragging in the cloud. It’s still early, but this is the right direction for real edge workloads.
At this moment my two questions for these things are:
1. Can I run a local LLM that allows me to control Home Assistant with natural language? Some basic stuff like timers, to do/shopping lists etc would be nice etc.
2. Can I run object/person detection on local video streams?
I want some AI stuff, but I want it local.
Looks like the answer for this one is: Meh. It can do point 2, but it's not the best option.
1. Probably, but not efficiently. But I'm just guessing I haven't tried local LLMs yet.
2. Has been possible in realtime since the first camera was released and has most likely improved since. I did this years ago on the pi zero and it was surprisingly good.
What’s the current state of the art in low power wake word and speech to text? Has anyone written a blog post on this?
I was able to run a speech to text on my old Pixel 4 but it’s a bit flaky (the background process loses the audio device occasionally). I just want to take some wake word and then send everything to remote LLM and then get back text that I do TTS on.
Maybe not SOTA but the HA Voice Preview Edition [1] in tandem with a Pi 5 or some similar low-power host for the Piper / Whisper pipeline is pretty good. I don't use it but was able to get an Alexa/Google Home-like experience going with minimal effort.
I was only using it for local Home Assistant tasks, didn't try anything further like retrieving sports scores, managing TODO lists, or anything like that.
Case closed. And that's extremely slow to begin with, the Pi 5 only gets what, a 32 bit bus? Laughable performance for a purpose built ASIC that costs more than the Pi itself.
> In my testing, Hailo's hailo-rpi5-examples were not yet updated for this new HAT, and even if I specified the Hailo 10H manually, model files would not load
Laughable levels of support too.
As another datapoint, I've recently managed to get the 8L working natively on Ubuntu 24 with ROS, but only after significant shenanigans involving recompiling the kernel module and building their library for python 3.12 that Hailo for some reason does not provide outside 3.11. They only support the Pi OS (like anyone would use that in prod) and even that is very spotty. Like, why would you not target the most popular robotics distro for an AI accelerator? Who else is gonna buy these things exactly?
"For example, the Hailo 10H is advertised as being used for a Fujitsu demo of automatic shrink detection for a self-checkout."
... why though? CV in software is good enough for this application and we've already been doing it forever (see also: Everseen). Now we're just wasting silicon.
Are they seeing a worthwhile niche for the tinkerers (or businesses?) who want to run local LLMs with middling performance but still need full set of GPIOs in a small package? Maybe. But maybe this is just Raspberry jumping on the bandwagon.
I don't blame them for looking to expand into new segments, the business needs to survive. But these efforts just look a bit aimless to me. I "blame" them for not having another "Raspberry Pi moment".
P.S. I can maybe see Frigate and similar solutions driving the adoption for these, like they boosted Coral TPU sales. Not sure if that's enough of a push to make it successful. The hat just doesn't have any of the unique value proposition that kickstarted the Raspberry wave.
But now if I want some low power linux PC replacement with display output, for the price of the latest RPi 5, I can buy on the used market a ~2018 laptop with a 15W quad core CPU, 8GB RAM, 256 NVME and 1080p IPS display, that's orders of magnitude more capable. And if I want a battery powered embedded ARM device for GPIO over WIFI, I can get an ESP32 clone, that's orders of magnitude cheaper.
Now RPi at sticker price is only good for commercial users since it's still cheaper than the dedicated industrial embedded boards, which I think is the new market the RPI company caters to. I haven't seen any embedded product company that hasn't incorporate RPis in its products they ship, or at least in their lab/dev/testing stage, so if you can sell your entire production stock to industrial users who will pay top dollar, why bother making less money selling to consumers, just thank them for all the fish. Jensen Huang would approve.
I'm also currently building a small device with 5" touchscreen that can control a midi fx padle of mine. It's just so easy to find images, code and documentation on how to use the GPIO pins.
Might be niche, but that is just what the Pi excels at. It's a board for tinkers and it works.
It's 10 bucks more. ¯\_(ツ)_/¯ Still half the price that I see intel NUCs for sale. Which of course are way more capable. But still, I don't mind the price that much.
I could go with a cheaper alternative, but then AFAIK you might have to fiddle with images, kernel and documentation. For me that is worth 10 bucks.
I don't really care how it compares to past models or inflation to justify its price tag. I was just comparing to to what you can buy on the used market today for the same price and it gets absolutely dunked on in the value proposition by notebooks.
60 Euros for 2GB and 100 for 8GB models is kind of a ripoff if you don't really need it for a specific niche use case.
And the thin clients when they are for sale tend to have their SSDs ripped out by IT for data security, so then it's a hassle to go out and buy and extra SSD, compared to just buying a used laptop that already comes with display , keyboard, etc.
I'm in the market to replace my aging Intel NUCs, but RPi is still cheaper.
But it won't be as reliable, mostly motherboards won't last long.
The ticking timebomb lemons with reliability or design issues, will just die in the first 2-4 years like clockwork, but if they've already survived 6+ years without any faults, they'll most likely be reliable from then on as well.
Ok, let us say they ll last 4 more years, so 10 years total lifespan.
A PI would last a lot longer.
Awful how? A SBC can take advantage of many software written from the dawn of x86.
The Picos are great for the smaller stuff, new Pis are great for bigger stuff, and old Pis and Zeros are still available. They've innovated around their segment.
The AI stuff is just an expression of that. People are doing AI on Pi5s and this is just a way to make that better.
I wouldn't dare suggest that. The RPi was never for everyone yet it turned out it was for many. It was small but powerful for the size, it was low power, it was extremely flexible, it had great software support, and last but not least, it was dirt cheap. There was nothing like that on the market.
They need to target a "minimum viable audience" with a unique value proposition otherwise they'll just Rube-Goldberg themselves into irrelevance. This hat is a convoluted way to change the parameters of an existing compromise and turn it into a different but equally difficult compromise. Worse performance, better efficiency, adds cost, and it doesn't differentiate itself from the competing Hailo-10H-based products that work with any system not just RPi (e.g. ASUS UGen300 USB AI Accelerator).
> the idea of miniaturising
If you aren't ditching the laptop you aren't miniaturizing, just splitting into discrete specialized components.
Almost nothing useful runs in 8.
This is the problem with this gen of “external AI boards” floating around. 8, 16, even 24 is not really enough to run much useful, and even then (ie. offloading to disk) they're so impractically slow.
Forget running a serious foundation model, or any kind of realtime thing.
The blunt reality is fast high memory GPU systems you actually need to self host are really really expensive.
These devices are more optics and dreams (“itd be great if…”) than practical hacker toys.
8GB RAM for AI on a Pi sounds underwhelming even from the headline
They seem very fast and I certainly want to use that kind of thing in my house and garden - spotting when foxes and cats arrive and dig up my compost pit, or if people come over when I'm away to water the plants etc.
[edit: I've just seen the updated version in Pimonori and it does claim usefulness for LLMs but also for VLMs and I suspect this is the best way to use it].
I buy a raspberry pi because I need a small workhorse - I understand adding RAM for local LLMs, but it would be like a raspberry pi with a GPU, why do i need it when a normal mini machine will have more ram, more compute capacity and better specs for cheaper?
I fail to see the use-case on a Pi. For learning you can have access to much better hardware for cheaper. Perhaps you can use it as a slow and expensive embedding machine, but why?
1. Can I run a local LLM that allows me to control Home Assistant with natural language? Some basic stuff like timers, to do/shopping lists etc would be nice etc.
2. Can I run object/person detection on local video streams?
I want some AI stuff, but I want it local.
Looks like the answer for this one is: Meh. It can do point 2, but it's not the best option.
2. Has been possible in realtime since the first camera was released and has most likely improved since. I did this years ago on the pi zero and it was surprisingly good.
I was able to run a speech to text on my old Pixel 4 but it’s a bit flaky (the background process loses the audio device occasionally). I just want to take some wake word and then send everything to remote LLM and then get back text that I do TTS on.
TinyML is a book that goes through the process of building a wake word model for such constrained environments.
I was only using it for local Home Assistant tasks, didn't try anything further like retrieving sports scores, managing TODO lists, or anything like that.
[1] https://www.home-assistant.io/voice-pe/
Case closed. And that's extremely slow to begin with, the Pi 5 only gets what, a 32 bit bus? Laughable performance for a purpose built ASIC that costs more than the Pi itself.
> In my testing, Hailo's hailo-rpi5-examples were not yet updated for this new HAT, and even if I specified the Hailo 10H manually, model files would not load
Laughable levels of support too.
As another datapoint, I've recently managed to get the 8L working natively on Ubuntu 24 with ROS, but only after significant shenanigans involving recompiling the kernel module and building their library for python 3.12 that Hailo for some reason does not provide outside 3.11. They only support the Pi OS (like anyone would use that in prod) and even that is very spotty. Like, why would you not target the most popular robotics distro for an AI accelerator? Who else is gonna buy these things exactly?
... why though? CV in software is good enough for this application and we've already been doing it forever (see also: Everseen). Now we're just wasting silicon.