Build anything small into a container on your laptop, push it to DockerHub or the Github package registry then host it on fly.io for free.

Great write up, thanks. For video learners, Wolfgang does a good step-by-step on YouTube

I’d love you to check back later with your conclusions.

Guide to Self Hosting LLMs with Ollama.

• Download and run Ollama
• Open a terminal, type ollama run llama3.2

If it’s an M1, you def can and it will work great. With Ollama.

Thanks, I ended up going with Garage, but it has the same issue. I assumed I could just specify some buckets with their keys in the docker-compose or garage.toml, but no - they had to be done through the api or command line.

This is correct, I already installed the minio cli, but when I came back and read this, I tried it out and yes, once garage is running in the container, you can

alias garage="docker exec -ti <container name> /garage"

so you can do the cli things like garage bucket info test-bucket or whatever. The --help for the garage command is pretty great, which is good since they don’t write it up much in the docs.

Thanks. I ended up going with Garage (in Docker), and installed the minio client cli for these tasks.

One I’m writing. I use the host file system (as I have a strong preference for simple) for it’s storage, but I’m interested in adding Litestream for replicating the database onto AWS.

"Convert this text to make it sound like from a random person: "

Love the effort you’ve put into this question. You’ve clearly done some quality research and thinking.

When I asked myself this same question a couple of years ago, I ended up just buying a second hand Synology NAS to use alongside my mini-pc. That would meet your criteria, and avoids the (I’m not sure what magnitude) reliability risk of using disks connected over USB. It’s more proprietary than I’d like, but it’s battle tested and reliable for me.