How To Protect Your Data With Self-Hosted LLMs And OpenFaaS Edge

With growing concerns about data privacy and NDAs, sending customer data to third-party AI services like OpenAI or Anthropic can be risky. Self-hosting Large Language Models (LLMs) using OpenFaaS Edge provides a secure alternative, ensuring your sensitive data never leaves your infrastructure.

You Should Know:

1. Setting Up OpenFaaS Edge for Self-Hosted LLMs

OpenFaaS Edge allows you to deploy serverless functions and LLMs on your own infrastructure. Here’s how to get started:

Install OpenFaaS CLI & faasd

 Install OpenFaaS CLI 
curl -sSL https://cli.openfaas.com | sudo sh

Deploy faasd (lightweight OpenFaaS) 
git clone https://github.com/openfaas/faasd 
cd faasd 
./hack/install.sh

Deploy Ollama for Local LLMs

Ollama lets you run open-source LLMs like Gemma, LLaMA, or Mistral locally:

curl -fsSL https://ollama.com/install.sh | sh 
ollama pull gemma:4b  Example model

2. Running LLMs with OpenFaaS

Create a serverless function to interact with your self-hosted LLM:

Python Function Example

import requests

def handle(event, context): 
prompt = event.body.decode("utf-8") 
ollama_url = "http://localhost:11434/api/generate" 
response = requests.post(ollama_url, json={"model": "gemma:4b", "prompt": prompt}) 
return response.json()["response"]

Deploy it using:

faas-cli deploy --name llm-query --image your-docker-username/llm-query --fprocess="python3 handler.py"

3. Securing Your Deployment

Enable TLS: Use `certbot` for HTTPS.
Network Isolation: Deploy in a private subnet.
Access Control: Use OpenFaaS’s built-in auth.

4. Monitoring & Logging

Check logs with:

journalctl -u faasd -f  For faasd logs 
kubectl logs -n openfaas-fn <pod-name>  If using Kubernetes

What Undercode Say

Self-hosting LLMs with OpenFaaS Edge ensures compliance with strict data policies while maintaining AI capabilities. By leveraging tools like Ollama and faasd, organizations can deploy private, scalable AI solutions without relying on third-party APIs.