Software Developer (Python, LLM/Meta Llama 3)

- Proven experience deploying open-source LLMs like Meta Llama 3 and Mistral / Mixtral in on-premise or private setups - Strong command of Python for LLM inference, prompt crafting, and integration - Background in CPU-based inference, model quantization, and performance optimization - Hands-on experience with open-source vector databases including Qdrant, Chroma, Milvus, or Pgvector - Demonstrated implementation of Retrieval-Augmented Generation (RAG) pipelines - Experience in generating and managing embeddings along with metadata filtering - Familiarity with data privacy, air-gapped deployments, and enterprise security protocols - Proven experience in establishing access controls and audit logging - Experience with LangChain or LlamaIndex is a plus - Exposure to Rust, Go, or C++ for high-performance applications is desirable - Knowledge of Docker and Kubernetes for on-premise deployments is advantageous - Understanding of inference frameworks such as vLLM, llama.cpp, or Hugging Face Transformers - Previous work experience in regulated or enterprise settings is beneficial