Technical Infrastructure
AI systems architecture, the guts
AI systems architecture and production infrastructure powering the security, OSINT, and automation platforms I design and build. I use a wide variety of tools, systems, and custom built models that adapt specifically to what YOU need.
Technology Stack
Core capabilities
Backend
↓- Python (FastAPI, async/await, Celery)
- Go/Golang for high-performance services
- Node.js / Express for real-time systems
- SQLAlchemy + Alembic migrations
Frontend
↓- React 18+ / Next.js 14
- TypeScript for type safety
- React Native for mobile
- TailwindCSS, shadcn/ui, Radix UI
- Vite & Webpack build systems
AI & Machine Learning
↓- Ollama local model deployment
- OpenAI, Anthropic & Vertex AI models
- LangGraph & CrewAI agent orchestration
- LangChain & Semantic Kernel integrations
- PyTorch & TorchVision
- OpenCV for computer vision
- scikit-learn for ML pipelines
Databases
↓- PostgreSQL (optimized queries, partitioning)
- Redis (caching, pub/sub, queues)
- Neo4j for graph relationships
- Weaviate vector database (250M+ vectors)
Infrastructure
↓- Google Cloud Platform (GCP / GKE / Cloud Run)
- Docker Swarm & Kubernetes (K8s) deployment
- Docker Compose for containerization
- Terraform for IaC (Infrastructure as Code)
- Nginx reverse proxy & load balancing
- CI/CD pipelines (GitHub Actions)
- Linux server management
Security
↓- JWT & OAuth 2.0 authentication
- bcrypt encryption
- Rate limiting & DDoS protection
- Vulnerability scanning (SCAFU)
- SSL/TLS certificate management
System Architecture
Production infrastructure
A modular production stack: gateway at the edge, services in the middle, knowledge and storage beneath. The specifics change per project—the structure stays consistent.
Edge & Gateway
Receives all external traffic, routes to internal services, enforces global policies, and handles TLS. First line of defense and traffic control.
Application Services
Stateless services handling REST/GraphQL APIs, real-time channels, and async workers. Message queues decouple workloads for resilience.
Orchestration
Automated orchestration pipelines handling cross-service coordination, data movement, and scheduled tasks. 450K+ agentic decisions executed across the lifecycle.
AI & ML Layer
Privacy-first AI utilizing local Ollama models and 18 custom fine-tuned adapters. 53B tokens processed across fine-tuning pipelines and production inference.
Data Plane
Multi-model persistence: PostgreSQL for transactions, Neo4j for relationships, Redis for sub-millisecond reads. Purpose-built for each data pattern.
Monitoring & Security
End-to-end observability with Prometheus metrics, Grafana dashboards, and real-time alerting. Security policies enforced at every layer.
All services communicate via internal Docker network with automatic service discovery. Prometheus monitors health metrics, Grafana visualizes performance, and automated backups run daily.
AI Infrastructure
Local AI deployment
Privacy-first AI architecture. All models run locally via Ollama—no data leaves your infrastructure. Cloud APIs (OpenAI, Claude) used only for non-sensitive workloads.
# Model routing logic
if task.is_sensitive:
if task.requires_deep_logic:
model = "claude-fable-5" # High-reasoning local agent
else:
model = "qwythos-9b" # Fast agent & code logic
elif task.is_low_latency:
model = "cms-1b" # Edge classification & routing
elif task.context_length > 32000:
model = "claude-3.5-sonnet" # Cloud fallback for long context
else:
model = "qwen3.5:27b" # Default heavy compute
Vector Database
RAG & semantic search
Weaviate vector database with 250M+ embeddings enables sub-100ms semantic search across millions of entities. Hybrid search combines vector similarity with keyword filtering for precision.
Embedding Pipeline: Documents chunked to 512 tokens → Voyage AI embeddings → Weaviate index → Redis cache hot queries. Nuculair uses this for instant profile context retrieval across 300+ data sources.
Hybrid Search: Vector similarity (cosine distance) + BM25 keyword matching + metadata filters. Weighted fusion algorithm combines scores for optimal relevance ranking.
Workflow Orchestration
Temporal.io orchestration
Resilient, code-defined distributed execution engine hosting Python and Go workflows. Replaces fragile visual queues with typed activities, handling complex multi-step agents, telemetry ingestion pipelines, and long-running security scans.
OSINT Pipelines
- Distributed worker orchestration (120+ platforms)
- Rate-limiting & proxy backoff policies
- Aggregated ingestion workflows
- Real-time event queries & data mutation
- Enrichment pipeline Sagas
Security Workflows
- Durable scheduled scans
- Resilient CVE data sync activities
- Automated report compilation tasks
- Failure compensation & alert channels
- Dynamic payload mutation monitoring
Agentic Orchestration
- Multi-agent state loops
- LLM latency tracking & routing
- Compensation paths for API degradation
- Durable context window caching
- Parallel batch embeddings
Frequently Asked Questions
Infrastructure FAQ
What infrastructure powers these AI systems?
The production stack runs on Docker Swarm orchestrating containers across multiple nodes. Core services include Ollama for local AI model inference, PostgreSQL for relational data, Neo4j for graph relationships, Redis for caching, and FastAPI/Python backends with React/Next.js frontends.
Why Docker Swarm instead of Kubernetes?
Docker Swarm provides sufficient orchestration while dramatically reducing operational complexity. For single-team operations running fewer than 50 services, Swarm's native Docker integration and lower resource overhead outweigh Kubernetes' advanced scheduling. However, for enterprise-scale deployments requiring auto-scaling and stateful set management, the architecture is designed to transition smoothly to Kubernetes / Google Kubernetes Engine (GKE).
How are local AI models deployed in production?
Ollama serves as the local model runtime, hosting quantized versions of Llama 3.1 and Mistral. A routing layer directs security-sensitive inference to local models and generic tasks to cloud APIs.
What database strategy supports both OSINT and security?
A polyglot persistence strategy: PostgreSQL for structured records, Neo4j for entity relationships, Redis for sub-millisecond caching, and vector stores for semantic search across 2.8 million embeddings.
How is monitoring handled?
Prometheus collects metrics, Grafana provides dashboards, and custom alerting monitors AI model latency, inference quality scores, and resource utilization.