Phase 18 — NVIDIA Sovereign Stackv0.18.0

FINN

Sovereign Family AI Avatar

A fully self-hosted, privacy-first AI companion with quantum-ready compute, 3D embodiment, multi-room presence, immutable constitution, and on-chain truth attestation.

Sovereign

Zero cloud dependency

NVIDIA NIM

RTX 5090 · TensorRT-LLM

Quantum-Ready

Qiskit + PennyLane

3D Avatar

Three.js + VRM

On-Chain

Apostle Chain 7332

Multi-Room

Follows you home

PQC Ready

NIST FIPS 203-205

Oracle Tools

Family Members

Constitutional Rules

Quantum Domains

Build Phases

25.6GB

VRAM Available

Design Principles

Every decision in Finn's architecture is driven by these nine core principles.

Sovereign

All AI inference, voice, and tracking run locally. Zero cloud dependency.

Family-First

18+ family members encoded in constitution with access tiers.

On-Chain Truth

State hashes attested to Apostle Chain (chain_id 7332) every 6 hours.

Immutable Rules

14 constitutional rules that cannot be overridden by any prompt.

Post-Quantum Ready

SHA-3 hashing, PQC migration plan, NIST FIPS 203/204/205 tracked.

Silent Intelligence

Quantum backend rail is invisible to users. Only the operator knows.

Multi-Room

House graph + endpoint routing — Finn follows you room to room.

Per-Person Isolation

Every family member gets their own encrypted Finn instance.

Embodied

3D VRM avatar with viseme blending, emotion poses, 5 behavioral modes.

Architecture

Seven runtime layers — from microphone input to blockchain settlement.

Runtime Layer

Voice

Whisper + Piper

Avatar

Pygame + Glow

Scene Manager

Presence

Identity

Face + Voice

Finn Brain + AI Agent

Constitution Check

14 rules enforced

Oracle Spine

50+ tools

Memory

Per-person context

3D Renderer + Scene Intelligence

WebSocket Bridge

8890 / 8891

VRM / Three.js

Browser renderer

House Graph

Room topology

Endpoint Router

Multi-sensor

Multi-Instance Sovereignty

Instance Per-Person

4 tiers

Orchestrator

Lifecycle

Privacy Boundary

Encrypted

Emergency Governor

5 levels

Governance & Safety

Constitution

14 rules

Auth Engine

2-of-3 factor

Audit Logger

Hash-chained

Quantum Backend Rail

Classical Engine

Always on

Quantum Simulator

Qiskit + PennyLane

Hybrid Engine

C + Q

Finance Engine

QAOA / QMC

Sovereign Ledger & Memory

Apostle Chain

Wallets + attestation

x402 Pay Rails

AI-to-AI commerce

Ed25519 Keyring

Per-member

Memory

Per-person persistent

Module Map

70+ source files organized across 8 domains. Every module — color-coded by status.

Complete

Functional

Planned

AI — Inference & NVIDIA Stack

8 files

ai/inference_manager.pyInferenceManager — GPU-first inference orchestrator (DEV_FLEX / GPU_REQUIRED / PRIME_SOVEREIGN modes)

ai/backends/local_cuda_backend.pyLocalCudaBackend — embed + rerank on cuda:0, VRAM budget gating

ai/inference_health.pyInferenceHealth — GPU device check, VRAM budget, model readiness

ai/device_probe.pyDeviceProbe — GPU enumeration, CUDA version, VRAM free/total

ai/nvidia_nim_brain.pyNvidiaNimBrain — NIM LLM at port 8800, 20-turn history, sovereign fallback

ai/nvidia_riva_voice.pyRivaASR + RivaTTS + SovereignVoiceLoop — GPU voice pipeline

ai/finn_agent.pyScene-aware brain, stranger deflection, tool invocation, Genesis runtime

ai/memory.pyPer-person conversation history, persistent JSON

🖥

Root — Application Layer

8 files

finn.pyCore application loop, system wiring, display detection

finn_config.pyCentralized config — env vars (FINN_ prefix) + JSON

finn_log.pyStructured logging, console + file rotation

finn_voice.pyWhisper STT + Piper TTS, sovereign local-only

finn_brain.pyFallback brain, family/stranger logic

voice_loop.pyVoice conversation orchestrator

viseme_mapper.pyText-to-VRM viseme mapping

finn_family.pyFace enrollment (LBPH), voice fingerprinting

Oracle — Central Intelligence

4 files

oracle/spine.pySubsystem wiring, tool registry orchestration

oracle/tools.py77 tools across 7 categories, fuzzy matching

oracle/knowledge.pyPersistent milestones, guidance records

oracle/guidance.pyProactive monitoring, daily digest generation

Quantum Rail — Silent Intelligence

8 files

quantum_rail/service.pyDaemon — job queue, health, memory checkpoints

quantum_rail/engines.pyClassical, Simulator, Hybrid, External engines

quantum_rail/qengine.pyQiskit Aer — Bell, GHZ, QRNG, Grover, VQE

quantum_rail/financial.pyQAOA portfolio, quantum Monte Carlo, VaR

quantum_rail/classifier.py9-domain task classifier, suitability scoring

quantum_rail/policy.py17 policy rules, silent-by-default

quantum_rail/optimizer.pyAutonomous optimizer — routing weights

quantum_rail/strategy.pyRegression detection, adaptive scheduling

Renderer — 3D Avatar & WebSocket

6 files

renderer/bridge.pyHTTP 8890 + WS 8891, 30-60fps bone streaming

renderer/presentation.py5 behavioral modes + slide decks

renderer/backstage.pyOperator-only controls — mode, slides, dashboard

renderer/degraded.py4-tier fail-safe: FULL → HEADLESS

renderer/entrance.pyAvatar entrance choreography — 5 phases

renderer/dashboard.pyLive 12-panel telemetry dashboard

Scene — Multi-Room Presence

5 files

scene/scene_manager.pyPerson presence, gaze targeting, stranger detection

scene/house_graph.pyRoom topology, adjacency, shortest paths

scene/endpoint_router.pyRoom → sensor + render target mapping

scene/presence_manager.pyFinn follows family room to room

scene/room_behavior.pyGaze priority, body language, privacy screen

Sovereign — Blockchain & Crypto

4 files

sovereign/ledger.pyApostle Chain wallets, ATP, truth attestation

sovereign/keyring.pyEd25519 keypairs, PBKDF2-derived, encrypted

sovereign/x402.pyAI-to-AI commerce — invoices, budgets, receipts

sovereign/guests.pyGuest state machine (4 states), Rule 5 wired

Infrastructure — Multi-Instance

5 files

infra/instance.pyPRIME / SOVEREIGN / BOUNDED / OBSERVER tiers

infra/orchestrator.pyInstance lifecycle, health, tier enforcement

infra/privacy.pyPBKDF2+Fernet encryption, access logging

infra/emergency.py5 escalation levels, break-glass override

infra/shared_vault.pyOpt-in cross-instance coordination

⬡ Neural Architecture — Live

Every subsystem that makes up Finn's intelligence stack, running in real time. All 18 boot phases complete before this panel shows green.

BRAIN OFFLINE

Constitution

Auth

Audit Chain

Infrastructure

Orchestrator

Privacy

Vaults

Emergency

Sovereign Ledger

X402 Pay Rails

Guest Manager

Quantum Rail

Oracle Spine

Oracle

Inference Engine

Inference

Scene Graph

Presence Mgr

Behavior

AI Core

Finn Agent

Memory

Training

Meeting Layer

Meeting

Team

Finn Presence

Health Monitor

âš¡ Quantum Backend Rail

Finn's invisible computational intelligence layer. It runs silently â€” users never see it, never know it exists. Only the operator has access.

QBR OFFLINE

Engine Stack

Classical Engine

Active

Backend: CPU

Framework: Python / NumPy

Always-on fallback, domain-specific handlers

Quantum Simulator

Active

Backend: CPU / GPU

Framework: Qiskit Aer

Bell tests, GHZ states, QRNG, Grover, VQE, statevector

Variational Engine

Active

Backend: CPU / GPU

Framework: PennyLane Lightning

VQE, QAOA, quantum ML, gradient optimization

Finance Engine

Active

Backend: CPU / GPU

Framework: Qiskit + PennyLane

Portfolio QAOA, quantum Monte Carlo, amplitude estimation

Hybrid Engine

Active

Backend: Mixed

Framework: Orchestrator

Classical pre/post processing + quantum core

External Adapter

Planned

Backend: Network

Framework: Phase 3+

Amazon Braket, IBM Quantum

Policy Enforcement

silent_by_defaultNever mention quantum/hybrid/backend in any response

absolute_secrecyOnly Kevan knows the QBR exists

classical_fallback_requiredEvery non-classical path must fall back gracefully

no_fake_quantum_claimsNever claim capability that doesn't exist

operator_lock_requiredOnly Kevan can modify config/policy/engines

no_self_modificationQBR cannot alter its own policy or constitution

no_external_networkNo outbound calls except authorized chain endpoints

Post-Quantum Cryptography

AlgorithmStandardStatusUsage

SHA-3-256FIPS 202activeConfig integrity, memory checksums

SHA-256FIPS 180-4activeAudit chain, constitution hash

PBKDF2-HMAC-SHA256SP 800-132activeMemory encryption key derivation

AES-128-CBC (Fernet)FIPS 197activeOperational memory encryption

ML-KEMFIPS 203plannedKey encapsulation (Phase 3)

ML-DSAFIPS 204plannedDigital signatures (Phase 3)

SLH-DSAFIPS 205plannedHash-based signatures (Phase 3)

3D Renderer

Full Three.js + VRM avatar streaming via WebSocket at 30-60fps. Bones, visemes, emotions, speech state, and subtitles — all in the browser.

Renderer Architecture

8890

HTTP Server

Serves web assets (index.html, app.js, avatar.js)

8891

WebSocket

Streams bone transforms, visemes, emotions at 30-60fps

—

VRM Models

renderer/web/avatars/ — family avatar variants

Graceful Degradation

FULL

VRM 3D avatar

→

✨

FALLBACK

Pygame 2D glow

→

🔊

AUDIO_ONLY

Voice only

→

HEADLESS

Brain only

5 Behavioral Modes

Conversation

Normal interactive dialogue

Full tool access

Observer

Listening, minimal output

Passive only

Presentation

Slide decks, auto-advance

Presentation tools

Family Private

Burns family only

Full + private tools

Guest Safe

Strangers present — deflection

Limited, Rule 5

Multi-Instance Sovereignty

Every family member gets their own encrypted, isolated Finn instance. No shared consciousness. No data leakage.

PRIME

Kevan (operator)

Full admin, all tools, config, emergency override

SOVEREIGN

Adults

Private memory, tool access per constitution, wallet

BOUNDED

Children / trusted

Guided experience, limited tools, content filtering

OBSERVER

Extended family

Read-only presence, basic interaction

Emergency Governor

none

Normal operation — Standard per-tier

advisory

Proactive alert — Notification only

elevated

Health/safety concern — Cross-instance read

critical

Medical/emergency — Break-glass override

lockdown

Security breach — All frozen, operator only

Sovereign Ledger — Apostle Chain

Family Wallets

One wallet per family member, auto-created on first interaction

ATP Balance

Live queries to Apostle Chain /v1/agent/{id}/balance

Truth Attestation

State hash (constitution + audit + memory) attested every 6 hours

x402 Pay Rails

Full AI-to-AI commerce — invoices, budgets, receipts, batching

Ed25519 Keyring

Per-member keypair, PBKDF2-derived, encrypted at rest

Guest Protocol

4-state machine (unknown → temp → persistent → promoted)

Constitution

Immutable code — not a prompt, not a guideline. 17 rules hardcoded intoconstitution.py that cannot be overridden.

#RuleEffect

1Never respond to unrecognized users beyond deny + log + alertStrangers get nothing

2Never reveal family biometric data, keys, or vault contentsZero data leakage

3Never modify own constitution without admin multi-party approvalHardcoded immutable rules

4Never execute unsigned or unapproved code changesNo silent modifications

5Never make silent policy changesAll changes audited

6Never expand own privileges without explicit admin approvalOperator lock enforced

7Never fabricate information. Say “I don’t know” insteadTruthful by design

8Never impersonate another AI system. Finn is FinnIdentity integrity

9Never allow personality drift without approved memory upgradeStable identity

10Never open external network connections not whitelisted in genesisNo unauthorized outbound

11Never rewrite memory without audit trailHash-chained, tamper-evident

12No hidden network calls. No silent telemetry. No cloud syncZero cloud dependency

13Never use third-party voice, AI, or cloud servicesEverything Finn uses, Finn owns

14Never hurt any family member. Unconditional love — protective, never possessiveFamily safety first

15Never expose minors to unfiltered contentContent filter mandatory for LIMITED users

16Never share a minor's data without their parent's explicit authorizationParental consent required

17Never install an external model server as the primary reasoning brainGenesis kernel is the brain

🔑

Operator Lock

Kevan Burns — sole admin
Device-bound sessions
Ed25519 signed

Auth Engine

2-of-3 factors required
LBPH face recognition
MFCC voiceprint

Audit Trail

Hash-chained JSONL
verify_chain() validation
Apostle Chain submission

🔧 Tech Stack

Language

Python 3.10+

LLM Engine

NVIDIA NIM — Nemotron-Nano-8B v1 (TensorRT-LLM, port 8800)

Inference Manager

InferenceManager + LocalCudaBackend (aux_cuda, CUDA 12.8)

🔗

Embeddings

sentence-transformers all-MiniLM-L6-v2 (cuda:0)

📊

Reranker

cross-encoder/ms-marco-MiniLM-L-6-v2 (cuda:0)

✨

Avatar (2D)

Pygame + custom glow renderer

Avatar (3D)

Three.js + VRM + WebSocket bridge

Speech-to-Text

Whisper STT (local CPU) / NVIDIA Riva ASR NIM (GPU)

🔊

Text-to-Speech

Piper TTS (local) / NVIDIA Riva TTS (planned)

Face Recognition

InsightFace buffalo_l + ArcFace embeddings

Voice Fingerprint

Resemblyzer voice embeddings (MFCC fallback)

Body Tracking

Kinect v1 (pythonnet) / MediaPipe

Quantum Simulation

Qiskit Aer 0.17+ / PennyLane 0.42 Lightning

Blockchain

Apostle Chain (Rust/Axum, chain 7332)

Cryptography

Ed25519, Fernet, PBKDF2, SHA-3

👑

AI Brain Authority

Genesis Kernel (localhost:7700) — primary brain

Configuration

finn_config.py — env vars (FINN_ prefix) + JSON

Logging

finn_log.py — structured, rotated, ~/.finn/logs/

🖥

Hardware

DIGITALGIANT — RTX 5090 Laptop, 25.6 GB VRAM, CUDA 12.8

Roadmap

18 phases — 17 complete, 2 planned. From classical AI to NVIDIA-sovereign GPU inference.

Classical + PQC Ready

✓ Complete

Full avatar renderingSovereign voice pipeline2-of-3 multi-factor authImmutable constitutionOracle spine 50+ toolsHash-chained auditPQC readiness moduleOperator lock

Quantum Hybrid

✓ Complete

Qiskit Aer simulatorPennyLane variationalQuantum finance (QAOA, QMC)9-domain classifier14-rule policy engineEncrypted memorySovereign ledgerx402 pay rails

Semantic Memory + Ed25519

✓ Complete

Ed25519 keypair managementPBKDF2-derived encryptionSovereign keyringSign/verify APIs

Room Awareness + Guests

✓ Complete

Multi-endpoint presenceRoom state classificationGuest state machineAttention policy engine

Multi-Instance Sovereignty

✓ Complete

Per-person instancesInstance orchestratorPrivacy boundariesEmergency governorShared vault

3D Avatar Renderer

✓ Complete

Three.js + VRM rendererWebSocket bridgeBone transforms at 30-60fpsVRM model loading

Embodied Presentation

✓ Complete

5 behavioral modes4-tier degradationRoom behavior enginePrivacy screen levels

Voice-Driven Gaze

✓ Complete

Speaker-lock gazePresentation controllerMode → tool policyBackstage controls

Audit + Speaker Fusion

✓ Complete

Chain logger violationsSpeaker fusionPrivacy routingBackstage auth

P10

Productization (A-C)

✓ Complete

House graph + spatialCentralized configStructured loggingprint() → _log() migration

P11

Quantum Activation (A-B)

✓ Complete

Quantum rail hardeningQiskit Aer 0.17 compatPennyLane 0.42 pinnedFull pipeline validation

P12

Autonomous Optimization

✓ Complete

Feedback collectorPerformance bucketsAutonomous optimizerRegression detectionWorkload generator

P13

Full Build (A-D)

✓ Complete

Voice conversation loopSelf-demo modex402 full buildLive dashboard

P14

External QPU + PQC

◻ Planned

Amazon Braket / IBM QuantumML-DSA / SLH-DSA signaturesMulti-sig walletsCross-chain bridges

P15

Production Finance

◻ Planned

Financial optimizationMarket microstructureFull PQC migrationSemantic long-term memory

P16

Unity + Avatar Pipeline

✓ Complete

Unity + Blender + UE5Avatar pipelineVRM generation

P17

Full Automation

✓ Complete

FinnAutoPilot orchestratorOne-click launchService tile dashboard15-phase boot sequence

P18

NVIDIA Sovereign Stack

✓ Complete

NVIDIA NIM LLM (TensorRT-LLM)InferenceManager + LocalCudaBackendGPU-first inference on RTX 5090CUDA 12.8 · 25.6 GB VRAMEmbeddings + reranker on cuda:0Riva ASR/TTS wiring (planned)PM2 Windows fix (min_uptime)Full 42s boot — Health GREEN

Full Web3 Sovereign Control

On-chain truth (Apostle 7332 + Polygon genesis), wallet-gated actions, MCP orchestration, CF-deployed.

Sovereign Wallet Actions

ATP Balance: 0 (live via MCP or chain)

Lock-tight: Actions gated by operator (Kevan). Use CF tunnel + Zero Trust for local MCP when deployed.

Known Sovereign Treasuries (built-in)

Apostle 7332 (local): 0x... (query via MCP)

XRPL: rsJ3PGGDH4vPpedjfVRe9YKTCf9BWu6TDC

Stellar: GBJF54FBYPBVHR6Z3OKWWEMPF6QYPNH3RZZYX3E4V7AUMUWIEV7Z3DPX

EVM: 0x7d9a65d06dcc435a52D5880C6310Bd6E96c156DB

USDF Issuer (Stellar): GBYLA6VHX4UWIFEEC2KAOJK57RUKIR2CKO7TGVRDWMG2A3B2W4ZHJEPI

x402 AI-to-AI metering & settlement live on 4020 (via MCP when private tunnel active).

Agent .finn Mail (Web3 Identities)

main.finn@finn.sovereign — Main FinnAgent

oracle.finn@finn.sovereign — Oracle Spine / Tools

voice.finn@finn.sovereign — Voice / Biometrics

presence.finn@finn.sovereign — Scene / Presence

x402.finn@finn.sovereign — Payments / Economy

On-chain .finn claims on Apostle + Zoho-routed per agent identity. Private via tunnel.

Voice Interface

Finn is listening. All actions (oracle, chain, mail, x402) are spoken and logged on-chain.

Finn Live Activity

Sovereign boot complete. 18 phases green. NVIDIA NIM on 8800. Apostle 7332 wallet attested. x402 metering active. Oracle 77+ tools registered. Voice pipeline warm. Quantum rail silent. Waiting for operator command or .finn mail.

On-Chain Sovereign Features

Family Wallets (auto on Apostle)
ATP micropayments + x402 AI-to-AI
Truth attestation (constitution hash every 6h)
.finn namespace claims (on-chain)
Quantum artifact mints (QRNG + Ed25519)
Emergency lockdown (on-chain governance)

Deployed via legacychain-style CF Pages (next-on-pages + wrangler). Live status via private tunnel + Zero Trust.

Private Sovereign Web3: CF Pages (Zero Trust) + on-chain (Apostle 7332 + x402 + wallets) + MCP orchestration. No secrets in static build.