Cloud-Assisted Verification ("Draft Local, Verify Cloud")
Here is my proposal which I think good to implement. (Note: I'm using LUMO itself to help me to produce a proposal based on my ideas)
Problem: Small on-device models (7B–20B) are prone to hallucination on complex tasks — inventing facts, generating incorrect syntax, or producing logically inconsistent output. Pure local inference is risky for high-stakes work (configuration changes, security analysis, medical/legal information).
Proposed Solution: A three-layer pipeline where local generation is verified by a cloud model — without exposing raw user data.
Architecture — "Local Generation + Cloud Verification":
Layer 1: Local Generation (Instant & Private)
Input (prompt + files) stays on device
Small, fast model generates a draft response in milliseconds
Zero data leaves the device at this stage
Layer 2: Web Search Augmentation (Optional, Lightweight)
If current information is needed, anonymized web search fetches relevant text
Results are fed into the local model to incorporate into the draft
Layer 3: Cloud Verification (Selective)
The generated output text only (not the raw input, not the user's files, not the conversation context) is sent to a large cloud model
The cloud model acts as a "judge" — it checks:
Fact-checking: Are claims accurate?
Logic consistency: Did the model contradict itself?
Hallucination detection: Is this command syntax real or fabricated?
Returns a confidence score or a corrected version
Only triggered for complex/high-stakes queries; skipped for simple tasks
Final Delivery:
Confidence > Threshold: Show local draft immediately with ✅ "Verified" badge
Confidence < Threshold: Replace with cloud-corrected version or flag as ⚠️ "Unverified — Review Recommended"
Trust Indicators (Transparent to User):
🟢 Fully Verified — Generated locally + web-augmented + cloud-verified
🟡 Web-Verified — Generated locally + web search (no cloud verification)
🔵 Local Only — On-device only, offline mode, best-effort accuracy
Real-World Example:
User asks: "Generate a Cisco IOS config to block SSH from subnet 192.168.10.0/24"
Local model generates: access-list 100 deny tcp 192.168.10.0 0.0.0.255 any eq 22
Cloud verifier checks: ACL syntax is correct, but missing permit ip any any at the end (implicit deny-all would break connectivity)
Final output: Corrected config with verification badge
Data Privacy Guarantee:
Raw files (configs, logs, documents): ❌ Never leave device
Conversation context/history: ❌ Never leaves device
Generated output text: ✅ Sent to cloud for verification (output is less sensitive than input)
User can disable cloud verification entirely ("Fast & Private" mode)
Review — Strengths:
Solves the hallucination problem without surrendering privacy or driving up costs
Cloud verification is exponentially cheaper than cloud generation (checking 500 words vs. generating 500 words)
Proton pays for cloud compute only on the ~20% of queries that genuinely need it
Enables a lower subscription tier (majority of work happens on user hardware)
Separates sensitive inputs (stay local) from generated outputs (can be verified)
Trust indicators give users transparency about response confidence — critical for professional use
Review — Risks:
Latency: Waiting for cloud verification could delay response display
Solution: Display local draft immediately with "Verifying..." indicator; update asynchronously if corrections are found
Bandwidth: Sending every response for verification adds network overhead
Solution: Selective verification — only verify complex queries; skip simple tasks (greetings, formatting, simple summaries)
Privacy of output: Even generated output could contain sensitive information
Solution: User toggle to disable cloud verification entirely; clear trust indicators so users know when verification is active
Feasibility: MEDIUM — The concept of using a large model to verify a smaller model's output is well-established in AI research. Implementation requires Proton to build the verification API endpoint and client-side routing logic.