NekoAI — AI-Powered Desktop Pet

NekoAI is an open-source, AI-powered desktop pet that lives on your screen. It wanders around your windows, reacts to what you do, and when you need it — it thinks, answers, and helps, right there on your desktop. A love letter to 90s/00s desktop companions (Neko, eSheep, Shimeji) rebuilt with a modern stack and a real AI brain inside.

Key Features

Feature	Status
Animated sprite pets that roam your desktop	Live
8-direction cursor following & movement	Live
AI chat via animated speech bubble	Live
Persistent memory — name, projects, preferences	Live
Multi-provider AI (Claude, OpenAI, Gemini, Ollama local)	Live
Dynamic mood — energy changes with time and idle time	Live
Multiple pets — Classic Neko, Ghost, Shiba	Live
Proactive nudges ("coding 90 min — take a break!")	Live
NekoCapture — screenshots with metadata footer	v0.5
NekoAnnotate — full-screen drawing overlay	v0.6

Architecture

src-tauri/ (Rust backend)
├── lib.rs              — App setup, tray, Tauri commands
├── desktop_monitor.rs  — Win32 APIs: active window, idle time
└── storage.rs          — SQLite: conversation history, user facts

src/ (TypeScript/React frontend)
├── ai/                 — Provider factory, memory, fact extraction
├── components/         — SpeechBubble, SettingsPanel, ContextMenu
├── hooks/              — usePetMovement, useMoodEngine, useDesktopContext
└── pets/               — PetRenderer, PetBrain, sprite loader

AI Memory System

NekoAI builds a persistent context for every conversation:

Pet personality defined per-pet in pet.json via system_prompt
User facts extracted from conversations and stored in SQLite (user_facts table)
Conversation history — last 20 messages as rolling context
Dynamic mood — energy/happiness/curiosity subtly influences AI tone

Technical Challenges Solved

Frameless window resizing on Windows: Creating a truly frameless window removes WS_THICKFRAME from the OS. JavaScript APIs cannot restore it at runtime. A Rust-side Tauri command calls window.set_size() directly, bypassing the JS API limitation. This allows the speech bubble and settings panel to expand/collapse dynamically.

Pixel-perfect sprite scaling: All pet sizes are integer multiples of the native 32px sprite (S=32, M=64, L=96, XL=128) to prevent anti-aliasing artifacts on pixel art.