Clicky: An Open-Source AI Teaching Buddy for macOS

Update: April 27, 2026

Farza here — the creator of Clicky. The open-source core of Clicky remains freely accessible for tinkering, experimentation, and personal hacking. If you want to explore, customize, or even start a company on top of this foundation, go for it — the MIT license invites you to do exactly that. For new, privately developed features, I’ll keep those private, but the public repository stays wide open for everyone to learn from and contribute to. To grab the latest public version, visit the project’s home, and you can also catch Farza’s updates on X/Twitter. The community is invited to “go crazy” with the repo.

Clicky is a friendly AI companion that lives beside your cursor. It isn’t just a passive helper; it can see your screen, speak with you, and point at UI elements as you work. Imagine having a patient, interactive teacher right there with you while you learn new tasks or troubleshoot problems. This combination of eye-in-screen awareness, natural conversation, and on-screen guidance makes Clicky feel like a real mentor shadowing your workflow.

Below is a detailed guide to understanding Clicky, getting started, and how the project is organized. If you’re curious about the visual aspect, check out the demo GIF that showcases Clicky in action.

---

What Clicky Is and Why It Matters

• An AI teacher that lives alongside your cursor, capable of visual guidance and dialogue.
• It can see your screen, talk to you, and point at interface elements to direct attention.
• The project’s core is open source under MIT license, inviting exploration, extension, and customization.
• The public open-source version is designed for learning, feature hacking, and understanding the underlying architecture.
• The private side of the project continues to be developed separately, with ongoing improvements that aren’t in the public repo.

This approach balances openness with iterative, private development. The open core provides a transparent view into how Clicky’s AI, screen capture, and UI overlays work together, while the private stream can push newer capabilities without exposing fragile prototypes to the public.

Sources of the latest public information include the project landing page and public announcements, as well as social posts sharing context and examples. If you want context or demo references, you can explore the original posts linked in the project notices.

---

Get started quickly: Claude Code path

For developers who want to jump in fast, Claude Code is the recommended entry point. It’s designed to help you bootstrap Clicky locally and then iterate.

• The fastest route is to use Claude Code. It can guide you through the setup, including the Cloudflare Worker, the proxy URLs, and building in Xcode.
• A sample prompt to Claude Code might look like this:
• “Hi Claude. Clone https://github.com/farzaa/clicky.git into my current directory. Then read the CLAUDE.md. I want to get Clicky running locally on my Mac. Help me set up everything — the Cloudflare Worker with my own API keys, the proxy URLs, and getting it building in Xcode. Walk me through it.”
• Claude Code will perform the clone, read the documentation, and walk you through the full setup so you can keep developing and extending Clicky.

This approach reduces friction and provides a guided path from a clean slate to a working environment, with future feature development ready to be continued in your own branch of experimentation.

---

Manual setup: a comprehensive guide

If you prefer to do everything by hand, here’s the practical path, broken into clear steps and prerequisites.

Prerequisites

• macOS 14.2+ (needed for ScreenCaptureKit)
• Xcode 15+ (required for building the macOS app and Swift code)
• Node.js 18+ (needed for working with the Cloudflare Worker)
• A Cloudflare account (free tier is sufficient)
• API keys for:
• Anthropic
• AssemblyAI
• ElevenLabs

These are the building blocks that let Clicky proxy secrets safely, handle real-time transcription, and convert text-to-speech for responses.

1) Set up the Cloudflare Worker

The Worker acts as a tiny, secure proxy that holds your API keys. This means your private keys never ship in the app binary. The typical workflow looks like this:

• In the worker directory:
• bash
  • cd worker
  • npm install
• Add your secrets. Wrangling with Wrangler will prompt you to paste each one:
• npx wrangler secret put ANTHROPICAPIKEY
• npx wrangler secret put ASSEMBLYAIAPIKEY
• npx wrangler secret put ELEVENLABSAPIKEY
• For the ElevenLabs voice ID (not sensitive), set it in wrangler.toml under [vars]:
• ELEVENLABSVOICEID = "your-voice-id-here"
• Deploy the Worker:
• npx wrangler deploy
• After deployment, you’ll get a URL like:
• https://your-worker-name.your-subdomain.workers.dev
• Copy that URL; you’ll point the app at it.

2) Run the Worker locally (for development)

If you want to test changes without deploying:

• cd worker
• npx wrangler dev

This starts a local server (often at http://localhost:8787) that behaves like the deployed Worker.

• Create a .dev.vars file in the worker/ directory with your keys, for example:

• ANTHROPICAPIKEY=sk-…

• ASSEMBLYAIAPIKEY=…

• ELEVENLABSAPIKEY=…

• ELEVENLABSVOICEID=…

• When developing, update the app’s proxy URLs to point to http://localhost:8787. Look for references to the deployed Worker URL and adjust accordingly (grep for clicky-proxy to locate them).

3) Update the proxy URLs in the app

The app has references to the Worker URL in a few places. Ensure everything points to the correct local or deployed endpoint:

• grep -r "clicky-proxy" leanring-buddy/

You’ll typically locate and update these files:

• CompanionManager.swift — Claude chat + ElevenLabs TTS
• AssemblyAIStreamingTranscriptionProvider.swift — AssemblyAI token endpoint

4) Open in Xcode and run

• Open the project:
• open leanring-buddy.xcodeproj
• In Xcode:

1. Select the leanring-buddy scheme (note: the deliberate typo)
2. Set your signing team under Signing & Capabilities
3. Hit Cmd + R to build and run

What you’ll see:

• The app will appear in your menu bar (not the Dock).
• Click the icon to open the panel, grant the requested permissions, and you’re good to go.

Permissions the app needs

• Microphone — for push-to-talk voice capture
• Accessibility — for the global keyboard shortcut (Control + Option)
• Screen Recording — for taking screenshots when you use the hotkey
• Screen Content — for ScreenCaptureKit access

These permissions enable Clicky to capture your inputs, associate them with the right UI regions, and deliver real-time guidance with minimal friction.

---

Architecture at a glance

If you want the high-level technical picture, here’s the short version of Clicky’s architecture:

• Menu-bar app with two NSPanel windows:
• A control panel dropdown
• A full-screen, transparent cursor overlay
• Push-to-talk streams audio to AssemblyAI (real-time transcription)
• Transcripts plus screenshots are sent to Claude via a streaming SSE connection
• Claude’s response is delivered back through ElevenLabs TTS
• Claude can embed special tags like [POINT:x,y:label:screenN] in its responses to guide the cursor toward specific UI elements across multiple monitors
• All three APIs (Claude, TTS, transcription) are proxied through the Cloudflare Worker for secure key handling

This combination creates a responsive, interactive AI tutor that can annotate your screen, narrate actions, and navigate to UI targets as needed.

---

Project structure (what’s inside the repo)

A quick tour of the main files helps you understand how Clicky is built and organized:

• leanring-buddy/ (Swift source; note the intentional typo in the name)
• CompanionManager.swift — The central state machine
• CompanionPanelView.swift — UI for the menu bar panel
• ClaudeAPI.swift — Claude streaming client
• ElevenLabsTTSClient.swift — Text-to-speech playback
• OverlayWindow.swift — The blue cursor overlay
• AssemblyAI*.swift — Real-time transcription integration
• BuddyDictation*.swift — Push-to-talk pipeline
• worker/ — Cloudflare Worker proxy
• src/index.ts — Three routes: /chat, /tts, /transcribe-token
• CLAUDE.md — Full architecture documentation (intended for agents/readers)
• Additional build and configuration files to wire up the environment

This structure is designed for clarity and ease of extension. The Swift components are responsible for the macOS UI and real-time behavior, while the Worker proxy keeps sensitive keys safe and accessible only to the server side.

---

Contributing and getting involved

Contributions are welcome. If you’re using Claude Code, the codebase already understands what you want to build. In that spirit, you can:

• Propose new features, bug fixes, or UI improvements
• Extend the tool to cover additional edge cases
• Create new integrations or plug-ins that work with the existing architecture

If you want to reach out for feedback or share progress, the project’s maintainer invites direct messaging on X (Twitter). You can find updates and context there and use the discussion channels to propose changes or gather help from the community.

---

How to think about using Clicky in practice

• Learn by doing: Use Clicky to guide you through tasks while you build new features or fix issues.
• Safe experimentation: The article emphasizes that the codebase is open for tinkering, with private work continuing on top of the public foundation.
• Hands-on learning: The combination of screen visibility, spoken guidance, and cursor-directed cues helps you internalize UI flows faster than static tutorials.
• Privacy-aware workflows: The Cloudflare Worker acts as a secure boundary so your private API keys don’t get embedded in the app binary.

Whether you’re a student, a developer, or a curious learner, Clicky offers an approachable path to understanding AI-assisted workflows on macOS while staying rooted in an open-source ethos.

---

Final thoughts

Clicky represents a bold fusion of AI tutoring, live screen interaction, and developer-friendly openness. The project’s dual-path approach — an open-source core with a private development stream — invites a broad community to learn, contribute, and innovate. The MIT license ensures you can adopt, modify, or even commercialize your own variations, provided you respect the terms.

If you’re ready to dive in, you can explore the public repository, follow the setup steps outlined above, and start building your own features or tweaks. The blend of Claude-based guidance, AssemblyAI transcription, and ElevenLabs TTS creates a dynamic feedback loop that can adapt to your learning pace and preferences.

The demo GIF provides a quick visual sense of Clicky in action, while the detailed architecture and project structure give you a blueprint for deeper exploration. So whether you’re a curious beginner or a seasoned developer, Clicky offers a rich playground for experimenting with AI-driven teaching tools on macOS.

For the latest updates and announcements, you can check the project’s public channels and links listed in the original documentation. And if you have ideas or feedback, don’t hesitate to reach out through the established communication channels.

End of guide. Happy tinkering with Clicky!